Skip to main content

Telco: the agent that has read the runbook

Track: Telco · In the room: Ericsson · Seed use case: incident triage across runbooks, grounded with AI Search.

Who this is for​

Ericsson. Essentially no Cloudflare developer-platform footprint today, which in this track is not a handicap — it means there is no legacy triage tooling to migrate off, only a greenfield hour and a network operations problem that every telco shares: an alert fires, and the person who picks it up has to reconstruct, from memory or from a wiki nobody has opened in months, which of the fifty runbooks applies.

The pain is never "we lack monitoring." Telco networks are drowning in telemetry. The pain is the translation step between an alert and an action — the minutes spent searching for the right procedure while a cell site, a core function, or a customer's service stays degraded. That translation step is exactly what a grounded agent is for.

Synthetic data only

Invented node ids, invented alert payloads, invented runbooks. Nothing that resembles real network topology, real customer impact, or a real internal procedure goes into a shared account today.

Three agent use cases​

1. Incident-to-runbook triage — the seed use case​

  • Trigger. A monitoring alert fires — a threshold breach, a flapping interface, a node reporting degraded health. This is pushed to the agent; nobody opens a dashboard.
  • What the agent decides. Which runbook procedure actually matches this alert's signature, how severe it is, and whether the indicated remediation is safe to apply without touching a live customer session or only safe with a human watching.
  • What it does — the tools. triage_incident matches the alert against the runbook corpus and returns a diagnosis with the specific passage it relied on. propose_remediation turns that into a concrete step — restart a process, fail over a link, rate-limit a neighbour — with the runbook citation attached.
  • Human in the loop? Always on the remediation, never on the read. Retrieving and citing the right procedure is pure gain — a NOC engineer reading a cited runbook passage sixty seconds faster is a win with no downside. Actually touching a live network element — restarting a node, failing over traffic — is irreversible in the way that matters here: it can turn a degraded service into an outage if the diagnosis was wrong, so it waits for a named engineer.

2. Recurrence and flap detector​

  • Trigger. The same triage runs repeatedly, because the same node keeps alerting.
  • What the agent decides. Whether this is the fourth unrelated alert today or the fourth occurrence of the same underlying fault today — the difference between a node worth an engineer's afternoon and one worth a dozen separate, individually-actioned tickets.
  • What it does — the tool. check_recurrence compares the current alert against that node's own recent history and raises the priority, and the ticket, once a pattern is confirmed rather than on every individual instance.
  • Human in the loop? Not for the correlation itself — flagging a pattern is informational. Yes for anything the correlation recommends, such as escalating a routine fault into a planned maintenance window, because that reprioritises other engineers' work, not just this one alert's.

3. Change-risk pre-check​

  • Trigger. A planned configuration change is about to be pushed to a node or a group of nodes.
  • What the agent decides. Whether this node has any open or recent incidents that make the change riskier than the change ticket assumes, and whether the runbook for this node's function class has a known interaction with the kind of change being proposed.
  • What it does — the tool. precheck_change cross-references the change against the node's incident history and the relevant runbook section, and returns a go / hold-for-review verdict with its reasoning.
  • Human in the loop? Yes, as advice a change-approval board can override. The agent never blocks a change outright — it adds a data point that either accelerates a safe change or gives a risky one a second look before it goes out.

The 60-minute cut​

Build use case 1. Cut it to: an alert arrives, a runbook is cited, a remediation waits for an engineer.

In scope:

  1. A hardcoded array of 6 synthetic alerts, each with an alert id, a node id, a signal name, and a short description. Make two clearly match one runbook each, one match nothing, and the rest overlap enough to need real disambiguation.
  2. A hardcoded array of 5 synthetic runbook entries — invented content only — each with an id, a fault signature it applies to, a severity, and a remediation step, plus whether that step is safe to auto-apply.
  3. One tool that matches an alert to the best-fitting runbook entry and states, honestly, when nothing matches well enough to cite.
  4. A Workflow that proposes the remediation and pauses on step.waitForEvent() for anything not marked auto-applyable.
  5. A one-page queue: every alert, its cited runbook id, its verdict, and its approval state.

Out of scope, and say so: a real alerting integration, real network protocols, ticketing system sync, authentication, and anything resembling a topology graph. If you are modelling actual network elements, you have lost the hour.

The demo that wins. Fire two alerts that look similar. Show the agent citing a different runbook passage for each, correctly, because the signatures actually differ. Then try to apply the riskier remediation yourself and get refused until an engineer approves it. A room full of network engineers trusts a system that shows its citation more than one that shows its confidence score.

Primitives — exactly three​

  1. AI Search over your own documents. This is the pattern Cloudflare's own product, AI Search, exists to formalise: index a corpus, retrieve the passage that actually supports an answer, and cite it — rather than asking a larger model to remember your runbooks from its training data, which it cannot, because your runbooks are not public. For the hour, simulate the pattern with a hardcoded corpus and a matching tool, as this playbook's prompt does below — wiring the real product onto your own docs is a follow-up your event's AI Search instance already makes possible, and it is what makes the mentor agent in this event answer questions about this event correctly.
  2. Workflows. Applying a remediation to live network infrastructure has to be exactly-once and has to be able to wait for an engineer who is on another call. step.waitForEvent() holds the run open without cost, and step.do() checkpointing means a retry cannot restart the same node twice.
  3. Persistent Durable Object state. An alert only means something in the context of that node's recent history — the same signal is routine on one node and a red flag on another that has already alerted three times today. An Agent instance addressed by node id, with embedded SQLite via this.sql, is where that history lives, strongly consistent and hibernating between alerts.

Skip schedules here — an incident is pushed by monitoring, not discovered by polling, and inventing a sweep would just add latency to something that is already an event. Skip Browser Rendering — there is no public page in this loop, only internal telemetry and your own runbook corpus.

Paste-ready starter prompt​

Scaffold the starter first:

npm create cloudflare@latest -- team-telco-incident-triage --template cloudflare/agents-starter
cd team-telco-incident-triage

Then paste this into your coding agent:

I'm working in a fresh clone of Cloudflare's agents-starter template. The agent code is in
src/server.ts and the React client is in src/client.tsx. I have 60 minutes. Keep the
existing chat agent working — extend it, don't replace it.

Build a network-incident triage agent that cites a runbook before recommending anything.

1. Add a hardcoded array of 5 synthetic runbook entries to a new file. Each has: id (like
   RB-204), a short fault-signature description, severity, a remediation string, and
   autoApplyable (boolean). Invented content only.

2. Add a hardcoded array of 6 synthetic alerts to src/server.ts: alertId, nodeId, signal
   name, description, and timestamp. Make two obviously match one runbook entry each, one
   match nothing well, and the rest ambiguous enough to need real comparison.

3. Add a tool called triage_incident. Given an alertId, it compares the alert description
   against every runbook entry using a Workers AI model and returns JSON: matchedRunbookId
   (or null if nothing fits well), confidence 0 to 1, diagnosis (two sentences that quote the
   matched entry's fault signature), and remediation. If matchedRunbookId is null, say so
   honestly instead of guessing.

4. Add a Cloudflare Workflow called IncidentRemediation in a new file, with steps as
   step.do():
     a) triage — run the triage logic and store the result
     b) await_approval — skip this step entirely if the matched runbook's autoApplyable is
        true. Otherwise use step.waitForEvent() with a generous timeout, waiting for an event
        of type "remediation-decision"
     c) apply — record that the remediation was applied (or declined), and by whom
   Import WorkflowEntrypoint, WorkflowStep and WorkflowEvent from "cloudflare:workers", and
   add the workflow binding to wrangler.jsonc.

5. Store each alert's triage result in the Agent's SQLite storage using this.sql, keyed by
   nodeId, and check that node's last 24 hours of results before triaging a new alert for it
   — if 2 or more prior results reference the same runbook id, flag isRecurring true.

6. Add routes:
     POST /alert/:alertId    starts an IncidentRemediation workflow for that alert
     GET  /queue             HTML page: every alert, its cited runbook id (or "no match"),
                              severity, recurrence flag, and approval state
     POST /decide/:alertId   sends the "remediation-decision" event with approved true or
                              false and an engineer name from the body

Requirements: TypeScript. Route every Workers AI call through AI Gateway by passing
{ gateway: { id: "agenthack" } } as the options argument. Never auto-apply a remediation that
isn't marked autoApplyable. Run `npx wrangler types` after changing wrangler.jsonc. Then
deploy with `npx wrangler deploy` and give me curl commands that prove an autoApplyable
remediation goes straight through and a non-autoApplyable one waits for /decide.

The Worker name team-telco-incident-triage follows the shared-account rule <team_prefix>-<slug>. Eight teams deploy into one account today, so if the Scoping agent issued you a different prefix, use that one instead.

What "production" looks like​

  1. Index your real runbooks in Cloudflare AI Search, not a hardcoded array. Point an instance at your actual procedure documentation and require every diagnosis to carry the real source passage. Then measure precision on a sample of historical alerts before you trust it on a live one — a wrong citation is worse than no citation, because it is confidently wrong.
  2. Make the approval a real identity, not a name in a form field. Put your identity provider in front of the decision endpoint so a remediation approval carries a named engineer, their on-call role, and the exact diagnosis they were shown, retained next to the agent's original recommendation.
  3. Feed resolved incidents back into the corpus. Every incident that gets manually resolved with a procedure that wasn't in the runbook is a gap in your documentation, not just a one-off fix. Closing that loop is what turns a static runbook corpus into a living one, and it is cheaper than any amount of prompt tuning.

Next: the Level Up page explains each primitive, and the cheat sheet has the snippets. Both are in the sidebar.