Building a multi-agent intake and triage workflow that survives production

TL;DR / Key Takeaways

• A multi-agent workflow is a pipeline with defined handoffs, not a swarm of free agents.
• The router is the only agent with judgment; the rest are tools it calls through a narrow, permissioned interface.
• Triage scoring against real historical cases beats a prompt that "reasons" about priority.
• Everything below a confidence threshold goes to a review queue, and the human's correction becomes the next training signal.
• If you cannot reconstruct any given run, you shipped an agent you cannot debug.

A multi-agent intake system is one workflow with a few defined stages, a router in front, and a human review step at the end. The "multi-agent" part is a router plus a small set of specialized steps that hand off to each other, not a collection of agents negotiating with one another.

This is a field note on the version of that system we actually build: an intake agent that classifies an inbound request, extracts the fields that matter, scores it against real cases, and either acts or routes it to a person. It is the same shape as the quoting and follow-up workflows, one layer of judgment up.

Start with the workflow, not the agents

The first mistake is drawing boxes for "agent A," "agent B," and "agent C" before defining the actual flow of work.

Write the workflow in plain language first:

1. A request arrives (email, form, message, document).
2. Someone — or something — decides what it is and how urgent it is.
3. The details get pulled into the right shape for the next step.
4. If the decision is clear, act on it. If not, a person looks at it.
5. The person's answer is recorded and used to make the next run better.

Only after that map exists should you decide which stages need a language model, which need code, and which need both.

The router does the judgment; the tools do the work

In practice the system is not several agents talking to each other. It is one router with a toolbelt.

The router's job is classification and extraction:

• What kind of request is this?
• Which account, order, or record does it relate to?
• What fields need to be filled in?
• How confident is the model in each answer?

The tools it calls are narrow and permissioned: look up a CRM record, check a pricing rule, pull a document, write a draft to a queue. Each tool has one job and does not call the model on its own.

This matters for two reasons. First, narrow tools are testable. Second, when something goes wrong you know which stage broke, because the stages are not blurred together.

Score against real cases, not a prompt

"Rank these by priority" in a prompt is a feeling, not a measurement. Triage needs a score you can reproduce.

The scoring step compares the incoming request to a set of real, labeled historical cases and produces a number — often a classification confidence plus a few extracted signals, such as account size, requested delivery date, or whether it is an existing customer.

Two things make this work in production:

• The reference set is real cases, not synthetic ones. If you score against made-up examples, you are measuring the prompt, not the business.
• The threshold is explicit. A request above the line routes automatically; a request below the line goes to review. "Above the line" and "below the line" are the two outcomes that matter.

The review queue is a feature, not a failure

Every low-confidence result routes to a person. That is the point of the design.

The review step is where the model's judgment is checked and corrected, and the correction is the cheapest labeled data you will ever collect. Each "this was actually a refund, not a support ticket" is a training example for the next scoring run.

If a workflow has no review step, it has no safety margin, and it has no way to get better. Agents that only run unattended never learn what they got wrong.

Boundaries before you ship

An intake agent that can read everything and write anywhere is an incident waiting to happen. Define the boundaries in code, not in the prompt:

• Which tools it can call
• Which data it can see (and which fields it cannot)
• Which actions it can take without approval, and which require it
• Where the review points sit
• What gets logged for every run

The prompt can ask nicely. The permissions should not care how nicely it asks.

Observability: you must be able to reconstruct a run

A wrong output that you cannot trace is not debuggable — it is only re-runnable, and re-running it gives you the same wrong output back.

For every run, capture:

• The raw input
• The classification and confidence
• Which tools were called, with which arguments, and what came back
• What the router produced
• Whether it went to review, and what the human did

Store the inputs and outputs, not just the final answer. The reconstruction is how you find the schema drift in the source document, the tool that started returning a different shape, or the model that started hallucinating a field it was never given.

Where these systems actually break

The failures are rarely the model being dumb. They are:

• Schema drift — the document, form, or API the agent reads changes shape silently, and the extraction starts returning garbage that still parses.
• Tool drift — a downstream system changes an error code or a field name, and the router keeps routing on stale assumptions.
• Threshold rot — the business changes what "urgent" means, but the scoring threshold was set once and never revisited.
• Permission creep — a new tool gets added with broad access because it was faster than scoping it.

Each of these is caught by the review queue and the run logs, provided both exist and someone actually looks at the miss rate over time.

Practical checklist

• Map the workflow in plain language before naming any agents.
• Keep the router as the single point of judgment; make the tools narrow and permissioned.
• Score against real, labeled cases with an explicit threshold.
• Route everything below the threshold to a human review queue.
• Capture raw input, tool calls, confidence, and human corrections for every run.
• Revisit the threshold and the reference set when the business changes.

FAQ

Do I actually need multiple agents, or one agent with several tools?

Usually one router with several tools. A separate "agent" only earns its place when a stage genuinely needs different tools, a different model, or different approval rules — not because the diagram looks better with more boxes.

What is the difference between this and plain automation?

The router exists because the input varies. If every request arrived with the same structure, a fixed script would be cheaper and more reliable. The agent earns its cost where language, ambiguity, or judgment enters the flow.

How do you know the triage is working?

By measuring the review queue, not the model's confidence. A rising miss rate, or a reviewer correcting the same category repeatedly, tells you more than any prompt-level evaluation.

Where does MCP fit into this?

MCP servers are one clean way to expose the tools — the CRM lookup, the pricing rule, the document reader — as a permissioned interface the router can call. The pattern is the same either way: narrow, tested tools behind a boundary.

Related Reading

• When to ship an agent, and when to just write SQL
• Why AI projects need boring plumbing before they need agents
• AI Agents and Orchestration