Multi-Agent Systems: Architecture, Patterns, and Real-World Examples

Anthropic reported that its multi-agent research system beat a single-agent setup by 90.2% on an internal evaluation. It also used about 15 times more tokens than a normal chat. That pairing matters. Multiple agents can widen a search, isolate context, and work in parallel, but every extra reasoning loop adds cost and another place for the system to fail.

A multi-agent system is software in which several autonomous or semi-autonomous agents interact inside a shared environment to pursue individual or collective goals. Modern versions may use language models, tools, memory, and structured messages. The idea predates LLMs by decades. Traffic control, robotics, market simulations, and distributed optimization all have a long multi-agent history.

This guide explains the full system: its architecture, five common patterns, examples that have made it into production, and the controls needed when several agents can act at once.

People who search for "multi agent systems" are usually trying to decide whether splitting work across agents will improve a real application. That decision is the thread running through every section below.

More agents do not create intelligence by arithmetic. They create a coordination problem that may be worth solving.

What is a multi-agent system?

Four properties separate a multi-agent system from a chain of API calls. It has multiple entities that can decide what to do, an environment or shared problem space, a way for those entities to affect one another, and goals that may be shared or local.

Autonomy comes in degrees. A routing agent that chooses one specialist from a fixed menu has little freedom. A research agent that creates subtasks, recruits subagents, changes its plan, and decides when the evidence is sufficient has much more. Both can belong to a multi-agent design. A script that calls three models in a fixed order cannot, unless those calls contain real decision authority.

Classical systems and LLM-based systems

Researchers studied interacting software agents long before ChatGPT. Classical multi-agent systems model local decisions, incomplete information, negotiation, cooperation, and conflict. A warehouse robot may reserve an aisle. A traffic signal may change its phase after receiving congestion data from nearby junctions. Neither needs natural language.

LLMs changed the interface. Agents can now delegate work in plain English, interpret loose instructions, call software tools, and adapt a plan when a result looks wrong. They did not repeal distributed-systems physics. State still becomes stale. Concurrent writes still collide. Messages still arrive late. Permissions still matter.

Workflow, single agent, or multi-agent system?

Zapier, Make, and n8n are good at known sequences. A flowchart encodes the path before the run begins. It does not become a multi-agent system because one box calls an LLM. Chatbots sit at the other end of the spectrum: one conversational loop, usually waiting for the next message. A multi-agent system can plan, delegate, wait on peers, merge artifacts, and continue without a person prompting every move.

For the anatomy inside one reasoning loop, read our guide to AI agent architecture. This article stays one level higher, where several loops have to behave as one system.

Multi-agent system architecture

The smallest useful diagram has six parts. Leave any one of them vague and the omission comes back as an incident.

Request or event
       |
Coordination layer
       |
Specialized agents <-> communication layer
       |                         |
Local state             shared state and artifacts
       |                         |
Tools and external systems
       |
Governance, traces, approvals, budgets, evaluation

Agents need real responsibility boundaries

Split agents along a boundary that the system can enforce. Useful boundaries include domain knowledge, tool access, security permission, context ownership, model cost, or organizational accountability. A billing specialist that can read invoices but cannot issue a refund without approval has a real boundary. Three agents called researcher, writer, and reviewer, all given the same prompt and tools, amount to persona theater.

Boundaries answer practical questions. Who owns the customer record? Which agent can change it? What happens when two agents disagree? Who pays for a retry? If the design document does not answer those questions, the runtime will answer them accidentally.

Coordination allocates work

A central supervisor can create tasks and combine results. A router can select one specialist. Peers can negotiate through direct messages or a shared board. Some systems encode handoffs in a graph; others let an agent choose its next collaborator at runtime.

This is the boundary with AI agent orchestration. Orchestration owns delegation, execution order, retries, and routing. System architecture asks which coordination model fits the job and what that model costs in latency, coupling, and accountability.

Separate local state, shared state, and artifacts

Local state belongs to one agent: scratch notes, temporary hypotheses, tool results that do not need to travel. Shared state includes task status, constraints, accepted facts, and the current plan. Artifacts are durable outputs such as code changes, tickets, reports, approvals, or database rows.

Dumping every message into a communal context is not state management. It burns tokens, leaks information across permission boundaries, and makes freshness hard to judge. Good designs name an owner for each shared field, add versions to mutable records, and treat durable artifacts as the source of truth when chat transcripts disagree.

Communication needs contracts

Agents can communicate through direct messages, queues, event buses, shared blackboards, or structured task records. Natural language works well for interpretation. It works poorly as the only contract for a payment amount, repository path, completion state, or permission request.

Use typed envelopes around flexible content. A handoff should carry an objective, inputs, constraints, output schema, deadline, budget, and error state. The prose inside can stay expressive. The edges cannot.

Google introduced the Agent2Agent protocol in 2025 with support from more than 50 technology partners. A2A addresses collaboration between agents. MCP gives models a standard way to access tools and outside context. Those are different integration seams.

The operational control plane

Once agents can spend money, modify records, send messages, or deploy code, the system needs controls that sit across every framework and model. Identity, least-privilege access, run budgets, trace correlation, approvals, pause and resume, retry policy, audit history, and evaluation all live here.

Rerun fits at this layer. It is a framework-agnostic operations product for observing agent work, applying governance, and placing humans at sensitive action boundaries. It does not define an agent's reasoning, communication topology, or framework runtime.

Five multi-agent system patterns

Patterns are topologies, not brand names. Each changes the path that information and authority take through the system.

1. Supervisor and workers

One agent decomposes the goal, assigns bounded tasks, reviews the returns, and builds a final answer. The supervisor carries the global plan while workers get narrow contexts. That isolation can reduce distraction and let different searches run at once.

The supervisor is also a single point of failure. Vague task descriptions produce redundant work. A weak synthesis can erase good findings. Put a hard ceiling on fan-out, require source-bearing outputs, and let workers report uncertainty instead of filling gaps.

2. Router and specialists

A router classifies an incoming request and sends it to a specialist with the right tools or access. Customer support is the obvious case: billing, technical support, account security, and cancellations each need different data and rules.

Routing errors deserve first-class treatment. Keep the original request attached to the handoff, record the classification and confidence, and give the receiving agent a refusal path. Silent bouncing between specialists is an expensive loop dressed up as collaboration.

3. Sequential handoffs

Each agent owns one stage and passes a structured artifact forward. A policy agent checks a proposed answer, a legal agent reviews disputed language, and a publishing agent releases the approved version. This design creates clean ownership when each stage has a different permission boundary.

Bad input travels. Later agents may trust an upstream label that was guessed. Save the underlying evidence with every decision and allow a downstream agent to send the artifact back with a typed reason.

4. Parallel fan-out and aggregation

Several agents inspect independent dimensions at the same time. One might search papers, another review customer data, another test a code path, and a final component reconcile their outputs. Read-heavy work fits because parallel agents do not compete to mutate the same object.

Write-heavy tasks are harder. Two coding agents touching the same module can create a merge problem larger than the code they produced. Parallelize investigation first. Serialize side effects unless the underlying system has safe concurrency controls.

5. Peer-to-peer collaboration

Agents coordinate without a permanent supervisor. They may publish observations to a shared environment, bid for tasks, or negotiate access to a resource. Robotics, traffic systems, and simulations use this model when no central actor has complete information or reliable control.

Accountability gets blurry. Add explicit termination rules, message budgets, deadlock detection, and an authority that can stop the run. An emergent loop does not care that every individual message looked sensible.

Our agentic design patterns guide goes deeper into implementation patterns. Keep topology and implementation separate when evaluating a design.

When should you use multiple agents?

Start with one agent. Add another only when an evaluation shows a measurable gain in quality, speed, isolation, or safety.

A multi-agent design earns its cost when work divides cleanly, parallel exploration matters, specialists need different permissions, one context window becomes noisy, independent critique catches expensive errors, or distinct systems own separate stages. Organizational reality can be a sound boundary too. A finance agent and a support agent may need different owners even if one model could technically do both jobs.

Stay with one agent when the task depends on a continuous shared context, steps are tightly sequential, latency dominates, a single tool set is safe, or the team cannot trace one agent reliably yet. Use an ordinary workflow when every branch can be written down in advance. Flowcharts remain excellent for deterministic work. They are simply a different control model.

Six questions before adding an agent

• Can the work split without constant synchronization?
• Does each proposed agent have a boundary we can state and enforce?
• Can every handoff use a structured contract?
• Is there one authoritative source of shared truth?
• Can the system recover when one agent fails halfway through?
• Does the measured gain justify more tokens, latency, and operational work?

If two answers are no, redesign before adding headcount to the machine.

{
  "role": "You are reviewing a proposed multi-agent system before production.",
  "inputs": ["agent list", "tool permissions", "state model", "handoff schemas", "evaluation baseline"],
  "checks": [
    "Identify agents that lack a distinct responsibility or permission boundary",
    "Find shared state without one owner or versioning rule",
    "List handoffs missing an objective, constraints, output schema, budget, or error state",
    "Map every irreversible action to an approval or rollback rule",
    "Compare the design with a single-agent baseline"
  ],
  "output": {
    "unnecessary_agents": [],
    "failure_paths": [],
    "permission_gaps": [],
    "required_tests": [],
    "ship_decision": "go, revise, or stop"
  }
}

One documented deployment and four reference architectures

Documented deployments and reference architectures should not be mixed. The first example below comes from a published engineering account. The others describe common system shapes, not claims about a named customer's production stack.

Anthropic's research system

Anthropic built a lead research agent that delegates searches to parallel subagents, then synthesizes their findings. In the company's June 2025 engineering report, the design improved performance by 90.2% over a single-agent setup on an internal research evaluation. Anthropic also reported that agents used about 15 times more tokens than ordinary chat interactions.

The qualifier is doing real work: internal evaluation, one research product, one task family. It does not prove that multi-agent designs win everywhere. It shows why breadth-first research is a good candidate. Independent context windows let subagents explore several directions without crowding one lead agent's context.

How we built our multi-agent research systemOn the the engineering challenges and lessons learned from building Claude's Research systemanthropic.com

Customer service routing and resolution

A triage agent can classify a request, then hand it to billing, technical, or account specialists. A compliance agent reviews regulated language. Refunds above a threshold pause for a person. All agents read the same customer identifier, but their fields and actions differ by permission.

This is where chatbot framing breaks down. The chat window is only the intake surface. The hard system is behind it: identity, shared customer state, routing confidence, side effects, escalation, and a trace that explains who changed what.

Software delivery and incident response

During an incident, one agent can inspect telemetry while another searches recent changes and a third reads the runbook. Parallel read-only investigation is useful. Deployment stays gated. A human reviews the proposed patch, evidence, blast radius, and rollback plan before any production write.

Concurrent coding is less forgiving. Agents can edit incompatible assumptions into neighboring files, pass isolated tests, and still break the build together. Give one owner the write set or use workspaces with explicit merge and review stages.

Supply-chain coordination

Demand, inventory, warehouse, supplier, and routing agents each observe a partial view. A warehouse agent may minimize picking time while the routing agent minimizes vehicle miles. Shared constraints stop a local win from damaging the whole network.

Classical multi-agent work has wrestled with this problem for years. The agents need not speak English or call an LLM. They need local decisions, interaction, and a common environment.

Traffic, robotics, and autonomous fleets

Robots can reserve paths, share hazards, and allocate jobs as conditions change. Traffic signals can exchange queue data with neighboring intersections. Fleet members may decide locally because a central controller cannot react quickly enough or see every obstacle.

Decentralization lowers dependence on one coordinator. It also makes global debugging harder. Replay needs synchronized events, agent identities, and the state each actor saw at decision time.

Why multi-agent systems fail in production

Handoffs lose the point

An agent asks another to "check this" without saying what success means, what evidence is allowed, or what shape the answer should take. The recipient returns plausible prose. Nobody notices that it answered a different question.

Treat handoffs like API contracts. Preserve the parent objective, assign one bounded task, declare the expected artifact, and carry constraints forward verbatim when they matter.

Shared state becomes a rumor

One agent reads an account balance. Another changes it. The first agent approves a purchase using stale data. Chat history cannot fix this. Use authoritative records, version checks, idempotency keys, and conflict handling around state mutations.

Errors and costs multiply

One hallucinated fact can pass through a reviewer that assumes the researcher checked it. Retries spawn more agents. Agents debate until a budget expires. Token cost rises with fan-out and chatter, while tool charges and latency accumulate outside the model bill.

Set budgets at run, agent, and tool level. Record why each child was created. A retry must have a reason that changes the next attempt, not blind hope.

Termination rules are weak

"Continue until done" is not a production stop condition. Define maximum turns, time, tokens, recursion depth, tool spend, and confidence thresholds. Route unresolved work to a dead-letter queue or person. Give operators a real kill switch.

Individual traces hide system failure

Each agent may complete its task while the combined result fails. A router classified correctly, a billing agent issued the requested refund, and a retention agent offered a discount at the same time. Local success created a contradictory customer outcome.

System evaluation follows the causal chain across delegation, tool calls, state changes, approvals, and the final business result. Per-agent accuracy is only one slice.

How to operate a multi-agent system safely

Write the responsibility map before the prompts. For each agent, name its owner, allowed inputs, tools, write permissions, budget, stop rule, and escalation path. Deny access by default. A specialist should not inherit every tool the supervisor has.

Then make side effects boring. Use typed commands, validation, idempotency keys, dry runs, approval thresholds, and rollback paths. Put a human checkpoint immediately before an irreversible or high-impact action, not after an alert announces that it already happened.

Trace one run as one run. Correlation IDs must survive every delegation and queue. Capture the state version, prompt or policy version, tool input, tool result, approval, and resulting artifact. Evaluate outcomes at both levels: did each agent follow its contract, and did the entire system achieve the intended result without unacceptable cost or risk?

The NIST AI Risk Management Framework organizes work through Govern, Map, Measure, and Manage. It gives teams a useful lifecycle vocabulary without prescribing an agent framework.

Rerun adds the operational layer across agent runtimes. Teams can watch work, inspect traces, apply controls, and bring a person into the loop where judgment or authority is required. Frameworks still build the agent behavior. Rerun helps people run it.

Frameworks build behavior; operations controls it

Multi-agent frameworks make different bets. Some model execution as a graph. Others use conversations, roles, or event-driven actors. State may live in a checkpoint store, a message thread, or an application database. Handoffs may be explicit edges or decisions made during a run.

Those differences matter when building. Our AI agent frameworks guide covers the selection question, while AutoGen vs CrewAI handles that specific comparison. Neither framework choice removes the need for cross-run visibility, permissions, budgets, approvals, and an audit trail.

Keep two layers on the diagram. The framework or runtime defines how agents reason and coordinate. The operations layer governs what happens across the deployed system. That separation lets a team change models or frameworks without rebuilding every control around them.

July 26, 2026

AI Agent Orchestration: How to Coordinate Multi-Agent Systems

AI agent orchestration coordinates multiple reasoning agents toward one goal. Learn the four patterns, the architecture, and how to run a multi-agent system with no code.

Build the smallest system that can pass the test

Multi-agent systems pay off when specialization, context isolation, permission separation, or parallel work produces a measured gain. They punish decorative complexity. Every agent adds handoffs, state transitions, new failure paths, and bills.

Begin with a single-agent baseline. Add one boundary. Test it against quality, latency, cost, and safety criteria. Keep the extra agent only if the numbers move enough to cover its operational burden. Production architecture grows from evidence, not an org chart made of prompts.