Multi-agent LLM systems fail mainly because of how they are organised, not because the model is weak. In the largest study of the question so far, UC Berkeley researchers traced failures to system design issues (44.2% of observed failures), inter-agent misalignment (32.3%) and weak task verification (23.5%), across 1,600+ annotated traces from 7 popular frameworks. So why do multi-agent LLM systems fail? Mostly because of how they are organised, and that changes what you should do about it.
The same paper opens on an awkward fact. Despite all the excitement, the performance gains of multi-agent systems "often remain minimal compared to single-agent frameworks" or even simple best-of-N sampling, and the seven open-source systems studied failed on 41% to 86.7% of their tasks. Add Gartner's forecast that over 40% of agentic AI projects will be canceled by the end of 2027, and the question stops being academic.
This guide walks through all 14 failure modes in the MAST taxonomy, with the current figures, a real example for each, and, most importantly, the signal that catches it in a live trace before a customer does.
In a hurry? Skip to the master table or start watching your own runs.
Key takeaways
- The 14 failure modes fall into 3 buckets: system design (44.2%), inter-agent misalignment (32.3%) and task verification (23.5%).
- The single most common failure is step repetition (15.7%), followed by reasoning-action mismatch (13.2%) and not knowing when to stop (12.4%).
- Better models move the failure mix around but do not remove it. The researchers expect model upgrades alone to be "insufficient".
- Prompt fixes help, structure fixes help more, and neither is enough. ChatDev went from 25.0% to 34.4% with better prompts and to 40.6% with a new topology.
- You cannot design every failure away, so production systems need step-level logs, run budgets, approvals before irreversible actions, and a way for agents to ask a human.
The short answer: it's the organisation, not the model
If you are new to the architecture itself, our guide to multi-agent systems covers what they are and when they are worth it. This article starts where that one stops: they are running, and they are failing.
The most useful sentence in the Berkeley paper is in section 5.3. The authors write that they "conjecture that improvements in the base model capabilities will be insufficient to address the full MAST", and argue that good multi-agent design "requires organizational understanding", because even organisations of brilliant individuals can fail catastrophically when the structure is wrong.
A team of experts with no clear roles, no handoff rules and no reviewer does not fail because the experts are dumb. Multi-agent systems fail the same way.
The data backs this up. On the same programming tasks, GPT-4o showed 39% fewer system design failures than Claude 3.7 Sonnet, yet both still failed regularly. Swapping the framework mattered just as much: MetaGPT had 60% to 68% fewer design and misalignment failures than ChatDev, but 1.56x more verification failures. Change the model or the framework and you reshuffle the failures. You do not get rid of them.
Anthropic, which runs a multi-agent research system in production, makes the same point from the other side. Its setup with a Claude Opus 4 lead agent and Sonnet 4 subagents beat single-agent Opus 4 by 90.2% on an internal research eval, but used about 15x more tokens than a chat. Multi-agent systems can work. They just cost more, and they break in more places.
Where the 14 failure modes come from: the MAST study
MAST stands for Multi-Agent System Failure Taxonomy. It was built by the UC Berkeley Sky Computing Lab (Mert Cemri, Melissa Pan, Matei Zaharia, Joseph Gonzalez, Ion Stoica and colleagues) and published at NeurIPS 2025.
Why Do Multi-Agent LLM Systems Fail?Despite enthusiasm for Multi-Agent LLM Systems (MAS), their performance gains on popular benchmarks are often minimal. This gap highlights a critical need for a principled understanding of why MAS fail. Addressing this question requires systematic identification and analysis of failure patterns. We introduce MAST-Data, a comprehensive dataset of 1600+ annotated traces collected across 7 popular MAS frameworks. MAST-Data is the first multi-agent system dataset to outline the failure dynamics in MAS for guiding the development of better future systems. To enable systematic classification of failures for MAST-Data, we build the first Multi-Agent System Failure Taxonomy (MAST). We develop MAST through rigorous analysis of 150 traces, guided closely by expert human annotators and validated by high inter-annotator agreement (kappa = 0.88). This process identifies 14 unique modes, clustered into 3 categories: (i) system design issues, (ii) inter-agent misalignment, and (iii) task verification. To enable scalable annotation, we develop an LLM-as-a-Judge pipeline with high agreement with human annotations. We leverage MAST and MAST-Data to analyze failure patterns across models (GPT4, Claude 3, Qwen2.5, CodeLlama) and tasks (coding, math, general agent), demonstrating improvement headrooms from better MAS design. Our analysis provides insights revealing that identified failures require more sophisticated solutions, highlighting a clear roadmap for future research. We publicly release our comprehensive dataset (MAST-Data), the MAST, and our LLM annotator to facilitate widespread research and development in MAS.
arXiv.org
How they built it matters, because it is why these numbers are worth trusting:
- Grounded theory, not a wish list. Instead of starting from categories, six expert annotators read 150 traces (each averaging more than 15,000 lines) and let the categories emerge from what actually went wrong.
- High agreement. The human annotators agreed at a Cohen's kappa of 0.88, which is very high for this kind of labelling.
- Scaled with a calibrated judge. An LLM annotator built on OpenAI's o1 was calibrated to a kappa of 0.77 against the humans, then validated at 0.79 on two unseen systems.
- Broad coverage. The resulting dataset, MAST-Data, holds 1,600+ annotated traces across ChatDev, MetaGPT, HyperAgent, AppWorld, AG2, Magentic-One and OpenManus.
A note on freshness: the figures in this article come from the v3 revision of the paper. Earlier write-ups quoting 41.8% / 36.9% / 21.3% for the three categories use the original v1 numbers. The authors are also honest that MAST is not exhaustive. It is a precise, shared vocabulary, not the final word.
Category 1: system design issues (44.2%)
These failures are baked in before the first message is sent. They come from the system's architecture, its prompts and how it manages state.
FM-1.1 Disobey task specification (11.8%)
What it looks like: an agent ignores a constraint the task stated clearly. Asked for a script in Python 3.10 with no external libraries, it imports one anyway.
Production signal: check the final output against the explicit constraints of the spec, automatically where you can, and by a human where the constraint is a business rule.
FM-1.2 Disobey role specification (1.5%)
What it looks like: an agent acts outside its role. In ChatDev, the CPO agent sometimes ended the conversation without the CEO agent's agreement.
What fixed it: a simple workflow change ensuring the CEO had the final say added +9.4% to the overall task success rate.
Production signal: an agent calling a tool or taking a decision that its role should not have access to. This shows up instantly in step-level logs, and is better still prevented with least-privilege tool access.
FM-1.3 Step repetition (15.7%, the most frequent mode)
What it looks like: the system redoes a step it has already done, often the same tool call with the same arguments, burning tokens and time. The paper notes OpenManus is particularly prone to it.
Production signal: the same tool call with identical arguments N times in one run. This is the cheapest alert you will ever set up, and it catches the single most common failure.
FM-1.4 Loss of conversation history (2.8%)
What it looks like: context is unexpectedly truncated and the agent reverts to an earlier state, or asks again for something it already received. Our guide to AI agent memory covers the mechanics behind this.
Production signal: an agent re-requesting information already present earlier in the trace.
FM-1.5 Unaware of termination conditions (12.4%)
What it looks like: the system does not recognise that the task is done, or that it should stop trying, and keeps going.
Production signal: a run that goes past its expected step count or token budget without ever reaching a "done" state.
Category 2: inter-agent misalignment (32.3%)
These failures happen between agents while the system is running. The paper's key insight here is uncomfortable for anyone hoping a protocol will fix it: solutions focused on context or communication formats "are often insufficient" for this category, which demands deeper "social reasoning". Agents can exchange perfectly formatted messages and still fail to work out what the other agent needs to know.
Cognition's Walden Yan described the same problem in Don't Build Multi-Agents: actions carry implicit decisions, and when two subagents act on conflicting assumptions, the pieces do not fit back together. How you route work between agents (supervisor, hierarchy, swarm) is its own topic, covered in our guide to AI agent orchestration.
FM-2.1 Conversation reset (2.2%)
What it looks like: a dialogue between agents restarts unexpectedly, losing the progress made so far.
Production signal: a run that suddenly repeats its opening exchanges mid-way through.
FM-2.2 Fail to ask for clarification (6.8%)
What it looks like: an agent proceeds on a wrong assumption instead of asking when information is unclear or missing.
Production signal: low-confidence decisions taken silently. This is exactly where a human-in-the-loop question beats a guess: an agent that can pause and ask "which account did you mean?" avoids the whole failure.
FM-2.3 Task derailment (7.4%)
What it looks like: the system drifts away from the stated objective and produces work that is irrelevant or unproductive.
Production signal: tool calls and outputs that no longer map to the original goal. A short goal check every few steps, or a human reviewing the plan before execution, catches most of it.
FM-2.4 Information withholding (0.85%)
What it looks like: an agent holds information another agent needs and does not share it. The paper's example is a Phone Agent that knows the username format an API requires but never tells the Supervisor Agent, who also never asks. The result is repeated failed logins and a failed task.
Production signal: repeated identical errors from one agent while the fix is visible elsewhere in the trace.
FM-2.5 Ignored other agent's input (1.9%)
What it looks like: an agent disregards a recommendation or correction from another agent. If your design delegates work to helpers, our explainer on subagents covers the delegation patterns that make this more or less likely.
Production signal: a reviewer agent flags an issue and the next step proceeds as if it had not.
FM-2.6 Reasoning-action mismatch (13.2%, the second most frequent)
What it looks like: the agent's stated reasoning says one thing and its action does another. It writes "I will check the invoice total before sending", then sends without checking.
Production signal: compare the logged reasoning with the tool call that follows. You can only do this if you log every step, not just the final answer. This mode alone is the strongest argument for step-level traces over output-only logs.
Multi-Agent Systems: Architecture, Patterns, and Real-World Examples
A practical guide to multi-agent system architecture, patterns, examples, failure modes, and the controls required to run multiple AI agents safely.
Category 3: task verification (23.5%)
These failures are about quality control of the final output. The paper's third insight is that a single, last-stage, low-level check is not enough: you need verification at several levels.
FM-3.1 Premature termination (6.2%)
What it looks like: the system stops before the objective is met, or before the information needed to meet it has been exchanged. AppWorld suffers from this most, likely because its star topology, with no predefined workflow, makes termination conditions less obvious.
Production signal: a run that reports success while required outputs or steps are missing.
FM-3.2 No or incomplete verification (8.2%)
What it looks like: checks exist but miss what matters. In the paper, a ChatDev-generated chess game passed review because it compiled, yet it broke the actual rules of chess and was unusable.
Production signal: verification steps that only test surface properties (it compiles, it is non-empty, no TODOs left) rather than the real objective.
FM-3.3 Incorrect verification (9.1%)
What it looks like: the verifier runs, but reaches the wrong conclusion and approves a bad output.
Production signal: this one is the hardest to see from inside the system, which is why high-stakes outputs need an independent check, often a human.
What fixed it: adding a verification step against the high-level task objective to ChatDev gave +15.6% task success on ProgramDev.
Verification during a run is not the same as a test suite you run before release. For building offline evals, see our guide to AI agent testing. This section is about checks that happen while the work is being done.
The 14 failure modes at a glance
This is the table to bookmark. Each row pairs a failure mode with what you will actually see in a trace and the control that catches it.
| Code | Failure mode | Share | What you'll see in a trace | Control that catches it |
|---|---|---|---|---|
| FM-1.1 | Disobey task specification | 11.8% | Output breaks a stated constraint | Spec check, human approval |
| FM-1.2 | Disobey role specification | 1.5% | Agent acts outside its role | Step logs, least privilege |
| FM-1.3 | Step repetition | 15.7% | Same tool call, same arguments, N times | Repetition alert |
| FM-1.4 | Loss of conversation history | 2.8% | Agent re-asks for known information | Step logs, memory design |
| FM-1.5 | Unaware of termination conditions | 12.4% | Run exceeds step or token budget | Step and token limits |
| FM-2.1 | Conversation reset | 2.2% | Opening exchanges repeat mid-run | Step logs |
| FM-2.2 | Fail to ask for clarification | 6.8% | Silent low-confidence decision | Human question |
| FM-2.3 | Task derailment | 7.4% | Actions stop mapping to the goal | Goal check, plan approval |
| FM-2.4 | Information withholding | 0.85% | Repeated error, fix visible elsewhere | Step logs, human question |
| FM-2.5 | Ignored other agent's input | 1.9% | Reviewer flag followed by no change | Step logs |
| FM-2.6 | Reasoning-action mismatch | 13.2% | Stated plan differs from tool call | Step-level reasoning logs |
| FM-3.1 | Premature termination | 6.2% | "Done" with required outputs missing | Done-state definition |
| FM-3.2 | No or incomplete verification | 8.2% | Checks test surface, not objective | Objective-level verifier |
| FM-3.3 | Incorrect verification | 9.1% | Verifier approves a bad output | Human approval |
The three categories add up to 100%: 44.2% design, 32.3% misalignment, 23.5% verification.
Why prompt fixes alone hit a ceiling
The paper did not just catalogue failures. It tried fixing them, and the results are the most practical part of the study.
| System and benchmark | Baseline | Better prompts | New topology |
|---|---|---|---|
| ChatDev on ProgramDev-v0 | 25.0% | 34.4% | 40.6% |
| ChatDev on HumanEval | 89.6% | 90.3% | 91.5% |
| AG2 on GSM-Plus (GPT-4) | 84.75% | 89.75% | 85.50% |
Three lessons stand out:
- Prompts help, but only so far. Better role prompts and verifier instructions lifted ChatDev by 9.4 points on ProgramDev-v0.
- Structure can beat prompts. Turning ChatDev's one-way pipeline into a loop with a review-gated exit pushed it to 40.6%.
- Neither closes the gap. Even after both fixes, most programming tasks still failed. And for AG2, the new topology did worse than better prompts.
If design fixes are partial, the rest has to be caught while the system runs. That is an operations problem, not a modelling problem.
How to catch multi-agent failures in production
You will never design out all 14 failure modes. What you can do is make each one visible, cheap to stop, and impossible to turn into an irreversible mistake. Four controls cover almost the whole table.
Log every step, not just the final answer
Catches FM-1.3, FM-1.4, FM-2.1 and FM-2.6. You need the reasoning, the tool call, the arguments and the result for every step of every agent, in order. Output-only logs cannot show a reasoning-action mismatch or a reset.
This matters more than it sounds, because finding the culprit is hard even with the full trace. In the Who&When study presented at ICML 2025, the best automated method identified the agent responsible for a failure only 53.5% of the time, and the exact step only 14.2% of the time. Without step-level logs, you are not even in the game. For the tooling side, dashboards and tracing standards, our guide to AI agent observability goes deeper.
Put budgets and stop conditions on every run
Catches FM-1.3, FM-1.5 and FM-3.1. Every run needs a step limit, a token or cost limit, and an explicit definition of "done" that the system has to reach and report. A run that hits a limit should stop and surface, not quietly retry.
Require approval before irreversible actions
Catches FM-1.1, FM-1.2, FM-2.3 and FM-3.3. Before an email goes out, a payment is made, or a record is overwritten, a human sees exactly what is about to happen and approves it. Incorrect verification is the failure an automated verifier cannot catch on itself, and a person can. Our guides on human-in-the-loop AI agents and AI agent guardrails cover where to draw that line.
Let agents ask a human instead of guessing
Catches FM-2.2 and FM-2.4. Most clarification failures are an agent choosing to guess because it has no way to ask. Give it one: a question that pauses the run and waits for an answer turns a silent wrong assumption into a ten-second reply.
Label your own traces with MAST
The authors packaged their LLM annotator as a Python library, available from the MAST GitHub repository. Running it on a sample of your failed traces tells you which failure modes dominate your system, so you fix the right thing first.
pip install agentdashimport os
from agentdash import annotator
openai_api_key = os.environ["OPENAI_API_KEY"]
trace = open("failed_run.txt").read() # one full trace as text
# Initialize the annotator with your OpenAI API key
MASTAnnotator = annotator(openai_api_key)
# Annotate one full multi-agent trace, passed as text
mast_annotation = MASTAnnotator.produce_taxonomy(trace)
# List the failure modes detected
for failure_mode_id, detected in mast_annotation["failure_modes"].items():
if detected:
print(failure_mode_id, MASTAnnotator.get_failure_mode_info(failure_mode_id))Your production checklist
Where Rerun fits, and what it is not
Rerun is not a framework. It will not replace LangGraph, CrewAI, AutoGen or the Claude Agent SDK, and it does not fix a badly designed topology for you. If your system has a role-specification problem, you still have to redesign the roles. Comparisons like LangGraph vs CrewAI are where that choice is made.
What Rerun does is the operations layer this article keeps coming back to. On Rerun, AI agents are called coworkers, and every coworker's steps are logged as it works: the tools it used, the time, the tokens. A "To Handle" tab collects the approvals and questions waiting for you, and a coworker pauses before a risky action until you approve it in the app. Coworkers run 24/7 on your workspace's private cloud machine, and you can watch the work live.
It is not a tracing dashboard either. Tools like LangSmith or Langfuse show you what went wrong after the fact. Rerun also stops the run before an irreversible step and waits for you.
And it is not a flowchart tool. Zapier, Make and n8n run fixed steps that someone wired by hand, which is great for deterministic tasks, but there is no reasoning to inspect, so they cannot show you a reasoning-action mismatch. And it is not a chatbot: a chatbot talks back, it does not act, so there is nothing to verify.
Rerun is one plan from $29 a month with 5 coworkers and unlimited seats, and there is a free 5-day trial.
Name it, see it, stop it
Multi-agent LLM systems fail the way organisations fail: unclear roles, missed handoffs, nobody checking the work. Better models will shift which failures you see, not end them. So treat reliability as an operations job. Name the failure with MAST, see it in a step-level trace, and put a human at every irreversible step.
Frequently asked questions
Why do multi-agent LLM systems fail?
According to the MAST study from UC Berkeley (NeurIPS 2025), which annotated 1,600+ traces across 7 frameworks, failures come from three sources: system design issues (44.2%), inter-agent misalignment (32.3%) and weak task verification (23.5%). In other words, most failures come from how the agents are organised, not from the model itself.
What is the MAST taxonomy?
MAST, the Multi-Agent System Failure Taxonomy, is a classification of 14 failure modes grouped into 3 categories. It was built by the UC Berkeley Sky Computing Lab from expert-annotated traces (inter-annotator agreement kappa 0.88) and is available as an open-source Python library called agentdash that labels traces automatically.
What is the most common multi-agent failure mode?
Step repetition (FM-1.3) is the most frequent at 15.7% of observed failures: the system redoes a step it has already completed, often the same tool call with the same arguments. It is followed by reasoning-action mismatch (13.2%) and being unaware of termination conditions (12.4%).
Will better LLMs fix multi-agent failures?
Not on their own. The MAST authors conjecture that base model improvements will be insufficient, because many failures are organisational. In their data, GPT-4o had 39% fewer design failures than Claude 3.7 Sonnet on the same tasks, yet both still failed often. A stronger model shifts the failure mix rather than removing it.
Do multi-agent systems fail more often than single agents?
They have more places to fail. The MAST paper notes that multi-agent gains are often minimal compared with single-agent frameworks or best-of-N sampling, and measured failure rates of 41% to 86.7% across 7 open-source systems. Anthropic shows multi-agent setups can win on hard research tasks, but at about 15x the tokens of a chat.
How do you detect multi-agent failures in production?
Log every step of every agent (reasoning, tool call, arguments, result), alert on repeated identical tool calls, set step and cost limits with an explicit done state, and require human approval before irreversible actions. On Rerun, every coworker step is logged and risky actions wait for your approval in the app.
Can prompt engineering fix multi-agent system failures?
Partly. In the MAST case studies, better prompts lifted ChatDev from 25.0% to 34.4% task success on ProgramDev-v0, and a new topology with a review-gated loop reached 40.6%. Both helped, but most tasks still failed, which is why runtime controls and human checkpoints remain necessary.
What is the difference between a multi-agent failure and a bug?
A bug usually reproduces the same way: the same input breaks the same way. Multi-agent failures are often probabilistic and emerge from interaction, such as one agent withholding information another needs, or a stated plan that differs from the action taken. That is why they only become visible in step-level traces, not in unit tests.
Clément Janssens
