An agent that can act can also act wrong, and by the time anyone notices, it has already sent the refund emails, booked the appointments, or pushed the charge.
OWASP's Top 10 for LLM Applications names excessive agency, an agent taking real-world actions beyond what a human actually approved, as one of the core risks of running autonomous systems in production. Rollback is what you do about that risk after the fact. Not a theoretical safeguard, the actual mechanics of undoing what already happened.
This guide walks through a four-step framework: classify, contain, compensate, reconcile. It assumes you already deploy agents and ship updates to them regularly. The question here is narrower and more operational: when a deployment goes wrong, what do you actually do, beyond git revert?
In a hurry? See a coworker's full action log live.
Why agent rollback isn't code rollback (the two-jobs problem)
A bad deploy has always meant two different problems. Most teams only built tooling for one of them.
Job 1: revert the artifact. The code, the prompt, the config, the model version. This is solved. Blue-green deploys, feature flags, git revert, a previous Docker tag. Minutes of work, well understood, decades of tooling behind it.
Job 2: undo the side effects. The emails the agent sent while running the bad version. The appointments it booked. The refunds it approved. The rows it wrote to a customer's CRM, the messages it posted to a shared channel, the invoice it sent to the wrong account. This is the job almost nobody has built for, because it didn't used to exist at this scale. A buggy web app serves a wrong page to a browser; a buggy agent takes a wrong action in the world, and the action doesn't wait for anyone to notice before it has consequences.
Reverting the artifact stops the bleeding. It does nothing about the wound already there. An agent that spent four hours running a flawed prompt before anyone caught it may have completed hundreds of real-world actions on that flawed logic, each one now sitting in someone's inbox, calendar, or bank statement. Job 1 is done the moment the new version deploys. Job 2 starts there, and it's usually the one that costs the afternoon.
The rest of this guide is about Job 2, because Job 1 is a solved problem and Job 2 usually isn't, not because nobody has thought about it, but because it has to be designed in before the incident, not improvised during one.
Classify every action first: read-only / reversible / compensatable / irreversible
You cannot decide how to undo an action you haven't classified. Classification has to happen at design time, when you write the tool, not during an incident when you're reading logs under pressure and guessing what a function actually touched.
Four buckets cover almost everything an agent does:
- Read-only. Fetching a record, searching the web, summarizing a document. No rollback needed, nothing changed in the world.
- Reversible. Updating a CRM field, moving a card on a board, changing a status. The old value is still known somewhere; writing it back undoes the action cleanly and completely.
- Compensatable. Sending a refund, booking then cancelling a meeting, posting then deleting a message. The original action can't be un-sent or un-booked, but a second, deliberate action neutralizes its practical effect.
- Irreversible. An email a human already read and acted on. A wire transfer that cleared. A public post that got screenshotted before it came down. Nothing undoes these; the only lever left is damage control and honest disclosure.
The classification work belongs in your agent governance layer, as a property on every tool definition, not as a note in a wiki someone updates twice a year and nobody reads before an incident. A tool schema that doesn't carry an action_class field is a tool nobody has actually thought about rolling back, and that gap only shows up once the action has already run.
{
"tool": "send_refund",
"action_class": "compensatable",
"compensation": "reverse_refund",
"max_auto_approve_amount_usd": 50
}Tag the tool once, and every downstream decision, what needs a checkpoint, what needs a human in the loop, what needs a kill switch, follows from that one field instead of being re-litigated action by action, under pressure, during the incident itself.
Checkpoints + idempotent tool calls
Once actions are classified, two mechanical pieces make rollback possible at all: checkpoints and idempotency. Neither is exotic, both are frequently skipped.
A checkpoint has to capture more than a code version. It needs a state snapshot (what did the database, the CRM, the inbox look like before this run started) and an action log (every tool call the agent made, its class, and its result). A checkpoint that only records "deployed commit abc123" tells you nothing about what the agent did between that deploy and the moment someone pulled the rollback trigger. Testing agent behavior should include testing the rollback path itself, not just the happy path, because a rollback procedure nobody has run before is a rollback procedure that will fail the first time it actually matters, usually at the worst possible moment.
Idempotency is what makes retries and replays safe instead of dangerous. Stripe's idempotent request pattern is the reference implementation worth copying directly: a client generates a unique key per logical operation, and the server guarantees the operation never runs twice for that key, even across retries, timeouts, or network failures. Apply the same pattern to every tool call your agent makes that writes to an external system. Without it, a rollback that retries a half-finished action risks running it a second time, which is frequently worse than the original failure, a duplicate refund on top of a wrong one, a second confirmation email contradicting the first.
This is runtime-layer work. It belongs next to whatever handles agent infrastructure in your stack, the layer that persists state, manages retries, and keeps a durable log, not something bolted onto each individual agent as an afterthought.
A checkpoint tells you where you were. An idempotency key tells you it's safe to get back there without doing any of it twice.
Compensating actions (saga pattern) for multi-step tasks
Most of what an agent breaks isn't a single action, it's a chain. The agent books a flight, then a hotel, then sends a confirmation email, then the flight step turns out to have used the wrong dates. You don't want to undo the whole chain by hand, re-reading logs to reconstruct what happened in what order.
The saga pattern was built for exactly this shape of problem in distributed systems, long before agents existed to make it worse. Microsoft's Azure Architecture Center documents the two classic implementations, choreography and orchestration, and the same split shows up in workflow engines built for long-running, message-driven processes.
Temporal Workflow message passing - Signals, Queries, & Updates | Temporal DocumentationBreak a transaction into a sequence of local steps, and define a compensating action for each one up front, at design time. If step three fails, the saga runs the compensations for steps two and one, in reverse order, instead of someone inventing an undo procedure after the fact under time pressure.
Applied to an agent workflow, every forward action gets its compensation written alongside it when the tool is built, not improvised during an incident:
| Forward action | Compensating action |
|---|---|
| Book flight | Cancel flight, refund if within policy |
| Book hotel | Cancel reservation |
| Send confirmation email | Send correction email, flag for human follow-up |
Long-running or background agents are where this matters most, because the gap between "action taken" and "problem discovered" is widest: a task that runs overnight or over several days can accumulate dozens of forward actions before anyone looks at the output. A saga that defines compensations only for the steps someone remembered to think about leaves exactly the gaps that show up during an actual incident, usually the ones nobody considered because they seemed unlikely.
Contain before you revert: kill switches and the unknown-outcome queue
Classification and compensation assume you know what the agent did. Containment is what you do when you don't, yet, and you need to stop the bleeding before you can even start cleaning up.
Kill switches need three levels, not one: per-agent (stop this specific run, leave everything else running), per-tool (disable send_email across every agent while you investigate, without taking the whole system down), and global (stop everything, for the rare case that actually warrants it). A single all-or-nothing kill switch is a blunt instrument that teams are reluctant to pull, which means in practice they pull it too late, once the damage is already wider than it needed to be. Three levels mean the response matches the actual blast radius instead of over-reacting or under-reacting by default.
Containment also means routing uncertain outcomes somewhere a human can actually look at them, rather than guessing and moving on. Call it the unknown-outcome queue: any action whose real-world result can't be confirmed programmatically (did that email actually get read before the correction went out, did that payment actually settle or just get submitted) gets logged there instead of assumed safe by default. This is the same mechanism good human-in-the-loop agent design already builds for approvals, just pointed at post-incident triage instead of pre-action gating. The agent pauses, a person looks at the specific ambiguous case, and the run resumes exactly where it stopped once a decision is made.
Choosing a combination by workflow risk (decision table)
Not every workflow needs every control. Matching the control to the actual risk keeps the system usable instead of grinding every single agent action to a crawl behind approval gates it doesn't actually need.
| Action class | Recommended control | Typical workflow |
|---|---|---|
| Read-only | Checkpoint only (for audit, not rollback) | Research, summarization, lookups |
| Reversible | Idempotent writes, automatic retry | CRM updates, ticket status changes |
| Compensatable | Saga with a pre-written compensation | Bookings, refunds, scheduled sends |
| Irreversible | Human-gated kill switch, no auto-approve | Wire transfers, public posts, legal filings |
Two worked examples make the table concrete:
-
A customer-support agent mostly moves tickets and drafts replies: reversible and compensatable territory, light on approval gates, fast on automatic retries, because the cost of a mistake is a correction, not a disaster.
-
A finance agent moves money: every irreversible action routes through a human approval by default, every compensatable one logs a written compensation before it fires, not after, and the kill switch sits one click away in the dashboard, not three menus deep where nobody finds it under pressure.
-
Every tool definition carries an
action_class -
Every compensatable action has a written compensation, not an improvised one
-
Idempotency keys cover every external write
-
Kill switches exist at agent, tool, and global level
-
Irreversible actions require human approval by default, not by exception
A team building this on n8n or Zapier hits a hard limit here: a node graph is a sequence of steps someone wired once and hoped would hold, with no native concept of compensation, action classification, or a kill switch that actually stops a run mid-flight instead of just the next scheduled trigger. On Rerun, these agents are called coworkers, and the action log, the classification, and the approval gate are part of how a coworker runs by default, not something bolted on top of a flowchart after the fact. A coworker's observability layer is what makes classification auditable in the first place: every action, its class, and its outcome, visible on a live dashboard instead of buried in a log file nobody reads until something has already broken.
A coworker whose entire job is owning this classification and audit trail is already in the library.

AI Governance Lead
Map Your AI Systems and Risks
Five minutes of setup, and every new tool an agent gets access to is tagged, reviewed, and logged before it ever runs against production data, instead of after the first incident forces the question.
Reconcile: close the loop after containment
Classify, contain, and compensate stop the damage and start undoing it. Reconcile is the step that confirms the undo actually worked and turns the incident into something the next deployment benefits from. Skip it, and the same gap reopens the next time a tool changes.
Reconciliation has three concrete parts, not a vague "do a post-mortem":
- Diff the action log against ground truth. For every compensatable and irreversible action taken during the bad window, confirm its actual real-world state (was the refund received, was the cancelled booking actually removed from the calendar) rather than trusting that the compensating action succeeded just because it ran without an error.
- Close every item in the unknown-outcome queue. Nothing stays in that queue past the incident. Each entry gets a confirmed outcome and, where the outcome can't be confirmed, an explicit note that it's accepted as unresolved, not silently dropped.
- Feed the gap back into classification. If an action that was tagged reversible turned out to need manual compensation, or an action nobody classified caused the incident in the first place, that's a correction to the
action_classfield, not just a lesson for a wiki page.
The NIST AI Risk Management Framework frames this kind of work as a continuous practice rather than a one-time checklist to clear, and reconciliation is where that framing becomes concrete: the loop doesn't end when the compensating actions finish running, it ends when someone has confirmed the real world actually matches what the log says happened.
The operating loop, not a one-time setup
Classify, contain, compensate, reconcile isn't a project you finish once and move on from. It's the loop that runs every time something goes wrong, which means the design-time work, tagging actions, writing compensations, wiring kill switches at three levels, is what actually determines whether that loop takes ten minutes or two days when it fires for real. You don't get to design it once, ship it, and consider the problem solved. Every new tool the agent gets access to reopens the same four questions, classify, contain, compensate, reconcile, and the teams that answer them before the incident are the ones who spend an afternoon on it instead of a week.
Frequently asked questions
What is AI agent rollback?
Agent rollback means reverting both the code behind an agent and the real world actions it already took, like sent messages, bookings, or payments, not just the deployed version.
How is agent rollback different from a normal code rollback?
A code rollback undoes the deployed artifact in minutes. Agent rollback also has to undo or compensate for actions the agent already executed in external systems, which a code revert cannot touch.
What is the saga pattern and why does it apply to agents?
The saga pattern breaks a multi step transaction into local steps, each with a pre written compensating action, so a failure partway through can be undone step by step instead of all at once.
What is an idempotency key and why do agents need one?
An idempotency key is a unique identifier attached to a tool call so retrying it never performs the action twice, which matters when a rollback retries a half finished agent action.
How should a kill switch work for an AI agent?
A kill switch needs three levels: stopping one agent run, disabling one tool across every agent, and stopping everything, so the response matches the actual scope of the problem.
When should a human review an agent's action before rollback?
Any irreversible action, and any action whose outcome can't be confirmed programmatically, should route to human review by default rather than being assumed safe.
How do you know a rollback actually worked?
A rollback is only confirmed once reconciliation closes it: the action log is diffed against real world ground truth, every unknown outcome queue item has a confirmed result, and any misclassified action is corrected for next time.








