Context Engineering for AI Agents: Patterns, Tools, and Best Practices

Research on long-context language models found a stubborn production problem: models often use information less reliably when the relevant evidence sits in the middle of a long input. The model may technically accept the text and still fail to use it. That is why a larger context window is not a substitute for context engineering.

Context engineering is the deliberate selection, structure, sequencing, and updating of everything an AI model receives at inference time. For an agent, that includes instructions, task state, retrieved documents, memory, tool definitions, tool results, permissions, examples, and output constraints.

The goal is not to fill the window. It is to give the agent the smallest, freshest, highest-signal set of information needed to make the next decision correctly.

A context window is capacity. Context engineering is deciding what deserves the model's attention.

Context engineering in brief

• Context engineering manages the model's full runtime information environment.
• Prompt engineering is one part of it, not a competing discipline.
• Good context is assembled for the current decision, not dumped in once for the entire task.
• Retrieval, memory, tools, permissions, state, and evaluation all belong in the design.
• More tokens can increase cost, latency, contradiction, and attack surface.

What is context engineering?

Context engineering is the practice of deciding what an AI system can see, what it should ignore, how information is organized, and when that information changes during a task. It treats context as a system that must be designed and tested, not as a text box that someone fills with clever instructions.

An agent's context can contain:

• System and developer instructions
• The current request and definition of done
• Conversation history
• A plan, completed steps, and open questions
• Retrieved documents and database records
• Working, episodic, semantic, and procedural memory
• Tool names, descriptions, schemas, and permissions
• Tool-call results
• Examples and output schemas
• Environmental facts such as the current time, user, workspace, or account

Context window vs. context

The context window is a capacity limit measured in tokens. Context is the information actually placed inside that capacity. A warehouse can hold ten thousand boxes, but that does not mean every box belongs beside the worker making one decision.

Large windows are useful. They let an agent inspect more source material and maintain longer trajectories. But capacity does not guarantee relevance, freshness, correct ordering, or security. The engineering work begins after the capacity is available.

Context engineering vs. in-context learning

In-context learning describes a model's ability to infer behavior from instructions and examples in its input. Context engineering designs the pipeline that chooses those instructions and examples, adds state and evidence, exposes tools, and updates the input after each action.

Context engineering vs. prompt engineering

The phrase became popular partly because teams discovered that rewriting a prompt could not repair missing data, stale memory, ambiguous tools, or broken state. Prompt engineering still matters. It simply operates inside a larger system.

Prompt engineering asks, “How should I explain this task?” Context engineering also asks:

• Did we retrieve the right source?
• Is that source current and authorized for this user?
• Does the agent know which steps are already complete?
• Are the available tools relevant to this decision?
• Can the agent distinguish trusted policy from untrusted webpage text?
• Will the result be recorded in a form the next step can use?

When a prompt problem is actually a context problem

If an agent invents a return policy, the prompt may not be the cause. The current policy may never have been retrieved. If it repeats work, the completed action may not have been persisted. If it calls the wrong tool, two descriptions may overlap. If it ignores a fact, that fact may be buried under pages of raw output.

Changing “Please be careful” to “You must be extremely careful” fixes none of these failures.

Why context engineering matters more for agents

A chatbot can answer one question and stop. An agent acts across multiple model calls while its environment changes. It reads, decides, calls a tool, receives new data, updates state, and decides again. Every transition creates a new context problem.

This is the key difference between an agent and a flowchart. Zapier, Make, and n8n execute branches that a person specified in advance. A flowchart does not need to decide which evidence matters. An agent does, and that decision must remain observable and governable.

A chatbot has a similar limitation from the other direction. It can discuss a refund, but it does not necessarily verify the account, inspect the transaction, apply policy, request approval, execute the refund, and record the result. Context engineering supports that operating loop.

The context window is an attention budget

The “lost in the middle” research showed that long-context performance can vary with the position of relevant evidence. The safe conclusion is not that every model always fails in the middle. It is that token capacity and effective use are different things, so ordering and selection need empirical testing.

Excess context can cause:

• Higher inference cost and latency
• Worse cache efficiency
• Conflicting instructions or facts
• Distraction from the current decision
• More untrusted text exposed to the model
• Harder trace review and debugging
• Lossy summaries that become mistaken for source truth

Security makes selection even more important. OWASP's prompt injection guidance explains why instructions hidden in external content can redirect a model. Context pipelines should label untrusted material, separate it from privileged instructions, constrain available tools, and require approval before consequential actions.

Effective context engineering for AI agentsAnthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.anthropic.com

Seven common context failures

The anatomy of an agent's context

Useful context has layers. Each layer has a different owner, update rate, and trust level.

Instructions and policies

Stable instructions define the role, boundaries, permissions, escalation rules, and output contract. Keep them separate from task-specific requests. Retrieved documents should never be allowed to masquerade as system policy.

Working state

Working state describes the current goal, completed steps, artifacts, unanswered questions, and next action. Structured state is often more dependable than replaying an entire transcript because it makes progress explicit and recoverable.

Retrieved knowledge

Retrieval-augmented generation finds candidate evidence. Context engineering decides which candidates reach the model, in what order, with which timestamps, permissions, and citations. Retrieval is a component, not a synonym.

Hybrid search, metadata filters, reranking, query rewriting, and recency weighting all help. Access control must happen before content enters the model, not after the answer has been generated.

Memory

Memory needs clear types and policies:

• Working memory: temporary state for the current task
• Episodic memory: prior interactions and events
• Semantic memory: durable facts and preferences
• Procedural memory: reusable workflows, rules, and skills

Every memory system needs write criteria, retrieval criteria, expiration, deduplication, and conflict resolution. Saving every conversation forever creates a landfill, not intelligence.

Tools and tool results

Tool names, descriptions, parameters, examples, permissions, and results all consume context. Overlapping descriptions create tool-selection errors. Raw API responses create noise. Exposing every connector at every step widens both the attention problem and the security surface.

Examples and output schemas

Few-shot examples should resemble the current decision. Output schemas should be typed and validated. A generic example that adds tokens but does not constrain behavior is decoration.

Four context engineering patterns

The most useful operating taxonomy is write, select, compress, and isolate, a framing also used in the external LangChain context engineering article.

Write context

Persist completed actions, verified facts, artifacts, and task state outside the active prompt. A durable record lets the next model call reconstruct the task without carrying every prior token. Writing is also a governance decision. Sensitive or transient data should not automatically become long-term memory.

Select context

Retrieve information just in time. Filter by account, role, date, task, and permissions. Rerank for relevance and authority. Load only the tools the current step could reasonably use. Selection is the most direct way to turn a large knowledge universe into a small decision packet.

Compress context

Summaries, structured extraction, tool-output pruning, and deduplication reduce noise. Compression should preserve obligations, unresolved questions, source links, and uncertainty. Keep raw evidence reachable for verification. A confident summary of a hallucination is still a hallucination.

Isolate context

Separate unrelated tasks, untrusted content, or specialized work into independent execution threads. Subagents can reduce interference, but their handoffs need explicit contracts. Isolation without a good handoff merely moves the context failure to a boundary.

A practical context engineering architecture

The architecture should revolve around decisions, not around the maximum amount of available data.

Task
  ↓
Decision definition → Context policy
  ↓                    ↓
Candidate retrieval → permission checks
  ↓
Rank + filter + resolve conflicts
  ↓
Assemble decision context
  ↓
Model or tool action
  ↓
Validate result → update state → evaluate trace

1. Define the decision

Name the exact decision for this step: classify the request, choose a tool, determine eligibility, draft a response, or request human approval. “Solve the support ticket” is too broad to guide context selection.

2. Establish a context policy

Specify mandatory rules, permitted sources, recency requirements, retrieval limits, latency budget, memory retention, security boundaries, and required provenance.

{
  "decision": "Determine whether a subscription refund may proceed",
  "required_context": ["verified identity", "payment event", "current refund policy"],
  "allowed_tools": ["read_billing_account", "request_refund_approval"],
  "freshness": {"refund_policy": "current version", "payment_event": "live"},
  "human_review": "required above $100 or when policy evidence conflicts",
  "output": ["eligibility", "evidence_ids", "next_action", "uncertainty"]
}

3. Generate candidates

Search the relevant document store, database, memory, files, and tool catalog. Apply authorization filters before ranking. An excellent reranker cannot repair a privacy breach caused by retrieving another customer's record.

4. Rank, filter, and resolve conflicts

Score candidates for relevance, authority, freshness, and permission. When sources disagree, surface the conflict and apply a declared rule. Silent concatenation forces the model to improvise authority.

5. Assemble predictable context

A useful default order is:

1. Stable instructions and safety boundaries
2. The current objective and definition of done
3. Structured task state
4. Selected evidence with timestamps and source IDs
5. Available tools and permissions
6. Output contract
7. The current decision request

Treat this as a hypothesis to test, not a universal law.

6. Execute and update state

Record the attempted action, validated result, relevant evidence, and new state. Avoid storing private chain-of-thought or every transient detail. Store the facts and artifacts that future steps need.

7. Evaluate the trace

Inspect what context was available at each decision. Final answers alone cannot reveal whether success was repeatable or accidental.

Worked example: a support refund agent

Consider an agent handling a subscription refund request.

The naïve context

The agent receives the entire conversation, the whole help center, every support tool, raw billing JSON, old customer interactions, and general company instructions. It costs more, takes longer, and may mix an obsolete policy with the current transaction.

The engineered context

For the eligibility decision, the agent receives:

• The customer's current request
• Verified identity and account ID
• The relevant subscription and payment events
• The current refund policy section with version and source ID
• A precomputed eligibility result
• Only billing-read and escalation tools
• A required output schema
• A human-review threshold

For the execution decision, it gets a different packet: the approved amount, approval record, idempotency key, refund tool, and audit requirements. The agent does not need the entire help center again.

The table makes no invented performance promise. Teams should measure token use, retrieval precision, tool accuracy, latency, unsupported claims, escalation accuracy, and task success on their own workflow.

Context engineering tools

Choose tools by capability, not by logo count.

Orchestration frameworks

LangGraph, LlamaIndex, Semantic Kernel, AutoGen, CrewAI, and the OpenAI Agents SDK can help manage state, tools, and multi-step execution. A custom state machine may be simpler for bounded workflows. Assess explicit state, checkpoints, context hooks, tool filtering, isolation, observability, and human approval.

A framework does not make context correct. It gives you places to implement the policy.

Independent technology assessments increasingly treat this as an architectural practice rather than a prompt-writing trick. Thoughtworks' Technology Radar entry is useful corroboration: the discipline spans the information and capabilities made available to a model, not merely the wording of one request.

Retrieval and memory infrastructure

SQL databases suit structured, frequently updated records. Full-text search handles exact terms. Vector search helps semantic discovery. Graph databases support relationship-heavy queries. Object storage keeps large artifacts. Dedicated memory layers may manage extraction and recall.

Do not put every fact into a vector database. Storage should match structure, update pattern, query type, and deletion requirements.

Tool interfaces

Function calling, JSON Schema, typed tool definitions, and the Model Context Protocol make capabilities machine-readable. Dynamic discovery can reduce prompt size, but permission checks and distinguishable descriptions remain essential.

Observability and operations

Production teams need to inspect the instructions, memories, evidence, tool definitions, approvals, and results available for every important decision. This is where Rerun fits: it is a framework-agnostic operations layer for running agents on dedicated infrastructure, watching actions live, reviewing logs, and pausing sensitive work for human approval. It is not the context framework itself, and it does not replace retrieval, memory, or evaluation code inside your agent.

That distinction matters. Rerun is not Zapier with an AI label, not another flowchart builder, and not a chatbot window. It provides the operational visibility and human control needed after an agent begins acting.

How to evaluate context quality

Evaluation should combine business outcomes with trace-level evidence.

Run golden task sets and inspect traces. Remove one context component at a time to see whether it contributes. Test adversarial retrieval, stale facts, conflicts, injection attempts, long trajectories, and recovery after tool failure.

An ablation test is especially useful. If removing three pages of “helpful background” leaves success unchanged while reducing latency, those pages were not useful context.

Context engineering best practices

1. Start with the decision the agent must make.
2. Treat context as a limited attention budget.
3. Separate stable instructions from dynamic evidence.
4. Prefer structured state over replaying the full transcript.
5. Retrieve just in time instead of preloading everything.
6. Preserve provenance, timestamps, permissions, and versions.
7. Define explicit memory-write, update, and expiry policies.
8. Make tool descriptions mutually distinguishable.
9. Compress selectively and retain access to raw evidence.
10. Isolate unrelated tasks and untrusted content.
11. Version prompts, retrieval policies, schemas, and memory logic.
12. Evaluate changes against task success, cost, and latency.

The best context is not the longest context. It is the context that supports the next correct decision.

Common mistakes to avoid

• Filling the window because capacity is available
• Treating RAG as the whole of context engineering
• Saving every interaction as durable memory
• Injecting unrestricted raw tool output
• Mixing trusted instructions with untrusted retrieved text
• Letting old summaries become unquestioned truth
• Exposing every tool at every step
• Omitting source timestamps and permissions
• Optimizing tokens without measuring task success
• Adding subagents where one isolated workflow is enough
• Reviewing only final responses instead of execution traces

August 13, 2026

AI Agent Memory: How Agents Store, Retrieve, and Learn

How AI agent memory works: short-term vs long-term, the episodic, semantic, and procedural types, how agents store and retrieve with vector search, how they learn, and how to govern memory in production.

Context engineering checklist

Use this before deploying an agent:

• The current decision is clearly defined.
• Every source is authoritative and permission-checked.
• State is structured, recoverable, and versioned.
• Tool definitions are unambiguous.
• Retrieved text is separated from system instructions.
• Evidence keeps provenance and timestamps.
• Memory writes and expiry are controlled.
• Context limits come from tests, not intuition.
• Every important execution can be traced.
• Injection, stale-data, and conflict cases are covered.
• Human approval blocks sensitive actions before execution.
• Cost, latency, and task success are measured together.

If a failure occurs, classify it before editing the prompt. Was the problem caused by the model, missing evidence, bad retrieval, stale memory, ambiguous tools, invalid state, unsafe permissions, or poor assembly? Change one policy at a time and compare the trace.

From context window to operating discipline

Context engineering turns an agent's information environment into an explicit architecture. Write durable state, select decision-relevant evidence, compress without discarding obligations, and isolate work that should not interfere. Then test the assembled context against real task outcomes.

Start with one consequential workflow. Trace its failures, label the context failure behind each one, and change a single policy. The aim is not a spectacular prompt. It is an agent whose next action can be understood, evaluated, and stopped when needed.