Context Engineering: The Most Important Skill for Building AI Agents

Everyone knows prompt engineering. But context engineering is what actually decides whether your AI agent performs well or falls apart. Anthropic recently shared its full experience on this topic, and it is worth understanding in depth.

📖 What Is Context Engineering

Prompt engineering is about writing good instructions. Context engineering is much broader: it is the overall strategy for managing every piece of information an LLM receives at inference time. That includes system instructions, tools, MCP, external data, message history, and everything else sitting inside the context window.

The goal is to find the smallest possible set of tokens that still carries the most signal, so the model produces the best possible result.

Context Rot: The Problem Every AI User Runs Into

As the context window grows longer, model performance tends to drop. Anthropic calls this "context rot." The root cause is the transformer architecture itself: every token has to attend to every other token, creating relationships that scale roughly with n squared. A context window behaves like human working memory, it has real limits.

Stuffing 100K tokens into the context does not mean the model remembers all of it. It will skip details, mix things up, and "forget" information that actually matters.

3 Principles for Designing Context

Principle 1: Find the right altitude. A system prompt needs to be specific enough to guide behavior, yet flexible enough to let the model handle edge cases on its own. Use XML tags or Markdown headers to organize the content.

Principle 2: Tools must be self-contained, robust to error, and extremely clear about their purpose. Avoid letting the tool set bloat with overlapping functionality.

Principle 3: Use diverse, canonical examples instead of trying to list every edge case. A handful of well-chosen examples work better than pages of documentation.

3 Techniques for Long Tasks

When a task outgrows the context window, there are three common ways to handle it:

Compaction: Summarize the conversation history and restart with a compressed context. The key is to preserve architectural decisions, unresolved bugs, and implementation details, and only drop the redundant output.

Structured Note-Taking: The agent writes notes outside the context window and reads them back when needed. A classic example: Claude playing Pokemon and keeping accurate track of game stats across thousands of steps by taking systematic notes.

Sub-Agent Architectures: Specialized agents handle small tasks and return a summary of roughly 1,000 to 2,000 tokens to a coordinator agent. This keeps responsibilities cleanly separated.

Just In Time Retrieval

Instead of cramming all the data into context upfront, keep lightweight references, such as file paths or URLs, and load the actual data only when it is needed. It mirrors how humans work: we do not memorize everything, we know how to look things up.

Claude Code is a good example of this in practice: CLAUDE.md is loaded upfront as fixed context, while grep and glob are used for dynamic discovery whenever needed. This hybrid approach works far better than loading everything upfront.

As models keep getting more capable, agents will need less prescriptive engineering. But context will always be a scarce, valuable resource. Learning how to manage it is a foundational skill for anyone building AI agents.