Building Tools for AI Agents: 5 Principles Anthropic Used to Hit SOTA on SWE-bench

An agent is only as good as the tools you hand it. Anthropic shared 5 tool-design principles that helped them reach state-of-the-art results on SWE-bench. The most interesting part: they used Claude itself to optimize tools for Claude.

Principle 1: Choose the Right Tools

More tools does not mean better tools. Instead of wrapping every API endpoint into its own tool, design tools around how an agent actually works. Agents have limited context, so tools need to search efficiently rather than dump everything back at once.

For example, use search_contacts instead of list_contacts. Combine multiple operations into a single tool: schedule_event (instead of separate list and create calls), or get_customer_context (which compiles every relevant piece of customer data into one place).

📛 Principle 2: Clear Namespacing

Group related tools with a consistent prefix, like asana_search and asana_projects_search. This cuts down confusion for the agent and saves context. Should you use a prefix or a suffix? You have to test both. Do not guess.

Principle 3: Return Meaningful Context

Return high-signal information, prioritizing relevance over flexibility. Instead of returning a uuid and mime_type, return a name and file_type. Agents perform better with natural language identifiers than with cryptic technical codes.

One useful technique: add a response_format parameter with an enum of "concise" (72 tokens) or "detailed" (206 tokens). The agent picks the level of detail it needs, similar to GraphQL field selection.

Principle 4: Save on Tokens

Implement pagination, filtering, range selection, or truncation with sensible defaults. Claude Code caps tool responses at 25,000 tokens by default.

When truncating, steer the agent toward a more efficient strategy with clear error messages. Do not return cryptic error codes. Write "Results truncated. Try narrowing your search" instead of "ERR_OVERFLOW".

Principle 5: Tool Descriptions Are Prompt Engineering

Tool descriptions have an outsized effect on performance. Write them as if explaining to a new colleague, turning implicit context into explicit instructions. Use clear parameter names, like user_id instead of just user.

Anthropic fine-tuned its tool descriptions for Claude Sonnet 3.5 and reached state-of-the-art SWE-bench performance. They put more effort into tool descriptions than into the overall prompts.

Using AI to Optimize Tools

Here is the most interesting part: Anthropic used Claude to analyze evaluation transcripts and propose tool improvements. Claude-optimized tools outperformed human-written versions on held-out test sets.

The process: write a tool prototype, run an eval, have Claude read the transcript, let Claude propose a fix, apply the fix, then run the eval again. Repeat until the results meet the bar. Use held-out test sets to avoid overfitting.

Designing tools for AI agents is a lot like UX design for humans. Invest in the agent-computer interface (ACI) just as much as the human-computer interface (HCI).