AI Grading Its Own Work Is Why Products Turn Out Bad (and Anthropic's Fix)

Anthropic just published how they build AI agents that actually produce great work, not just "good enough" output. The core problem they found is simple: AI is way too easy on itself when grading its own work.

What Is Context Degradation

When you run Claude on long, complex tasks, the model starts losing the thread. As the context window fills up, Claude Sonnet 4.5 can show a kind of "context anxiety," wrapping up work early even when it isn't finished. The result is a half-built product.

The second problem is even worse: the AI rates its own output far too generously. Ask it "does this code work?" and it says "yes, looks great." In reality, the app is completely broken.

The Fix: Separate the Generator From the Evaluator

Anthropic borrowed an architecture inspired by GAN networks. Instead of one AI doing everything, they split the work across three agents. The Planner turns a short prompt into a detailed spec. The Generator builds the app using React, FastAPI, and SQLite. The Evaluator uses Playwright to actually run the app in a real browser, clicking through every button, then scores it against four criteria: design quality, originality, craft, and functionality.

This loop runs 5 to 15 times until the product meets the bar.

The Results

In a test building a retro game maker, a single pass took 20 minutes and cost about $9, but the game was broken, with faulty entity-runtime wiring. Running it through the full harness took 6 hours and cost about $200, but produced a complete game with polished UI and a built in Claude agent for generating sprites from prompts.

One surprising moment: in one run, the Generator abandoned the standard museum website layout entirely and rebuilt it as a spatial 3D experience using CSS perspective. That kind of creative leap only shows up when there's a feedback loop in place, it is not something you get from a single pass generation.

With Claude Opus 4.6, Anthropic has already started stripping scaffolding away, because the model plans well enough on its own now. Sprint decomposition is no longer needed, and QA has moved from per-sprint checks to a single pass.

The Big Lesson

A harness architecture has to keep adapting as the underlying model gets stronger. When the model improves, drop the parts that are no longer necessary and add new capabilities instead. The interesting design space for a harness does not shrink as models get more capable, it just shifts to a new set of problems.

If you are building an AI agent and everything it produces comes out "fine" but never actually great, this is likely why. The AI needs a separate agent to give it honest feedback, not just pat itself on the back.