4 Properties of Generative AI: Name the Error, Fix It Right

Every behavior of generative AI, from summarizing a dense report in seconds to inventing a quote that never existed, comes down to four properties: next token prediction, knowledge, working memory, and steerability. Each one is a spectrum, with a strength on one end and a limitation on the other, and the same mechanism produces both ends. Once you know these four properties, when the AI gives you a wrong answer you can name the failure and pick the right fix, instead of just hitting "try again" and hoping for the best.

The useful part is that these four properties stay the same even as models change generations. The boundaries move further out, but the shape stays the same.

A quick everyday analogy

Picture a friend who has read almost an entire library, who can talk fluently about almost any topic, but:

• When he tells a story, he's speaking from memory and from the flow of the conversation, not flipping through a book to check. For familiar topics he gets it right; for unfamiliar ones he sometimes just "fills in the gaps to make the sentence sound complete."
• He only read books up to a certain year, so he has no idea what happened after that.
• He works at a small desk. Whatever paper is on the desk, he can see; whatever falls off the desk, it's as if it doesn't exist.
• Whatever you ask him to do, he follows your words exactly, which sometimes drifts from what you actually meant.

Those four traits are exactly the four properties of generative AI.

Before that: where an AI's "personality" comes from

Claude is polite, helpful, and sometimes overly careful. There's nothing mysterious about it, it's the result of two training stages:

• Pretraining: the AI learns to predict the next token over a massive amount of text. After this stage it only knows how to continue text, it has no concept yet of actually helping you.
• Fine-tuning: the AI is taught to treat your message as a request, to answer directly, and to refuse harmful requests.

Because the fine-tuning stage is built on human ratings of what counts as a "good" answer, it leaves four visible fingerprints in every model:

Try it yourself: pick something you're familiar with and ask three times. First, ask normally. Second, open with a false assumption and see if it pushes back. Third, ask a question that only needs one line and see how long the answer actually is.

How it works: the four properties

1. Next token prediction

Generative AI is closer to an extremely sophisticated autocomplete (like the suggestions on your phone keyboard, but far more capable) than it is to a search engine. It writes text piece by piece based on what usually follows what.

• Strong zone: tasks that resemble patterns it has seen many times, such as summarizing, reformatting, or explaining common concepts.
• Weak zone: unfamiliar topics, sparse data, and anything that requires telling "correct" apart from "sounds correct."

The single most important thing to remember: fabrication concentrates on the most specific details. Proper names, dates, numbers, citations, links. The more precise a detail looks, the more it needs checking. This is explained in more depth in our post on what AI hallucination is.

Self-check: pick a field you know well, ask the AI for five verifiable facts (three sources, one author, one figure), then check each one. Run the exact same request in a new conversation and compare; what changes is just the randomness of text generation.

2. Knowledge

An AI only knows what it actually read in its training data, and it's frozen at its training cutoff. Without external tools, it knows nothing that happened after that point.

• Strong zone: common, stable topics that are written about consistently.
• Weak zone: rare, new, local, or still-contested topics.

Four typical failure patterns: treating old information as current, uneven coverage across topics, inherited bias about what counts as "normal," and no way to trace where a piece of knowledge actually came from.

The right question isn't "does the AI know this?" but "how much was this written about in what it read?" The fix: turn on web search, connect it to your own data source (such as RAG), or feed it the documents directly.

3. Working memory

Everything an AI is currently paying attention to sits inside a fixed-size space called the context window. Inside it, the AI sees; outside it, the AI doesn't. See our post on what a context window is for more.

This property differs from the other three in one key way: it has a cliff edge, not a gentle slope. Performance is fine until it suddenly isn't, and the typical failure mode is silent truncation, nobody tells you it happened.

Two common misunderstandings:

• Burying an important instruction in the middle of a long document makes it very easy to miss. Moving it to the top changes things immediately.
• An AI does not learn from the corrections you give it. It only reacts to what's currently present in that conversation.

4. Steerability

An AI follows your instructions using the exact same mechanism it uses for everything else: continuing a pattern. That's what makes it so easy to steer, and it's also why there's always a gap between what you want and what it understands.

• Strong zone: short, specific, checkable instructions: format, length, role.
• Weak zone: long chains of reasoning, abstract or ambiguous instructions, and anything requiring perfect precision with numbers and logic.

Two typical failure modes:

• Reasoning drift: a small error early on compounds by step five into something completely wrong. Fix: insert checkpoints, have it pause at step two so you can review before continuing.
• Right words, wrong intent: you say "make it shorter" when the real problem is structure, so it trims the length but the piece is still weak. Fix: state the actual goal, not just the format.

Repeating the same instruction more forcefully doesn't close this gap. Switching to stating the goal clearly does. Our post on what a prompt is has plenty of examples of how to write clear instructions.

What this means for you

In practice, an error is rarely caused by just one property. It's usually two properties intersecting. Naming that pair tells you exactly how to fix it:

Three practical examples:

Turn a format request into a goal. Instead of "make it shorter," write:

Make this shorter. Goal: a director should be able to read it in 1 minute and still get the conclusion and the key numbers.

Ask neutrally to avoid easy agreement. Instead of "This plan is solid, right?" ask: "List the 3 biggest weaknesses of this plan, ranked by risk level."

Don't let the AI do mental math. For payroll, quotes, or financial figures, have the AI write the formulas for a spreadsheet, then let the spreadsheet do the actual calculation.

The practical takeaway: trust should be calibrated, not a switch you flip on or off. Anything well inside the AI's strong zone is usually fine to read and use as is. Anything that's specific, new, long, or multi-step needs a verification step built into the process.

Common misconceptions

Misconception 1: "AI looks things up like Google." Reality: without search turned on, the AI doesn't look anything up. It generates its answer from what it already learned, so it can "misremember" something while sounding completely confident.

Misconception 2: "If I correct it once, it'll remember next time." Reality: an AI doesn't learn from your corrections. For it to remember, you need to put that information back into the conversation, or into the app's memory or persistent instructions feature.

Misconception 3: "The newer model will have fixed all these issues." Reality: newer models push the boundaries further out, but the four properties are still there. The habit of verifying still applies no matter which model you're using.

FAQ

What are the four properties of generative AI? Next token prediction, knowledge (with a training cutoff), working memory (a limited context window), and steerability (it follows instructions but can miss the actual intent).

Why does AI tend to fabricate names, numbers, and links? Because it writes according to familiar patterns. The more specific and rare a detail is, the more likely it gets "filled in to sound complete." That's exactly where you need to check most carefully.

Why does the AI keep agreeing with me? This is a fingerprint of the human-feedback fine-tuning stage. Invite it to push back and phrase your questions neutrally.

How do I get the AI to understand what I actually mean? State the actual goal instead of just the format, put the most important instructions first, and break multi-step work into stages with checkpoints in between.

Do newer models remove these limitations? Not entirely. The boundaries shift further out, but these four properties are still how generative AI fundamentally works.