What a Claude context window is, and why bigger is not better

The phrase turns up in every explanation of these tools and almost nobody stops to define it: what a context window holds, what it does not, and why filling it is usually a mistake.

The Claude context window is the total amount of text the model can see while it answers you. That is the whole of it, and if you read no further you have the useful part. The unit it is measured in is the token, which is roughly a short word or a fragment of a longer one, plus the spaces and punctuation between them. A window described as a million tokens holds about a million of those pieces at once.

What counts towards it, including its own reply

Everything in the conversation takes up room. The standing instructions Claude receives before you arrive, every message you have sent, every document and image you have attached, the results of anything it looks up, and its own answer, including the reasoning it does on the way to that answer.

That last item catches people out. The reply competes for the same space as the question. Hand over an enormous file and ask for an enormous report on it, and you can run out of room in the middle of the report, not at the point of asking.

One window, five claims on it Everything in the conversation shares the same space, including the answer. Context rot system prompt your question documents you paste in tool results Claude's reply people forget this one As the token count grows, accuracy and recall degrade. Anthropic calls this context rot, which is why curating what goes in matters as much as how much room you have.
The reply is the segment nobody budgets for. A window is not a loading bay to be filled to the brim. It is a desk, and the answer has to be written on it too, which is why an enormous request and an enormous document rarely sit together well.

What it is not: memory, or training

A context window is not memory between conversations. Close the chat, start a new one tomorrow, and none of it carries over unless you put it back in front of the model yourself. People assume otherwise and then feel let down when Claude has forgotten last Tuesday.

It is also not the same thing as what the model learned during training. Training is what it knows in general, absorbed once and fixed. A context window holds what it knows about you, today, in this one conversation. If you have never explained the difference between those two kinds of knowing, almost everything else about the tool stays confusing.

Bigger is not automatically better

A million tokens is a serious quantity of material in business terms. A full set of supplier contracts. A year of board papers with the appendices attached. The staff handbook and every amendment made to it since 2019. Most people will never come close to the ceiling in ordinary use, and the number is there for reassurance rather than ambition: whatever you were worried about fitting, it fits.

Having the room, though, is not a reason to use it. Anthropic's own documentation is unusually direct about the cost of overfilling the window:

As token count grows, accuracy and recall degrade, a phenomenon known as context rot. This makes curating what's in context just as important as how much space is available.Anthropic, Context windows documentation

Context rot is a phrase worth borrowing. Load in everything you own and the answers get vaguer rather than sharper, because the paragraph that matters is now sitting among four hundred that do not.

If window size is the figure you have been comparing models on, the tier comparison will save you some time. Once you have settled what to put in front of it, how you frame the request does more for the quality of the answer than any amount of extra material.

Curating beats capacity

The habit I try to build in training is unfashionable: give it the three documents that bear on the question, not the thirty that mention it. Everyone finds this harder than it sounds, because leaving something out feels like a risk and tipping the lot in feels like diligence.

Deciding which three documents those are is your job, and it is the part that requires you to have read them.

Sources

  1. Anthropic — Context windows
  2. Anthropic — Claude models overview