What is context rot and how do you avoid it?

Short answer

Context rot is the decline in an AI model's accuracy and recall as the amount of text in its context window grows. Chroma's 2025 study of 18 models found performance gets less reliable with longer input, even on simple tasks. You avoid it by keeping the context small and relevant: retrieve only what the task needs, clear old tool output, compact long sessions and start fresh when the task changes.

What the research shows

Anthropic’s documentation describes it in one line: “As token count grows, accuracy and recall degrade, a phenomenon known as context rot.” A well-known measurement is Chroma’s technical report “Context Rot” from July 2025, by Kelly Hong, Anton Troynikov and Jeff Huber. They tested 18 models, including GPT-4.1, Claude 4, Gemini 2.5 and Qwen3. Their main findings:

Finding What it means in practice
Performance gets less reliable as input gets longer, even on simple tasks A model that finds a fact perfectly in a short document can miss it in a long one
A single distracting passage already lowers performance, and four lower it more Near-miss content (an old version, a similar customer) is worse than unrelated text
On LongMemEval, focused prompts with only the relevant part did much better than the full input of around 113k tokens Giving the model the whole conversation history is worse than giving it the right slice
Claude models more often declined when unsure; GPT models more often answered confidently and wrongly when distractors were present Rot shows up differently per model, sometimes as refusals, sometimes as errors

Note what the study does not claim: it does not give a final explanation of why this happens. The effect is measured; the mechanism is still open.

The common misunderstanding

Most people assume the context window is the limit that matters: if it fits, the model can use it. With current Claude models offering a 1M-token window, it is tempting to load the whole repo, all contracts or the full mail archive. Context rot says the opposite. Fitting is not the same as being used well. Anthropic’s own documentation frames the window as working memory and stresses that more context is not automatically better, so curation counts.

How to avoid it

  1. Retrieve, don’t paste. Keep references in context (file paths, links, queries) and let the AI load what the current step needs. Anthropic calls this just-in-time retrieval.
  2. Clear old tool output. Search results and file dumps that are no longer needed only distract, though unresolved errors and references you may need again are worth keeping. Claude’s API supports clearing old tool results and thinking blocks through context editing.
  3. Compact long sessions. Replace older turns with a summary and keep recent turns word for word. The Claude API offers compaction on demand and at a token threshold, both in beta. See what is context compaction.
  4. Use sub-agents for side tasks. A sub-agent searches or reads in its own clean window and returns a short summary, so the main session stays focused.
  5. Start fresh when the task changes. A new task in an old session drags along everything from the old one. A short handover note beats 200k tokens of history.
  6. Remove near-duplicates. Old versions of a document, outdated rules and contradicting instructions add plausible but misleading context; remove them when they are not needed.

What it means for a team

Context rot has a team version. When knowledge lives in long personal chat histories, the only way to “give the AI context” is to keep that history going, which is exactly what rots. And when ten people each paste their own version of the pricing sheet or the style guide into their sessions, the AI meets the kind of similar-but-wrong passages that Chroma used as distractors.

The fix is the same as for one session, but organized: one current version of each piece of team knowledge, stored outside any chat, and retrieved per question. Then a fresh session needs little re-briefing, because nothing important lived only in the old one, although retrieval can still miss something. More on setting that up in how to set up context engineering for a team, and on sizes in what is an LLM context window.

Where Bombos fits

Bombos keeps team knowledge out of long chat histories: it is a plugin that gives every AI tool your team uses one shared memory, and it retrieves only what fits the question at hand, so the window stays small. Learned rules and the knowledge already in mail, documents and repos sit there in one current version, and you can see what each answer was based on. One person can try it first for their own agents.

Related questions

Does context rot happen with 1M-token models too?

Yes. A larger window lets you fit more, but Anthropic's documentation says accuracy and recall still degrade as token count grows. The limit you hit first is usually quality, not size.

Is context rot the same as hallucination?

No, but they are related. Chroma found that with distracting content in the input, GPT models more often gave confident wrong answers, while Claude models more often declined when unsure.

Why does context rot happen?

There is no settled explanation yet. The measurable part is the effect: longer and noisier input means less reliable output, so the practical response is curation.

How do I know a session has rotted?

Typical signs are the AI forgetting an instruction from early on, mixing up two files or versions, or repeating a mistake you already corrected. That is the moment to compact or start a new session with a short summary.

Sources

  1. Chroma Technical Report: Context Rot
  2. Claude API docs: Context windows
  3. Anthropic: Effective context engineering for AI agents
  4. Claude API docs: Compaction overview
The first half hour · free

What does your AI forget?

  1. 01You tell us which AI you use and where it keeps starting over.
  2. 02We show what a shared memory changes there, on your own examples.
  3. 03You keep the plan, with or without Bombos.
Piet Baudoin
Piet BaudoinYou talk to Piet, from the Bombos teamDutch team from Utrecht and Nijmegen · data in Europe
We reply the same working day.