What the research shows
Anthropic’s documentation describes it in one line: “As token count grows, accuracy and recall degrade, a phenomenon known as context rot.” A well-known measurement is Chroma’s technical report “Context Rot” from July 2025, by Kelly Hong, Anton Troynikov and Jeff Huber. They tested 18 models, including GPT-4.1, Claude 4, Gemini 2.5 and Qwen3. Their main findings:
| Finding | What it means in practice |
|---|---|
| Performance gets less reliable as input gets longer, even on simple tasks | A model that finds a fact perfectly in a short document can miss it in a long one |
| A single distracting passage already lowers performance, and four lower it more | Near-miss content (an old version, a similar customer) is worse than unrelated text |
| On LongMemEval, focused prompts with only the relevant part did much better than the full input of around 113k tokens | Giving the model the whole conversation history is worse than giving it the right slice |
| Claude models more often declined when unsure; GPT models more often answered confidently and wrongly when distractors were present | Rot shows up differently per model, sometimes as refusals, sometimes as errors |
Note what the study does not claim: it does not give a final explanation of why this happens. The effect is measured; the mechanism is still open.
The common misunderstanding
Most people assume the context window is the limit that matters: if it fits, the model can use it. With current Claude models offering a 1M-token window, it is tempting to load the whole repo, all contracts or the full mail archive. Context rot says the opposite. Fitting is not the same as being used well. Anthropic’s own documentation frames the window as working memory and stresses that more context is not automatically better, so curation counts.
How to avoid it
- Retrieve, don’t paste. Keep references in context (file paths, links, queries) and let the AI load what the current step needs. Anthropic calls this just-in-time retrieval.
- Clear old tool output. Search results and file dumps that are no longer needed only distract, though unresolved errors and references you may need again are worth keeping. Claude’s API supports clearing old tool results and thinking blocks through context editing.
- Compact long sessions. Replace older turns with a summary and keep recent turns word for word. The Claude API offers compaction on demand and at a token threshold, both in beta. See what is context compaction.
- Use sub-agents for side tasks. A sub-agent searches or reads in its own clean window and returns a short summary, so the main session stays focused.
- Start fresh when the task changes. A new task in an old session drags along everything from the old one. A short handover note beats 200k tokens of history.
- Remove near-duplicates. Old versions of a document, outdated rules and contradicting instructions add plausible but misleading context; remove them when they are not needed.
What it means for a team
Context rot has a team version. When knowledge lives in long personal chat histories, the only way to “give the AI context” is to keep that history going, which is exactly what rots. And when ten people each paste their own version of the pricing sheet or the style guide into their sessions, the AI meets the kind of similar-but-wrong passages that Chroma used as distractors.
The fix is the same as for one session, but organized: one current version of each piece of team knowledge, stored outside any chat, and retrieved per question. Then a fresh session needs little re-briefing, because nothing important lived only in the old one, although retrieval can still miss something. More on setting that up in how to set up context engineering for a team, and on sizes in what is an LLM context window.
Where Bombos fits
Bombos keeps team knowledge out of long chat histories: it is a plugin that gives every AI tool your team uses one shared memory, and it retrieves only what fits the question at hand, so the window stays small. Learned rules and the knowledge already in mail, documents and repos sit there in one current version, and you can see what each answer was based on. One person can try it first for their own agents.
Related questions
Does context rot happen with 1M-token models too?
Yes. A larger window lets you fit more, but Anthropic's documentation says accuracy and recall still degrade as token count grows. The limit you hit first is usually quality, not size.
Is context rot the same as hallucination?
No, but they are related. Chroma found that with distracting content in the input, GPT models more often gave confident wrong answers, while Claude models more often declined when unsure.
Why does context rot happen?
There is no settled explanation yet. The measurable part is the effect: longer and noisier input means less reliable output, so the practical response is curation.
How do I know a session has rotted?
Typical signs are the AI forgetting an instruction from early on, mixing up two files or versions, or repeating a mistake you already corrected. That is the moment to compact or start a new session with a short summary.
