
Context Rot: What It Is, Why It Happens and How to Fix It
Why AI Gets Worse the Longer You Chat, and What Actually Helps
Disclosure: Membase is our product. Most fixes for context rot below need no product at all, and we cover them first. Research findings were checked against Chroma’s technical report, Anthropic’s engineering guide and the papers linked at the end in November 2026.
Quick answer
Context rot is the drop in an AI model’s accuracy as the amount of text in its context window grows. The model still has room for more, but it gets worse at finding and using what’s there.
- It’s measurable. Chroma tested 18 models in 2025, including GPT-4.1, Claude 4 and Gemini 2.5, and found that performance grows less reliable as input length grows, even on simple tasks.
- It’s not the same as running out of space. A long chat can rot well before it hits the context limit.
- Irrelevant text makes it worse. The more off-topic or look-alike material sits in context, the harder it is for the model to pick out what matters.
How to fix it: keep the context short and on-topic. Start a fresh chat or session for each task, summarize before you continue, and keep long-term facts in memory outside the chat so only the relevant parts come back in.
In this guide: What is context rot · What the research found · Signs of context rot · Why it happens · How to fix it in chat apps · How to fix it in coding agents · How to prevent it in your own agents · Where memory fits · FAQ
What is context rot?
Every chat with an AI model runs inside a context window: the text the model can see at once, including your messages, its replies, files, tool results and instructions. Context windows now reach hundreds of thousands or millions of tokens.
A bigger window doesn’t mean every token gets equal attention. Context rot is the name for what happens as that window fills: the model becomes less accurate at recalling and reasoning over what’s in it.
The term was popularized by Chroma’s July 2025 report, Context Rot: How Increasing Input Tokens Impacts LLM Performance. Anthropic describes it the same way in its guide to context engineering: as the number of tokens in the context window grows, the model’s ability to recall information from it goes down, and while some models degrade more gently than others, it shows up in all of them.
| Context rot | Context window limit | |
|---|---|---|
| What happens | Answers get less accurate as context grows | The model can’t take any more text |
| When | Gradually, well before the limit | All at once, at the limit |
| What you notice | Forgotten instructions, mixed-up details, confident wrong answers | An error, a cut-off, or a notice such as ChatGPT’s “Chat memory full” |
| Fix | Less, more relevant context | A new chat or compaction |
What the research found
Chroma (2025): 18 models, simple tasks. The team extended the classic needle-in-a-haystack test, where a model must find one fact in a long document, to make it closer to real use. Across GPT-4.1, Claude 4, Gemini 2.5, Qwen3 and others, performance became less uniform as input grew. Key findings:
- Look-alike questions suffer most. The less the question’s wording matches the answer’s wording, the faster accuracy drops as input grows.
- Distractors hurt. Even one passage that looks relevant but isn’t lowers accuracy, and four distractors lower it further. Models also differed in how they failed: in Chroma’s tests, Claude models hallucinated least, and GPT models most often gave confident wrong answers when distractors were present.
- Structure matters, in a surprising way. Models did worse when the surrounding text flowed logically than when it was shuffled.
- Real conversations show it too. On LongMemEval, a benchmark of long chat histories, models did well when given only the relevant parts of a conversation, and consistently worse when given the full history of about 113,000 tokens.
Lost in the middle (2023). An earlier Stanford study found that models use information best when it’s at the start or end of the context, and worst when it’s in the middle.
Long-horizon search (2026). A 2026 study of deep-search agents found a newer failure: with lots of context, models give up or return uncertain wrong answers long before they run out of context window.
The common thread: what’s in the context, and how much of it, matters as much as whether the right fact is there at all.
Signs of context rot
You’re likely seeing context rot when, deep into a long chat or agent session:
- The AI forgets instructions you gave at the start, such as a format, a tone or a rule.
- It mixes up details: names, numbers, file paths or decisions from different parts of the conversation.
- It repeats suggestions you already rejected, or undoes fixes it made earlier.
- It answers confidently but wrongly about something you discussed earlier.
- Answers get slower, vaguer or more generic.
- A coding agent edits the wrong file or loses track of the plan.
If the same question gets a sharp answer in a fresh chat, the long context was the problem.
Why context rot happens
- Attention is a budget. Anthropic compares a model’s attention to a limited budget: every token relates to every other token, which is n² relationships for n tokens. The more tokens, the thinner that attention is spread.
- Long context turns one task into two. With only relevant text, the model just has to reason. With a long, mixed history, it must first find the relevant parts, then reason, in one pass. Chroma’s LongMemEval results show the cost of that extra step.
- Similar-but-wrong text competes. Old drafts, rejected ideas and earlier versions of a file all look relevant. The model can pick the wrong one.
- Position matters. Facts in the middle of a long context get used least.
How to fix context rot in ChatGPT, Claude and Gemini
- One chat per task. Don’t keep a single endless chat for everything. Start a new one when the topic changes.
- Summarize, then restart. Ask for a short summary of decisions and open questions, then paste it into a new chat. ChatGPT offers this automatically when a chat shows “Chat memory full”; see ChatGPT Memory: “Memory Full”, Limits and How to Manage It.
- Put key instructions where they stick. Use custom instructions, a project’s instructions, or your assistant’s memory for rules that should always apply, instead of saying them once at the top of a long chat.
- Paste less. Share the relevant section of a document, not the whole thing.
- Restate what matters. Before a key question in a long chat, repeat the facts the answer depends on.
- Edit instead of piling on. If an answer goes wrong, edit your earlier message rather than adding correction after correction.
How to fix context rot in Claude Code, Codex and Cursor
Coding agents fill context fast: every file read, command output and failed attempt stays in the window.
- Clear between tasks. In Claude Code, run /clear when you switch to something unrelated. In Codex or Cursor, start a new session or chat.
- Compact long sessions. In Claude Code, /compact replaces the history with a summary. Add what to keep, for example /compact keep the API decisions and the open bugs.
- Keep instruction files short. CLAUDE.md and AGENTS.md load into every session. Claude Code suggests staying under 200 lines per CLAUDE.md, and Codex stops loading AGENTS.md files at 32 KiB by default. Move long procedures into skills, which load only when needed; see AGENTS.md vs CLAUDE.md vs SKILL.md.
- Use subagents for exploration. Let a subagent search the codebase or read logs in its own context, and return only the result.
- Write decisions down. Before clearing, ask the agent to save important decisions to a notes file or to memory, so the next session starts with them.
More on each agent’s memory: Claude Code · Codex · OpenClaw.
How to prevent context rot in your own agents
If you build agents, the goal is what Anthropic calls finding the smallest possible set of high-signal tokens for each step. Its guide describes three main techniques for long-running work:
| Technique | What it does | Trade-off |
|---|---|---|
| Compaction | Summarizes the conversation as it nears the limit and starts a fresh window with the summary | Details the summary drops are gone |
| Structured note-taking (agentic memory) | The agent writes notes outside the context window and pulls them back in later | Notes need to be searchable and kept up to date |
| Sub-agents | Focused tasks run in their own clean context; only a short result comes back | More moving parts to coordinate |
Also:
- Retrieve just in time. Keep references (file paths, IDs, links) in context and load the content only when a step needs it.
- Trim tool output. Return the few lines that matter, not full logs or full API responses.
- Put the most important material first or last.
- Test at realistic lengths. Evaluate your agent with long, messy inputs, not only short clean ones.
Where memory fits
Context rot is the reason “just give the model the whole history” doesn’t work as memory. The research points the other way: give the model less, more relevant context.
That’s the job of a memory layer. Instead of carrying every past conversation in the window, the agent stores what it learns outside the window and recalls only what the current question needs. Each new chat or session starts short and focused, but still knows the decisions that matter.
Membase is a memory layer for AI agents that works this way:
- It saves your conversations and documents into Memories outside any single chat, tool or machine.
- It brings back only the relevant parts, with the source each one came from.
- The same memory works across ChatGPT, Claude, Gemini, Claude Code, Codex, Cursor and your own agents.
- On LongMemEval_S, the same benchmark Chroma used to show context rot, Membase’s engine scores 92.6% while using up to 11× fewer tokens than the full history (our own numbers; method).
To start, add the Membase Chrome extension for ChatGPT, Claude, Gemini and other chat apps, or connect a coding agent with one command:
npx add-mcp 'https://api.app.membase.com/mcp-http'
Membase is in beta and free during the beta. Sign up at app.membase.com.
Quick links
- ChatGPT Memory: “Memory Full”, Limits and How to Manage It: what to do when a chat fills up
- Claude Code Memory: How It Works and How to Add Persistent Memory: CLAUDE.md, auto memory and /compact
- AGENTS.md vs CLAUDE.md vs SKILL.md: keep instruction files short
- Best MCP Memory Servers in 2026: memory for coding agents
- Membase research: LoCoMo and LongMemEval results and method
Sources
- Chroma: Context Rot: How Increasing Input Tokens Impacts LLM Performance (July 2025)
- Anthropic: Effective context engineering for AI agents
- Lost in the Middle: How Language Models Use Long Contexts (2023)
- LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory
- Diagnosing and Mitigating Context Rot in Long-horizon Search (2026)