Short answer
Context rot is the drop in a language model's accuracy and recall as the text in its context grows, long before the window is full. Chroma's 2025 study of 18 models found it on simple tasks and found that close-but-wrong passages hurt most. Reduce it by sending less, and by keeping stale copies out of what the assistant retrieves.
Context rot is the drop in a language model's accuracy as the amount of text in its context grows. It happens well before the window is full, on simple tasks, in every model that has been tested for it. Most of what is written about it treats it as a problem with long chats, and the advice stops at "start a new session". The part a team controls is somewhere else: the notes your assistant retrieves can feed the rot directly, because the text that hurts a model most is text that looks like the answer and is not. In a team's knowledge, that is usually last quarter's version of the right page. Baalda is built so that the old version never comes back in a search, and the rest of this post explains why that matters and where it stops mattering.
What is context rot?
Anthropic's documentation gives the short definition: "As token count grows, accuracy and recall degrade, a phenomenon known as context rot." Its engineering post on context engineering (29 September 2025) puts the same thing in terms of a budget: models have an "attention budget" they draw on when reading long input, and "Every new token introduced depletes this budget by some amount, increasing the need to carefully curate the tokens available to the LLM."
The point both sentences make is that a context window is not a bucket you fill until it is full. Capacity and usefulness are different numbers. As of 23 October 2026, Anthropic lists a 1M-token context window for its current models, including Opus 5.5, Sonnet 5.5 and Haiku 5.5, and the same page says plainly that "more context isn't automatically better".
It is also not the model forgetting something it once knew. The window is rebuilt from the transcript on every request (why nothing in it persists), so context rot is a property of one request: the more you send, the less reliably the model uses any one part of it.
What did Chroma's context rot research find?
The study most pages cite is Chroma's technical report "Context Rot: How Increasing Input Tokens Impacts LLM Performance", published 14 July 2025 by Kelly Hong, Anton Troynikov and Jeff Huber. It tested 18 models, "including the state-of-the-art GPT-4.1, Claude 4, Gemini 2.5, and Qwen3 models", and its headline result is that "models do not use their context uniformly; instead, their performance grows increasingly unreliable as input length grows."
Four of its findings matter for anyone deciding what to put in front of a model:
| Experiment | What Chroma found |
|---|---|
| How closely the question matches the answer | "as needle-question similarity decreases, model performance degrades more significantly with increasing input length" |
| Distractors (passages on the topic that do not answer the question) | "Even a single distractor reduces performance relative to the baseline", four compound it, and the effect "amplifies as input length grows" |
| Structure of the surrounding text | "structural coherence consistently hurts model performance": shuffled filler scored better than coherent essays across all 18 models |
| Focused vs full prompt (LongMemEval) | Focused prompts averaging about 300 tokens beat full prompts averaging about 113k tokens, "Across all models" |
The last row is the one to remember. The same questions, with the same answer present in both prompts, got better answers when everything except the relevant few hundred tokens was taken away. The gap persisted with reasoning modes switched on.
The distractor row is the one a team should read twice. Some distractors hurt more than others, and the wrong answers models gave were often lifted from a distractor. A passage that is about the right thing and says the wrong thing is worse than a passage about nothing.
Can your own notes cause context rot?
Yes, and this is the half the glossary pages leave out. Chroma's distractors were written by hand for an experiment. A team produces them for free.
Think about what a retrieval step returns when an assistant asks your notes "what is our refund window?". If the knowledge lives in a shared drive or a synced folder, it is likely to find refund-policy.md, refund-policy-v2.md, the copy someone pasted into the onboarding doc in 2025, and the meeting note where the change was discussed. Every one of those is on topic. Only one of them is current. To the model they arrive as peers, with no signal about which superseded which, and Chroma's result says that a close-but-wrong passage is the most damaging thing you can add.
Chroma did not test stale versions of a document specifically, so treat this as our reading of their result rather than their claim. But it is the obvious case of it: an outdated copy of the right page is a distractor by construction.
Replicated folders make extra copies on their own. Dropbox keeps both sides of a conflict and renames one a "conflicted copy". Syncthing renames the older one <filename>.sync-conflict-<date>-<time>-<modifiedBy>.<ext>. Each of those is a real file on disk, and a filesystem-based assistant indexes it like any other note (how each sync tool handles a conflict).
How do you reduce context rot?
The model-side advice is sound and well documented. Anthropic's context engineering post states the principle as "finding the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome", and lists the working techniques:
- Load just in time. Agents "maintain lightweight identifiers (file paths, stored queries, web links, etc.) and use these references to dynamically load data into context at runtime using tools", rather than loading everything up front.
- Compact. Summarise a conversation near the limit and restart with the summary, accepting that "overly aggressive compaction can result in the loss of subtle but critical context".
- Take notes outside the window. "Structured note-taking, or agentic memory, is a technique where the agent regularly writes notes persisted to memory outside of the context window."
- Split the work. Sub-agents explore with clean windows and return "a condensed, distilled summary of its work (often 1,000-2,000 tokens)".
- Start fresh. A new session drops everything the client was replaying.
Every one of those manages the window. None of them helps if the thing you load just in time is the wrong version. That is a property of the store, and it is the store a team actually owns.
What should the store do so retrieval does not feed the rot?
Three things, and they are how Baalda's notes and its MCP server are built.
One searchable row per note, always the current text. Baalda keeps a search index with one row per note. Every edit rewrites that row in place:
INSERT INTO note_index (doc_id, vault_id, title, content, vector, updated_at)
VALUES ($1, $2, $3, $4, $5::jsonb, now())
ON CONFLICT (doc_id) DO UPDATE
SET vault_id = EXCLUDED.vault_id, title = EXCLUDED.title, content = EXCLUDED.content,
vector = EXCLUDED.vector, updated_at = now()When two people edit the same note, their changes merge in real time over CRDTs into that one document. There is no second copy to reconcile and nothing to rename.
History kept, but never searched. Earlier states are not thrown away. Baalda captures a version once a note has been quiet for ten minutes, keeps up to 50 per note, and also keeps the text from just before an edit that removed most of a note. Those versions live in a separate table that search never reads. So "what did this page say in June" has an answer a person can open and restore, and the June text cannot turn up as a search hit beside the current one. Deleted notes are excluded the same way: they are soft-deleted with their history, and search only scores notes that have not been deleted.
Search returns pointers, not text. When an assistant calls search_notes over MCP, each hit is a docId, title, path, score and kind. No note bodies. The model reads the list, picks the note it needs and calls read_note for that one. That is the just-in-time pattern Anthropic describes, enforced by the tool shape rather than by hoping the client trims its results. File text is bounded the same way: read_attachment_text returns 20,000 characters by default, with a truncated flag if there was more.
Search also only scores notes the person behind the token is allowed to read, so a private folder is never in the candidate set at all. That was built for permissions, but it has the side effect of keeping someone else's drafts out of your assistant's results (connecting an assistant over MCP).
Delete Baalda from that picture and you are back to the five techniques above, applied to a folder that may contain three versions of every important page. To be fair to the hosted tools: Notion and Confluence also edit a page in place, so they avoid the sync-copy problem too. What they do not give you is the page as a markdown file on your own disk, which is the other half of what a team is choosing.
Where does this not help?
The default search is lexical. Baalda's built-in embedder is a 256-dimension hashed bag-of-words: it matches words, not meaning, and it will not connect "refund window" to a note that only says "returns period". That is exactly the low question-to-answer similarity case Chroma found hardest, moved one step earlier into retrieval. The embedder is a swappable function and the index can be rebuilt against a better one (what that involves), but out of the box it is a keyword search with a vector shape.
It cannot stop people writing a second page. If someone creates refund-policy-new.md instead of editing the old note, Baalda indexes both, because they are two notes. The system removes the copies that tools make. It cannot remove the ones people make.
There is one case where Baalda makes a copy. If two people independently create a new note at the same path before either has synced, the desktop app keeps both and renames one <name> (conflict YYYY-MM-DD).md. That file is a real note and is searchable until someone merges it.
It does nothing inside a session. A long conversation still rots, and a 4,000-line note read in full is still a long input. Keeping notes short and splitting them by topic is still your job. Baalda keeps the wrong version out of the results. It does not make the right version concise.
A single person with a tidy folder may not need any of this. If you are the only writer and you edit files in place, your folder has no stale copies to retrieve. The problem grows with the number of people writing and the number of tools replicating, which is why it shows up on teams first.