Short answer
A second brain is the notes a team writes and keeps. RAG is one way to feed notes to a model: chunk them, embed the chunks, retrieve the nearest few per question. For a team vault, an assistant that searches, reads whole notes and follows links is usually enough and easier to check. RAG pays off once the corpus outgrows that.
A second brain and RAG are not two products that do the same job. A second brain is what a team writes down and keeps; RAG (retrieval-augmented generation) is one technique for feeding pieces of a body of text to a model at question time. So "second brain vs RAG" is really asking something narrower: does your notes system need a vector index before an AI assistant can use it?
For most teams the answer is no. A vault of a few thousand linked notes is small enough for an assistant to search, open whole notes and follow the links between them, the same way a person would. This is how Baalda, the open source team second brain this site is for, gives assistants access to a team's notes: plain markdown files, an index of every [[wikilink]], and a keyword search, all reachable over MCP. RAG starts paying for itself when the corpus is much larger than that, or when the questions depend on meaning that the words on the page do not carry.
What is the difference between a second brain and RAG?
They sit at different layers, which is why comparing them head to head confuses people.
| Second brain | RAG | |
|---|---|---|
| What it is | The notes themselves, and how they are organised | A pipeline that fetches text for a model |
| Who it is for | People first, assistants second | A model |
| The unit | A whole note, with links to other notes | A chunk, usually a few hundred tokens |
| How things are found | Search, links, folders, titles | Nearest neighbours in a vector index |
| What you can check | Every note is a file you can open | The retriever's choice per question |
The term RAG comes from a 2020 paper by Patrick Lewis and colleagues, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. It describes models that "combine pre-trained parametric and non-parametric memory", where the non-parametric memory "is a dense vector index of Wikipedia, accessed with a pre-trained neural retriever". Read that again with a team vault in mind. The design was built for millions of encyclopaedia passages that nobody on your team wrote and nobody links together by hand.
A second brain is the opposite kind of corpus: small, written by the people asking the questions, and already full of structure. The method for building one is covered in how to build a second brain. The point here is only that a team's notes arrive with titles, folders and links that a Wikipedia dump has to have inferred for it.
What does RAG actually do to your notes?
It cuts them up, turns each piece into a vector, and at question time hands the model whichever pieces sit nearest to the question. Three things follow from that.
A chunk loses the context the note gave it. Anthropic's own Contextual Retrieval write-up (19 September 2024) uses this example: a chunk that reads "The company's revenue grew by 3% over the previous quarter" does not say which company or which quarter, "making it difficult to retrieve the right information or use the information effectively." In a team note, the missing part is usually the heading two screens up or the decision the paragraph was arguing against.
Getting retrieval good is real engineering. The same post reports that adding contextual embeddings and contextual BM25 cut its failed retrievals by 49%, from 5.7% to 2.9%, and by 67% to 1.9% with reranking on top. Those are good results, and they come from a chunking step, an embedding model, a keyword index, a reranker and a model call per chunk to write its context. That is a system somebody has to run and keep in step with the notes.
The index is a copy, and copies go stale. Every edit to a note means the vectors for that note are out of date until something re-embeds it. If nobody wires that up, the assistant answers from last month's version of a page your colleague fixed this morning.
None of this makes RAG wrong. It makes it a cost, and the question is whether your notes are big enough to need what it buys.
How does an assistant use a second brain without a vector index?
By navigating it. Instead of receiving five chunks someone else chose, the assistant searches, opens what it found, and follows the links the authors left.
Anthropic describes exactly this pattern for Claude Code in Effective context engineering for AI agents (29 September 2025): CLAUDE.md files go into context up front, "while primitives like glob and grep allow it to navigate its environment and retrieve files just-in-time, effectively bypassing the issues of stale indexing and complex syntax trees." The same post notes that "folder hierarchies, naming conventions, and timestamps all provide important signals that help both humans and agents understand how and when to utilize information." A well-kept vault is made of those signals.
Baalda builds a second brain around that idea. The parts that matter for an assistant:
- Notes are plain markdown files on your disk. The desktop app keeps a local SQLite index over them, with an FTS5 full-text table and a
linkstable that records every[[wikilink]]by note id, so a link survives a rename. The README is explicit that this index is "entirely derived", deletable and rebuilt from the files. - The server keeps the same structure for synced vaults. Every time a note is saved, the indexer parses its
[[wikilinks]]into anote_linkstable and writes the note's text intonote_index, debounced by two seconds. The source code calls both tables "a rebuildable cache derived from the canonical Yjs state". Delete them and nothing is lost but time. - Search returns whole notes, not chunks. The MCP tool
search_notesreturns up to 10 hits by default and 50 at most, each one a note (or the text of an attached file) with its id, title, path and score. It only ever scores notes the caller is allowed to read. - Reading is the whole file.
read_notereturns the note's full markdown and arevision. Pass that revision back with an edit and the write is refused if somebody changed the note in between.
A session looks like a person working through a vault, which is the point. The assistant calls search_notes, reads the best match in full with read_note, and when that note links somewhere, searches for the linked title and reads that note too. If a person wrote "superseded by" above a link to the newer note, the assistant follows it and ends up on the note that replaced the one it found. A vector index would most likely score both notes as close matches and hand over chunks of each, with nothing to say which one still holds. Links carry exactly the judgement that retrieval has to guess at.
The same vault is reachable from Claude, ChatGPT, Cursor and other MCP clients, through one endpoint at POST /api/mcp on a server you can host yourself. How that compares with other ways of wiring notes to an assistant is in an MCP second brain is only as good as where its notes live.
When does a second brain actually need RAG?
There are three sizes to think about, not two.
Small enough to paste in. Anthropic's guidance in the Contextual Retrieval post is blunt: "If your knowledge base is smaller than 200,000 tokens (about 500 pages of material), you can just include the entire knowledge base in the prompt that you give the model, with no need for RAG or similar methods." That holds for a project folder or a small team's handbook. It has a catch, which is that long contexts degrade as they fill, covered in does the context window reset.
Too big to paste, small enough to navigate. This is where most team vaults sit: hundreds or a few thousand notes, written by the people using them, with titles and links that mean something. Search plus links works here, and it has an advantage RAG does not: every step is a note a person can open and check.
Too big to navigate. RAG earns its cost when the corpus is far larger than anyone could walk through, or when it has little structure for an assistant to follow. Years of support tickets, a document archive inherited from an acquisition, a mailbox export, research papers by the thousand. It also earns it when the questions are about meaning rather than wording, because a keyword search will miss a note that says "release" when you asked about "deploy".
If you are in the third case, your notes can still be the source. A Baalda vault is a folder of markdown, so any RAG pipeline can index it. What you take on is keeping that index in step with the files, which is the work discussed in why the search index has to be yours to rebuild.
Where does this approach fall short?
Baalda's search is honest about being simple, and you should know where it stops before you rely on it.
- It matches words, not meaning. The MCP tool describes itself as "semantic + keyword search", but the embedder the server uses by default is a 256-dimension hashed bag-of-words. It is offline and needs no API key, and it does not know that two different words mean the same thing. Vector and hybrid search are listed as planned work in the project's status document, not as something that exists today.
- There is no backlinks tool over MCP yet. The app has a backlinks panel and a graph view built on the links tables, but an assistant follows links by reading a note and searching for the linked title. It can go forward along a link easily; finding everything that points at a note takes more calls.
- Every query scores every note you can read. The server walks the readable notes in batches of 500 and keeps only an id and a score per note, which keeps memory flat, but the work still grows with the size of the vault. At team scale that is fine. At millions of documents it is the wrong design, and that is where a real vector index belongs.
- Navigation costs tool calls. An assistant that searches, reads and follows three links has made six requests where a RAG system made one. You trade some latency for being able to see, and correct, every step.
If what you have is a large, unlinked archive and you mostly ask fuzzy questions of it, a dedicated retrieval product will serve you better than any notes app, Baalda included.
How should a team decide?
Ask three questions in order.
- Would the whole thing fit in about 500 pages? Then neither RAG nor navigation is the bottleneck. Keep the notes clean and let the assistant read them.
- Is it written and linked by the people who use it? Then give the assistant search and the links, keep the notes as files you own, and skip the vector database until the vault proves it needs one.
- Is it huge, unstructured, or mostly asked about in other words than it uses? Then build retrieval on top of the files, and treat the index as a cache you can throw away, never as the place the knowledge lives.
In every case the second brain comes first. RAG is a way of reading it, and the notes should outlive whichever way of reading them you pick this year.