Is RAG the same as AI memory?

Short answer

No. RAG (retrieval-augmented generation) is a retrieval technique: it finds matching passages in a source, usually existing documents, and puts them into the prompt. Memory is keeping, updating and later recalling what the AI learns, such as preferences, decisions and corrections, and many memory systems use RAG to find those memories again, but RAG alone does not decide what to keep or which fact is out of date.

What each one does

Both address the same problem: the model only knows your business through what is in its context window, and that window is limited and temporary. But they sit at different levels. RAG is a way of retrieving. Memory is a system that decides what to keep, updates it and recalls it later, and the recalling is often done with RAG. The original RAG paper already calls its document index a “non-parametric memory”. In practice, people usually compare a RAG setup over documents with a memory of conversations; the table compares those two setups.

Aspect RAG over documents Memory
Source Documents that already exist: wiki, manuals, contracts, tickets Mostly what happens while the AI works: conversations, corrections, decisions
When it is written When documents are indexed During or after a task
Typical question “What does our refund policy say?” “How does this client want updates written?”
Handles change Re-index when the document changes Update or invalidate the old fact
Typical failure Finds an outdated or almost-relevant passage Remembers something nobody wrote down anywhere else, or keeps a stale fact

Anthropic’s engineering team describes two ways to get knowledge into the window. Up-front retrieval, the classic RAG setup, uses embeddings to find relevant passages before the model starts. Just-in-time retrieval lets the agent load data through tools while it works, for example by opening files by path or running a query, without everything having to be indexed first. What is often called agentic RAG resembles that second pattern.

Memory products add a step that a basic RAG setup does not have: deciding what is worth keeping. Mem0 extracts facts and entities from the messages you send it and returns only the relevant ones later. Zep keeps facts in a temporal graph: when new data contradicts an earlier fact, the earlier one is marked invalid with a timestamp instead of silently living on next to it.

The misunderstanding: “we have RAG, so the AI remembers”

A team indexes its documents, connects them to a chatbot, and expects the AI to know how the team works. Then someone corrects the AI (“we never quote prices in the first email”) and the next day it makes the same mistake. Nothing went wrong with the retrieval. The correction lived in a conversation, and in this setup conversations are not in the index. You could index them, but then you face the memory question anyway: which sentence from which conversation still holds?

The reverse also happens. Personal chat memory knows a lot about how one person likes to work, but it cannot answer a question about the contract in SharePoint if that contract was never made available to it.

How to decide

  1. Mostly stable, written knowledge (policies, product documentation, manuals): RAG over those documents.
  2. Knowledge that only exists in people’s heads and conversations (preferences, exceptions, lessons from mistakes): memory.
  3. Both, which is the normal case for a team: one layer that reads existing documents and also stores what is learned, with a clear rule for which source wins when they disagree.

An example of such a rule. The price list (a document) says onboarding costs $500. Last week the team decided to waive it for existing clients, and someone told the AI (memory). A good rule: a more recent, explicit correction from someone authorized wins over an older document, and the AI mentions both and asks for the document to be updated. Done right, the correction is stored once, updated when the decision changes again, and comes back through retrieval whenever someone asks about onboarding costs. Without a rule, the model picks whatever happens to score highest in retrieval.

The third point is the hard part. If memory lives in each person’s ChatGPT or Claude account and the documents live in a separate RAG index, nobody knows which version is current, and nobody else benefits from one person’s corrections. For the different memory types, see What are the types of memory in AI agents?. For the team side, see How can a team share one AI memory?.

What this means for a team

Often someone builds a RAG index for one tool, while each assistant also keeps its own memory. The result is several partial memories that do not know about each other. A colleague using Cursor does not get the rule your Claude learned, and the RAG index behind your support bot does not know about either.

Where Bombos fits

Bombos does both in one shared layer. It reads what already exists (mail, documents, software packages, repositories) and stores what the team learns, including rules that the AI has ready for the next question. Retrieval combines similarity, keywords, closeness in a context graph and rules that only apply in a certain context, and each answer shows the sources it used. The same layer is available to every AI tool the team uses, in our cloud or on your own server.

Related questions

What is agentic RAG?

Retrieval where the agent itself decides when and what to search, using tools during the task, instead of one search before the prompt. Anthropic describes a related pattern, just-in-time retrieval, and names Claude Code as a hybrid: CLAUDE.md is loaded up front, files are searched on demand with glob and grep.

Can I use a vector database as memory?

Yes, many memory products store memories in a searchable index. The difference is what goes in: an extraction step that turns conversations into facts, and a rule for what happens when a new fact contradicts an old one.

Is context engineering the same as RAG?

No. Context engineering is the broader job of choosing what goes into the context window: instructions, tools, examples, retrieved documents and memory. RAG is one technique inside it.

Why not just paste everything into the 1M token window?

Because accuracy and recall drop as the window fills, which Anthropic calls context rot, and because the window is empty again in the next session.

Sources

  1. Effective context engineering for AI agents (Anthropic, September 2025)
  2. Context windows (Claude API docs)
  3. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., 2020)
  4. Mem0 overview
  5. Concepts (Zep docs)
The first half hour · free

What does your AI forget?

  1. 01You tell us which AI you use and where it keeps starting over.
  2. 02We show what a shared memory changes there, on your own examples.
  3. 03You keep the plan, with or without Bombos.
Piet Baudoin
Piet BaudoinYou talk to Piet, from the Bombos teamDutch team from Utrecht and Nijmegen · data in Europe
We reply the same working day.