RAG vs context engineering: is RAG still needed?

Short answer

RAG (retrieval-augmented generation) fetches passages from a knowledge source and puts them in the prompt. Context engineering is the wider job of deciding everything the model sees on each step: instructions, tools, examples, memory, history and retrieved documents. RAG is one technique inside it. What changed is how retrieval is used: agents increasingly search when they need something, instead of getting one batch of chunks before they start.

Different levels, not rivals

RAG comes from a 2020 paper by Patrick Lewis and eleven co-authors, presented at NeurIPS 2020: a language model paired with a retriever that searches a dense vector index of Wikipedia and supplies passages during generation. It is a specific technique for getting outside knowledge into a model.

Context engineering is a name for the whole job. Anthropic defines it as “the set of strategies for curating and maintaining the optimal set of tokens” during inference. Google’s November 2025 whitepaper lists what that set contains: system instructions, tool definitions, few-shot examples, long-term memory, external knowledge “often using” RAG, tool outputs, sub-agent outputs, conversation history and the user’s prompt. The 2025 survey of context engineering by Lingrui Mei and colleagues, based on over 1,400 papers, likewise treats RAG as one implementation next to memory systems, tool-integrated reasoning and multi-agent systems.

RAG Context engineering
Question it answers Which passages from a source fit this query? What should the model see at this step, and what should it not?
Scope Retrieved documents Instructions, tools, examples, memory, history, retrieval, output of other agents
Typical moment Once, before the model answers Every turn and every tool call
Typical failure Wrong or missing chunk Too much, stale or contradicting context; important context lost after compaction
Who owns it Search or data engineer Whoever builds the agent or the team’s AI setup

What changed in how retrieval is used

The classic RAG pipeline searches once and stuffs the top results into the prompt. Anthropic describes this as “embedding-based pre-inference time retrieval” and contrasts it with a just-in-time approach: the agent keeps lightweight identifiers such as file paths, stored queries and links, and loads the data with tools when it needs it.

Most real systems are a hybrid. Anthropic’s own example is Claude Code: CLAUDE.md files are loaded into context up front, while glob and grep let it find files just in time. Anthropic notes that loading more up front may suit work with less dynamic content, such as legal or finance work.

So the practical choice is not RAG or context engineering. It is:

  1. What must always be there? Rules, definitions and instructions that apply to every task. Load them up front, and keep them short.
  2. What should be found on demand? Documents, code, past tickets. Give the agent a search tool (vector, keyword or both) and good identifiers.
  3. What should be remembered? Facts and corrections from earlier work. That is memory, which needs extraction and updating, not just indexing.
  4. What should be removed? Old tool output, finished sub-tasks, outdated versions. Compaction and pruning are context engineering too.

The common misunderstanding

“We have RAG, so our AI has context” is the most common version. RAG answers only the second question above. A team can have an excellent document index and still get poor answers because the rules that matter are not loaded, because the search returns three versions of the same price list, or because a long session has buried the instruction from the start. The opposite mistake also happens: dropping retrieval because the window is now 1M tokens. Google’s whitepaper notes that cost and latency grow with context, and Anthropic warns that accuracy degrades as it fills. See What is context rot?

What this means for a team

For a team, the retrieval part is usually the easiest to buy: most AI tools can now search connected documents. The harder parts are the ones RAG does not cover. Which version of a document is current, which rules always apply, and which corrections a colleague already made. Those live in people’s heads and in individual chats, and each tool keeps its own. If memory is also on your list, Is RAG the same as AI memory? explains that difference, and What is a context graph? shows one way to connect the pieces.

Where Bombos fits

Bombos is a context graph with retrieval: it combines similarity, keywords, nearness in the graph and rules that only apply in a certain context, and serves that to every AI tool your team uses through one plugin. It covers the parts plain RAG leaves out, shared team knowledge and learned rules, next to what already sits in mail, documents and repos, and shows what each answer rests on. It runs in our cloud or on your own server.

Related questions

Did context engineering replace RAG?

No. Retrieval is still how an agent gets knowledge it was not trained on. The change is that retrieval is now one tool the agent calls during a task, combined with memory, compaction and tool design, rather than the whole architecture.

What is the difference between RAG and agentic RAG?

Classic RAG runs one search before the model answers. In agentic or just-in-time retrieval the model decides when and what to search, often several times, using tools such as search, grep or a database query.

Do I still need a vector database?

Often yes, for large text collections where keyword search misses synonyms. But Anthropic points out that Claude Code finds files with glob and grep instead of embeddings, so it depends on the data.

Is long context an alternative to RAG?

Only partly. A large window fits more, but accuracy and recall drop as it fills, which is called context rot. Retrieving the right slice usually beats pasting everything.

Sources

  1. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., 2020)
  2. Effective context engineering for AI agents (Anthropic, September 2025)
  3. Context Engineering: Sessions & Memory (Google whitepaper on Kaggle, November 2025)
  4. A Survey of Context Engineering for Large Language Models (Mei et al., 2025)
The first half hour · free

What does your AI forget?

  1. 01You tell us which AI you use and where it keeps starting over.
  2. 02We show what a shared memory changes there, on your own examples.
  3. 03You keep the plan, with or without Bombos.
Piet Baudoin
Piet BaudoinYou talk to Piet, from the Bombos teamDutch team from Utrecht and Nijmegen · data in Europe
We reply the same working day.