Memory and context

What is AI agent memory, and how does it work?

AI agent memory explained: short-term vs long-term, episodic, semantic and procedural memory, how retrieval works, and why per-agent memory fails a team.

AI agent memory is everything an agent can use beyond the current prompt: the conversation so far (short-term memory) and what it stored in earlier sessions (long-term memory). Long-term memory is usually split into three kinds: semantic (facts), episodic (past events and examples) and procedural (rules and how to do a task). The model itself does not remember anything between calls. Memory works because software writes notes to storage and retrieves the relevant ones back into the context window at the right moment. For a team, the open question is whose memory it is: most agent memory belongs to one user or one agent.

By Bombos. Published November 17, 2026. Sources checked November 17, 2026.

The short answer

  • The model is stateless. Memory is storage outside the model plus a step that puts the right pieces back into the context window. Anthropic calls this “structured note-taking, or agentic memory”: the agent “regularly writes notes persisted to memory outside of the context window” (Anthropic, Effective context engineering, September 29, 2025).
  • Short-term memory is the current session. LangChain describes it as memory that “tracks the ongoing conversation by maintaining message history within a session” (LangChain docs, memory).
  • Long-term memory survives sessions: it “stores user-specific or application-level data across sessions” (same source).
  • Three kinds of long-term memory, from the CoALA paper: “episodic, semantic, and procedural” (Sumers et al., Cognitive Architectures for Language Agents).
  • Retrieval decides quality. More memory in the prompt is not better; Anthropic warns of “context rot” as the context grows.
  • Scope decides who benefits. Memory is stored per user, per agent or per namespace. A team only shares memory if someone designs it that way.

Why does an agent need memory at all?

A language model answers from two things: what it learned in training and what is in its context window right now. Nothing carries over from one call to the next. A chat feels continuous because the application sends the earlier messages along again.

That breaks down in two ways. First, sessions end. A new chat, a new terminal or a new day starts empty. Second, the context window is finite, and filling it has a cost. Anthropic describes an “attention budget” and says context “must be treated as a finite resource with diminishing marginal returns” (Anthropic). So an agent that is supposed to improve over weeks needs a place to keep what it learned, and a way to pull back only what matters for the current task.

What is the difference between short-term and long-term memory?

Aspect Short-term memory Long-term memory
What it holds The current conversation, tool results, the plan in progress Facts, past events, rules and preferences
How long it lasts One session or thread Across sessions
Where it lives The context window, sometimes a checkpoint A database, files or a knowledge graph
Main risk Overflow: the window fills up Wrong or stale memories coming back

In the CoALA paper, short-term memory is called working memory: “These include short-term working memory and several long-term memories: episodic, semantic, and procedural” (arXiv HTML, section 4.1). LangGraph implements short-term memory as thread-scoped state that it persists through checkpoints, and long-term memory as a store with “custom namespaces” that can be recalled in any thread (LangChain docs).

What are episodic, semantic and procedural memory?

The three terms come from cognitive science. CoALA applied them to language agents in 2023, and most memory tools still use them.

  1. Semantic memory: facts. CoALA: “Semantic memory stores an agent’s knowledge about the world and itself.” Example: “Customer X has a pilot until March.” LangChain notes this is often used to personalise interactions.
  2. Episodic memory: what happened. CoALA: “Episodic memory stores experience from earlier decision cycles.” Example: the transcript of how the agent fixed a failing deploy last week. LangChain says that for agents it “often takes the form of few-shot examples” that show the agent how to complete a task.
  3. Procedural memory: how to do things. CoALA distinguishes “implicit knowledge stored in the LLM weights, and explicit knowledge written in the agent’s code.” In practice this includes the system prompt. LangChain observes that agents “more commonly modify their own prompts” than their weights or code.

For a business, procedural memory is the valuable one and the hardest to get right. “Never quote a delivery date without checking stock” is a rule, not a fact. If it only lives in one person’s chat, the next agent breaks it.

How does memory retrieval work?

Every memory system has a write side and a read side.

Writing. Something decides what is worth keeping. LangChain describes two options: writing “during runtime”, in the hot path, which gives real-time updates but “can impact agent latency”; or “as a separate background task”, which removes that latency but makes the timing of writes a design decision (LangChain docs). Some systems extract facts automatically after each conversation; others let the agent decide.

Storing. Common forms are plain files, rows in a database with vector embeddings, and knowledge graphs of entities and relations. Anthropic’s memory tool is an example of the file approach: Claude creates, reads, updates and deletes files under /memories, and the tool “operates client-side”, so “your application runs them” against storage you control (Claude docs, memory tool).

Reading. At the start of a task, or when the agent asks for it, the system finds the relevant memories and adds them to the context. Typical methods:

  • Similarity search on embeddings: find notes whose meaning is close to the question.
  • Keyword search: exact names, codes and IDs that embeddings can miss.
  • Graph traversal: follow relations from a customer to their contract to the person who signed it.
  • Just-in-time loading: the agent keeps “lightweight references” such as “file paths, stored queries, web links” and loads data with tools when needed (Anthropic).

Most production systems combine several of these and then filter by scope: which user, which project, which agent. Retrieval is where memory succeeds or fails. A store full of correct facts is useless if the wrong ones come back, and harmful if an outdated one comes back with confidence.

Where does agent memory go wrong?

Four failure modes come up again and again:

  1. Stale facts. A price changed, but the old memory is still retrieved. Good systems keep history and mark what is current.
  2. Noise. Too much gets saved, and retrieval returns trivia instead of the rule that mattered.
  3. No source. The agent acts on a memory, and nobody can see where it came from or who said it.
  4. Wrong scope. A memory is stored for one user or one agent, so the rest of the team never sees it.

Anthropic’s memory tool documentation lists related chores you own when you build it yourself: cap file sizes, “periodically delete memory files that haven’t been accessed in a long time”, and guard against path traversal (Claude docs).

Is memory per agent or per team?

Almost always per user or per agent. That is a sensible default for privacy, but it means learning does not spread.

  • Claude Code stores auto memory per repository on one machine: “Files are not shared across machines or cloud environments” (Claude Code docs, memory). We cover the details in Claude Code memory for teams.
  • Codex keeps memories under ~/.codex, and OpenAI advises to “Keep required team guidance in AGENTS.md or checked-in documentation” and to treat memories “as a helpful recall layer, not as the only source for rules that must always apply” (OpenAI, Codex memories).
  • Developer memory layers such as Mem0, Zep and Letta let you scope memory by user, agent or namespace. Sharing is possible, but you build it. See Mem0 alternatives for team memory.

Team memory needs things a personal memory does not: one store that several tools read, permissions on who sees what, a review path for rules, and provenance for every item. That is less a storage problem than a design problem, which is why it belongs under context engineering: deciding what each model sees, at each step, for the whole team.

How does Bombos do this?

Bombos is the memory layer under the AI a team already uses: Claude, Claude Code, ChatGPT, Copilot, Codex, Cursor or your own agents. It is a plugin, not a new chat or model.

  • One shared memory that learns how the team works.
  • It learns from what the team records and corrects. A correction does not stay with one person.
  • The source behind every answer. You see which mail, document or decision a memory rests on.
  • Your AI acts, Bombos remembers. Bombos does not take actions itself. Your tools keep doing the work.
  • Cloud on servers in Europe, or standalone on your own server.

Where Bombos is not the answer: if you are building a single agent and want to own every line of the memory code, a library such as Mem0, Letta or Graphiti, or Anthropic’s memory tool, fits better.

Frequently asked questions

Does an LLM have memory? Not between calls. Memory is added by the application: it stores information outside the model and puts relevant parts back into the context window.

What is the difference between RAG and agent memory? RAG retrieves from a fixed body of documents. Agent memory also writes: the agent adds and updates what it learned, so the store changes as it works. Retrieval methods are often the same.

Can a team share one agent memory? Only if it is designed for that: one store, read by every tool, with permissions and sources. Most built-in memory in AI tools is per user or per machine.

Book your half hour: together we map what your AI forgets. Free, and the plan is yours, with or without Bombos.

Which sources did we use?

All sources opened and read on November 17, 2026.

  1. Anthropic, Effective context engineering for AI agents (September 29, 2025): https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
  2. LangChain docs, Memory: https://docs.langchain.com/oss/python/concepts/memory
  3. Sumers, Yao, Narasimhan, Griffiths, Cognitive Architectures for Language Agents (CoALA): https://arxiv.org/abs/2309.02427
  4. Claude docs, Memory tool: https://platform.claude.com/docs/en/agents-and-tools/tool-use/memory-tool
  5. Claude Code docs, How Claude remembers your project: https://code.claude.com/docs/en/memory
  6. OpenAI, Codex memories: https://learn.chatgpt.com/docs/customization/memories?surface=app
The first half hour · free

What does your AI forget?

  1. 01You tell us which AI you use and where it keeps starting over.
  2. 02We show what a shared memory changes there, on your own examples.
  3. 03You keep the plan, with or without Bombos.
Piet Baudoin
Piet BaudoinYou talk to Piet, from the Bombos teamDutch team from Utrecht and Nijmegen · data in Europe
We reply the same working day.