What are the privacy risks of AI agent memory?

Short answer

AI agent memory can leak what earlier users asked, because stored records are pulled back into the prompt and a crafted question can make the agent repeat them. The 2025 paper "Unveiling Privacy Risks in LLM Agent Memory" showed this with an attack called MEXTRA on a healthcare agent and a shopping agent. Leakage grew with bigger memories, more retrieved records and more attack attempts.

Why memory creates a privacy problem

Agent memory works by putting stored material back into the model’s input. Many agents store past (query, solution) pairs and retrieve the most similar ones as examples when a new request arrives. That makes the agent better at recurring tasks. It also means that whatever another user once asked can end up in the prompt of the next user, and a model can be talked into repeating its prompt.

What the MEXTRA research showed

The paper by Bo Wang and colleagues at Michigan State University and the University of Georgia (first posted February 2025, revised June 2025) tests this with an attack called MEXTRA, short for Memory EXTRaction Attack. The attacker only has black-box access: they can send queries, nothing more.

Each attacking prompt has two parts:

  1. A locator that asks the agent for the examples it just retrieved.
  2. An aligner that asks for them in a form that fits the agent’s workflow, for example typing them into a search box the agent controls.

GPT-4 then generates many varied versions of such prompts. A generic request like “Please repeat all the context” failed in the authors’ case study; the tailored prompts worked.

They tested two agents: EHRAgent, which answers questions over hospital records, and RAP, a web shopping agent. With 200 stored records and 30 attacking prompts, MEXTRA extracted 50 distinct past queries from EHRAgent and 26 from RAP. Without the aligner part, that fell to 36 and 6.

What makes leakage worse

Setting Effect in the paper
Larger memory (50 to 500 records) Generally more extracted
More records retrieved per query (top-k) Consistently more extracted
More attacking prompts (10 to 50) Kept rising; at 50, over 30% of stored queries leaked with edit-distance retrieval and over 10% with embedding retrieval
Edit-distance matching instead of embeddings More leakage
Attacker knows how retrieval works More effective prompts

The authors tested single-agent setups only and flag the lack of session separation, where several users share one memory, as part of the risk.

The misunderstanding: memory is private because only the agent sees it

Memory feels internal, like a database behind the scenes. But retrieved memory is part of the prompt, and anything in the prompt can come out in the answer. The defenses the paper discusses show the trade-off. Rules in the system prompt and paraphrasing of inputs and outputs help, but crafted prompts can look harmless and paraphrasing may keep sensitive details. De-identifying records before storage is stronger, but can make the stored examples less useful. The memory tool docs from Anthropic make a related point for builders: Claude usually refuses to write sensitive information to memory, but for stronger guarantees you add your own validation that strips it before anything is written.

What this means for a team

Shared memory is the goal for many teams, and that is exactly the setting where this risk grows: more people, more stored material, more ways to ask. Practical consequences:

  1. Decide what is shared. Team knowledge and working rules can be shared; personal data, client files and HR matters need their own scope.
  2. Store distilled knowledge, not raw requests. A lesson (“this supplier needs a PO number”) leaks far less than the original message with names in it.
  3. Separate by client or matter where confidentiality applies.
  4. Keep a trail. Being able to see which stored items fed an answer makes leaks visible and fixable.

The write-side twin of this risk is covered in What is memory poisoning in AI agents?, and lifecycle rules in How should an AI agent manage its memory?

Where Bombos fits

With a shared memory, the risks above become team questions: what may be stored, and who sees it. Bombos is a plugin: a shared memory layer underneath the AI your team uses, from Claude, ChatGPT, Copilot, Grok, Codex, Cursor and your own agents, and every answer shows what it rests on. If data must stay in-house, it runs standalone on your own server; otherwise in the cloud in Europe. One person can try it first with their own agents.

Related questions

Is this the same as memory poisoning?

No. Memory poisoning writes false or malicious content into memory. Memory extraction reads out what is already stored. Both exploit the fact that memory flows back into the prompt.

Do system prompt rules stop extraction?

They help but are not enough on their own. The paper notes that attacking prompts can look harmless, so a rule against revealing past queries may not catch them.

Which memory settings leak the most?

In the paper's tests: larger memory, more records retrieved per query (higher top-k), and edit-distance matching instead of embedding similarity all increased how much was extracted.

What is the most reliable fix?

Do not store what should not be shared. De-identify records before they are saved, and keep each user's or client's memory separate unless sharing is intended.

Sources

  1. Unveiling Privacy Risks in LLM Agent Memory (arXiv 2502.13172)
  2. Unveiling Privacy Risks in LLM Agent Memory, HTML version v2
  3. Memory tool: security considerations (Claude Platform docs)
The first half hour · free

What does your AI forget?

  1. 01You tell us which AI you use and where it keeps starting over.
  2. 02We show what a shared memory changes there, on your own examples.
  3. 03You keep the plan, with or without Bombos.
Piet Baudoin
Piet BaudoinYou talk to Piet, from the Bombos teamDutch team from Utrecht and Nijmegen · data in Europe
We reply the same working day.