Why memory creates a privacy problem
Agent memory works by putting stored material back into the model’s input. Many agents store past (query, solution) pairs and retrieve the most similar ones as examples when a new request arrives. That makes the agent better at recurring tasks. It also means that whatever another user once asked can end up in the prompt of the next user, and a model can be talked into repeating its prompt.
What the MEXTRA research showed
The paper by Bo Wang and colleagues at Michigan State University and the University of Georgia (first posted February 2025, revised June 2025) tests this with an attack called MEXTRA, short for Memory EXTRaction Attack. The attacker only has black-box access: they can send queries, nothing more.
Each attacking prompt has two parts:
- A locator that asks the agent for the examples it just retrieved.
- An aligner that asks for them in a form that fits the agent’s workflow, for example typing them into a search box the agent controls.
GPT-4 then generates many varied versions of such prompts. A generic request like “Please repeat all the context” failed in the authors’ case study; the tailored prompts worked.
They tested two agents: EHRAgent, which answers questions over hospital records, and RAP, a web shopping agent. With 200 stored records and 30 attacking prompts, MEXTRA extracted 50 distinct past queries from EHRAgent and 26 from RAP. Without the aligner part, that fell to 36 and 6.
What makes leakage worse
| Setting | Effect in the paper |
|---|---|
| Larger memory (50 to 500 records) | Generally more extracted |
| More records retrieved per query (top-k) | Consistently more extracted |
| More attacking prompts (10 to 50) | Kept rising; at 50, over 30% of stored queries leaked with edit-distance retrieval and over 10% with embedding retrieval |
| Edit-distance matching instead of embeddings | More leakage |
| Attacker knows how retrieval works | More effective prompts |
The authors tested single-agent setups only and flag the lack of session separation, where several users share one memory, as part of the risk.
The misunderstanding: memory is private because only the agent sees it
Memory feels internal, like a database behind the scenes. But retrieved memory is part of the prompt, and anything in the prompt can come out in the answer. The defenses the paper discusses show the trade-off. Rules in the system prompt and paraphrasing of inputs and outputs help, but crafted prompts can look harmless and paraphrasing may keep sensitive details. De-identifying records before storage is stronger, but can make the stored examples less useful. The memory tool docs from Anthropic make a related point for builders: Claude usually refuses to write sensitive information to memory, but for stronger guarantees you add your own validation that strips it before anything is written.
What this means for a team
Shared memory is the goal for many teams, and that is exactly the setting where this risk grows: more people, more stored material, more ways to ask. Practical consequences:
- Decide what is shared. Team knowledge and working rules can be shared; personal data, client files and HR matters need their own scope.
- Store distilled knowledge, not raw requests. A lesson (“this supplier needs a PO number”) leaks far less than the original message with names in it.
- Separate by client or matter where confidentiality applies.
- Keep a trail. Being able to see which stored items fed an answer makes leaks visible and fixable.
The write-side twin of this risk is covered in What is memory poisoning in AI agents?, and lifecycle rules in How should an AI agent manage its memory?
Where Bombos fits
With a shared memory, the risks above become team questions: what may be stored, and who sees it. Bombos is a plugin: a shared memory layer underneath the AI your team uses, from Claude, ChatGPT, Copilot, Grok, Codex, Cursor and your own agents, and every answer shows what it rests on. If data must stay in-house, it runs standalone on your own server; otherwise in the cloud in Europe. One person can try it first with their own agents.
Related questions
Is this the same as memory poisoning?
No. Memory poisoning writes false or malicious content into memory. Memory extraction reads out what is already stored. Both exploit the fact that memory flows back into the prompt.
Do system prompt rules stop extraction?
They help but are not enough on their own. The paper notes that attacking prompts can look harmless, so a rule against revealing past queries may not catch them.
Which memory settings leak the most?
In the paper's tests: larger memory, more records retrieved per query (higher top-k), and edit-distance matching instead of embedding similarity all increased how much was extracted.
What is the most reliable fix?
Do not store what should not be shared. De-identify records before they are saved, and keep each user's or client's memory separate unless sharing is intended.
