---
title: "How to build a knowledge base for AI agents | Bombos"
description: "Put in what an agent cannot guess: rules, decisions, terms and which source leads. Keep it short, one topic per file, loaded when relevant, and dated."
image: "https://bombos.ai/assets/og-en-v2.png"
---

# How do you build a knowledge base for AI agents?

Short answer

A knowledge base for AI agents holds what an agent cannot find or guess on its own: the team's rules, the decisions behind them, the words it uses, and which source is leading per topic. Keep the always-loaded part short, split the rest into one topic per file that loads only when relevant, and give every entry an owner and a date so outdated knowledge gets removed.

*Piet Baudoin · published November 17, 2026 · November 17, 2026*

## What belongs in it, and what does not

An agent can already read your files, search your docs and browse the web. A knowledge base earns its place by holding what the agent cannot work out from those sources. In practice that is five kinds of knowledge:

| Kind | Example | Why the agent cannot guess it |
| --- | --- | --- |
| Rules | “Quotes over €10,000 need a second reviewer.” | Nowhere written, or written in an old email |
| Decisions with their reason | “We dropped supplier X in March because of delivery times.” | The documents show the result, not the why |
| Glossary | “A ‘run’ means a production batch, not a software job.” | Internal words mean something else outside |
| Sources of truth | “Prices: the sheet in Finance/2026, not the PDFs.” | Several versions exist side by side |
| Who knows what | “Contracts: ask the office manager.” | Not in any system |

What does not belong: copies of whole documents (link to the source instead), anything that changes daily and lives in a system already, and personal preferences of one person dressed up as team rules.

## How to structure it, step by step

1.  **Start from questions, not documents.** Collect the twenty questions colleagues and agents got wrong or asked twice last month. Each one points to a missing rule, decision or source.
2.  **Keep a short core that always loads.** Coding agents read a fixed file at the start of every session: AGENTS.md for Codex, Cursor, GitHub Copilot’s coding agent and many others, CLAUDE.md for Claude Code. Anthropic’s docs advise keeping each CLAUDE.md under about 200 lines, since longer files take more context and are followed less reliably. The AGENTS.md site describes its contents as what you would give a new teammate: overview, commands, conventions, security notes.
3.  **Put the rest in one topic per file.** Claude Code supports a `.claude/rules/` folder where each file covers one topic and can be scoped to certain paths, so it only loads when relevant. In a large repo, AGENTS.md files can be nested, and the one closest to the edited file wins. Outside code, the same idea works as a folder of Markdown pages with an index page listing each with one line.
4.  **Let the agent fetch, not swallow.** Anthropic’s guidance on context engineering treats context as a finite resource: keep references (paths, titles) up front and let the agent load the full text just in time with search tools.
5.  **Date and own every entry.** Add who wrote it and when. A rule without a date cannot be judged when it collides with a newer one.
6.  **Prune on a schedule.** Anthropic’s docs warn that two contradicting instructions make the model pick one arbitrarily. A monthly pass that removes outdated and conflicting entries does more for answer quality than adding new ones.

## The misunderstanding: more knowledge makes better agents

The opposite is often true. Every line that is always loaded competes for the model’s attention with the task at hand, and every stale entry is a chance to be confidently wrong. The best knowledge bases for agents are small at the top, specific, and ruthlessly current. A useful test: if a line would not change what the agent does, delete it.

A related pattern is Andrej Karpathy’s LLM Wiki, in which the agent itself maintains a set of linked Markdown pages from raw sources and periodically checks it for contradictions and stale claims. See [What is Karpathy’s LLM Wiki?](https://bombos.ai/en/questions/team-knowledge/karpathy-llm-wiki-knowledge-base/)

## What this means for a team

The files above work per tool and often per repository. The rules your developers put in AGENTS.md do not reach the colleague asking ChatGPT about the same customer, and the decision recorded in one project does not reach another. A team also produces new knowledge daily, mostly as corrections in chats, and those disappear unless someone moves them into the knowledge base. For making existing documents findable, which is a different job, see [How do you make company documents searchable for AI?](https://bombos.ai/en/questions/team-knowledge/how-to-make-company-documents-searchable-for-ai/) and for the background on context, [How to give AI agents your company context](https://bombos.ai/en/knowledge-base/how-to-give-ai-agents-your-company-context/).

## Where Bombos fits

Bombos keeps this kind of knowledge in one shared place underneath every AI tool the team uses: Claude, Claude Code, ChatGPT, Copilot, Grok, Codex, Cursor and your own agents. It is a plugin that learns how your team works, draws on what already sits in mail, documents and repos, and shows with every answer which notes it rests on. It runs in the cloud in Europe or standalone on your own server, and one person can try it first with their own agents.

## Related questions

Should I just upload all our documents?

Documents are the raw material, not the knowledge base. An agent can search them, but it still needs to know which version leads, what was decided later and which rules apply. Those are short notes you write on top.

Which file format works best?

Plain Markdown is read well by every current agent and diffs cleanly in git. Coding agents look for AGENTS.md or CLAUDE.md by name; for other agents, a folder of Markdown files with clear titles and an index works.

How long should the always-loaded instructions be?

Short. Anthropic advises keeping each CLAUDE.md under about 200 lines, because longer files use more context and are followed less consistently. Move the rest into topic files that load only when needed.

Who should maintain it?

Name an owner per topic. Agents can propose updates, for instance after a correction, but someone has to approve what becomes a team rule.

## Sources

1.  [Effective context engineering for AI agents (Anthropic, 29 September 2025)](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents)
2.  [How Claude remembers your project (Claude Code docs)](https://code.claude.com/docs/en/memory)
3.  [AGENTS.md](https://agents.md/)
4.  [LLM Wiki (Andrej Karpathy, GitHub Gist, April 2026)](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f)

## What does your AI forget?

1.  01You tell us which AI you use and where it keeps starting over.
2.  02We show what a shared memory changes there, on your own examples.
3.  03You keep the plan, with or without Bombos.

![Piet Baudoin](https://bombos.ai/assets/team/piet-klein.jpg)

**Piet Baudoin**You talk to Piet, from the Bombos teamDutch team from Utrecht and Nijmegen · data in Europe

## More questions about Team knowledge and AI

-   [What is a knowledge base in AI?](https://bombos.ai/en/questions/team-knowledge/what-is-a-knowledge-base-in-ai/)
-   [How do you set up an AI chatbot on your company knowledge base?](https://bombos.ai/en/questions/team-knowledge/ai-chatbot-with-company-knowledge-base/)
-   [What is Karpathy's LLM Wiki, and does it work as a team knowledge base?](https://bombos.ai/en/questions/team-knowledge/karpathy-llm-wiki-knowledge-base/)
-   [How can AI help onboard a new team member?](https://bombos.ai/en/questions/team-knowledge/how-to-onboard-a-new-hire-with-ai/)
-   [What happens to your team's AI knowledge when the AI champion leaves?](https://bombos.ai/en/questions/team-knowledge/what-happens-when-your-ai-champion-leaves/)
-   [How do you make company documents searchable for AI?](https://bombos.ai/en/questions/team-knowledge/how-to-make-company-documents-searchable-for-ai/)
-   [What working agreements should a team make about using AI?](https://bombos.ai/en/questions/team-knowledge/working-agreements-for-ai-in-a-team/)
-   [Claude Projects or NotebookLM for a team's documents?](https://bombos.ai/en/questions/team-knowledge/claude-projects-vs-notebooklm-for-team-documents/)

[All questions about Team knowledge and AI →](https://bombos.ai/en/questions/team-knowledge/)


---

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://bombos.ai/#organization","name":"Bombos","url":"https://bombos.ai/","logo":"https://bombos.ai/assets/bombos-bee.png","description":"Bombos is the memory layer under every AI your team uses. One shared memory for Claude, ChatGPT, Copilot, Cursor and your own agents, that learns how you work.","sameAs":["https://www.linkedin.com/company/bombos-ai","https://www.youtube.com/@truebombos","https://www.instagram.com/truebombos/","https://bsky.app/profile/bombostrue.bsky.social","https://www.threads.com/@truebombos","https://mastodon.social/@Bombos","https://www.tiktok.com/@truebombos","https://t.me/truebombos","https://www.pinterest.com/truebombos/","https://github.com/bombos-ai","https://linktr.ee/bombos.ai","https://bombosai.substack.com","https://hashnode.com/@bombosai","https://hackernoon.com/u/bombos","https://dev.to/bombos"],"email":"info@bombos.ai","areaServed":["NL","Europe"],"knowsLanguage":["nl","en"],"contactPoint":{"@type":"ContactPoint","email":"info@bombos.ai","contactType":"sales","url":"https://bombos.ai/en/contact/","availableLanguage":["nl","en"]}},{"@type":"WebSite","@id":"https://bombos.ai/#website","name":"Bombos","url":"https://bombos.ai/","inLanguage":["nl-NL","en"],"publisher":{"@id":"https://bombos.ai/#organization"}}]}

{"@context":"https://schema.org","@type":"Article","headline":"How do you build a knowledge base for AI agents?","description":"Put in what an agent cannot guess: rules, decisions, terms and which source leads. Keep it short, one topic per file, loaded when relevant, and dated.","abstract":"A knowledge base for AI agents holds what an agent cannot find or guess on its own: the team's rules, the decisions behind them, the words it uses, and which source is leading per topic. Keep the always-loaded part short, split the rest into one topic per file that loads only when relevant, and give every entry an owner and a date so outdated knowledge gets removed.","datePublished":"2026-11-17T00:00:00.000Z","dateModified":"2026-11-17T00:00:00.000Z","author":{"@type":"Person","name":"Piet Baudoin"},"publisher":{"@type":"Organization","@id":"https://bombos.ai/#organization","name":"Bombos","logo":{"@type":"ImageObject","url":"https://bombos.ai/assets/bombos-bee.png"}},"image":"https://bombos.ai/assets/og-en-v2.png","mainEntityOfPage":"https://bombos.ai/en/questions/team-knowledge/how-to-build-a-knowledge-base-for-ai-agents/","inLanguage":"en-US","isPartOf":{"@type":"CollectionPage","name":"Team knowledge and AI","url":"https://bombos.ai/en/questions/team-knowledge/"},"citation":[{"@type":"CreativeWork","name":"Effective context engineering for AI agents (Anthropic, 29 September 2025)","url":"https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents"},{"@type":"CreativeWork","name":"How Claude remembers your project (Claude Code docs)","url":"https://code.claude.com/docs/en/memory"},{"@type":"CreativeWork","name":"AGENTS.md","url":"https://agents.md/"},{"@type":"CreativeWork","name":"LLM Wiki (Andrej Karpathy, GitHub Gist, April 2026)","url":"https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f"}]}

{"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Bombos","item":"https://bombos.ai/en/"},{"@type":"ListItem","position":2,"name":"Questions","item":"https://bombos.ai/en/questions/"},{"@type":"ListItem","position":3,"name":"Team knowledge and AI","item":"https://bombos.ai/en/questions/team-knowledge/"},{"@type":"ListItem","position":4,"name":"How do you build a knowledge base for AI agents?","item":"https://bombos.ai/en/questions/team-knowledge/how-to-build-a-knowledge-base-for-ai-agents/"}]}

{"@context":"https://schema.org","@type":"FAQPage","mainEntity":[{"@type":"Question","name":"Should I just upload all our documents?","acceptedAnswer":{"@type":"Answer","text":"Documents are the raw material, not the knowledge base. An agent can search them, but it still needs to know which version leads, what was decided later and which rules apply. Those are short notes you write on top."}},{"@type":"Question","name":"Which file format works best?","acceptedAnswer":{"@type":"Answer","text":"Plain Markdown is read well by every current agent and diffs cleanly in git. Coding agents look for AGENTS.md or CLAUDE.md by name; for other agents, a folder of Markdown files with clear titles and an index works."}},{"@type":"Question","name":"How long should the always-loaded instructions be?","acceptedAnswer":{"@type":"Answer","text":"Short. Anthropic advises keeping each CLAUDE.md under about 200 lines, because longer files use more context and are followed less consistently. Move the rest into topic files that load only when needed."}},{"@type":"Question","name":"Who should maintain it?","acceptedAnswer":{"@type":"Answer","text":"Name an owner per topic. Agents can propose updates, for instance after a correction, but someone has to approve what becomes a team rule."}}]}
```
