What belongs in it, and what does not
An agent can already read your files, search your docs and browse the web. A knowledge base earns its place by holding what the agent cannot work out from those sources. In practice that is five kinds of knowledge:
| Kind | Example | Why the agent cannot guess it |
|---|---|---|
| Rules | “Quotes over €10,000 need a second reviewer.” | Nowhere written, or written in an old email |
| Decisions with their reason | “We dropped supplier X in March because of delivery times.” | The documents show the result, not the why |
| Glossary | “A ‘run’ means a production batch, not a software job.” | Internal words mean something else outside |
| Sources of truth | “Prices: the sheet in Finance/2026, not the PDFs.” | Several versions exist side by side |
| Who knows what | “Contracts: ask the office manager.” | Not in any system |
What does not belong: copies of whole documents (link to the source instead), anything that changes daily and lives in a system already, and personal preferences of one person dressed up as team rules.
How to structure it, step by step
- Start from questions, not documents. Collect the twenty questions colleagues and agents got wrong or asked twice last month. Each one points to a missing rule, decision or source.
- Keep a short core that always loads. Coding agents read a fixed file at the start of every session: AGENTS.md for Codex, Cursor, GitHub Copilot’s coding agent and many others, CLAUDE.md for Claude Code. Anthropic’s docs advise keeping each CLAUDE.md under about 200 lines, since longer files take more context and are followed less reliably. The AGENTS.md site describes its contents as what you would give a new teammate: overview, commands, conventions, security notes.
- Put the rest in one topic per file. Claude Code supports a
.claude/rules/folder where each file covers one topic and can be scoped to certain paths, so it only loads when relevant. In a large repo, AGENTS.md files can be nested, and the one closest to the edited file wins. Outside code, the same idea works as a folder of Markdown pages with an index page listing each with one line. - Let the agent fetch, not swallow. Anthropic’s guidance on context engineering treats context as a finite resource: keep references (paths, titles) up front and let the agent load the full text just in time with search tools.
- Date and own every entry. Add who wrote it and when. A rule without a date cannot be judged when it collides with a newer one.
- Prune on a schedule. Anthropic’s docs warn that two contradicting instructions make the model pick one arbitrarily. A monthly pass that removes outdated and conflicting entries does more for answer quality than adding new ones.
The misunderstanding: more knowledge makes better agents
The opposite is often true. Every line that is always loaded competes for the model’s attention with the task at hand, and every stale entry is a chance to be confidently wrong. The best knowledge bases for agents are small at the top, specific, and ruthlessly current. A useful test: if a line would not change what the agent does, delete it.
A related pattern is Andrej Karpathy’s LLM Wiki, in which the agent itself maintains a set of linked Markdown pages from raw sources and periodically checks it for contradictions and stale claims. See What is Karpathy’s LLM Wiki?
What this means for a team
The files above work per tool and often per repository. The rules your developers put in AGENTS.md do not reach the colleague asking ChatGPT about the same customer, and the decision recorded in one project does not reach another. A team also produces new knowledge daily, mostly as corrections in chats, and those disappear unless someone moves them into the knowledge base. For making existing documents findable, which is a different job, see How do you make company documents searchable for AI? and for the background on context, How to give AI agents your company context.
Where Bombos fits
Bombos keeps this kind of knowledge in one shared place underneath every AI tool the team uses: Claude, Claude Code, ChatGPT, Copilot, Grok, Codex, Cursor and your own agents. It is a plugin that learns how your team works, draws on what already sits in mail, documents and repos, and shows with every answer which notes it rests on. It runs in the cloud in Europe or standalone on your own server, and one person can try it first with their own agents.
Related questions
Should I just upload all our documents?
Documents are the raw material, not the knowledge base. An agent can search them, but it still needs to know which version leads, what was decided later and which rules apply. Those are short notes you write on top.
Which file format works best?
Plain Markdown is read well by every current agent and diffs cleanly in git. Coding agents look for AGENTS.md or CLAUDE.md by name; for other agents, a folder of Markdown files with clear titles and an index works.
How long should the always-loaded instructions be?
Short. Anthropic advises keeping each CLAUDE.md under about 200 lines, because longer files use more context and are followed less consistently. Move the rest into topic files that load only when needed.
Who should maintain it?
Name an owner per topic. Agents can propose updates, for instance after a correction, but someone has to approve what becomes a team rule.
