What is llms.txt — and should your docs have one?

If you maintain documentation, there's a decent chance an LLM is reading it right now. Someone asks ChatGPT or Claude how to configure your product, the model goes looking for an answer, and it lands on your docs — or some stale, half-garbled version of them. The llms.txt convention is a small, pragmatic attempt to make that encounter go better.

The idea in one file

llms.txt is a markdown file you put at the root of your site — yourdomain.com/llms.txt. It's a curated map of your documentation written for machines: a short description of what your project is, followed by a tidy list of links to the pages that actually matter, each with a one-line note on what it covers.

That's it. It's not a new format to learn. It's just markdown with a loose convention: an H1 with the project name, an optional blockquote summary, and then sections of links. A trimmed example:

# Acme API

> Acme is a payments API for marketplaces.

## Docs
- [Quickstart](https://acme.dev/docs/quickstart.md): Get a first charge working in 5 minutes
- [Authentication](https://acme.dev/docs/auth.md): API keys, scopes, rotation
- [Webhooks](https://acme.dev/docs/webhooks.md): Event types and signature verification

## Reference
- [API endpoints](https://acme.dev/docs/api.md): Full REST reference

There's an optional companion, llms-full.txt, which is the opposite instinct: instead of a curated index of links, it's your entire docs concatenated into one giant markdown file. The index version helps a model decide what to read; the full version hands it everything in one fetch so it doesn't have to crawl. Some sites ship both.

Why it exists

The problem it solves is mundane: rendered HTML is a terrible thing to feed a model. A docs page in the browser is wrapped in navigation, cookie banners, sidebars, search widgets, analytics scripts, and layout markup. A human eye skips all of it instinctively. A model has to wade through it, burning context window on chrome that isn't the content — and sometimes mis-reading the page structure entirely.

Markdown sidesteps that. It's close to how the model "thinks" about text, it's compact, and it puts the signal right up front. llms.txt is the same reflex that gave us robots.txt (here's what crawlers may touch) and sitemap.xml (here's the map of the site) — except aimed at LLMs and answering a different question: here is the clean, curated version of what I want you to understand about this project.

The keyword is curated. The value isn't only the format; it's that a human chose which pages matter and described them. You're not dumping your whole site at a model and hoping it guesses what's important — you're pointing it at the quickstart, the auth page, and the API reference, and skipping the changelog archive and the marketing fluff.

How to write a good one

• Lead with a real description. One or two sentences on what the project actually is and who it's for — the context a model uses to frame everything else.
• Link to clean source, not rendered pages. Where you can, point at the raw markdown (.md) version of each page rather than the HTML. That's the whole point — give the model the unwrapped text.
• Annotate every link. "Authentication: API keys, scopes, and rotation" is far more useful than a bare "Authentication." The descriptions help the model pick the right page before it spends a fetch.
• Curate ruthlessly. Include the pages that answer real questions; leave out the noise. If everything is linked, nothing is prioritized.
• Keep it current. A stale index that points at moved or deleted pages is worse than none. This works best when it's generated from the same source as your docs, so it can't drift.

If your docs are already markdown-native, most of this is close to free — you're publishing text you already have, in the shape it's already in.

The honest caveats

It would be a disservice to sell llms.txt as a settled standard, because it isn't.

It's a proposed convention, not a ratified one. It was put forward by Jeremy Howard in 2024 and picked up traction in developer-tooling circles, but there's no governing body behind it and no guarantee it sticks. Treat it as a community proposal gaining momentum, not a spec you're obligated to follow.

Adoption — and actual use — varies. Plenty of sites now publish one. How much any given model or AI tool actually reads it is far less certain, and changes as products evolve. Some agent frameworks and docs tools look for it; many models still just crawl your HTML like always. Publishing one is a low-cost bet, not a guaranteed pipeline into every assistant.

It's read-only and static. This is the big one. llms.txt is a file — a snapshot a model fetches. It can't run a query, respect permissions, take an action, or reflect data that changed a minute ago. For anything dynamic or interactive you want a live interface, and that's exactly what the Model Context Protocol (MCP) is for. MCP gives a model real tools — read and write, scoped and authenticated, against live state. The two aren't competitors: llms.txt is the cheap, public, static map of your docs; MCP is the live, capable connection into a running system. A good setup can have both, doing different jobs.

So the answer to "should your docs have one?" is a measured yes. It's cheap, it's harmless, and on the off chance a model reads it, you've handed it a clean version of the truth instead of leaving it to scrape your HTML. The downside is mostly the upkeep. Just don't expect it to move mountains on its own.

Where this is heading

The deeper shift behind llms.txt is that documentation is becoming something machines consume, not just something humans read. For years we wrote docs for a person with a browser; increasingly the reader is a model answering on that person's behalf. That changes what "good docs" means — clean source text, stable URLs, honest structure, and machine-readable entry points start to matter as much as visual polish.

This is partly why we built tela the way we did. It's an open-source (AGPL — the whole product, not a crippled core), self-hostable team wiki that's markdown-native from the ground up, so the content is already in the form machines want. It ships an llms.txt at telawiki.com/llms.txt. And it goes a step past the static file with a built-in MCP server exposing 39 tools — the live, read-and-write counterpart to the static map. There's even a feature called Atlas that auto-generates a cited wiki from your git history and Jira, which is the same idea pointed inward: turn the raw material machines produce back into docs both people and models can use.

llms.txt is a small file. But it's a clear signal about where docs are going — and writing one is a cheap way to start treating your machine readers as real readers. The code's on GitHub if you want to see how the pieces fit.