SEO explained
llms.txt, and whether you need one
llms.txt is a proposed file at the root of your domain that hands large language models a curated, markdown summary of your site — what it is, which pages matter, and where the plain-text versions live. It is a convention put forward in 2024, not a standard, and no major AI provider has publicly committed to reading it.
What it is trying to solve
A model that arrives at a modern web page has to work through navigation, banners, scripts and markup to reach a few hundred words of substance, and it has a context window to spend. llms.txt proposes handing it the substance directly: a short markdown file listing the pages that matter, with one-line descriptions.
The comparison people reach for is robots.txt, and it is misleading. Robots.txt tells crawlers where not to go and is honoured by every major crawler. This tells them what is worth reading and is honoured by nobody in particular.
What goes in it
Markdown, at /llms.txt. An H1 with the site or product name, an optional blockquote summarising it in a sentence, then H2 sections grouping links — each a markdown link plus a short description of what the page covers.
The convention also suggests shipping a plain-markdown twin of each page at the same URL with .md appended, and an llms-full.txt containing the whole documentation inline. Those are the expensive parts, and the parts least likely to be read.
The honest state of support
As of 2026 no major assistant has documented reading it. Anthropic, OpenAI and Google publish crawler names and robots.txt behaviour; none of them promise anything about this file. Server logs are the only way to know whether anything fetches yours, and mostly they will show that nothing does.
That is not an argument against shipping one. It is an argument against reporting it as a completed optimisation, and against the llms-full.txt variant, which is real work for a convention that may not survive.
What to do instead, or as well
The things that demonstrably affect whether a model can use your page are unglamorous: let the crawlers in, put the answer near the top in plain words, and keep the markup semantic. A model that can read your page does not need a map to it.
If you want to influence AI answers, the measurable work is on the other side — knowing which prompts name you today, so you can tell whether anything you changed mattered.
Questions people ask
- Does llms.txt actually do anything today?
- There is no public evidence that a major assistant reads it. Treat it as a cheap bet on a convention rather than a change with a measurable effect, and check your own server logs before claiming otherwise.
- Is it the same as robots.txt?
- No. Robots.txt restricts crawling and is universally honoured; llms.txt suggests what to read and is honoured by no major provider. Only the filename convention and the location are similar.
- Will llms.txt stop AI from training on my site?
- No — it has nothing to do with permission. Blocking training or crawling is done in robots.txt with the provider's crawler name, such as GPTBot, and each provider documents its own.
Tools that answer this on your own site
- Site Audit
Crawl your site and group what is broken
- Brand Lookup
How often models mention and cite your brand
Read the theory, then check your own site
The free plan opens every tool with 500 credits and no card, so you can look at your own numbers rather than an example.
Start free