llms.txt: what it is, who actually reads it, and when it's worth having

Publishing an llms.txt file has become one of the most repeated recommendations in GEO and AEO advice over the last two years, that is, in advice on preparing a site for generative search and answer assistants. Here's what it actually is, who reads it according to primary sources, and what to do once you find out that, for most sites, that's close to nobody.

What the file contains

llms.txt is a markdown file at the root of a site (/llms.txt) meant to give language models a concise, machine-readable overview of a website: a heading, a short summary and a list of links to key pages. The canonical source is llmstxt.org: only one thing is required, an H1 heading; on top of that comes an optional blockquote summary, free-form text, and H2-headed sections listing links in the form - [name](url): note.

Sites also commonly publish llms-full.txt, the full text of every linked page in one file. That is not part of the core specification; it's a convention introduced by tooling around llms_txt2ctx (FastHTML). Presenting llms-full.txt as part of an official standard isn't accurate.

Who reads it, according to primary sources

This is the question most llms.txt guides skip: whether anyone downloads it at all. It's answerable directly from primary sources, not from blog posts:

Google's own documentation says it outright: "Google Search itself doesn't use them." OpenAI, Anthropic, Perplexity, Microsoft and Mistral don't declare reading llms.txt either, and their crawler documentation talks exclusively about robots.txt. The one group with documented llms.txt consumption is coding tools.

We almost fell into a related trap ourselves: OpenAI, Anthropic, Perplexity, Microsoft and Mistral all publish their own llms.txt for their documentation sites. A lot of SEO content frames that as "industry support for the standard", but it's actually the opposite. Those companies are publishers of content for other tools to consume (mostly code editors), not consumers of the files you publish.

Numbers worth trusting, and one that isn't

The best available adoption and traffic measurement comes from Ahrefs (137,210 domains, June 2026): 28% of tracked sites publish the file, but 97% of them never receive a single request. Among the requests that do arrive, the largest group is traffic from SEO audit tools, not AI assistants.

A number circulating online, "408 requests out of 500 million events", is often misattributed to OtterlyAI. Don't cite it: it comes from a small analytics vendor with no published methodology, and the actual OtterlyAI study has entirely different numbers (84 requests out of 62,100 on a single site). It's the kind of statistic that spreads because it sounds convincing, not because it's verified.

When it's actually worth having

Two situations where llms.txt makes real sense today: documentation sites and developer tools, where coding assistants demonstrably fetch it, and sites where you simply want a dense, machine-readable overview on hand for the future, because adoption among AI systems is growing (from a low base), and the convention could still gain wider traction.

In both cases the same rule applies: llms.txt is a reasonable addition to a site that's already indexable and machine-readable, not a substitute for it. A site AI crawlers can't read in the first place won't be saved by an llms.txt file (see JavaScript and AI crawlers).

Concrete criteria you can use to decide whether it's worth it for your own site:

• Your site has documentation, an API reference, or guides a coding assistant might read, because that's where a real consumer exists.
• Your site has hundreds or thousands of pages and you want one dense overview on hand, as a practical convenience for future tooling rather than an AI visibility play.
• You've already got indexability, robots.txt and sitemap.xml sorted, so llms.txt is something you add on top, not instead of them.

How to write one, if you decide to

llms.txt and sitemap.xml solve different problems and aren't interchangeable. A sitemap is a machine-generated, uninterpreted list of URLs for indexing, and it can easily hold thousands of entries. llms.txt is the opposite: a short, hand-curated, interpreted overview, a few dozen links at most, each with a note explaining what it is. An auto-generated llms.txt listing every URL on your site defeats the purpose, because it gives a model no more of an overview than it would get by crawling the site itself.

The minimal shape per llmstxt.org: an H1 with the site name, one summary sentence in a blockquote, then sections of links:

• # Site name
• > One-sentence summary of what the site is.
• ## Documentation: links to key pages, each with a short note
• ## Optional: an optional section of links a consumer may skip when it needs a shorter context

The location is fixed (/llms.txt at the domain root, not a subfolder) and the format is plain markdown, with no HTML tags and no embedded scripts. Update it when the site's structure changes meaningfully, and whenever it contains details that change, such as prices.

What this means for your site

Generating the file is a reasonable, low-cost step. What you can't honestly promise is that llms.txt will improve your visibility in AI answers or increase the odds a model cites you. None of the major AI companies declare reading other sites' llms.txt, and Google's own documentation says outright it doesn't use it. Anyone promising otherwise today is promising more than they know.

Torumata generates both llms.txt and llms-full.txt for download as part of an audit, as an indexability add-on, not a citation guarantee. Both files state the audit date their content reflects. If they pick up prices from your site, we point that out right next to the file: after you change your pricing or offer, regenerate them with a new audit. What actually, measurably increases citation odds is covered in What documentably increases your chance of being cited by AI.

Want to see where your own site stands? Run a free audit.