ACC-02

The site has no llms.txt file

InfoCrawler access

What the check measures

The audit sends a single request to /llms.txt at the root of your domain (under our own User-Agent, so your logs show who asked) and looks at nothing but the status code. HTTP 200 means the file exists; anything else (404, 5xx, timeout, DNS failure) counts as “not there”. ACC-02 is raised only in the second case.

What the check does not do, worth knowing before you act on it:

  • The file's contents are never downloaded or stored. An empty llms.txt passes exactly like a carefully written index.
  • Nothing is validated against the llmstxt.org specification; a missing H1 or broken links inside the file go unnoticed.
  • It is not compared against your site map or sitemap. If the file points at pages that no longer exist, ACC-02 will not tell you.
  • /llms-full.txt is not checked. It is not part of the core specification; it is a convention introduced by tooling.

So this is a check of presence, not quality. It applies to the whole site rather than individual pages, so it appears in the report at most once. And because it is informational, it costs you no score at all: informational findings carry a weight of zero in our model.

How strong the evidence is

Effect not demonstrated

We recommend it because it does no harm or has some other benefit, but we promise nothing about whether it makes language models cite you. Nobody has demonstrated that yet.

Every other “optimise for AI” guide opens with llms.txt. We will say it plainly instead: there is no evidence that it affects whether language models cite you. That is not caution on our part; it is what the operators' own documentation says.

WhoReads other sites' llms.txt?How we know
Google SearchNo, explicitlyGoogle's own documentation, added June 2026
OpenAI (GPTBot, OAI-SearchBot)Not declaredtheir crawler overview describes robots.txt only
Anthropic (ClaudeBot, Claude-SearchBot)Not declaredtheir crawling support article never mentions llms.txt
Perplexity, Microsoft, Meta, MistralNot declaredtheir crawler documentation covers robots.txt only
Coding toolsYes, documentedserver logs in the Ahrefs study, Claude-Code among the user agents

Add the traffic measurement across 137,210 domains (Ahrefs, June 2026): 28 % of sites publish the file, and 97 % of those never receive a single request for it. Among the requests that do arrive, the largest group is SEO audit tools, tools like ours, not language models.

Two arguments circulate about llms.txt that do not hold up. First: “OpenAI and Anthropic support the standard, they publish their own llms.txt.” They do, as publishers of content for other people's tools, not as consumers of anyone else's file. That is the opposite of what gets inferred from it. Second: the figure “408 requests out of 500 million events” travels with no methodology at all, and misattributed to a company whose actual study reported entirely different numbers. We do not cite it.

Why we tell you this when it weakens our own sales pitch. A tool that admits “this one has no measured effect” is worth believing on everything else, and the rest of the audit contains checks backed by Google's own documentation or by a peer-reviewed experiment. If we sold you llms.txt as a route to citations in ChatGPT, you would have to apply that same discount to every other number we give you.

We still generate the file and still recommend it, just for a different reason than the one usually given. On documentation and developer-facing sites the benefit is immediate and documented, because coding tools genuinely fetch it. And adoption keeps growing year on year, so the bet on the future costs a few minutes.

How to fix it

Start with the decision, not the writing. Do you run documentation, an API, guides, or anything else developers read? Then deploy llms.txt; it is the cheapest documented win in the whole audit. Is this an ordinary company site or a shop? Then it is optional; put it behind ACC-01 (so AI crawlers are allowed in at all) and ACC-03 (so they find something without running JavaScript). Those two have the backing that ACC-02 lacks.

If you go ahead, the minimum valid file is short. It is plain markdown and must be served at exactly https://your-domain/llms.txt: the root, not a subdirectory, and not a redirect to another domain (the check reads the status code, so a redirect chain ending in 404 is an absent file as far as it is concerned).

  • An H1 heading: the only mandatory part. The name of the site or product.
  • A blockquote (> …): two sentences on what the site is. The specification places it as the summary right after the heading; what any given tool does with it is not documented.
  • Optional free text with no headings: context that fits nowhere else.
  • H2 sections holding link lists in the form - [title](url): short note. The note after the colon is optional, but it is the part that decides whether the file is any use.
  • The special ## Optional section: what a consumer may skip when it needs shorter context.

Three mistakes we see most often: the file is generated once and left for years (its links rot faster than the content does; build it alongside your sitemap, from the same source); it lists the entire site instead of the twenty pages that matter (the point of the file is selection, not completeness); and it is served as Content-Type: text/html because a framework rendered it as a page, which technically passes, but it is markdown and belongs under text/plain or text/markdown.

And the part that matters most: do not expect better visibility in AI assistants' answers. That cannot be demonstrated today. Expect that the tool searching your documentation on a developer's behalf will understand it better, and that when the convention does take hold, you will already have it.

What the report says about it

Finding description

No /llms.txt was found on the domain, the llmstxt.org convention, a structured signpost a site offers language models to navigate its content. We report it as information, not a fault: in June 2026 Google explicitly added to its documentation that Search does not use the file, for rankings or for AI Overviews, and neither OpenAI nor Anthropic states in their crawler documentation that they read it. What demonstrably does fetch it are coding tools that need to understand someone else's documentation or API, which is why sites like Stripe, Vercel, and Cloudflare publish one.

Recommendation

If you run documentation, an API, or content aimed at developers, deploy llms.txt: there the benefit is immediate and the work takes minutes. For an ordinary company site it is a bet on the future: adoption is growing year over year, the standard may well take hold, and having the file certainly does no harm. Don't expect it to improve your visibility in AI answers though; that cannot currently be demonstrated. Prioritise robots.txt (so AI crawlers may enter at all) and structured data, where the effect is documented. Sources: specification https://llmstxt.org/ (opens in a new window) | Google's position https://developers.google.com/search/docs/fundamentals/ai-optimization-guide (opens in a new window) | adoption and traffic measured across 137,000 domains https://ahrefs.com/blog/llmstxt-study/ (opens in a new window)

Sources

Text verified 2026-09-12