robots.txt blocks advertising bots

The audit reads robots.txt the same way as for ACC-01 and looks for groups that disallow the whole site (Disallow: / or Disallow: /* without Allow: /) for bots that serve advertising. In config/ai-bots.json their purpose is ads_validation, and today there are two: OAI-AdsBot (OpenAI, ads in ChatGPT) and Meta-ExternalAds (Meta).

Until 4 Nov 2026 ACC-01 watched them together with the other AI bots, so a site that blocked only an advertising bot got the critical finding "you block AI crawlers". That wasn't true: these bots have nothing to do with search or with AI assistants' answers. That's why they have an informational finding of their own.

What the check does not do:

• It doesn't change the score. The finding is informational, weight zero.
• It doesn't check whether you advertise. We can't tell that from the site, so the finding only says whom the block may concern.
• It doesn't verify that the bot obeys the rule. For OAI-AdsBot, OpenAI's documentation doesn't say; see below.
• Partial blocks (Disallow: /offers/) are not reported, just as with ACC-01.

The "undocumented" label doesn't mean the finding promises something without support. It means it promises no effect on AI visibility at all: it only describes what your file blocks and whom that may concern. The support is both operators' documentation. OpenAI writes about its bot (verified 4 Nov 2026):

OAI-AdsBot is used to validate the safety of web pages submitted as ads on ChatGPT. When you submit an ad, OpenAI may visit the landing page to ensure it complies with our policies. We may also use content from the landing page to determine when it’s most relevant to show the ad to users. OAI-AdsBot only visits pages submitted as ads, and the data collected by OAI-AdsBot is not used to train generative AI foundation models.OpenAI, Overview of OpenAI crawlers

About Meta-ExternalAds, Meta says only that it crawls the web for use cases such as improving advertising and other business-related products and services. Meta doesn't have the "only pages submitted as ads" sentence for it, so the finding doesn't attribute it to Meta. For citations in Meta AI's answers Meta has a different bot, Meta-WebIndexer, just as OpenAI has OAI-SearchBot for search.

What we don't know: whether OAI-AdsBot follows robots.txt. The introduction of OpenAI's page names only OAI-SearchBot and GPTBot as robots.txt tokens, and the OAI-AdsBot section says nothing about robots.txt. So we don't promise that the block applies to it, or that allowing it changes anything. For Meta-ExternalAds, Meta states in general that you block its crawlers with a rule in robots.txt, and its sample file gives it a group of its own.

If you don't advertise in ChatGPT or with Meta and don't plan to, there's nothing to change; you can keep the block. The fixed robots.txt the audit offers for download leaves it alone too.

If you do advertise, allow the relevant bot at least for the pages your ads lead to. The simplest way is a group of its own:

User-agent: OAI-AdsBot
Allow: /

User-agent: Meta-ExternalAds
Allow: /

The ad's landing page should also open without a login and without a redirect off your domain; this check doesn't measure that, though.