> ## Content Index
> Fetch the complete content index at: https://adjacent.media/llms.txt
> Use this file to discover other available public pages before exploring further.

# Most websites' AI bot instructions go completely unread
- URL: https://adjacent.media/signals/most-websites-ai-bot-instructions-go-completely-unread/
- Published: 2026-06-16T16:12:10.000Z
- Updated: 2026-06-16T16:12:10.000Z
- Description: Ahrefs’ analysis of 137,000 domains reveals that the llms.txt protocol—meant to guide how AI systems crawl and use website content—is almost entirely ignored in practice, with 97% of files receiving zero requests from AI bots.
- Author: Jonathan Greene
- Tags: #signal, theme-consumer, attention economy, platform dynamics, ai adoption

Source: [Search Engine Journal](https://www.searchenginejournal.com/97-of-llms-txt-files-got-no-requests-ahrefs-data-shows/579478/?ref=adjacent.media)

Ahrefs' analysis of 137,000 domains reveals that the llms.txt protocol—meant to guide how AI systems crawl and use website content—is almost entirely ignored in practice, with 97% of files receiving zero requests from AI bots. Websites are creating these files to appear responsible, while AI companies' crawlers largely bypass them, leaving the protocol functionally useless as a control mechanism. For publishers and brands worried about content scraping, protection will come through legal leverage, technical barriers, or direct deals with major AI labs—not voluntary machine-readable instructions.