// search indexing

Source: Search Engine Journal

Website operators attempting to prevent AI training bots from scraping their content are accidentally blocking Googlebot, creating a painful tradeoff between machine-learning privacy and organic search visibility. The tools lack the granularity to distinguish between Google's indexing crawler and other bots, forcing publishers into a binary choice between protecting their data and maintaining search rankings. As more sites implement Cloudflare's AI blocking, search index coverage for a meaningful slice of the web could degrade.