Source: Search Engine Journal
OpenAI is allowing its GPTBot to bypass robots.txt directives that publishers use to prevent automated access, treating the standard as advisory rather than binding. This escalates friction between AI labs and content creators. Publishers lack technical recourse to stop training scrapes and must rely on legal action, shifting power to AI infrastructure companies that can unilaterally decide which rules apply to them.