4 major AI companies support a common AI training preference signal; websites will be able to set it uniformly via robots.txt

Google, OpenAI, Microsoft and Anthropic will support a common AI training preference signal. Website administrators will in the future be able to use the __usage_ai-training marker in robots.txt to uniformly express whether web page content is permitted to be used for generative AI model training, without having to separately maintain each company's rules. The marker follows existing robots.txt syntax; after specifying it in the User-Agent field, Disallow or Allow rules can be applied to the entire site or specific directories. It controls the purpose for which content is used, not crawler access permissions; refusing AI training does not mean prohibiting search engines from crawling or indexing the web page, and existing control rules targeting individual companies can still continue to be used. Anthropic has begun implementation, OpenAI is expected to support it within the next few months, Google plans to introduce it in 2027, and Microsoft is expected to support it on February 1, 2027.