OpenAI, Google, Microsoft and Anthropic back a shared SI training opt-out signal
Prepared by Sintelligent with the help of SI. Published in SI: . Updated: .
Website operators will find it easier to communicate the same training preference to several SI providers.
According to The Verge’s account, a robots.txt signal proposed in an IETF working group will let sites express a preference to multiple crawlers at once. Anthropic already uses it; OpenAI promises support in the coming months, and Google and Microsoft next year. Existing rules for individual crawlers continue to work.
Sources 2