The AI crawling standard proposed by Kevin Kelley of Anthropic uses a shared policy token within a website's robots.txt file to block scrapers collecting generative AI training data across multiple providers simultaneously

Opening sign-in…