Cloudflare lets sites block AI models from using their content
Cloudflare just introduced its Content Signals Policy, letting website owners decide how AI systems—like those from Google—use their content online.
The move comes after worries that AI has been scooping up content without asking, especially since Google uses the same crawler for both search and AI.
With this policy, over 3.8 million sites (about 20% of the internet managed by Cloudflare) get more say in what happens to their content.
The new policy adds 3 signals to robots.txt
The new policy adds three signals to robots.txt: 'search' (for search engines), 'ai-input' (for AIs using site info), and 'ai-train' (for training AI models).
By default, sites will still show up in searches but block AI training. If companies ignore these signals, they could face contractual obligations or legal implications.
It's Cloudflare's way of helping publishers protect their work as the web keeps changing.