Cloudflare will block mixed-use AI crawlers from ad-supported pages
Cloudflare is shaking things up for AI companies: starting September 15, 2026, it will block "mixed-use" AI crawlers from accessing web pages with ads by default.
The idea is to help publishers keep more control over their content and protect their intellectual property as AI tools get smarter.
Now, AI firms will need to use dedicated crawlers for different tasks like search or training.
Cloudflare introduces pay-per-use for AI
This policy will hit all new users, sites made by existing customers, and free accounts. It is Cloudflare's way of balancing visibility for websites with fair treatment.
Publishers have been worried about losing revenue and ownership when AIs scrape their stuff without paying up.
On top of that, Cloudflare is rolling out a pay-per-use model so websites can actually charge AI companies when their content gets used in things like search results or premium features.
Ceramic.ai and You.com are already testing this system out.