AI bots must pay to scrape content under new rules
The IAB Tech Lab just introduced the Content Monetization Protocol (CoMP) v1.0, a set of rules that makes AI companies get permission from publishers before grabbing their content.
This follows reports that some publishers have seen search referral traffic declines exceeding 50%.
The protocol is open for public feedback until April 9, 2026.
How does it work?
With CoMP, AI bots have to ask before scraping anything. If a publisher says no, the bot gets redirected to a licensing site like realsimplelicensing.com.
If they strike a deal, bots explain why they want access and get special tokens as proof.
The system supports pay-per-crawl, pay-per-use, and other flexible payment options, making it easier for both sides to agree on terms.
Why is this needed?
This protocol aims to cut down on messy custom integrations and lower costs by giving everyone one clear process.
Marketplaces are expected to surface content according to agreed dimensions such as relevance, quality, recency, and latency.
Achim Schlosser from Bertelsmann said that scalable, robust compensation frameworks, together with visibility and attribution for content usage, are essential to sustaining high-quality journalism and premium content.
Pricing details haven't been shared yet, but this could be a big step toward fairer deals online.