Cloudflare's new policy: block mixed crawlers by default, push AI companies to pay
Original title:Cloudflare’s new policy pushes AI companies to pay for publishers’ content
AI summary
Cloudflare announced that starting September 15, 2026, its default settings will block "mixed-use" crawlers used for search, AI agents and training from accessing pages that host ads, unless site owners manually adjust settings. The move aims to protect publishers' content from being used without compensation. It also upgrades the existing "Pay Per Crawl" model to "Pay Per Use," allowing publishers to charge AI companies when content creates value, with Ceramic.ai and You.com as initial partners. Cloudflare data shows that more than 50% of AI crawler fetch traffic is wasted on repeatedly retrieving unchanged pages. The new policy applies to new customers, new sites from existing customers and all existing free customers.
Key Takeaways
- Cloudflare announced that starting September 15,2026,its default settings will block "mixed-use"…
- The move aims to protect publishers' content from being used without compensation.
- It also upgrades the existing "Pay Per Crawl" model to "Pay Per Use," allowing publishers to…
- ai and You.
- com as initial partners.
Quick Facts
- Published
- Updated
- Source
- TechCrunch:AI(RSS)
- Category
- Industry
- AI score
- 72
Cloudflare用基础设施商的话语权重新划定了AI内容付费线,默认屏蔽混合爬虫这一刀切下去,从Google到创业公司都躲不开,出版商到底该分多少钱的讨论终于有了一个落地的支点。
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: Cloudflare's new policy: block mixed crawlers by default, push AI companies to pay. July 2, 2026. https://new.nat6.net/en/news/cmr2dvczx075wsl8znz2joowc/