Industry

Cloudflare's new policy: block mixed crawlers by default, push AI companies to pay

Original title:Cloudflare’s new policy pushes AI companies to pay for publishers’ content

AI summary

Cloudflare announced that starting September 15, 2026, its default settings will block "mixed-use" crawlers used for search, AI agents and training from accessing pages that host ads, unless site owners manually adjust settings. The move aims to protect publishers' content from being used without compensation. It also upgrades the existing "Pay Per Crawl" model to "Pay Per Use," allowing publishers to charge AI companies when content creates value, with Ceramic.ai and You.com as initial partners. Cloudflare data shows that more than 50% of AI crawler fetch traffic is wasted on repeatedly retrieving unchanged pages. The new policy applies to new customers, new sites from existing customers and all existing free customers.

Key Takeaways

  • Cloudflare announced that starting September 15,2026,its default settings will block "mixed-use"…
  • The move aims to protect publishers' content from being used without compensation.
  • It also upgrades the existing "Pay Per Crawl" model to "Pay Per Use," allowing publishers to…
  • ai and You.
  • com as initial partners.

Quick Facts

Published
Updated
Source
TechCrunch:AI(RSS)
Category
Industry
AI score
72
Read the original Read the full article at the original source
Why it matters

Cloudflare用基础设施商的话语权重新划定了AI内容付费线,默认屏蔽混合爬虫这一刀切下去,从Google到创业公司都躲不开,出版商到底该分多少钱的讨论终于有了一个落地的支点。

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: Cloudflare's new policy: block mixed crawlers by default, push AI companies to pay. July 2, 2026. https://new.nat6.net/en/news/cmr2dvczx075wsl8znz2joowc/

View all →