Cloudflare Pay Per Crawl (AI Crawl Control)
charging AI crawlers per request
Cloudflare Pay Per Crawl (AI Crawl Control) official (mechanics) + secondary (status, yield) GA since Aug 2025; consolidated across plans Apr 2026 (secondary)
20 AI data/licensing · Web licensing · rail
Sells or earns: charging AI crawlers per request
one flat per-request price across all Charge crawlers; Block/Charge/Allow per crawler; only successful responses billed; robots.txt, sitemap.xml, security.txt and crawlers.json never charged; Cloudflare cut not published
Payout: Cloudflare aggregates, charges crawler, distributes · Eligibility: site on Cloudflare · Notes: ~1B HTTP 402s/day; ~$20-$200 per million pageviews at $0.001-$0.01 per crawl (secondary)
Fields
- category
- 20 AI data/licensing
- subcategory
- Web licensing
- name
- Cloudflare Pay Per Crawl (AI Crawl Control)
- url
- https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/faq/
- type
- rail
- what_you_sell_or_earn
- charging AI crawlers per request
- economics
- one flat per-request price across all Charge crawlers; Block/Charge/Allow per crawler; only successful responses billed; robots.txt, sitemap.xml, security.txt and crawlers.json never charged; Cloudflare cut not published
- payout_threshold
- Cloudflare aggregates, charges crawler, distributes
- eligibility
- site on Cloudflare
- status
- GA since Aug 2025; consolidated across plans Apr 2026 (secondary)
- verified_date
- 2026-09-05
- verification
- official (mechanics) + secondary (status, yield)
- source_url
- https://blog.cloudflare.com/introducing-pay-per-crawl/
- notes
- ~1B HTTP 402s/day; ~$20-$200 per million pageviews at $0.001-$0.01 per crawl (secondary)
Machine-readable: category JSON · record id 20-cloudflare-pay-per-crawl-ai-crawl-control