Cloudflare's Pay per Crawl feature enables website owners to impose a fixed fee for each page request made by AI crawlers. This automates the previously manual negotiation process for content access. The system utilizes the HTTP status code 402, indicating that payment is required before serving content. Publishers can set a price in the Cloudflare dashboard and define actions for known crawlers like allowing access, charging for requests, or blocking entirely. Cloudflare manages the payment process, acting as the merchant of record and simplifying financial transactions for publishers.
Cloudflare's Pay per Crawl feature allows website owners to charge AI crawlers a fixed fee for each page request, automating the billing process at the edge.
This feature utilizes HTTP status code 402, which indicates that payment is required before content is served, streamlining billing for content access.
Collection
[
|
...
]