Cloudflare and GoDaddy Combat AI Bot Exploitation of Web Content

Cloudflare and GoDaddy have partnered to enhance website owners' control over AI bots accessing their content. This collaboration introduces Cloudflare's AI Crawl Control tool into GoDaddy's platform, allowing site operators to manage AI interactions by granting access, blocking, or implementing paywalls. The initiative aims to address the economic challenges faced by publishers as AI-generated summaries reduce web traffic to original sources. Cloudflare's chief strategy officer, Stephanie Cohen, emphasized the need for a new internet business model that protects creators' interests. The companies also support the Agent Name Service and Web Bot Auth standards to differentiate legitimate crawlers from malicious ones. With Cloudflare protecting approximately 20% of global websites, this partnership represents a significant step towards establishing enforceable web engagement rules. The ongoing issue of unauthorized AI scraping continues to threaten the viability of online content creators.

Key Points: • Cloudflare and GoDaddy have launched AI Crawl Control to manage AI bot access to websites. • The partnership aims to protect content creators from economic losses due to AI-generated summaries. • New standards like Agent Name Service and Web Bot Auth are endorsed to identify legitimate AI crawlers.