The web infrastructure revolt over Google's AI Overviews
a day ago
- #Cloudflare
- #AI
- Cloudflare updated robots.txt files to influence Google's crawling practices for AI products.
- The change, called Content Signals Policy, responds to publishers' concerns over revenue loss due to Google's AI Overviews.
- Publishers argue AI answer engines like Google's reduce traffic to original content sources.
- Cloudflare's influence is significant, supporting nearly 20% of the web.
- AI companies are willing to pay for content but fear Google's free access creates an unfair advantage.
- Google's dominance pressures publishers to allow content use in ways they might not prefer.
- Since 2023, Google allows opting out of LLM training but indexing requires accepting AI Overviews via RAG.