Generative AI Threatens Web Survival, Founder Warns

Image credit: Abondance

Generative artificial intelligence is causing the irreversible degradation of the internet’s crawlable web by disrupting the economic model for content creators, Sylvain Peyronnet, founder of IBOU, said Tuesday.

Peyronnet modeled the crawlable web as an economic common, demonstrating the existence of a unique threshold beyond which the collective corpus of online content irreversibly declines and disappears.

Peyronnet’s analysis indicates that generative AI breaks an implicit contract between publishers and search engines by extracting content value without returning traffic, leading to significant losses in publisher revenue.

Google traffic to publishers has dropped by 30 percent in one year, and clicks on the first search result position fall by two-thirds when an AI Overview is displayed, according to data cited by Peyronnet.

As content extraction rates increase, three simultaneous degradations occur: higher-quality content moves behind paywalls, the rate of content renewal slows, and content increasingly shifts towards real-time data or low-quality AI-generated material, often referred to as ‘AI slop’.

The mathematical model developed by Peyronnet proves the existence of a critical threshold where the renewal of the web’s content corpus no longer compensates for its degradation, leading to an accelerating decline in overall quality and availability.

A significant number of websites are already taking action against AI crawlers. Approximately 25 percent of the top 1,000 global websites currently restrict AI crawlers, with the most reliable media sources being the most active in blocking, according to data from Ahrefs and Chartbeat.

This trend suggests a growing effort by publishers to protect their content from unauthorized AI extraction, the Reuters Institute reported.

Peyronnet proposed seven corrective mechanisms to maintain the system below the degradation threshold. These mechanisms range from mandating traffic commitments from AI providers to implementing collective rights management systems for content.

These proposed solutions aim to re-establish an economic balance that incentivizes content creation and ensures the long-term viability of the open web, Peyronnet stated.

Source: Abondance