Google intensifies fight against AI scraping, rank tracking

Google is fighting back against SERP tracking, and the explosion of AI agents may be giving it reason to clamp down harder. — Image credit: Search Engine Journal

Google is intensifying its efforts to combat search engine results page (SERP) scraping and rank tracking globally, driven by a significant surge in AI agent traffic and the high computational costs associated with generating AI-driven search results.

This increased focus comes as AI agent website traffic has spiked by more than 1,700 percent in the past year, contributing to more than half of all internet traffic now being non-human, according to data from Cloudflare.

Generating AI-driven search results is substantially more expensive computationally compared to traditional search queries, posing a significant financial challenge for Google, company officials indicated.

Google is actively working to reduce the cost per query for its AI Mode responses through improvements to large language models (LLMs), hardware optimizations and various engineering enhancements.

Industry observers suggest Google’s actions against rank tracking may also be an attempt to prevent companies from reverse engineering its search algorithm through data distillation.

Ryan Jones, founder of SERPrecon, noted that rank trackers distort actual keyword inventory data and contribute to increased cost per query, which can negatively impact Google’s click and traffic metrics.

Google CEO Sundar Pichai has previously acknowledged the company’s commitment to improving the efficiency of its AI services.

The company has not issued a public statement directly addressing a new policy or specific actions against rank tracking, but industry professionals have reported increased enforcement.

The proliferation of AI agents conducting automated searches places additional strain on Google’s infrastructure, necessitating more aggressive measures to manage traffic and costs.

These developments underscore a broader industry shift where search providers are grappling with the economic and technical implications of advanced AI integration into core services.

Source: Search Engine Journal