Google Teams Offer Conflicting Guidance on LLMs.txt File

Image credit: Search Engine Journal

Google’s internal product teams are providing conflicting guidance to website owners regarding the implementation and necessity of the llms.txt file for generative artificial intelligence features and agentic browsing.

The discrepancy stems from Google Search’s official optimization guide, which states llms.txt is not required for generative AI features, while the Google Chrome Lighthouse tool now advises site owners to create the file for AI agent interaction.

Google Search team members, including John Mueller, Gary Illyes, and Amir Taboul, have consistently communicated that Google Search is not adopting llms.txt for its indexing or ranking processes.

Conversely, Lighthouse version 13.3 introduced an llms.txt audit under a new ‘Agentic Browsing’ category, explicitly recommending its creation.

Lighthouse documentation describes llms.txt as an ’emerging convention’ designed to help large language models (LLMs) and AI agents comprehend website content and interact appropriately.

The conflicting advice has created confusion for web developers and site administrators attempting to optimize their online presence for evolving AI technologies.

Industry analyst Lidia Infante noted the contradictory signals from different Google divisions, highlighting the challenge for webmasters.

Dave Smart, a web development expert, also pointed out the inconsistency, emphasizing the need for clear, unified guidance from Google.

An llms.txt file was briefly observed on Google’s Search Central developer documentation and other Google developer properties, suggesting an internal deployment via a Content Management System (CMS) rather than a direct decision by the Search team.

This temporary appearance further fueled speculation and uncertainty among the web development community.

Google has not yet issued a unified statement to reconcile the differing directives from its Search and Chrome divisions regarding the llms.txt file.

Website owners are left to balance potential future AI agent interactions with current Search optimization practices.

Source: Search Engine Journal