Google’s Mueller advises LLMs.txt creation for AI platform requests

Image credit: Abondance

Google search advocate John Mueller said webmasters should create an LLMs.txt file if an artificial intelligence platform specifically requests it for client acquisition purposes.

Mueller’s guidance addresses the evolving interaction between websites and AI agents, suggesting a pragmatic approach to the relatively new protocol for search engine optimization.

While many AI agents currently disregard the LLMs.txt file, its utility remains contingent on the specific requirements of individual AI platforms, according to Mueller. He emphasized that the primary SEO optimization for AI agents involves not blocking them if a site wishes to be crawled.

The advice from the Google representative is considered logical by industry observers, who noted that Google’s own non-utilization of LLMs.txt does not preclude other AI agents from adopting and using the file for their operational needs.

The LLMs.txt file is designed to provide instructions to large language models and other AI crawlers, similar to how robots.txt guides traditional search engine bots. However, its adoption and enforcement across the AI sector are still in nascent stages.

Mueller’s comments indicate a growing need for webmasters to understand and potentially implement new protocols as AI platforms become more integrated into content discovery and client interaction. He highlighted that the decision to create such a file should be driven by direct requests from AI services that aim to bring traffic or clients to a website.

This clarification underscores a nuanced approach to webmaster guidelines, where the necessity of certain files like LLMs.txt is determined by external AI platform demands rather than universal adoption.

Source: Abondance