The llms.txt file, a proposed but unofficial standard for guiding AI crawlers, has demonstrated little practical utility for most websites in 2026, according to recent traffic logs and industry studies released November 5.
Despite its introduction in September 2024, the file offers no measurable benefit for typical websites seeking improved visibility with Google or other AI engines, data indicates.
Analysis of Abondance.com’s traffic logs between June 1 and November 4, 2026, revealed that only 10 of 1,657 requests for llms.txt or llms-full.txt originated from identified AI robots. Of those, only two requests were attributed to ChatGPT.
Google Search officially disregards llms.txt files, stating they neither positively nor negatively affect search rankings or AI-related features, according to Google’s John Mueller.
Studies conducted in May 2026 by SEO analytics firms Ahrefs and SE Ranking found no correlation between the presence of llms.txt files and citations by large language models. These studies reported that 97 percent of such files received no queries from large language models.
The primary entities accessing llms.txt files are SEO tools, GEO tools, and various scanners, which accounted for 35.3 percent of requests on Abondance.com, rather than major AI bots, Abondance reported.
The concept for llms.txt was initially proposed by Mintlify, Meta, Common Crawl, and OVH, among others, to provide a standardized method for website owners to control how AI models access and use their content.
However, the file has not gained traction as a widely adopted web standard, leading to its limited impact.
Organizations like Apple, ByteDance, and the Internet Archive were also involved in early discussions regarding the file’s potential.
Experts noted that without broader adoption and recognition from major AI developers and search engines, the llms.txt file remains largely ineffective for its intended purpose of managing AI access and content usage.
Source: Abondance
