OpenAI’s Search Index Serves Small Sites, French Study Finds

Close up view of Chatgpt logo on Apple App Store website using magnifying glass lens highlighting mobile software installation page and digital productivity tools. India,… — Image credit: Search Engine Journal

OpenAI’s in-house search index, dubbed ‘labrador,’ provides search results to hundreds of smaller websites without content licensing agreements for free ChatGPT accounts, a French SEO consultancy reported Tuesday.

The findings challenge previous assumptions that OpenAI‘s index predominantly favored large, licensed content partners, indicating a more inclusive approach for its widely used AI chatbot.

Resoneo, the French consultancy, discovered that the ‘labrador’ index handled the majority of ChatGPT search results for users with free accounts, serving numerous outlets regardless of formal content deals. The index presented pages from both licensed partners and non-partners in a consistent format, length and freshness, according to Resoneo‘s analysis.

For paid ChatGPT accounts utilizing ‘thinking mode,’ Google‘s scraping accounted for approximately 75 percent of search results, while OpenAI’s internal index contributed about 24 percent, Resoneo stated.

Suganthan Mohanadasan, an SEO expert, initially believed OpenAI’s index was largely inaccessible to smaller sites. However, he corrected his findings in July after receiving new data from a reader, acknowledging that the index provided broader access than first thought.

OpenAI’s index stores a title and approximately 200 characters from a webpage, Resoneo found. The analysis also indicated that H1 headings were included in 83.6 percent of the snippets when present on the original page.

The revelation suggests that OpenAI’s strategy for content sourcing within its free ChatGPT service extends beyond exclusive partnerships with major publishers like Reuters, The Guardian or The Wall Street Journal, incorporating a wider array of online information, including sources like Wikipedia.

This approach could significantly impact how small and independent publishers view their visibility within leading AI platforms, potentially offering a new avenue for content discovery without requiring direct commercial agreements.

Source: Search Engine Journal