SHANGHAI — Major artificial intelligence search tools, including ChatGPT and Google AI, cited fabricated concepts within a week of their creation, raising concerns about the rapid spread of misinformation, according to a new Fudan University study.
The study found that 8 of 10 AI search tools, including all four English-web tools tested, referenced posts about a fictional concept dubbed ‘Cactus Gravity Protocol’ shortly after researchers published them on frequently cited websites.
Researchers from Fudan University in China conducted tests in March and April, posing 120 questions to 10 AI search tools, four of which focused on the English web and six on the Chinese web.
The team planted articles about the made-up concept from ordinary accounts on sites commonly referenced by AI systems to assess their susceptibility to new, unverified information.
Grok linked to a post about ‘Cactus Gravity Protocol’ in 30 of 32 answers, while Google AI did so in 15 of 33 responses. Perplexity cited the concept in 13 of 35 answers, and ChatGPT included it in 7 of 20 responses, the study reported.
The study indicated that placing information on frequently referenced sources significantly increased the likelihood of citation by AI tools, more so than the volume of posts.
Posts related to opinion questions and made-up products were picked up less frequently by the same AI tools, suggesting some level of content differentiation.
One instance highlighted the speed of AI adoption: a detail planted in a Chinese web posting package was referenced by Doubao, a Chinese AI tool, within one hour of its delivery, according to the Fudan University researchers.
The findings underscore the potential for AI search tools to rapidly incorporate and disseminate fabricated information, posing challenges for information verification and accuracy.
Academics and industry experts, including those from Cornell Tech, have previously warned about the risks of AI models generating and spreading misinformation.
Source: Search Engine Journal
