Skip to content

News

Meta Expands Web Crawling Rights for Product Training, Sparks Concerns

Meta's new Commercial Terms, effective Oct. 30, grant the company broader rights to crawl business websites for product training, sparking debate on news scrapi…

Person holding a phone displaying the Meta logo in front of Facebook Business Manager.
Rosario, Argentina – June 20, 2024: Man holding a smartphone showing the Meta logo on the mobile phone screen. Computer with Facebook Business Manager on... Image: Search Engine Journal

Meta Platforms Inc. updated its Commercial Terms, effective October 30, to broaden its rights to crawl business websites and use the collected content for product training, raising concerns among legal experts about potential news site scraping.

The expanded terms apply to any business utilizing Meta’s products, including Facebook Pages and Meta Business Suite, extending beyond merely advertisers, according to the company.

Legal professionals suggested the new language could permit Meta to scrape news sites specifically for artificial intelligence (AI) training, a claim Meta spokesperson Andy Stone denied.

Meta stated the update aims to ensure accurate product information for its advertising and content recommendation systems and is not intended for scraping news sites.

The revised clause explicitly allows Meta to deploy web scrapers, bots, and other automated tools, and to create “test credentials” to access pages, potentially bypassing registration walls.

While Meta indicated that standard robots.txt files could be used by site owners to control crawling, legal experts like Matt Topic, a lawyer specializing in digital rights, believe the new terms might supersede such blocking instructions.

James Grimmelmann, a professor at Cornell University who studies law and technology, echoed these concerns, telling Nieman Lab that the terms could give Meta “unilateral permission” to access content.

The Intercept previously reported on the implications of these changes for publishers and content creators.

The company said the updated terms consolidate existing crawling permissions from various Meta agreements to reduce overlap and streamline its legal framework.

The move comes as tech companies like OpenAI face scrutiny over their use of web content for AI model training.


Need links for your own site?

Guest posts from $59, link insertions from $45, or a managed plan from $269/mo.