AI & Business

Meta took the word 'train' out of its new business terms. The test logins, the crawling and the word 'develop' are still in.

Meta says the new clause is only about ads and that robots.txt still works. Then why is robots.txt nowhere in the clause, and why does it need to create logins on your website?

Rohan KumarSansser
13 min read
Halftone photo of the entrance sign at Meta's headquarters in Menlo Park, California
The entrance sign at Meta's headquarters in Menlo Park, May 2022. Photo: Nokia621, CC BY-SA 4.0, via Wikimedia Commons. Halftone by Sansser.

Meta's new business terms, officially called the Commercial Terms, are now due to start on 20 November 2026. They let Meta send bots to any website "associated with" a business that uses Facebook, Instagram or WhatsApp business tools. They let Meta "create test credentials," which means making accounts on your site to get inside it. And they let Meta use what it collects to "provide, develop, improve, research, and maintain" its advertising and business products. After people pushed back, Meta removed the word "train" from the clause. It kept everything else, including the test logins and the phrase "associated with," which it never defines. There is still no way to opt out other than stopping business use of Meta altogether. And when I checked the India-English page on 27 October, it showed two different start dates on the same page.

I read the old terms and the new terms in my own Indian browser. I also read the saved copies from before and after Meta's edit, Meta's own guide to its web crawlers, and Senator Adam Schiff's letter to Meta. Then I read what lawyers, publishers and business owners have said on Hacker News and X since the story broke. Most coverage has treated this as a fight between Meta and big news publishers. The people with the most to lose are much smaller than that, and many of them are in India.

What the clause says, in plain words

This is the new paragraph, exactly as it appears in the India-English edition of the new terms.

Meta Commercial Terms preview, India-English edition, showing the paragraph that lets Meta use web scrapers and create test credentials to crawl websites associated with a business
Meta Commercial Terms preview, India-English edition, read 27 Oct 2026. Highlight added by Sansser.

If you read it slowly, it breaks into four parts. The first part is who does the crawling, and the answer is "us (or any entity acting on our behalf)." That means Meta and any company it hires to do the job. The second part is how Meta gets in. It can use "automated software (such as web scrapers, bots, robots, spiders, crawlers and user agents)." It can also use "test credentials," which means it can make an account on your site the way a customer would. The third part is which websites it covers. The clause names any web pages "that are associated with your products or services." It also covers pages "that are made available through your or users' use of Meta Company Products and services." That includes your own site and also links that other people share about you. The fourth part is the reason. The clause says Meta does this "to provide, develop, improve, research and maintain advertising and business products." It also gives a second reason, "the integrity and security of Meta Company Products."

Nowhere in that paragraph will you find the words robots.txt, paywall or opt-out. And "associated with" is not defined anywhere in the terms. It could mean the website on your Facebook Page, the link in your Instagram bio, the checkout page your ads point to, or the site where your Meta Pixel sits. Meta declined to answer Nieman Lab's list of questions about the new terms, and the terms themselves don't say.

What Meta changed, and what it quietly kept

The first version of this clause was live from early October, according to saved copies on the Wayback Machine. It ended with the sentence that started the whole fight. That sentence read: "Meta may use any content from such domains, web pages, or URLs to provide, develop, improve, research, train, and maintain the integrity of, the Meta Company Products, services, and features…" Nieman Lab reported it on 15 October, and legal experts told the site this allowed Meta to train AI on publishers' websites.

Meta's spokesperson Andy Stone answered on X the next day. "This is false. There's no change to how Meta uses web crawlers. As always, content owners can use robots.txt flags to indicate their preferences," he wrote. By the early hours of the next morning, UTC, the US-English page had changed. The Wayback Machine shows the "train" sentence on the page at 16:20 UTC on 16 October, and gone by 03:19 UTC on 17 October. At 03:14 UTC, Stone posted that "the crawling terms we consolidated only applies to our ads and business products, and we've now clarified that in the terms themselves." But the UK-English edition still said "train" on 18 October, according to PPC Land.

Then on 23 October, Meta pushed the start date from 30 October to 20 November. It said this was "to allow more time for us to explain to our partners and stakeholders what's changing and what's not."

From India, that delay looks messier. On 27 October, the India-English page had a banner at the top saying the updates "will come into effect on 20 November 2026." The bottom of the very same page still said "Effective date: 30 October 2026."

Top and bottom of Meta's India-English Commercial Terms preview page on 27 October 2026, showing 20 November in the banner and 30 October in the footer
Top: banner on the preview page. Bottom: footer of the same page. Both read 27 Oct 2026. Highlight added by Sansser.

It looks like a small thing, but small things like this matter in a contract. Meta has not said which edition an Indian business is bound by. And when a company rewrites the rules for millions of businesses, it should at least get the date right on every page.

What survived the edit is the important part. The words "test credentials," "collect, retrieve" and "associated with your products or services" are all still there. Links from "users' use" of Meta's apps are still covered, and so is "develop, improve, research." James Grimmelmann, a law professor at Cornell, told Nieman Lab after the edit that "'develop…advertising and business products' [language] is still broad enough to drive a truck through." Lawyer Matt Topic went further: "I think 'develop' is broad enough to include 'train.'" Meta's ad tools already use generative AI to write and design ads. So an AI model that helps "develop" ad products counts as a business product, and the clause covers it.

Meta's defence, and why it doesn't hold

Meta's main argument is that nothing here is new. Stone called the crawling terms "consolidated," which means Meta says it just joined older permissions into one document. This claim is easy to check, so I went and read the older terms.

The Ad Creative Generative AI Terms from 2024 let Meta crawl "domains, webpages or URLs provided by you for your use in connection with your ads." The Product Catalog Terms from 2015 say that "if you provide a URL to for product catalog content," Meta may pull updates from it. Both of these are about links you gave Meta yourself, for your own ads or your own product list. The new clause adds sites that are only "associated with" you, links that other people share and test credentials. It also lets Meta use what it finds to build and test all of its ad and business tools. Calling that "consolidating" is like adding a new floor to your house and calling it "tidying up the rooms."

Meta's second argument is that robots.txt still works. Robots.txt is a small file on your website that tells bots what they may and may not visit. Stone has said robots.txt is respected. Meta's statement to Reuters said "content owners will still be able to use robots.txt to signal preferences," and Meta repeated it to Politico. If that is true, it costs Meta one sentence to write it into the clause, and Meta still hasn't done that. Senator Adam Schiff made this point well in his letter of 24 October. "Meta's verbal assurances that it will not supersede robots.txt files, which are voluntary server flags, do not appear to constitute enforceable legal protections," he wrote.

Senator Adam Schiff's letter to Mark Zuckerberg dated 24 October 2026, saying the right to create test credentials is an alarming opening to bypass paywalls and other technical measures
Sen. Adam Schiff's letter to Meta, page 1, dated 24 Oct 2026. Highlight added by Sansser.

Meta's own crawler guide also shows why the robots.txt promise is weaker than it sounds. Meta lists a bot called meta-externalfetcher, which in 2026 it says supports "evaluating and improving agentic AI capabilities—including helping AI navigate websites to complete tasks for users. Accordingly, this crawler may bypass robots.txt rules." So Meta's own guide already says one of its bots "may bypass" robots.txt. And it is the bot built for AI agents that move around websites and do tasks. Put that next to "create test credentials," and the clause starts to look less like ad housekeeping. It looks more like groundwork for AI agents that sign up for websites and click through them.

So my callout goes to Meta, and to Andy Stone by name, since he has made the robots.txt promise in public at least twice. If robots.txt is respected, put the word robots.txt in the clause. If paywalls are respected, put the word paywall in the clause. A promise on X is not a contract, and every business signing this knows the difference.

"It's just legalese, and businesses can leave if they don't like it"

This is the strongest pushback, and plenty of smart people believe it. On Hacker News, nradov wrote: "This is an agreement between two businesses, not a clickwrap license being forced upon consumers. Businesses have plenty of alternatives to Meta; they can just close their accounts if they don't like the terms." Another user, olalonde, added: "In Meta's defense, I don't think you actually need a business' consent to scrape their website."

That second point is partly right, and it is a little funny too. In the US, scraping public web pages is mostly legal, and Meta knows this better than anyone. In 2024, Meta lost a court fight against a scraping company called Bright Data. The judge held that "Meta's terms of service may not be construed to prohibit logged-off scraping of data that is publicly available on Meta's platforms." So if this clause only covered public pages, it would add very little.

But the clause doesn't only cover public pages, and it gives Meta three things the law doesn't hand it for free. The first is logged-in access through test credentials, which can reach the parts of a site that are not public. The second is your signed consent, which makes it much harder for a business to complain later. The third is how it sits next to a wider promise and a wider indemnity. An indemnity means you agree to cover Meta's costs if someone brings a legal claim. You promise that you "own or have secured all rights necessary" for the content you share through Meta's products. That promise now includes the right to "create derivative works." You also cover Meta if "you or anyone" brings a claim. Read strictly, that promise covers what you share through Meta, not your whole website. But the terms never say how it connects to content Meta collects from your site under the crawling clause. That silence is a risk that sits on the business, not on Meta.

The "just leave" argument also breaks down when you look at how small businesses actually sell. A home baker in Pune or a tailor in Surat does not have "plenty of alternatives" when her customers find her on Instagram and order on WhatsApp. University of Wisconsin law professor Benjamin Sobel called this "tying," which is the competition-law word for forcing one deal on top of another. "If you want to use the Meta products that require acceptance of the commercial terms, you have to also acquiesce to the scraping and crawling permissions," he said. No official body has called it tying yet, but they are watching closely. The European Commission welcomed Meta's delay and, according to Reuters, pointed to EU competition rules and the Digital Markets Act. The UK's competition regulator, the CMA, said it is in urgent talks with Meta. And Norwegian publishers have asked a court in Oslo to block the terms.

So I don't buy the legalese argument. The side with all the power wrote this contract, and "you can always leave" only works for businesses that don't need the customers.

Who is actually exposed

Big news publishers have lawyers and industry groups to fight for them. The people I worry about are the ones who have neither.

If you sell a course, run a paid community or keep client files behind a login, read the "test credentials" line twice. A test account is a normal account, so on its own it can't get past a real payment wall. But anything a free sign-up can see, Meta's bots can now see with your written permission. From now on, treat everything behind a free sign-up as public.

If you share stock photos, licensed music, client work or your customers' posts through Meta's tools, the rights promise and the indemnity matter as much as the crawling. That covers your ads, your product catalog and your Page. DannyBee, a Hacker News user who says he is an IP lawyer, put it plainly. "Copyright is strict liability … Unintentional copyright violation, or believing you have a license, is not a defense to infringement," he wrote. You are promising Meta that you hold the rights to what you share. With the crawling clause in the same document, it is worth asking a lawyer where Meta thinks that promise ends.

The scale of Meta's crawling is not small either. DataDome, a company that sells bot blocking, counted 9.1 billion requests from Meta's AI and search crawlers in the second quarter of 2026 alone. That count covers only DataDome's own customers, and those bots sent almost no visitors back in return. At the same time, Nieman Lab reports that Meta has started limiting how many outside links some accounts can post without paying. So links going out of Meta now cost money, while crawling into your site, under these terms, costs Meta nothing.

What this means for businesses in India

The India angle here is real, and almost nobody has covered it. The new terms apply to any business use of Meta's products "regardless of the entity type." That includes running ads, managing a Page, listing products and using Meta Business Suite. Meta's Terms for the WhatsApp Business Platform say they are "supplemental to the Meta Commercial Terms," so every business on the WhatsApp Business API is covered. Indian advertisers sign Meta's Self-Serve Ad Terms with Facebook India Online Services Private Limited, and the new version of those ad terms has the same crawling clause. If a dispute comes up, a business outside the US has to take Meta Platforms to court in California, or Meta Ireland to court in Ireland.

Indian law also protects creators less than many people think. In July 2026, in the case ANI v OpenAI, the Delhi High Court gave an early ruling on AI training. It held, for now, that storing copyrighted work to train an AI model falls under the fair dealing exception in India's Copyright Act, which is India's version of fair use. The court refused to stop OpenAI. So if Meta uses your content to build a product, the legal protection you can count on is weaker in India than in the US or Europe.

There is one Indian case that points the other way. In November 2024, the Competition Commission of India fined Meta ₹213.14 crore over WhatsApp's 2021 privacy update, which forced users to "accept or lose access." In November 2025, the appeals tribunal, the NCLAT, upheld that fine. It is the closest thing India has to the "take it or leave it" problem in these new terms. Yet as of 27 October, I could not find a single statement on the new terms from MeitY, the CCI, the Digital News Publishers Association or the Indian Newspaper Society. European publishers went to court within days, and The Hindu and MediaNama have written about it here, but the people in charge in India have stayed silent.

What I would do this week

Start with robots.txt, because it is free and takes five minutes. Meta's crawler page lists the names of its bots, and these are the lines that turn away its AI crawler and its ads crawler:

User-agent: meta-externalagent
Disallow: /

User-agent: meta-externalads
Disallow: /

Do not block facebookexternalhit. That bot makes your link previews on Facebook and WhatsApp, and blocking it makes your shared links look broken. Meta says robots.txt changes can take up to 24 hours to work. Meta itself says meta-externalfetcher "may bypass robots.txt rules." If you want that bot gone, you need to block it at your firewall or CDN, for example with a Cloudflare rule. I can't tell you yet whether blocking meta-externalads affects your ads or your catalog, because Meta hasn't said. So try it on a less important site first.

After that, move anything valuable behind a payment, not just behind a free sign-up. Check the third-party content you share through Meta's tools, like catalog photos and ad designs, because you are promising Meta you hold the rights to it. Save a PDF of the terms on the day you accept them, in your own country's edition, since the editions have already disagreed with each other. And if you are a US business, Meta's terms give you 30 days from when you first accept them to send a written notice opting out of arbitration.

The bigger point is that you have more power here than it feels like. Meta wants your product pages, because its ads and its AI shopping tools are only as good as the product details behind them. That gives a business owner a real say. So decide on purpose what Meta gets, instead of letting the words "associated with" decide for you.

My prediction

Meta will not add the words "robots.txt" or "paywall" to the crawling clause of its Commercial Terms by 20 November 2026. If either word appears in the live clause on that date, in the US or the India edition, I was wrong.

I am confident about this because Meta has now made the robots.txt promise at least four times in public. It made it on X, to Reuters, to Nieman Lab and to Politico, and each time it chose a statement over a sentence in the contract. Senator Schiff asked for exactly that sentence and set a reply deadline of 2 November. By 20 November, a company that wanted to make this promise binding will have had five weeks to do it.

Questions people are asking

When do Meta's new business terms take effect? Meta says 20 November 2026, after moving the date from 30 October. On 27 October, the India-English page still showed 30 October at the bottom, while the banner at the top said 20 November.

Did Meta remove the AI training part? It removed the word "train" and the sentence about using "any content." The crawling, the test credentials and the use of your content to "develop, improve, research" ad and business products are all still there.

Can I opt out without leaving Facebook or Instagram? Meta offers no opt-out form for these terms. The terms say that if you disagree, you can stop all business or commercial use of Meta's products.

Does unlinking my website from my Facebook Page stop the crawling? Nobody can say for sure, because "associated with" is not defined, and the clause also covers links that other users share. Blocking the crawlers in robots.txt and at your firewall is the step you control.

Do these terms apply to WhatsApp Business in India? They apply to businesses on the WhatsApp Business Platform, which is the API version, because Meta's WhatsApp platform terms sit on top of these terms. Meta has not clearly said how they apply to the separate WhatsApp Business app.

What did Senator Schiff ask Meta to do? In his letter dated 24 October, he asked Meta not to bring in the terms on 20 November while they require "unilateral off-platform scraping." He also asked for "explicit guarantees" that paywalls and robots.txt will be respected, and for a reply by 2 November.

Sources

  1. Meta, Commercial Terms preview (live, read 27 Oct 2026):facebook.com/legal/…
  2. Meta, current Commercial Terms (effective 6 Mar 2024):facebook.com/legal/…
  3. Wayback Machine, Commercial Terms preview with 'train', 16 Oct 2026:web.archive.org/web/…
  4. Wayback Machine, revised preview, 17 Oct 2026:web.archive.org/web/…
  5. Meta, Self-Serve Ad Terms preview:facebook.com/legal/…
  6. Meta, Ad Creative Generative AI Terms (2024):facebook.com/legal/…
  7. Meta, Product Catalog Terms (2015):facebook.com/legal/…
  8. Meta, country-specific ad terms (India):facebook.com/legal/…
  9. Meta, Terms for WhatsApp Business Platform:facebook.com/legal/…
  10. Meta for Developers, Meta web crawlers:developers.facebook.com/docs/…
  11. Andy Stone on X, 16 Oct 2026:x.com/andymstone/…
  12. Andy Stone on X, 17 Oct 2026:x.com/andymstone/…
  13. Sen. Adam Schiff, press release and letter to Meta, 26 Oct 2026:schiff.senate.gov/news/…
  14. Sen. Adam Schiff, letter to Mark Zuckerberg dated 24 Oct 2026 (PDF via Politico):politico.com/f
  15. Sen. Adam Schiff on X, 26 Oct 2026:x.com/SenAdamSchiff/…
  16. Politico, Decoded newsletter (Meta reply to Schiff), 26 Oct 2026:politico.com/newsletters/…
  17. Reuters, Sen. Marsha Blackburn on Meta terms, 23 Oct 2026:reuters.com/world/…
  18. Nieman Lab, 15 Oct 2026:niemanlab.org/2026/…
  19. Nieman Lab, 17 Oct 2026:niemanlab.org/2026/…
  20. Nieman Lab, 23 Oct 2026:niemanlab.org/2026/…
  21. PPC Land, UK edition still carried 'train':ppc.land/metas-ai-training-clause-still-in-oct-30-terms-after-report-of-removal
  22. Press Gazette, Norwegian publishers go to court, 27 Oct 2026:pressgazette.co.uk/news/…
  23. The Hindu, 'Meta's quiet rewrite of the rules for the open web', 23 Oct 2026:thehindu.com/sci-tech/…
  24. MediaNama, 19 Oct 2026:medianama.com/2026/…
  25. SpicyIP, ANI v OpenAI (Delhi High Court), 24 Jul 2026:spicyip.com/2026/…
  26. Quinn Emanuel, Meta v. Bright Data ruling:quinnemanuel.com/the-firm/…
  27. Free Press Journal, NCLAT on CCI's WhatsApp order:freepressjournal.in/tech/…
  28. Let's Data Science, DataDome count of Meta bot requests, Q2 2026:letsdatascience.com/news/…
  29. Hacker News discussion:news.ycombinator.com/item
Newsletter

One email a week. Written by people.

The AI stories worth your time, the tools we tested and what we think, in one short email.

Free · Unsubscribe anytime