🔍 Read the full analysis: What The Shared AI Training Opt-Out Means For Your Information on ThorstenMeyerAI.com
Get privacy and security gear delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
OpenAI, Google, Microsoft and Anthropic have announced support for a shared robots.txt signal asking them not to use a website’s content to train generative AI models. The signal is voluntary, will take effect on different schedules, and does not block crawling, govern AI answers based on live web content, or remove material from models already trained.
OpenAI, Google, Microsoft and Anthropic said on October 30 that they will honor a shared website signal asking AI providers not to use a site’s content for generative AI training. The proposed robots.txt rule could replace separate training opt-outs for participating companies, but it is a voluntary request, not a technical block or an adopted internet standard.
The proposed signal uses a robots.txt group with User-Agent: __usage_ai-training and Disallow: /. A site owner adding it would express a preference against having the site’s content used to train generative AI models. The text appears in an individual Internet-Draft titled “Machine-Readable Training Preference Signal,” published October 30 by Anthropic standards lead Kevin Kelley.
Companies have set different timelines. Anthropic says it has already started implementing support, while warning it will take time to add support across its services. OpenAI says it aims to implement the signal “in the coming months.” Google says it aims to support it in 2027, with its crawler documentation specifying early 2027. Microsoft says it will honor the preference from February 1, 2027, as it crawls websites.
Microsoft also announced a separate page-level preference for AI Grounding—content used to inform answers in Bing, Copilot and Microsoft 365. Microsoft says both preferences take effect February 1, 2027. Its older NoArchive and NoCache signals will stop counting as preferences for training or grounding after March 1, 2027. The companies’ existing crawler controls otherwise remain relevant while support rolls out.
Infographic 01 · Who backs it, and when
One line in robots.txt, four companies, four timelines.
A shared signal against AI training. Only Anthropic says it has started; the others name dates or months. The standard itself is still a draft.
- 30 Oct 2026Draft published, four companies announce supportdoneIndividual Internet-Draft by Kevin Kelley (Anthropic)
- 30 Oct 2026Anthropic: “has already started implementing support”startedIts standards lead on the IETF list: “It’ll take us some time to add support across our services.”
- Coming monthsOpenAI aims to implement itplanExisting GPTBot and OAI-SearchBot rules keep applying
- 16 + 19 Nov 2026Working group meets at IETF 127, San FranciscoopenOn 21 Oct a chair said no consensus has been declared yet
- 1 Feb 2027Microsoft honors the new preferences as it crawlsplanTraining in robots.txt or per page; grounding per page
- 1 Mar 2027Bing stops reading NoArchive / NoCache as AI preferencesdeadlinePublish the new settings before this date
- 2027Google plans support; its docs say “early 2027”planA “no” maps to Google-Extended unless you set that token yourself
- 3 May 2027Draft version 00 expires unless updatedInternet-Drafts are valid for at most six months
Binding? No, voluntary
- OpenAI calls it a “voluntary effort”.
- The vocabulary draft does not “ensure that preferences are followed”.
- The draft “has no formal standing in the IETF standards process”.
- Private agreements may override it, the draft says.
Who else?
- Meta supports a common signal, but did not say whether it will honor it.
- Apple, Amazon, xAI, Perplexity: no statement in the sources reviewed for this article.
- The draft’s acknowledgments list contributors “for identification only”, not supporters.
Sources: IETF draft, 30 Oct 2026 · Anthropic on the IETF list · AIPREF minutes, 21 Oct 2026 · Google, 30 Oct 2026 · Google crawler docs · OpenAI, 30 Oct 2026 · Microsoft Bing, Oct 2026 · Meta (Andy Stone on X). Company plans are company statements.
Powered by Thorsten Meyer AI
A Simpler Request, Not a Block
For publishers, the shared rule could reduce the work of expressing a training preference to several participating companies. Until now, site operators have needed company-specific controls, such as rules for GPTBot, Google-Extended and ClaudeBot, and Microsoft-specific page tags. A common signal gives publishers one more direct way to state that preference, though they may need to retain existing rules until each company’s implementation is in place.
The distinction between a request and an enforcement mechanism matters. Robots.txt communicates instructions to crawlers; this shared signal does not technically prevent a company from accessing a page. Its practical effect depends on providers choosing to honor it. Nor does an opt-out create payment, traffic or inclusion in search or AI services. Microsoft says neither choice guarantees any of those outcomes.
The announcement also separates model training from other uses of web content. The shared signal is described as a preference for foundational model training, not for every way an AI service might use a page. Microsoft’s separate grounding control illustrates that distinction: content used to help produce an answer may be governed by another setting. Publishers will need to decide whether their preferences differ for training, AI answers and ordinary search.
Products Worth Considering
Samsung 27 Odyssey OLED G5 (G50SF) QHD QD-OLED Gaming Monitor
Samsung 27-inch Odyssey OLED G6 QD-OLED QHD Gaming Monitor
LG 34GX90SA-W 34-inch Ultragear WQHD OLED Curved Gaming Monitor
As an affiliate, we earn on qualifying purchases.
Company Controls Remain in Use
The proposal is being discussed through the AI Preferences working group at the Internet Engineering Task Force, which develops internet standards. But the draft is not an adopted IETF standard. At a working-group meeting on October 21, a chair said, “We haven’t declared consensus on anything yet.” The announcement reflects commitments by individual companies, not a universal rule for the industry.
Google says that if a site uses the new token and has no Google-Extended rule, it plans to treat the shared preference as if Google-Extended had been disallowed. OpenAI says its existing crawler controls will continue to apply. Google has also said Google-Extended does not affect Google Search, and Microsoft says its preferences do not change a page’s eligibility for Bing search results. These distinctions mean a training opt-out should not be read as a general instruction to remove a site from search.
Meta’s communications chief, Andy Stone, voiced support for publishers’ ability to express preferences and called a common signal an important step, but did not commit Meta to honoring it. The source material reports no statement from Apple, Amazon, xAI or Perplexity. The announced commitments therefore cover only the companies that have stated a position, and their implementation schedules vary.
““has already started implementing support for this draft””
— Anthropic
Coverage and Enforcement Remain Open
The signal’s reach depends on which companies implement it and whether they follow the request. Meta has expressed support without making an implementation commitment, and the material reviewed does not establish plans by several other major AI companies. It is also unclear when Anthropic’s support will cover all of its services or whether the proposed wording will change during the standards process.
The draft has no formal standing in the IETF standards process at this stage. The companies’ announcements are commitments or plans attributed to those companies, not evidence that every service already recognizes the rule. The signal also cannot undo training on content that has already been incorporated into a model, and it does not, on its own, govern AI answers that use live web material.
Publishers should not assume that adding the shared line immediately replaces every existing control. In particular, Google, OpenAI and Anthropic still have company-specific crawler rules, while Microsoft has set dates for its new preferences and the retirement of its older tags. The precise behavior of each service during the rollout remains tied to its published documentation and implementation.
Watch the Rollout Dates
The next developments are whether the IETF working group considers the draft and whether the shared wording changes. Publishers should also watch each provider’s documentation for confirmation that its systems recognize the signal. Anthropic says implementation has started; OpenAI’s stated target is the coming months; and Google’s target is early 2027.
Microsoft’s announced dates are February 1, 2027, for its shared training preference and separate grounding preference, followed by March 1, 2027, when NoArchive and NoCache stop counting as training or grounding preferences. Site operators using those Microsoft tags will need to review their rules ahead of the later date. Until the providers’ changes are in effect, the practical approach described in the source material is to keep existing company-specific controls alongside the new signal if opting out of training is the goal.
Key Questions
What does the shared AI training opt-out do?
It adds a robots.txt preference asking participating AI providers not to use a website’s content to train generative AI models. It does not technically block access to the site.
Does the signal stop AI companies from crawling a website?
No. The signal is a request about training use, not a technical barrier to crawling. Its effect depends on providers honoring it.
Does it prevent a site from appearing in search or AI answers?
Not by itself. The announced preference concerns training, while AI answers and search may involve separate controls. Microsoft has announced a distinct preference for AI Grounding; Google says Google-Extended does not affect Google Search.
When will the companies honor the signal?
Anthropic says implementation has begun. OpenAI aims for the coming months, Google aims for early 2027, and Microsoft’s announced start date is February 1, 2027. The schedules are not the same.
Does this setting apply to my personal AI chats?
No. The signal described here is for a website’s robots.txt file and its published content. It is not an account setting for a person’s conversations with an AI assistant.
Source: ThorstenMeyerAI.com
