AI Training Opt-Out Signal: What Google, OpenAI, Microsoft, and Anthropic Agreed On

Google, Microsoft, OpenAI, and Anthropic said on October 30, 2026 that they will honor a single shared AI training opt-out signal in robots.txt. Google’s Search Central Blog announced its own support the same day, calling it a way for website owners to set AI training preferences “across multiple AI providers in a single step.” If you run a site and currently juggle separate rules for GPTBot, ClaudeBot, and Google-Extended, this is the first real attempt to collapse that list into one line.

Nothing changes in your robots.txt file today. The companies are committing to a shared draft standard, not shipping a finished feature. Google says it aims to support the signal in 2027, and the other three companies gave their own, slightly different timelines. Here is what the AI training opt-out signal actually is, how the token works, where each company stands right now, and what is worth checking in your own file before any of this goes live.

What Google Just Announced About the AI Training Opt-Out Signal

The announcement came from John Mueller, posting as Google Search Advocate on the Search Central Blog. He said Google plans to support a shared AI training control built on work from the AIPREF working group at the Internet Engineering Task Force, the body that sets technical standards for the web.

Google has offered its own AI training control since 2023 through Google-Extended, a token site owners can name directly in robots.txt to stop their content from feeding certain Google AI training. That control still works. The new piece is a second, shared token that multiple AI providers agree to read the same way, so a site owner does not have to list every company’s crawler by name to express one broad preference.

Mueller’s post frames this as industry pressure finally landing somewhere concrete. He wrote that the proposal “reflects feedback from publishers, civil society groups, regulators and other stakeholders that have called for industry-wide controls on AI training,” and he tied it to the EU’s Code of Practice for General-Purpose AI Model Providers, which already treats robots.txt as the standard place for publishers to state machine-readable preferences.

Separate reporting from PPC Land confirms that Microsoft, OpenAI, and Anthropic made matching announcements on the same day, each describing how the shared signal fits into their own existing controls. That four-company alignment, on the same date, is the part that makes this worth paying attention to now rather than waiting for 2027.

How the AI Training Opt-Out Token Actually Works in robots.txt

The mechanism is simpler than it sounds, and it does not invent new robots.txt syntax. Instead of a new directive, it reuses the existing User-agent, Allow, and Disallow lines you already know, pointed at a special shared name rather than a specific crawler.

Sites write a group in robots.txt where the User-agent line reads __usage_ai-training instead of naming GPTBot or Google-Extended. Standard Allow and Disallow path rules apply inside that group, following the same longest-match logic defined in RFC 9309, the formal robots.txt standard. Disallowing the whole site while allowing one folder, such as a press kit, works exactly like it does for any other crawler group.

A few details matter if you plan to use it once it is live:

• The token is matched without regard to case, so capitalization in the User-agent line will not break it.
• A generic User-agent: * group does not count as using this signal. You have to name the token specifically.
• If no group in your robots.txt names the token at all, you have expressed no preference, and nothing changes by default.
• The signal covers AI training use, not crawling access and not retrieval at answer time. It is a data-use preference layered on top of an access-control file, similar to how Google-Extended already works for Google specifically.
• Private licensing agreements between a publisher and an AI company can still override the signal.

The underlying document is draft-kelley-aipref-training-preference-signal, an individual Internet-Draft written by Anthropic’s Kevin Kelley. It has no formal IETF endorsement yet and is set to expire on May 3, 2027 unless the working group revises and extends it. That status matters: this is a draft four major companies have agreed to honor, not a ratified web standard.

Google, Microsoft, OpenAI, and Anthropic: Where Each One Stands

The four companies are not moving at the same speed, and they are not all treating the token identically. The table below summarizes what each one committed to on October 30, 2026, based on their own statements and PPC Land’s reporting.

Microsoft’s version is the most detailed of the four so far. Its AI Training control can be set through a meta tag or an HTTP header, which overrides robots.txt entirely, and its AI Grounding control adds a DisallowExtended option that limits grounding to Microsoft’s own products and excludes third-party grounding services. Sites currently using NoArchive or NoCache as a stand-in for AI preferences get a window, opening December 15, 2026, to move those settings into Bing Webmaster Tools before Microsoft stops reading the old signals on March 1, 2027.

How This Differs From Google-Extended and Your Existing AI Crawler Rules

It helps to be precise about what this signal replaces and what it does not. DevByteDaily’s recent breakdown of robots.txt rules for AI crawlers covered how Google’s own parser handles named bots like GPTBot and ClaudeBot, and why there has never been a single rule that blocks “all AI crawlers” at once. That remains true. The shared AI training opt-out signal does not block crawling by any bot. It is a preference about what happens to content after a crawler, which already chose to fetch it, has it.

That distinction is the same one DevByteDaily flagged in our look at the AI crawler blocking myth: robots.txt is an access-preference protocol, not a data-use agreement, with Google-Extended as the one existing exception that actually changes how collected content gets used. The new shared token extends that same idea, data use rather than access, to more companies at once, but it still does not stop a crawler from fetching your pages in the first place. Blocking GPTBot or ClaudeBot from crawling still requires naming those tokens individually, exactly like before.

My take: treating this as a replacement for your existing crawler-specific rules would be a mistake. The shared token is additive. It gives you one broad lever for AI training preferences across participating companies, while your named-crawler rules keep doing the narrower job of controlling who fetches your pages at all. Site owners who run both will end up with the clearest, most durable setup once this actually ships.

Why Four Competitors Agreed to One Signal Now

Publishers have tried piecemeal fixes to the same problem for a couple of years. Cloudflare built Content Signals as a network-level option. A separate community effort pushed llms.txt as a plain-language file for AI companies to read. Neither reached the point where a handful of major AI providers agreed to honor it the same way, and Google has previously confirmed that tools like these do not change what an AI company actually does with content it has already crawled.

Regulation is a bigger part of the story than any single company’s product roadmap. Mueller’s post ties the new signal directly to the EU’s Code of Practice for General-Purpose AI Model Providers, a framework under the EU AI Act that already names robots.txt as the standard place for publishers to state machine-readable preferences. A shared, cross-industry token gives AI companies operating in the EU a cleaner way to show they are honoring publisher preferences at scale, rather than defending a patchwork of one-off tokens.

That regulatory pressure helps explain why Google, Microsoft, OpenAI, and Anthropic coordinated a joint announcement instead of each shipping a competing format. An IETF working group draft, even an unendorsed one, is a more defensible reference point for regulators and publishers alike than four separate company blog posts describing four slightly different robots.txt tokens. It also means the draft’s wording, not any one company’s interpretation, will likely settle most of the open questions, like how the signal interacts with licensing deals or regional law, as the AIPREF group keeps meeting.

What to Check in Your robots.txt Before Early 2027

None of this requires an edit today. Google, Microsoft, Anthropic, and OpenAI have not shipped a live implementation yet, and none of them recommend adding the token before their own rollout starts. A few things are still worth doing now, while the standard is still a draft.

1. Read your current robots.txt and note every AI-related rule you already have, including Google-Extended, GPTBot, ClaudeBot, and any others, so you know your starting point before a shared token gets layered on top.
2. Decide, in plain terms, what your actual AI training preference is: opt out everywhere, allow everywhere, or allow specific companies like Google while opting out of the rest. Writing this down now saves a scramble once the token is live.
3. If you run WordPress through Yoast, Rank Math, or a similar SEO plugin, check whether your robots.txt is generated dynamically or edited as a static file, since that affects how quickly you can add a new token once providers confirm support.
4. Bookmark the Google Search Central announcement and Microsoft’s Bing Webmaster Tools changelog, since both companies say they will post implementation updates rather than flip a switch silently.
5. If your site uses NoArchive or NoCache as an informal AI opt-out signal for Bing, plan to move that preference into Bing Webmaster Tools between December 15, 2026 and March 1, 2027, before Microsoft stops reading those tags for that purpose.
6. Treat any third-party plugin or tool that claims to “support” this signal today with caution. With a draft that expires in May 2027 and no live reader among the four companies yet, an early claim of full support is getting ahead of the standard itself.

Agencies managing several client robots.txt files have a reasonable argument for building a short internal checklist now, even without a live deadline. The four companies moved in lockstep on the announcement date, which suggests the actual rollout windows in 2027 could compress just as tightly once one provider ships first.

Frequently Asked Questions

What is the AI training opt-out signal?
It is a shared robots.txt token, written as __usage_ai-training, that lets a website express one AI training preference that multiple AI companies have agreed to read the same way, instead of writing separate rules for each provider.

Does the AI training opt-out signal replace Google-Extended, GPTBot, or ClaudeBot rules?
No. It is additive. Named crawler controls still govern whether a bot can fetch your pages at all. The shared signal only covers what happens to content after a participating company’s crawler already has it.

Will this affect my Google Search rankings?
No. Google’s announcement states explicitly that search rankings are unaffected by how a site sets its AI training preference through this signal.

Do I need to update my robots.txt file right now?
No. None of the four companies has a live implementation yet. Google targets 2027, Microsoft begins honoring the signal on February 1, 2027, and OpenAI and Anthropic have not set fixed dates.

Where can I read the actual technical specification?
The mechanism comes from draft-kelley-aipref-training-preference-signal, an individual Internet-Draft submitted to the IETF’s AIPREF working group. It is not yet a ratified standard.

Where the AI Training Opt-Out Signal Stands Today

Four companies that rarely coordinate publicly agreed, on the same day, to read one new robots.txt token the same way. That alone is a bigger signal than the token’s technical details. Google, Microsoft, OpenAI, and Anthropic each have their own reasons to want a simpler, industry-wide preference mechanism instead of fielding separate crawler lists from every publisher who asks.

For now, the practical move is to understand the mechanism, note your own AI training preference, and watch for the implementation updates each company says it will publish. Early 2027 is the earliest any of this goes live, and the draft itself is not guaranteed to survive to that date unchanged. Nothing about your current robots.txt file needs to change today, but the next few months of IETF working group activity are worth following if AI training use is something your site cares about controlling.