The Short Answer

If someone adds a 'block AI' line to your website this month, the line that is coming does one job: it tells AI companies not to train their models on your pages. It does not decide whether ChatGPT or Google names your business. On 30 October 2026 Google and OpenAI each published a page saying they will honour one shared robots.txt signal for AI training preferences, based on work at the Internet Engineering Task Force. Google says it aims to support it in 2027; OpenAI says in the coming months. OpenAI's documentation keeps its search crawler separate from its training crawler, and Google's says its training control does not affect inclusion in Google Search. One catch: on Google's reading, the shared opt-out also closes the door to Gemini's answers unless you name Google-Extended yourself.

Short on time? Ask an AI to summarise this post:

On 30 October 2026 Google and OpenAI each published a page saying the same thing. Both will honour one shared line in a website's robots.txt file that says, in effect, do not use this site to train your AI models. Today the line is a plan. Google gives 2027 as its date, OpenAI says the coming months, and neither has published the final wording a site should use.

So why write about it now? Because the people who manage Nigerian business websites will start adding it before then. Security plugins already offer a "block AI" button. Developers will be asked by a client whether to press it. And the owner who is asked will have one question in mind: if I block AI, do I disappear from ChatGPT? This post answers that from both companies' own pages, with the dates we read them, and ends with what to check on your own site this week.

What Google and OpenAI announced on 30 October 2026

Google's post, written by John Mueller and published on the Search Central blog on 30 October 2026, opens: "Today, we're announcing our plans to support a shared AI training control in robots.txt." It says the control "will help websites express their AI training preferences across multiple AI providers in a single step" and is "based on work from the AIPREF working group at the Internet Engineering Task Force", the body that writes the technical standards the web runs on.

An open laptop and a black mug of coffee on a pale wooden desk in morning light, the ten minutes it takes to read a robots.txt file
Photo by Engin_Akyurt on Pixabay

The mechanism, in Google's words: "Instead of managing separate instructions for every AI developer, website owners will be able to use a shared 'policy token' within robots.txt to set broad preferences across participating AI providers." Google's crawlers page, last updated the same day, gives the token it intends to read, "user-agent: __usage_ai-training", and the date, "early 2027".

OpenAI's page of 30 October 2026 says: "OpenAI will honor a common signal in robots.txt, the file that lets websites express their preferences about the use of their content for foundational generative AI training." It describes the problem the signal solves: "Previously, website operators would identify individual crawlers and maintain separate rules for each and every provider." And it gives its own timing: "We aim to implement the signal in the coming months, and will share another update with details about our implementation when we do so."

That is the whole announcement. One line, read by several companies, about training. Nothing in either page changes how a search result or an AI answer is assembled.

What the switch does not switch off

The word in both announcements is training. Training is the process of building the model. It is separate from the moment a customer asks ChatGPT for a tailor in Surulere and ChatGPT goes to the web to find one. Both companies document that separation.

OpenAI's crawler page, read on 3 November 2026, lists four crawlers. OAI-SearchBot "surfaces websites in ChatGPT's search features". GPTBot crawls content "that may be used to train OpenAI's generative AI foundation models". ChatGPT-User visits a page when a user's question requires it. OAI-AdsBot checks ad landing pages. Each has its own robots.txt rule. Blocking GPTBot says nothing to OAI-SearchBot, and OpenAI's page says a site that blocks OAI-SearchBot will not be shown in ChatGPT search results. OpenAI's 30 October page adds: "This new signal will not replace the controls we already provide through robots.txt. Our existing crawler controls, which provide web publishers flexibility and control over their pages, will continue to apply and be respected."

Google has had a training control since 2023, named Google-Extended. Its crawlers page says: "Google-Extended does not impact a site's inclusion in Google Search nor is it used as a ranking signal in Google Search." Google's AI features page says a page appears in AI Overviews and AI Mode when it is indexed and eligible for a snippet, and it names no training control as a condition.

So on the pages as they read today, a business that opts out of training with the new line stays in Google Search, stays in Google's AI features, and stays eligible for ChatGPT search answers, provided the search crawlers themselves are still allowed in. That last clause is where most of the damage happens, and it is covered in the steps below.

The one catch: Gemini

Google's post contains a sentence that an owner should read twice. "When a website uses this policy token to opt out of training in its robots.txt file (and Google-Extended is not explicitly specified), Google plans to interpret that as if the publisher had disallowed Google-Extended."

Google-Extended does more than training. Google's crawlers page says it manages whether crawled content may be used for training future Gemini models "and for grounding", which it defines as "providing content from the Google Search index to the model at prompt time to improve factuality and relevancy". Grounding is the moment a Gemini answer reaches into the search index and pulls your page in. Google's post says it plainly: "Websites that specifically allow Google-Extended are eligible to appear in responses and receive clicks via grounding in Gemini Apps."

Put the two together. An owner who adds the shared opt-out line and says nothing about Google-Extended has, on Google's own reading, also told Gemini's apps not to pull their pages into answers. An owner who wants to opt out of training and stay in Gemini's answers has to write both lines: the shared token to opt out, and an explicit allow for Google-Extended. Google's post says the company-specific controls "will continue to function alongside this new joint signal", so the two lines can live in the same file.

Whether Gemini's answers matter to your customers is your call. Our view is that a Lagos business should not close that door by accident.

Should a Nigerian business block AI training at all?

We cannot answer that for you, and we are suspicious of anyone who can. Nobody publishes a figure for whether allowing training raises a business's chance of being named in an answer. The honest position is that training and being named are separate on the documentation, and the decision about training is a decision about your content, not about your visibility.

For a news publisher or a company whose product is its writing, opting out of training is a serious commercial question. For a tailor, a clinic or a logistics firm whose website is a service page, an address and a price list, the content being protected is the content you want every machine to repeat. Our own robots.txt, last updated 11 June 2026, allows every crawler it names, including Google-Extended and GPTBot. That is our decision for an agency that wants to be found and cited. It is not advice.

What we do advise is that the decision be made on purpose and written down, so that the next developer or the next plugin update does not make it for you.

What is not live yet

Three things from the pages, so that nobody sells you an urgent fix:

  • Google says it aims to support the control "in 2027", and its crawlers page says "early 2027".
  • OpenAI says "in the coming months" and promises an update with implementation details.
  • The exact line a site should write is still being settled. Google's crawlers page names the token it intends to read. OpenAI has not yet published what it will read.

A line copied from a vendor's blog post or an agency's checklist in November 2026 is a guess. Wait for the two companies' own documentation, which is what we will quote when it changes.

What to do this week

Five steps. None of them needs a developer for more than ten minutes, and the first two need nobody at all.

  1. Read your own robots.txt. Type yourdomain.com/robots.txt into a browser. You will see either a short text file of allow and disallow lines, a blank page, or a not-found error. A blank page or an error means no rules, which means every crawler is allowed, which is fine for most small businesses.
  2. Look for the names that matter. The crawlers that put you in answers are Googlebot, OAI-SearchBot, ChatGPT-User and PerplexityBot. The training names are GPTBot and Google-Extended. You will see each name after the words "User-agent:" with "Allow" or "Disallow" under it.
  3. Make the training decision on paper. Write one sentence: we allow AI training, or we do not, and why. You will see that the sentence takes two minutes and settles every future conversation with a plugin or a developer.
  4. If you opt out, name Google-Extended yourself. Ask whoever runs the site to write both the shared opt-out, once Google and OpenAI publish the final wording, and an explicit line for Google-Extended that matches your Gemini decision. You will see two groups of lines in the file rather than one, and Google's post says both are honoured.
  5. Never block the search crawlers. Whatever you decide about training, confirm that Googlebot, OAI-SearchBot and PerplexityBot are not disallowed. OpenAI's page says a robots.txt change takes about 24 hours to reach its systems, and Perplexity's says up to 24 hours. You will see, a day after any change, that the test in our six-step post still names you, or that it has stopped.

Why the file is worth ten minutes

The people asking AI engines about businesses are not a niche, and the named surveys say so:

49%
US adults who use AI chatbots, up from 33% in 2024
Pew Research Center, Americans and AI 2026
45%
US consumers who used AI for local business recommendations in the past year, up from 6%
BrightLocal, Local Consumer Review Survey 2026

A robots.txt file is the cheapest place to lose those customers and the cheapest place to keep them. One disallow line aimed at the wrong crawler removes a business from the answers, and nothing on the website tells the owner it happened.

The honest limits of this post

Everything above rests on four pages as they read between 31 October and 3 November 2026: Google's blog post and crawlers page, OpenAI's announcement and crawler page. Both companies say the control is coming rather than live, and OpenAI has not said how it will map the shared signal onto its own crawlers. When either company publishes its implementation, the wording in this post may be out of date on that point, and we will say so in a dated update rather than edit this one quietly. What we do not expect to change is the separation both companies document today: the crawler that trains the model and the crawler that finds your business for a customer are different crawlers, with different switches.

Want to know whether the machines can read your site at all?

Our free AI Visibility checklist covers the 12 checks that decide whether ChatGPT, Gemini and Google can find, read and name your business, starting with the crawler lines in robots.txt. Enter your email and we will send it over.

✓ Check your inbox, your checklist is on the way!

We respect your privacy. Unsubscribe at any time.

Key Takeaway

The shared switch is for training, and the search crawlers that put your name in an answer are different switches. Read your robots.txt this week, know which lines are in it, and decide the training question as a business decision rather than letting a plugin decide it for you.

One forward before you go: open yourdomain.com/robots.txt in a browser, screenshot it, and send it to whoever runs your website with one question: which of these lines blocks a crawler that puts us in search or AI answers? If the file is empty or does not load, that is also an answer, and a cheaper one to fix.

Frequently asked questions

If I block AI training on my website, will ChatGPT stop recommending my business?

Not on OpenAI's documentation as it read on 3 November 2026. OpenAI's crawler page lists GPTBot, which gathers content that may be used to train its models, and OAI-SearchBot, which surfaces sites in ChatGPT's search features, as separate crawlers with separate robots.txt rules. A site that blocks GPTBot and allows OAI-SearchBot remains eligible to appear in ChatGPT search answers. A site that blocks OAI-SearchBot does not, and that is the line to protect. OpenAI's 30 October 2026 page says the new shared signal will not replace its existing crawler controls.

Will the 'no AI training' switch remove me from Google or from AI Overviews?

Google's common crawlers page, last updated 30 October 2026, says Google-Extended does not impact a site's inclusion in Google Search and is not used as a ranking signal. Google's AI features page says a page appears in AI Overviews and AI Mode when it is indexed and eligible for a snippet, and names no training control as a condition. So the training switch and Google Search are separate on Google's own pages. The one place it does reach is Gemini's apps, covered in the next answer.

What is Google-Extended, and why does it matter here?

Google-Extended is a name you can address in robots.txt. Google's crawlers page says it manages whether content Google crawls may be used for training future Gemini models and for grounding, which is Google's word for pulling pages from its search index into a Gemini answer at the moment someone asks. Google's 30 October 2026 post says that if a site uses the new shared token to opt out of training and does not mention Google-Extended, Google plans to treat that as disallowing Google-Extended, and that sites which specifically allow Google-Extended are eligible to appear in responses in Gemini Apps. So the shared switch, used alone, also switches off Gemini's answers on Google's reading.

What is robots.txt, in plain words?

It is a small text file at yourdomain.com/robots.txt that tells crawlers which parts of your site they may read. Each rule names a crawler and says allow or disallow. Google documents its syntax and the names of its crawlers; OpenAI and Perplexity document theirs. Anyone can read yours by typing the address into a browser, which is why the first step this week is to look at it.

My website plugin has a 'block AI bots' button. Should I press it?

Not without reading what it writes. Some of those buttons block training crawlers only, which is a business decision you can make either way. Others block every crawler whose name contains 'GPT' or 'AI', including OAI-SearchBot and ChatGPT-User, and on OpenAI's documentation that removes you from ChatGPT search answers. Press it, then open robots.txt and read the lines it added. If OAI-SearchBot, Googlebot or PerplexityBot is disallowed, undo it.

When does the shared switch start working?

Google's 30 October 2026 post says it aims to support the control along with other AI providers in 2027, and its crawlers page says early 2027 with the token user-agent: __usage_ai-training. OpenAI's page of the same date says it aims to implement the signal in the coming months and will publish details when it does. Until then, the lines that work are the company-specific ones both companies already document, and a line copied from a vendor's blog post is not one of them.

Is this the same as SEO?

It overlaps with it. Managing robots.txt has been part of SEO since the file existed, and a developer who says so is right. The new part is that the same file now carries a training preference that Google and OpenAI will read, and that one of those lines, used alone, can affect whether Gemini pulls your page into an answer. SEO tools report rankings; nothing in them reports whether an AI answer names you. GEO and AEO are the trade names some agencies use for that second measurement.

References & sources

  1. Google Search Central blog, Supporting a shared AI training control for websites (John Mueller, 30 October 2026) (read 31 October and 3 November 2026)
  2. OpenAI, Making it easier for websites to express AI training preferences (30 October 2026) (read 31 October 2026)
  3. Google Search Central, Google's common crawlers (Google-Extended entry and AIPREF note) (last updated 30 October 2026; read 1 and 3 November 2026)
  4. OpenAI, OpenAI crawlers documentation (OAI-SearchBot, OAI-AdsBot, GPTBot, ChatGPT-User) (read 3 November 2026)
  5. Google Search Central, AI features and your website (eligibility: indexed and eligible for a snippet)
  6. Google Search Central, Introduction to robots.txt
  7. Perplexity, Perplexity crawlers (PerplexityBot, Perplexity-User) (read 3 November 2026)
  8. Pew Research Center, Americans and AI 2026 (5,119 US adults, February 2026)
  9. BrightLocal, Local Consumer Review Survey 2026 (1,002 US adults)
A
Abdulmuiz Adebowale
AI Product Manager & Builder, Nexum-sol

Abdulmuiz helps Nigerian businesses build websites and digital systems that get found, by people and by AI. He writes about practical SEO, AI search, and growth systems for SMEs at Nexum-sol.