Google-Extended
A Google robots.txt token with no crawler of its own. It controls whether crawled content may be used to train Gemini models and for grounding in Gemini Apps and Vertex AI.
| Operator | |
|---|---|
| Purpose | Training — Gemini Apps and Vertex AI (training and grounding) |
| robots.txt token | Google-Extended |
| User-Agent | No crawler of its own: robots.txt token only |
| Obeys robots.txt | Yes, according to the publisher |
| Official IP ranges | Not published |
| Reverse DNS | Not documented |
| Official documentation | https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers#google-extended |
Live activity of Google-Extended
Google-Extended is not a crawler that visits sites: it is a token read from robots.txt by Google's existing crawlers, so it never shows up in logs.
Last updated: 2026-10-25 12:07:01 UTC. See the Observatory
What Google-Extended does
Google-Extended is not a crawler, and no HTTP request ever carries that name. Crawling is done with Google's existing user agents; the token is only used to state a preference in robots.txt.
According to Google, the token lets publishers decide whether content crawled from their site may be used:
- to train future generations of the Gemini models that power Gemini Apps and the Vertex AI API for Gemini;
- for grounding, meaning content from the Google Search index is provided to the model at prompt time to improve factuality and relevance, in Gemini Apps and in Grounding with Google Search on Vertex AI.
Google states that Google-Extended does not affect a site's inclusion in Google Search and is not used as a ranking signal.
The grounding part is the detail most people miss. Disallowing Google-Extended is not just a training opt-out; it also removes your content from grounded answers in Gemini Apps and Vertex AI. Based on the token's description, you cannot use it alone to stay available as a Gemini source while refusing training.
Since no crawler uses this name, nothing shows up under Google-Extended in server logs, and there is no IP range or reverse DNS hostname to check.
Block or allow Google-Extended
Google should stop using your content to train future Gemini models and for grounding in Gemini Apps and Vertex AI. Google says your inclusion and ranking in Google Search are unaffected.
To block it, add this to your robots.txt:
User-agent: Google-Extended
Disallow: /To allow it explicitly (copy any private rules you have):
User-agent: Google-Extended
Allow: /A group naming Google-Extended replaces the User-agent: * group for it. Build a complete robots.txt with the generator and check it with the tester.
Other Google crawlers
- Googlebot — Search