Googlebot
Google's main crawler, in smartphone and desktop variants. It feeds Google Search, including Discover and all Search features, as well as Google Images, Video mode and Google News.
| Operator | |
|---|---|
| Purpose | Search — Google Search (including Discover, Images, News) |
| robots.txt token | Googlebot |
| User-Agent | Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) |
| Obeys robots.txt | Yes, according to the publisher |
| Official IP ranges | https://developers.google.com/static/crawling/ipranges/common-crawlers.json — 317 ranges, list dated 2026-10-23 |
| Reverse DNS | googlebot.com |
| Official documentation | https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers#googlebot |
Live activity of Googlebot
Over the last 30 days, 4675 visits from Googlebot to iabig.com: 4673 IP-verified, 2 spoofed, 0 unverifiable. Last seen: 2026-10-25.
Last updated: 2026-10-25 14:07:01 UTC. See the Observatory
What Googlebot does
Googlebot is Google's main crawler. It comes in two variants, Googlebot Smartphone and Googlebot Desktop, which both obey the same Googlebot robots.txt token. Rules addressed to this token apply to Google Search (including Discover and all Search features) as well as Google Images, Video mode and Google News.
What Google states:
- Common crawlers, Googlebot included, always respect robots.txt rules for automatic crawls.
Chrome/W.X.Y.Zin the string is a placeholder for the Chrome version in use, which tracks Chromium releases, so log filters should use a wildcard instead of a fixed version.- The user-agent string can be spoofed. Verify requests with a reverse DNS lookup (hostnames under
googlebot.com, includinggeo.googlebot.com) followed by a forward lookup, or against the IP ranges incommon-crawlers.json.
On the generative AI side, whether crawled content may be used to train Gemini models and for grounding in Gemini Apps and Vertex AI is controlled by the separate Google-Extended token, which Google says has no effect on Google Search.
Googlebot is therefore the crawler to allow if you want to be in Google. Blocking it does not merely opt you out of AI uses; it stops your pages from being crawled for search.
Block or allow Googlebot
Your pages are no longer crawled for Google Search, Discover, Images and News, which severely hurts your presence in Google. To opt out of Gemini uses only, target Google-Extended instead.
To block it, add this to your robots.txt:
User-agent: Googlebot
Disallow: /To allow it explicitly (copy any private rules you have):
User-agent: Googlebot
Allow: /A group naming Googlebot replaces the User-agent: * group for it. Build a complete robots.txt with the generator and check it with the tester.
Verify a Googlebot visit
Anyone can write “Googlebot” in a User-Agent. Paste the IP address and User-Agent from your logs into the IP verifier to know whether the visit is genuine.
Other Google crawlers
- Google-Extended — Training