GPTBot
OpenAI's crawler for web content that may be used to train its generative AI foundation models. Its robots.txt setting is independent of the one that governs ChatGPT search.
| Operator | OpenAI |
|---|---|
| Purpose | Training — OpenAI foundation models (training) |
| robots.txt token | GPTBot |
| User-Agent | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot |
| Obeys robots.txt | Yes, according to the publisher |
| Official IP ranges | https://openai.com/gptbot.json — 18 ranges, list dated 2026-09-22 |
| Reverse DNS | Not documented |
| Official documentation | https://developers.openai.com/api/docs/bots |
Live activity of GPTBot
No visit from GPTBot recorded on iabig.com over the last 30 days.
Last updated: 2026-10-25 12:07:01 UTC. See the Observatory
What GPTBot does
GPTBot crawls the web to gather content that may be used to train OpenAI's generative AI foundation models. OpenAI says the goal is to make those models more useful and safe.
What OpenAI states:
- Disallowing
GPTBotin robots.txt signals that a site's content should not be used to train these models. - The
GPTBotsetting is independent ofOAI-SearchBot, so a site can opt out of training and still appear in ChatGPT search. - When both bots are allowed, OpenAI may use the results of a single crawl for both purposes to avoid duplicate crawling. That is why one of the two may show up less often in your logs.
- When fetching robots.txt, OpenAI may add a
robots.txtmarker to the user-agent string so those requests are easy to tell apart.
The published string includes a version number (GPTBot/1.4) that may change, so match on the GPTBot token rather than the full string.
OpenAI publishes its IP ranges as a JSON file in the usual prefixes format and regenerates it regularly. No reverse DNS hostname is documented, so checking the source address against that file is the only supported way to authenticate a request.
The documentation does not say what happens to content collected before an opt-out rule was added.
Block or allow GPTBot
Your pages should no longer be used to train OpenAI's foundation models. According to OpenAI this does not affect ChatGPT search, which is governed by OAI-SearchBot.
To block it, add this to your robots.txt:
User-agent: GPTBot
Disallow: /To allow it explicitly (copy any private rules you have):
User-agent: GPTBot
Allow: /A group naming GPTBot replaces the User-agent: * group for it. Build a complete robots.txt with the generator and check it with the tester.
Verify a GPTBot visit
Anyone can write “GPTBot” in a User-Agent. Paste the IP address and User-Agent from your logs into the IP verifier to know whether the visit is genuine.
Other OpenAI crawlers
- OAI-SearchBot — Search
- ChatGPT-User — Browsing