Claude-SearchBot
Anthropic's crawler for improving the relevance and accuracy of Claude's search responses. Blocking it may reduce how visible a site is in those results.
| Operator | Anthropic |
|---|---|
| Purpose | Search — Claude web search |
| robots.txt token | Claude-SearchBot |
| User-Agent | Full string not published by the operator |
| Obeys robots.txt | Yes, according to the publisher |
| Official IP ranges | https://claude.com/crawling/bots.json — 14 ranges, list dated 2026-10-20 |
| Reverse DNS | Not documented |
| Official documentation | https://support.claude.com/en/articles/8896518 |
Live activity of Claude-SearchBot
No visit from Claude-SearchBot recorded on iabig.com over the last 30 days.
Last updated: 2026-10-25 12:07:01 UTC. See the Observatory
What Claude-SearchBot does
Claude-SearchBot navigates the web to improve the quality of search results shown to Claude users. Anthropic says it analyzes online content specifically to make search responses more relevant and accurate.
What Anthropic states:
- Disabling it prevents your content from being indexed for search optimization, which may reduce your site's visibility and accuracy in user search results.
- Like all of Anthropic's bots, it honors robots.txt directives and does not bypass CAPTCHAs.
- Rules go in the robots.txt of each subdomain concerned.
Claude-SearchBot is separate from ClaudeBot, which collects training data: a site can refuse training by blocking ClaudeBot while still letting Claude-SearchBot index it. It is also separate from Claude-User, which fetches pages at the moment a user asks a question.
Requests are authenticated against Anthropic's single IP list, which covers all three bots without telling them apart. Anthropic publishes neither a full user-agent string nor a reverse DNS hostname for this bot. In your logs, the Claude-SearchBot name is the only identifier, and the IP list is the only way to confirm the request really came from Anthropic.
Block or allow Claude-SearchBot
Anthropic can no longer index your pages for Claude's search, which may reduce their visibility and the accuracy of answers that draw on them.
To block it, add this to your robots.txt:
User-agent: Claude-SearchBot
Disallow: /To allow it explicitly (copy any private rules you have):
User-agent: Claude-SearchBot
Allow: /A group naming Claude-SearchBot replaces the User-agent: * group for it. Build a complete robots.txt with the generator and check it with the tester.
Verify a Claude-SearchBot visit
Anyone can write “Claude-SearchBot” in a User-Agent. Paste the IP address and User-Agent from your logs into the IP verifier to know whether the visit is genuine.
Other Anthropic crawlers
- ClaudeBot — Training
- Claude-User — Browsing