AI crawler observatory
Which AI crawlers actually browse the web, how often, and how websites respond to them.
Period: last 30 days. Last updated: 2026-10-25 12:07:01 UTC. Updated hourly.
AI crawler visits to iabig.com
Every visit is checked against the IP ranges published by the crawler's operator. A spoofed visit comes from a program pretending to be that crawler; unverifiable means the operator publishes no IP ranges.
3974 visits from identified crawlers, 2 of them spoofed.
| Crawler | Visits | Verified | Spoofed | Unverifiable | Last seen | |
|---|---|---|---|---|---|---|
| Googlebot Google · Search | 3866 | 3866 | 3864 | 2 | 0 | 2026-10-25 |
| ClaudeBot Anthropic · Training | 100 | 100 | 100 | 0 | 0 | 2026-10-25 |
| Bingbot Microsoft · Search | 8 | 8 | 8 | 0 | 0 | 2026-10-25 |
Visits per day (excluding spoofed)
What scanned websites reveal
2 website(s) checked so far. Statistics appear from 10 websites so that they are meaningful. Check yours.
Method
Crawlers are identified by User-Agent, then verified against the official IP ranges published by OpenAI, Anthropic, Perplexity, Google, Microsoft, Apple, Mistral AI, Common Crawl and DuckDuckGo (refreshed daily). Site statistics use the latest check of each domain; no visitor IP address is kept. Raw data as JSON: /api/v1/stats. Detailed profiles: all AI crawlers.