Robot par robot
| Robot | Nouveaux blocages | Déblocages | Restreint en partie | Restrictions levées |
|---|---|---|---|---|
| Perplexity-User Perplexity | 1 | 2 | ||
| Amazonbot Amazon | 2 | |||
| Applebot-Extended Apple | 2 | |||
| Bytespider ByteDance | 2 | |||
| CCBot Common Crawl | 1 | 1 | ||
| ChatGPT-User OpenAI | 2 | |||
| Claude-SearchBot Anthropic | 2 | |||
| Claude-User Anthropic | 2 | |||
| ClaudeBot Anthropic | 2 | |||
| GPTBot OpenAI | 2 | |||
| Google-Extended Google | 2 | |||
| Googlebot Google | 2 | |||
| OAI-SearchBot OpenAI | 2 | |||
| PerplexityBot Perplexity | 2 | |||
| meta-externalagent Meta | 2 | |||
| Applebot Apple | 1 | |||
| Bingbot Microsoft | 1 | |||
| DuckAssistBot DuckDuckGo | 1 | |||
| MistralAI-User Mistral AI | 1 | |||
| meta-externalfetcher Meta | 1 |
Tous les changements de la semaine
- déblocagepartielmeraki.com débloque GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, PerplexityBot, Perplexity-User, Google-Extended, Applebot-Extended, meta-externalagent, meta-externalfetcher, Amazonbot, MistralAI-User, Bytespider, DuckAssistBot; lève des restrictions partielles pour Googlebot, Bingbot, Applebot, CCBot.
Rang Tranco 176 · 8 novembre 2026diff du robots.txt (groupes concernant l IA)
@@ -1,75 +0,0 @@ -User-agent: 360spider -User-agent: 360spider-image -User-agent: 360spider-video -User-agent: adsbot-google -User-agent: adsbot-google-mobile -User-agent: adsbot-google-mobile-apps -User-agent: adidxbot -User-agent: applebot -User-agent: applenewsbot -User-agent: baiduspider -User-agent: baiduspider-image -User-agent: baiduspider-news -User-agent: baiduspider-video -User-agent: bingbot -User-agent: bingpreview -User-agent: bublupbot -User-agent: ccbot -User-agent: cliqzbot -User-agent: coccoc -User-agent: coccocbot-image -User-agent: coccocbot-web -User-agent: daumoa -User-agent: dazoobot -User-agent: deusu -User-agent: duckduckbot -User-agent: duckduckgo-favicons-bot -User-agent: euripbot -User-agent: exploratodo -User-agent: facebot -User-agent: feedly -User-agent: findxbot -User-agent: googlebot -User-agent: googlebot-image -User-agent: googlebot-mobile -User-agent: googlebot-news -User-agent: googlebot-video -User-agent: haosouspider -User-agent: ichiro -User-agent: istellabot …
- déblocagepartielcnbc.com débloque GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, PerplexityBot, Perplexity-User, Google-Extended, Applebot-Extended, meta-externalagent, Amazonbot, CCBot, Bytespider; lève des restrictions partielles pour Googlebot.
Rang Tranco 486 · 8 novembre 2026diff du robots.txt (groupes concernant l IA)
@@ -1,77 +0,0 @@ -User-agent: googlebot -Disallow: /*native-android-mobile -Disallow: /*native-android-tablet -Disallow: /*mobile-native -Disallow: /preview/ -Disallow: /undefined/ -Disallow: /proplayer -Disallow: /appchart/* -Disallow: /search/* -Disallow: /zephr/* -Disallow: /blaize/* - -User-agent: gptbot -User-agent: perplexitybot -User-agent: amazonbot -User-agent: anthropic-ai -User-agent: awariorssbot -User-agent: awariosmartbot -User-agent: bytespider -User-agent: ccbot -User-agent: huggingfacebot -User-agent: chatgpt-user -User-agent: claudebot -User-agent: claude-web -User-agent: dataforseobot -User-agent: diffbot -User-agent: facebookbot -User-agent: google-extended -User-agent: google-notebooklm -User-agent: google-agent -User-agent: magpie-crawler -User-agent: newsnow -User-agent: news-please -User-agent: omgili -User-agent: omgilibot -User-agent: peer39_crawler -User-agent: peer39_crawler/1.0 -User-agent: scrapy -User-agent: turnitinbot …
- llms.txttarget.com publie un llms.txt.
Rang Tranco 608 · 8 novembre 2026diff du robots.txt (groupes concernant l IA)
@@ -0,0 +1,81 @@ +User-agent: * +Disallow: /*/Ntk +Disallow: /*/Ntt +Disallow: /*/Ntx +Disallow: /*%7Cd_ +Disallow: /*/schoollist/ +Disallow: /*BTWN +Disallow: /[path]/ +Disallow: /7078046/ +Disallow: /7079046/ +Disallow: /AddToList +Disallow: /AddToRegistry +Disallow: /admin +Disallow: /advancedGiftRegistrySearchView +Disallow: /AjaxSearchNavigationView +Disallow: /Allons_voter +Disallow: /bp/c/ +Disallow: /bp/guest_mfg_brand +Disallow: /bp/p/ +Disallow: /CallToActionModalView +Disallow: /cgi-bin +Disallow: /cgi-local +Disallow: /Checkout +Disallow: /CheckoutEditItemsDisplayView +Disallow: /CheckoutOrderBillingView +Disallow: /CheckoutOrderShippingView +Disallow: /CheckoutSignInView +Disallow: /co- +Disallow: /common +Disallow: /coupons. +Disallow: /custom-reviews/ +Disallow: /data +Disallow: /database/philboard.mdb +Disallow: /deals/ +Disallow: /dir_on_server/ +Disallow: /EmailCartView +Disallow: /EnlargedImageView +Disallow: /ESPDisplayOptionsViewCmd +Disallow: /ESPModal …
- blocagedw.com bloque Perplexity-User.
Rang Tranco 1 390 · 8 novembre 2026diff du robots.txt (groupes concernant l IA)
@@ -63 +63,4 @@ Disallow: / + +User-agent: perplexity-user +Disallow: /
Tous les changements · Semaine suivante →
robots.txt et llms.txt des 2 000 premiers domaines Tranco, vérifiés deux fois par jour ; un changement figure ici après avoir été observé lors de deux vérifications consécutives. Fonctionnement.