Block or Allow AI Crawlers on Wix - The robots.txt Editor and Its Limits
Edit robots.txt on Wix to block GPTBot or ClaudeBot without blocking ChatGPT Search, Claude and Perplexity, where to find the editor and the pitfalls to avoid.
Wix generates every site's robots.txt file automatically and updates it when you hide a page or change your SEO settings. An editor in the dashboard lets you add your own rules, which is enough to separate training crawlers from search crawlers. As far as we know, Wix offers no dedicated setting that sorts AI crawlers by purpose: everything goes through this editor. This guide shows where to find it, gives you a ready-to-paste file, and flags an important point: the example Wix itself publishes also blocks agents that are used to cite you.
What Wix puts in robots.txt by default
According to the Help Center, the default file allows access to every page, except those crawlers cannot reach: password-protected pages, members-only pages, and pages excluded from indexing in the Editor or in the SEO settings. It also contains the sitemap URL. Wix says it updates the file automatically after a modified page is published, and immediately after a site setting changes.
On the Wix sites we looked at in October and November 2026, the default file looked like this (this content is not documented and may change):
User-agent: *
Allow: /
Disallow: *?lightbox=
# Optimization for Google Ads Bot
User-agent: AdsBot-Google-Mobile
User-agent: AdsBot-Google
Disallow: /_partials*
Disallow: /pro-gallery-webapp/v1/galleries/*
# Block PetalBot
User-agent: PetalBot
Disallow: /
# Crawl delay for overly enthusiastic bots
User-agent: dotbot
Crawl-delay: 10
User-agent: AhrefsBot
Crawl-delay: 10
Sitemap: https://www.example.com/sitemap.xml
No AI crawler is named: they all follow the * group and can crawl the site.
Where to edit the file
In your site's dashboard:
- Open SEO & GEO.
- Scroll down to Tools and settings.
- Click Robots.txt Editor.
- Click View File.
- Enter your directives under This is your current file.
- Click Save Changes, then Save.
Depending on the language and version of your dashboard, the labels may differ slightly. Wix describes editing this file as an advanced feature and says its Customer Care team cannot help with the changes you make to it.
A ready-to-paste robots.txt
Add these two groups at the top of the existing file, without deleting Wix's default groups. Remove from the first group any crawler you are willing to accept.
# 1. Model training: blocked
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: CCBot
User-agent: meta-externalagent
User-agent: MistralAI-Training
User-agent: Applebot-Extended
User-agent: Google-Extended
Disallow: /
# 2. Search and on-demand retrieval: allowed
User-agent: OAI-SearchBot
User-agent: ChatGPT-User
User-agent: Claude-SearchBot
User-agent: Claude-User
User-agent: PerplexityBot
User-agent: Perplexity-User
Allow: /
Disallow: *?lightbox=
Group 2 repeats the Disallow rule from the * group in your file, because a crawler that finds a group with its own name follows only that group. If your * group contains other lines, copy them too. As for Google-Extended, remember that this token also covers grounding of answers in the Gemini apps: block it only if you accept no longer being cited there.
Wix's own example goes further than needed
Wix's help article on blocking AI crawlers suggests pasting this block:
User-agent: CCBot
Disallow: /
User-agent: GPTBot
Disallow: /
User-agent: ChatGPT-User
Disallow: /
User-agent: BingAI
Disallow: /
Two remarks before you use it as is:
ChatGPT-Useris not a training crawler. OpenAI describes it as the agent that visits a page when a user asks a question in ChatGPT. Blocking it reduces your chances of being consulted and cited, without changing anything about training, which is handled byGPTBot. OpenAI also states thatrobots.txtrules may not apply to this agent.BingAIdoes not appear in Bing's official list of crawlers. Microsoft's page describing Bing's crawlers does not mention this name. A line that matches no real crawler blocks nothing.
The same article notes that this blocking can limit NLWeb, the Wix feature that exposes your site's structured content to AI agents: if you use it, don't block the crawlers involved.
Wix-specific limits and pitfalls
- No per-purpose setting. Wix provides no switch that separates training from search: it is up to you to write the groups, and to keep them current as AI companies publish new tokens.
- Sites without a custom domain. A crawler only reads
robots.txtat the root of the hostname. For a site published at an address likeusername.wixsite.com/my-site, the file that is read isusername.wixsite.com/robots.txt. On a freewixsite.comsite we checked in November 2026, that file existed, had the same default groups as above, noSitemapline, and ended with the comment# Auto generated, go to SEO Tools > Robots.txt Editor to change this. Wix's documentation doesn't say whether edits made on a free site apply to that shared subdomain file, so check the served file after saving rather than assuming. - Mistakes are costly. Wix warns that changing this file can stop your site from appearing in search results. A
Disallow: /accidentally pasted underUser-agent: *blocks every crawler. - Resetting erases everything. The Reset to Default button in the same editor restores the original file and therefore deletes your groups. Keep a copy of your version.
- Hiding a page is not a job for robots.txt. To exclude a page from search results, Wix recommends the page settings (excluding it from indexing, the meta robots tag), which update the file themselves. Separately, Wix's AI-crawler article points to the
nosnippetcheckbox under SEO basics > Advanced SEO > Robots meta tag in each page's settings, which it says stops the page's content from being used as direct input for AI overviews; it also removes text snippets from ordinary search results, so use it page by page, not site-wide.
Check the result
After saving, open https://www.example.com/robots.txt: your two groups should appear above Wix's groups, each exactly once. Wix points out that search engines update their copy of the file on their next visit; RFC 9309 asks crawlers not to cache it for more than 24 hours, so allow a day. Wix itself reminds you that some crawlers ignore robots.txt and that this is "not an airtight solution". Then check a few URLs for GPTBot, OAI-SearchBot and PerplexityBot with the IABIG robots.txt tester: the home page should be disallowed for the first and allowed for the other two.
Official sources
- Wix, Editing Your Site's Robots.txt File: https://support.wix.com/en/article/editing-your-sites-robotstxt-file
- Wix, Blocking AI Crawlers from Your Site: https://support.wix.com/en/article/blocking-ai-crawlers-from-your-site
- Wix, Enabling NLWeb for AI Agent Access: https://support.wix.com/en/article/enabling-nlweb-for-ai-agent-access
- OpenAI, Overview of OpenAI Crawlers: https://developers.openai.com/api/docs/bots
- Microsoft, Overview of Bing crawlers: https://www.bing.com/webmasters/help/which-crawlers-does-bing-use-8c184ec0
- RFC 9309, Robots Exclusion Protocol: https://www.rfc-editor.org/rfc/rfc9309
Free tools
Free AI visibility checker
Check for free whether ChatGPT, Claude, Perplexity and Gemini can read and cite your website - robots.txt, llms.txt, JavaScript and hidden text.
AI robots.txt generator
Build a robots.txt in one click that blocks AI training while keeping your site visible in ChatGPT Search, Claude and Perplexity. Free.
AI crawler blocklists
Ready-to-use files, refreshed hourly, to block AI training (or all AI crawlers): robots.txt, nginx, Apache, Cloudflare and the official IP ranges published by AI companies.
robots.txt tester for AI crawlers
Paste your robots.txt and instantly see, crawler by crawler, whether GPTBot, ClaudeBot, PerplexityBot or Google-Extended can access a page.
Related guides
- How to Block AI Crawlers on IIS - robots.txt, URL Rewrite, Request Filtering and IP RestrictionsOn IIS, publish robots.txt, deny AI training crawlers with a URL Rewrite rule in web.config and learn the limits of request filtering and IP restrictions.
- How to Block AI Crawlers in nginx - User-Agent Rules, Official IP Ranges and Rate LimitsA tested nginx configuration that denies AI training crawlers, lets genuine AI search crawlers through, rate-limits them and writes them to a dedicated log.
- How to Block AI Crawlers in Apache 2.4 - robots.txt, .htaccess, Require not env and IP RangesDeny AI training crawlers in Apache 2.4 with .htaccess, mod_setenvif or mod_rewrite, and only accept AI search crawlers coming from their official IP ranges.
- Block or Allow AI Crawlers on WordPress - Virtual robots.txt, Physical File and PHP FilterWhere to edit robots.txt on WordPress to block GPTBot or ClaudeBot yet stay visible in ChatGPT Search, Claude and Perplexity, by file, PHP filter or plugin.