# Block or Allow AI Crawlers on Wix - The robots.txt Editor and Its Limits

> Edit robots.txt on Wix to block GPTBot or ClaudeBot without blocking ChatGPT Search, Claude and Perplexity, where to find the editor and the pitfalls to avoid.

Source: https://iabig.com/guides/platforms/wix-ai-crawlers/ — Guides CC BY 4.0, extraits de code MIT

Wix generates every site's `robots.txt` file automatically and updates it when you hide a page or change your SEO settings. An editor in the dashboard lets you add your own rules, which is enough to separate training crawlers from search crawlers. As far as we know, Wix offers no dedicated setting that sorts AI crawlers by purpose: everything goes through this editor. This guide shows where to find it, gives you a ready-to-paste file, and flags an important point: the example Wix itself publishes also blocks agents that are used to cite you.

## What Wix puts in robots.txt by default

According to the Help Center, the default file allows access to every page, except those crawlers cannot reach: password-protected pages, members-only pages, and pages excluded from indexing in the Editor or in the SEO settings. It also contains the sitemap URL. Wix says it updates the file automatically after a modified page is published, and immediately after a site setting changes.

On the Wix sites we looked at in October and November 2026, the default file looked like this (this content is not documented and may change):

```text
User-agent: *
Allow: /
Disallow: *?lightbox=

# Optimization for Google Ads Bot
User-agent: AdsBot-Google-Mobile
User-agent: AdsBot-Google
Disallow: /_partials*
Disallow: /pro-gallery-webapp/v1/galleries/*

# Block PetalBot
User-agent: PetalBot
Disallow: /

# Crawl delay for overly enthusiastic bots
User-agent: dotbot
Crawl-delay: 10
User-agent: AhrefsBot
Crawl-delay: 10

Sitemap: https://www.example.com/sitemap.xml
```

No AI crawler is named: they all follow the `*` group and can crawl the site.

## Where to edit the file

In your site's dashboard:

1. Open **SEO & GEO**.
2. Scroll down to **Tools and settings**.
3. Click **Robots.txt Editor**.
4. Click **View File**.
5. Enter your directives under **This is your current file**.
6. Click **Save Changes**, then **Save**.

Depending on the language and version of your dashboard, the labels may differ slightly. Wix describes editing this file as an advanced feature and says its Customer Care team cannot help with the changes you make to it.

## A ready-to-paste robots.txt

Add these two groups at the top of the existing file, without deleting Wix's default groups. Remove from the first group any crawler you are willing to accept.

```text
# 1. Model training: blocked
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: CCBot
User-agent: meta-externalagent
User-agent: MistralAI-Training
User-agent: Applebot-Extended
User-agent: Google-Extended
Disallow: /

# 2. Search and on-demand retrieval: allowed
User-agent: OAI-SearchBot
User-agent: ChatGPT-User
User-agent: Claude-SearchBot
User-agent: Claude-User
User-agent: PerplexityBot
User-agent: Perplexity-User
Allow: /
Disallow: *?lightbox=
```

Group 2 repeats the `Disallow` rule from the `*` group in your file, because a crawler that finds a group with its own name follows only that group. If your `*` group contains other lines, copy them too. As for `Google-Extended`, remember that this token also covers grounding of answers in the Gemini apps: block it only if you accept no longer being cited there.

## Wix's own example goes further than needed

Wix's help article on blocking AI crawlers suggests pasting this block:

```text
User-agent: CCBot
Disallow: /
User-agent: GPTBot
Disallow: /
User-agent: ChatGPT-User
Disallow: /
User-agent: BingAI
Disallow: /
```

Two remarks before you use it as is:

- **`ChatGPT-User` is not a training crawler.** OpenAI describes it as the agent that visits a page when a user asks a question in ChatGPT. Blocking it reduces your chances of being consulted and cited, without changing anything about training, which is handled by `GPTBot`. OpenAI also states that `robots.txt` rules may not apply to this agent.
- **`BingAI` does not appear in Bing's official list of crawlers.** Microsoft's page describing Bing's crawlers does not mention this name. A line that matches no real crawler blocks nothing.

The same article notes that this blocking can limit NLWeb, the Wix feature that exposes your site's structured content to AI agents: if you use it, don't block the crawlers involved.

## Wix-specific limits and pitfalls

- **No per-purpose setting.** Wix provides no switch that separates training from search: it is up to you to write the groups, and to keep them current as AI companies publish new tokens.
- **Sites without a custom domain.** A crawler only reads `robots.txt` at the root of the hostname. For a site published at an address like `username.wixsite.com/my-site`, the file that is read is `username.wixsite.com/robots.txt`. On a free `wixsite.com` site we checked in November 2026, that file existed, had the same default groups as above, no `Sitemap` line, and ended with the comment `# Auto generated, go to SEO Tools > Robots.txt Editor to change this`. Wix's documentation doesn't say whether edits made on a free site apply to that shared subdomain file, so check the served file after saving rather than assuming.
- **Mistakes are costly.** Wix warns that changing this file can stop your site from appearing in search results. A `Disallow: /` accidentally pasted under `User-agent: *` blocks every crawler.
- **Resetting erases everything.** The **Reset to Default** button in the same editor restores the original file and therefore deletes your groups. Keep a copy of your version.
- **Hiding a page is not a job for robots.txt.** To exclude a page from search results, Wix recommends the page settings (excluding it from indexing, the meta robots tag), which update the file themselves. Separately, Wix's AI-crawler article points to the `nosnippet` checkbox under **SEO basics > Advanced SEO > Robots meta tag** in each page's settings, which it says stops the page's content from being used as direct input for AI overviews; it also removes text snippets from ordinary search results, so use it page by page, not site-wide.

## Check the result

After saving, open `https://www.example.com/robots.txt`: your two groups should appear above Wix's groups, each exactly once. Wix points out that search engines update their copy of the file on their next visit; RFC 9309 asks crawlers not to cache it for more than 24 hours, so allow a day. Wix itself reminds you that some crawlers ignore robots.txt and that this is "not an airtight solution". Then check a few URLs for `GPTBot`, `OAI-SearchBot` and `PerplexityBot` with the [IABIG robots.txt tester](/tools/robots-txt-tester/): the home page should be disallowed for the first and allowed for the other two.

## Official sources

- Wix, Editing Your Site's Robots.txt File: <https://support.wix.com/en/article/editing-your-sites-robotstxt-file>
- Wix, Blocking AI Crawlers from Your Site: <https://support.wix.com/en/article/blocking-ai-crawlers-from-your-site>
- Wix, Enabling NLWeb for AI Agent Access: <https://support.wix.com/en/article/enabling-nlweb-for-ai-agent-access>
- OpenAI, Overview of OpenAI Crawlers: <https://developers.openai.com/api/docs/bots>
- Microsoft, Overview of Bing crawlers: <https://www.bing.com/webmasters/help/which-crawlers-does-bing-use-8c184ec0>
- RFC 9309, Robots Exclusion Protocol: <https://www.rfc-editor.org/rfc/rfc9309>
