IABIGFR · Français

AI visibility · Updated 2026-11-06 · Markdown version

llms.txt - Exact Format, Examples and Real Adoption by AI Companies

The llms.txt file as specified on llmstxt.org, with examples for a business site and a documentation site, and what AI companies actually confirm about it.

The llms.txt file is a proposal published in September 2024 by Jeremy Howard (Answer.AI): a Markdown file placed at the root of a website that summarizes the site and points to its most useful pages in a form that is easy for a language model to read. The proposal was revised in August 2026 ("v2"). This guide describes the format as currently specified on llmstxt.org, gives two complete examples, explains how to produce the file, and reviews, with sources, what AI companies publicly say they do with it. That last part is less clear-cut than most articles suggest.

What llms.txt is for

A web page is built for humans: navigation, banners, scripts, ads. For an agent that needs to answer a question quickly, extracting the useful text from that HTML costs time and tokens, and the result is approximate. llms.txt offers a single, short, hand-written entry point that says "this is what this site is, and this is where the detailed information lives."

The proposal states that the file is mainly meant for inference (an agent consulting the site to help a user) rather than for model training. It replaces neither robots.txt (which expresses access permissions) nor sitemap.xml (which lists every indexable page). It is a curated, annotated selection.

The exact format

The file is named llms.txt. It can live at the root (/llms.txt) or under a subpath (/docs/llms.txt). Since v2, a file covers the URLs located under its path, and when several files apply, the agent should use the most specific one. The following sections must appear in this order.

Order Element Status
1 Byte order mark (BOM) optional
2 H1 heading with the name of the project or site the only required element
3 Blockquote (>) with a short summary containing the key information recommended
4 Paragraphs or lists with details, no headings optional
5 Sections delimited by H2 headings, each containing a "file list" optional

Each item in a file list must contain a Markdown link [name](url), optionally followed by : and a note. Here is the template given in the specification.

# Title

> Optional description goes here

Optional details go here

## Section name

- [Link title](https://link_url): Optional link details

## Optional

- [Link title](https://link_url)

The ## Optional section. By convention, it groups secondary information that an agent can skip if it needs a shorter context. In v1 it had a mechanical effect: the llms_txt2ctx tool, which expanded the file into a single block of context, left it out. V2 removed that tool from the proposal. The section is still allowed and useful, but it no longer has any special technical meaning.

Markdown versions of pages. The proposal recommends publishing a clean Markdown version of each useful page, at the same URL with .md appended (page.html.md) or replacing the extension (page.md). For a URL without a file name, use index.html.md or index.md. Links in llms.txt should point to these versions. V2 adds a discovery mechanism based on standard link relations, in HTML or in an HTTP header.

<link rel="alternate" type="text/markdown" href="/docs/page.html.md">
<link rel="describedby" href="/docs/llms.txt">
Link: </docs/page.html.md>; rel="alternate"; type="text/markdown", </docs/llms.txt>; rel="describedby"

What about llms-full.txt?

llms-full.txt is not part of the llmstxt.org specification, in either v1 or v2. It is a convention popularized by documentation platforms. Mintlify, for example, automatically generates a /llms-full.txt that "combines your entire documentation site into a single file," with each page's title, URL and full Markdown content. Anthropic, Perplexity, Cloudflare and Next.js publish such a file for their documentation.

In practice, llms.txt is the index and llms-full.txt is the full content. The latter can be very large (tens of megabytes for some documentation sets), which makes it useful for a tool that downloads it once, not for an agent that has to load everything into its context window.

Complete example for a small business website

For a small business, the point is to state in a few lines the facts an assistant would otherwise guess: what you do, where, how to reach you, published prices.

# Hollis Woodworks

> Custom woodworking shop in Portland, Maine, since 2009. We build and install wood windows, staircases and custom built-ins for homeowners and businesses within 40 miles of Portland.

- Free quotes by appointment; typical lead time is 6 to 8 weeks.
- We do not sell vinyl or aluminum windows.
- Contact: (207) 555-0100, contact@hollis-woodworks.example, Monday to Friday.

## Services

- [Wood windows and patio doors](https://hollis-woodworks.example/windows.md): wood species, energy performance, available rebates
- [Custom staircases](https://hollis-woodworks.example/staircases.md): staircase types, project steps, price ranges
- [Custom built-ins](https://hollis-woodworks.example/built-ins.md): closets, bookcases, kitchens

## Practical information

- [Service area and lead times](https://hollis-woodworks.example/service-area.md): towns served, typical schedule
- [Company information](https://hollis-woodworks.example/company.md): legal name, contractor license, insurance

## Optional

- [Portfolio](https://hollis-woodworks.example/portfolio.md): annotated photos of recent projects
- [News](https://hollis-woodworks.example/news.md): trade shows, holiday closures

Complete example for a documentation site

For technical documentation, which the proposal itself names as the main use case, favor links to reference pages and the most requested guides, with notes that help the agent choose.

# Kestrel API

> Kestrel is a REST geocoding API. Current version is v3 (v1 and v2 are no longer maintained). Authentication uses a key in the Authorization header; rate limit is 50 requests per second.

Important notes:

- Coordinates are always returned in latitude, longitude order.
- The official JavaScript SDK is `@kestrel/sdk`; there is no official Python SDK.

## Getting started

- [Quickstart](https://docs.kestrel.example/quickstart.md): get a key and make a first request
- [Authentication](https://docs.kestrel.example/auth.md): key rotation, scopes, 401 and 403 errors

## Reference

- [/geocode endpoint](https://docs.kestrel.example/reference/geocode.md): parameters, response format, examples
- [/reverse endpoint](https://docs.kestrel.example/reference/reverse.md): reverse geocoding
- [Error codes](https://docs.kestrel.example/reference/errors.md): full list and how to handle each

## Guides

- [Migrating from v2 to v3](https://docs.kestrel.example/guides/migration-v3.md): breaking changes
- [Batch geocoding](https://docs.kestrel.example/guides/batch.md): uploading CSV files

## Optional

- [Changelog](https://docs.kestrel.example/changelog.md)
- [Full documentation in one file](https://docs.kestrel.example/llms-full.txt)

Generating and publishing the file

By hand. For a site with a few dozen pages, this is the best option: the file should be a selection, not an export. The proposal recommends concise language, informative link descriptions, no unexplained jargon, and testing the file by asking questions to an agent that has only this file as its starting point.

With a tool. llmstxt.org lists integrations: documentation platforms (Mintlify, GitBook), WordPress plugins (Yoast SEO, AIOSEO), Wix, a Drupal recipe, and plugins for Docusaurus and VitePress. IABIG's own llms.txt generator helps you draft a starting file that you then edit by hand. Always review the output: a file generated from every page quickly ends up looking like a sitemap.

With a script. If your pages already have metadata (title, description), a build script can produce the file. Here is a minimal example in Node.js.

// generate-llms.js: builds llms.txt from a list of pages
import { writeFileSync } from "node:fs";

const site = {
  name: "Hollis Woodworks",
  summary: "Custom woodworking shop in Portland, Maine: wood windows, staircases and built-ins.",
};
const sections = {
  Services: [
    { title: "Wood windows", url: "https://hollis-woodworks.example/windows.md", note: "species and performance" },
    { title: "Custom staircases", url: "https://hollis-woodworks.example/staircases.md", note: "steps and prices" },
  ],
  Optional: [
    { title: "Portfolio", url: "https://hollis-woodworks.example/portfolio.md" },
  ],
};

let out = `# ${site.name}\n\n> ${site.summary}\n`;
for (const [name, pages] of Object.entries(sections)) {
  out += `\n## ${name}\n\n`;
  for (const p of pages) out += `- [${p.title}](${p.url})${p.note ? `: ${p.note}` : ""}\n`;
}
writeFileSync("public/llms.txt", out, "utf8");

Once it is live, check that the file returns a 200 status with a text content type, and that no firewall rule blocks it.

curl -sI https://www.example.com/llms.txt | grep -iE "^HTTP|content-type"
curl -s https://www.example.com/llms.txt | head -20

Real adoption, what is confirmed and what is not

This is where most articles extrapolate. Here is what can be verified in official sources as of early November 2026.

Company What is public
Google Search Its page "Optimizing your website for generative AI features on Google Search" says you don't need to create AI text files such as llms.txt to appear in Google Search, because Google Search ignores them. Creating them for other services is "completely fine" and "will neither harm nor help" your visibility or rankings in Google Search.
Chrome Lighthouse An audit documented with Lighthouse's agentic browsing audits checks llms.txt: it fails only on a server error when fetching the file; a missing file (404) is marked "not applicable," and the page says providing the file "is optional at the moment." Lighthouse is a diagnostic tool, not a search ranking system.
OpenAI Publishes an llms.txt for its developer documentation. Its crawler page (OAI-SearchBot, GPTBot, ChatGPT-User, OAI-AdsBot) does not mention llms.txt.
Anthropic Publishes llms.txt and llms-full.txt for its documentation. Its help article on ClaudeBot, Claude-SearchBot and Claude-User does not mention llms.txt.
Perplexity Publishes llms.txt and llms-full.txt for its documentation. Its page on PerplexityBot and Perplexity-User does not mention llms.txt.

In other words, the major AI labs publish llms.txt files so that developers and their coding agents can read their documentation, but none has stated in its official documentation that its crawlers or answer engines read the llms.txt of third-party sites, or that it influences which sources get cited. Google Search has stated the opposite for its own use. The v2 text states that "thousands of sites now publish an llms.txt file" and that "coding agents use them reliably"; that is the view of the proposal's author, not a confirmation from the providers. For a measured view of how many large sites actually publish one, the weekly AI crawler census checks llms.txt on the top 10,000 sites of the Tranco list.

What is certain is that an agent given the URL of your llms.txt by a user, or one that comes across it while browsing, can read it like any other page. It is just Markdown.

Should you publish one?

The cost is low and the file does no harm. It is clearly useful for technical documentation read by developers who work with coding agents. For a business website, it is a clean, up-to-date fact sheet, but don't expect a measurable effect on your presence in ChatGPT, Claude, Perplexity or Google. The documented levers are elsewhere: letting AI search crawlers through in robots.txt and at the firewall (see how to block AI training while staying visible), and serving content in the initial HTML rather than through JavaScript (see why JavaScript sites can be invisible to AI).

To measure real usage, look at your server logs: which requests target /llms.txt, with which user agent, and how often (grep '"GET /llms.txt' /var/log/nginx/access.log is enough to start; see tracking AI crawler traffic in server logs). To check your own site, use the IABIG checker.

Sources

Free tools

Related guides