llms.txt Explained: The File That Helps AI Cite Your Site

llms.txt is a plain-text Markdown file you place at the root of your website (at yourdomain.com/llms.txt) that gives AI models a clean, curated map of your most important pages and the facts you want them to know. Think of it as a table of contents written for machines: instead of forcing ChatGPT, Claude, Gemini, Perplexity, or Grok to wade through your navigation menus, cookie banners, and ad markup, you hand them a short, structured summary and a list of the pages that actually matter. It will not force any engine to cite you, and adoption is still early — but it is a low-cost, low-risk way to make your site easier for AI to read, understand, and quote accurately.

What is llms.txt, exactly?

The llms.txt standard was proposed by Jeremy Howard (co-founder of Answer.AI and fast.ai) on September 3, 2024. The idea borrows its shape from two files the web already knows: robots.txt, which tells crawlers where they may go, and sitemap.xml, which lists every URL on a site. llms.txt is different from both. It is not about permission and it is not an exhaustive list. It is a human-curated, machine-readable brief — a short document, written in Markdown, that says “here is who we are, and here are the pages worth reading” in a format a language model can parse in one pass.

The problem it solves is real. Large language models have limited context windows, and a typical HTML page is mostly noise: scripts, styling, navigation, legal footers, tracking pixels. When an AI system tries to understand your business from raw HTML, it burns budget on markup and can misread or miss the parts that matter. A Markdown file is almost pure signal. By publishing one, you reduce the chance an engine summarizes you wrong, and you increase the chance it pulls the exact facts — your name, your category, your locations, your best content — that you want it to repeat.

It helps to be precise about how the three files relate. robots.txt answers “may you crawl this?” — it is a permission layer. sitemap.xml answers “what URLs exist?” — a complete machine index, often thousands of links deep with no priority or meaning attached. llms.txt answers a third question those two never touch: “what should you read first, and what does it mean?” It is opinionated and short by design. You are not listing everything; you are telling a model, in plain language, which pages carry the facts you most want repeated and why each one matters. That editorial curation is the entire value.

The llms.txt format and spec

The specification (published at llmstxt.org) is deliberately simple. A valid file uses standard Markdown and follows this structure, in order:

• An H1 with the name of your project or site. This is the only required element in the entire file.
• A blockquote summary (a line starting with >) that captures the key information needed to understand everything below it — who you are and what you do, in one or two sentences.
• Zero or more Markdown sections — paragraphs, bullet lists, any content except headings — giving extra context an AI should know.
• Zero or more H2 sections, each containing a list of links. Every link is a Markdown hyperlink [name](url), optionally followed by a colon and a short note describing what the page is.
• An optional H2 named ## Optional. Links placed under this heading are explicitly marked as skippable — a model working with a tight context budget can drop them first.

That is the whole spec. No new syntax to learn, no schema to validate against a parser, no required fields beyond the title. Because it is ordinary Markdown, you can write it by hand in any text editor.

A realistic llms.txt example

Here is what a complete, well-formed file looks like for a fictional coffee roaster. Notice the H1 title, the blockquote summary, a short context section, grouped H2 link lists, and an Optional section at the end:

# Summit Coffee Co.

> Summit Coffee Co. is a specialty coffee roaster and cafe based in Boulder, Colorado, roasting small-batch single-origin and blend coffees since 2014 and shipping nationwide.

We roast to order and publish detailed origin, tasting, and brewing information for every coffee. Use the links below for accurate product, brewing, and company facts.

## Core pages

- [About Summit Coffee](https://example.com/about): Company history, roasting philosophy, and Boulder cafe locations
- [Shop all coffee](https://example.com/shop): Current single-origin and blend offerings with roast levels and tasting notes
- [Wholesale & cafe program](https://example.com/wholesale): Pricing and onboarding for cafes, offices, and restaurants

## Brewing guides

- [Pour-over guide](https://example.com/brew/pour-over): Ratios, grind size, and step-by-step method
- [Espresso dial-in](https://example.com/brew/espresso): Dose, yield, and time for our espresso blend
- [Cold brew recipe](https://example.com/brew/cold-brew): Concentrate ratio and steep times

## Company

- [Contact & hours](https://example.com/contact): Cafe addresses, phone, and opening hours
- [Shipping & returns](https://example.com/shipping): Delivery timelines, costs, and return policy

## Optional

- [Blog](https://example.com/blog): Roaster notes, origin travel, and coffee education
- [Careers](https://example.com/careers): Open roles at the roastery and cafes

A person can read that in ten seconds and know exactly what the business is. So can a model — and that is the point.

Where it goes, and what llms-full.txt is

The file lives at the root of your domain: https://yourdomain.com/llms.txt. That fixed, predictable location is how an AI system knows where to look, exactly like robots.txt. Serve it as text/plain (or Markdown), keep it publicly reachable with no login wall, and make sure the URLs inside it are absolute and correct.

You will also see llms-full.txt mentioned. This is a companion, expanded version. Where llms.txt is the concise index — a summary plus links — llms-full.txt inlines the actual content of those key pages into one long Markdown document, so a model can ingest your core material without following a single link. The trade-off is size: the full file can run very long, so keep it to genuinely important pages rather than dumping your entire site. Many teams publish both: the short llms.txt as the map, and llms-full.txt as the full text for tools that want everything in one fetch. Publishing the short file first is the sensible starting point.

How AI engines actually use it (an honest look)

This is where you need the truth rather than the hype, because the five major engines do not treat llms.txt the same way, and none of them treat it as a ranking lever.

• Claude (Anthropic): Anthropic has indicated that Claude’s retrieval workflows can respect llms.txt, and curated files tend to show the most positive signal here.
• Perplexity: Perplexity has publicly said it retrieves llms.txt and can use it to help prioritize which pages to pull, which makes it one of the more receptive engines.
• ChatGPT (OpenAI): OpenAI has not formally confirmed that ChatGPT or its search surface consumes llms.txt as a ranking input. OpenAI’s crawlers — GPTBot, OAI-SearchBot, and the on-demand ChatGPT-User fetcher — primarily read your live HTML.
• Gemini (Google): Google has publicly stated it does not support llms.txt and has no plans to. Gemini and AI Overviews rely on Google’s existing crawl and index, governed by the Google-Extended token — not by this file.
• Grok (xAI): xAI has not published support for llms.txt. Assume Grok reads your site the ordinary way.

It also helps to separate the two ways an engine can touch your site, because llms.txt only affects one of them. Training crawlers like GPTBot and ClaudeBot gather text to build models. Search and answer systems — OAI-SearchBot for ChatGPT Search, Claude’s and Perplexity’s retrieval, on-demand fetchers like ChatGPT-User — pull pages in real time to answer a live question. It is the second group, the answer surfaces, where a clean summary and link map can plausibly change what gets read and quoted. Nothing about this file overrides the permission controls; it simply makes comprehension easier for the systems that already have access.

So does llms.txt work? The honest answer in 2026 is: it is emerging, not established. Independent studies have found adoption sitting at roughly one in ten sites, and analyses of AI-crawler logs show the bots overwhelmingly fetch HTML directly and rarely request /llms.txt at all. There is no credible evidence that simply adding the file guarantees more citations across the board. What some teams do report is a modest uplift in how accurately and how often they get quoted — most visibly on Anthropic and Perplexity — when the file is genuinely well curated. Treat it as one clean input among many, not a magic switch. The bigger wins still come from strong, well-structured published content, which is exactly what our GEO content strategy guide covers.

Best practices for a file that actually helps

• Lead with a sharp summary. The blockquote is the single most-read line. State who you are, what you do, and where — plainly, no marketing fluff.
• Curate, don’t dump. Link the pages that carry your best facts and content. A tight list of 10–30 high-value URLs beats a mirror of your sitemap.
• Write clean notes. The text after each link’s colon tells the model what the page is for. Make it specific: “Wholesale pricing and onboarding,” not “Learn more.”
• Keep it factual and current. Names, locations, hours, product lines — get them right, because an engine may repeat them verbatim. Update the file whenever those facts change.
• Use absolute URLs and check every link. A broken or relative link is a dead end for a crawler.
• Don’t treat it as access control. llms.txt is guidance, not a gate. If you want to block AI crawlers, that is robots.txt and crawler-specific tokens like Google-Extended — a separate job.
• Ship it alongside real content. The file points to your pages; it cannot make thin pages worth citing. Pair it with the work in our guide to getting cited by AI.

Keeping llms.txt correct without the manual upkeep

The catch with a hand-written file is drift. You publish a great llms.txt on launch day, then you add products, move pages, open a location — and six months later the file quietly points AI models at stale facts and dead links. Because engines may repeat what they read, a neglected file can actively mislead.

This is one of the things Emergeo handles for you. Emergeo auto-generates your llms.txt from a set of verified business facts plus the content you publish, and keeps it updated as new content goes live — so the file an AI reads always reflects what is actually true about your business, without you editing Markdown by hand. It is one piece of a broader system that also runs weekly tests across ChatGPT, Claude, Gemini, Perplexity, and Grok to measure whether those engines actually recommend you, and shows you the receipts. For the wider playbook that llms.txt fits into, see our complete guide to generative engine optimization.

The bottom line

llms.txt is a small, sensible file: a Markdown summary and a curated link map, served at your root, that makes your site easier for AI to read and quote correctly. It is not a ranking switch, adoption is still emerging, and the engines vary in how much they use it — Anthropic and Perplexity lean in, Google explicitly opts out. But it costs almost nothing to publish, it can only make you clearer to machines, and paired with genuinely strong content it is a clean win. Write a good one, keep it accurate, and let it do its quiet job.

Want to see how AI actually answers when customers ask about your business today? Run a free AI-visibility check at emergeo.ai — we’ll test the major engines, show you where you stand, and generate the llms.txt that helps them answer with you.