llms.txt: The New Standard AI Models Actually Read

⏱ 5 min read · AI Readiness

Robots.txt tells crawlers where they can't go. Sitemaps tell them where they should go. But neither file tells AI models what your site is actually about, what content matters most, or how you want your work to be cited.

That's where llms.txt comes in. A simple markdown file at the root of your domain that gives AI models the context they need to understand, cite, and reference your content correctly. And unlike robots.txt, which has been around for decades, llms.txt emerged organically from the AI community itself.

Why AI Models Need Context

When a language model crawls your site, it doesn't understand your site structure the way a human would. It sees HTML, text, and links — but it doesn't know which pages are authoritative, which content is evergreen, or what your site's core purpose is.

That matters when the model is deciding whether to cite you. If an AI can't quickly determine what your site is about and whether it's trustworthy, it's more likely to cite a competitor with clearer signals. llms.txt solves this by providing a human-readable summary that models can parse instantly.

What Goes in an llms.txt File

The format is deliberately simple: markdown, structured in sections, written for both humans and AI. A typical llms.txt includes your site name, a one-sentence description, key pages, content usage policy, and preferred citation format.

The content usage policy is critical. This is where you explicitly state whether AI models can use your content for training, for inference, or not at all. It's not legally binding, but it's a clear signal that models can check before deciding how to handle your content.

The preferred citation format tells models how to attribute your work. Do you want a link to the homepage? A specific author name? A particular URL structure? Most models will try to honor this if it's clearly stated.

How AI Models Use It

When an AI model encounters your site, it checks for llms.txt first. If the file exists, the model reads it before crawling deeper. This gives the model context that influences how it interprets your content.

For example, if your llms.txt states that you're a medical information site and lists your key authoritative pages, a model is more likely to cite you for health-related queries. If your llms.txt says you're a satirical news site, the model knows not to treat your content as factual.

This isn't guaranteed — models don't always follow llms.txt directives perfectly. But the major AI labs have started training models to recognize and respect these files, because it improves citation accuracy and reduces hallucinations.

llms.txt is what robots.txt should have been: a way to communicate intent, not just restrictions.

The Adoption Curve

llms.txt started as a grassroots standard. Developers and site owners who wanted better AI citations began adding these files to their domains. AI researchers noticed and started training models to look for them. Now, major AI labs are recommending llms.txt as a best practice.

It's not yet universal. Many sites still don't have one. But the sites that do are seeing better citation rates in AI-generated content. When Perplexity or ChatGPT cites a source, sites with llms.txt files are disproportionately represented.

What This Means for SEO

Traditional SEO optimized for Google's algorithm. GEO — Generative Engine Optimization — optimizes for AI models. And llms.txt is one of the clearest GEO signals you can send.

It's not a ranking factor in the traditional sense. But it's a trust signal. A site with a well-written llms.txt file is signaling that it understands the AI landscape, cares about accurate citations, and has thought about how its content should be used.

That matters when an AI model is deciding between two similar sources. All else being equal, the site with better metadata wins. And llms.txt is the best metadata you can provide.

How to Write One

Start with your site's core purpose in one sentence. Then list 3-5 key pages that represent your best content. Add a content usage policy — be explicit about training vs. inference. Include a preferred citation format. Keep it under 500 words.

The file should live at yourdomain.com/llms.txt, just like robots.txt. It should be plain text or markdown, not HTML. And it should be written for both AI models and humans — because humans will read it too, especially if they're deciding whether to trust your site.

The Future of AI-Readable Metadata

llms.txt is just the beginning. As AI models become better at understanding web content, we'll see more structured metadata formats emerge. But for now, llms.txt is the standard that actually works — because it was built by the people who needed it, not by a committee.

If you want AI models to cite your content correctly, you need an llms.txt file. It's that simple.