LLMS.TXT: WHAT IT DOES, WHAT IT DOESN’T, AND HOW TO ADD ONE

Because llms.txt was built for AI agents and coding tools, not for search rankings.

According to llmstxt.org, OpenAI, Anthropic and Google’s Gemini team publish llms.txt files for their developer documentation. That can look like a contradiction, since Google Search says it ignores the file.

It isn’t. Google Search and an AI agent reading docs are two different jobs.

Google Search crawls and indexes your whole website, then ranks it. An agent helping a developer only needs the right page, right now, inside a limited context window.

An llms.txt file helps with the second job. It points the agent to clean, short content instead of pages full of menus and scripts.

That’s why documentation platforms like Mintlify and GitBook, and WordPress plugins like Yoast and AIOSEO, can now create an llms.txt file for you. Wix creates one for each Wix website.

What does this mean for your business? If developers or AI agents use your docs, an llms.txt file is worth the hour it takes.

If your customers simply ask ChatGPT which brand to buy, the file won’t decide that answer. Your content, reviews and mentions on other websites will.

Not in Google, and there’s no good evidence that it does in ChatGPT either.

People search for “llms.txt in seo” hoping it’s a shortcut into AI answers. Here’s what the data says.

Google’s guide to generative AI search says Google Search ignores llms.txt files. Creating one neither harms nor helps your rankings, and that includes AI Overviews and AI Mode.

For other AI tools, SE Ranking studied about 300,000 domains. It found no measurable link between having an llms.txt file and how often a domain got cited.

Google’s John Mueller also said server logs show chatbots fetching normal pages, not the llms.txt file.

Could that change? It might. The convention is young, and a big AI company could start reading it tomorrow.

But you shouldn’t plan your AI SEO around a maybe.

What does help you show up in AI search?

Content that answers your customers’ questions clearly, pages Google can crawl and index, accurate schema markup, and mentions of your brand on websites AI tools already trust. Those are the best uses of your time.

My advice: add an llms.txt file if it’s easy for you, then spend the rest of your time on those four things.

Mostly AI coding assistants, AI agents and a few tools that check websites for agent readiness.

The llms.txt proposal says the file is used on demand, when an agent needs information while helping someone. It isn’t built for search indexing.

Here’s where you’ll see it used.

Coding assistants reading developer docs.

This is where llms.txt files are most common, according to llmstxt.org.

Browser agents completing tasks on a website, like comparing products or booking a slot. Chrome’s Lighthouse now has an experimental llms.txt audit for this kind of agentic browsing.

Your own team’s AI tools. For example, if you load your website into an internal assistant, an llms.txt file can tell it which content matters.

Who doesn’t seem to read it? Google Search says it ignores the file. And Google’s John Mueller said consumer chatbots don’t fetch it in the server logs he’d seen.

You can check this yourself. Ask your developer or hosting provider for server logs, and search them for requests to /llms.txt.

If nobody requests it after a few months, you’ve spent an hour on housekeeping, not on a strategy. That’s an acceptable cost.

Your business name, a one-line summary, and links to your best content with a short note on each.

People search for “llms.txt example” because the spec is written for software docs. A business version is simpler.

Here’s a structure you can use.

An H1 with your business name. This is the only required part.

A short summary in a blockquote: what you do, who you serve and where. For example, “Dental clinics in Pune offering implants, aligners and root canal treatment.”

A few lines on what an AI tool might get wrong about you, for example the areas you serve or services you don’t offer.

H2 sections with lists of links, such as Services, Locations, Pricing and Policies. Add a short note after each link.

An Optional section, by convention, for content an agent can skip.

Keep it short.

Your file should fit in an AI tool’s context easily, with the detail behind the links.

And keep it accurate. An llms.txt file with old prices or a closed branch might be worse than none, because it states the wrong facts with confidence.

You can use the free llms.txt generator on this website, then edit the result by hand.

Only if you don’t want AI tools to use your content. For most businesses that want AI search visibility, blocking is the wrong move.

People search for “llms.txt vs robots.txt” because the two files sit side by side. They do different jobs.

A robots.txt file tells crawlers what they may access.

That’s where you allow or block bots like Googlebot or OpenAI’s crawlers.

An llms.txt file doesn’t control access at all. It only offers a summary and links for tools that choose to read it.

That means an llms.txt file can’t stop anyone from using your content. If you want control, robots.txt is the tool.

Here’s the mistake I see. A website blocks AI crawlers in robots.txt, sometimes through a security setting, then adds an llms.txt file and wonders why AI tools ignore it.

Check your robots.txt first. Make sure the crawlers you want, including Googlebot, can reach your content.

If you publish premium content, like paid reports, blocking some AI crawlers might make sense for you. That’s a business decision, not an AI SEO one.

Get robots.txt right, and treat llms.txt as an optional extra on top.

llms-full.txt is a single file with the full text of your key content, and most business websites don’t need it.

The llms.txt proposal also suggests offering clean Markdown versions of your pages, at the same URL with .md added. Some documentation platforms create an llms-full.txt file that bundles everything into one download.

Why does this exist? An AI coding assistant can load a whole set of docs at once, instead of crawling page by page.

That’s helpful for a software library with hundreds of pages of reference content.

For your clinic, shop or services firm, it’s likely overkill.

Your website might have twenty pages, and an AI tool can read them directly.

There’s also a maintenance cost. Each Markdown copy and each llms-full.txt file has to stay in sync with your live content. If they drift, AI tools might read the old version.

Google’s guide also says you don’t need Markdown files to appear in Google Search, because Google Search doesn’t use them.

Here’s the order I’d follow. Add a short llms.txt file, keep your real pages fast and readable, and skip llms-full.txt unless you publish developer docs.

Not in Google Search, but a careless one can cause other problems.

Google says creating an llms.txt file neither harms nor helps your visibility or rankings in Google Search. That part is settled.

Here’s where it can go wrong for you.

Old information. If your llms.txt file lists last year’s prices or a closed location, any AI tool that reads it gets the wrong facts.

Links you didn’t mean to share. Some generators crawl everything, including staging pages, admin paths or private PDFs. Check each link before you publish.

Server errors. Chrome’s Lighthouse flags a server error on /llms.txt. A missing file is fine, but a broken one fails the audit.

Wasted budget. The biggest cost is paying for llms.txt work while your content, reviews and technical AI SEO still need attention.

You can avoid all four with a simple check. Open the file in a browser, click each link, and add a reminder to review it whenever your content changes.

Use an llms.txt checker if you like, but your own eyes are the best check for a short file.

The bottom line: a small, accurate llms.txt file is safe. A big, stale one is a liability.