llms.txt: what it is, and whether it does anything yet
Siddhant AryanLead Designer · AI AutomationListen to this article
The short answer
`llms.txt` is a community proposal — a Markdown file at your site root that gives language models a clean, curated map of your most important content, in the way `robots.txt` gives crawlers rules and `sitemap.xml` gives them URLs. It is not an official standard, and no major AI provider has publicly committed to reading it. So treat it as cheap, harmless housekeeping with possible future upside, not as an AI visibility tactic. The things that demonstrably affect whether AI answers cite you are crawler access in `robots.txt`, extractable on-page answers, valid structured data, and third-party corroboration — in that order.
On this page
Every few months a file appears that the internet decides is the new SEO. `llms.txt` is the current one, and I have watched clients get quoted real money to 'implement llms.txt for AI optimisation'. So let me be plain about what it is, because the honest version is still useful: it is a good idea, it costs an hour, we ship it on our own site, and it is almost certainly not why anyone gets cited in 2026.
What the file actually is
The proposal is simple. Language models work with limited context and struggle with the navigation, scripts and boilerplate around real web pages. So publish `/llms.txt` — a Markdown document listing your key pages with one-line descriptions, grouped by section, optionally pointing at clean Markdown versions of each page. A model or agent that wants to understand your site can read one small file instead of crawling and parsing fifty.
It is deliberately unlike `robots.txt`. It grants nothing and forbids nothing; it is a curated index, closer in spirit to a sitemap written for a reader than to an access-control file.
What is llms.txt?
A proposed Markdown file at a site's root (`/llms.txt`) that gives large language models a curated, boilerplate-free map of the site's most useful content. A community convention — not a specification published or endorsed by any major AI provider.
Who actually reads it
As of 2026, the honest answer is: no major provider has publicly committed to honouring it. Some developer tools, documentation platforms and agent frameworks do look for it, and a number of technical-product sites publish one. That is a real ecosystem, but it is not ChatGPT, Gemini or Perplexity promising to fetch your file before answering a question about your category.
Which means the correct framing is optionality. The file costs an hour to write, adds no risk, and if adoption grows you already have it. Anyone selling it as the reason your AI visibility will improve is selling a hypothesis as a service.
How to hold this
Publish it because it is cheap and tidy. Do not move budget to it from crawler access, content structure, schema or reputation work, all of which have observable effects today.
What a good one looks like
Keep it short and curated. The value is in the choosing: an H1 with your business name, a blockquote summarising what you do, then grouped links to the pages that genuinely represent you — services, pricing or packages, case studies with results, your best guides, and contact details. One line of description per link, written for a reader rather than a keyword.
Resist the temptation to list everything. A file with 400 links recreates exactly the problem it exists to solve.
A workable structure
- 1`# Business name` — then a one-line blockquote: what you do, for whom, where.
- 2`## Services` — your real service pages, one line each on what the service is.
- 3`## Pricing` — anything with numbers on it. Models and buyers both want this.
- 4`## Proof` — case studies with named outcomes, not a portfolio grid.
- 5`## Guides` — your strongest explanatory content, the pages you would want quoted.
- 6`## Contact` — name, address, phone, email, hours. Identical to everywhere else you publish them.
Generate it, don't hand-maintain it
On this site /llms.txt is a route that builds itself from the same content data the pages use, so it cannot drift out of date. A hand-written file is accurate for about six weeks.
The things that actually move AI visibility
If you have an hour for AI visibility, spend it in this order. Check `robots.txt` for accidental blocks on GPTBot, OAI-SearchBot, PerplexityBot and Google-Extended — a block makes everything else pointless. Then make sure your important pages answer their question in the first 60 words in a way that survives being quoted. Then check your structured data is valid and says who you are. Then go look at what other sites say about you.
`llms.txt` sits below all of those, and that ordering is the entire practical message of this article.
| Action | Effort | Observable effect today |
|---|---|---|
| Unblock AI crawlers in robots.txt | 10 minutes | High — it is a gate, not a lever |
| Answer-first rewrites on key pages | A few hours per page | High |
| Valid Organization / LocalBusiness / FAQ schema | Half a day | Medium — removes ambiguity |
| Third-party corroboration (reviews, case studies, mentions) | Ongoing | High, slow |
| Publish llms.txt | One hour | None proven yet — cheap option on the future |
If you publish one, do it properly
Serve it as `text/plain` at exactly `/llms.txt`, keep it under a few hundred lines, use absolute URLs, and make sure every link returns 200 — a curated index of broken links is worse than no index. If you also generate clean Markdown versions of pages, link them, but do not let those become a second, stale copy of your site.
Then forget about it. It is infrastructure, not a campaign, and the moment you find yourself reporting on it monthly you have lost the thread.
1 hour
Reasonable budget for llms.txt. If a quote for it has more digits than that implies, you are paying for a hypothesis.
Key takeaways
- llms.txt is a community proposal, not an adopted standard — no major AI provider has committed to reading it, so treat it as cheap housekeeping rather than a visibility tactic.
- A good one is curated, not exhaustive: business summary, services, pricing, proof, best guides, contact — and generated from your content data so it cannot drift.
- Spend the hour on robots.txt access, answer-first rewrites, valid schema and third-party corroboration first; those have observable effects today.
Frequently asked questions
What is llms.txt?
A proposed Markdown file at your site root (/llms.txt) that gives language models a curated, boilerplate-free map of your most useful pages with a one-line description of each. It is a community convention rather than a standard published by any AI provider, and it grants or restricts nothing — it is an index, not an access rule.
Do ChatGPT, Gemini or Perplexity read llms.txt?
None of them has publicly committed to it as of 2026. Some developer tools, documentation platforms and agent frameworks look for the file, so there is a real if narrow ecosystem, but you should not expect a mainstream assistant to fetch it before answering a question about your business.
Is llms.txt worth adding to my website?
Yes, if it costs you an hour and you treat it as optionality — it is harmless, tidy, and already there if adoption grows. No, if it displaces budget from crawler access, answer-first content, valid schema or reputation work, all of which measurably affect whether AI answers cite you today.
How is llms.txt different from robots.txt and sitemap.xml?
robots.txt sets access rules for crawlers. sitemap.xml lists every URL for discovery. llms.txt is a human-readable, curated summary of the content that best represents your site, written in Markdown so a model can consume it cheaply. Different jobs — and llms.txt is the only one of the three that nothing is obliged to read.
Should llms.txt list every page on my site?
No. The value is entirely in the curation — if you list everything you have rebuilt the problem the file exists to solve. Include the pages you would actively want quoted: services, pricing, proof with real outcomes, your strongest guides, and contact details.
Tools & next steps
Put this into practice, go deeper, or see how we'd do it for you.
Written by
Mr. Siddhant Aryan
Lead Designer & AI Automation, Global Info Edge
Lead designer and AI-automation specialist at Global Info Edge with 5 years building fast, conversion-focused websites and the workflows that run behind them.
View full profile