Quick answer
If your goal is Google Search — including AI Overviews and AI Mode — you do not need llms.txt. Google says Search does not use it for visibility or rankings. It can still be useful for developer documentation and compatible agent workflows, so treat it as optional interoperability rather than an SEO requirement.
Contents
- What is llms.txt?
- Does Google Search use llms.txt?
- Why Lighthouse checks for llms.txt
- Who actually uses llms.txt?
- Does llms.txt help ChatGPT, Claude or other AI assistants find your site?
- Should you create an llms.txt file?
- How to publish llms.txt without creating another maintenance problem
- What should you focus on for Google AI Search instead?
- Will Elegant Matter use llms.txt?
What is llms.txt?
llms.txt is a proposed convention for a Markdown file, usually placed at /llms.txt, that gives language models and AI agents a concise description of a site and links to useful machine-readable resources.
Jeremy Howard published the original proposal on 3 September 2024. The current v2, published 10 August 2026, adds standard link relations that help agents discover Markdown versions of pages and the llms.txt file that covers them.
The basic idea is deliberately simple: instead of forcing an agent to infer a site’s structure from navigation, JavaScript and hundreds of HTML pages, a site can provide a small curated index that points to the material an agent is most likely to need.
Version 2 also adds standardized ways for pages to advertise machine-readable alternatives and their related llms.txt file. In practice, that makes the proposal more useful for agents that are actively navigating documentation rather than for search engines deciding what should rank.
The common shorthand “robots.txt for AI” is misleading. robots.txt controls crawler access; llms.txt provides no such control. A curated map or reading list for compatible agents is a closer description.
Does Google Search use llms.txt?
No.
Google’s current guidance is unusually explicit: Google Search does not use AI text files such as llms.txt to determine visibility in Search, including its generative AI features. Google also says maintaining one will not positively or negatively affect rankings.
That means llms.txt is not required for:
- AI Overviews
- AI Mode
- normal Google rankings
- being crawled or indexed by Google
- Google’s generative-search visibility
A file can still be discovered or indexed like many other file types. That does not give it special treatment.
A lot of early llms.txt advice framed the file as an AEO or GEO tactic. Google’s 2026 guidance settles that question for Google Search.
Why Lighthouse checks for llms.txt
Lighthouse is measuring how well an AI agent can use a site. Google Search ranking is a separate system.
Chrome’s Lighthouse documentation describes llms.txt as an emerging convention that can help AI agents understand a site’s high-level structure and primary content. Lighthouse includes an llms.txt check in its agentic-browsing category.
The audit treats the file as optional. If /llms.txt returns a normal 404 because the site does not provide one, Lighthouse marks the audit Not Applicable rather than failing the page. It flags server errors when the file cannot be retrieved cleanly.
The two systems are checking different things:
- Google Search: llms.txt is not used for search visibility or rankings.
- Lighthouse agentic browsing: llms.txt may be useful as an optional machine-readable summary for agents.
Lighthouse evaluates agent usability; Search ranking does not depend on the file.
| Question | Google Search | Compatible AI agents/tools |
|---|---|---|
| How is the site accessed? | Normal crawling and indexing. | May use llms.txt as a guide to relevant content. |
| Does llms.txt matter? | Google says it is not used for Search visibility or rankings. | Support depends on the tool and workflow. |
| Main use case | Search results, AI Overviews and AI Mode. | Documentation, agentic browsing and machine-readable navigation. |
| Should most sites prioritize it? | No reason to prioritize llms.txt for Google SEO. | Only when there is a concrete agent or documentation use case. |
Who actually uses llms.txt?
The clearest real-world use case today is developer documentation.
OpenAI publishes an llms.txt index for its API documentation, linking to Markdown versions of guides and a combined llms-full.txt export.
Anthropic also publishes an llms.txt file for its documentation.
Microsoft’s Teams SDK documentation explicitly tells developers that its llms.txt files are designed to give AI coding assistants the right context.
X publishes llms.txt and llms-full.txt for its API documentation and describes them as structured access for tools such as coding assistants and agents.
These examples matter because they show that llms.txt is not purely theoretical. But they do not prove that publishing one improves rankings, AI citations or visibility in every assistant.
They demonstrate a much narrower point: when an AI tool deliberately looks for or is pointed to an llms.txt file, the format can be a convenient way to expose documentation.
Does llms.txt help ChatGPT, Claude or other AI assistants find your site?
The examples above only show that these companies publish the format for their own documentation. They do not establish how every consumer search or retrieval product treats third-party llms.txt files.
OpenAI and Anthropic publishing llms.txt for their own documentation demonstrates that the format is useful in at least some agent and documentation workflows. It does not establish that every ChatGPT or Claude search/retrieval system automatically looks for llms.txt on every website.
The same caution applies to Perplexity, Gemini and other AI products. Their crawling, retrieval and citation systems can change independently, and support should be verified platform by platform.
We would not recommend adding llms.txt solely because somebody says it “helps LLMs understand your brand.” That claim is too broad to be useful.
Should you create an llms.txt file?
It depends on what your site is for.
If you run developer documentation or an API reference, llms.txt is increasingly reasonable. Your users may already be working through coding agents, and providing a compact index plus Markdown versions of documentation can reduce friction.
If you run a product knowledge base, it may also be worth considering when customers or support agents regularly need structured access to documentation.
If you run a normal editorial, business or portfolio website, it is optional. Creating a small correct file is unlikely to be a major burden, but there is no reason to prioritize it over the work that actually affects discoverability and usefulness.
If your only goal is Google SEO, do not create it for that reason.
Create llms.txt when you can name a real agent or workflow that benefits from it. If the only justification is a vague promise about “AI SEO,” it can probably wait.
How to publish llms.txt without creating another maintenance problem
An llms.txt file should complement your website, not become a parallel publishing system you struggle to maintain.
- Keep the important information available in normal crawlable HTML.
- Use canonical pages as the source of truth.
- Keep the llms.txt index concise and curated.
- Point agents to useful, current resources rather than dumping every URL on the site.
- Update the file when important URLs or documentation change.
- Do not hide information from users while exposing a richer version only to machines.
- Do not describe the file internally as a ranking or citation hack.
The v2 proposal gives publishers more options for linking pages to Markdown alternatives and to the llms.txt file that describes them. Those features are most useful when you already maintain high-quality machine-readable documentation.
What should you focus on for Google AI Search instead?
Google’s 2026 generative-AI guidance is much less exotic than the AEO/GEO industry sometimes makes it sound. Our guide to Google’s 2026 AI Search guidance breaks down the full set of recommendations, myths and new Search Console controls.
Google recommends continuing to focus on the foundations of Search: crawlable and indexable pages, useful internal links, accurate structured data where appropriate, good page experience, and content that provides something genuinely useful and non-commodity.
It explicitly says you do not need to:
- create llms.txt for Google
- break every article into arbitrary tiny “RAG-friendly” chunks
- rewrite normal prose specifically for AI systems
- add special AI schema
- manufacture mentions around the web
Google has also introduced dedicated generative-AI performance reporting in Search Console, which is a much better place to measure whether your pages are appearing in AI Overviews and AI Mode than trying to infer success from whether an AI crawler fetched a text file.
If you have an hour to spend on AI-search visibility, improving a weak page is likely to be a better use of it than writing an llms.txt file for Google.
Will Elegant Matter use llms.txt?
Not as a launch priority.
For an editorial site like Elegant Matter, the immediate value is limited. Our priority is making the actual articles useful, crawlable, well sourced and easy to navigate.
As the publication grows, a concise llms.txt file may become worthwhile as a low-maintenance index for agents — especially if agentic browsing and direct content retrieval become a meaningful source of use. At that point, we would add it for interoperability, not because we expect it to improve Google rankings.
That is also a useful test for other sites: if you cannot explain who the file is for, it can probably wait.
For most sites, it can wait
As of September 2026, Google Search gives site owners no SEO reason to add llms.txt. Adoption is easier to see in developer documentation and agent tooling, where the format can save an agent from reconstructing a site’s structure from scratch.
That makes llms.txt a small interoperability feature with a real but limited use case. Sites with documentation-heavy or agent-facing workflows have the clearest reason to consider it. Everyone else can spend the time elsewhere and revisit the question if their users or tools start asking for it.
Sources and verification
- Google Search Central — Guide to optimizing for generative AI features on Google Search
- Google Search Central — Documentation updates: clarification on llms.txt
- Chrome for Developers — Lighthouse llms.txt audit
- llms.txt specification — Version 2 and change log
- OpenAI API docs — llms.txt
- Anthropic documentation — llms.txt
- Microsoft Learn — Teams SDK llms.txt
- X Developer docs — llms.txt and llms-full.txt