llms.txt: what the file does and how to test it
llms.txt is a proposed Markdown file that points AI agents to the parts of your website that matter. Treat it as a curated entry point to public content that someone keeps current and tests, and do not expect it to win you more citations.
Start with the proposal and its limits
The llms.txt proposal describes a short overview with links to detailed material; checked on 16 November 2026. It suggests a way to present information. No search system has promised to read it.
Google says it ignores llms.txt for Search visibility; guidance checked on 16 November 2026. So ask what problem the file solves for your readers. A product with heavy documentation may need a short route into its references. A small site with clear pages probably has a more urgent problem somewhere else. Decide what the file is for before you take on another thing to publish and maintain.
Keep website files tied to their purpose
| File | Purpose to distinguish |
|---|---|
| llms.txt | A curated overview that points an agent to relevant information |
| robots.txt | Crawler access rules for agents that follow them |
| sitemap.xml | A list of your URLs for discovery |
| The public page | The explanation a person can read and you have to keep accurate |
Curate the links around a useful task
Pick one task, such as working out which integrations your product supports. Link to the page that answers it and say what the reader will find there. Pasting your whole site navigation into a list helps nobody.
Product availability and limits in the file should come from the same approved sources as the public site. If the file and the product page disagree, fix the fact first. Until then the file does more harm than good.
Use a small, clearly labelled structure
The proposal uses a project heading, an optional summary and grouped links. For a recruitment site, a short entry can point straight to a real service page.
# Scalerr> GTM recruitment for tech companies.## Recruitment- [Key Talent Search](https://scalerr.com/md/key-talent-search.md): GTM role recruitment.
That example links to Scalerr's public service description. Check every destination, and write each description so a reader can tell which link they need.
Test whether it helps answer the intended question
Hand a colleague the file and a real product question. Ask them to follow the links and show you the evidence behind their answer. Note where the route gets confusing or where a page fails to answer.
You can run the same test with an agent, but check its answer against the linked pages. A fluent reply does not prove the file supplied the right evidence. Save the source passage next to the result so someone else can check it.
Assign maintenance to the publication workflow
Review the file whenever a linked product page changes or moves. Check that it still describes what you sell and that every link still works. An out-of-date file is one more version of your company story that someone has to reconcile.
For a wider review, use the AEO checklist and the AI visibility audit. Keep your reason for maintaining llms.txt apart from any claim about search performance.
People also ask
Does llms.txt guarantee AI citations? No. The file can give an agent a useful route to your information, but no answer system has committed to cite it. Test the use you have in mind and report only what you observe.
Does llms.txt replace robots.txt? No. llms.txt is a guide to your information; robots.txt sets crawler access rules. Your website owner should maintain the access policy on its own.
Who should keep llms.txt up to date? Give it to one owner who works with the people who maintain the linked pages. They should review it whenever a product explanation changes, so the file never contradicts the site.
Hiring a marketing leader to own this work? Brief Scalerr on the role or use the enquiry form below.
Hiring someone to own search?
Scalerr recruits marketing leaders for tech companies, from a first Head of Marketing to a CMO. Brief us on a marketing leadership hire, or use the form below.
Request a call