Does llms.txt actually work?

What the evidence actually says

Google's own guidance for AI Overviews and AI Mode says standard SEO foundations still apply, and that site owners do not need to create new machine-readable files, AI text files, or special schema to appear in those features. That does not make llms.txt harmful or useless for every possible client. It means the file should be presented as an optional convention for documented consumers, not as a Google Search or AI recommendation requirement.

Do not mix up training crawlers and answer retrieval

A robots.txt rule for a named crawler, a training opt-out, a live-search fetcher, and an llms.txt summary all do different jobs. GPTBot, ClaudeBot, OAI-SearchBot, Googlebot, Google-Extended and other agents are not interchangeable signals. Blocking a crawler can matter when that crawler is the one a service uses to fetch pages, but it does not prove lost recommendation share on its own, and a present llms.txt file does not prove a model read or cited the site.

Why we still check for it

Because the file is cheap to maintain and can be useful when a tool or internal workflow explicitly consumes it. What changed is the weight: a missing llms.txt should not be a critical defect, a score driver, or a promised route to AI mentions. The useful version of the check is whether the file, if present, is accurate, current and clearly subordinate to the stronger signals: crawlable pages, visible content, consistent entity facts and measurable recommendation outcomes.

Structured data is still useful, with limits

Structured data helps search systems interpret page facts and can make a page eligible for supported search features when the markup matches visible content. It does not guarantee ranking, a rich result, AI citation, or inclusion in AI Overviews. Treat schema as a clarity and eligibility layer, not a shortcut around helpful visible content or independent corroboration.

Some rich results you may still be chasing no longer exist

Google's Search documentation updates say the FAQ rich result no longer appears in Google Search results, and the old feature documentation was removed in June 2026. FAQPage markup can still be retained for another documented consumer if it accurately matches visible questions and answers, but it should not be recommended as a way to earn an expandable Google FAQ result.

Measure recommendations directly

The outcome that matters is not crawler hits, total page count, or whether a file exists. For AI recommendation work, measure a fixed panel of realistic prompts, run it on a schedule, and record whether the business is mentioned, how it is described, which competitors appear, and what sources the answer cites. That gives you a recommendation-share baseline instead of a proxy metric dressed up as demand.

What to do with an afternoon

Check that important public pages are crawlable, indexable, and readable as text. Confirm robots.txt rules reflect a current policy rather than an old blanket block. Make the business facts on the site specific enough to quote, then reconcile the same facts across profiles, directories and review platforms. Keep llms.txt if it helps a known workflow, but spend the serious time on facts, access and corroboration.

Frequently asked questions

Turn website findings into an action plan

Create an account to start a Full Audit of your website, with supporting evidence and a prioritised action plan.