Most GEO checklists published this year include llms.txt, usually near the top, usually with no evidence attached. I publish one on my own site, plus a much larger llms-full.txt, so I have a stake in the answer. The honest position is that there is now enough public data to say what llms.txt does not do, and not yet enough to say much about what it does. This article covers what the file is, who has said they read it, what the 2026 log studies found, how to check your own logs in ten minutes, and where I have landed. It is deliberately narrow. For the broader question of how to get cited by AI tools, my AI marketing guide is the better starting point.
What llms.txt actually is
llms.txt is a proposal, not a standard. Jeremy Howard published it at llmstxt.org in September 2024 as a way of adding a Markdown file at the root of a site to give language models LLM-friendly content. The format is simple: an H1 with the site or project name, which is the only required part, a short blockquote summary, optional notes, and sections of Markdown links to the pages that matter, with an Optional section for links an agent can skip. The proposal also suggests clean Markdown copies of key pages at the same URL with .md added. llms-full.txt is a related convention, not part of the original page, where the site puts the full text of its important content into a single file. Mine is generated automatically from the same content source as the site, so every article, its short answer and its FAQs appear there within an hour of publishing, with no separate file to maintain. None of this is like robots.txt. robots.txt controls access and crawlers are built to obey it. llms.txt is a suggestion about what to read, and nothing obliges any crawler to fetch it.
What Google says: Search ignores it
Google has been unusually direct. Its guide on optimising for generative AI features in Google Search, published in May 2026, says you do not need to create new machine-readable files, AI text files, markup or Markdown to appear in Google Search, and that creating them will neither harm nor help your visibility or rankings in Google Search because Google Search ignores them. That covers AI Overviews and AI Mode. The picture inside Google is not entirely consistent, though. Search Engine Journal's Matt G. Southern reported in May 2026 that Lighthouse 13.3 added an experimental Agentic Browsing audit that checks whether a site provides an llms.txt file, framed as useful for browser-based agents rather than as a search factor. So the most accurate summary is that Google Search ignores the file, while some of Google's developer tooling treats it as a possible convenience for agents. For OpenAI, Anthropic and Perplexity, I could not find any statement in their crawler documentation that their bots use llms.txt. Absence of a statement is not proof they ignore it, but it means anyone claiming llms.txt helps you rank in ChatGPT is guessing.
What the log studies found
The best public data comes from server logs. Ahrefs (Louise Linehan, June 2026) looked at 137,210 domains in Ahrefs Web Analytics that had traffic in May 2026. About 28% published an llms.txt file, which Ahrefs calls an upper bound because its customers skew technical. Of those files, 97% received no requests at all that month. Of the requests that did arrive, 96% came from bots, and AI bots accounted for 19.5%, with GPTBot the top named AI fetcher; SEO audit tools, unidentified bots and general crawlers made up much of the rest. Ahrefs also found no AI bot requests for llms.txt paths that returned a 404, which suggests AI crawlers are not routinely probing for the file. Two caveats keep me from over-reading this. Ahrefs sells SEO tools and its sample is its own customers. And a fetch is not use: a crawler reading the file tells you nothing about whether a model ever drew on it in an answer. Still, it is the largest dataset I know of, and it points one way. For most sites, almost nobody is reading the file.
How to check your own logs
You do not need a study to answer this for your own site. You need about ten minutes with your logs. If you have raw access logs, search them for requests to /llms.txt and /llms-full.txt, then group the matches by user agent. On a typical Linux server that is a grep for the path piped into a count of the user-agent field. If you are on a CDN or a platform like Vercel or Cloudflare, filter the request logs by path instead. Compare the result with requests for /robots.txt over the same period, which every serious crawler fetches regularly, so you have a baseline. Then verify any AI bot hits by IP, because user agents can be faked; OpenAI, Anthropic and Perplexity all publish their crawler IP ranges. On my own site this is built in. The same server-side logger that records AI crawlers also records requests to robots.txt, llms.txt, llms-full.txt and the sitemap, and my admin report has a Discovery files panel showing which bots fetched each file and when, plus a live check that each file is serving correctly. I describe that logger in my piece on tracking AI crawlers.
Where it might still help
There are a few situations where I think llms.txt earns its place, all of them outside classic search. Coding assistants and developer tools are the clearest case. Ahrefs found an AI coding tool among the top named fetchers, and documentation sites are where the format was born: a developer pastes your llms.txt or llms-full.txt into an assistant to give it clean context about your product. Browser agents are the second case, which is what the Lighthouse audit is aimed at. An agent asked to compare three vendors may find a clean summary easier to work with than a JavaScript-heavy page. The third case is internal. Writing an llms.txt forces you to decide, in a few lines, what your company does and which ten pages matter, and that summary is useful for your own team, your sales deck and any chatbot you build. I also use my llms-full.txt as a single file to hand to an assistant when I want it to answer questions from my own published content. None of these depend on Google or ChatGPT search reading the file.
My conclusion: keep it cheap and keep it honest
I keep llms.txt on my site, and I would add one for most B2B clients, on three conditions. It must be cheap: generated from your existing content, not a hand-maintained document that drifts out of date. It must be accurate: the same claims as your pages, because a summary that overstates what you do is worse than no summary. And it must not displace the work that does matter for AI visibility. Google's own guide is clear that Search ignores the file, and the log data says most AI crawlers rarely fetch it. If you have limited time, crawlable HTML, a clear answer near the top of each page, an accurate robots.txt policy, internal links to your key pages, and content with a real point of view will do far more. Then measure. If your logs show AI bots reading the file, good. If they show nothing after a few months, you have lost an hour of work, not a strategy. That is a much better position than the one most llms.txt advice puts people in.
Sources
llmstxt.org, Jeremy Howard, The /llms.txt file (3 September 2024): https://llmstxt.org/. Google Search Central, Optimizing your website for generative AI features on Google Search: https://developers.google.com/search/docs/fundamentals/ai-optimization-guide. Ahrefs, Louise Linehan, We analyzed 137K sites: 97% of llms.txt files never get read (15 June 2026): https://ahrefs.com/blog/llmstxt-study/. Search Engine Journal, Matt G. Southern, Google's llms.txt guidance depends on which product you ask (20 May 2026): https://www.searchenginejournal.com/googles-llms-txt-guidance-depends-on-which-product-you-ask/575431/
FAQ
No. Google's guide on generative AI features in Search says Google Search ignores AI text files like llms.txt, so the file neither helps nor harms visibility in AI Overviews or AI Mode.
None of their crawler documentation says so. Log studies show occasional fetches, with GPTBot the top named AI fetcher in Ahrefs' data, but a fetch does not show the content was used in an answer.
Search your server or CDN logs for requests to /llms.txt, group them by user agent, compare with robots.txt requests over the same period, and verify AI bot IPs against the ranges each vendor publishes.
Little, if it is generated from your real content and kept accurate. The risk is spending time on it instead of on crawlable pages, clear answers and a sensible robots.txt policy.