Discover your website pages
RankNova reads robots.txt, follows declared sitemaps and sitemap indexes, and falls back to homepage links when no sitemap responds.
RankNova LLMs.txt Generator
Free LLMs.txt Generator & Checker
Create an llms.txt file from your real website content.
RankNova discovers your important pages, checks your existing llms.txt, filters unsuitable URLs, organizes useful content and generates an editable file you can copy or download.
Free•No signup required
Existing llms.txt checkerSitemap discoveryPage recommendationsCopy and download
Quick answer
What Is llms.txt?
llms.txt is a proposed Markdown-based convention for giving AI agents a concise, curated map of a website's most useful content. Instead of making an agent parse an entire site, the file points to the pages that best explain what you offer, each with a short note.
- It does not control crawler access. That is what robots.txt is for. A blocked page stays blocked even if you list it.
- It does not replace sitemap.xml. A sitemap lists every indexable URL; llms.txt is a short curated selection. Keep both.
- It does not guarantee AI citations or visibility. Support depends on each platform choosing to use the convention.
See the shape of the file
llms.txt Example
A complete llms.txt file is short and readable. Here is a small example using the structure described in the proposal:
llms.txt
# Example Company > Example Company provides analytics software for online businesses. ## Products - [Analytics Platform](https://example.com/product): Analyze website and customer performance. ## Documentation - [Getting Started](https://example.com/docs/getting-started): Learn how to configure the platform. ## Guides - [Analytics Guide](https://example.com/guides/analytics): Learn the fundamentals of website analytics.
You do not need to type every URL by hand. RankNova builds this structure from the pages it actually discovers on your website, then lets you adjust the selection, sections, and descriptions before you export.
Required vs optional
What Is the Correct llms.txt Format?
llms.txt is plain Markdown. Under the current proposal only the H1 is strictly required — everything else is optional, in this order:
| Element | Markdown | Required? | Purpose |
|---|---|---|---|
| H1 heading | # Site or project name | Required | The name of the site or project. Under the current proposal this is the only strictly required element. |
| Blockquote summary | > Short summary | Optional | A brief summary carrying the key information someone needs before reading further. |
| Contextual detail | Plain Markdown paragraphs | Optional | Any additional non-heading Markdown that adds context, placed before the link sections. |
| H2 sections | ## Section name | Optional | Zero or more sections that group related resources, such as Products, Documentation, or Guides. |
| Link entries | - [Page title](https://example.com/page): Note | Optional | A Markdown link is required inside a section list; the note after the colon is optional. |
The file can live at your site root as /llms.txt or at a subpath such as /docs/llms.txt. Each file covers the URLs beneath its own path, and where more than one applies, agents are expected to use the most specific one.
Seven steps
How the RankNova LLMs.txt Generator Works
Filter low-value or unsuitable URLs
Login screens, carts, checkouts, search results, pagination, and non-HTML files are set aside with a reason you can review.
Identify important pages
Each page gets a deterministic Page Importance score from indexability, sitemap presence, navigation links, click depth, and content depth.
Organize pages into sections
Pages are grouped into sections such as Tools, Products, Documentation, and Guides using URL structure, headings, and structured data.
Generate descriptions
Descriptions come from the page's own metadata and content first. Optional AI assistance is grounded in the same fetched facts.
Validate your llms.txt
Before export, RankNova checks for duplicate, malformed, off-domain, broken, redirecting, and noindex URLs, and flags weak descriptions.
Copy or download
Review the preview, adjust anything you disagree with, then copy the file or download it as plain UTF-8 llms.txt.
Content selection
What Should Be Included in llms.txt?
Include the pages someone would need in order to understand what your website offers. The exact mix depends on the website, so treat the list below as a starting point rather than a rule.
Products
Pages that describe what you sell, including collection or catalog pages that give useful context.
Services
Service and solution pages that explain what you do and who it is for.
Tools
Free or public tools on your website that people can use directly.
Documentation
Setup guides, API references, and developer documentation.
Guides
Long-form tutorials and how-to content that answers real questions.
Important resources
Pricing, feature overviews, case studies, and other pages people reference when evaluating you.
Company information
About and contact pages that establish who is behind the website.
Keep it concise
What Should Usually Be Excluded?
Context matters, so none of these are absolute. RankNova sets them aside by default and always shows the reason, and you can include any of them from the editor.
Login and account pages
Sign-in, registration, and account screens are utilities rather than content.
Cart and checkout
These pages show per-visitor state and change constantly.
Search result pages
Generated per query, so they rarely describe anything stable.
Duplicate URLs
Non-canonical variants and parameter duplicates repeat what a primary page already says.
Noindex pages
If a page carries a noindex directive, listing it sends a mixed signal.
Broken pages
URLs returning 404, 410, or a server error add nothing useful.
Low-value utility pages
Empty tag archives, pagination, and thin routes dilute a concise file.
Two different files
llms.txt vs llms-full.txt
These names look related but only one of them is described by the proposal.
| File | Role | What it contains | Status |
|---|---|---|---|
llms.txt | Curated map | A short list of links to your most useful pages, each with an optional note. Designed to be read quickly and to fit comfortably in a context window. | Described by the llms.txt proposal |
llms-full.txt | Expanded content file | A filename some tools and sites use for a much larger document that inlines page content rather than linking to it. | Not defined by the core proposal |
llms-full.txt is not currently defined by the core llms.txt proposal and should not be treated as a required standard. RankNova generates the curated llms.txt file described by the proposal.
Beyond the file itself
llms.txt v2 and Markdown Versions of Your Pages
The current proposal goes further than a single file. It also describes offering Markdown versions of individual pages, and using standard HTML link relations so an agent can discover them.
rel="alternate" with type="text/markdown"
Points from an HTML page to a Markdown version of that same page, so an agent can read clean content instead of parsing the rendered markup.
rel="describedby"
Points from a resource to the llms.txt file that describes it, which helps an agent find the right file when several exist at different paths.
Most-specific file wins
Because a file covers the URLs beneath its path, a /docs/llms.txt is expected to take precedence over a root file for pages under /docs.
RankNova generates and validates the llms.txt file itself. It does not currently check rel="alternate" or rel="describedby" link relations on your pages, so treat this section as background rather than something the tool verifies today.
A direct answer
Does llms.txt Improve AI Visibility?
An llms.txt file can give agents that choose to use the convention a concise, structured path to your most useful content. That is the whole of what it does. It does not guarantee any of the following:
- ChatGPT citations
- Claude citations
- Gemini citations
- Perplexity citations
- Google AI Overview inclusion
- AI ranking or recommendation
Crawler access, content quality, relevance, authority, each platform's retrieval system, and its own policies all remain separate factors, and support depends entirely on each platform choosing to adopt the convention. If AI crawlers cannot reach your pages at all, the file changes nothing. Check that first with the AI Crawler Checker and the Website Crawl Test, then look at AI visibility tracking for what actually shows up in AI answers.
Three different files
llms.txt vs robots.txt vs sitemap.xml
These files answer different questions and are not interchangeable. llms.txt complements robots.txt and sitemap.xml rather than replacing either, so a website can and usually should have all three.
| File | Purpose | What it contains | Status |
|---|---|---|---|
| llms.txt | Curated content map for agents | A short list of useful pages with a note for each one, read on demand when an agent needs information about a topic. | Proposal, adoption varies |
| robots.txt | Crawler access rules | Tells crawlers which paths they may or may not request, per user agent. It governs access; llms.txt does not. | Widely supported |
| sitemap.xml | URL discovery | Lists URLs so crawlers can find them, usually every indexable page on the site, with no curation or descriptions. | Widely supported |
Related RankNova tools
What to Check After You Publish llms.txt
Publishing the file is the first step, not the finish line. Work through these checks in order to confirm the pages you listed are actually reachable and worth surfacing.
AI Crawler CheckerListing a page in llms.txt grants no access. Check whether GPTBot, ClaudeBot, PerplexityBot, and Google-Extended can actually reach the pages you selected.Website Crawl TestReview robots.txt rules, sitemap discovery, and homepage crawlability for the whole host, so the URLs in your file are reachable and indexable.Meta Tag CheckerYour titles and meta descriptions become the link text and notes in llms.txt. Inspect and improve them to get stronger entries.GEO ScoreSee how RankNova measures AI search readiness across a site, well beyond what any single file can show.AI Visibility TrackingTrack how your brand actually appears in AI-generated answers, which is the outcome llms.txt alone cannot guarantee.Technical SEOUnderstand how crawlability, indexability, and site structure fit into wider technical health.
Direct answers
llms.txt Frequently Asked Questions
What is llms.txt?
llms.txt is a proposed convention for publishing a concise, curated Markdown map of a website's most useful content. Agents that support it can read the file to find relevant pages quickly instead of parsing a whole site. It does not control crawler access and it does not replace sitemap.xml.
Is llms.txt an official web standard?
No. llms.txt is a proposal published at llmstxt.org, not a specification ratified by the W3C or IETF. Support depends on each individual tool or AI platform choosing to adopt it, so treat it as an emerging convention rather than a guaranteed standard.
How do I generate an llms.txt file?
Enter your website URL in the generator above. RankNova discovers your pages through robots.txt and your sitemaps, filters out unsuitable URLs, groups the useful ones into sections, and writes a description for each entry. You can then edit the selection and copy or download the file.
Where should I put llms.txt?
The usual place is your site root, so it is reachable at https://example.com/llms.txt. The proposal also allows llms.txt files at a subpath, such as /docs/llms.txt, where each file covers the URLs beneath its own path. When more than one file applies, agents are expected to use the most specific one.
What should an llms.txt file contain?
An H1 with your site or project name is the only strictly required element. Most files also add an optional blockquote summary, optional context in plain Markdown, and H2 sections holding lists of links with short notes. Point those links at the pages that best explain what you offer.
Should every page be included in llms.txt?
No. The file works best when it stays short and curated. Login screens, carts, checkouts, search results, duplicate URLs, and broken pages add noise without adding understanding, which is why RankNova filters them out by default and tells you why.
Is llms.txt the same as robots.txt?
No. robots.txt tells crawlers which paths they may request, so it governs access. llms.txt describes which pages are useful and what they contain, and grants no access of its own. A page blocked in robots.txt stays blocked even if you list it in llms.txt.
Is llms.txt the same as sitemap.xml?
No. A sitemap is a machine-readable list of URLs for discovery and usually includes every indexable page. llms.txt is a short curated selection with a human-readable note for each entry. The two complement each other, and you should keep both.
Does llms.txt improve SEO rankings?
There is no evidence that llms.txt affects traditional search rankings, and no major search engine has said it uses the file as a ranking signal. Treat it as a way to describe your content for agents, not as an SEO tactic.
Does llms.txt improve ChatGPT or AI visibility?
Creating the file does not guarantee that a website will appear, rank, be cited, or be recommended by ChatGPT, Claude, Gemini, Perplexity, Google AI Overviews, or any other AI system. Crawler access, content quality, relevance, authority, each platform's retrieval system, and its policies all remain separate factors.
What is llms-full.txt?
llms-full.txt is a filename some tools and sites use for a much larger file containing full page content rather than a curated link list. It is not defined by the core llms.txt proposal, so treat it as an optional extra convention rather than part of the specification.
What if my website already has llms.txt?
RankNova detects the existing file, parses it, and reports broken, redirecting, duplicate, off-domain, and noindex URLs along with recommended pages that are missing from it. You get an LLMs.txt Health score for the current file and a regenerated version you can compare against. Your website is never modified.