Skip to content
regensbot
Audit a URL freeSign in
DEV-URL-05Should5 of 11 in URL

Publish XML sitemaps listing only canonical, indexable URLs with accurate lastmod

Level
Should

Recommended: skip it only for a reason you can name.

Documentation

Documented

At least one cited Google document states or supports it.

Area
URLURLs, links and discovery
Covered

Automatic

Decided from one URL.

Why it matters

A sitemap listed in robots.txt can be picked up by any crawler; one that is not listed has to be submitted, for example in Search Console. Sitemap inclusion is a weak canonical signal, so a sitemap full of redirects, parameters or noindexed URLs sends mixed signals.

How to meet it

Generate sitemaps from the same source as the canonical tags: only URLs that return 200, are indexable and are their own canonical. Change lastmod only for significant updates (main content, structured data or links), never for footer, copyright or timestamp changes; Google uses lastmod only when it is consistently accurate. Split files at 50,000 URLs or 50 MB uncompressed under a sitemap index, and add hreflang (DEV-INT-05), image (DEV-IMG-04) and video (DEV-VID-04) entries where used. Reference the index in robots.txt and submit it in Search Console.

How to test it

Crawl the sitemap URLs: every one returns 200, has a self-referencing canonical and no noindex. The Sitemaps report shows each file as read, and the Page indexing report filtered by sitemap shows how many listed URLs are indexed.

How Regensbot covers it

AutomaticNeedsOne URL

Decided from one URL.

An audit reads the response, the headers, the HTML, robots.txt, the sitemaps and the structured data, and the finding shows the evidence it saw.

How verdicts workWhat you connect

Sources

Google documents04

  1. 01Build and submit a sitemapGoogle Search Central (opens in a new tab)
  2. 02How Google interprets the robots.txt specificationGoogle (opens in a new tab)
  3. 03How to specify a canonical URL with rel="canonical" and other methodsGoogle Search Central (opens in a new tab)
  4. 04Optimize your crawl budgetGoogle (opens in a new tab)

Claims in the knowledge base11

  1. D2-C017ConfirmedWelcome to indexing day!Shown on screen
  2. D2-C018ConsistentWelcome to indexing day!Shown on screen
  3. D2-C391SourceHandling web duplicationGoogle's documentation
  4. D2-C390ConsistentHandling web duplicationShown on screen
  5. D1-C067n/aHow crawling worksThe author's interpretation
  6. D2-C833SourceWelcome to indexing day!Google's documentation
  7. D2-C842ConfirmedWelcome to indexing day!Said on stage
  8. D2-C843ConsistentWelcome to indexing day!Said on stage
  9. D1-C475ConfirmedQ&ASaid on stage
  10. D1-C337ConsistentHow crawling worksSaid on stage
  11. D1-C336ConfirmedHow crawling worksSaid on stage

Claim IDs refer to the Search Central Live 2026 knowledge base. The knowledge base (opens in a new tab)

Confirmed
Google's documentation states the same thing.
Consistent
Google's documentation supports it without stating it.
Source
The claim is the cited document itself.
n/a
Nothing to verify: a quotation, framing, opinion or analysis.

Audit

Check it on your site.

One audit checks a URL against all 138 requirements and returns one finding for each, with its source. One URL decides 66; for each of the others, the finding says which source to connect.

Audit a URL freeBrowse all 138