Skip to content
regensbot
Audit a URL freeSign in
DEV-MON-03Should3 of 11 in MON

Watch the Page indexing report and treat its not-indexed reasons as a work list

Level
Should

Recommended: skip it only for a reason you can name.

Documentation

Documented

At least one cited Google document states or supports it.

Area
MONTesting and monitoring
Covered

Assisted

Proposed with the evidence. A person confirms it.

Why it matters

The Page indexing report is where index selection problems show. "Discovered - currently not indexed" means Google knows the URL but has not crawled it yet, and "Crawled - currently not indexed" means Google processed the page but did not keep it, which was described at Search Central Live as most of the time a quality issue; such pages need no resubmission. Soft 404s are also looked up in this report, while other crawl issues are debugged in the Crawl stats report (DEV-MON-04).

How to meet it

Export not-indexed URLs by reason and template regularly. For Discovered, first rule out slow responses and server errors; for Crawled, compare with indexed pages of the same type for thin, duplicate or soft-404-like content and for template differences. Do not resubmit the same URLs again and again.

How to test it

A trend of indexed and not-indexed URLs per sitemap and template is reviewed, and spikes after releases are investigated.

How Regensbot covers it

AssistedNeedsSearch Console, Merchant Center or BigQuery

Regensbot proposes a verdict with the evidence; a person on your team confirms it.

The evidence comes from Search Console, Merchant Center or BigQuery once you connect them.

One consent screen grants read-only Search Console access. Merchant Center and BigQuery are requested separately, only where they apply, and Regensbot only reads from them. URL Inspection samples and Search Analytics add Google's own view of your pages.

How verdicts workWhat you connect

Sources

Google documents03

  1. 01Page indexing reportGoogle Search Console Help (opens in a new tab)
  2. 02Optimize your crawl budgetGoogle (opens in a new tab)
  3. 03Crawl Stats reportGoogle Search Console Help (opens in a new tab)

Claims in the knowledge base10

  1. D2-C717ConsistentDeciding what goes in the index?Said on stage
  2. D2-C718ConfirmedDeciding what goes in the index?Said on stage
  3. D2-C707SourceDeciding what goes in the index?Google's documentation
  4. D2-C712SourceDeciding what goes in the index?Google's documentation
  5. D2-C711ConsistentDeciding what goes in the index?Said on stage
  6. D2-C706UndocumentedDeciding what goes in the index?Said on stage
  7. D2-C714UndocumentedDeciding what goes in the index?Said on stage
  8. D2-C708n/aDeciding what goes in the index?The author's interpretation
  9. D2-C716n/aDeciding what goes in the index?The author's interpretation
  10. D1-C508ConfirmedHow crawling errors affect SearchSaid on stage

Claim IDs refer to the Search Central Live 2026 knowledge base. The knowledge base (opens in a new tab)

Confirmed
Google's documentation states the same thing.
Consistent
Google's documentation supports it without stating it.
Undocumented
Said or shown at the event, not in the documentation.
Source
The claim is the cited document itself.
n/a
Nothing to verify: a quotation, framing, opinion or analysis.

Audit

Check it on your site.

One audit checks a URL against all 138 requirements and returns one finding for each, with its source. One URL decides 66; for each of the others, the finding says which source to connect.

Audit a URL freeBrowse all 138