Skip to content
regensbot
Audit a URL freeSign in

RequirementsSearch Central Live Deep Dive Europe 2026

138 requirements, each with its sources.

The developer kit of Search Central Live Deep Dive Europe 2026, in 16 areas. Each requirement says what to do, why it matters, how to meet it and how to test it, and cites the Google documents and claims it rests on.

By level

  • Must37
  • Should67
  • May15
  • Avoid19

Coverage with every source connected

  • Automatic105
  • Assisted29
  • Guided4

Pack v2.13.0Data through 2026-10-02The knowledge base (opens in a new tab)

SRV01 / 16

Server, status codes and crawling

11 requirements

How the server answers Google's crawlers decides whether anything else on the site matters. These requirements cover reachability through firewalls and CDNs, status codes for outages and overload, robots.txt on every host, HTTP versions and caching headers.

  1. DEV-SRV-01MustLet verified Google crawlers through firewalls, CDNs and bot protectionAssistedServer logs
  2. DEV-SRV-02AvoidNever serve bot challenges or error pages to crawlers with a 200 statusAutomaticOne URL
  3. DEV-SRV-03MustAnswer planned maintenance and short outages with 503, for a day or two at mostAssistedYour site
  4. DEV-SRV-04MustKeep robots.txt reachable on every host: 200 with rules, or 404 if there are noneAutomaticOne URL
  5. DEV-SRV-05MustRely only on the fields Google supports (user-agent, allow, disallow, sitemap) and keep robots.txt under 500 KiBAutomaticOne URL
  6. DEV-SRV-06AvoidDo not put Content-Signal or other non-standard lines between user-agent linesAutomaticOne URL
  7. DEV-SRV-07MustAnswer crawlers over HTTP/1.1 (HTTP/2 optional); never run a crawlable host on HTTP/3 onlyAutomaticOne URL
  8. DEV-SRV-08ShouldSupport conditional requests with ETag or Last-Modified and answer 304 when nothing changedAutomaticOne URL
  9. DEV-SRV-09AvoidDo not answer crawlers with 401, 402 or 403 on pages that should be in SearchAutomaticGoogle
  10. DEV-SRV-10MustGive a crawler its own robots.txt group only when needed, and make that group completeAutomaticOne URL
  11. DEV-SRV-11AvoidNever use robots.txt to hide private paths; protect them with authenticationAutomaticOne URL

REN03 / 16

Rendering and JavaScript

10 requirements

Google renders pages with a headless Chromium in a separate, queued system and indexes what is in the DOM when rendering ends. Content and links that depend on clicks, scrolling, blocked resources, permission prompts or slow APIs can be missing from what Google indexes.

  1. DEV-REN-01ShouldServer-render the main content and everything indexing depends on (SSR, static generation or hybrid)AutomaticRendering
  2. DEV-REN-02MustHave all indexable content in the DOM after load, without clicks, typing or scrollingAutomaticRendering
  3. DEV-REN-03MustLazy-load with native lazy loading or an IntersectionObserver, never with scroll eventsAutomaticRendering
  4. DEV-REN-04AvoidDo not block JavaScript, CSS or API endpoints that pages need for rendering in robots.txtAutomaticRendering
  5. DEV-REN-05AvoidDo not add, change or remove robots meta tags with JavaScriptAutomaticRendering
  6. DEV-REN-06ShouldMake rendering robust: no slow API dependencies and no unresolved placeholdersAutomaticRendering
  7. DEV-REN-07ShouldKeep prices, offer counts and other commercial data identical in raw and rendered HTMLAutomaticRendering
  8. DEV-REN-08ShouldFeature-detect permission-based APIs and WebGL, and render the content without themAutomaticRendering
  9. DEV-REN-09ShouldMonitor JavaScript console errors and Content Security Policy violations on key templatesAutomaticRendering
  10. DEV-REN-10ShouldKeep indexable content out of iframes, and check web components in the rendered HTMLAutomaticRendering

IDX04 / 16

Indexing and snippet controls

13 requirements

robots.txt controls crawling; the robots meta tag and the X-Robots-Tag header control indexing and how a page appears, including how much of it AI Overviews and AI Mode may use. Search Console and the Google-Extended token add controls for generative AI.

  1. DEV-IDX-01MustUse noindex to keep a page out of Search, and leave that URL crawlableAutomaticOne URL
  2. DEV-IDX-02AvoidDo not use noindex, nofollow or crawl-delay to save crawl budgetAutomaticServer logs
  3. DEV-IDX-03ShouldSend robots rules for PDFs, images and other non-HTML files in the X-Robots-Tag headerAutomaticOne URL
  4. DEV-IDX-04ShouldAllow full snippets and large previews with max-snippet:-1 and max-image-preview:largeAutomaticOne URL
  5. DEV-IDX-05AvoidDo not add nosnippet or a short max-snippet unless you accept losing snippets and AI feature useAutomaticOne URL
  6. DEV-IDX-06ShouldPrefer rel attributes on individual links to a page-level nofollowAutomaticOne URL
  7. DEV-IDX-07MayUse data-nosnippet on a span, div or section to keep one detail out of snippetsAutomaticOne URL
  8. DEV-IDX-08MayUse unavailable_after on pages with a known end dateAutomaticOne URL
  9. DEV-IDX-09ShouldLeave notranslate and noimageindex out of templates unless translation or image indexing must be preventedAutomaticOne URL
  10. DEV-IDX-10MayUse the Search generative AI setting in Search Console only to take a property out of AI Overviews and AI ModeGuidedYour site
  11. DEV-IDX-11MayUse the Google-Extended robots.txt token to control Gemini training and groundingAutomaticOne URL
  12. DEV-IDX-12ShouldOn sites with explicit content, keep it on its own subdomain or domain and label it with rating markupAssistedYour site
  13. DEV-IDX-13ShouldFor an urgent removal, use the Removals tool in Search Console and also return 404 or noindexAssistedYour site

CAN05 / 16

Canonicals, redirects and duplicates

11 requirements

Google clusters duplicate pages, picks one representative URL (the canonical) and forwards signals to it. Redirects, rel=canonical, internal links and sitemaps tell it which URL you prefer: when they agree Google follows them, and when they conflict it decides on its own.

  1. DEV-CAN-01MustUse permanent server-side redirects (301 or 308) for moved URLs, straight to the final URLAutomaticOne URL
  2. DEV-CAN-02MustServe one host over HTTPS with a valid certificate and redirect every other origin to itAutomaticOne URL
  3. DEV-CAN-03MustPut exactly one absolute rel=canonical in the server-rendered head of every indexable pageAutomaticOne URL
  4. DEV-CAN-04MustPoint every canonical directly at a final URL that returns 200, is indexable and is linkedAutomaticCrawl
  5. DEV-CAN-05MustMake all canonical signals agree: redirects, rel=canonical, internal links, sitemaps and hreflangAutomaticCrawl
  6. DEV-CAN-06AvoidDo not canonicalise one language version to another; connect them with hreflangAutomaticOne URL
  7. DEV-CAN-07AvoidNever leave staging, preview or mirror copies of the site publicly crawlableAutomaticOne URL
  8. DEV-CAN-08ShouldMake pages that share a URL pattern differ in their main contentAutomaticCrawl
  9. DEV-CAN-09ShouldKeep migration redirects in place long term and judge a move in Search Console, not with site: queriesAssistedYour site
  10. DEV-CAN-10MustRedirect an old URL only to a page with the same intent, and return 404 or 410 when there is noneAssistedYour site
  11. DEV-CAN-11AvoidNever route a crawlable URL's redirects through a URL that robots.txt blocksAutomaticOne URL

ERR06 / 16

Error pages and soft 404s

4 requirements

A page that should be an error but returns 200 is a soft 404: Google keeps it out of the index but keeps crawling it, may cluster many such pages together, and recognises it from the main content rather than from keywords. Every response needs the status code that matches the state of the page.

  1. DEV-ERR-01MustReturn 404 or 410 for removed and non-existent URLsAutomaticOne URL
  2. DEV-ERR-02MustReturn a real 404 for unknown routes in single-page apps, or use a documented client-side fallbackAutomaticRendering
  3. DEV-ERR-03AvoidNever serve error states, empty results or failed data loads with a 200 statusAutomaticOne URL
  4. DEV-ERR-04ShouldKeep temporarily out-of-stock product pages live with 200 and their current availabilityAutomaticOne URL

INT07 / 16

International and multilingual sites

12 requirements

Google reads a page's language from its content, stores one language per page and uses hreflang to show searchers the right version. Each version needs its own URL, crawlable links to its alternates, valid and reciprocal annotations and content that really differs.

  1. DEV-INT-01MustGive every language and country version its own URLAutomaticOne URL
  2. DEV-INT-02AvoidDo not redirect visitors automatically by IP address or browser languageAutomaticOne URL
  3. DEV-INT-03MustMake hreflang complete: a self-reference, return links and fully qualified URLsAutomaticCrawl
  4. DEV-INT-04MustUse valid hreflang codes: an ISO 639-1 language, an optional ISO 15924 script and an optional ISO 3166-1 Alpha-2 regionAutomaticOne URL
  5. DEV-INT-05ShouldSupply hreflang through one method only: HTML head, HTTP Link header or sitemapAutomaticOne URL
  6. DEV-INT-06MustBuild language and market selectors as <a href> links to each alternate URLAutomaticRendering
  7. DEV-INT-07ShouldMake each page's language unambiguous by translating the template with the main contentAutomaticOne URL
  8. DEV-INT-08ShouldDifferentiate same-language country versions beyond boilerplate and mark them with region codesAutomaticCrawl
  9. DEV-INT-09MayChoose ccTLDs, subdomains or subdirectories by business needs, and never URL parametersAutomaticOne URL
  10. DEV-INT-10ShouldPublish machine-translated language versions only after human review and localisationAssistedCrawl
  11. DEV-INT-11ShouldWrite pages in the words, spellings and script the audience searches with, and never add synonym or variant listsAssistedYour site
  12. DEV-INT-12ShouldMark the text direction of right-to-left and mixed-script titles, product names and URLsAssistedYour site

HTM08 / 16

Page structure and HTML

7 requirements

Google parses each page into a DOM, tells header, navigation, main content and footer apart, and weighs words by where they appear. Templates decide whether the main content is easy to find and whether controls in the HTML work as intended.

  1. DEV-HTM-01ShouldWrap each page's primary content in one clearly delimited main areaAutomaticOne URL
  2. DEV-HTM-02ShouldPlace the words a page should rank for in its main content, not in footers or sidebarsAssistedYour site
  3. DEV-HTM-03MustGive every indexable page a unique, descriptive title element in the server HTMLAutomaticCrawl
  4. DEV-HTM-04ShouldUse semantic, accessible HTML: real buttons, labelled forms and a logical heading orderAssistedYour site
  5. DEV-HTM-05ShouldWrite valid HTML and close every elementAutomaticOne URL
  6. DEV-HTM-06AvoidNever hide text from users, such as white-on-white or off-screen keyword blocksAssistedRendering
  7. DEV-HTM-07MayUse real heading, title and emphasis elements instead of styling aloneAutomaticRendering

SDA09 / 16

Structured data

15 requirements

Structured data makes pages eligible for rich results, gives Google precise facts such as prices, dates and identifiers more cheaply and accurately than model-based extraction, and the same processed data feeds classic results and AI features. It must describe what users can see.

  1. DEV-SDA-01ShouldWrite structured data as JSON-LD by defaultAutomaticOne URL
  2. DEV-SDA-02MustMark up only content that is visible on the page and relevant to itAssistedYour site
  3. DEV-SDA-03ShouldChoose the specific types Google documents for its features, and skip generic or irrelevant onesAutomaticOne URL
  4. DEV-SDA-04ShouldIdentify entities with unique identifiers: url, @id and stable IDsAutomaticCrawl
  5. DEV-SDA-05ShouldWrite dates in ISO 8601 and give every date-time a UTC offsetAutomaticOne URL
  6. DEV-SDA-06ShouldEmit each structured data entity once per page, and audit CMS plug-ins for duplicatesAutomaticOne URL
  7. DEV-SDA-07ShouldDescribe only the page's main product in Product markup, not related-product carouselsAutomaticOne URL
  8. DEV-SDA-08ShouldMark up merchant listings completely, with validity dates for sale pricesAutomaticOne URL
  9. DEV-SDA-09ShouldDeclare return, shipping and loyalty policies once at organisation levelAutomaticOne URL
  10. DEV-SDA-10ShouldPut the same structured data on duplicate URLs as on the canonicalAutomaticOne URL
  11. DEV-SDA-11MustValidate markup in the Rich Results Test before it reaches templates, and monitor it after releaseAssistedCI
  12. DEV-SDA-12MayWire Google's SHACL validation rules into the build once they are publishedGuidedYour site
  13. DEV-SDA-13ShouldAdd Article markup with headline, images, dates and authors to every article templateAutomaticOne URL
  14. DEV-SDA-14AvoidDo not mark up self-serving review ratings of your own businessAutomaticOne URL
  15. DEV-SDA-15ShouldState the preferred site name with WebSite markup on the home pageAutomaticOne URL

SHP10 / 16

Shopping data (Merchant Center, product markup and Storebot)

4 requirements

Product rich results come from structured data, the Shopping tab needs a Merchant Center feed, and Google Shopping's own crawler, Storebot-Google, checks both against the shop. AI shopping experiences ask for richer product data than classic results, delivered first through the feed.

  1. DEV-SHP-01ShouldUse Product markup for rich results and a Merchant Center feed for the Shopping tab, with identical valuesAutomaticGoogle
  2. DEV-SHP-02ShouldLet Storebot-Google crawl product, cart and checkout pages, through robots.txt and bot protectionAutomaticServer logs
  3. DEV-SHP-03ShouldSend conversational product data in the Merchant Center feed: Q&A, product details, highlights and related productsAutomaticGoogle
  4. DEV-SHP-04MayTreat product Q&A and specification markup in schema.org as an extra on top of the feedAutomaticOne URL

IMG11 / 16

Images

8 requirements

Google finds images through img elements in the HTML and through image sitemaps, understands them through alt text and the surrounding text, and can show them in web search, Google Images, Discover and AI features.

  1. DEV-IMG-01MustPut every image that should be found in an <img> element with a src attributeAutomaticRendering
  2. DEV-IMG-02MustDescribe every meaningful image in its alt attributeAutomaticOne URL
  3. DEV-IMG-03ShouldPlace images next to text that explains themAssistedYour site
  4. DEV-IMG-04ShouldList important images in an image sitemapAutomaticOne URL
  5. DEV-IMG-05ShouldServe modern, compact image formats with a widely supported file in the img srcAutomaticOne URL
  6. DEV-IMG-06ShouldKeep images out of Search with robots.txt or X-Robots-Tag, not with CSS tricksAutomaticOne URL
  7. DEV-IMG-07ShouldGive every article a large, relevant lead image and declare it in og:image or markupAutomaticOne URL
  8. DEV-IMG-08MayKeep the IPTC and C2PA metadata that marks AI-generated imagesAssistedYour site

VID12 / 16

Video

6 requirements

Google extracts videos from the page together with the data around them and passes them to its media indexing. A dedicated watch page, a video element that loads without interaction and VideoObject markup make a video eligible for video features such as key moments.

  1. DEV-VID-01MustEmbed each video in a video, iframe, embed or object element that loads without user actionAutomaticRendering
  2. DEV-VID-02ShouldGive each important video its own watch pageAssistedGoogle
  3. DEV-VID-03ShouldDescribe videos with VideoObject markup and mark key moments with Clip or SeekToActionAutomaticOne URL
  4. DEV-VID-04ShouldList videos in a video sitemapAutomaticOne URL
  5. DEV-VID-05MayAllow video previews with max-video-preview:-1AutomaticOne URL
  6. DEV-VID-06ShouldServe videos fast in a common container with standard codecs, and never block the filesAutomaticOne URL

PRF13 / 16

Performance and server capacity

4 requirements

Speed matters to crawling as much as to users: Google adjusts how much it crawls a host to its connect time, time to first byte and error rate, and that capacity is shared by every Google crawler. Fast rendering keeps content from timing out.

  1. DEV-PRF-01ShouldKeep connect time and time to first byte low and stable under crawler loadAutomaticServer logs
  2. DEV-PRF-02ShouldMake JavaScript-generated content render quicklyAutomaticOne URL
  3. DEV-PRF-03MayLoad per-session-billed third-party tools only after a user interactionAutomaticOne URL
  4. DEV-PRF-04ShouldKeep each HTML response well under 2 MB and put the main content and JSON-LD earlyAutomaticOne URL

AIF14 / 16

AI features (AI Overviews, AI Mode, Gemini)

6 requirements

AI Overviews and AI Mode use the same crawling and the same index as Search, and query fan-out sends generated queries to that index and gets documents back with their snippets. There is no separate AI index, markup or file to build for; the controls for AI features are covered in the indexing controls above.

  1. DEV-AIF-01MustKeep pages indexable and snippet-eligible, the whole technical requirement for AI featuresAutomaticOne URL
  2. DEV-AIF-02ShouldSpend no effort on llms.txt, AI text files or AI-specific markup for GoogleAutomaticOne URL
  3. DEV-AIF-03ShouldStructure content for readers instead of splitting it into small chunks for AIAssistedCrawl
  4. DEV-AIF-04ShouldDo not block the AI agents you want as visitorsAssistedServer logs
  5. DEV-AIF-05MaySet robots.txt policy per AI crawler by its own token, and know that user-triggered fetchers ignore itAutomaticOne URL
  6. DEV-AIF-06MayTry WebMCP to declare the actions a site offers to AI agentsAutomaticRendering

SPM15 / 16

Spam policies and AI-generated content

5 requirements

Google's spam policies cover practices that deceive users or manipulate Search, including its generative AI responses, and several of them are built in code: history manipulation, cloaking, generated page sets and hacked pages. AI is a tool; how its output is used decides whether content is helpful or spam.

  1. DEV-SPM-01AvoidNever manipulate browser history so that Back does not return users to where they came fromAutomaticRendering
  2. DEV-SPM-02AvoidDo not generate large sets of low-value pages from templates, feeds or LLM outputAssistedCrawl
  3. DEV-SPM-03AvoidNever serve Googlebot different content from users, and do not build doorway pagesAutomaticOne URL
  4. DEV-SPM-04MustKeep the CMS and plug-ins patched and watch for injected pages and linksAssistedCrawl
  5. DEV-SPM-05ShouldPublish AI-generated text, images and markup only after human review, and show only real authorsAssistedCrawl

MON16 / 16

Testing and monitoring

11 requirements

Search Console shows what Google crawled, rendered, chose as canonical and indexed. Render one URL per template in a headless browser before every release, check it in URL Inspection right after, and watch the reports that reveal crawling, rendering and index selection problems.

  1. DEV-MON-01MustVerify every production host in Search Console before launchAssistedGoogle
  2. DEV-MON-02MustRender every template in a headless browser before release and check it in URL Inspection right afterAssistedCI
  3. DEV-MON-03ShouldWatch the Page indexing report and treat its not-indexed reasons as a work listAssistedGoogle
  4. DEV-MON-04ShouldMonitor Crawl stats and server logs for verified Googlebot trafficAssistedServer logs
  5. DEV-MON-05ShouldRun recurring rendering checks: raw versus rendered HTML, placeholders and console errorsAutomaticRendering
  6. DEV-MON-06ShouldAudit canonical groups with a crawler on a scheduleAutomaticCrawl
  7. DEV-MON-07MayTrack AI feature visibility in the Generative AI performance reportGuidedYour site
  8. DEV-MON-08ShouldSet up the Search Console bulk data export to BigQuery on large sitesAutomaticGoogle
  9. DEV-MON-09MayAdd the brand's YouTube, Instagram, TikTok and X accounts to Search Console as platform propertiesGuidedYour site
  10. DEV-MON-10ShouldPlan releases and fixes around Google's processing times, and judge results only after themAssistedYour site
  11. DEV-MON-11ShouldMeasure the time from publishing a URL to Googlebot's first crawl of itAutomaticServer logs

Audit

Check it on your site.

One audit checks a URL against all 138 requirements and returns one finding for each, with its source. One URL decides 66; for each of the others, the finding says which source to connect.

Audit a URL free