Why it matters
Googlebot fetches only the first 2MB of an HTML URL, HTTP headers included: bytes past that cutoff are not fetched, rendered or indexed. Google warns that bloated inline base64 images, large blocks of inline CSS or JavaScript and megabytes of menus can push a page's text or structured data past the limit, a real risk for single-page apps that inline large hydration state. Scripts, stylesheets and API responses the page loads each have their own limit.
How to meet it
Move base64 images, large inline scripts and styles and serialised application state (hydration JSON) to external files or endpoints, keep navigation markup compact, and place the meta tags, title, canonical, hreflang and essential JSON-LD high in the head, with the main content before long secondary blocks.
How to test it
For every template, curl -s <url> | wc -c stays under about 1.5 MB, and the main heading and the JSON-LD appear within the first 2 MB of the response (curl -s <url> | head -c 2000000 | grep -c 'application/ld+json').
How Regensbot covers it
Decided from one URL.
An audit reads the response, the headers, the HTML, robots.txt, the sitemaps and the structured data, and the finding shows the evidence it saw.
Sources
Google documents02
- 01Inside Googlebot: demystifying crawling, fetching, and the bytes we processSearch Central blog (31 March 2026) (opens in a new tab)
- 02Overview of Google crawlers and fetchers (user agents)Google (opens in a new tab)
Claims in the knowledge base02
- D1-C136SourceHow crawling worksGoogle's documentation
- D1-C137SourceHow crawling worksGoogle's documentation
Claim IDs refer to the Search Central Live 2026 knowledge base. The knowledge base (opens in a new tab)
- Source
- The claim is the cited document itself.