Let Storebot-Google crawl product, cart and checkout pages, through robots.txt and bot protection

Google Shopping crawls with its own user agent, Storebot-Google, because it needs fresh prices, availability and shipping details. It visits product, cart and checkout pages, can fill in checkout forms and records price, shipping, availability, coupons and payment methods to verify the data merchants send. robots.txt rules addressed to Storebot-Google affect all Google Shopping surfaces, and a crawler without its own group follows the * group, so a shop that disallows /cart/ and /checkout/ for every crawler also blocks these checks.

Give Storebot-Google its own robots.txt group that repeats the general rules but leaves cart and checkout paths open; verify it like Googlebot, as one of Google's common crawlers (DEV-SRV-01), and exempt it from rate limits, CAPTCHAs and bot challenges on product, cart and checkout flows. Block it only if the shop deliberately stays out of Google Shopping.

Automatisch entschieden, sobald Sie Ihre Server-Logs verbinden.

regensbot logs push behält nur Zeilen aus den IP-Bereichen, die Google und KI-Agenten veröffentlichen. So verlässt keine Besucher-IP Ihre Systeme. Googlebot wird per IP und DNS verifiziert.

So funktionieren die UrteileWas Sie verbinden