Shopify storefront. Public product, collection, page, blog, policy, cart, and localized HTML is crawlable.
User-agent: * Allow: / Allow: /products/account Allow: /products/orders Allow: /products/checkout Allow: //products/account Allow: //products/orders Allow: //products/checkout Allow: /collections/account Allow: /collections/orders Allow: /collections/checkout Allow: //collections/account Allow: //collections/orders Allow: //collections/checkout Allow: /pages/checkout Allow: /*/pages/checkout Allow: /blogs/*account Allow: /blogs/*orders Allow: /blogs/checkout Allow: //blogs/account Allow: //blogs/orders Allow: //blogs/*checkout
Private / transactional
Disallow: /admin Disallow: /cart/ Disallow: //cart/ Disallow: /checkout Disallow: //checkout Disallow: /checkouts/ Disallow: //checkouts/ Disallow: /orders Disallow: //orders Allow: /account/login Allow: //account/login Disallow: /account Disallow: //account Disallow: /52539064489 Disallow: /cdn/wpm/*.js
Shopify-internal endpoints not meant for crawlers
Disallow: /services Disallow: /sf_*
AJAX surfaces: agents should use UCP/MCP instead
Disallow: /cart.js Disallow: //cart.js Disallow: /recommendations/products Disallow: //recommendations/products
Filters, sort, previews, language-picker crawl traps
Disallow: /collections/sort_by Disallow: //collections/sort_by Disallow: /collections/+* Disallow: /collections/%2B Disallow: /collections/%2b Disallow: //collections/+* Disallow: //collections/%2B* Disallow: //collections/%2b* Disallow: /collections/filter&filter Disallow: //collections/filter&filter Disallow: /blogs/+* Disallow: /blogs/%2B Disallow: /blogs/%2b Disallow: //blogs/+* Disallow: //blogs/%2B* Disallow: //blogs/%2b* Disallow: /?ls=&ls= Disallow: /?ls%3ls%3 Disallow: /?oseid= Disallow: /?preview_theme_id= Disallow: /*?preview_script_id=
Google adsbot ignores robots.txt unless specifically named, some rules must be repeated.
User-agent: adsbot-google Allow: /products/ Allow: //products/ Allow: /collections/ Allow: //collections/ Allow: /pages/ Allow: //pages/ Allow: /blogs/ Allow: //blogs/ Allow: /pages/checkout Allow: //pages/checkout Allow: /blogs/checkout Allow: //blogs/checkout Disallow: /checkout Disallow: //checkout Disallow: /checkouts/ Disallow: //checkouts/ Disallow: /orders Disallow: //orders Disallow: /services Disallow: /sf_ Disallow: /52539064489 Disallow: /cdn/wpm/*.js