groas Knowledge Graph / Article

The AI Crawler Swipe File: Rules and Page Blocks for Live-Answer Citations

Most sites that “block AI” are blocking the wrong bot. I spent a year disallowing GPTBot and thinking I had opted out of ChatGPT, while the user agent that could fetch a page for a buyer’s live question was a different one entirely. Bot…

Stable URI: https://graph.groas.com/entity/the-ai-crawler-swipe-file-rules-and-page
Canonical page: https://groas.com/post/the-ai-crawler-swipe-file-copy-paste-rob
Last verified: 2026-11-16T07:01:32.062Z · Raw JSON-LD: the-ai-crawler-swipe-file-rules-and-page.json

Provenance

FieldSource
urlcustomer_site
namecustomer_site
imagecustomer_site
headlinecustomer_site
descriptioncustomer_site
dateModifiedcustomer_site
datePublishedcustomer_site

JSON-LD

{
  "@context": "https://schema.org",
  "@id": "https://graph.groas.com/entity/the-ai-crawler-swipe-file-rules-and-page",
  "url": "https://groas.com/post/the-ai-crawler-swipe-file-copy-paste-rob",
  "name": "The AI Crawler Swipe File: Rules and Page Blocks for Live-Answer Citations",
  "@type": "Article",
  "image": "https://groas.com/media/blog/2b1aadd4ff0dcc345b73d978e89f5c1e37c44b06a92d1ac8bf09722b93507727.png",
  "headline": "The AI Crawler Swipe File: Rules and Page Blocks for Live-Answer Citations",
  "description": "Most sites that “block AI” are blocking the wrong bot. I spent a year disallowing GPTBot and thinking I had opted out of ChatGPT, while the user agent that could fetch a page for a buyer’s live question was a different one entirely. Bot…",
  "dateModified": "2026-11-10T21:24:53.016Z",
  "datePublished": "2026-10-04T05:34:36.479Z"
}