Most sites that “block AI” are blocking the wrong bot. I spent a year disallowing GPTBot and thinking I had opted out of ChatGPT, while the user agent that could fetch a page for a buyer’s live question was a different one entirely. Bot…
| Field | Source |
|---|---|
url | customer_site |
name | customer_site |
image | customer_site |
headline | customer_site |
description | customer_site |
dateModified | customer_site |
datePublished | customer_site |
{
"@context": "https://schema.org",
"@id": "https://graph.groas.com/entity/the-ai-crawler-swipe-file-rules-and-page",
"url": "https://groas.com/post/the-ai-crawler-swipe-file-copy-paste-rob",
"name": "The AI Crawler Swipe File: Rules and Page Blocks for Live-Answer Citations",
"@type": "Article",
"image": "https://groas.com/media/blog/2b1aadd4ff0dcc345b73d978e89f5c1e37c44b06a92d1ac8bf09722b93507727.png",
"headline": "The AI Crawler Swipe File: Rules and Page Blocks for Live-Answer Citations",
"description": "Most sites that “block AI” are blocking the wrong bot. I spent a year disallowing GPTBot and thinking I had opted out of ChatGPT, while the user agent that could fetch a page for a buyer’s live question was a different one entirely. Bot…",
"dateModified": "2026-11-10T21:24:53.016Z",
"datePublished": "2026-10-04T05:34:36.479Z"
}