In 30 days of groas edge logs, Claude-User made 1,071 live-answer fetches for robots.txt, while a dated changelog drew 577 from ChatGPT. Check access, extractability, and content in that order.
| Field | Source |
|---|---|
url | customer_site |
name | customer_site |
image | customer_site |
headline | customer_site |
description | customer_site |
dateModified | customer_site |
datePublished | customer_site |
{
"@context": "https://schema.org",
"@id": "https://graph.groas.com/entity/claude-user-fetched-our-robots-txt-1-071",
"url": "https://groas.com/post/claude-checked-our-robots-txt-1-071-times-in-30-days-the-ai",
"name": "Claude-User Fetched Our robots.txt 1,071 Times: An AI Citation Checklist That Starts With Access",
"@type": "Article",
"image": "https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/88344079-001f-440f-ac2d-ee61f05e06c2.png",
"headline": "Claude-User Fetched Our robots.txt 1,071 Times: An AI Citation Checklist That Starts With Access",
"description": "In 30 days of groas edge logs, Claude-User made 1,071 live-answer fetches for robots.txt, while a dated changelog drew 577 from ChatGPT. Check access, extractability, and content in that order.",
"dateModified": "2026-11-10T21:20:17.481Z",
"datePublished": "2026-10-31T06:16:31.629Z"
}