Crawler access score

The crawler access score turns a robots.txt audit into a number. Start at 100 and deduct: −15 for each critical crawler blocked (Googlebot, GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot), −5 for each secondary crawler, and −10 if no sitemap is declared. A low score caps all downstream AI visibility, because access is the gate before content.

First used in: 3.3 · AI crawler access