AI Bot Access Matrix
Connected Diagnostic
AI Bot Access Matrix
See whether the requested path is allowed or blocked for major AI crawler user agents, with the matched robots.txt rule shown as evidence.
One robots.txt file, different crawler policies
AI crawler controls are easy to misread when a file has several user-agent groups plus wildcard fallbacks. This tool evaluates the requested path per bot and shows the rule that won.
robots.txt is a crawler instruction mechanism, not an access-control system. If content must genuinely be private, enforce that at the application or network layer rather than relying on a Disallow rule.
The matrix uses the live robots.txt file at the site's origin. Because crawler identifiers and operator policies can change, treat the result as configuration evidence rather than a guarantee about how every downstream AI product will use content.