The AI Crawler Access Auditor reads your policy files (robots.txt, ai.txt, meta robots). This tool tests the operational layer: it sends a real HTTP request to your URL using each AI bot's published User-Agent string and reports whether the response was actual content, a Cloudflare/WAF challenge page, or a hard block. If your CDN is silently blocking the bots, this tool will tell you in 15 seconds.
Cloudflare bot management, AWS WAF, Akamai Bot Manager, and similar CDN protections often return a 403 challenge page to non-browser User-Agents by default. AI crawlers cannot solve the JavaScript challenge, so they read an empty page with noindex,nofollow and walk away. Your robots.txt and ai.txt directives are irrelevant if the bot never reaches them. Read the full explanation: Cloudflare is blocking AI crawlers from your site. Here's how to fix it.
Probes are sequential to be polite to your origin (one bot at a time, ~1s apart). Total run takes about 15 seconds. Each probe goes through the jwatte.com Netlify proxy, which sends the bot's actual UA string but originates from Netlify IPs — so this is a UA-only test. If your WAF also IP-allowlists Cloudflare's verified-bot ranges, those checks will only pass when the request actually originates from a Cloudflare-verified IP, which this tool cannot fully simulate. Treat ALLOWED here as "UA-only allow" and verify with server logs.