← jwatte.com

robots.txt Simulator

Paste a robots.txt, a list of URLs, and a bot. The simulator applies the User-agent group precedence rules per Google's robots.txt spec (RFC 9309) and reports per-URL allow/disallow.

Context and background

Read the story behind this tool: Why robots.txt Simulator Exists →

📖 Context & background

Read the story behind this tool: Why this tool exists — AEO + crawler simulation.