robots.txt tester
Paste robots.txt, enter a user-agent and URL, then check whether a search crawler can read that path before publishing crawl rules.
robots.txt test
Paste robots.txt, then enter the crawler and URL you want to test.
Test robots.txt rules before publishing
A single robots.txt line can block an important page or leave a path open that you meant to block. When several User-agent groups, Allow rules, and Disallow rules are mixed together, the actual result can be hard to judge by sight.
This tool finds the matching user-agent group in your pasted robots.txt and compares its rules with the path from the test URL. It follows the common search engine interpretation where the longest matching pattern wins, and Allow wins when patterns are the same length.
The test runs only inside your browser. robots.txt content, user-agent, and URLs are not sent to berryfy. Search engines can differ in small details, so confirm important production changes with search console tools as well.
Related tools
Continue with tools that fit the same task flow.