robots.txt Tester

Fetch and test any site's robots.txt file to see if a specific path is allowed or blocked, online, free.

Enter a URL

Path (optional)

User agent (optional)

How to Convert robots.txt Tester

1
Enter the site's URL.
2
Optionally enter a path to test and a user-agent.
3
Click "Check" to fetch robots.txt and see whether the path is allowed.

Frequently Asked Questions

Meta Tag Analyzer SSL Checker