Robots.txt Generator
Build a robots.txt file with clear crawl instructions.
How to use the robots.txt generator
Enter one disallowed path per line and an optional sitemap URL to produce a simple robots.txt file.
Example
Enter /admin and /private to request that crawlers avoid those paths.
Good to know
Robots.txt is a crawl instruction, not access control. Do not use it to protect private files.