Free Tool
Robots.txt Generator
Toggle a few options. Copy the file. Upload to your site root.
Allow search engines to crawl your siteTurn off only if you want to hide your entire site from Google
Block AI training botsPrevents GPTBot, ClaudeBot, CCBot and others from scraping your content for AI training
Block /wp-admin/For WordPress sites. Keeps admin pages out of search results
Optional. Tells Google where your sitemap lives.
Your robots.txt
User-agent: * Allow: /
Save this as robots.txt in your website's root directory.
Want to check if your current robots.txt is set up correctly?
Run Free Audit →