Robots.txt Generator

Build a correct robots.txt with presets for WordPress and WooCommerce, AI-bot controls and sitemap lines.

About this tool

Pick a preset, add your rules, choose which AI crawlers to block and add your sitemap. The tool outputs a clean robots.txt ready to upload.

How to use the Robots.txt Generator

Pick a preset.
Add custom rules and your sitemap.
Choose AI bots to block (optional).
Upload as /robots.txt and test it.

Frequently asked questions

Does Disallow stop indexing?

No. A blocked URL can still be indexed without content if other sites link to it. Use noindex for that.

Is Crawl-delay supported?

Bing honors it; Google ignores it.

Related tools