SEO & Content
Robots.txt Generator (with AI bots)
Create a robots.txt file in seconds. Choose whether AI companies may use your content for training and whether AI search assistants may read it to answer questions.
Runs entirely in your browser — nothing you enter is sent or stored.
How it works
Training crawlers collect pages to train future models; search and assistant bots fetch pages to answer a user's question right now and usually link back. Blocking the second group can remove you from AI answers, so most publishers block training (if at all) and keep search. robots.txt is a request that reputable bots honour — it is not access control.
Frequently asked questions
Where do I put robots.txt?
At the root of your domain: https://yourdomain.com/robots.txt. WordPress serves a virtual one; SEO plugins let you edit it.
Does blocking Google-Extended hurt my Google rankings?
No. Google-Extended only controls use of your content for Gemini models; normal Googlebot crawling and ranking are unaffected.