# AI Crawler robots.txt Policy Builder

Choose which AI crawlers to allow or block and generate a robots.txt file, with a plain-English explanation of what each rule does.

- URL: https://toolstackhub.co.uk/tools/ai-crawler-policy-builder/
- Category: SEO & Marketing Tools
- Price: free, no sign-up
- Privacy: runs in the browser; input is not uploaded
- Last updated: 2026-10-08

## About
Pick a starting policy, add exceptions, and the builder writes a robots.txt covering the best-known AI and search crawlers, with a note on what each one does. It separates training crawlers, AI search crawlers and bots that fetch a page when a user asks. robots.txt is a request, not a lock: well-behaved crawlers follow it and others may not. Blocking Google-Extended does not affect Google Search, which is controlled by Googlebot.

## How to use
1. Paste or type your input into the box above.
2. Choose any options you need. Results update as you type.
3. Copy the result with the Copy button.

## FAQ
**Does blocking GPTBot remove me from ChatGPT search?** No. OAI-SearchBot is a separate crawler for search results. The builder keeps them apart so you can allow search and block training.

**Does Google-Extended affect my search ranking?** No. It is a control token for whether your content can be used by Google's Gemini models. Search is controlled by Googlebot.

**Will every bot obey it?** Reputable operators say they do, but robots.txt is voluntary. It cannot stop a scraper that ignores it.

**Is the AI Crawler robots.txt Policy Builder free?** Yes, with no sign-up or limits.