100% Free & Instant

AI Robots.txt Generator

Granularly allow or block specific AI crawlers and search agents.

Paths to Block (Comma separated)

Leave empty to block the entire website (/)for the selected “Block” bots.

1. OpenAI (ChatGPT)

ChatGPT User Browsing

User-agent: ChatGPT-User

GPTBot (LLM Training)

User-agent: GPTBot

OAI-SearchBot

User-agent: OAI-SearchBot

2. Google Gemini

Google Extended

User-agent: Google-Extended

Google AI Training

User-agent: Google-AI

3. Anthropic (Claude)

Anthropic AI

User-agent: Anthropic-AI

4. AI Search Engines & Copilots

Perplexity AI

User-agent: PerplexityBot

Microsoft Bing AI (Copilot)

User-agents: BingPreview / AdIdxBot

Want advanced analysis and a complete technical audit?

Try Full AEO Audit Pro →

robots.txt (Output)

# AI Crawler Directives generated by AEO Audit Pro

# --- OpenAI (ChatGPT) ---
User-agent: ChatGPT-User
Disallow:

User-agent: GPTBot
Disallow:

User-agent: OAI-SearchBot
Disallow:

# --- Google AI & Gemini ---
User-agent: Google-Extended
Disallow:

User-agent: Google-AI
Disallow:

# --- Anthropic (Claude) ---
User-agent: Anthropic-AI
Disallow:

# --- AI Search Engines ---
User-agent: PerplexityBot
Disallow:

User-agent: BingPreview
Disallow:

User-agent: AdIdxBot
Disallow:

# Standard Search Crawlers Configuration
User-agent: *
Allow: /

💡 How to use: Copy this text and add it into the robots.txt file located in the root directory of your website (e.g., yoursite.com/robots.txt).