# AI crawlers (firm's decision, 2026-10-25; Cloudflare set up this way since 2026-10-26): # - AI search and assistant crawlers (OAI-SearchBot, ChatGPT-User, # Claude-SearchBot, Claude-User, PerplexityBot, Perplexity-User…) fall under # "*" and may read the site, so it can be cited in AI answers. # - Crawlers that only collect content to train AI models are listed below and # asked to stay out. Cloudflare blocks them one by one in AI Crawl Control. # - Cloudflare → Security → Settings → AI bot policies: Search, Agent and # Training are all set to Allow. # WARNING: never set Cloudflare's "Training" policy to Block. Until 2026-10-26 # it also turned away Googlebot, Bingbot and Applebot, which keeps the site out # of Google, Bing and Apple search. User-agent: GPTBot User-agent: ClaudeBot User-agent: anthropic-ai User-agent: Google-Extended User-agent: Applebot-Extended User-agent: CCBot User-agent: Bytespider User-agent: Amazonbot User-agent: meta-externalagent User-agent: Meta-ExternalAgent User-agent: cohere-ai User-agent: cohere-training-data-crawler User-agent: Diffbot User-agent: DeepSeekBot User-agent: AI2Bot User-agent: Ai2Bot-Dolma User-agent: Timpibot User-agent: ImagesiftBot User-agent: omgili User-agent: omgilibot Disallow: / User-agent: * Allow: / Sitemap: https://www.guerrabravolaw.com/sitemap-index.xml