Cloudflare Block Settings Now Hit Googlebot: Disallow AI Training Keeps Search

Effective September 15, 2026, Cloudflare's Block and "Block on pages with ads" settings cover mixed-use crawlers such as Applebot, Bingbot, and Googlebot. Either setting now affects search as well as AI training, so a block aimed at trainers can cost you indexing.

To stop training while keeping search, you must select the new Disallow AI Training setting. It publishes the preference in robots.txt and blocks every other training crawler, including training-only crawlers from Amazon, Anthropic, Meta, and OpenAI. Cloudflare deprecated "Block AI Bots" in favor of granular Search, Training, and Agent controls. No Disallow setting exists for Agent yet; that waits on ai-prefs maturing.

Cloudflare's stated figures: under 1% of its sites block Search bots, while 17% enable some mechanism to block training. It also names Apple, Google, and Microsoft as meeting its "Accountable" designation, which is Cloudflare's own label, not an independent audit.

Check which state your domain landed in. Legacy "Block AI Bots" domains migrate to Search: Allow plus Disallow AI Training, so they keep search traffic; verify yours. Mixed-use crawlers fall under the new controls only if you select Block or "Block on pages with ads", the choice that can cost you Googlebot and Bingbot. Bing will not honor the Disallow AI Training preference in robots.txt until Microsoft ships support, targeted for early 2027. Want them gone entirely? Select Block.