Cloudflare: Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives

3. Aug. 2025
1 Min. Lesezeit
Aktualisiert: 13. Sept. 2025
Key Takeaways:
Perplexity recrawls websites with stealth, undeclared crawlers when initially blocked from crawling (impersonating Google Chrome on macOS)
Findings based on test of Perplexity crawling behavior
Cloudflare detected 3-6 million undeclared crawler hits daily compared to 20-25 million hits from the official crawler


![ICYMI Metehan: Statistics & Fact Checking in AI Chats and beyond the Web [Experiment]](https://static.wixstatic.com/media/24c590_b0a8e1392c1c47f28a0a46a67d59df71~mv2.jpg/v1/fill/w_980,h_653,al_c,q_85,usm_0.66_1.00_0.01,enc_avif,quality_auto/24c590_b0a8e1392c1c47f28a0a46a67d59df71~mv2.jpg)
