Segonzac: How AI Search Agents see the Web

Key Takeaways:

• Segonzac analyzed the behavior of AI Search Agents on how they view websites

  • The Good

     - Static HTML is visible to everyone (your safest bet)

     - <noscript> fallbacks work for ChatGPT, Claude & Gemini

  • The Surprising

     - Only 3 crawlers execute JavaScript​: Bing Copilot, Grok & DeepSeek

     - Bing Copilot is the ONLY one that can read iframes and Shadow DOM

     - Some crawlers identify themselves honestly (ChatGPT, Claude, Gemini), others operate in full stealth mode (Bing, Grok)

  • The Broken

     - Perplexity can't decode compressed responses (empty Accept-Encoding headers)

     - Google AI Mode doesn't fetch URLs at all, it only uses Google's existing index

     - Content loading after 2 seconds? Invisible to everyone

• GEO Takeaways:

  1. Put critical content in static HTML

  2. Use <noscript> fallbacks for JS-dependent content

  3. Avoid Shadow DOM for important information

  4. Get indexed by Google

  5. Disable compression for Perplexity-User

Sources:

• https://www.linkedin.com/posts/resoneo_we-tested-how-ai-assistants-actually-see-activity-7415040888088862720-pL2n