AI agent directory
Every AI agent that visits websites.
44 crawlers, fetchers, AI browsers and personal agents: who runs each one, why it visits, how Airglow identifies it and how to allow or block it.
AI browsers and personal agents
They click, scroll and type for a person, and look like Chrome. Standard analytics counts them as people.
- ChatGPT AtlasOpenAI, acts for a person
- Perplexity CometPerplexity, acts for a person
- Claude in ChromeAnthropic, acts for a person
- Claude desktop browserAnthropic, acts for a person
- Edge Copilot ModeMicrosoft, acts for a person
- DiaThe Browser Company, acts for a person
- Meta MuseMeta, acts for a person
- OpenAI DotsOpenAI, acts for a person
- OpenClawOpenClaw (OSS), acts for a person
- InstinctSpear Street Technology, acts for a person
AI fetchers
They read one page when someone asks an assistant about it, without running JavaScript.
AI crawlers
They crawl pages to train models or build AI search indexes.
- GPTBotOpenAI, training
- OAI-SearchBotOpenAI, search
- ClaudeBotAnthropic, training
- Claude-SearchBotAnthropic, search
- Meta external agentMeta, training
- AmazonbotAmazon, training and search
- BytespiderByteDance, training
- CCBotCommon Crawl, training
- DiffbotDiffbot, training
- cohere-aiCohere, training
- YouBotYou.com, search
- DuckAssistBotDuckDuckGo, acts for a person
- Meta web indexerMeta, search
- PerplexityBotPerplexity, search
- ShapBotParallel Web Systems, search
- Google-ExtendedGoogle, training
Search and SEO crawlers
Not AI agents, but shown next to them so you can compare.
Browser automation
Frameworks and hosted browsers that AI agents and scripts run on.
The list is the same one Airglow uses to label visits, and grows as new agents appear. Missing one? Tell us.