AI agent directory / SemrushBot-SI
SemrushBot-SI
Crawler from Semrush
Fetches pages for Semrush's On Page SEO Checker and similar tools.
SemrushBot-SI is the Semrush bot for the On Page SEO Checker and similar tools, such as the SEO Content Template. It fetches the pages a Semrush user wants optimization ideas for.
Semrush's bot page gives only the robots.txt token for this bot, not a full user-agent string.
| Run by | Semrush |
|---|---|
| Type | Crawler |
| Purpose | Undeclared |
| Acts for | Its operator |
| robots.txt token | SemrushBot-SI |
| Documentation | Semrush's documentation |
Verifying it's really SemrushBot-SI
For allowlisting, Semrush lists 85.208.98.53 and 85.208.98.0/24 with the SemrushBot-SI user agent, on ports 80 and 443.
How Airglow identifies it
- DeclaredIt names itself in the User-Agent:
SemrushBot-SI.
Every visit in Airglow says which of these applied, so you can decide how much to trust it.
Seeing it on your site
It reads pages without running JavaScript, so Airglow counts it through a server connection (Cloudflare). It's labeled as context, not replayed.
Allowing or blocking SemrushBot-SI
SemrushBot-SI reads the SemrushBot-SI group in your robots.txt. To keep it out of the whole site:
User-agent: SemrushBot-SI Disallow: /
Airglow shows, for every agent that visits, whether your robots.txt lets it in, and helps you write rules by purpose (training, search, acting for a person).
Blocking one Semrush tool doesn't block the others
Semrush uses a separate robots.txt token for each tool. A group for SemrushBot-SI does not cover SemrushBot, SiteAuditBot or the other tokens on Semrush's bot page. Add a group for each one you want to stop.
Questions
What is SemrushBot-SI?
It is the bot behind Semrush's On Page SEO Checker and similar tools. It fetches a page so the tool can suggest on-page improvements.
How do I block SemrushBot-SI?
Add User-agent: SemrushBot-SI and Disallow: / to the robots.txt in your site's top directory. Semrush documents this token on its bot page.
Why does it ignore my page when I set a crawl delay?
Semrush says On Page SEO Checker crawlers accept a crawl delay of 1 second at most. A higher Crawl-delay makes the crawler skip the page.
Other agents from Semrush
Sources
Last reviewed November 10, 2026.
See when SemrushBot-SI visits your site
Airglow names every AI agent on your website, replays what it did and links the visitors it sends you. Free for one website.