AI agent directory / AhrefsSiteAudit
AhrefsSiteAudit
Crawler from Ahrefs
Crawls and renders a site's pages for Ahrefs' Site Audit tool, which reports technical and on-page SEO issues.
AhrefsSiteAudit is the crawler behind Ahrefs' Site Audit tool. Ahrefs users point Site Audit at a website, and this bot crawls and renders its pages to find technical and on-page SEO issues.
It is separate from AhrefsBot, the crawler that builds Ahrefs' link index and the Yep search engine. It has its own robots.txt token, and it is not an AI agent.
| Run by | Ahrefs |
|---|---|
| Type | Crawler |
| Purpose | Undeclared |
| Acts for | Its operator |
| robots.txt token | AhrefsSiteAudit |
| IP ranges | Published by Ahrefs |
| Documentation | Ahrefs's documentation |
AhrefsSiteAudit user-agent strings
AhrefsSiteAudit (desktop) (source):
Mozilla/5.0 (compatible; AhrefsSiteAudit/6.1; +http://ahrefs.com/robot/site-audit)
AhrefsSiteAudit (mobile) (source):
Mozilla/5.0 (Linux; Android 13) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/151.0.7922.173 Mobile Safari/537.36 (compatible; AhrefsSiteAudit/6.1; +http://ahrefs.com/robot/site-audit)
Match on AhrefsSiteAudit; the rest of the string changes between versions.
Verifying it's really AhrefsSiteAudit
Ahrefs documents two checks, shared by AhrefsSiteAudit and AhrefsBot:
- IP list. Both bots crawl from published IPs, available as IP ranges or individual IPs. The lists do not say which bot uses which address.
- Reverse DNS. The hostname for a genuine Ahrefs IP always ends in
ahrefs.comorahrefs.net.
A user agent alone proves nothing, because anyone can copy it.
How Airglow identifies it
- DeclaredIt names itself in the User-Agent:
AhrefsSiteAudit.
Every visit in Airglow says which of these applied, so you can decide how much to trust it.
Seeing it on your site
It reads pages without running JavaScript, so Airglow counts it through a server connection (Cloudflare). It's labeled as context, not replayed.
Allowing or blocking AhrefsSiteAudit
AhrefsSiteAudit reads the AhrefsSiteAudit group in your robots.txt. To keep it out of the whole site:
User-agent: AhrefsSiteAudit Disallow: /
Airglow shows, for every agent that visits, whether your robots.txt lets it in, and helps you write rules by purpose (training, search, acting for a person).
AhrefsSiteAudit vs. AhrefsBot
The two bots are controlled separately. A User-agent: AhrefsBot rule does not apply to AhrefsSiteAudit, and the reverse is also true. To block both, add a group for each token.
AhrefsBot crawls the web on its own schedule. AhrefsSiteAudit crawls a site because an Ahrefs user set up an audit of it, which may be the site's owner or someone else.
When it can ignore robots.txt
By default AhrefsSiteAudit obeys robots.txt and Crawl-delay. Ahrefs lets the owner of a site ask it to ignore both on that site, so the owner can audit sections that are normally blocked. Ahrefs says only verified site owners can let it ignore robots.txt.
Site Audit is capped at 30 URLs per minute. Owners who verify their site in Site Audit can crawl it faster.
Why your logs may show bursts
Site Audit renders pages like a browser. When it does, it may fetch several images, scripts and stylesheets at once. Ahrefs says crawl delay is followed for HTML pages but can't be followed when rendering JavaScript, so logs can show more requests than your delay allows.
Questions
What is AhrefsSiteAudit?
It is the crawler for Ahrefs' Site Audit tool. It crawls and renders a website's pages to report technical and on-page SEO problems.
How do I block AhrefsSiteAudit?
Add User-agent: AhrefsSiteAudit and Disallow: / to robots.txt. Blocking AhrefsBot does not block it, so list both tokens if you want neither.
Why is AhrefsSiteAudit crawling my site when I don't use Ahrefs?
Ahrefs users can run Site Audit on websites they want to analyze, not only their own. It still obeys your robots.txt. Only a verified owner of the site can tell it to ignore robots.txt there.
Is AhrefsSiteAudit the same as AhrefsBot?
No. AhrefsBot builds Ahrefs' web index and feeds the Yep search engine. AhrefsSiteAudit only runs audits in the Site Audit tool. Each has its own robots.txt token.
Does AhrefsSiteAudit have a mobile version?
Yes. Ahrefs publishes a desktop user agent and an Android mobile user agent. Both contain AhrefsSiteAudit/6.1 and use the same robots.txt token.
Other agents from Ahrefs
Sources
Last reviewed November 10, 2026.
See when AhrefsSiteAudit visits your site
Airglow names every AI agent on your website, replays what it did and links the visitors it sends you. Free for one website.