aiHitBot
aiHitBot is blocked by 94 sites in the October 2026 survey, or 1.5% of readable files. The community reference lists its operator as aiHit.
Who runs aiHitBot?
| Fact | aiHitBot reference entry |
|---|---|
| Operator | “aiHit” |
| Purpose | “A massive, artificial intelligence/machine learning, automated system.” |
| Does aiHitBot obey robots.txt? | “Yes” |
| Frequency | “No information provided.” |
| Cloudflare managed list for aiHitBot | No |
These facts quote the stored community ai.robots.txt reference.
“Scrapes data for AI systems.”
This entry’s stated use is quoted above. An owner may want a page included in search and available on request without offering its contents for future model development.
How many sites block it?
The name occurs in 98 out of 6,171 readable files, a 1.6% share. Across the full sample, the whole-site Disallow: / block count is 94, or 1.5%. Files without that ban number 6,077.
Tranco’s top 1,000 sites yielded 552 readable files, including 23 files with its name, or 4.2%, and 21 files blocking it. Blocking is more common among the leading sites, at 3.8%. Outside those ranks, 73 of 5,619 files block it, or 1.3%.
| Readable files | Named by | Named share | Blocked by | Blocked share |
|---|---|---|---|---|
| aiHitBot, all ranks | 98 | 1.6% | 94 | 1.5% |
| aiHitBot, top 1,000 | 23 | 4.2% | 21 | 3.8% |
| aiHitBot, remaining ranks | 75 | 1.3% | 73 | 1.3% |
Only whole-site blocks recorded against this crawler’s name count here. Rules addressed to every crawler are outside this count. Of 94 blocks, 5 do not also appear in our count of files naming it. The naming total includes 89 files with a ban on every path and 9 files without that ban. Elsewhere, 6,073 files do not name it. We ignore capitalization and repeated names.
How to block or allow it
Use your site’s root file, robots.txt. Specific groups for aiHitBot take precedence over User-agent: *, according to the robots.txt standard.
Block the whole site
User-agent: aiHitBot
Disallow: /Allow access
User-agent: aiHitBot
Allow: /It's the 61st most-named crawler of the 171 we counted in the crawler index.
How we counted
For aiHitBot, our 19 October 2026 collection used Tranco’s top 10,000 sites. Files had an 8-second timeout and a 262,144-byte limit. Missing, unreachable and oversized files do not contribute to these shares. Reading files again used scripts/robots_2026_08_06.py (robots.py at app commit 07104ff) as well. See the full survey method.
Common questions
Does a User-agent line mean it is blocked?
We found 9 files without a whole-site ban among 98 that name it.
What do the top-ranked sites show?
Its name appears in 4.2% of readable leading files and 1.6% overall. Blocking affects 3.8% in the leading group and 1.5% in all ranks.
Which nearby entry can I compare?
For comparison, Factset_spyderbot has 96 files naming it and 92 blocks, or 1.5%. Our counts for this entry are 98 files with the name and 94 blocking the whole site, or 1.5%.