WARDBot
WARDBot is blocked by 92 sites in the October 2026 survey, or 1.5% of readable files. The community reference lists its operator as WEBSPARK.
Who runs WARDBot?
| Fact | WARDBot reference entry |
|---|---|
| Operator | “WEBSPARK” |
| Purpose | “AI Data Scrapers” |
| Does WARDBot obey robots.txt? | “Unclear at this time.” |
| Frequency | “Unclear at this time.” |
| Cloudflare managed list for WARDBot | No |
These facts quote the stored community ai.robots.txt reference.
“WARDBot is an AI data scraper operated by WEBSPARK. It's not currently known to be artificially intelligent or AI-related.”
This entry is classed as a crawler collecting data for AI. A requested page visit can answer a person now, whereas training collects material for a model. Owners may apply different access rules.
How many sites block it?
Instructions to stay out of the whole site appear in 92 files, or 1.5% of 6,171 readable files. We counted 92 with its name in a User-agent line, a 1.5% share. Another 6,079 lack a recorded Disallow: / ban.
Across Tranco’s top 1,000 ranks, there were 552 readable files. Among them, 8 files carry the name, or 1.4%, with 8 files blocking the whole site. The leading sites have a lower block share, 1.4%. Farther down, 84 of 5,619 files record a block, 1.5% of that group.
| Readable files | Named by | Named share | Blocked by | Blocked share |
|---|---|---|---|---|
| WARDBot, all ranks | 92 | 1.5% | 92 | 1.5% |
| WARDBot, top 1,000 | 8 | 1.4% | 8 | 1.4% |
| WARDBot, remaining ranks | 84 | 1.5% | 84 | 1.5% |
Only whole-site blocks recorded against this crawler’s name count here. Rules addressed to every crawler are outside this count. Of 92 blocks, 1 does not also appear in our count of files naming it. Looking just at named files gives 91 blocks and 1 without a whole-site block. For 6,079 readable files, we found no mention by name. We ignore capitalization and repeated names.
How to block or allow it
Put these in robots.txt at the site root. Per the robots.txt standard, a group naming WARDBot takes precedence over User-agent: *.
Block the whole site
User-agent: WARDBot
Disallow: /Allow access
User-agent: WARDBot
Allow: /It's the 65th most-named crawler of the 171 we counted in the crawler index.
How we counted
For WARDBot, on 19 October 2026, we requested robots.txt from Tranco’s top 10,000 sites. An 8-second wait or a file over 262,144 bytes ended a read. Missing, unreachable and oversized files do not contribute to these shares. The file readings, including repeats, use scripts/robots_2026_08_06.py (robots.py at app commit 07104ff). See the full survey method.
Common questions
Does a User-agent line mean it is blocked?
The 92 files naming it include 1 file without a whole-site ban.
What do the top-ranked sites show?
Leading files record 1.4% with the name and 1.4% blocking it. Across all readable files, the corresponding shares are 1.5% and 1.5%.
Which nearby entry can I compare?
The nearby Bravebot entry reports 90 files that name it and 81 whole-site blocks, 1.3% of readable files. Here, 92 files mention the name and 92 block access, or 1.5%.