WRTNBot
WRTNBot is blocked by 68 sites in the October 2026 survey, or 1.1% of readable files. Its operator is unclear in the community reference.
Who runs WRTNBot?
| Fact | WRTNBot reference entry |
|---|---|
| Operator | “Unclear at this time.” |
| Purpose | “Undocumented AI Agents” |
| Does WRTNBot obey robots.txt? | “Unclear at this time.” |
| Frequency | “Unclear at this time.” |
| Cloudflare managed list for WRTNBot | No |
These facts quote the stored community ai.robots.txt reference.
The reference leaves this agent’s purpose undocumented. Helping people find a page through search is a different use from using its contents to train a model, so owners may allow those visits separately.
How many sites block it?
The name occurs in 70 out of 6,171 readable files, a 1.1% share. Across the full sample, the whole-site Disallow: / block count is 68, or 1.1%. Files without that ban number 6,103.
Across Tranco’s top 1,000 ranks, there were 552 readable files. Among them, 21 files carry the name, or 3.8%, with 21 files blocking the whole site. Blocking is more common among the leading sites, at 3.8%. Farther down, 47 of 5,619 files record a block, 0.8% of that group.
| Readable files | Named by | Named share | Blocked by | Blocked share |
|---|---|---|---|---|
| WRTNBot, all ranks | 70 | 1.1% | 68 | 1.1% |
| WRTNBot, top 1,000 | 21 | 3.8% | 21 | 3.8% |
| WRTNBot, remaining ranks | 49 | 0.9% | 47 | 0.8% |
Here, the total counts whole-site blocks recorded for this crawler. It leaves out general User-agent: * instructions. Of 68 blocks, 2 do not also appear in our count of files naming it. The naming total includes 66 files with a ban on every path and 4 files without that ban. Elsewhere, 6,101 files do not name it. We ignore capitalization and repeated names.
How to block or allow it
Place the group in your root robots.txt file. Per the robots.txt standard, a named group for WRTNBot overrides User-agent: *.
Block the whole site
User-agent: WRTNBot
Disallow: /Allow access
User-agent: WRTNBot
Allow: /It's the 87th most-named crawler of the 171 we counted in the crawler index.
How we counted
For WRTNBot, our 19 October 2026 collection used Tranco’s top 10,000 sites. A request timed out after 8 seconds, with files capped at 262,144 bytes. Missing, unreachable and oversized files do not contribute to these shares. The file readings, including repeats, use scripts/robots_2026_08_06.py (robots.py at app commit 07104ff). See the full survey method.
Common questions
Can a file name it and still allow it?
The 70 files that mention its name do not all record a whole-site ban. Those without one total 4.
How do the shares compare by rank?
Leading files record 3.8% with the name and 3.8% blocking it. Across all readable files, the corresponding shares are 1.1% and 1.1%.
What other entry is close in the index?
Crawl4AI is named by 69 files and blocked by 71, or 1.2%. For this crawler, 70 files name it and 68 block it, a 1.1% share.