Anomura
Anomura is listed as a search crawler, and the community reference lists its operator as Direqt. In October 2026, 44 sites block it, a 0.7% share of readable files.
Who runs Anomura?
| Fact | Anomura reference entry |
|---|---|
| Operator | “Direqt” |
| Purpose | “Collects data for AI search” |
| Does Anomura obey robots.txt? | “Yes” |
| Frequency | “No information provided.” |
| Cloudflare managed list for Anomura | No |
These facts quote the stored community ai.robots.txt reference.
“Anomura is Direqt's search crawler, it discovers and indexes pages their customers websites.”
The listed use here is search. An owner may allow a search crawler to list a page or an assistant to retrieve it for someone, yet decline to supply the text for training.
How many sites block it?
The name occurs in 45 out of 6,171 readable files, a 0.7% share. Across the full sample, the whole-site Disallow: / block count is 44, or 0.7%. Files without that ban number 6,127.
Tranco’s top 1,000 sites yielded 552 readable files, including 17 files with its name, or 3.1%, and 17 files blocking it. Blocking is more common among the leading sites, at 3.1%. Outside those ranks, 27 of 5,619 files block it, or 0.5%.
| Readable files | Named by | Named share | Blocked by | Blocked share |
|---|---|---|---|---|
| Anomura, all ranks | 45 | 0.7% | 44 | 0.7% |
| Anomura, top 1,000 | 17 | 3.1% | 17 | 3.1% |
| Anomura, remaining ranks | 28 | 0.5% | 27 | 0.5% |
This is a count of blocks attached to the named crawler, covering the whole site. Blanket rules for every crawler do not enter it. The overlap between naming and blocking is 44 files. Naming without a whole-site ban accounts for 1, and 6,126 files never name it. We ignore capitalization and repeated names.
How to block or allow it
Place the group in your root robots.txt file. Per the robots.txt standard, a named group for Anomura overrides User-agent: *.
Block the whole site
User-agent: Anomura
Disallow: /Allow access
User-agent: Anomura
Allow: /It's the 123rd most-named crawler of the 171 we counted in the crawler index.
How we counted
For Anomura, our 19 October 2026 collection used Tranco’s top 10,000 sites. Requests could run for 8 seconds, with a 262,144-byte file limit. Shares exclude unreachable, unavailable and oversized files. Repeat readings are included, using scripts/robots_2026_08_06.py (robots.py at app commit 07104ff). See the full survey method.
Common questions
Does a User-agent line mean it is blocked?
This survey includes 1 file with the name and no whole-site ban, out of 45 naming files.
What do the top-ranked sites show?
At the top of the ranking, 3.1% of readable files name it and 3.1% block it. Across all ranks, those shares are 0.7% and 0.7%.
Which nearby entry can I compare?
The nearby Operator entry reports 44 files that name it and 36 whole-site blocks, 0.6% of readable files. Here, 45 files mention the name and 44 block access, or 0.7%.