DuckAssistBot
DuckAssistBot is blocked by 237 sites in the October 2026 survey, or 3.8% of readable files. Its operator is unclear in the community reference.
Who runs DuckAssistBot?
| Fact | DuckAssistBot reference entry |
|---|---|
| Operator | “Unclear at this time.” |
| Purpose | “AI Assistants” |
| Does DuckAssistBot obey robots.txt? | “Yes” |
| Frequency | “Unclear at this time.” |
| Cloudflare managed list for DuckAssistBot | No |
These facts quote the stored community ai.robots.txt reference.
“DuckAssistBot is a web crawler that scans websites to collect content for DuckDuckGo's AI-assisted answers feature, which generates brief responses to search queries usin…”
The stated role here is an AI assistant. Search indexing is aimed at helping people find pages. Feeding text into model training is another use an owner may decline.
How many sites block it?
A User-agent line names it in 320 files, 5.2% of 6,171 readable files. Whole-site bans, written Disallow: /, total 237, or 3.8%. That leaves 5,934 without a recorded ban.
Breaking out the leading 1,000 sites gives 552 readable files, with 56 naming the crawler, or 10.1%. Whole-site blocking appears in 36 of them. Blocking is more common among the leading sites, at 6.5%. Beyond those ranks, 5,619 files have a blocking total of 201, 3.6% of that sample.
| Readable files | Named by | Named share | Blocked by | Blocked share |
|---|---|---|---|---|
| DuckAssistBot, all ranks | 320 | 5.2% | 237 | 3.8% |
| DuckAssistBot, top 1,000 | 56 | 10.1% | 36 | 6.5% |
| DuckAssistBot, remaining ranks | 264 | 4.7% | 201 | 3.6% |
This is a count of blocks attached to the named crawler, covering the whole site. Blanket rules for every crawler do not enter it. Of 237 blocks, 1 does not also appear in our count of files naming it. The overlap between naming and blocking is 236 files. Naming without a whole-site ban accounts for 84, and 5,851 files never name it. We ignore capitalization and repeated names.
How to block or allow it
Place the group in your root robots.txt file. Per the robots.txt standard, a named group for DuckAssistBot overrides User-agent: *.
Block the whole site
User-agent: DuckAssistBot
Disallow: /Allow access
User-agent: DuckAssistBot
Allow: /It's the 27th most-named crawler of the 171 we counted in the crawler index.
How we counted
For DuckAssistBot, we surveyed Tranco’s top 10,000 sites on 19 October 2026. A request timed out after 8 seconds, with files capped at 262,144 bytes. Shares exclude unreachable, unavailable and oversized files. Later readings are included in results from scripts/robots_2026_08_06.py (robots.py at app commit 07104ff). See the full survey method.
Common questions
What does naming this crawler tell me?
The 320 files that mention its name do not all record a whole-site ban. Those without one total 84.
Are leading sites different?
Leading files record 10.1% with the name and 6.5% blocking it. Across all readable files, the corresponding shares are 5.2% and 3.8%.
How does the nearby crawler compare?
For comparison, Meta-ExternalFetcher has 298 files naming it and 509 blocks, or 8.2%. Our counts for this entry are 320 files with the name and 237 blocking the whole site, or 3.8%.