Linguee Bot
Linguee Bot has 0 sites asking it to stay out in October 2026, or 0.0% of the files we could read. However, the reference says it does not obey robots.txt, and the community reference lists its operator as Linguee.
Who runs Linguee Bot?
| Fact | Linguee Bot reference entry |
|---|---|
| Operator | “Linguee” |
| Purpose | “AI powered translation service” |
| Does Linguee Bot obey robots.txt? | “No” |
| Frequency | “Unclear at this time.” |
| Cloudflare managed list for Linguee Bot | No |
These facts quote the stored community ai.robots.txt reference.
“Linguee Bot is a web crawler used by Linguee to gather training data for its AI powered translation service.”
The reference says this crawler ignores robots.txt. Keeping a site available to readers may include allowing search and requested fetches. An owner may still keep the text out of training collections.
How many sites block it?
We have 6,171 readable files for this entry, 41 of them naming it, a 0.7% share. Whole-site blocks written Disallow: / total 0. That puts the count without a recorded ban at 6,171.
For leading ranks, naming still appears in 17 files, 3.1% of 552 readable files from Tranco’s top 1,000. The remaining 24 naming files come from 5,619 readable lower-ranked files, a 0.4% share. Neither rank group contributes any recorded blocks.
| Readable files | Named by | Named share | Blocked by | Blocked share |
|---|---|---|---|---|
| Linguee Bot, all ranks | 41 | 0.7% | 0 | 0.0% |
| Linguee Bot, top 1,000 | 17 | 3.1% | 0 | 0.0% |
| Linguee Bot, remaining ranks | 24 | 0.4% | 0 | 0.0% |
Here, the total counts whole-site blocks recorded for this crawler. It leaves out general User-agent: * instructions. The counted overlap is 0 files, with 41 files naming it without banning all paths. Missing names account for 6,130 files. Repeating a name or changing its capitalization does not increase the naming total.
How to block or allow it
Place the group in your root robots.txt file. Per the robots.txt standard, a named group for Linguee Bot overrides User-agent: *.
Block the whole site
User-agent: Linguee Bot
Disallow: /Allow access
User-agent: Linguee Bot
Allow: /It's the 129th most-named crawler of the 171 we counted in the crawler index.
How we counted
For Linguee Bot, on 19 October 2026, we requested robots.txt from Tranco’s top 10,000 sites. An 8-second wait or a file over 262,144 bytes ended a read. Shares exclude unreachable, unavailable and oversized files. scripts/robots_2026_08_06.py (robots.py at app commit 07104ff) parsed both initial and repeat readings. See the full survey method.
Common questions
Does a User-agent line mean it is blocked?
A name identifies the crawler addressed by a group. The survey found 41 files with that name and no recorded whole-site ban, out of 41.
What do the top-ranked sites show?
Within readable robots.txt files near the top of the ranking, 3.1% name it and 0.0% block it. Overall, naming reaches 0.7% and blocking 0.0%.
Which nearby entry can I compare?
The nearby ERNIEBot entry reports 38 files that name it and 39 whole-site blocks, 0.6% of readable files. Here, 41 files mention the name and 0 block access, or 0.0%.