ERNIEBot
ERNIEBot has 39 recorded blocks in October 2026, or 0.6%, and 38 files naming it. The community reference lists its operator as Baidu.
Who runs ERNIEBot?
| Fact | ERNIEBot reference entry |
|---|---|
| Operator | “Baidu” |
| Purpose | “Collects public web content for Baidu's ERNIE large language models.” |
| Does ERNIEBot obey robots.txt? | “Unclear at this time.” |
| Frequency | “Unclear at this time.” |
| Cloudflare managed list for ERNIEBot | No |
These facts quote the stored community ai.robots.txt reference.
“ERNIEBot is a crawler operated by Baidu to collect public web content used for its ERNIE (Wenxin) large language models.”
This entry’s stated use is quoted above. Search referrals or a user’s question can give an owner a reason to allow access while declining collection of page text for training.
How many sites block it?
Instructions to stay out of the whole site appear in 39 files, or 0.6% of 6,171 readable files. We counted 38 with its name in a User-agent line, a 0.6% share. Another 6,132 lack a recorded Disallow: / ban.
Tranco’s top 1,000 sites yielded 552 readable files, including 3 files with its name, or 0.5%, and 3 files blocking it. The leading sites have a lower block share, 0.5%. Outside those ranks, 36 of 5,619 files block it, or 0.6%.
| Readable files | Named by | Named share | Blocked by | Blocked share |
|---|---|---|---|---|
| ERNIEBot, all ranks | 38 | 0.6% | 39 | 0.6% |
| ERNIEBot, top 1,000 | 3 | 0.5% | 3 | 0.5% |
| ERNIEBot, remaining ranks | 35 | 0.6% | 36 | 0.6% |
Only whole-site blocks recorded against this crawler’s name count here. Rules addressed to every crawler are outside this count. Of 39 blocks, 1 does not also appear in our count of files naming it. Among files listing the crawler, 38 record a ban on all paths. The number not naming it is 6,133. For naming, repeats and different case are merged.
How to block or allow it
Add these lines to robots.txt at your site root. For ERNIEBot, the robots.txt standard gives a named group priority over User-agent: *.
Block the whole site
User-agent: ERNIEBot
Disallow: /Allow access
User-agent: ERNIEBot
Allow: /It's the 130th most-named crawler of the 171 we counted in the crawler index.
Which other crawlers does this operator run?
How we counted
For ERNIEBot, on 19 October 2026, we requested robots.txt from Tranco’s top 10,000 sites. An 8-second wait or a file over 262,144 bytes ended a read. Only available files we could read enter the percentages. Repeat readings are included, using scripts/robots_2026_08_06.py (robots.py at app commit 07104ff). See the full survey method.
Common questions
Can a file name it and still allow it?
Every one of the 38 files naming it also has a whole-site ban.
How do the shares compare by rank?
Leading files record 0.5% with the name and 0.5% blocking it. Across all readable files, the corresponding shares are 0.6% and 0.6%.
What other entry is close in the index?
The nearby amazon-QBusiness entry reports 37 files that name it and 44 whole-site blocks, 0.7% of readable files. Here, 38 files mention the name and 39 block access, or 0.6%.