Bravebot
Bravebot is blocked by 81 sites in the October 2026 survey, or 1.3% of readable files. The community reference lists its operator as https://safe.search.brave.com/help/brave-search-crawler.
Who runs Bravebot?
| Fact | Bravebot reference entry |
|---|---|
| Operator | “https://safe.search.brave.com/help/brave-search-crawler” |
| Purpose | “AI Data Providers” |
| Does Bravebot obey robots.txt? | “Yes” |
| Frequency | “Unclear at this time.” |
| Cloudflare managed list for Bravebot | No |
These facts quote the stored community ai.robots.txt reference.
“Bravebot is a web crawler by Brave that indexes pages for Brave Search, providing search data and AI-optimized context to power chatbots, agents, and RAG pipelines.”
Its listed role is supplying data for AI tools. An owner may want a page included in search and available on request without offering its contents for future model development.
How many sites block it?
A User-agent line names it in 90 files, 1.5% of 6,171 readable files. Whole-site bans, written Disallow: /, total 81, or 1.3%. That leaves 6,090 without a recorded ban.
Tranco’s top 1,000 sites yielded 552 readable files, including 11 files with its name, or 2.0%, and 8 files blocking it. Blocking is more common among the leading sites, at 1.4%. Outside those ranks, 73 of 5,619 files block it, or 1.3%.
| Readable files | Named by | Named share | Blocked by | Blocked share |
|---|---|---|---|---|
| Bravebot, all ranks | 90 | 1.5% | 81 | 1.3% |
| Bravebot, top 1,000 | 11 | 2.0% | 8 | 1.4% |
| Bravebot, remaining ranks | 79 | 1.4% | 73 | 1.3% |
Only whole-site blocks recorded against this crawler’s name count here. Rules addressed to every crawler are outside this count. Of 81 blocks, 3 do not also appear in our count of files naming it. Names and bans overlap in 78 files. We also found 12 files with a name and no whole-site ban, plus 6,081 files omitting the name. Case changes and repeated spellings are merged.
How to block or allow it
Edit the root robots.txt file on your site. The robots.txt standard gives a Bravebot group precedence over User-agent: * for this crawler.
Block the whole site
User-agent: Bravebot
Disallow: /Allow access
User-agent: Bravebot
Allow: /It's the 66th most-named crawler of the 171 we counted in the crawler index.
How we counted
For Bravebot, our 19 October 2026 collection used Tranco’s top 10,000 sites. Files had an 8-second timeout and a 262,144-byte limit. Only available files we could read enter the percentages. scripts/robots_2026_08_06.py (robots.py at app commit 07104ff) parsed both initial and repeat readings. See the full survey method.
Common questions
Does a User-agent line mean it is blocked?
A name identifies the crawler addressed by a group. The survey found 12 files with that name and no recorded whole-site ban, out of 90.
What do the top-ranked sites show?
Within readable robots.txt files near the top of the ranking, 2.0% name it and 1.4% block it. Overall, naming reaches 1.5% and blocking 1.3%.
Which nearby entry can I compare?
imageSpider is named by 90 files and blocked by 104, or 1.7%. For this crawler, 90 files name it and 81 block it, a 1.3% share.