bedrockbot
Few readable files block bedrockbot in October 2026, at 53 sites or 0.9%. The community reference lists its operator as Amazon.
Who runs bedrockbot?
| Fact | bedrockbot reference entry |
|---|---|
| Operator | “Amazon” |
| Purpose | “Data scraping for custom AI applications.” |
| Does bedrockbot obey robots.txt? | “Yes” |
| Frequency | “Unclear at this time.” |
| Cloudflare managed list for bedrockbot | No |
These facts quote the stored community ai.robots.txt reference.
“Connects to and crawls URLs that have been selected for use in a user's AWS bedrock application.”
This entry’s stated use is quoted above. A requested page visit can answer a person now, whereas training collects material for a model. Owners may apply different access rules.
How many sites block it?
The name occurs in 60 out of 6,171 readable files, a 1.0% share. Across the full sample, the whole-site Disallow: / block count is 53, or 0.9%. Files without that ban number 6,118.
Among 552 readable files from the top 1,000 sites, we found 9 files naming it, a 1.6% share, with 9 files blocking the whole site. Blocking is more common among the leading sites, at 1.6%. Outside this group, 5,619 files contribute 44 blocks, or 0.8%.
| Readable files | Named by | Named share | Blocked by | Blocked share |
|---|---|---|---|---|
| bedrockbot, all ranks | 60 | 1.0% | 53 | 0.9% |
| bedrockbot, top 1,000 | 9 | 1.6% | 9 | 1.6% |
| bedrockbot, remaining ranks | 51 | 0.9% | 44 | 0.8% |
Only whole-site blocks recorded against this crawler’s name count here. Rules addressed to every crawler are outside this count. Of 53 blocks, 1 does not also appear in our count of files naming it. We found 52 files naming it and blocking the whole site, along with 8 files naming it without that ban. Another 6,111 files omit the name. We ignore capitalization and repeated names.
How to block or allow it
These lines go in robots.txt at the root. User-agent: * yields to a group naming bedrockbot under the robots.txt standard.
Block the whole site
User-agent: bedrockbot
Disallow: /Allow access
User-agent: bedrockbot
Allow: /It's the 104th most-named crawler of the 171 we counted in the crawler index.
Which other crawlers does this operator run?
How we counted
For bedrockbot, on 19 October 2026, we requested robots.txt from Tranco’s top 10,000 sites. Requests could run for 8 seconds, with a 262,144-byte file limit. Only available files we could read enter the percentages. The file readings, including repeats, use scripts/robots_2026_08_06.py (robots.py at app commit 07104ff). See the full survey method.
Common questions
Does a User-agent line mean it is blocked?
The 60 files naming it include 8 files without a whole-site ban.
What do the top-ranked sites show?
Leading files record 1.6% with the name and 1.6% blocking it. Across all readable files, the corresponding shares are 1.0% and 0.9%.
Which nearby entry can I compare?
The nearby Manus-User entry reports 60 files that name it and 56 whole-site blocks, 0.9% of readable files. Here, 60 files mention the name and 53 block access, or 0.9%.