Amzn-SearchBot
Amzn-SearchBot is listed as a search crawler, and its operator is unclear in the community reference. In October 2026, 307 sites block it, a 5.0% share of readable files.
Who runs Amzn-SearchBot?
| Fact | Amzn-SearchBot reference entry |
|---|---|
| Operator | “Unclear at this time.” |
| Purpose | “AI Search Crawlers” |
| Does Amzn-SearchBot obey robots.txt? | “Unclear at this time.” |
| Frequency | “Unclear at this time.” |
| Cloudflare managed list for Amzn-SearchBot | No |
These facts quote the stored community ai.robots.txt reference.
The listed use here is search. Search indexing is aimed at helping people find pages. Feeding text into model training is another use an owner may decline.
How many sites block it?
Our readable sample has 307 whole-site blocks, 5.0% of 6,171 files, with Disallow: / as the instruction. Of that full sample, 109 name this crawler, or 1.8%. The count without a whole-site block is 5,864.
For the top 1,000 ranks, its name appears in 16 files out of 552, or 2.9%. Whole-site blocks total 51. Blocking is more common among the leading sites, at 9.2%. Lower in the ranking, blocking reaches 256 files out of 5,619, a 4.6% share.
| Readable files | Named by | Named share | Blocked by | Blocked share |
|---|---|---|---|---|
| Amzn-SearchBot, all ranks | 109 | 1.8% | 307 | 5.0% |
| Amzn-SearchBot, top 1,000 | 16 | 2.9% | 51 | 9.2% |
| Amzn-SearchBot, remaining ranks | 93 | 1.7% | 256 | 4.6% |
Both named groups and general User-agent: * groups can contribute to blocking the whole site. Of 307 blocks, 221 come only from that general rule, without naming this crawler. Looking just at named files gives 86 blocks and 23 without a whole-site block. For 6,062 readable files, we found no mention by name. We ignore capitalization and repeated names.
Has that changed since August?
Between August and October, 15 sites that had blocked it stopped and 17 started, so 2 more of the same 5,889 sites block it. A further 258 blocked it in both months, and 5,599 did so in neither. Keeping the sites the same lets us count changes in their rules.
August’s full sample had 293 blocks among 6,236 readable files. October’s was 307 among 6,171. Those monthly totals also reflect which files we could read on each visit.
| August to October | Files |
|---|---|
| Started blocking | 17 |
| Stopped blocking | 15 |
| Blocked in both | 258 |
| Blocked in neither | 5,599 |
| Net change | +2 |
How to block or allow it
These lines go in robots.txt at the root. User-agent: * yields to a group naming Amzn-SearchBot under the robots.txt standard.
Block the whole site
User-agent: Amzn-SearchBot
Disallow: /Allow access
User-agent: Amzn-SearchBot
Allow: /It's the 56th most-named crawler of the 171 we counted in the crawler index.
How we counted
For Amzn-SearchBot, our 19 October 2026 collection used Tranco’s top 10,000 sites. Each file had 8 seconds to arrive, up to 262,144 bytes. Shares exclude unreachable, unavailable and oversized files. Later readings are included in results from scripts/robots_2026_08_06.py (robots.py at app commit 07104ff). See the full survey method.
Common questions
What does naming this crawler tell me?
This survey includes 23 files with the name and no whole-site ban, out of 109 naming files.
Are leading sites different?
Its name appears in 2.9% of readable leading files and 1.8% overall. Blocking affects 9.2% in the leading group and 5.0% in all ranks.
How does the nearby crawler compare?
ISSCyberRiskCrawler is named by 109 files and blocked by 111, or 1.8%. For this crawler, 109 files name it and 307 block it, a 5.0% share.