AddSearchBot
AddSearchBot is listed as a search crawler, and its operator is unclear in the community reference. In October 2026, 68 sites block it, a 1.1% share of readable files.
Who runs AddSearchBot?
| Fact | AddSearchBot reference entry |
|---|---|
| Operator | “Unclear at this time.” |
| Purpose | “AI Search Crawlers” |
| Does AddSearchBot obey robots.txt? | “Unclear at this time.” |
| Frequency | “Unclear at this time.” |
| Cloudflare managed list for AddSearchBot | No |
These facts quote the stored community ai.robots.txt reference.
“AddSearchBot is a web crawler that indexes website content for AddSearch's AI-powered site search solution, collecting data to provide fast and accurate search results.”
The listed use here is search. A site can welcome a page fetch prompted by a person while declining the collection of that text to build future models.
How many sites block it?
Of 6,171 robots.txt files we could read, 68 block the whole site with Disallow: /, or 1.1%. Another way to read the sample is by name, with 67 files naming it in a User-agent line, or 1.1%. The 6,103 remaining files have no recorded whole-site block.
Breaking out the leading 1,000 sites gives 552 readable files, with 20 naming the crawler, or 3.6%. Whole-site blocking appears in 20 of them. Blocking is more common among the leading sites, at 3.6%. Beyond those ranks, 5,619 files have a blocking total of 48, 0.9% of that sample.
| Readable files | Named by | Named share | Blocked by | Blocked share |
|---|---|---|---|---|
| AddSearchBot, all ranks | 67 | 1.1% | 68 | 1.1% |
| AddSearchBot, top 1,000 | 20 | 3.6% | 20 | 3.6% |
| AddSearchBot, remaining ranks | 47 | 0.8% | 48 | 0.9% |
Here, the total counts whole-site blocks recorded for this crawler. It leaves out general User-agent: * instructions. Of 68 blocks, 6 do not also appear in our count of files naming it. Names and bans overlap in 62 files. We also found 5 files with a name and no whole-site ban, plus 6,104 files omitting the name. Case changes and repeated spellings are merged.
How to block or allow it
Use your site’s root file, robots.txt. Specific groups for AddSearchBot take precedence over User-agent: *, according to the robots.txt standard.
Block the whole site
User-agent: AddSearchBot
Disallow: /Allow access
User-agent: AddSearchBot
Allow: /It's the 91st most-named crawler of the 171 we counted in the crawler index.
How we counted
For AddSearchBot, our 19 October 2026 collection used Tranco’s top 10,000 sites. An 8-second wait or a file over 262,144 bytes ended a read. Only available files we could read enter the percentages. Later readings are included in results from scripts/robots_2026_08_06.py (robots.py at app commit 07104ff). See the full survey method.
Common questions
What does naming this crawler tell me?
The 67 files that mention its name do not all record a whole-site ban. Those without one total 5.
Are leading sites different?
For naming rules, 3.6% of the leading sample mentions it, compared with 1.1% overall. Whole-site blocking is 3.6% at leading ranks and 1.1% across the full sample.
How does the nearby crawler compare?
The nearby GoogleAgent-Mariner entry reports 67 files that name it and 57 whole-site blocks, 0.9% of readable files. Here, 67 files mention the name and 68 block access, or 1.1%.