QueritBot
QueritBot has 20 recorded blocks in October 2026, or 0.3%, and 19 files naming it. The community reference lists its operator as Querit, a company providing a search API for large language model integration.
Who runs QueritBot?
| Fact | QueritBot reference entry |
|---|---|
| Operator | “Querit, a company providing a search API for large language model integration” |
| Purpose | “AI Data Providers” |
| Does QueritBot obey robots.txt? | “Unclear at this time.” |
| Frequency | “Unclear at this time.” |
| Cloudflare managed list for QueritBot | No |
These facts quote the stored community ai.robots.txt reference.
“QueritBot is a web crawler operated by Querit, a company providing a search API for large language model integration. This bot indexes web content to power the real-time …”
Its listed role is supplying data for AI tools. Search referrals or a user’s question can give an owner a reason to allow access while declining collection of page text for training.
How many sites block it?
Of 6,171 robots.txt files we could read, 20 block the whole site with Disallow: /, or 0.3%. Another way to read the sample is by name, with 19 files naming it in a User-agent line, or 0.3%. The 6,151 remaining files have no recorded whole-site block.
Among 552 readable files from the top 1,000 sites, we found 3 files naming it, a 0.5% share, with 3 files blocking the whole site. Blocking is more common among the leading sites, at 0.5%. Outside this group, 5,619 files contribute 17 blocks, or 0.3%.
| Readable files | Named by | Named share | Blocked by | Blocked share |
|---|---|---|---|---|
| QueritBot, all ranks | 19 | 0.3% | 20 | 0.3% |
| QueritBot, top 1,000 | 3 | 0.5% | 3 | 0.5% |
| QueritBot, remaining ranks | 16 | 0.3% | 17 | 0.3% |
Here, the total counts whole-site blocks recorded for this crawler. It leaves out general User-agent: * instructions. Of 20 blocks, 1 does not also appear in our count of files naming it. 19 files have the name plus a whole-site ban. Outside that set, 6,152 files never name it. Repeated user-agent mentions or capitalization variants are counted once.
How to block or allow it
These lines go in robots.txt at the root. User-agent: * yields to a group naming QueritBot under the robots.txt standard.
Block the whole site
User-agent: QueritBot
Disallow: /Allow access
User-agent: QueritBot
Allow: /It's the 152nd most-named crawler of the 171 we counted in the crawler index.
Which other crawlers does this operator run?
How we counted
For QueritBot, we surveyed Tranco’s top 10,000 sites on 19 October 2026. Each file had 8 seconds to arrive, up to 262,144 bytes. Only available files we could read enter the percentages. The file readings, including repeats, use scripts/robots_2026_08_06.py (robots.py at app commit 07104ff). See the full survey method.
Common questions
Does a User-agent line mean it is blocked?
In the 19 files naming this crawler, every one also blocks the whole site.
What do the top-ranked sites show?
At the top of the ranking, 0.5% of readable files name it and 0.5% block it. Across all ranks, those shares are 0.3% and 0.3%.
Which nearby entry can I compare?
The nearby AIWebIndex entry reports 18 files that name it and 24 whole-site blocks, 0.4% of readable files. Here, 19 files mention the name and 20 block access, or 0.3%.