Shap-User
Shap-User is blocked by 60 sites in the October 2026 survey, or 1.0% of readable files. Its operator is unclear in the community reference.
Who runs Shap-User?
| Fact | Shap-User reference entry |
|---|---|
| Operator | “Unclear at this time.” |
| Purpose | “AI Assistants” |
| Does Shap-User obey robots.txt? | “Unclear at this time.” |
| Frequency | “Unclear at this time.” |
| Cloudflare managed list for Shap-User | No |
These facts quote the stored community ai.robots.txt reference.
“Shap-User accesses web content on behalf of users of Parallel Web Systems products. It identifies user-initiated requests rather than automatic web crawling.”
The stated role here is an AI assistant. An owner may want search visitors and answers that link back to a page while keeping its text out of model-training datasets.
How many sites block it?
Within 6,171 readable robots.txt files, the name appears in 61, or 1.0%. Instructions blocking every path, Disallow: /, appear in 60, a 1.0% share. No whole-site ban is recorded for 6,111 files.
From the highest 1,000 ranks, 552 files could be read. Its naming count is 6, or 1.1%, and the block count is 6. Blocking is more common among the leading sites, at 1.1%. Compare 54 blocks among the 5,619 readable files below those ranks, or 1.0%.
| Readable files | Named by | Named share | Blocked by | Blocked share |
|---|---|---|---|---|
| Shap-User, all ranks | 61 | 1.0% | 60 | 1.0% |
| Shap-User, top 1,000 | 6 | 1.1% | 6 | 1.1% |
| Shap-User, remaining ranks | 55 | 1.0% | 54 | 1.0% |
This total covers whole-site bans recorded for this name. A blanket User-agent: * rule alone is outside this measure. The overlap between naming and blocking is 60 files. Naming without a whole-site ban accounts for 1, and 6,110 files never name it. We ignore capitalization and repeated names.
How to block or allow it
Use your site’s root file, robots.txt. Specific groups for Shap-User take precedence over User-agent: *, according to the robots.txt standard.
Block the whole site
User-agent: Shap-User
Disallow: /Allow access
User-agent: Shap-User
Allow: /It's the 103rd most-named crawler of the 171 we counted in the crawler index.
How we counted
For Shap-User, our 19 October 2026 collection used Tranco’s top 10,000 sites. Files had an 8-second timeout and a 262,144-byte limit. Only available files we could read enter the percentages. The file readings, including repeats, use scripts/robots_2026_08_06.py (robots.py at app commit 07104ff). See the full survey method.
Common questions
What does naming this crawler tell me?
The 61 files that mention its name do not all record a whole-site ban. Those without one total 1.
Are leading sites different?
Within readable robots.txt files near the top of the ranking, 1.1% name it and 1.1% block it. Overall, naming reaches 1.0% and blocking 1.0%.
How does the nearby crawler compare?
For comparison, bedrockbot has 60 files naming it and 53 blocks, or 0.9%. Our counts for this entry are 61 files with the name and 60 blocking the whole site, or 1.0%.