Webzio-Extended
Webzio-Extended is blocked by 270 sites in the October 2026 survey, or 4.4% of readable files. Its operator is unclear in the community reference.
Who runs Webzio-Extended?
| Fact | Webzio-Extended reference entry |
|---|---|
| Operator | “Unclear at this time.” |
| Purpose | “AI Data Scrapers” |
| Does Webzio-Extended obey robots.txt? | “Unclear at this time.” |
| Frequency | “Unclear at this time.” |
| Cloudflare managed list for Webzio-Extended | No |
These facts quote the stored community ai.robots.txt reference.
“Webzio-Extended is a web crawler used by Webz.io to maintain a repository of web crawl data that it sells to other companies, including those using it to train AI models.”
This entry is classed as a crawler collecting data for AI. Allowing people to find and read a page need not include permission to gather it for model training. Owners can express those choices in crawler-specific groups.
How many sites block it?
A User-agent line names it in 281 files, 4.6% of 6,171 readable files. Whole-site bans, written Disallow: /, total 270, or 4.4%. That leaves 5,901 without a recorded ban.
From the highest 1,000 ranks, 552 files could be read. Its naming count is 44, or 8.0%, and the block count is 42. Blocking is more common among the leading sites, at 7.6%. Compare 228 blocks among the 5,619 readable files below those ranks, or 4.1%.
| Readable files | Named by | Named share | Blocked by | Blocked share |
|---|---|---|---|---|
| Webzio-Extended, all ranks | 281 | 4.6% | 270 | 4.4% |
| Webzio-Extended, top 1,000 | 44 | 8.0% | 42 | 7.6% |
| Webzio-Extended, remaining ranks | 237 | 4.2% | 228 | 4.1% |
For this entry, we counted whole-site bans attached to its own name, leaving general rules for all crawlers out of the total. Of 270 blocks, 4 do not also appear in our count of files naming it. We found 266 files naming it and blocking the whole site, along with 15 files naming it without that ban. Another 5,890 files omit the name. We ignore capitalization and repeated names.
How to block or allow it
The examples include the spelling variants counted together here. Use your site’s root file, robots.txt. Specific groups for Webzio-Extended take precedence over User-agent: *, according to the robots.txt standard.
Block the whole site
User-agent: Webzio-Extended
User-agent: webzio-extended
Disallow: /Allow access
User-agent: Webzio-Extended
User-agent: webzio-extended
Allow: /It's the 30th most-named crawler of the 171 we counted in the crawler index.
How we counted
For Webzio-Extended, we surveyed Tranco’s top 10,000 sites on 19 October 2026. An 8-second wait or a file over 262,144 bytes ended a read. Shares exclude unreachable, unavailable and oversized files. The file readings, including repeats, use scripts/robots_2026_08_06.py (robots.py at app commit 07104ff). See the full survey method.
Common questions
What does naming this crawler tell me?
The 281 files naming it include 15 files without a whole-site ban.
Are leading sites different?
Its name appears in 8.0% of readable leading files and 4.6% overall. Blocking affects 7.6% in the leading group and 4.4% in all ranks.
How does the nearby crawler compare?
The nearby AI2Bot entry reports 278 files that name it and 251 whole-site blocks, 4.1% of readable files. Here, 281 files mention the name and 270 block access, or 4.4%.