GoogleOther

By Matt Kerr·Published 11 November 2026

GoogleOther is blocked by 371 sites in the October 2026 survey, or 6.0% of readable files. The community reference lists its operator as Google.

Who runs GoogleOther?

Fact	GoogleOther reference entry
Operator	“Google”
Purpose	“Scrapes data.”
Does GoogleOther obey robots.txt?	“Yes”
Frequency	“No information.”
Cloudflare managed list for GoogleOther	No

These facts quote the stored community ai.robots.txt reference.

A matching description is used for 2 other entries. Counts below are specific to this crawler.

This entry’s stated use is quoted above. Search referrals or a user’s question can give an owner a reason to allow access while declining collection of page text for training.

How many sites block it?

Our readable sample has 371 whole-site blocks, 6.0% of 6,171 files, with Disallow: / as the instruction. Of that full sample, 232 name this crawler, or 3.8%. The count without a whole-site block is 5,800.

Breaking out the leading 1,000 sites gives 552 readable files, with 40 naming the crawler, or 7.2%. Whole-site blocking appears in 65 of them. Blocking is more common among the leading sites, at 11.8%. Beyond those ranks, 5,619 files have a blocking total of 306, 5.4% of that sample.

Readable files	Named by	Named share	Blocked by	Blocked share
GoogleOther, all ranks	232	3.8%	371	6.0%
GoogleOther, top 1,000	40	7.2%	65	11.8%
GoogleOther, remaining ranks	192	3.4%	306	5.4%

Counts include a whole-site ban in a User-agent: * group if no named group overrides it. Of 371 blocks, 198 come only from that general rule, without naming this crawler. The overlap between naming and blocking is 173 files. Naming without a whole-site ban accounts for 59, and 5,939 files never name it. We ignore capitalization and repeated names.

Has that changed since August?

Between August and October, 15 sites that had blocked it stopped and 66 started, so 51 more of the same 5,889 sites block it. A further 268 blocked it in both months, and 5,540 did so in neither. Keeping the sites the same lets us count changes in their rules.

August’s full sample had 305 blocks among 6,236 readable files. October’s was 371 among 6,171. Those monthly totals also reflect which files we could read on each visit.

August to October	Files
Started blocking	66
Stopped blocking	15
Blocked in both	268
Blocked in neither	5,540
Net change	+51

How to block or allow it

Edit the root robots.txt file on your site. The robots.txt standard gives a GoogleOther group precedence over User-agent: * for this crawler.

Block the whole site

User-agent: GoogleOther
Disallow: /

Allow access

User-agent: GoogleOther
Allow: /

It's the 36th most-named crawler of the 171 we counted in the crawler index.

Which other crawlers does this operator run?

How we counted

For GoogleOther, our 19 October 2026 collection used Tranco’s top 10,000 sites. An 8-second wait or a file over 262,144 bytes ended a read. Shares exclude unreachable, unavailable and oversized files. scripts/robots_2026_08_06.py (robots.py at app commit 07104ff) parsed both initial and repeat readings. See the full survey method.

Common questions

Does a User-agent line mean it is blocked?

A name identifies the crawler addressed by a group. The survey found 59 files with that name and no recorded whole-site ban, out of 232.

What do the top-ranked sites show?

For naming rules, 7.2% of the leading sample mentions it, compared with 3.8% overall. Whole-site blocking is 11.8% at leading ranks and 6.0% across the full sample.

Which nearby entry can I compare?

The nearby ICC-Crawler entry reports 230 files that name it and 229 whole-site blocks, 3.7% of readable files. Here, 232 files mention the name and 371 block access, or 6.0%.