GoogleAgent-Mariner
Few readable files block GoogleAgent-Mariner in October 2026, at 57 sites or 0.9%. The community reference lists its operator as Google.
Who runs GoogleAgent-Mariner?
| Fact | GoogleAgent-Mariner reference entry |
|---|---|
| Operator | “Google” |
| Purpose | “AI Agents” |
| Does GoogleAgent-Mariner obey robots.txt? | “Unclear at this time.” |
| Frequency | “Unclear at this time.” |
| Cloudflare managed list for GoogleAgent-Mariner | No |
These facts quote the stored community ai.robots.txt reference.
“GoogleAgent-Mariner is an AI agent created by Google that can use a web browser. It can intelligently navigate and interact with websites to complete multi-step tasks on …”
The reference places this crawler among AI agents. Search indexing is aimed at helping people find pages. Feeding text into model training is another use an owner may decline.
How many sites block it?
The name occurs in 67 out of 6,171 readable files, a 1.1% share. Across the full sample, the whole-site Disallow: / block count is 57, or 0.9%. Files without that ban number 6,114.
Among 552 readable files from the top 1,000 sites, we found 21 files naming it, a 3.8% share, with 20 files blocking the whole site. Blocking is more common among the leading sites, at 3.6%. Outside this group, 5,619 files contribute 37 blocks, or 0.7%.
| Readable files | Named by | Named share | Blocked by | Blocked share |
|---|---|---|---|---|
| GoogleAgent-Mariner, all ranks | 67 | 1.1% | 57 | 0.9% |
| GoogleAgent-Mariner, top 1,000 | 21 | 3.8% | 20 | 3.6% |
| GoogleAgent-Mariner, remaining ranks | 46 | 0.8% | 37 | 0.7% |
Blocking here counts whole-site bans recorded for it by name. A general ban for all crawlers is excluded. Of 57 blocks, 1 does not also appear in our count of files naming it. Names and bans overlap in 56 files. We also found 11 files with a name and no whole-site ban, plus 6,104 files omitting the name. Case changes and repeated spellings are merged.
How to block or allow it
These lines go in robots.txt at the root. User-agent: * yields to a group naming GoogleAgent-Mariner under the robots.txt standard.
Block the whole site
User-agent: GoogleAgent-Mariner
Disallow: /Allow access
User-agent: GoogleAgent-Mariner
Allow: /It's the 92nd most-named crawler of the 171 we counted in the crawler index.
Which other crawlers does this operator run?
How we counted
For GoogleAgent-Mariner, on 19 October 2026, we requested robots.txt from Tranco’s top 10,000 sites. Each file had 8 seconds to arrive, up to 262,144 bytes. Shares exclude unreachable, unavailable and oversized files. Reading files again used scripts/robots_2026_08_06.py (robots.py at app commit 07104ff) as well. See the full survey method.
Common questions
Does a User-agent line mean it is blocked?
The 67 files that mention its name do not all record a whole-site ban. Those without one total 11.
What do the top-ranked sites show?
Its name appears in 3.8% of readable leading files and 1.1% overall. Blocking affects 3.6% in the leading group and 0.9% in all ranks.
Which nearby entry can I compare?
The nearby KunatoCrawler entry reports 66 files that name it and 70 whole-site blocks, 1.1% of readable files. Here, 67 files mention the name and 57 block access, or 0.9%.