YaK
YaK is listed as a search crawler, and the community reference lists its operator as Meltwater. In October 2026, 71 sites block it, a 1.2% share of readable files.
Who runs YaK?
| Fact | YaK reference entry |
|---|---|
| Operator | “Meltwater” |
| Purpose | “According to the Meltwater Consumer Intelligence page 'By applying AI, data science, and market research expertise to a live feed of global data sources, we transform unstructured data into actionable insights allowing better decision-making'.” |
| Does YaK obey robots.txt? | “Unclear at this time.” |
| Frequency | “Unclear at this time.” |
| Cloudflare managed list for YaK | No |
These facts quote the stored community ai.robots.txt reference.
“Retrieves data used for Meltwater's AI enabled consumer intelligence suite”
The listed use here is search. Helping people find a page through search is a different use from using its contents to train a model, so owners may allow those visits separately.
How many sites block it?
Within 6,171 readable robots.txt files, the name appears in 77, or 1.2%. Instructions blocking every path, Disallow: /, appear in 71, a 1.2% share. No whole-site ban is recorded for 6,100 files.
Among 552 readable files from the top 1,000 sites, we found 29 files naming it, a 5.3% share, with 26 files blocking the whole site. Blocking is more common among the leading sites, at 4.7%. Outside this group, 5,619 files contribute 45 blocks, or 0.8%.
| Readable files | Named by | Named share | Blocked by | Blocked share |
|---|---|---|---|---|
| YaK, all ranks | 77 | 1.2% | 71 | 1.2% |
| YaK, top 1,000 | 29 | 5.3% | 26 | 4.7% |
| YaK, remaining ranks | 48 | 0.9% | 45 | 0.8% |
Blocking here counts whole-site bans recorded for it by name. A general ban for all crawlers is excluded. The counted overlap is 71 files, with 6 files naming it without banning all paths. Missing names account for 6,094 files. Repeating a name or changing its capitalization does not increase the naming total.
How to block or allow it
Put these in robots.txt at the site root. Per the robots.txt standard, a group naming YaK takes precedence over User-agent: *.
Block the whole site
User-agent: YaK
Disallow: /Allow access
User-agent: YaK
Allow: /It's the 77th most-named crawler of the 171 we counted in the crawler index.
How we counted
For YaK, we surveyed Tranco’s top 10,000 sites on 19 October 2026. Requests could run for 8 seconds, with a 262,144-byte file limit. Missing, unreachable and oversized files do not contribute to these shares. Repeat readings are included, using scripts/robots_2026_08_06.py (robots.py at app commit 07104ff). See the full survey method.
Common questions
Can a file name it and still allow it?
The 77 files that mention its name do not all record a whole-site ban. Those without one total 6.
How do the shares compare by rank?
Leading files record 5.3% with the name and 4.7% blocking it. Across all readable files, the corresponding shares are 1.2% and 1.2%.
What other entry is close in the index?
KimiBot is named by 76 files and blocked by 77, or 1.2%. For this crawler, 77 files name it and 71 block it, a 1.2% share.