Applebot-Extended
Applebot-Extended is blocked by 694 sites in the October 2026 survey, or 11.2% of readable files. The community reference lists its operator as Apple.
Who runs Applebot-Extended?
| Fact | Applebot-Extended reference entry |
|---|---|
| Operator | “Apple” |
| Purpose | “Powers features in Siri, Spotlight, Safari, Apple Intelligence, and others.” |
| Does Applebot-Extended obey robots.txt? | “Yes” |
| Frequency | “Unclear at this time.” |
| Cloudflare managed list for Applebot-Extended | Yes |
These facts quote the stored community ai.robots.txt reference.
“Apple has a secondary user agent, Applebot-Extended ... [that is] used to train Apple's foundation models powering generative AI features across Apple products, including Apple Intelligence, Services, and Developer Tools.”
This entry’s stated use is quoted above. A site can welcome a page fetch prompted by a person while declining the collection of that text to build future models.
How many sites block it?
Instructions to stay out of the whole site appear in 694 files, or 11.2% of 6,171 readable files. We counted 658 with its name in a User-agent line, a 10.7% share. Another 5,477 lack a recorded Disallow: / ban.
Tranco’s top 1,000 sites yielded 552 readable files, including 88 files with its name, or 15.9%, and 83 files blocking it. Blocking is more common among the leading sites, at 15.0%. Outside those ranks, 611 of 5,619 files block it, or 10.9%.
| Readable files | Named by | Named share | Blocked by | Blocked share |
|---|---|---|---|---|
| Applebot-Extended, all ranks | 658 | 10.7% | 694 | 11.2% |
| Applebot-Extended, top 1,000 | 88 | 15.9% | 83 | 15.0% |
| Applebot-Extended, remaining ranks | 570 | 10.1% | 611 | 10.9% |
Both named groups and general User-agent: * groups can contribute to blocking the whole site. Of 694 blocks, 184 come only from that general rule, without naming this crawler. The counted overlap is 510 files, with 148 files naming it without banning all paths. Missing names account for 5,513 files. Repeating a name or changing its capitalization does not increase the naming total.
Has that changed since August?
Between August and October, 165 sites that had blocked it stopped and 54 started, so 111 fewer of the same 5,889 sites block it. A further 600 blocked it in both months, and 5,070 did so in neither. Keeping the sites the same lets us count changes in their rules.
August’s full sample had 869 blocks among 6,236 readable files. October’s was 694 among 6,171. Those monthly totals also reflect which files we could read on each visit.
| August to October | Files |
|---|---|
| Started blocking | 54 |
| Stopped blocking | 165 |
| Blocked in both | 600 |
| Blocked in neither | 5,070 |
| Net change | -111 |
How to block or allow it
Add these lines to robots.txt at your site root. For Applebot-Extended, the robots.txt standard gives a named group priority over User-agent: *.
Block the whole site
User-agent: Applebot-Extended
Disallow: /Allow access
User-agent: Applebot-Extended
Allow: /It's the 10th most-named crawler of the 171 we counted in the crawler index.
How we counted
For Applebot-Extended, we surveyed Tranco’s top 10,000 sites on 19 October 2026. An 8-second wait or a file over 262,144 bytes ended a read. Shares exclude unreachable, unavailable and oversized files. The file readings, including repeats, use scripts/robots_2026_08_06.py (robots.py at app commit 07104ff). See the full survey method.
Common questions
Can a file name it and still allow it?
We found 148 files without a whole-site ban among 658 that name it.
How do the shares compare by rank?
Leading files record 15.9% with the name and 15.0% blocking it. Across all readable files, the corresponding shares are 10.7% and 11.2%.
What other entry is close in the index?
For comparison, anthropic-ai has 642 files naming it and 718 blocks, or 11.6%. Our counts for this entry are 658 files with the name and 694 blocking the whole site, or 11.2%.