FacebookBot
FacebookBot collects content for model training, and the community reference lists its operator as Meta/Facebook. Our October 2026 survey found 370 sites blocking it, or 6.0% of the files we could read.
Who runs FacebookBot?
| Fact | FacebookBot reference entry |
|---|---|
| Operator | “Meta/Facebook” |
| Purpose | “Training language models” |
| Does FacebookBot obey robots.txt? | “Yes” |
| Frequency | “Up to 1 page per second” |
| Cloudflare managed list for FacebookBot | No |
These facts quote the stored community ai.robots.txt reference.
“Officially used for training Meta "speech recognition technology," unknown if used to train Meta AI specifically.”
Model training is this crawler’s listed use. Search indexing is aimed at helping people find pages. Feeding text into model training is another use an owner may decline.
How many sites block it?
The name occurs in 452 out of 6,171 readable files, a 7.3% share. Across the full sample, the whole-site Disallow: / block count is 370, or 6.0%. Files without that ban number 5,801.
Among 552 readable files from the top 1,000 sites, we found 60 files naming it, a 10.9% share, with 46 files blocking the whole site. Blocking is more common among the leading sites, at 8.3%. Outside this group, 5,619 files contribute 324 blocks, or 5.8%.
| Readable files | Named by | Named share | Blocked by | Blocked share |
|---|---|---|---|---|
| FacebookBot, all ranks | 452 | 7.3% | 370 | 6.0% |
| FacebookBot, top 1,000 | 60 | 10.9% | 46 | 8.3% |
| FacebookBot, remaining ranks | 392 | 7.0% | 324 | 5.8% |
Blocking here counts whole-site bans recorded for it by name. A general ban for all crawlers is excluded. The overlap between naming and blocking is 370 files. Naming without a whole-site ban accounts for 82, and 5,719 files never name it. We ignore capitalization and repeated names.
How to block or allow it
Edit the root robots.txt file on your site. The robots.txt standard gives a FacebookBot group precedence over User-agent: * for this crawler.
Block the whole site
User-agent: FacebookBot
Disallow: /Allow access
User-agent: FacebookBot
Allow: /It's the 18th most-named crawler of the 171 we counted in the crawler index.
Which other crawlers does this operator run?
How we counted
For FacebookBot, we surveyed Tranco’s top 10,000 sites on 19 October 2026. A request timed out after 8 seconds, with files capped at 262,144 bytes. Shares exclude unreachable, unavailable and oversized files. Parsing includes files read again, through scripts/robots_2026_08_06.py (robots.py at app commit 07104ff). See the full survey method.
Common questions
Does a User-agent line mean it is blocked?
The 452 files that mention its name do not all record a whole-site ban. Those without one total 82.
What do the top-ranked sites show?
Within readable robots.txt files near the top of the ranking, 10.9% name it and 8.3% block it. Overall, naming reaches 7.3% and blocking 6.0%.
Which nearby entry can I compare?
For comparison, YouBot has 451 files naming it and 583 blocks, or 9.4%. Our counts for this entry are 452 files with the name and 370 blocking the whole site, or 6.0%.