PerplexityBot
PerplexityBot is listed as a search crawler, and the community reference lists its operator as Perplexity. In October 2026, 622 sites block it, a 10.1% share of readable files.
Who runs PerplexityBot?
| Fact | PerplexityBot reference entry |
|---|---|
| Operator | “Perplexity” |
| Purpose | “Search result generation.” |
| Does PerplexityBot obey robots.txt? | “Yes” |
| Frequency | “No information.” |
| Cloudflare managed list for PerplexityBot | No |
These facts quote the stored community ai.robots.txt reference.
“Crawls sites to surface as results in Perplexity.”
The listed use here is search. An owner may allow a search crawler to list a page or an assistant to retrieve it for someone, yet decline to supply the text for training.
How many sites block it?
The name occurs in 784 out of 6,171 readable files, a 12.7% share. Across the full sample, the whole-site Disallow: / block count is 622, or 10.1%. Files without that ban number 5,549.
Tranco’s top 1,000 sites yielded 552 readable files, including 120 files with its name, or 21.7%, and 89 files blocking it. Blocking is more common among the leading sites, at 16.1%. Outside those ranks, 533 of 5,619 files block it, or 9.5%.
| Readable files | Named by | Named share | Blocked by | Blocked share |
|---|---|---|---|---|
| PerplexityBot, all ranks | 784 | 12.7% | 622 | 10.1% |
| PerplexityBot, top 1,000 | 120 | 21.7% | 89 | 16.1% |
| PerplexityBot, remaining ranks | 664 | 11.8% | 533 | 9.5% |
The survey includes relevant User-agent: * instructions alongside instructions addressed to this crawler. Of 622 blocks, 181 come only from that general rule, without naming this crawler. Names and bans overlap in 441 files. We also found 343 files with a name and no whole-site ban, plus 5,387 files omitting the name. Case changes and repeated spellings are merged.
Has that changed since August?
Between August and October, 35 sites that had blocked it stopped and 28 started, so 7 fewer of the same 5,889 sites block it. A further 562 blocked it in both months, and 5,264 did so in neither. Keeping the sites the same lets us count changes in their rules.
August’s full sample had 626 blocks among 6,236 readable files. October’s was 622 among 6,171. Those monthly totals also reflect which files we could read on each visit.
| August to October | Files |
|---|---|
| Started blocking | 28 |
| Stopped blocking | 35 |
| Blocked in both | 562 |
| Blocked in neither | 5,264 |
| Net change | -7 |
How to block or allow it
Edit the root robots.txt file on your site. The robots.txt standard gives a PerplexityBot group precedence over User-agent: * for this crawler.
Block the whole site
User-agent: PerplexityBot
Disallow: /Allow access
User-agent: PerplexityBot
Allow: /It's the 6th most-named crawler of the 171 we counted in the crawler index.
Which other crawlers does this operator run?
How we counted
For PerplexityBot, we surveyed Tranco’s top 10,000 sites on 19 October 2026. Files had an 8-second timeout and a 262,144-byte limit. Missing, unreachable and oversized files do not contribute to these shares. Collection and repeat readings used scripts/robots_2026_08_06.py (robots.py at app commit 07104ff). See the full survey method.
Common questions
What does naming this crawler tell me?
This survey includes 343 files with the name and no whole-site ban, out of 784 naming files.
Are leading sites different?
At the top of the ranking, 21.7% of readable files name it and 16.1% block it. Across all ranks, those shares are 12.7% and 10.1%.
How does the nearby crawler compare?
The nearby ChatGPT-User entry reports 728 files that name it and 541 whole-site blocks, 8.8% of readable files. Here, 784 files mention the name and 622 block access, or 10.1%.