LCC
Few readable files block LCC in October 2026, at 46 sites or 0.7%. Its operator is unclear in the community reference.
Who runs LCC?
| Fact | LCC reference entry |
|---|---|
| Operator | “Unclear at this time.” |
| Purpose | “AI Data Scrapers” |
| Does LCC obey robots.txt? | “Unclear at this time.” |
| Frequency | “Unclear at this time.” |
| Cloudflare managed list for LCC | No |
These facts quote the stored community ai.robots.txt reference.
This entry is classed as a crawler collecting data for AI. An owner who wants search traffic may allow indexing and visits requested by users, yet decline to have the same content gathered for training datasets.
How many sites block it?
Within 6,171 readable robots.txt files, the name appears in 48, or 0.8%. Instructions blocking every path, Disallow: /, appear in 46, a 0.7% share. No whole-site ban is recorded for 6,125 files.
Tranco’s top 1,000 sites yielded 552 readable files, including 6 files with its name, or 1.1%, and 6 files blocking it. Blocking is more common among the leading sites, at 1.1%. Outside those ranks, 40 of 5,619 files block it, or 0.7%.
| Readable files | Named by | Named share | Blocked by | Blocked share |
|---|---|---|---|---|
| LCC, all ranks | 48 | 0.8% | 46 | 0.7% |
| LCC, top 1,000 | 6 | 1.1% | 6 | 1.1% |
| LCC, remaining ranks | 42 | 0.7% | 40 | 0.7% |
Blocking here counts whole-site bans recorded for it by name. A general ban for all crawlers is excluded. We found 46 files naming it and blocking the whole site, along with 2 files naming it without that ban. Another 6,123 files omit the name. We ignore capitalization and repeated names.
How to block or allow it
Edit the root robots.txt file on your site. The robots.txt standard gives a LCC group precedence over User-agent: * for this crawler.
Block the whole site
User-agent: LCC
Disallow: /Allow access
User-agent: LCC
Allow: /It's the 120th most-named crawler of the 171 we counted in the crawler index.
How we counted
For LCC, the survey covered 10,000 top Tranco sites, collected on 19 October 2026. Requests could run for 8 seconds, with a 262,144-byte file limit. Only available files we could read enter the percentages. Repeat readings are included, using scripts/robots_2026_08_06.py (robots.py at app commit 07104ff). See the full survey method.
Common questions
Does a User-agent line mean it is blocked?
The 48 files naming it include 2 files without a whole-site ban.
What do the top-ranked sites show?
For naming rules, 1.1% of the leading sample mentions it, compared with 0.8% overall. Whole-site blocking is 1.1% at leading ranks and 0.7% across the full sample.
Which nearby entry can I compare?
For comparison, ApifyWebsiteContentCrawler has 46 files naming it and 51 blocks, or 0.8%. Our counts for this entry are 48 files with the name and 46 blocking the whole site, or 0.7%.