AI crawlers / Hosts / WordPress.com

Does WordPress.com block AI bots by default?

WordPress.com sharing settings says: WordPress.com does not state a starting block rule for these six names in its public-site guide. It says enabling Prevent third-party sharing adds known AI bots to robots.txt and excludes public content from its partner network.

Open Settings > Reading > Site Visibility. Inspect Prevent third-party sharing for this site and save any change. The guide says the request depends on AI platforms honoring it. It gives no list of the known bots covered.

Rules for each bot

GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Applebot-Extended, CCBot

WordPress.com sharing settings says: The docs do not say. The sharing checkbox adds known AI bots without naming this bot or its starting state. Review Settings > Reading > Site Visibility > Prevent third-party sharing, then inspect the live file.

What we measured

We did not measure any customer site on this host.

Start by checking site visibility

WordPress.com sharing settings says: The guide explains how to make a WordPress.com site public. In Settings, choose Reading, find Site Visibility and select Public. If Public is absent, the guide says the site has not been launched. Use the current published address when checking bot access. A reading from a different site in your account cannot establish this site’s choices.

Public visibility also offers Discourage search engines from indexing this site. The guide says some search engines may ignore that request and enabling it also keeps blog posts out of the WordPress.com Reader. Read that setting alongside the AI sharing checkbox when investigating a broad loss of crawler access. Each setting expresses a different request.

What Prevent third-party sharing changes

WordPress.com sharing settings says: WordPress.com says it works with third-party partners that license content for research and related uses, including AI training. Enabling Prevent third-party sharing excludes this site’s public content from that network. It also adds known AI bots to the disallow list in robots.txt. That combines a partner-sharing choice with a request to web crawlers. The guide does not name every bot on that list.

To enable it, keep Public selected in Site Visibility, tick Prevent third-party sharing and choose Save Changes at the bottom. Inspect each website you manage because the instructions say to update each site separately. If you later change your choice, read the saved setting and the public file again. Keep the before and after text so the result can be tied to that one site.

Verify the public request

WordPress.com sharing settings says: Fetch robots.txt from the public domain. Search the served text for GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Applebot-Extended and CCBot, then inspect any group for all crawlers. The six names are a checklist for your own inspection. Their appearance or absence in your file is more specific than the guide’s phrase known AI bots. Do not assume a missing name is included by the checkbox.

Next, request a public post using the browser and named headers below. Record the headers and status separately from the robots.txt instructions. WordPress.com says AI platforms decide whether to honor the request, so an ordinary HTTP response is compatible with a file asking a bot to stay away. A curl request supplies a name, without proving that a real visit from that company will follow the file.

curl -sS -L https://example.com/robots.txt
curl -sS -L -D - -o /dev/null -A 'Mozilla/5.0' https://example.com/
curl -sS -L -D - -o /dev/null -A 'GPTBot' https://example.com/
curl -sS -L -D - -o /dev/null -A 'ClaudeBot' https://example.com/
curl -sS -L -D - -o /dev/null -A 'PerplexityBot' https://example.com/
curl -sS -L -D - -o /dev/null -A 'Google-Extended' https://example.com/
curl -sS -L -D - -o /dev/null -A 'Applebot-Extended' https://example.com/
curl -sS -L -D - -o /dev/null -A 'CCBot' https://example.com/

Common questions

Is this guide for every self-hosted WordPress site?

The source describes WordPress.com’s own public-site and partner-sharing controls. Check that you are using the WordPress.com setting shown here before following the dashboard steps.

Do I need to update every WordPress.com website?

Yes. The support guide tells owners of multiple websites to update Prevent third-party sharing for each site they want to exclude. One site’s saved checkbox does not describe another site’s choice.

Which six bots does the sharing checkbox cover?

The saved guide says known AI bots without listing names. It does not confirm coverage for any particular name here. Read the public robots.txt after saving to inspect the request your site serves.

Will Discourage search engines affect WordPress.com Reader?

The guide says enabling that option excludes blog posts from the Reader. Check it separately from Prevent third-party sharing when you review the Site Visibility controls.

Check your site

Start with the free AI crawler checker to read the rules for your address. Then download Check Engine for an audit on your computer.

Sources