Which pages failed during the crawl?
It separates server errors, missing pages, refusals and requests that could not finish.
What Check Engine measures
Responses marked as challenges are excluded from the main status groups. Server responses with HTTP status from 500 up to but below HTTP status 600 are critical. Missing responses with HTTP status 404 or HTTP status 410 are separated from refusals with HTTP status 401 or HTTP status 403 and other rejected client requests. A zero status with an error is divided into timeouts, malformed links, connection refusals on otherwise answering hosts and hard failures. Prose-like URL paths are guarded against misleading missing-page claims. For example, a refusal with HTTP status 403 needs a different review from a missing-page response with HTTP status 404.
The check uses response statuses, errors and incoming links from the crawl. Refused or challenged requests may leave the destination unchecked. Keep those outcomes in view when deciding which link to change.
How to read the result
Severity ranges from medium for several refusal or timeout cases to high for hard failures and broken internal links, and critical for server errors. A refusal means the audit could not read the page. It is not automatically proof that a visitor sees a missing page.
Why it matters
Different failures need different fixes. Separating a missing address from a server rejecting requests avoids telling you to delete a useful page when the real issue is access or crawl pressure.
How to fix it
Open the cited source link and review the saved response. Correct a malformed destination, restore genuinely missing content or redirect an obsolete address to a relevant replacement. Investigate server errors and request refusals with logs. Retest timeouts before declaring the page dead, especially when other pages on the same host still answer.
'Could not be reached' is not a diagnosis
What we found in field audits
Problems in Crawlability & indexing were found on 12 of the 14 sites we audited. These are area totals. Read the field-audit method.
Method note
backend/seo_engine/checks/indexability.py, broken_pages, revision 0bb8e8d3.
Common questions
Is an HTTP 403 response the same as a missing page?
No. HTTP status 403 represents a refusal here, while HTTP status 404 represents a missing response. Access restrictions need their own review.
How serious are server errors in the failed-page check?
The result is critical for server errors. Missing addresses, refusals and timeouts have separate treatment.
Are challenge pages treated as ordinary broken-page responses?
No. Responses marked as challenges are kept outside the main status groups. They can leave the destination unchecked.