ShippedChange check — first crawl stores a baseline

robots.txt blocks monitored URLs

What this page tests

Check robots.txt blocking (indexability.robots_txt_blocked), alert indexability.robots_txt_blocked.

robots.txt now disallows crawling of monitored URLs. Google may stop fetching those pages, so content updates and indexation can stall.

Why it matters

robots.txt now disallows crawling of monitored URLs. Google may stop fetching those pages, so content updates and indexation can stall.

How the fail is produced

robots.txt Disallow matches only this path. Other monitored URLs stay allowed.

This page is part of the TechDash Test Site used to exercise the TechDash desktop app. Each fixture isolates a single check so you can add one path at a time and see a clear pass or fail. Search engines and users rely on stable HTTP responses, index signals, metadata, and site files. When those signals drift, organic visibility, crawl efficiency, and measurement can all suffer. Keep the homepage as the healthy control origin, then monitor the specific path you want to fail. Substantial body copy is included so thin-content and soft-404 heuristics do not fire unless that is the intended test.