Index Coverage Delta — The Diff Between What GSC Sees and What You Actually Publish

Google Search Console tells you a page is "Discovered, currently not indexed." It does not tell you which of the six likely reasons applies. Bing Webmaster Tools tells you "Crawled but not indexed" and similarly leaves the diagnosis to you.

The diagnosis is almost always a mismatch between three sets of URLs:

1. The URLs your site links to (what a crawler discovers naturally)
2. The URLs in your sitemap.xml (what you explicitly advertised to indexers)
3. The URLs search engines have already discovered and tried to fetch (visible in GSC Coverage and Bing Site Explorer)

A healthy site has those three sets roughly equal. A site with indexing problems has gaps between them, and the gaps, once surfaced, usually point at a specific fix.

The Index Coverage Delta tool exists to surface sets 1 and 2 in one pass so you can compare them against set 3 (what GSC shows) and know exactly what to fix.

Why this tool was built

Existing tools cover adjacent problems:

• Sitemap Audit validates that every URL in sitemap.xml returns 200.
• Internal Link Auditor finds broken internal links.
• Link Graph visualizes the link topology.
• Search Console + Bing Importer diffs your sitemap against a GSC or Bing CSV export.

None of them perform a live-crawl vs sitemap diff. That diff is what predicts whether GSC will index a URL:

• A URL in the sitemap with no internal links pointing at it is what Google calls "Discovered, currently not indexed." Google saw it, decided it did not look important enough (no internal link equity), and declined to index it.
• A URL with inbound internal links that is missing from the sitemap is an orphan from the sitemap's perspective. Bing in particular relies heavily on sitemap submission; it may never discover the URL.
• A URL that appears in both but has a canonical pointing elsewhere is silently self-rejecting for indexing.
• A URL that appears in both but returns 404 is the worst case, you told the crawler "index me," it tried, and you gave it an error. Over time this de-trusts the whole sitemap.

The three buckets

The tool produces three buckets:

Matched: the URL is in the sitemap and is reachable from the homepage via internal links within the crawl depth limit. These are the healthy URLs, they will be discovered via the sitemap AND via natural crawl AND ranked with link-equity signals.

Orphans: the URL is reachable via internal links but not in the sitemap. Two fixes: either add it to the sitemap (then submit a freshness ping via IndexNow so Bing picks it up), or decide it should not be indexed (in which case add a noindex tag or remove the internal links).

Ghosts: the URL is in the sitemap but nothing on the site currently links to it. This is what GSC reports as "Discovered, currently not indexed." The fix is almost always to add an internal link from a hub page (category, main menu, homepage section) to re-introduce the URL to Google's crawl. If the URL is genuinely not meant to be public, remove it from the sitemap.

The issue overlays

On top of the three-bucket classification, the tool layers issue detection:

• HTTP 4xx/5xx: the URL returns an error when probed
• redirect 3xx: the URL redirects, and sitemaps should point at the final URL not the redirect source
• noindex-in-sitemap: the URL is in the sitemap AND carries <meta name="robots" content="noindex">, a direct contradiction
• canonical→<other-url>: the page's canonical tag points to a different URL, which tells Google to treat that other URL as the real one and the current one as a duplicate

Each issue has a mechanical fix: remove from sitemap, update canonical, remove noindex, or replace the URL with its redirect target.

The IndexNow-ready output

The tool emits a JSON body formatted for the IndexNow API (https://api.indexnow.org/indexnow). IndexNow is a push-notification protocol backed by Bing, Yandex, Seznam, and others, you POST a URL list and crawlers are notified within minutes rather than waiting for the next sitemap crawl.

If you find 30 orphans, copy the IndexNow JSON, replace the key placeholder with your site's IndexNow key, POST it, and Bing will recrawl those URLs on average within 30 minutes. GSC does not participate in IndexNow yet but does respect sitemap resubmission, so the complementary move is "update sitemap, submit to GSC."

Why not just trust GSC?

Two reasons.

First, GSC and Bing index coverage reports lag the live state of your site by days to weeks. If you shipped 50 new pages yesterday, neither report reflects them yet. The Index Coverage Delta runs on your live site and your live sitemap, the picture is always current.

Second, GSC truncates. If your site has 10,000 URLs, GSC shows you aggregates plus sampled examples. The delta tool shows you the complete per-URL list up to the crawl limit, which matters when you're trying to fix hundreds of "Discovered, currently not indexed" URLs and need to know which hub page to add links from.

Running it

Paste your site's base URL. Optional knobs: max pages (default 60), max depth (default 3). The tool:

1. Fetches /sitemap.xml (auto-following sitemap-index children up to 12 deep)
2. BFS-crawls your site from the homepage, respecting <meta name="robots" content="nofollow"> and rel="nofollow" on individual links
3. For each crawled page, parses <link rel="canonical"> and <meta name="robots"> so it can flag conflicts
4. HEAD-probes the first 30 ghost URLs to catch 404s without a full crawl
5. Classifies every URL into match / orphan / ghost, layers on the issue overlays, and produces the report

Everything is client-side. No crawler servers, no OAuth, no data leaves your browser.

The companion workflow

The best workflow is:

1. Run Index Coverage Delta on your site weekly or after each deploy
2. Fix orphans by either adding them to the sitemap or removing the internal links
3. Fix ghosts by adding internal links from hub pages, not by re-submitting the sitemap
4. Run the Sitemap Audit monthly to confirm the sitemap itself is valid and every URL still returns 200
5. Cross-check against GSC and Bing Webmaster once a month using the Search Console + Bing Importer

Seven days after applying the fixes from the first two tools, you will see "Submitted and indexed" climb and "Discovered, currently not indexed" fall in GSC. That is the only real metric of whether the fix worked.

Related reading

• Sitemap Delta: Finding the URLs Your Site Links To That Are Not in Your Sitemap, the concept post
• Hosting & Indexing Health Checks: Fix GSC Errors Fast
• Hreflang Must Be Bidirectional, Why Google Demotes Multilingual Sites, a related indexing pathology
• Sitemap-Image.xml Guide

If you run a site with 1,000+ URLs and sell anything, books, services, SaaS, an indexing regression costs real money. The $97 Launch walks through the full indexing-hygiene checklist I use on the sites I build: The $97 Launch.

Fact-check notes and sources

• Google "Discovered, currently not indexed" definition: Google Search Central, Coverage report (accessed 2026-04-20)
• IndexNow protocol specification: indexnow.org, supported by Microsoft Bing, Yandex, Seznam, Naver as of 2026
• Hreflang bidirectional requirement: Google Search Central, hreflang documentation