Meta AI Crawler + Link Preview Audit

Meta publishes no Muse crawler, user agent or website standard (checked 2026-09-24). This audit tests only what Meta documents: the five crawler tokens in robots.txt, the Facebook and WhatsApp link-preview limits, and what its crawlers actually receive from your server.

Context and background

Read the story behind this tool: There is no Muse standard. Here is what Meta actually documents.

Meta publishes no Muse crawler, user agent or website standard (checked 2026-09-24). This audit tests only what Meta documents. Every row links its source. Rows built on inference are labelled INFERRED or REPORTED and are not scored. There is no "Muse-ready" badge and no overall score, because there is no standard to score against.

Page URL

The first URL gets the full audit: robots.txt for Meta's five tokens, the page as seven user agents, Meta's own Range request, and the og:image as Facebook's crawler fetches it. Everything runs through jwatte.com's fetch proxy from a US data center, so timings and blocks are what that network sees, not what Meta's network sees.

More pages and options

Extra URLs (optional, up to 10, one per line) Or sample pages from the sitemap (used only when the box above is empty)

Each extra page costs two requests: the HTML as facebookexternalhit and a HEAD on its first og:image.

WhatsApp rules: head inside the first 300 KB, og:image under 600 KB, at least 300 px wide User-agent parity (first URL): Meta's crawlers, WhatsApp and a stock Chrome on Linux baseline Range test: Meta's sample request, Range: bytes=0-524288 Muse and Meta agents (informational rows, never scored)