A working MCP connector and a connector that works for people are not the same thing. The standard check is the protocol handshake: send initialize, send tools/list, confirm the tools come back. Every endpoint we publish passed that check. Then Fable, the AI model I use for this kind of analysis, ran a second suite that behaves like real assistants and real questions, and the results were humbling.
When we tested ours:
- 25 tools on 5 servers rejected a normal call. An assistant sent
nullfor an optional argument it had left unset, and our schema validation answered "must be an integer." - A blog connector answered "latest articles" with a May post. Its search ranked by keyword relevance, so the word "latest" matched an old article that happened to use it.
- Availability tools reported rooms free in January 2020 and built booking links for those dates, because nothing rejected a date in the past.
None of these would show up in a handshake test. All three would show up the first time a real person asked a real question. Here are the eleven checks we now run before every release and every week, and a free script that runs the generic ones against your own connector.
Why the handshake is not enough
The handshake proves the server speaks the protocol. It does not prove the server handles the way clients actually fill in arguments, or that its answers are current, correct and safe to hand to a person. Those are separate questions, and each needs its own check.
The eleven checks
1. Null optional arguments
Some assistant clients send null for an optional argument the model did not set, instead of leaving it out. A strict schema validator then rejects the whole call. In our testing ChatGPT did this, and it accounted for all 25 failures above.
Check: call every tool with each optional argument set to null. It must behave exactly as if the argument were absent.
Fix: before schema validation, drop optional arguments whose value is null, unless the schema explicitly allows null.
2. _meta on tool calls
The MCP specification reserves a _meta field for clients to attach their own metadata, and some clients send it on every tools/call (MCP specification).
Check: call every tool with params._meta present. Nothing should change.
3. Freshness: "what is new"
A keyword search cannot answer a recency question. "Latest," "newest" and "this week" are not keywords to match; they are requests to sort by date.
Check: compare the connector with your site's own feed or sitemap. The newest few items must come back, newest first, from a date-sorted listing tool, and each must also be findable by searching for its own title. A query made only of a recency word should be answered by date.
Fix: add a listing tool that sorts by date with limit, offset and since arguments, and route recency-only queries to it. We added one to our own blog connector, and its search now answers recency and date questions by publication date.
4. Date validation
Any tool that takes dates needs to refuse the impossible ones.
Check: past dates, a check-out before the check-in, and a stay longer than your maximum. Each should return a clear error with a next step, not availability.
5. Drift
The facts in a connector's answer have to match the page it cites. Prices are the easiest place to see drift, because a site's prices change and a connector's copy of them can lag behind.
Check: for every answer that cites a source page, confirm each price in the answer appears on that page. Compare amounts written with cents only, because general prose often rounds.
6. Parity
If the same question can be asked through two surfaces, such as a hub connector and a single-location connector, or a full endpoint and a read-only endpoint built for one assistant, the answers must agree.
Check: ask both and compare the totals.
7. Links
Every URL a connector hands to a person is a promise.
Check: every link in every answer must open with an HTTP status below 400. Skip the MCP endpoint itself: it usually answers a plain browser request with an error or an event stream, so it looks broken even when it works.
8. Clean failures
An unknown tool, a wrong-typed argument and a lookup that finds nothing are all normal events. The specification separates protocol errors, such as an unknown tool or invalid arguments, which use JSON-RPC error code -32602, from tool execution errors, which come back in the result with isError: true (MCP tools specification).
Check: each of these must come back as isError or -32602 with a useful next step, never as an HTTP 5xx.
9. Language and wording
If your site has pages in more than one language, check which tools accept a language argument. In one of ours, only 3 of 9 tools did. Then read the error wording. A valid choice that one location simply does not offer should not be called "unknown."
10. Description length
Tool descriptions are what an assistant reads to decide which tool to call. Long ones get hard to use. Our check warns on any description over 1,024 characters.
11. Read-only safety and pacing
A test suite should never change anything. Ours never calls a tool unless its readOnlyHint annotation is exactly true, it waits about a second between requests to the same host, and it sends no credentials. It runs as a release gate and again every week, because drift and freshness problems appear between releases, not at them.
Run the generic checks yourself
The free script mcp-connector-scenario-tests.mjs runs checks 1, 2, 3, 7, 10 and 11, plus the unknown-tool and wrong-type parts of check 8, against any Streamable HTTP connector you run or have permission to test. It completes the handshake the way a client does, including any session ID the server issues. It needs Node 18 or later and has no dependencies.
# Contract and compatibility checks
node mcp-connector-scenario-tests.mjs --endpoint https://example.com/mcp
# Add a freshness check against your own feed
node mcp-connector-scenario-tests.mjs --endpoint https://example.com/mcp \
--feed https://example.com/feed.xml --search-tool search_site \
--recent-tool list_recent_articles --n 5
If a tool has required inputs that a placeholder cannot satisfy, give it a real example with --sample tool_name='{"slug":"your-slug"}'. The script exits with 1 if anything fails, so it can sit in a deploy pipeline. Checks 4, 5, 6 and 9 depend on what your business sells and how your site is built, so the script leaves them to you. The descriptions above are the specification for writing them.
What to do with the results
- Fix the null and
_metafailures first. They are small code changes, and they block real users of the biggest assistants. - Add a date-sorted listing tool if you publish anything that changes: articles, events, menus, inventory.
- Put the suite in your release process, and schedule it weekly. A connector that passed on launch day can drift by the next price change.
- Keep the human boundary clear. A test that hands a person a broken link, or a room in 2020, is a test of trust, not just code.
Related reading
- Where to List Your MCP Connector in 2026, Step by Step: the readiness checks before any directory.
- Your MCP Connector Works. Here Is What Getting Listed Takes: review, documentation and revalidation after listing.
- Build Your Own MCP Connector for Programs, Events and Business: blueprints, with a test record for each release.
- MCP servers for small business: what an MCP server is, and how to vet someone else's.
Fact-check notes and sources
- Findings ("25 tools on 5 servers", the "latest articles" answer, the January 2020 availability): from our own scenario run across every connector we publish, November 2026. Server names are omitted on purpose. Since the run, the null-argument fix and a date-sorted listing tool have shipped on our blog connector, where a rerun of the script now passes, and the past-date fix has shipped for the lodging tools. The null-argument fix was still pending on our other servers when this was published.
_meta: MCP specification, basic protocol. Protocol errors,-32602,isError: MCP specification, tools.- The script is published under the site's terms at /downloads/mcp-connector-scenario-tests.mjs. We ran it against our own blog connector on November 21, 2026 (Pacific time) before publishing, and it was reviewed twice by Fable and once by Opus; version 1.2 includes those review fixes.
This post is informational, not legal or security advice. Run tests only against servers you operate or have permission to test.