Conformance Testing

MCP-TUI ships two testing subcommands: verify runs a small suite of targeted behavior probes, and conform runs the complete protocol matrix (every protocol scenario plus every probe). Both exit non-zero when anything fails, so they drop straight into CI.

capabilities — what did the server negotiate?

Before testing behavior, confirm the handshake. capabilities prints the negotiated server and client capabilities as JSON:

mcp-tui --transport http --url http://localhost:8000/mcp capabilities

server prints the server’s name, version, and protocol.

verify — behavior probes

Each probe sends a single targeted request and reports PASS, WARN or FAIL plus a human-readable fix suggestion. WARN marks a server that breaks a SHOULD in the spec: it is reported with its fix but does not fail the run. A probe the target cannot run is reported as SKIP with the reason (probe requires a stdio command target) and, as in conform, counts as passing. Every probe drives its own short-lived connection, so there is no persistent-session overhead.

mcp-tui verify http://localhost:8000/mcp                 # all probes; seterror-content skipped
mcp-tui verify --probe cross-origin http://localhost:8000/mcp
mcp-tui verify --json http://localhost:8000/mcp | jq '.results[]|select(.pass==false)'

Probe	Checks
cross-origin	Server rejects POST with a foreign Origin (SDK v1.4.1+)
dns-rebind	Server rejects 127.0.0.1 with a foreign Host header (SDK v1.4.0+)
content-type	Server rejects POST with a non-JSON Content-Type
origin-header	Origin enforcement is scoped to POST, not GET/HEAD
mcp-method-headers	Server tolerates SEP-2243 MCP-Method/MCP-Name headers
seterror-content	Tool-result errors preserve the Content payload (SDK v1.6.0+)
tool-names	Every tool name is 1-128 characters of A-Z a-z 0-9 _ - . (SEP-986)
list-order	tools/list returns the same order twice (2026-07-28 SHOULD; WARN when it does not)
protocol-violations	Listing tools, resources and prompts draws no message that breaks JSON-RPC 2.0 or the negotiated MCP version (stray or second responses, undefined methods, non-JSON-RPC lines)

The first five probes require a URL target. tool-names, list-order and protocol-violations take a URL or a stdio --cmd. Outside verify, every command reports the same violations on stderr after its output (see Protocol violations). list-order makes sure both lists reach the server: when the SDK answered the second from its TTL cache, it asks again on a new session. seterror-content runs against a stdio server and takes an optional --tool, a tool that fails by design, with its arguments in --tool-args (key=value or key:=<json>, as tool call takes them). Without --tool it calls echo, and is skipped when the server has no echo tool:

mcp-tui verify --probe seterror-content --cmd ./demo-server --args -stdio \
  --tool lookup_customer --tool-args customer_id=C-4040

conform — the full matrix

conform runs every scenario plus every verify probe and prints a per-scenario PASS/WARN/FAIL/SKIP summary. Warnings pass (in a JUnit report they are passing test cases with the warning in system-out). Skipped scenarios (features the server does not advertise, or probes the target kind cannot run, reported as skipped: probe requires a … target) count as passing.

mcp-tui conform http://localhost:8000/mcp
mcp-tui conform --cmd npx --args "@modelcontextprotocol/server-everything,stdio"
mcp-tui conform --scenario tools.list http://localhost:8000/mcp

Scenarios: initialize, tools.list, tools.call, tools.call.isError, resources.list, resources.read, resources.templates.list, prompts.list, prompts.get, sampling.createMessage, elicitation.create, notifications, completion.complete, and every probe as verify.<probe-name>.

Driving the harder scenarios

Some scenarios need a hint about which tool triggers a server callback, or how to shape a completion request:

Flag	Purpose
--sampling-trigger-tool	Tool that triggers sampling/createMessage (default sampleLLM)
--sampling-trigger-args	An argument for that tool, key=value or key:=<json> as in tool call; repeat for more
--elicit-trigger-tool	Tool that triggers elicitation/create (default startElicitation)
--elicit-trigger-args	An argument for that tool, as --sampling-trigger-args
--tool	Tool that fails by design, for tools.call.isError and verify.seterror-content (as verify --tool)
--tool-args	An argument for --tool in tools.call.isError and verify.seterror-content, as --sampling-trigger-args
--completion-prompt	Prompt name (or template URI with --completion-resource) for completion/complete
--completion-resource	Treat --completion-prompt as a resource template URI
--completion-arg	Argument name for completion/complete
--completion-prefix	Prefix value for completion/complete

The sampling and elicitation scenarios reply with the stub flags described in Client features. They pass only when calling the trigger tool made the server send sampling/createMessage (resp. elicitation/create) and the stub answered it; a trigger tool that returns without asking fails. Arguments are converted and checked against the tool’s input schema like tool call’s, and nothing is made up for a required argument: a trigger tool that needs arguments you did not pass is skipped with a message naming the flag.

mcp-tui conform http://localhost:8000/mcp \
  --sampling-trigger-tool draft_reply --sampling-trigger-args ticket_id=T-1042 \
  --sampling-stub "Thanks, we are on it."

tools.call calls non-destructive tools whose input schema accepts no arguments, in order, and passes on the first that answers without a tool error; it is skipped when none does. completion.complete is skipped when the server does not declare the completions capability.

tools.call.isError checks that a failing tool call comes back as isError: true with non-empty content. Name the failing call with --tool and, when it needs them, --tool-args; the scenario then fails if the tool is missing, succeeds, or returns an error with no content. Without --tool it picks a tool whose name contains error, fail or invalid (else the first tool), calls it with no arguments, and is skipped when that call succeeds.

mcp-tui conform http://localhost:8000/mcp --scenario tools.call.isError \
  --tool lookup_customer --tool-args customer_id=C-4040

CI with JUnit

--report-junit writes a JUnit XML report your CI dashboard can render, while the non-zero exit code fails the job:

mcp-tui conform --report-junit conform.xml --transport http --url "$MCP_URL"

- name: MCP conformance
  run: mcp-tui conform --report-junit conform.xml --transport http --url ${{ env.MCP_URL }}
- name: Publish report
  if: always()
  uses: your-favorite/junit-report-action@v1
  with:
    report_paths: conform.xml