← All posts

MCP Compatibility Test for Claude Code and Codex

A platform team can pass a source-level MCP review and still ship a server that disappears inside one coding agent, truncates the wrong part of an error in another, or exposes a schema that the model never uses correctly. That is the practical problem behind every MCP compatibility test in 2026.

The server is not only talking to a protocol. It is being translated into Claude Code tools, Codex tools, prompts, resources, approvals, visible status, model context, and sometimes slash commands. Compatibility has to cover that full path.

The useful target is not "does this server support MCP?" The useful target is: can the same MCP server behave predictably across Claude Code, Codex, and mixed coding-agent clients, with enough evidence that a platform team can upgrade clients without guessing?

The decision frame

For platform teams, MCP compatibility is a release gate. It decides whether one shared server can serve every coding-agent surface, whether a client-specific shim is needed, or whether a known limitation must be documented before rollout.

Layer What can pass What can still fail
Protocol JSON-RPC, _meta, version metadata, auth, notifications, result envelopes The client accepts the exchange but later hides or rewrites important details.
Server contract tools/list, tools/call, prompts, resources, schemas, annotations A valid schema becomes too large, too nested, or too ambiguous for client translation.
Client adaptation Tools are visible, resources attach, prompts become usable commands, approvals render A tool is deferred, dropped, truncated, renamed, or loaded differently by mode.
Model behavior The model sees enough of the tool contract to choose and call it correctly The catalog is technically present, but the model omits required fields or chooses the wrong tool.

That is why a compatibility suite needs two records for every test: the wire evidence and the model-visible evidence. The wire trace says what the server advertised. The model-visible trace says what the agent could actually use.

Start with the 2026-07-28 MCP baseline

The 2026-07-28 MCP revision changes several assumptions that older servers and harnesses may still carry. The brief points to a stateless protocol model where clients send protocol version and client capabilities in _meta on each request, and servers should not depend on connection or session identity for context.

That matters for coding agents because a long-lived stdio process is not the same thing as a durable conversation. A server that quietly stores conversation state by process can look fine in one local test and fail when a client changes runtime, reconnects, or sends independent requests.

The baseline should include these expectations from the brief:

  • Requests are stateless and self-contained.
  • Every request includes required _meta fields.
  • List endpoints should not vary by connection identity.
  • Tools, prompts, and resources are separate server-facing capabilities.
  • Elicitation, sampling, and roots are client-facing capabilities.
  • Progress, cancellation, authorization, logging, subscriptions, and extensions need explicit tests.
  • Request-scoped notifications such as progress remain tied to the request response stream.

Use the MCP 2026-07-28 specification, the MCP basic overview, and the MCP key changes as the source baseline. Then test each client against the server behavior, not against an assumption that "MCP support" means equivalent support.

Claude Code and Codex expose different surfaces

Claude Code and Codex both support MCP, but the brief shows that they optimize for different operational surfaces.

Claude Code documentation is especially explicit about the client adaptation layer. It covers HTTP, SSE fallback, stdio, WebSocket, OAuth, dynamic headers, project, user, and local scopes, resources in @ mentions, prompts exposed as slash commands, elicitation, output limits, schema handling, tool search, tool deferral, and v1/v2 runtime negotiation. It also documents client-side constraints such as output truncation, root-level combinator handling, invalid schema exclusion, and Anthropic-specific _meta annotations.

Codex documentation is strongest around shared configuration, authentication, and governance controls. The brief notes support across ChatGPT desktop, Codex CLI, and IDE extension with shared config.toml. It also lists stdio, streamable HTTP, bearer tokens, OAuth including CIMD/DCR, ChatGPT session auth for trusted first-party servers, server instructions, startup timeouts, tool timeouts, required servers, enabled and disabled tool lists, approval modes, per-tool output limits, OAuth callback controls, HTTP headers, and header helpers.

Comparatively:

Area Claude Code test focus Codex test focus
Discovery Tool search, tool deferral, prompts as slash commands, resources in mentions Shared config, server startup, tool discovery, required vs optional servers
Schemas Invalid schema exclusion, root combinators, headless vs interactive loading Large schema compaction, nested required fields, tool visibility after list changes
Auth OAuth, scoped OAuth, dynamic headers, connector auth, command-line login Bearer env vars, static and env headers, header helpers, OAuth CIMD/DCR, callbacks
Operations Runtime negotiation, output truncation, interaction annotations Startup and tool timeout budgets, approval modes, output limits, config parity

The compatibility suite should reflect those differences instead of forcing one vendor's mental model onto the other.

Build the suite as a matrix, not a demo

A demo proves that one happy path works once. A compatibility matrix proves which combinations keep working as the server, schema, client, and runtime mode change.

Use a matrix with these axes:

  • Client: Claude Code, Codex, and any mixed coding-agent clients you support.
  • Surface: Claude Code terminal, Claude Code desktop, claude -p, Codex CLI, Codex desktop, Codex IDE extension, ChatGPT web, and any plugin-hosted MCP surface in scope.
  • Transport: stdio, streamable HTTP, SSE fallback, WebSocket where relevant.
  • Capability: tools, prompts, resources, elicitation, sampling, roots, progress, cancellation, auth, subscriptions, logging, extensions.
  • Runtime mode: interactive, headless, first turn, resumed turn, required server, optional server.
  • Evidence: raw MCP exchange, client-visible catalog, model-visible declaration, actual model call, user-visible result.

For every cell, record pass, fail, degraded, or not supported. "Degraded" is important. A tool that appears but loses enum detail is not the same as a tool that disappears, but both can break production workflows.

Test tools and schemas harder than you think

The field reports in the brief point to schemas as a main compatibility risk. SineFrame M3 tested ten MCP features against Claude Code 2.1.287 and Codex 0.162.0 on October 9, 2026. Five features split the clients, and the failures did not overlap. The same report says the full run covered 66 executions, with Claude Code averaging 9.8 seconds per execution and Codex averaging 18.2 seconds.

The individual findings are more useful than the score. Reported split failures included Claude Code dropping a tool with an ids[] argument name, losing middle content in a long error, and losing text when a result included both text and structuredContent. Codex reportedly missed a new tool after a tool-list change and dropped required nested fields in a 14 KB schema.

Additional probes in the brief suggest a practical schema test pack:

  • Plain root object with simple required fields.
  • Root oneOf, anyOf, and allOf, both alone and beside properties.
  • Deeply nested required fields in a large schema.
  • Large enum and composition combinations.
  • Tuple schemas with prefixItems.
  • Unusual property names, including names with brackets.
  • Long tool names, similar tool names, and names that differ only by punctuation.
  • Tool count boundaries, including very large catalogs.
  • Output schemas paired with text-only results.

Use a conservative server-author baseline even if a client supports more. Prefer plain root object schemas, stable and short tool names, concise descriptions, and clear required fields. Put critical facts early in tool descriptions and error messages because truncation may remove the middle.

Test tools, resources, and prompts separately

A tool-only server should not have to pretend that tools are resources. That sounds obvious, but the brief cites OpenAI Codex issue #14242 as a compatibility pattern where /mcp could show a server enabled while Codex still tried resources/list, saw no resources, and failed to discover or use tools.

The test suite should include four server shapes:

  • Tool-only server with no resources and no prompts.
  • Resource-only server with no tools.
  • Prompt-only server with no tools or resources.
  • Mixed server with tools, prompts, resources, and templates.

For Claude Code, verify whether resources are visible through @ mentions and whether prompts become usable slash commands. For Codex, verify the client-visible behavior separately in CLI, desktop, and IDE extension, because the brief notes that ChatGPT web and plugin-hosted tools may not behave like local Codex MCP surfaces.

Auth compatibility is more than connection success

A server that connects once has not passed auth compatibility. Platform teams need to test setup, token refresh, scope mismatch, issuer mismatch, dynamic headers, credential redaction, and unauthorized tool-call errors.

Claude Code and Codex offer different auth setup paths. Claude Code covers OAuth, scoped OAuth, dynamic headers, connector auth, and command-line login. Codex covers bearer env vars, static and env headers, header helpers, OAuth CIMD/DCR, callback controls, and ChatGPT session auth for trusted first-party servers.

Run the same auth scenarios across clients:

  • Clean first connection.
  • Expired access token.
  • Expired refresh token.
  • Missing scope for a sensitive tool.
  • Wrong issuer or tenant.
  • Revoked credential.
  • Dynamic header missing or malformed.
  • Unauthorized tool call after server discovery succeeds.

The result should show what the user sees, what the model sees, and what the logs show. Those three views often differ.

Streaming and progress need user-visible tests

MCP progress and notifications are protocol concepts. The question for an agent workflow is whether the model and the user can use the information.

The brief cites Anthropic Claude Code issue #9767 as a streaming and events gap: a Codex MCP server emitted events visible to MCP Inspector, but Claude Code reportedly filtered intermediate events so the model could not see the session_id required for multi-turn replies. Treat that as a test case category, not as a permanent verdict on current behavior.

For long-running calls, test:

  • Progress notifications during an active request.
  • Large final output with important facts at the beginning, middle, and end.
  • Error output that exceeds client limits.
  • Mixed text and structuredContent.
  • Cancellation while the server is working.
  • Request-scoped notification visibility to the model and to the human.

The compatibility result should not simply say "progress supported." It should say where progress is visible, whether it affects model decisions, and whether final results preserve the required identifiers.

Client-specific probes to add

Generic MCP tests catch protocol and contract issues. Client-specific probes catch the failures that show up after translation.

For Claude Code, include:

  • Interactive mode vs headless claude -p.
  • Tool search enabled by default.
  • Deferred or lazy tool loading.
  • v1 vs v2 runtime negotiation where applicable.
  • Top-level schema combinators in interactive and headless modes.
  • Malformed connector schema behavior when a full tool list is sent.
  • Resources as @ mentions and prompts as slash commands.

The brief cites Anthropic Claude Code issues #56174 and #64113 for malformed schema risk. Issue #64113 is especially useful because it distinguishes interactive Claude Code, where deferred behavior avoided the bad schema until invocation, from headless claude -p, where eager loading caused a 400.

For Codex, include:

  • Codex CLI vs desktop vs IDE extension.
  • Shared config.toml behavior across local surfaces.
  • Required vs optional MCP servers.
  • Startup timeout and tool timeout behavior.
  • Tool discovery on first turn when many MCP servers are configured.
  • Tool-list changes after startup.
  • Per-tool approval modes and output limits.

The brief cites OpenAI Codex issue #21318 for startup and tool discovery as a first-turn performance hazard. Optional MCP servers can still hurt the first prompt if tools/list sits on the critical path.

Make visible tool lists part of startup

Silent disappearance is worse than a clear failure. A startup check should compare the server's advertised catalog with each client's visible tool list.

Use three snapshots:

  1. The server manifest: what the MCP server says it exposes.
  2. The client catalog: what Claude Code or Codex reports as available.
  3. The model-facing declaration: what the model can actually choose from.

Diff those snapshots on every compatibility run. Flag missing tools, renamed tools, changed required fields, truncated descriptions, dropped enum values, missing resources, missing prompts, and tools that become visible only after search or invocation.

A good failure message says, "Codex CLI 0.162.0 saw 47 of 48 tools. Missing: ticket.batch_update. Server advertised it in tools/list. Model declaration did not include it." That is actionable. "MCP failed" is not.

Compatibility budgets for server authors

The brief leaves several open questions about budgets for tool description length, schema size, output size, tool count, and error-result size. Until your organization has measured budgets, set conservative defaults and test exceptions explicitly.

Budget Conservative default Test when exceeded
Tool names Short, stable, distinct names without near-collisions Name collision, punctuation variants, and long names
Input schemas Plain root object, shallow nesting, clear required fields Root combinators, tuples, large compositions, 10 KB plus schemas
Descriptions Put the selection rule in the first few sentences Long descriptions and deferred tool search
Outputs Return a concise text summary plus structured fields when needed Long errors, mixed text and structured content, truncation points
Catalog size Group tools around clear workflows Hundreds or thousands of tools on one server

The direct workaround pattern is simple: design for the least surprising model-facing declaration, then add richer forms only where tests prove both clients preserve them.

Put the suite in CI before client upgrades

The suite should run when the MCP server changes, when schemas change, when auth settings change, and before upgrading Claude Code or Codex versions across the team.

A practical CI gate has four stages:

  1. Spec pass: validate protocol behavior, required _meta, list endpoints, result envelopes, auth responses, and cancellation.
  2. Catalog pass: compare advertised tools, prompts, and resources with each client-visible catalog.
  3. Invocation pass: run deterministic calls that verify required fields, schema translation, output shape, errors, progress, and truncation.
  4. Agent pass: ask each agent to complete a realistic workflow and verify the selected tool, arguments, result handling, and user-visible status.

Pin client versions in the report. The SineFrame M3 result is useful partly because it named Claude Code 2.1.287 and Codex 0.162.0. Without versions, a compatibility report becomes folklore.

Decision checklist

Use this checklist before shipping one MCP server across Claude Code, Codex, and mixed coding-agent clients:

  • Does the server behave as stateless MCP 2026-07-28, without hidden connection-state assumptions?
  • Do tools, prompts, resources, and templates pass independently?
  • Does every target client show the expected catalog?
  • Does the model-facing declaration preserve required fields and critical descriptions?
  • Do interactive and headless modes behave the same where your workflows require it?
  • Do auth failures produce useful user-visible and log-visible errors?
  • Are progress and streaming events visible where the workflow depends on them?
  • Are output truncation points tested for normal results and errors?
  • Are tool-list changes tested after startup?
  • Is every client version pinned in the compatibility report?

Related reading: coding-agent observability, MCP security for AI agents and MCP permission gateway for Claude Code.

The bottom line

An MCP compatibility test is not a validator run. It is a record of what survives the full path from server contract to client adaptation to model-visible behavior.

For platform teams, the safest default is to keep the server simple, make the catalog observable, test Claude Code and Codex as different clients, and gate upgrades with pinned evidence. When a feature works in both clients, ship it as shared MCP. When it works in only one, decide deliberately: shim it, simplify it, or document the gap before engineers depend on it.

Get started

Deploy your fleet.

Put a fleet of sandboxed agents to work on your own infrastructure, provisioned in seconds and watched live from one console.

Get started →

Admin-provisioned · Self-host in one command · Your data never leaves your VM