Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 54 additions & 11 deletions .github/workflows/conformance.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ on:
branches: [main]
workflow_dispatch:

env:
CONFORMANCE: '@modelcontextprotocol/conformance@0.2.0-alpha.12'

jobs:
server:
name: Server Conformance
Expand All @@ -26,13 +29,48 @@ jobs:
mvn exec:java -pl conformance-tests/server-servlet -Dexec.mainClass="io.modelcontextprotocol.conformance.server.ConformanceServlet" &
timeout 30 bash -c 'until curl -s http://localhost:8080/mcp > /dev/null 2>&1; do sleep 0.5; done'

- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '22'

- name: Run conformance tests
uses: modelcontextprotocol/conformance@v0.1.16
run: |
npx --yes "$CONFORMANCE" server \
--url http://localhost:8080/mcp \
--suite active \
--spec-version 2025-11-25 \
--expected-failures ./conformance-tests/conformance-baseline.yml

modern-server:
name: Modern Server Conformance (2026-07-28)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Set up JDK 17
uses: actions/setup-java@v4
with:
mode: server
url: http://localhost:8080/mcp
suite: active
expected-failures: ./conformance-tests/conformance-baseline.yml
java-version: '17'
distribution: 'temurin'
cache: 'maven'

- name: Build and start server
run: |
mvn clean install -DskipTests
mvn exec:java -pl conformance-tests/server-servlet-modern &
timeout 30 bash -c 'until curl -s http://localhost:8081/mcp > /dev/null 2>&1; do sleep 0.5; done'

- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '22'

- name: Run conformance tests
run: |
npx --yes "$CONFORMANCE" server \
--url http://localhost:8081/mcp \
--requirements 2026-07-28

client:
name: Client Conformance
Expand All @@ -53,13 +91,18 @@ jobs:
- name: Build client
run: mvn clean install -DskipTests

- name: Run conformance test
uses: modelcontextprotocol/conformance@v0.1.16
- name: Set up Node.js
uses: actions/setup-node@v4
with:
mode: client
command: 'java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-*-SNAPSHOT.jar'
scenario: ${{ matrix.scenario }}
expected-failures: ./conformance-tests/conformance-baseline.yml
node-version: '22'

- name: Run conformance test
run: |
npx --yes "$CONFORMANCE" client \
--command 'java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-*-SNAPSHOT.jar' \
--scenario ${{ matrix.scenario }} \
--spec-version 2025-11-25 \
--expected-failures ./conformance-tests/conformance-baseline.yml

auth:
name: Auth Conformance
Expand Down
6 changes: 3 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,13 +114,13 @@ Records in `McpSchema` are serialized directly to the MCP JSON wire format. The

When the MCP specification marks a field as required, callers must not be able to construct a structurally invalid record, but the wire parser must still tolerate peers that fail to send it. Follow these rules in addition to the relevant Case A rules (annotation, naming, append-only).

1. **Reject `null` in the compact constructor.** Use `Assert.notNull` for required objects or `Assert.hasText` for required `String` identifiers (`name`, `uri`, `uriTemplate`, `version`). This throws `IllegalArgumentException` at construction time instead of producing a record that fails later in serialization or protocol handling. Overrides Case A Rule 7 for this field.
1. **Reject `null` in the compact constructor** with `Assert.notNull`. This throws `IllegalArgumentException` at construction time instead of producing a record that fails later in serialization or protocol handling. Overrides Case A Rule 7 for this field. The compact constructor is the one path that both Java callers and `fromJson` (Rule 2) go through, so it may only enforce what a wire default can satisfy. Stricter Java-side checks, such as non-empty identifiers (`Assert.hasText` on `name`, `uri`, `uriTemplate`, `version`), belong in the required-first builder factory (Rule 3).
2. **Add a `@JsonCreator` static `fromJson` factory** alongside the canonical constructor. When a required field is absent from the wire, substitute a documented safe default (`""` for strings, `[]` for collections, `{}` for maps, `0` / `0.0` for numerics, `INFO` for `LoggingLevel`, etc.) and log at `WARN` naming the field and the value used. The SDK must not halt the conversation because of a missing field. Place `@JsonCreator` on this `fromJson` factory, never on the canonical constructor (Case A Rule 6 still applies to the canonical constructor itself).
- Exception: `JSONRPCResponse.JSONRPCError` fails fast on missing `code` / `message` because a malformed JSON-RPC error envelope is unrecoverable.
- Exception: do not substitute a default when the missing value would make the message unprocessable anyway. This covers `JSONRPCResponse.JSONRPCError` (`code` / `message`), since a malformed JSON-RPC error envelope is unrecoverable, and the identifiers a request is dispatched on (e.g. `name` of `tools/call` and `prompts/get`, `uri` of `resources/read`, `ref` of `completion/complete`). A missing dispatch identifier must fail fast with an `INVALID_PARAMS` error, not an internal error.
3. **Provide a required-first builder factory** `builder(req1, req2, …)` and remove the corresponding setters from the `Builder`. A no-arg `builder()` factory must not exist on a record that has required fields. If one already exists for source compatibility, mark it `@Deprecated`.
4. **Add tests per required field**:
- Constructing the record with `null` for the field throws `IllegalArgumentException`.
- Deserializing JSON *without* the field succeeds and yields the documented default.
- Deserializing JSON *without* the field succeeds and yields the documented default, or, for a Rule 2 exception, fails.
- Deserializing JSON with an extra *unknown* field still succeeds (Case A Rule 8 also applies).

### Example
Expand Down
76 changes: 68 additions & 8 deletions conformance-tests/VALIDATION_RESULTS.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
# MCP Java SDK Conformance Test Validation Results

Last validated: **2026-08-17** against conformance suite
**`@modelcontextprotocol/conformance@0.2.0-alpha.11`** (SDK at `main`, 2.0.1-SNAPSHOT), targetting
version 2025-11-25 (`--spec-version 2025-11-25`).
Last validated: **2026-10-02** against conformance suite
**`@modelcontextprotocol/conformance@0.2.0-alpha.12`** (2.1.0-SNAPSHOT), targeting version 2025-11-25
(`--spec-version 2025-11-25`) for the legacy server and clients, and the 2026-07-28 requirement set
(`--requirements 2026-07-28`) for the modern server. Auth results below were last validated with
`0.2.0-alpha.11`.

## Summary

**Modern Server Tests (2026-07-28 requirements):** 37/37 required scenarios passed (`server-servlet-modern`)
**Server Tests (active suite):** 73/73 checks passed (31 scenarios, 100%)
**Server Tests (SEP-1613 `json-schema-2020-12`):** 5/5 checks passed (SEP-2106 checks skipped — post-2025-11-25 spec additions)
**Client Tests:** 3/4 scenarios passed; `sse-retry` fails (tracked in `conformance-baseline.yml`)
Expand All @@ -14,6 +17,34 @@ version 2025-11-25 (`--spec-version 2025-11-25`).
Baseline check passed on every run: all failures are expected per
[`conformance-baseline.yml`](conformance-baseline.yml).

## Modern Server Test Results (2026-07-28)

The `server-servlet-modern` module serves the stateless 2026-07-28 revision with
`io.modelcontextprotocol.modern.server.McpServer` over `HttpServletMcpTransport`.

### Required — Passing (37/37 scenarios)

- **Stateless lifecycle (SEP-2575):** `_meta` validation, `server/discover`, version negotiation,
`MCP-Protocol-Version`/`Mcp-Method` header mismatch, `-32021` capability enforcement, removed
methods answered `404`/`-32601`, `subscriptions/listen` acknowledgement and filtering
- **Tools, Resources, Prompts, Completion:** all content types, progress, resource templates,
SEP-2164 not-found errors
- **Caching (SEP-2549):** `ttlMs`/`cacheScope` on list results and `resources/read`
- **InputRequiredResult / MRTR (SEP-2322):** all 14 scenarios, including multi-round, tampered
`requestState` and capability checks
- **Security:** DNS rebinding protection, SSE streams

### Not scored for 2026-07-28

These run but don't count: the frozen requirement set marks extensions and anything that was pending
in the anchor release (`0.2.0-alpha.10`) as `not_scored`. Pending scenarios are still spec requirements.

- **Passing:** `json-schema-2020-12` (8/8 checks, including the SEP-2106 `allOf`/`anyOf`,
`if`/`then`/`else` and `$anchor` checks), `http-header-validation` (14/14)
- **Failing — `http-custom-header-server-validation` (SEP-2243 custom headers):** not implemented,
see [Known Limitations](#known-limitations)
- **Failing — tasks extension (`tasks-*`, SEP-2663):** not implemented

## Server Test Results

### Active Suite — Passing (31/31 scenarios, 73/73 checks)
Expand Down Expand Up @@ -66,9 +97,38 @@ of the 0.2.0-alpha auth suite.

1. **Client SSE Retry:** client doesn't parse or respect the `retry:` field,
reconnects immediately, and doesn't send the `Last-Event-ID` header
2. **Modern server: SEP-2243 custom headers (`Mcp-Param-{Name}`) are not supported.** SEP-2243 is part
of 2026-07-28 and the server-side requirements are MUSTs; only the scenario's pending status keeps
it out of the score. Today the five custom-header checks report "not testable" because no tool
carries `x-mcp-header`. Adding such a tool would turn them into real failures, because nothing
validates the headers yet. Supporting it needs SDK work:
- **Tool definitions:** `Tool.inputSchema` is a free-form map, so `x-mcp-header` can already be
written, but nothing enforces the definition rules: value non-empty, ASCII without space or `:`,
case-insensitively unique per tool, only on `integer`/`string`/`boolean` parameters (not `number`).
- **Request validation on `tools/call`:** for each designated parameter, Base64-decode
`=?base64?…?=` values, check the header matches the body value (integers as decimal strings,
booleans as `true`/`false`), reject headers with invalid characters, don't expect a header when
the value is null or omitted, and reject a missing required parameter. Failures are answered with
`400` and `-32020`.
- **Where it lives:** the check needs the called tool's `inputSchema`, which
`HttpServletMcpTransport` doesn't have. `ToolsFeature` already looks the `Tool` up through
`McpSyncToolRepository#find` / `McpAsyncToolRepository#find` and validates the arguments against
`inputSchema` before calling the tool, so the header check fits there. What's missing is a way to
get the raw `Mcp-Param-*` headers from the transport to the feature (e.g. through the
`McpTransportContext` on `McpRequestContext`).

## Running Tests

### Modern Server (2026-07-28)
```bash
./mvnw clean install -DskipTests
./mvnw exec:java -pl conformance-tests/server-servlet-modern

# In another terminal
npx @modelcontextprotocol/conformance@0.2.0-alpha.12 server \
--url http://localhost:8081/mcp --requirements 2026-07-28
```

### Server (active suite)
```bash
# Build and start server
Expand All @@ -77,21 +137,21 @@ mvn exec:java -pl conformance-tests/server-servlet \
-Dexec.mainClass="io.modelcontextprotocol.conformance.server.ConformanceServlet"

# Run tests (in another terminal, from the repo root)
npx @modelcontextprotocol/conformance@0.2.0-alpha.11 server \
--url http://localhost:8080/mcp --suite active \
npx @modelcontextprotocol/conformance@0.2.0-alpha.12 server \
--url http://localhost:8080/mcp --suite active --spec-version 2025-11-25 \
--expected-failures ./conformance-tests/conformance-baseline.yml
```

### Server (SEP-1613 scenario)
```bash
npx @modelcontextprotocol/conformance@0.2.0-alpha.11 server \
--url http://localhost:8080/mcp --scenario json-schema-2020-12
npx @modelcontextprotocol/conformance@0.2.0-alpha.12 server \
--url http://localhost:8080/mcp --scenario json-schema-2020-12 --spec-version 2025-11-25
```

### Client
```bash
for scenario in initialize tools_call elicitation-sep1034-client-defaults sse-retry; do
npx @modelcontextprotocol/conformance@0.2.0-alpha.11 client \
npx @modelcontextprotocol/conformance@0.2.0-alpha.12 client --spec-version 2025-11-25 \
--command "java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-*.jar" \
--scenario $scenario \
--expected-failures ./conformance-tests/conformance-baseline.yml
Expand Down
1 change: 1 addition & 0 deletions conformance-tests/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
<module>client-jdk-http-client</module>
<module>client-spring-http-client</module>
<module>server-servlet</module>
<module>server-servlet-modern</module>
</modules>

</project>
34 changes: 34 additions & 0 deletions conformance-tests/server-servlet-modern/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# MCP Conformance Tests - Modern Servlet Server

Conformance server for the stateless **2026-07-28** MCP revision, built on
`io.modelcontextprotocol.modern.server.McpServer` and `HttpServletMcpTransport` in an embedded Tomcat.
The legacy (2025-11-25) server lives in [`server-servlet`](../server-servlet).

See [VALIDATION_RESULTS.md](../VALIDATION_RESULTS.md) for the latest results.

## Running

```bash
./mvnw clean install -DskipTests
./mvnw exec:java -pl conformance-tests/server-servlet-modern
```

The server listens on `http://localhost:8081/mcp` (override the port with `-Dport=<port>`).

```bash
npx @modelcontextprotocol/conformance@0.2.0-alpha.12 server \
--url http://localhost:8081/mcp --requirements 2026-07-28
```

## Fixtures

Besides the standard `test_*` tools, resources and prompts, the server exposes the fixtures the
2026-07-28 scenarios probe for:

- **SEP-2575 diagnostics:** `test_missing_capability`, `test_streaming_elicitation`, `test_logging_tool`,
`test_trigger_tool_change`, `test_trigger_prompt_change`
- **SEP-2322 MRTR:** `test_input_required_result_*` tools and the `test_input_required_result_prompt` prompt
- **SEP-1613 / SEP-2106:** `json_schema_2020_12_tool`

SEP-2243 custom headers (`x-mcp-header` / `Mcp-Param-{Name}`) are not supported yet; see
[Known Limitations](../VALIDATION_RESULTS.md#known-limitations).
75 changes: 75 additions & 0 deletions conformance-tests/server-servlet-modern/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>io.modelcontextprotocol.sdk</groupId>
<artifactId>conformance-tests</artifactId>
<version>2.1.0-SNAPSHOT</version>
</parent>
<artifactId>server-servlet-modern</artifactId>
<packaging>jar</packaging>
<name>MCP Conformance Tests - Modern Servlet Server</name>
<description>Servlet Server conformance tests for the modern (2026-07-28) Java MCP SDK server</description>
<url>https://github.com/modelcontextprotocol/java-sdk</url>

<scm>
<url>https://github.com/modelcontextprotocol/java-sdk</url>
<connection>scm:git:git://github.com/modelcontextprotocol/java-sdk.git</connection>
<developerConnection>scm:git:ssh://git@github.com/modelcontextprotocol/java-sdk.git</developerConnection>
</scm>

<properties>
<module.name>io.modelcontextprotocol.sdk.conformance.server.servlet.modern</module.name>
<maven.deploy.skip>true</maven.deploy.skip>
</properties>

<dependencies>
<dependency>
<groupId>io.modelcontextprotocol.sdk</groupId>
<artifactId>mcp</artifactId>
<version>2.1.0-SNAPSHOT</version>
</dependency>

<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>${slf4j-api.version}</version>
</dependency>

<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>${logback.version}</version>
</dependency>

<dependency>
<groupId>jakarta.servlet</groupId>
<artifactId>jakarta.servlet-api</artifactId>
<version>${jakarta.servlet.version}</version>
<scope>provided</scope>
</dependency>

<dependency>
<groupId>org.apache.tomcat.embed</groupId>
<artifactId>tomcat-embed-core</artifactId>
<version>${tomcat.version}</version>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>3.1.0</version>
<configuration>
<mainClass>io.modelcontextprotocol.conformance.server.modern.ModernConformanceServlet</mainClass>
<skip>false</skip>
</configuration>
</plugin>
</plugins>
</build>

</project>
Loading
Loading