Skip to content

fix(upstream): re-initialize a terminated Streamable HTTP session and retry once (Spec 113-e) - #1501

Merged
Dumbris merged 3 commits into
mainfrom
113-e-session-terminated-reinit
Oct 6, 2026
Merged

Dumbris merged 3 commits into
mainfrom
113-e-session-terminated-reinit

Conversation

@Dumbris

@Dumbris Dumbris commented Oct 5, 2026 •

Copy link
Copy Markdown
Member

Problem

When a remote Streamable HTTP upstream forgets a session (restart, expiry) it answers HTTP 404 to the session id mcpproxy holds. mcp-go v1.0.0 maps every such 404 to transport.ErrSessionTerminated and clears the transport's session id. Today that error text ("terminated") matches isConnectionError, so the server flips to Error and goes through a full reconnect, and the failing call is lost. Gap G8 of the Anthropic MCP-proxy talk audit (Spec 113, US5).

What changed

  • core/session_reinit.go: SessionSnapshot() (transport session id + whether the negotiated protocol is stateless) and ReinitializeSession(ctx), a fresh initialize on the same mcp-go client/transport. connectionEpoch is never touched.
  • managed/session_reinit.go: single-flight re-init keyed by the stale session id. Concurrent callers share one flight; a caller that finds a newer session proceeds; a request about to go out with an empty session id while one is known joins the flight instead of sending without a session. The flight does initialize plus a synchronous tools/list, refreshes the identity baseline, and schedules the normal discovery callback if the toolset changed.
  • Wired into: the tools/call path, the ListTools leader (so the cached-tool-count path is covered), prompts list/get, and the health ping. List/get/ping are retried once. tools/call is retried once only for read-only tools (readOnlyHint and not destructive) whose identity hash is unchanged (FR-081); otherwise the call returns ErrSessionReestablished (session re-established, call not repeated) and the server is not marked Error. A pinned call whose tool changed gets ErrConnectionGenerationChanged.
  • Only the final error reaches isConnectionError (retry also 404, or re-init fails: existing path).
  • Info log per re-init (session ids as 8-char prefixes) and a per-client counter exposed as session_reinit_count in GetConnectionStatus.
  • docs/setup.md: troubleshooting note.

mcp-go v1.0.0 verification (FR-088)

Pinned by TestMCPGo_SessionTerminated_ReinitializeOnSameTransport: a 404 on a non-initialize POST gives ErrSessionTerminated and clears the session id; a request with no session id that is 404'd gives it too; a second Initialize on the same client/transport stores a new session id and the client works again. No divergence from the plan.

Tests run

  • New: core (mcp-go behaviour pin, snapshot/reinit) and managed (TestSessionReinit_*: single read call = 1 re-init and state stays Ready; 10 concurrent calls = exactly 1 re-init; write tool not repeated and next call works; changed hash not retried and pinned call refused; ListTools retried; ping-first then call sends no session-less request; retry also 404 fails; pinned call epoch unchanged). -race -count=15 stable.
  • go test -race ./internal/upstream/... green; go build ./cmd/mcpproxy and go build -tags server -o /dev/null ./cmd/mcpproxy green.
  • golangci-lint (CI config, bare and --build-tags server) on ./internal/upstream/...: only 2 pre-existing govet reflect.Ptr findings in files this PR does not touch (core/connection_stdio.go:375, core/client_secret_test.go:452).
  • ./scripts/test-api-e2e.sh on an isolated port, run alone: 65/66; the one failure ("Audit log: server-edition binary present") is environmental and fails identically on origin/main. Earlier runs overlapped with sibling agents' e2e runs (shared launcher port 39933) and showed launcher failures that vanished when run alone.

Assumptions

  • Retrying tools/call is limited to read-only tools per FR-081 (a non-conforming server or intermediary could 404 after executing). Spec 018 operation type is not visible to managed.Client, so only the readOnlyHint annotation is used; tools without that hint get the "re-established, not repeated" error. A follow-up could pass the call_tool_read variant down.
  • FR-083a listener restore: mcpproxy does not enable mcp-go's continuous GET listener on the upstream hop in production (WithContinuousListening is only used in an e2e test), so there is no listener to restore; not implemented.
  • Prometheus counter: no observability seam was touched; counter is in the connection status/diagnostics map only.
  • error_class=session_terminated labelling belongs to 113-c; this PR exposes the sentinel managed.ErrSessionReestablished for it to map.
  • Not covered by a dedicated test: modern-protocol and non-404 no-retry branches (guarded by SessionSnapshot modern flag and errors.Is check).

Spec: #1495
Review: opencode github-copilot/gpt-6.1-sol, 2 rounds, clean; follow-ups: (1) Spec 018 call_tool_read variant not passed down so unannotated read tools are not retried; (2) residual narrow window where a request that snapshotted the old session id is sent on the new id before the flight's tools/list verifies (closing it would serialize sends behind long tool calls); (3) tool-change epoch bump is not a mechanism in this codebase (FR-083 relies on the differential update path); (4) test hardening: RetryAlso404 reaching the retry, listener fixture; (5) a caller ctx deadline while waiting on a flight is classed as a connection error like any other deadline.

… retry once (Spec 113-e)

mcp-go maps HTTP 404 for an unknown session id to ErrSessionTerminated and
clears the session id. Previously that error flipped the server to Error and
forced a full reconnect. The managed client now re-initializes the session in
place (single-flight per stale session id), retries list/prompt/ping requests
once, and retries tools/call once only for read-only tools whose identity
hash is unchanged. connectionEpoch is not touched.
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Deploying mcpproxy-docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: 8889f63
Status: ✅  Deploy successful!
Preview URL: https://382eddda.mcpproxy-docs.pages.dev
Branch Preview URL: https://113-e-session-terminated-rei.mcpproxy-docs.pages.dev

View logs

…r re-init, keep recovery error (Spec 113-e review r1)
@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

📦 Build Artifacts

Workflow Run: View Run
Branch: 113-e-session-terminated-reinit

Available Artifacts

  • archive-darwin-amd64 (31 MB)
  • archive-darwin-arm64 (28 MB)
  • archive-linux-amd64 (19 MB)
  • archive-linux-arm64 (17 MB)
  • archive-windows-amd64 (31 MB)
  • archive-windows-arm64 (27 MB)
  • frontend-dist-pr (0 MB)
  • installer-dmg-darwin-amd64 (27 MB)
  • installer-dmg-darwin-arm64 (24 MB)
  • smart-mcp-proxymcpproxy-goL8G0X1.dockerbuild (0 MB)

How to Download

Option 1: GitHub Web UI (easiest)

  1. Go to the workflow run page linked above
  2. Scroll to the bottom "Artifacts" section
  3. Click on the artifact you want to download

Option 2: GitHub CLI

gh run download 37440084534 --repo smart-mcp-proxy/mcpproxy-go

Note: Artifacts expire in 14 days.

@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 78.69565% with 49 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
internal/upstream/managed/session_reinit.go 78.40% 25 Missing and 13 partials ⚠️
internal/upstream/core/session_reinit.go 71.87% 5 Missing and 4 partials ⚠️
internal/upstream/managed/client.go 83.33% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

…ted-reinit

# Conflicts:
#	internal/upstream/managed/client.go
@Dumbris
Dumbris enabled auto-merge (squash) October 6, 2026 17:24
@Dumbris
Dumbris merged commit 479188e into main Oct 6, 2026
58 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants