MCP proxy does not relay streaming tool-call responses — buffers and collapses to a single result

[hunner]

Summary

Atryum's MCP proxy cannot stream a tools/call response from an upstream MCP server through to the connected agent. The upstream response is fully buffered, reduced to the single terminal JSON-RPC result, and returned to the agent as one application/json message. Any intermediate events the upstream emits for that call — notifications/progress, logging, partial/incremental content — are silently discarded, and nothing reaches the agent as it is produced.

Current behavior

The tools/call path is unary at every layer, so end-to-end streaming is structurally impossible today:

• Upstream read (HTTP / Streamable HTTP): the client advertises Accept: application/json, text/event-stream, but reads the entire response body to EOF into a buffer before parsing. If the upstream answered with text/event-stream, the SSE body is scanned only to extract the event whose JSON-RPC id matches the request — the final result is correct, but no intermediate event survives and nothing is delivered incrementally.
• Upstream read (stdio): the reader returns the first message carrying any id/result/error field, with no request-id correlation — a "first response wins" read.
• Invocation pipeline: the result is carried as a single Body []byte, persisted to a single response column, and surfaced as one result value. The pipeline cannot carry more than one result per call.
• Agent-facing write: the response is emitted as one application/json JSON-RPC message with no flusher; the agent-facing GET SSE endpoint is keepalive-only.

Even if the transport read the stream correctly, there is no channel or relay on the tools/call path to forward intermediate events onward.

End goal (per MCP spec)

Per the Streamable HTTP transport (MCP spec 2025-11-25, https://modelcontextprotocol.io/specification/2025-11-25/basic/transports): a request-shaped POST may receive a text/event-stream response carrying intermediate JSON-RPC messages before the terminal response. Atryum should behave as a policy-aware live relay rather than a buffer:

• When an upstream tools/call POST returns text/event-stream, start the downstream SSE response immediately and flush headers early.
• Relay upstream SSE events (progress, logging, partial content) to the agent as they arrive, preserving event framing.
• Detect the terminal JSON-RPC response for correlation/audit and close the downstream stream after it.
• Keep non-streaming application/json upstream responses behaving exactly as today.
• Preserve approval/governance: approval-gated calls still pause before upstream execution.
• Don't advertise resumability or server-initiated features (sampling, elicitation, roots) that Atryum can't actually broker end to end.

Acceptance criteria

• A fake upstream that delays and emits multiple SSE events before its terminal response is relayed live: the agent observes an intermediate event before the upstream's terminal response exists.
• Progress/logging notifications reach the agent instead of being dropped.
• Non-streaming JSON tool responses are unchanged.
• Audit records capture streamed events with enough metadata to correlate downstream request, upstream request, and invocation.