Skip to content

Possible ressource leak / race condition in streamable_http_client #1805

Open

Description

@h-filzer

Initial Checks

Description

Observed Issue

When using the MCP SDK with the streamable-http transport, the cpu usage spikes and never goes back down after sending multiple requests and exiting the client context. I observed this bahavior especially in the google adk-python when the mcp-toolset tries to close the client.

When running the testcode to reproduce i get:

Client-Side Testcode shows Thread Leaks

Iteration 1: 1 thread  → 2 threads
Iteration 2: 2 threads → 7 threads
Iteration 3: 7 threads → 7 threads
Final: 7 threads (6 leaked asyncio_* threads)

Server-Side: Exceptions

Session crashed: unhandled errors in a TaskGroup (1 sub-exception)

ClosedResourceError at session.py:349
  → _write_stream.send() fails - stream already closed

BrokenResourceError at streamable_http.py:638
  → SSE writer has no receiver

Possible Cause

  1. Client sends multiple requests via write_stream.send()
  2. Client exits async with streamable_http_client context
  3. tg.cancel_scope.cancel() is called during cleanup
  4. Server is still processing requests (e.g., slow_echo with delay)
  5. Server tries to send responses via _write_stream.send()
  6. Stream is already closed → ClosedResourceError
  7. Background asyncio threads handling responses become orphaned
  8. Threads never terminate → memory/resource leak

Expected Behavior

  • No thread leaks after exiting streamable_http_client context
  • Graceful handling of client disconnection on server side

Impact

  • Memory leak from accumulated threads
  • Resource exhaustion in long-running applications

Affected Code Paths

  • mcp/client/streamable_http.py - streamable_http_client() cleanup
  • mcp/shared/session.py:237-238 - cancel_scope.cancel() and __aexit__
  • mcp/server/streamable_http.py:638 - SSE response handling

Example Code

"""
This script demonstrates a possible bug in the MCP SDK where cleanup of
streamable_http_client causes BrokenResourceError and leaks threads.

Bug?: When tg.cancel_scope.cancel() is called during cleanup, child tasks
spawned by post_writer (like _handle_json_response) are still trying to
use read_stream_writer. The stream gets closed before they finish,
causing BrokenResourceError and preventing proper httpx client cleanup.
"""

import asyncio
import threading
import httpx
from mcp.client.streamable_http import streamable_http_client
from mcp.shared.message import SessionMessage
from mcp.types import JSONRPCRequest, JSONRPCMessage


async def make_mcp_request(url: str, trigger_race: bool = False) -> None:
    """Make an MCP request and observe cleanup behavior."""

    print(f"[Before] Active threads: {threading.active_count()}")
    print(f"[Before] Thread names: {[t.name for t in threading.enumerate()]}")

    # Create custom httpx client with short timeouts
    http_client = httpx.AsyncClient(
        headers={"Authorization": "test"},
        timeout=httpx.Timeout(5.0, read=10.0),
    )

    try:
        async with streamable_http_client(
            url=url,
            http_client=http_client,
        ) as (read_stream, write_stream, get_session_id):
            print(f"[Connected] Session ID: {get_session_id()}")
            print(f"[Connected] Active threads: {threading.active_count()}")

            # Send an initialize request
            init_request = JSONRPCRequest(
                jsonrpc="2.0",
                id="1",
                method="initialize",
                params={
                    "protocolVersion": "2024-11-05",
                    "capabilities": {},
                    "clientInfo": {"name": "test-client", "version": "1.0.0"},
                },
            )
            await write_stream.send(SessionMessage(JSONRPCMessage(init_request)))
            print("[Sent] Initialize request")

            # Read the response
            async for message in read_stream:
                print(f"[Received] {type(message).__name__}")
                if isinstance(message, Exception):
                    print(f"[Error in stream] {message}")
                break

            if trigger_race:
                # Send multiple requests quickly to create race during cleanup
                for i in range(5):
                    tool_request = JSONRPCRequest(
                        jsonrpc="2.0",
                        id=f"tool-{i}",
                        method="tools/call",
                        params={
                            "name": "slow_echo",
                            "arguments": {"message": f"test-{i}"},
                        },
                    )
                    await write_stream.send(SessionMessage(JSONRPCMessage(tool_request)))
                print("[Sent] 5 tool requests - exiting immediately to trigger race")
                # Exit immediately without reading responses - this should trigger the race

    except Exception as e:
        import traceback
        print(f"[Error] {type(e).__name__}: {e}")
        traceback.print_exc()

    print(f"[After cleanup] Active threads: {threading.active_count()}")
    print(f"[After cleanup] Thread names: {[t.name for t in threading.enumerate()]}")

    # Wait and check if threads persist
    await asyncio.sleep(5)
    print(f"[After 5s] Active threads: {threading.active_count()}")
    print(f"[After 5s] Thread names: {[t.name for t in threading.enumerate()]}")


async def main():
    # Replace with your MCP server URL
    MCP_URL = "http://localhost:8000/mcp"

    print("=" * 60)
    print("MCP SDK Cleanup Race Condition Reproduction")
    print("=" * 60)

    # Run multiple iterations to accumulate leaked threads
    for i in range(3):
        print(f"\n--- Iteration {i + 1} ---")
        # Trigger race condition on iterations 2 and 3
        await make_mcp_request(MCP_URL, trigger_race=(i > 0))
        await asyncio.sleep(2)

    print("\n" + "=" * 60)
    print("Final State")
    print("=" * 60)
    print(f"Active threads: {threading.active_count()}")
    for t in threading.enumerate():
        print(f"  - {t.name} (daemon={t.daemon})")


if __name__ == "__main__":
    asyncio.run(main())


-----

"""Minimal MCP server for race condition testing."""

from mcp.server.fastmcp import FastMCP

mcp = FastMCP("Test Server")


@mcp.tool()
def echo(message: str) -> str:
    """Echo the message back."""
    return f"Echo: {message}"


@mcp.tool()
def slow_echo(message: str) -> str:
    """Echo with a delay to simulate work."""
    import time

    time.sleep(0.5)
    return f"Slow Echo: {message}"


if __name__ == "__main__":
    mcp.run(transport="streamable-http")

Python & MCP Python SDK

- Python: 3.13
- MCP SDK: 1.25.0
- Transport: streamable-http

Activity

  1. Brid9e commented on Dec 21, 2025

    @Brid9e

    I'm experiencing the exact same issue: when connections fail or I manually shut down the streamable_http service, the CPU usage spikes to 100% instantly.

  2. ivanbelenky commented on Dec 23, 2025

    @ivanbelenky

    Just did a couple of tests here,

    Background asyncio threads handling responses become orphaned

    Not really as far as Im concerned, the asyncio threads are due to shutdown when the event loop ends exits. What is happening here is the following

    • when you send a request, getaddrinfo gets called as part of the connect_tcp call in httpcore.
    • getaddrinfo as any other blocking syscall library function that hosts many blocking syscalls, gets executed on its own thread (lets say it gets called at $t_0$)
    • this threads have no idle timeout or anything
    • asyncio default Executor is executor = concurrent.futures.ThreadPoolExecutor(thread_name_prefix='asyncio') (you can find that line in the cpython stdlib here). This means that the max_workers setting is set by default to the maximum, usually 2 times the cores you got.
    • executing getaddrinfo takes approximately $\delta t$
    • (HERE is where stuff happens) if any other call to getaddrinfo happens during the range $[t_0, t_0+\delta t]$ the executor will see no worker available and will spin a new thread.

    If you modify your codebase slightly and introduce a minor delay after the send you will get no more threads, just the one that got started since the first call. Furthermore, if you want to constrain this further, you can always run something like

    loop.set_default_executor(concurrent.futures.ThreadPoolExecutor(max_workers=2)) # or whatever number rocks your boat

    Threads should be spending 0 cpu cycles @Brid9e, would you mind providing something for reproducibility?

  3. challenger71498 commented on Dec 26, 2025

    @challenger71498

    Hi @h-filzer ,

    I also had same issue you've encountered, and find out the BaseSession cleanup logic is being cancelled on __aexit__, causing respond streams never finish.

    I think your issue is could be fixed by this PR #1817, you may look into it if you are interested in.

  4. ivanbelenky commented on Dec 26, 2025

    @ivanbelenky

    @challenger71498 awesome fix, thx for it, I was able to replicate the issue above by @h-filzer rebasing locally to your fork, so I think your PR has no effect on this. I still believe this is happening because of what was explained here.

  5. challenger71498 commented on Dec 26, 2025

    @challenger71498

    @ivanbelenky Yup, seems like this issue is not related to the my PR.

    I will remove this issue from the PR. Thx 👍

  6. added
    bugSomething isn't working
    needs confirmationNeeds confirmation that the PR is actually required or needed.
    on Dec 31, 2025
  7. h-filzer commented on Jan 2, 2026

    @h-filzer
    Author

    Hey @challenger71498 , Thanks for digging into it. It seems that your PR is on the right track, as propagating cancel events might be bogus in other places too? I tried your PR, but unfortunately the thread count stays at the number of CPUs. I follow your work in google-adk python :-) Thx 👍

  8. major commented on Feb 25, 2026

    @major

    Can confirm this. Seeing the same behavior with mcp==1.26.0 / anyio==4.12.1 / Python 3.12.

    My setup uses streamable HTTP transport to a FastMCP server. After the first tool call completes and the client context exits, the process sits at 90%+ CPU permanently with zero incoming requests.

    py-spy consistently catches the main thread in _deliver_cancellation:

    Thread 1 (active+gil): "MainThread"
        _call_soon (asyncio/base_events.py:819)
        call_soon (asyncio/base_events.py:803)
        _deliver_cancellation (anyio/_backends/_asyncio.py:594)
        _run (asyncio/events.py:88)
        _run_once (asyncio/base_events.py:1999)
        run_forever (asyncio/base_events.py:645)
    

    2 out of 3 rapid samples land there — the third catches select() momentarily between bursts. Tasks stuck in _must_cancel state keep should_retry=True, so _deliver_cancellation reschedules itself via call_soon() every tick.

  9. akshan-main commented on Feb 25, 2026

    @akshan-main
    Contributor

    Opened a fix at #2147 - closes streams before cancelling the task group so blocked send() calls exit cooperatively, and guards all send paths against ClosedResourceError/BrokenResourceError during teardown.

    Also applied the same pattern to the SSE transport and BaseSession.aexit.

  10. billoneil commented on Mar 24, 2026

    @billoneil

    Any updates on the proposed fix? #2147

  11. maxisbey commented on Apr 8, 2026

    @maxisbey
    Contributor

    From looking into this, the CPU spin happens when the client is opened in one
    asyncio.Task and closed from a different one, typically via an
    AsyncExitStack that gets aclose()d from a cleanup/shutdown path. That's
    the pattern adk's MCPSessionManager uses, and it's what produces the
    _deliver_cancellation loop in @major's py-spy trace.

    The repro in the issue body (single-task async with) doesn't trigger it. The
    threads it shows are asyncio's getaddrinfo executor pool (capped, idle, not a
    leak). @ivanbelenky's analysis is right on that.

    minimal cross-task repro
    import asyncio
    from contextlib import AsyncExitStack
    from mcp import ClientSession
    from mcp.client.streamable_http import streamablehttp_client
    
    async def opener(stack, ready):
        rs, ws, _ = await stack.enter_async_context(streamablehttp_client("http://localhost:8001/mcp"))
        s = await stack.enter_async_context(ClientSession(rs, ws))
        await s.initialize(); await s.call_tool("echo", {"message": "hi"})
        ready.set(); await asyncio.sleep(100)
    
    async def closer(stack, ready):
        await ready.wait()
        try: await stack.aclose()
        except Exception as e: print(e)  # RuntimeError; spin continues afterward
    
    async def main():
        stack, ready = AsyncExitStack(), asyncio.Event()
        await asyncio.wait([asyncio.create_task(opener(stack, ready)),
                            asyncio.create_task(closer(stack, ready))])
        await asyncio.sleep(3)  # 100% CPU here
    
    asyncio.run(main())

    These context managers wrap anyio.create_task_group(), which requires
    enter/exit from the same task. There isn't a clean anyio-portable way to make
    them cross-task-safe from inside the SDK, so for now:

    • We'll document the same-task constraint on the client transports +
      ClientSession / Client.
    • Consumers that need cross-task close should run the client inside a single
      owned task and signal shutdown via anyio.Event. adk's SessionContext
      already does this for the inner layer; the outer MCPSessionManager wrapping
      it in another AsyncExitStack and closing that cross-task is what trips it.

    We might add a helper for this in the SDK (something that takes a caller
    task group and runs the client in it for you), but I'm not sure yet whether
    that's worth the extra surface area. Open to thoughts.

    Same root cause as #577 / #915 (and closed #79 / #521 / #252 / #788).

    AI Disclaimer

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingneeds confirmationNeeds confirmation that the PR is actually required or needed.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions