Skip to content

Evict the longest-idle session instead of refusing at the connection limit (v1.9.7) - #29

Merged
christophdb merged 1 commit into
mainfrom
fix/evict-idle-sessions
Nov 18, 2026
Merged

christophdb merged 1 commit into
mainfrom
fix/evict-idle-sessions

Conversation

@christophdb

Copy link
Copy Markdown
Member

Problem

ChatGPT opens a new MCP session for every tool call and never sends DELETE (initialize → notifications/initialized → a GET it drops after ~1 ms → one tools/call). Every such session counted as used, so it kept its connection slot for the full 10-minute idle timeout. An intensive ChatGPT user hit 429 Too many concurrent connections after 20 calls in 10 minutes.

This is the third time the pattern showed up. The earlier fixes treated the symptom: raising the limit from 5 to 20 (91ef335), then a 30-second timeout for unused sessions (f62351b). That timeout never fired for ChatGPT, because the initialized notification and the GET already marked a session as used. Production logs for the incident: 72 sessions, 38 tool calls on 38 different sessions, 65 of 65 idle closes with used=true.

Change

  • Eviction instead of refusal. When a token's 20 slots are full, the session of the same token that has been idle longest and has no POST in flight is closed to make room. A client returning to it gets 404 and must re-initialize (MCP spec, Session Management). Only when all 20 sessions are mid-request does the client get 429. Other tenants' sessions are never touched.
  • Simpler lifecycle. The used flag and unusedSessionTimeoutMs are removed. One idle timeout (10 min) remains, and the sweeper skips sessions with a request in flight.
  • Visibility:
    • Token fingerprint in Session initialized, Session evicted and the per-token 429.
    • mcp_rate_limit_exceeded_total{type} now also counts connections, pre_auth and oauth.
    • New mcp_sessions_evicted_total.
    • Log line for initialize requests that end without a session.
    • Tool call logs carry the session fingerprint instead of the raw session ID.

The per-IP and per-token request limits are unchanged.

Tests

  • tests/sessionEviction.spec.ts replays the incident: 29 sessions over two minutes, each with the full ChatGPT sequence. It was red before this change. It also covers 404 on an evicted session, the choice of the longest-idle session, the in-flight guard (429 only when all sessions are busy), tenant isolation, and the logs and metrics.
  • Mutation check: removing the in-flight guard or the same-token restriction turns the corresponding tests red.
  • tests/connectionSlots.spec.ts: the unused-timeout tests are removed. The DELETE test now asserts that no eviction was needed.
  • npm test (566), npm run lint and npm run typecheck pass.

After merge

Build and scan the image locally, then tag release-v1.9.7. The changelog should say that mcp_rate_limit_exceeded_total has new type values and that mcp_sessions_evicted_total is new, in case dashboards or alerts filter on these.

🤖 Generated with Claude Code

…limit (v1.9.7)

ChatGPT opens a new MCP session for every tool call and never sends
DELETE: initialize, notifications/initialized, a GET it drops after a
millisecond, one tools/call. Each of those sessions counted as used, so
it held its slot for the full 10-minute idle timeout, and an intensive
ChatGPT user ran into "429 Too many concurrent connections" after 20
calls. The 30-second timeout for unused sessions never fired, because
the initialized notification and the GET already marked a session used.

The pool now limits how many sessions of a token may work at once, not
how many it has walked away from. When it is full, the session of the
same token that has been idle longest and has no POST in flight is
closed to make room. A client returning to it gets 404 and initializes
again, as the MCP spec requires. Only when all 20 are mid-request does
the client get 429. The used flag and the 30-second timeout are gone.

Visibility, which is why this went unnoticed twice:
- "Session initialized", "Session evicted" and the per-token 429 log
  the token fingerprint, so a burst can be attributed to a tenant.
- Rejections by the connection limit, the pre-auth limit and the OAuth
  limit are counted in mcp_rate_limit_exceeded_total; evictions in the
  new mcp_sessions_evicted_total.
- An initialize request that ends without a session gets a log line.
- Tool call log lines carry the session fingerprint, not the raw ID.

tests/sessionEviction.spec.ts replays the incident: 29 sessions over
two minutes, each used for one tool call. It failed before this change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@christophdb
christophdb merged commit 8a2eceb into main Nov 18, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant