Skip to content

perf(mcp): fewer token verifications on the request path - #93

Merged
Gintm merged 2 commits into
developfrom
perf/mcp-auth-verify-cache
Nov 12, 2026
Merged

Gintm merged 2 commits into
developfrom
perf/mcp-auth-verify-cache

Conversation

@Gintm

@Gintm Gintm commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Why

Every MCP request whose token is not cached waits on /v1/auth/verify before the tool runs. That call takes 302 ms at p50 and 2 s at p95 (Sentry, zernio-mcp, 7 days), and Axiom shows about 1.9 verifications per real tool call (55k/h vs 28k/h). The cache only lived 60 s, and concurrent requests with the same token each went upstream.

What

  • Concurrent requests with the same token share one upstream verification (a cancelled request does not cancel it).
  • The positive cache lives 10 minutes instead of 60 s. Revocation does not wait for it: every tool call still sends the key to the API, and an API 401 evicts the token, so the next MCP request re-verifies and the client gets the 401 that makes it refresh.
  • The verifier keeps one AsyncClient with 60 s keepalive instead of a new client (DNS + TCP + TLS) per verification.
  • Each tool call is tagged mcp.auth = cache / upstream / joined / grace, with mcp.auth.wait_ms, in Sentry and the logs, so the effect can be measured after deploy.

Not changed: the API connection pool. A local simulation of the anyio thread model at 8 calls/s showed 3 to 5% new connections, mostly warm-up, and a new connection to the edge costs about 20 to 70 ms. A longer keepalive or shared pools would save about 1 ms per call on average.

Follow-ups

  • After deploy: compare mcp.auth shares and wait_ms in Sentry, and the hourly count of python-httpx /v1/auth/verify calls in Axiom against the 55k/h baseline.

🤖 Generated with Claude Code

Gintm and others added 2 commits November 12, 2026 15:24
One upstream verification per token at a time, a 10 minute cache that an
API 401 evicts, and a long-lived verify client. Tool calls are tagged with
how their request's token was resolved so the effect is measurable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Gintm

Gintm commented Nov 12, 2026

Copy link
Copy Markdown
Contributor Author

Review (code-review high --fix)

Finding Result
The ChatGPT surface never evicted a token on an API 401, so with a 10 min TTL ChatGPT would not get the 401 that makes it refresh Fixed, with a test
Duplicated guarded get_access_token lookup in telemetry Fixed (_current_token)
support_ask uses the SDK's async client, which bypasses the evicting transport Skipped: rare, and the next sync tool call evicts the token
Non-key 401s also evict the cached token No change: costs one extra verification, never a wrong grant
The keepalive test reads httpx private attributes Skipped: httpx exposes no public accessor

QA against production (local verifier against zernio.com/api/v1/auth/verify, no credentials)

Check Result
5 concurrent requests with the same invalid token 1 upstream call, all 5 rejected (401)
New client per verification (old behaviour), 8 calls median 196 ms
Long-lived client (new), 8 calls after the first median 109 ms (measured from Spain; Railway sits next to sfo1, so its saving is smaller)
Unit tests 1786 passed

Metrics

Metric Before Expected after
python-httpx /v1/auth/verify calls (Axiom late, middleware rows) 1.45M / 24h (≈55k/h) well below the real-call count (≈28k/h); today it is 1.9x
Verify latency paid on a cache miss (Sentry, zernio-mcp) p50 302 ms, p95 2,026 ms the same per miss, but far fewer misses
Share of tool calls whose request waited on verify not measured before new mcp.auth tag: cache / upstream / joined / grace, with mcp.auth.wait_ms

Post-deploy check: the same 24 h Axiom query of verify calls by hour, divided by the hour's late-python-sdk calls from the MCP IPs (control: the real-call count, which this PR does not change), plus the mcp.auth split in Sentry.

@Gintm
Gintm merged commit 7261492 into develop Nov 12, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant