Repository navigation
perf(mcp): fewer token verifications on the request path - #93
Merged
Merged
Conversation
One upstream verification per token at a time, a 10 minute cache that an API 401 evicts, and a long-lived verify client. Tool calls are tagged with how their request's token was resolved so the effect is measurable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
Review (code-review high --fix)
QA against production (local verifier against
|
| Check | Result |
|---|---|
| 5 concurrent requests with the same invalid token | 1 upstream call, all 5 rejected (401) |
| New client per verification (old behaviour), 8 calls | median 196 ms |
| Long-lived client (new), 8 calls after the first | median 109 ms (measured from Spain; Railway sits next to sfo1, so its saving is smaller) |
| Unit tests | 1786 passed |
Metrics
| Metric | Before | Expected after |
|---|---|---|
python-httpx /v1/auth/verify calls (Axiom late, middleware rows) |
1.45M / 24h (≈55k/h) | well below the real-call count (≈28k/h); today it is 1.9x |
| Verify latency paid on a cache miss (Sentry, zernio-mcp) | p50 302 ms, p95 2,026 ms | the same per miss, but far fewer misses |
| Share of tool calls whose request waited on verify | not measured before | new mcp.auth tag: cache / upstream / joined / grace, with mcp.auth.wait_ms |
Post-deploy check: the same 24 h Axiom query of verify calls by hour, divided by the hour's late-python-sdk calls from the MCP IPs (control: the real-call count, which this PR does not change), plus the mcp.auth split in Sentry.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Every MCP request whose token is not cached waits on
/v1/auth/verifybefore the tool runs. That call takes 302 ms at p50 and 2 s at p95 (Sentry, zernio-mcp, 7 days), and Axiom shows about 1.9 verifications per real tool call (55k/h vs 28k/h). The cache only lived 60 s, and concurrent requests with the same token each went upstream.What
AsyncClientwith 60 s keepalive instead of a new client (DNS + TCP + TLS) per verification.mcp.auth= cache / upstream / joined / grace, withmcp.auth.wait_ms, in Sentry and the logs, so the effect can be measured after deploy.Not changed: the API connection pool. A local simulation of the anyio thread model at 8 calls/s showed 3 to 5% new connections, mostly warm-up, and a new connection to the edge costs about 20 to 70 ms. A longer keepalive or shared pools would save about 1 ms per call on average.
Follow-ups
mcp.authshares andwait_msin Sentry, and the hourly count of python-httpx/v1/auth/verifycalls in Axiom against the 55k/h baseline.🤖 Generated with Claude Code