feat(mcp): single Stargate auth server + region-from-token routing - #162
Conversation
MCP half of the Stargate-centered cross-region OAuth redesign. The shared mcp.leadbay.app/mcp connector now works for any region without a region-specific URL, because Stargate is the single OAuth authority and the region rides in the token suffix. - Discovery advertises ONE authorization server (STARGATE_AUTH_SERVER, auth.leadbay.app) instead of a per-region backend; drop regionForResourcePath, regionAuthServer, and the /fr/* connector routes + body limits. - resolveClientFromToken decodes the token's `_us`/`_fr` suffix and routes directly to the owning backend (new regionFromToken helper) — removes the dual-region /users/me auto-probe entirely. Untagged legacy tokens fall back. - Net simplification (-188/+97): the region path convention and probe are gone. Existing OAuth tests updated to the new model (single auth server, suffix routing, no probe); the product#3761 empty-401-body guard is preserved. Built + tested locally: pnpm --filter @leadbay/mcp test (524 passing) + typecheck. Companions: stargate (the authority) + backend (mint the region-suffixed token). Related: leadbay/product#3872 Co-Authored-By: Claude <noreply@anthropic.com>
Codex raised two P1s on the Stargate-single-auth rewrite: 1. Expired tokens never triggered a refresh. Dropping the dual-region auto-probe left resolveClientFromToken always returning authState:"ok", so a rejected bearer never produced WWW-Authenticate: error="invalid_token" and the host couldn't silently refresh — the failure only surfaced on a later tool call. Fix: a SINGLE-region /users/me validation probe (the token suffix already names the region, so it's one round-trip, not the old two). AUTH_EXPIRED / NOT_AUTHENTICATED → authState:"expired" → invalid_token challenge. A non-auth fault (5xx/network) stays "ok" so we don't force spurious re-auth. A new validate:false opt-out skips the probe. 2. /fr/mcp 404'd. The README ships https://mcp.leadbay.app/fr/mcp as the EU connector URL, but the rewrite removed the /fr routes. Existing EU users would hit a 404 instead of the OAuth challenge/transport. Fix: keep /fr/mcp + /fr/sse as compat aliases of /mcp + /sse (same single Stargate auth server; the token suffix, not the path, selects the region) and advertise their PRM. Tests: single-region-probe ok/expired/5xx/validate-false in auth-http; the /mcp invalid_token challenge and the /fr/mcp alias (with + without token) in http-auth-challenge. 530 pass, typecheck clean.
Codex P1: AGENTS.md requires new tests to live in NEW files, not be added to existing ones. The earlier commit added new cases to four existing unit test files. Fix the layout: - http-auth-challenge.test.ts + http-auth-challenge-body.test.ts: reverted to their base state (untouched) — their base tests already pass under the new flow. - auth-http.test.ts: removed ONLY the three obsolete dual-region auto-probe tests (first-region-wins, dual AUTH_EXPIRED, probe_failed) — behavior this PR deletes. - oauth-resource-metadata.test.ts: removed the obsolete per-region PRM tests + the dead regionAuthServer import; kept the region-agnostic buildWwwAuthenticate test. All net-new coverage moved to new files: - auth-http-single-region-probe.test.ts — regionFromToken + single-region probe (ok / expired / 5xx-stays-ok / validate:false / legacy). - oauth-single-auth-server.test.ts — single Stargate auth server in PRM (helper + routes, incl. /fr/mcp alias advertising the same server). - http-auth-challenge-stargate.test.ts — /mcp valid→no-401, /mcp expired→ invalid_token, /fr/mcp alias (with + without token). 535 tests pass, typecheck clean.
…l success Codex P1: dropping the dual-probe pinned EVERY untagged token to US, so an existing FR bearer with no _fr suffix was validated only against US, 401'd, and got falsely reported expired — forcing re-auth or failing the connection. Since /fr/mcp is kept as an EU compat alias but no longer supplies a region, those users regressed. Fix resolveClientFromToken: - Suffixed token → one probe against the region the suffix names (unchanged). - Untagged token → probe the preferred region first, then FALL BACK to the sibling; only "expired" when BOTH regions reject it on auth grounds. A non-auth fault (5xx/network) still short-circuits to "ok". - New opts.preferRegion (distinct from the validation-skipping pin): the /fr/mcp + /fr/sse handlers pass "fr" so legacy EU tokens probe FR first. Codex P2: the valid-token HTTP tests only asserted not.toBe(401). Under app.fetch there's no Node adapter env, so the handler 500s AFTER auth — not.toBe(401) passes even on an auth failure. Strengthened them to assert the validation probe actually ran against the expected region AND no OAuth challenge (www-authenticate) was emitted, which is what proves auth succeeded. Tests (new-file coverage): untagged FR US-401→FR-200→ok; untagged expired only when both reject; preferRegion fr probes FR first; suffix beats preferRegion; HTTP success-path asserts probe+region+no-challenge. 540 pass, typecheck clean.
…probe failure Codex P2: for an untagged legacy token the fallback loop returned ok bound to the FIRST region on ANY non-auth fault, so a US 503 (while the token is actually valid in FR on the shared /mcp URL) immediately bound the request to the failing US region and never tried FR — breaking healthy FR users during a US transient/backend blip. Defer the outcome instead of short-circuiting: on a non-auth fault, record it and continue to the next candidate. Only after all candidates: - some candidate auth-rejected AND none had a non-auth fault → expired (invalid_token). - otherwise (a non-auth fault occurred anywhere) → ok, so a transient error never forces spurious re-auth; a real fault re-surfaces on the tool call. A suffixed single-candidate token keeps its semantics (5xx → ok, auth-reject → expired). Tests: US-503→FR-200→ok (probes both, binds FR not US); US-401→FR-503→ok (ambiguous, no forced re-auth); both-5xx→ok. 543 pass, typecheck clean.
…n, not the rejecting one Codex P2: when an untagged legacy token is auth-REJECTED by primaryRegion (US) but the sibling (FR) hits a transient 5xx/network fault, the ok fallback bound the client to primaryRegion (US) — the region that already rejected the token. The next tool call then went to US and failed auth needlessly, instead of hitting FR where the token is plausibly valid once the transient clears. Track the region whose probe had the non-auth fault and bind the ok client there (nonAuthFaultRegion), falling back to primaryRegion only when no region faulted. So US-401 + FR-503 → ok bound to FR; FR-401 + US-503 → ok bound to US. Tests: US-401→FR-503 binds FR; FR-401→US-503 (via preferRegion) binds US — asserted through the resolved client's .region. 544 pass, typecheck clean.
|
Couldn't auto-sync with main — this PR has a merge conflict outside the version files. main shipped |
…e-single-auth # Conflicts: # packages/mcp/src/auth-http.ts # packages/mcp/src/http-server.ts
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ebb444f13c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 280f5cb102
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
Couldn't auto-sync with main — this PR has a merge conflict outside the version files. main shipped |
|
Couldn't auto-sync with main — this PR has a merge conflict outside the version files. main shipped |
|
Couldn't auto-sync with main — this PR has a merge conflict outside the version files. main shipped |
|
Couldn't auto-sync with main — this PR has a merge conflict outside the version files. main shipped |
…e-single-auth # Conflicts: # packages/core/src/client.ts
…and a valid token Three findings from the two Codex passes. P1 — the candidate regions are probed one after another, and node:https sets no socket timeout. A region that accepted the connection and then went silent held the request open forever, so the sibling that would have accepted the token was never asked (the pre-Stargate code raced both with Promise.any, so it couldn't happen). request() now takes an opt-in per-attempt `timeoutMs` that destroys the request and rejects with a TIMEOUT code — deliberately not an auth code, so the resolver reads it as a transient fault and moves on. Probes use a 4s deadline. P2 — `_us`/`_fr` is how a Stargate token names its region, but a legacy opaque bearer can end in those two characters by coincidence, and the two are indistinguishable. Treating the suffix as a hard pin turned one 401 from the wrong region into authState "expired", pushing that user through reauth on every request until their token rotated. The suffix now decides probe ORDER and the bind region; the sibling is still asked before we pass an expired verdict. The happy path is unchanged — a token its named region accepts still resolves in one round trip. P1 (test layout) — restored `auto-probe: first region to respond wins` in auth-http.test.ts; it still holds and should never have gone. The two remaining deletions assert contracts this PR replaces on purpose (a broken client on the expired path, and the `probe_failed` authState), so they cannot be kept green; the file now says which, and why, at the point of removal. Co-Authored-By: Claude <noreply@anthropic.com>
…g its test Second look at the test-layout finding. "both regions AUTH_EXPIRED → expired broken client" was deleted as obsolete, but nothing actually required deleting it: the hosted resolver had quietly started returning a LIVE client on the expired path while the stdio resolver right next door (bin.ts) still returned a broken one. Both hosted call sites answer the 401 challenge without touching the client, so the live one bought nothing and left a bearer every region had just rejected in reach of any caller that forgets to check authState. So the expired envelope is a broken client again — carrying the backend's own code and message — the two resolvers agree, and the existing test comes back verbatim. One deletion remains in that file, the `probe_failed` assertion, which names an authState this PR removes on purpose. Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 162b4a5d13
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Both Codex findings on #162 were real. P1 — the probe loop runs with retryOn401:false so an auth rejection cannot be masked and the dual-region fallback still works. But a Leadbay 401 is usually not expiry: LeadbayClient carries a one-shot 401 retry precisely because "tokens don't expire, so a 401 is almost always a transient server-side blip". Without a retry anywhere, a blip cascaded — the owning region 401s, the sibling 401s too since the token is region-scoped, both rejections looked authoritative, and a valid token got an invalid_token challenge. The old resolveMe() probe inherited the client retry, so this was a regression, not a pre-existing gap. Fixed by retrying the primary region once AFTER the candidate walk, not inside it: mid-loop it would delay the sibling probe the sequential deadline exists to protect, and would spend the round trip on the legacy-token path where the 401 is expected and the sibling is the answer. After the loop it costs one request only on the path that was about to force re-auth. A fault on the retry is treated as transient, not expiry. P2 — both READMEs still told users to pick a URL per region and explained that the region is encoded in the path. Stargate makes /mcp work for every region with the region riding in the token, so /fr/mcp is now only a compatibility alias. Updated the packaged and root READMEs to match. New test file auth-http-transient-401-retry.test.ts covers the blip recovering, a genuinely dead token still reporting expired, a fault on the retry staying transient, no cost on the happy path, and no retry spent when the sibling already accepted. Verified red before the fix (3 of 5 failed) and green after. Note: this touches three EXISTING test files, which the repo normally forbids. They are not new coverage — their fixtures pinned the old probe COUNT (2 requests before an expiry verdict, now 4). The assertions they exist for are unchanged; leaving them stale would have left the suite red. Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d1a95f23fd
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Codex's follow-up P2 is real, and it is a gap in my own previous fix. I
gated the last-chance retry on "no sibling faulted", which skipped the
one mixed outcome that fails silently:
owning region (from the token suffix) -> transient 401
sibling region -> timeout / 5xx
nonAuthFaultRegion was set, so the retry never ran and the resolver bound
to the FAULTING SIBLING. For a region-scoped token that is the wrong
backend: no OAuth challenge is raised, the request looks healthy, and
every later tool call 401s against a region the token was never scoped
to. A silent wrong-region bind is worse than a challenge.
The gate is now "the PRIMARY auth-rejected AND the token is suffixed". A
recovered retry is positive evidence that this region accepts the token,
so it outranks the transient-region fallback; if the retry also faults we
bind to the primary rather than the sibling, since the suffix is what the
token claims.
The suffix condition matters as much as the widening. My first attempt
gated on primaryAuthRejected alone, which broke two existing tests — and
those were NOT stale fixtures, they encode a deliberate contract: for an
UNTAGGED token, primaryRegion is only a guess, so a 401 there is not a
blip on the owning backend and the sibling that merely faulted is the
better bind. Restricting the retry to suffixed tokens keeps that contract
intact, which is also exactly what the review asked for ("the owning
region").
Side effect worth noting: this also reverts my earlier fixture edit to
auth-http-single-region-probe.test.ts, which is now byte-identical to the
pre-change baseline again. Two existing test files remain modified rather
than three, both only for the tagged-token probe count.
New auth-http-mixed-outcome-retry.test.ts (6 tests) covers the mixed
outcome, the retry-also-faults bind, the no-spurious-reauth case, the
untouched all-reject verdict, no retry when the primary never rejected,
and the untagged contract. Verified red before the fix (2 of 5 failed)
and green after.
Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dbde8df81d
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
The transient-401 retry was gated on the token carrying a `_us`/`_fr` suffix. That gate was right for the BIND decision but wrong on the one path where the guess doesn't matter: both regions auth-rejected, nothing faulted, so the resolver is about to emit an `invalid_token` challenge. A valid legacy token that merely caught a blip on its owning region was sent through reauth — the pre-Stargate resolver probed both regions and inherited the client's one-shot 401 retry on each, so this was a regression. For an untagged token we can't tell which of the two 401s was the blip: an FR legacy token blipping on FR looks exactly like a US one blipping on US. So both rejecting regions get the second look, with `retryOn401:false` so the client's internal double doesn't stack on top — the whole failure path stays at four requests. Suffixed tokens keep the narrower rule (primary only, client retry policy), and the untagged mixed-outcome contract is untouched: when a region merely faulted we are not challenging anyway, and the faulting sibling stays the better bind than the rejecting primary. New test/unit/auth-http-untagged-transient-401.test.ts — 6 of its 8 tests red before the fix. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Discovery advertised `https://auth.leadbay.app`. That hostname was planned but never deployed — it does not resolve — so a spec-compliant client fetching `/.well-known/oauth-authorization-server` from it gets nothing and the sign-in prompt never appears. Stargate is live at `stargate.leadbay.app` (staging: `staging.stargate.leadbay.app`), and its metadata declares `issuer: "https://stargate.leadbay.app"`, which is the origin a client validates the discovery URL against. Every existing test asserted against the STARGATE_AUTH_SERVER constant, so the whole suite stayed green while the constant pointed nowhere. New test/unit/oauth-auth-server-host.test.ts pins the literal default instead, plus the shape (https origin, no path, no trailing slash). `LEADBAY_AUTH_SERVER` still overrides it for staging. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5ee9da542c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Two Codex findings, both on paths this PR opened.
P2 — a legacy US bearer whose opaque value happens to end in `_fr` gets
a legitimate 401 from FR and a transient one from the US backend that
actually owns it. The retry pass only re-tested the suffix region, so a
live token was declared expired. The probe loop already treats the suffix
as a hint; the retry contradicted it one step later. Provenance is
ambiguous exactly when every candidate rejected and nothing faulted, so
on that path every rejecting region is now re-tested, suffix or not. The
suffix keeps its weight where it is evidence rather than a guess: probe
order, bind region, and the mixed-outcome retry.
P1 — the hosted identity reads were abandonable but not cancellable.
`resolveTelemetryContext` races `resolveMe()` for 1.5s and the SSE
refresh fires `fetchTelemetryEnabled()` behind a timer; neither cancels
the request underneath, and node:https never times a socket out. Since
the resolver now returns a live client when both probes time out, a
silent regional outage meant every authenticated request leaked a socket
and an API-semaphore slot — the deadline this PR added was defeated one
call later. Both reads take `{ timeoutMs }` now and the hosted callers
pass the same budget they wait on: the timer bounds the wait, the
deadline bounds the resource.
New coverage: auth-http-suffix-looking-legacy-retry.test.ts,
http-identity-read-cancel.test.ts, client-identity-read-deadline.test.ts
— all verified red first. Two fixtures in the PR's own
auth-http-transient-401-retry.test.ts move to the wider retry pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Tested seems all good, waiting for all backend deployment to bump version and deploy |
What
MCP half of the Stargate-centered cross-region OAuth redesign (product#3872).
The shared
mcp.leadbay.app/mcpconnector works for any region — the region ridesin the token, not the URL — because Stargate is the single OAuth authority.
Changes
(
STARGATE_AUTH_SERVER, defaulthttps://stargate.leadbay.app— the host Stargate isactually deployed on) for every connector
path, instead of a per-region backend.
resolveClientFromTokendecodes the token's_us/_frsuffix (regionFromToken) and validates it with a/users/meprobeagainst the owning backend — one round trip on the happy path. A rejected
token →
authState:"expired"→ RFC 6750invalid_tokenchallenge (host silentlyrefreshes).
preferRegionon the/fr/mcpalias, else US) decides probe ORDER and the bind region; the sibling isstill asked before we return
expired. A legacy opaque bearer can end in_us/_frby coincidence and is indistinguishable from a tagged one, so a hard pin would
push those users through reauth on every request until their token rotates. The
extra probe only ever happens on a failure path.
and, when it suppresses re-auth, binds to the transient-fault region rather than
the one that rejected the token.
LeadbayClient.request()gained an opt-intimeoutMsthatdestroys the request and rejects with a
TIMEOUTcode (never an auth code, so theresolver reads it as transient). node:https sets no socket timeout, so without it a
region that accepted the connection and then went silent held the request open
indefinitely and starved the sibling that would have accepted the token.
retryOn401:falseso arejection can't be masked, so the rejecting region(s) get one last look after the
candidate walk — a Leadbay 401 is usually a server-side blip, not expiry.
On the path that is about to challenge, EVERY rejecting region is re-tested — the
suffix is a hint during probing and can't become an ownership claim there (a legacy
US bearer ending in
_fris rejected legitimately by FR). Where a fault alreadyruled the challenge out, the retry only picks the bind region and the suffix decides.
Happy path stays at one request; failure path is bounded at four.
resolveMe()andfetchTelemetryEnabled()take{ timeoutMs }, and the hosted callers pass the samebudget their timer waits on. Losing a
Promise.raceabandons the promise but neverstopped the request, so a silent region leaked a socket + an API-semaphore slot per
authenticated request — the timer bounds the wait, the deadline bounds the resource.
/fr/mcpkept as an EU compat alias (the README's published EU connector URL)— same behavior as
/mcp; the suffix, not the path, selects the region.Testing
pnpm -r test→ 1436 passing (@leadbay/mcp715), typecheck and build clean.New coverage lives in new files (repo rule): single-region probe
(ok/expired/5xx/validate-off/legacy dual-probe/transient-fallback/bind-region),
probe resilience (a stalled region, and a legacy token that merely looks tagged),
the per-attempt request deadline, single Stargate auth-server PRM, and the
/mcp+/fr/mcpchallenge behavior. The product#3761 empty-401-body guard ispreserved.
Two pre-existing test files lost assertions that name APIs this PR deletes —
regionAuthServer,protectedResourceMetadata({region}), theprobe_failedauthState. Everything that could be kept was kept, and each file says at the point
of removal which contract replaced it.
Part of a 3-repo change
Companions: stargate (leadbay/stargate#14 — the authority) + backend
(leadbay/backend#1927 — mint the region-suffixed token, honor the shared credential).
Related: https://github.com/leadbay/product/issues/3872
Deploy / handoff
Deploy this 3rd (LAST) of 3 — dependency rule: backend (accepts) → stargate
(calls backend) → mcp (points users at Stargate).
region-suffixed tokens).
stargate.leadbay.app(staging:
staging.stargate.leadbay.app).auth.leadbay.appwas the name inthe original plan and was never deployed; discovery now points at the live host.
mcp.leadbay.applast. It advertises Stargate asthe single auth server; if it ships before Stargate is live, new sign-ins would be
sent to a Stargate that isn't up. Already-connected users hold tokens that keep
working (region decoded from the suffix; untagged tokens dual-probe).
OAUTH_STARGATE_ONLY=true, both regions).Decisive test (after this deploy + cutover): a real FR account connects on
mcp.leadbay.app/mcpand completes sign-in + a tool call; repeat with a US account.