Skip to content

fix(dashboard): reconnect quietly on a relay token renewal or drain - #34119

Merged
KiroCrewCaptain merged 1 commit into
mainfrom
fix/relay-crew-typed-states
Nov 10, 2026
Merged

KiroCrewCaptain merged 1 commit into
mainfrom
fix/relay-crew-typed-states

Conversation

@tim-finnigan

@tim-finnigan tim-finnigan commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A Crew dashboard opened through the Kiro relay stops flashing "offline" each time the relay turns its socket over, and says why when the connection really is gone.

Closes #34111

The relay closes the dashboard's /api/ws socket with 4401 when the access token it opened with expires, about every five minutes, after the app holding the token has already renewed it. It closes with 1012 when it drains. The next socket opens about a second later. The dashboard treats both like any other drop, so the top-bar capsule turns red and the phone strip says "Gateway offline — reconnecting" every few minutes. When the token really could not be renewed, it says the same thing, which points at a reconnect that will never succeed.

Relay build plan item 7B: CREW-24633.

Why it matters

Anyone viewing a Crew through Kiro Mobile or the desktop app's relay Crew window sees a false outage every token turnover. The relay design treats a close at token expiry as not an error: the client reconnects and shows nothing.

Not a goal

  • The native screens in Kiro Mobile and the desktop app (Crew list, refusals from CreateCrewAccessToken, "relay unavailable"), which the relay design leaves to the clients.
  • Sending the token as the first WebSocket message. That fallback depends on the iOS WebView cookie test, whose result is not in yet.
  • The plain browser launch page, deferred in the relay design.
  • The Mochi panel, Crew Companion and terminal sockets.

What changed (motivation → approach → change)

The dashboard socket now reads its close code (relayClose.ts). 4401 is a token renewal, 1012 a relay drain, 4400 to 4499 a relay refusal, and everything else a plain drop. Crew's own gateway sends none of these codes on /api/ws, so a direct connection behaves as before.

For a renewal or a drain of a socket that had opened, connection.ts reconnects at once and leaves the dashboard connected. If the new socket opens, nothing was shown. If it fails, or has not opened after 5 s, the dashboard goes offline; after a 4401 the reason is session_ended, and the dashboard stops retrying. The relay refuses an expired token with a plain 401 at the upgrade, which the browser reports as a 1006 close with no open, the same as a network drop, so only the 4401 before it identifies the case. Retrying with that token cannot succeed. The health probe's reconnect, which runs when /api/status answers 200 again, still brings the session back, and if that socket drops before it opens the normal backoff resumes. A refusal goes offline at once with reason refused. The reason (offlineCause in dashboardSlice.ts) stays until a socket opens. appShell.tsx turns it into the capsule's accessible name, which shellHeader.tsx's phone strip now shows too: "Relay session ended — reopen this Crew" or "Relay refused the connection — retrying". With no reason, both keep "Gateway offline — reconnecting". The two strings are added to en.manual.json and the eleven translated catalogs, and en-XA.json is regenerated.

Backwards compatibility

Compatible: a socket closed with any code outside 4401, 1012 and 4400 to 4499 is handled exactly as on main, and Crew's gateway sends none of those on /api/ws. markGatewayDisconnected and sseDisconnected take an optional reason; existing callers pass none.

Tests

  • website/src/test/useWebSocket.relayQuietReconnect.test.ts drives the real hook with a mock socket. It checks that 4401 and 1012 reconnect at once and never show offline, including across repeated renewals. It checks a failed reconnect after 4401 (refused at the upgrade, or still opening after the grace) shows session_ended, and that the reason survives later failed reconnects, including a health-probe replacement after the grace. It also checks that a failed reconnect after a 4401, whether it fails at once or after the grace, opens no further socket for two minutes, that the health probe still brings it back and an open clears the reason, that a probe reconnect which drops goes back to the backoff, and that a direct dashboard whose handshakes keep failing stays plain offline and keeps retrying. It also checks that a failed reconnect after a drain shows a plain offline, that 4429 shows refused at once, and that a plain drop still goes offline at once.
  • website/src/hooks/websocket/relayClose.test.ts covers the code table. website/src/test/offlineReason.test.ts covers the wording.
  • website/src/test/gatewayConnectedPairing.test.ts and website/src/test/useWebSocket.ownership.test.ts now admit the reason argument and the new owner module.

Proof: not provable -- frontend test (prove.py runs Python only)

Run by hand instead: with connection.ts reverted to main, the hook test file fails 12 of 14. With only the new stop rule reverted, the two stop-retrying tests fail. With the stop keyed on the displayed reason instead of the failed reconnect, the probe-reconnect test fails. With the stop rule widened to every reconnect after a first connect, the direct-dashboard test and the plain-drop test fail.

tsc -p tsconfig.app.json is clean. The 269 test files that reference the changed modules all pass, as do the i18n parity and duplicate-key tests. The full local gate did not run: scripts/local-gate.py selects 1,244 backend and 138 frontend files for this diff, and a vitest related run over the changed sources timed out at 25 minutes on this host, so the change is held to the 269 files above, tsc and CI.

Manual verification

Run against the production build with Playwright, socket closes faked through page.routeWebSocket (phone viewport, where the strip shows). The appearances column counts every time the offline strip appeared during the run. The first one in every row is page load, before the first socket opens.

Scene main this PR
4401, next socket opens 2 (load, then a flash) 1 (load only)
4401, next sockets fail ends on "Gateway offline — reconnecting" ends on "Relay session ended — reopen this Crew"
4429, next sockets fail ends on "Gateway offline — reconnecting" ends on "Relay refused the connection — retrying"

A manual relay test on the previous head (bbb813f), with a host Crew on the beta relay and a Firefox viewer holding a session cookie that was never renewed, found the gap this head fixes. After the token expired, the page tried /api/ws 12 times over about two minutes with growing backoff. Every try got HTTP 401 at the upgrade, so no socket opened and no close code arrived, and the page kept retrying a token that could no longer work. On this head, the first reconnect that fails after the 4401 shows "Relay session ended" and no further tries are made. That is pinned by the unit tests above and has not been re-run against the relay.

Not checked: a real Kiro Mobile WebView or the desktop relay Crew window. Those renew the token, so the quiet path is the one they take.

Screenshots / video

Shot on 20d8d80 (after) and main (before) from the production build, socket closes faked with Playwright, cropped to the top bar. Each pair is before, then after.

Desktop, 1300 px. On desktop the reason is the capsule's accessible name and tooltip, not visible text, so the pairs look alike; the measured name is under each.

4401, then the token is not renewed. Before: "Gateway offline — reconnecting". After: "Relay session ended — reopen this Crew".

before, desktop, session ended

after, desktop, session ended

  1. Before: "Gateway offline — reconnecting". After: "Relay refused the connection — retrying".

before, desktop, refused

after, desktop, refused

4401, then the next socket opens. Both end connected; counting every offline appearance during the run, main showed offline twice (page load, then the flash) and this PR once (page load only).

before, desktop, renewal

after, desktop, renewal

Phone, 390 px, where the strip shows the reason as text.

before, phone, session ended

after, phone, session ended

before, phone, refused

after, phone, refused

In this harness /api/status answers 200, so the health probe keeps bringing the socket back; on the real relay it gets a 401 too, which is why Tim's run on this head saw one failed try and no more.

Related Issues

Closes #34111

Sibling sweep

Swept: git grep -n "onclose = " -- website/src ':!.test.'
Sibling: website/src/apps/mochi/panel/panelBridge.ts -- not affected -- its socket never moves the dashboard connected flag or its offline wording
Sibling: website/src/apps/crew-companion/sessionWatch.ts -- not affected -- its socket never moves the dashboard connected flag or its offline wording

Pattern harvest

Not generalizable: the relay's close codes are read in one place, the dashboard socket, and no other socket shows the dashboard's offline state.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 9, 2026
@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 20d8d80577f10c05118047df98e75e0da29361c9 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

No design-level concerns. The close-code check sits in one owner module, only the relay's codes change behaviour, and the health probe still restores the stopped-retry path once the token is renewed.

Notes

  • The description links an internal tracker (taskei.amazon.dev). The internal-content-scan check may flag it, so drop the link before merge.

[DESIGN-REVIEWED] 20d8d80

Description read

[DESCRIPTION-READ] 58c44c95628c890b8910dce5b73c877360db7a03cd0f938230139aa12c7d446e

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 20d8d80577f10c05118047df98e75e0da29361c9 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. The quiet reconnect, the stop-retrying rule and the two reasons each come from the relay protocol fixture (test/fixtures/relay_protocol_v1) or from the manual relay test the description reports.

What this change ships

Inventory (8 items) — 8 justified

Intent: someone viewing a Crew through the Kiro relay stops seeing a false "offline" at every token turnover, and is told why when the connection really is gone. FIX.

  1. A 4401 or 1012 close of a socket that had opened reconnects at once, with no offline flash for up to 5 s — justified
  2. A reconnect after a 4401 that fails shows "Relay session ended — reopen this Crew" — justified
  3. After that failed reconnect, automatic retries stop; the health probe brings the session back — justified
  4. A 4400–4499 close goes offline at once with "Relay refused the connection — retrying" — justified
  5. The phone offline strip now shows the capsule's status message instead of the fixed generic one — justified
  6. The dashboard store keeps an offline reason until a socket opens — justified
  7. markGatewayDisconnected / sseDisconnected take an optional reason; existing callers are unchanged — justified
  8. Two new strings in the 12 locale catalogs — justified

[FIRST-PRINCIPLES-REVIEWED] 20d8d80

Description read

[DESCRIPTION-READ] 02d7e18217503fc8c714089fdc2144502fc304ea86e0cb74d635b0319ce6c172

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ✅ PASS

UX-level review of 20d8d80577f10c05118047df98e75e0da29361c9 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

UX-Verdict: PASS

No blocker meets a Trigger/Outcome bar: the new strings say what happened, and every new state is in a screenshot. The one action the copy asks for happens outside the page.

Notes

  • relay_session_ended "Relay session ended — reopen this Crew": the blind reader saw "no button for doing that" and didn't know "how you reopen one". Say where to do it, e.g. "Relay session ended — open this Crew again from Kiro".
  • The strip looks the same in every state (same red, same pulsing dot), so "Relay session ended" (you need to act) and "Relay refused the connection — retrying" (wait) can't be told apart. The reader: "I can't tell which one needs me to act." Use a different treatment or icon for the state that needs action.
  • "Relay refused the connection" made the reader think "I'm not allowed in… my fault". If the refusal is temporary, something like "Relay busy — retrying" would read better.

[UX-REVIEWED] 20d8d80

Description read

[DESCRIPTION-READ] 87edcb0d1c706ab271b2999728751d7d5fe16c5abfbe087fe861c3da485e18c1

That sha256 names everything this verdict read: the PR description, plus the 7 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 20d8d80577f10c05118047df98e75e0da29361c9 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 20d8d80

Verdict parsed from the review's SHA-scoped output markers for commit 20d8d80577f10c05118047df98e75e0da29361c9.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 20d8d80577f10c05118047df98e75e0da29361c9 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 20d8d80577f10c05118047df98e75e0da29361c9 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 20d8d80

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 20d8d80577f10c05118047df98e75e0da29361c9 false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 9, 2026
@tim-finnigan
tim-finnigan force-pushed the fix/relay-crew-typed-states branch from d463ff1 to bbb813f Compare November 9, 2026 21:28
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 9, 2026
@tim-finnigan

Copy link
Copy Markdown
Contributor Author
  • fixed: Persist the grace-expiry cause before replacing the socket span=2ec338771dc9

fixed in bbb813f: the grace timer now writes session_ended to causeRef before it dispatches, so a health-probe forceReconnect that drops quietRef no longer loses the reason.
New test keeps the reason when the health probe replaces a reconnect that outlived the grace fails with the old timer and passes with the fix.
self-added: yes
mechanism: quiet-reconnect grace timer and the sticky relay reason

@tim-finnigan

Copy link
Copy Markdown
Contributor Author
  • rebutted: Render relay connection failures through ErrorNotice span=4011460c0959

Not an error surface under the rule's own definition. errors-use-error-notice covers the outcome of something that FAILED (a rejected request, a caught exception, a query error, a backend body); it excludes status text. The strip at shellHeader.tsx:713 is the existing connection-status readout: on main it already renders Gateway offline — reconnecting as role="status" text for every socket drop, and the socket keeps reconnecting underneath it.
This PR changes only which status sentence that readout shows. Its value comes from the socket's close code, not from a failed request, and the state clears by itself when a socket opens.
The ErrorNotice hand-off cannot work in this state: it navigates to chat, which needs the very socket that is down, so askAgent would be a dead end.
Desktop is unchanged in kind: the capsule turns red and carries the reason in its accessible name, as it already did for "Gateway offline".

@tim-finnigan
tim-finnigan force-pushed the fix/relay-crew-typed-states branch from bbb813f to 7414c44 Compare November 10, 2026 04:34
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 10, 2026
@tim-finnigan
tim-finnigan force-pushed the fix/relay-crew-typed-states branch from 7414c44 to 20d8d80 Compare November 10, 2026 05:15
@tim-finnigan

Copy link
Copy Markdown
Contributor Author
  • fixed: A failed health-probe reconnect permanently stops recovery span=2ec338771dc9

fixed in 20d8d80: the stop is keyed on the closing socket being the failed reconnect after the 4401 (failedQuiet?.kind === 'token_renewal'), not on the sticky session_ended reason. A health-probe reconnect that drops before opening falls back to the normal backoff and keeps the reason on screen.
New test a health-probe reconnect that drops after the stop goes back to retrying fails on 7414c44 and passes here; the two stop-retrying tests still fail on bbb813f.
self-added: yes
mechanism: stop retrying after a failed reconnect that follows a 4401

@tim-finnigan

Copy link
Copy Markdown
Contributor Author
  • fixed: if (causeRef.current === 'session_ended') return span=7ca3fcb3d516

fixed in 20d8d80: the stop is keyed on the closing socket being the failed reconnect after the 4401 (failedQuiet?.kind === 'token_renewal'), not on the sticky session_ended reason. A health-probe reconnect that drops before opening falls back to the normal backoff and keeps the reason on screen.
New test a health-probe reconnect that drops after the stop goes back to retrying fails on 7414c44 and passes here; the two stop-retrying tests still fail on bbb813f.
self-added: yes
mechanism: stop retrying after a failed reconnect that follows a 4401

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 10, 2026
@tim-finnigan
tim-finnigan marked this pull request as ready for review November 10, 2026 05:59
@tim-finnigan
tim-finnigan requested review from a team and KiroCrewCaptain as code owners November 10, 2026 05:59
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 10, 2026
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention readiness: passed Eligible automated validation passed for the current revision and removed readiness: passed Eligible automated validation passed for the current revision readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Nov 10, 2026
@KiroCrewCaptain
KiroCrewCaptain enabled auto-merge (squash) November 10, 2026 07:45

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. On this head the review traced every close path in connection.ts: the quiet 4401/1012 reconnect, the 5 s grace timer, the stop rule after a failed renewal reconnect, and the reset when a socket opens. It also confirmed the health probe still brings a stopped session back, and that a direct gateway connection takes the same path as main because the gateway only closes with 1008. The new hook, classifier and offline-reason tests ran on this head and passed. Reverting connection.ts to main, deleting the stop rule, or keying the stop rule on the shown reason each fails the tests named in the PR body.
Well done: the stop rule depends on the socket that is closing, not on the reason shown on screen, so a probe reconnect that drops goes back to normal backoff. A dedicated test pins this, and the relay close codes live in one small owner module with its own test.

Optional:

  • #33825 also edits connection.ts and removes resetBackoff, the open hook where this PR clears the quiet timer and the remembered cause. Whichever PR merges second should move that clearing into the open hook that remains, and both bodies should name the merge order.
  • Deleting causeRef.current = undefined in resetBackoff fails no test. A test for refused, then open, then a plain 1006 drop, asserting no relay reason, would pin it.
  • The PR body still links an internal tracker. Please remove that line. A body edit only re-runs readiness.

@KiroCrewCaptain
KiroCrewCaptain merged commit 930867f into main Nov 10, 2026
112 of 120 checks passed
@KiroCrewCaptain
KiroCrewCaptain deleted the fix/relay-crew-typed-states branch November 10, 2026 07:45
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Relay Crew dashboard flashes offline on every 4401 and 1012 close

2 participants