Skip to content

fix(ws): replace a silent mochi or crew-companion socket - #32931

Merged
bolichen97 merged 1 commit into
mainfrom
fix/14225-app-socket-silence-watchdog
Nov 8, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
fix/14225-app-socket-silence-watchdog

Conversation

@bolichen97

@bolichen97 bolichen97 commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: The Mochi pet panel and the Crew Companion recover on their own when their /api/ws socket goes quietly dead, instead of sitting stale until the page is reloaded.

Both app sockets reconnect only from onclose. A browser can keep a WebSocket whose transport is gone (a phone that changed networks, a tab the OS froze and resumed) with its readyState still OPEN and onclose never firing. The socket then delivers nothing and nobody notices: the pet's chat stream silently stops, and the companion's "tell me when sessions are done" bubble never fires again. The dashboard's own socket got a silence watchdog in #14222; these two did not.

Why it matters

A dead-but-open socket is invisible to the user: the UI looks connected while every live frame is lost, and the only fix is a manual reload. The gateway already pushes a dashboard status frame on every socket every 5s, so the liveness signal to detect this is already there; these two sockets were not using it.

Not a goal

What changed (motivation → approach → change)

The Mochi panel and the Crew Companion now replace their /api/ws socket when it stays open but goes silent, the same way the dashboard does, so they come back without a reload.

Socket On a socket silent for 20 s of visible time
Dashboard (hooks/useWebSocket.ts) replaced through forceReconnect, as before
Mochi panel (apps/mochi/panel/panelBridge.ts) closed and reopened at once
Crew Companion (apps/crew-companion/sessionWatch.ts) closed and reopened at once; its in-flight turns are settled by the fresh socket's first slots frame

The silence rule used to live inside the dashboard hook, so copying it into two more sockets would fork its timing and its hidden-tab bookkeeping three ways. It now lives once in website/src/utils/socketSilenceWatchdog.ts (SilenceWatchdog), and all three sockets drive it; each supplies only how to read whether its socket is open and how to replace it. The dashboard hook keeps one instance for its lifetime, so the count of silent replacements in a row survives a re-run of its effect. The helper sits in utils/ because hooks/websocket/ is the dashboard facade's private directory.

On a watchdog replacement the Crew Companion does not report its running turns as failed: the socket died, not the gateway, so those turns usually still run. The fresh socket's first slots frame decides instead: a turn it lists as running finishes through its own chat_done, and a turn it does not list is reported as failed, as a dropped socket reports it. When that frame lists the slot running under a different turn_gen/turn than the one the watcher recorded, the earlier turn ended unseen: it is reported as failed in the same way, and the new turn starts fresh with its own error and stop flags. When either identity is unknown (an older gateway), the running turn is kept. If the fresh socket closes before that frame, the turns fail as before. docs/system-specs/modules/learn-cron-dashboard.md names the helper and both app sockets, and test/test_ws_status_cadence_contract.py reads the timing constants from their new file.

Builds on #22652 by @dodgeblaster, rebased onto current main.

Backwards compatibility

Compatible: the dashboard socket keeps its behaviour (its existing watchdog specs in useWebSocket.silenceWatchdog.test.ts run unchanged against the shared helper), and the two app sockets only gain a replacement path for a state, open but silent, in which they did nothing before.

Removes nothing: no path, record, validator or public function is removed or tightened; useWebSocket keeps re-exporting WS_SILENCE_MS, WS_SILENCE_CHECK_MS and WS_SILENCE_MAX_MS, now from the shared helper.

Tests

  • website/src/test/socketSilenceWatchdog.test.ts: the helper alone replaces a silent OPEN socket, keeps one that delivers frames, skips a socket that is not OPEN, skips while hidden, doubles the window per silent replacement up to the cap, keeps the widened window across stop() + start(), and detaches on stop().
  • website/src/apps/mochi/test/mochiSocketSilence.test.ts: the Mochi socket is replaced after the window, is kept while it receives the 5 s status frame, and is left alone on a hidden page.
  • website/src/apps/crew-companion/sessionWatch.socketSilence.test.ts: the same three for the Crew Companion socket, plus what onDone receives on a replacement: nothing at the replacement, one success for a turn the first slots frame lists running, one failure for a turn it does not list, a successor under a new turn or turn_gen inherits neither the earlier turn's error nor its stop (and keeps its own stop from the settling frame), only the first slots frame settles, and a close before that frame fails the turn.
  • website/src/test/useWebSocket.ownership.test.ts: the facade's effect-order pin names the watchdog effect by its silenceHostRef wiring.
  • test/test_ws_status_cadence_contract.py: the pin between the frontend silence window and the gateway's _WS_STATUS_INTERVAL reads the constants and the thaw grace from socketSilenceWatchdog.ts; the contract is unchanged.

With sessionWatch.ts and panelBridge.ts put back to main, six of the eleven per-socket specs fail; with this change all pass. Every spec uses fake timers and an injected or mocked clock, with no real sleeps.

Proof: not provable -- frontend test (prove.py runs Python only)

Manual verification

N/A: unit coverage is sufficient. The behaviour is a timing rule over socket state, fully driven by fake timers and a mocked clock, with no rendered output to inspect.

Screenshots / video

Why no screenshot: this changes when an open but silent socket is replaced; no component, layout, theme or string renders differently.

Related Issues

Closes #14225

Sibling sweep

Swept: git grep -nE 'new WebSocket(' -- website/src
Sibling: website/src/hooks/useWebSocket.ts -- changed here
Sibling: website/src/hooks/useStreamingStt.ts:304 -- not affected -- /api/ws/stt carries no periodic status frame, so silence there is not a liveness signal
Sibling: website/src/apps/meetings/hooks/useMeetingTranscription.ts:629 -- not affected -- the same /api/ws/stt endpoint, no periodic status frame
Sibling: website/src/utils/terminalRegistry.ts:521 -- not affected -- a terminal socket is silent whenever the shell prints nothing

Pattern harvest

Rule candidate: review-prompt
Pattern: a per-connection liveness or silence watchdog copied inline into each socket instead of shared, so the timing constants drift between copies.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ✅ PASS

UX-level review of 11dc686f8f7a9fecc71493fa0ddbb552776a84ca — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

UX-Verdict: PASS

Nothing to check. The diff only changes when the socket reconnects. No component, string or layout renders differently, so the missing screenshot leaves no evidence gap.

[UX-REVIEWED] 11dc686

Description read

[DESCRIPTION-READ] 5d93ab3f1278782f5b4163327f1f2df4fd35f13d0a86f5964cbe89998516b6cf

That sha256 names everything this verdict read: the PR description, plus the 3 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 8, 2026
@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 11dc686f8f7a9fecc71493fa0ddbb552776a84ca — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the shared watchdog is the dashboard's existing silence check moved into one helper. It runs on the gateway's existing 5 s status frame, and the change can be cleanly reverted.

Notes

  • The Crew Companion also compares turn identities on the first slots frame (turnIdentities, turn_gen:turn). That frame now reports a predecessor turn as failed when a different turn holds its slot. The description's settle rule doesn't mention this, so add one line about it.

[DESIGN-REVIEWED] 11dc686

Description read

[DESCRIPTION-READ] 3e431b7e05a5abd74dfdce6d92ce175616400c90de072b665610cb4348a0069d

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 11dc686f8f7a9fecc71493fa0ddbb552776a84ca — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. The fix covers all three /api/ws sockets in website/src, and both new sockets reuse the dashboard's existing silence rule through one shared helper instead of keeping their own copies.

What this change ships

Inventory (8 items) — 7 justified

Intent: the Mochi panel and the Crew Companion reconnect on their own when their socket stays open but stops delivering, with no page reload needed. This is a FIX (Closes #14225).

  1. The Mochi panel replaces an open socket that has been silent for 20 s while the page is visible. — justified
  2. The Crew Companion replaces its silent open socket the same way. — justified
  3. On a silent replacement, the Companion waits for the first slots frame from the new socket before reporting in-flight turns as failed. — justified
  4. The Companion reports a turn as failed when the gateway now runs that slot under a different turn/turn_gen, and the new turn starts with clean error and stop flags. — justified
  5. The dashboard's silence rule moved into utils/socketSilenceWatchdog.ts and all three sockets use it. The dashboard behaves the same. — justified
  6. The timing constants moved to the helper; useWebSocket still re-exports them. — justified
  7. The Mochi online indicator briefly shows offline during a silent replacement (setOnline(false)). — undeclared
  8. The spec paragraph, the cadence contract test and the effect-order pin now point at the helper. — justified

[FIRST-PRINCIPLES-REVIEWED] 11dc686

Description read

[DESCRIPTION-READ] d663f13f1b0148d7ecb2158ed1616e38eba1bc42162fdf171e77ef75c0668d8c

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 11dc686f8f7a9fecc71493fa0ddbb552776a84ca — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 11dc686

Verdict parsed from the review's SHA-scoped output markers for commit 11dc686f8f7a9fecc71493fa0ddbb552776a84ca.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 11dc686f8f7a9fecc71493fa0ddbb552776a84ca false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 11dc686f8f7a9fecc71493fa0ddbb552776a84ca and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 11dc686

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 11dc686f8f7a9fecc71493fa0ddbb552776a84ca false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 8, 2026
@bolichen97
bolichen97 enabled auto-merge (squash) November 8, 2026 05:14
@bolichen97
bolichen97 force-pushed the fix/14225-app-socket-silence-watchdog branch from 13ce59c to bb776b6 Compare November 8, 2026 09:09
@bolichen97

Copy link
Copy Markdown
Contributor Author
  • New core spec imports a specific built-in app span=d6d085cb64a6

fixed in bb776b6: the spec moved from website/src/test/CrewCompanionSocketSilence.test.ts to website/src/apps/crew-companion/sessionWatch.socketSilence.test.ts, next to sessionWatch.ts, and imports ./sessionWatch and ../../utils/socketSilenceWatchdog. No core file imports the app any more for this change. The test body is unchanged and still passes (vitest, 4 files, 28 tests).
The same push rebases onto main 43ac0cd; the patch is otherwise identical (same git patch-id).

[operator: bolichen97]

AI-assisted, posted on behalf of bolichen97

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 8, 2026
The Mochi panel and Crew Companion sockets reconnected only from
onclose, so a socket that stayed OPEN with its transport gone sat stale
until a reload. The dashboard's silence rule is extracted into one
framework-agnostic SilenceWatchdog (website/src/utils/
socketSilenceWatchdog.ts) that the dashboard hook, the Mochi panel
bridge and the Crew Companion session watcher all drive.

The dashboard hook keeps one watchdog instance for its lifetime, so the
consecutive-replacement streak survives a re-run of its arming effect.

On a watchdog replacement the Crew Companion leaves its in-flight turns
to the fresh socket's first slots frame: a slot still running is closed
by its own chat_done, a slot no longer running is reported as failed.
A close before that frame fails them as any dropped socket does.

Builds on #22652 by @dodgeblaster.

Closes #14225

Co-authored-by: Gary Jennings <garysjennings@gmail.com>
@bolichen97
bolichen97 force-pushed the fix/14225-app-socket-silence-watchdog branch from bb776b6 to 11dc686 Compare November 8, 2026 09:37
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 8, 2026
@bolichen97

Copy link
Copy Markdown
Contributor Author
  • Reconnection carries the previous turn's state into its successor span=d9c9f6fd13b9

fixed in 11dc686: the watcher records each running slot's turn_gen:turn from its slots rows (the gateway sends both on a running dashboard turn). When the first slots frame after a watchdog replacement lists the slot running under a different identity, the predecessor is ended first, reported as failed the same way a turn absent from that frame is, and the successor then starts fresh as an assumed start with its own flags. A stopping: true in that same frame now belongs to the successor. When either identity is unknown (an older gateway), the running turn is kept as before.
Tests in website/src/apps/crew-companion/sessionWatch.socketSilence.test.ts: does not carry an error into a successor with identity $turn_gen:$turn (a new turn, and a new turn_gen), does not carry a deliberate stop into a successor with a different identity, and keeps the successor stop from the settling frame without suppressing its predecessor. All four cases failed on bb776b6 and pass here.

[operator: bolichen97]

AI-assisted, posted on behalf of bolichen97

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Nov 8, 2026

@buluoray buluoray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed head 11dc686. A silent mochi or crew-companion socket is now replaced, with bounded reconnects. Checked against builder-security: no new listener or auth change.

@bolichen97
bolichen97 merged commit 9e4587d into main Nov 8, 2026
200 of 232 checks passed
@bolichen97
bolichen97 deleted the fix/14225-app-socket-silence-watchdog branch November 8, 2026 12:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

readiness: passed Eligible automated validation passed for the current revision

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Apply the silent-socket watchdog to the mochi and crew-companion /api/ws sockets

2 participants