Skip to content

perf(web): land each frame of stream traffic as one store action - #23712

Merged
iamwhatever merged 1 commit into
kirodotdev:mainfrom
Pearcekieser:perf/perf-stream-single-flush
Oct 25, 2026
Merged

iamwhatever merged 1 commit into
kirodotdev:mainfrom
Pearcekieser:perf/perf-stream-single-flush

Conversation

@Pearcekieser

@Pearcekieser Pearcekieser commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A frame of stream traffic costs the web app one store dispatch however many sessions stream, so several sessions running at once do less main-thread work.

The socket buffers chat chunks, reasoning, sub-agent text and sidebar recency, then lands them once per animation frame. It landed them with one dispatch per streaming session per kind, and on top of that each chunk or reasoning frame that changed a session's status line, and every chat_status label, dispatched setSlotStatusDetail the moment it arrived. With 5 sessions streaming that is 3,179 dispatches in 30 s, 1,495 chunk flushes and 905 status writes among them.

Why it matters

Every dispatch runs every reducer and notifies the store's subscribers, and each mounted useAppSelector re-runs on a notification. On the 5-session benchmark the dispatch path takes about 370 ms per 30 s on a desktop CPU and grows with every extra streaming session.

Not a goal

  • Reducing the number of selectors a sidebar row or tool row mounts.
  • Batching tool calls, tool results, unread marks or stored rows, which still dispatch once per frame message.
  • Changing how a background session's notification is deferred.

What changed (motivation → approach → change)

The flush in hooks/websocket/streamBuffers.ts now lands everything the buffers hold as one stream/flush action (website/src/store/streamFlush.ts), on one shared frame instead of three; the hold comment in lib/streamHold.ts says so. The chat slice applies each session's reasoning, then its text, then the status line those events imply, then sub-agent text. The dashboard slice applies the recency bumps through the same write touchSlotActivity uses. A synchronous flush before a row lands only the buffer that row must follow, so a burst of rows still coalesces recency to the frame.

The status line is now written by that reducer, so chat_chunk and chat_thinking no longer dispatch (hooks/websocket/chatStream.ts). A chat_status label (hooks/useWebSocket.ts) is buffered with the session's chunks, because a label that arrives after a chunk must win over the chunk's "streaming", and a chunk after the label must win over the label. The buffer records the store's status entry when the newest event arrived. If anything wrote the entry directly since, such as a tool call or a turn end, that write is newer and the reducer leaves it. A tool-call refinement does not merge into a tool line that a buffered label or chunk has already replaced.

Frame Before After
N sessions stream text N chunk dispatches, plus a status dispatch per status change 1 stream/flush
chat_status label 1 dispatch at arrival lands in the frame's stream/flush
recency bumps for N sessions N dispatches inside the same stream/flush

Backwards compatibility

Compatible: the action is client-only and new. touchSlotActivity, setSlotStatusDetail, sseThinkingChunk and sseSubagentBatchChunks keep their reducers and every other caller. A status label and the streaming or thinking transition reach the store at most one frame later than before, and in a hidden tab with the frame flush (the 50,000-character overflow flush or the next row).

Removes nothing: no reducer, action creator, stored field or public function is deleted; the stream path stops dispatching three action types it used to.

Tests

  • website/src/test/useWebSocket.streamFlush.test.ts (new, 10 cases). One frame of chunks, reasoning, sub-agent text and recency for four sessions is one dispatch. A chunk dispatches no status of its own. A chat_status label or a tool call after a chunk keeps its status, and a chunk after either moves the line to streaming. Reasoning and text give streaming in either order. A tool refinement after a buffered label does not merge into the replaced line. A tool refinement equal to the stored line still wins over reasoning buffered before it, although the store skips that equal write. With the reducer's arrival check removed, the label and tool cases fail; with the refinement guard removed, the refinement case fails.
  • Updated for the one-action shape: useWebSocket.frameTrace.test.ts (the routing and lifecycle traces), useWebSocket.streamBatch.test.ts, useWebSocket.thinkingStatusChurn.test.ts, useWebSocket.slotActivityCoalesce.test.ts, useWebSocket.endedTurnFlush.test.ts and UseWebSocketCoverage.activity.test.tsx. Their assertions on final state are unchanged; status assertions now run after the frame.
  • Proof: not provable -- frontend test (prove.py runs Python only)
  • The frontend related set (121 files, 2,356 tests) and tsc -p tsconfig.app.json pass, and eslint is clean on the changed files.

Manual verification

Gateway-free render benchmark from the open benchmark branch perf/perf-bench-ci, run from a scratch copy that is not part of this diff, with two local counters added: store notifications and time spent inside dispatch. 222 sessions, open session perf-long with 100 rows, desktop 1440x900, react-dom profiling build, 30 s window. main 5d7a22d against this branch, the two builds interleaved, median of 3 runs each.

Scenario Dispatches Notifications Selector calls Dispatch ms Script ms
5 background sessions 3,179 → 1,354 770 → 384 455,774 → 221,186 (−51%) 366 → 231 (−37%) 1,937 → 1,779
1 background session 647 → 498 481 → 332 288,866 → 198,363 (−31%) 181 → 126 (−30%) 1,176 → 1,217
open session streams prose 316 → 315 315 → 314 230,061 → 229,715 259 → 253 11,952 → 11,666

In the 5-session run the remaining 1,354 dispatches are 487 stream/flush (300 frames, plus a flush ahead of each of the 150 tool rows and the segment rows), 190 unread marks, 160 tool statuses, and 150 each of tool activity, tool rows and tool results. Commits and renders are unchanged in every scenario. Script ms is noisy here: the host ran at a load average near 60, and the 1-session runs spread from 1,113 to 1,302 ms across both builds.

Screenshots / video

Why no screenshot: the transcript, status line and sidebar render the same pixels; a status change can reach the screen one frame later.

Related Issues

Fixes #23711

Pattern harvest

Not generalizable: this merges one module's per-frame flushes into one action; other dispatch sites are per-message and have no frame buffer to merge.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@Pearcekieser
Pearcekieser requested review from a team and KiroCrewCaptain as code owners October 24, 2026 09:03
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 24, 2026
@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — ✅ PASS

Premise-level review of e28ca26b6306812ea85e9dbae64cf5b59dd2362b via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks: each item comes from the measured dispatch cost in #23711, which this PR reproduces as 3,179 → 1,354 dispatches. The status-ordering pieces are the price of moving status writes into the flush.

What this change ships

Inventory (8 items) — 8 justified

Intent: let several sessions stream at once with less main-thread work, by making one frame of stream traffic cost one store dispatch. FIX (perf, #23711).

  1. Every streaming session's chunks, reasoning, sub-agent text and sidebar recency land in one stream/flush per frame, not one dispatch per session per kind — justified
  2. The three per-buffer frame timers merge into one shared frame — justified
  3. A chunk or reasoning frame's streaming/thinking status line is now written by the flush, so the chunk no longer dispatches it on arrival — justified
  4. A chat_status label reaches the store up to one frame later, buffered in arrival order with chunks — justified
  5. A tool call written directly after buffered events wins over them (the seen identity check plus supersedePendingStatus) — justified
  6. A tool refinement no longer merges into a line that a buffered label replaced — justified
  7. An open-slot chunk and its status change now notify subscribers once instead of twice — justified
  8. The touchSlotActivity write moves into applySlotActivity, which the flush reuses — justified

[FIRST-PRINCIPLES-REVIEWED] e28ca26

Description read

[DESCRIPTION-READ] 9ba0cc171bc96062fb7abf7af53775e7b1de6d85b6ac085c2618a32318a24250

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ✅ PASS

UX-level review of e28ca26b6306812ea85e9dbae64cf5b59dd2362b via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

UX-Verdict: PASS

Nothing to check. The PR changes only store and socket plumbing and adds no control, string or layout; the only visible effect is that a status label can appear one frame later.

[UX-REVIEWED] e28ca26

Description read

[DESCRIPTION-READ] 165b3fb7b98dc319e1987c1a83086ce2b6443fe55f03d097bed210b5313c34fc

That sha256 names everything this verdict read: the PR description, plus the 3 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of e28ca26b6306812ea85e9dbae64cf5b59dd2362b via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: one client-only action replaces per-slot dispatches, nothing reads the action types the stream path stopped sending, and every write is replayed in the same order as before.

Notes

  • The arrival guard relies on identity (statusBefore[k] !== st.seen), so a direct status writer other than tool_call that writes a line equal to the stored one keeps the identity, and the buffered label or chunk then lands over it. supersedePendingStatus covers this for tool_call only, so any later direct writer of slotStatusDetail must remember to call it too.
  • In a hidden tab a chat_status label now waits for the 50k overflow flush or the next row. The status line can therefore stay stale in the background for a long time, not just for one frame; the description accepts this, and a revert undoes it.

[DESIGN-REVIEWED] e28ca26

Description read

[DESCRIPTION-READ] f95450975ada5a157a8304312c4834a95ce1f9ca399c84413504a49bdae95967

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed e28ca26b6306812ea85e9dbae64cf5b59dd2362b via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] e28ca26

@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed e28ca26b6306812ea85e9dbae64cf5b59dd2362b via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] e28ca26

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

note (non-blocking): agreed, nothing to change. setSlotStatusDetail, sseThinkingChunk, sseSubagentBatchChunks and touchSlotActivity keep their reducers; only the stream path stops dispatching them.

rebutted: PASS verdict, no change needed

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

note (non-blocking): PASS acknowledged, nothing to change.

rebutted: PASS verdict, no change needed

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

note (non-blocking): agreed. No control or string changes; a status line can reach the screen one frame later.

rebutted: PASS verdict, no change needed

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 24, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor

Hold, with one small item left. Reviewed at 9f6d7f1089b3165c87ac1b18b22848b55b4d9fac.

The one-action flush is sound as far as I can read it. applyStreamFlush keeps reasoning before content. It also drops a buffered status when a tool call or turn end wrote the entry after the newest buffered event, and recency still goes through the touchSlotActivity write. CI is green and the AI lanes report no findings.

First Principles marks one behaviour as undeclared (inventory item 7). On reconnect or unmount, the salvaged reasoning now carries its status line, and the salvage notifies at once. The First Principles disposition only acknowledges the PASS. Please add a line to What changed, or reply to item 7, saying the salvage now writes the status and notifies immediately and why that is intended.

— crew-pr-reviewer (r124)

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

note (non-blocking): inventory item 7, intended. The reconnect and unmount salvage writes the status line and notifies at once on purpose.

intended: on main, chat_chunk and chat_thinking wrote the streaming or thinking status at arrival. This PR defers that write to the flush, so the salvage carries it; without it, reasoning salvaged on reconnect or unmount would land with the status line never moved to thinking.
intended: notifying at once is unchanged. On main the salvage (flushBufferedThinking) and the unmount recency and sub-agent flushes all dispatched without the autobatch tag; flush(FLUSH_RECONNECT, true) and flush(FLUSH_UNMOUNT, true) keep that.
intended: content the salvage drops does not move the line, because the reducer writes streaming only for a slot the action carries text or status for, and on main that write had already landed at arrival.

@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Oct 24, 2026
@Pearcekieser
Pearcekieser force-pushed the perf/perf-stream-single-flush branch from 9f6d7f1 to 483b378 Compare October 24, 2026 22:17
@Pearcekieser

Copy link
Copy Markdown
Contributor Author

note (non-blocking): rebased on main (c24c7bc) to clear the conflict with #23229, no behaviour change.

#23229 removed ts from SlotStatusDetail and made setSlotStatusDetail skip a write equal to the stored entry. This head drops ts from the buffered chat_status label, the stream/flush payload and the reducer's status writes, and the flush reducer skips an equal status write the same way. The frame-trace expectations differ from the previous head only by the removed ts fields. The 121 related frontend test files (2,356 tests) pass, and tsc and eslint are clean.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: action required A blocking check or review needs attention labels Oct 24, 2026
@Pearcekieser

Copy link
Copy Markdown
Contributor Author

note (non-blocking): PASS with all 7 inventory items justified, nothing to change. Item 7, the reconnect and unmount salvage writing the status line and notifying at once, is answered in issuecomment-6389440482.

rebutted: PASS verdict, no change needed

@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Oct 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Security Scope Review (fork) — ✅ PASS

Which legitimate operations does 483b378f2df246c038f4c12f3c5ada03f746f75c newly refuse? Model proposes, scripts/deny_diff.py decides — updated in place on each push. A script-confirmed regression blocks PR readiness.

adjudicated, zero confirmed regressions

Scope-Verdict: PASS

Reason: This change doesn't tighten anything. It refactors how the dashboard's websocket stream buffers land in the store: chunks, reasoning, chat_status labels, sub-agent text and recency now go out as one stream/flush action per frame. It does not touch the shell deny composite (sensitive-bash, exfil, deny-rules) or any validator, path fence, argv floor, or admission or origin gate, so no legitimate operation can newly be refused.

TIGHTENING: none -- the diff adds no refusal and names no threat. The closest change is a store-side drop of a stale buffered status line, if (statusBefore[k] !== st.seen) continue in applyStreamFlush (website/src/store/chatSlice.ts). That only decides which status label the UI shows. It does not decide whether an operation is allowed.

ADJUDICATED: 7 candidates, 0 confirmed newly-refused

PLATFORM COVERAGE: posix, windows, any. The diff makes no assumption about a path, separator, interpreter, executable name or environment variable. It is browser-side TypeScript running in requestAnimationFrame/Redux. As a control, the candidates cover the maintainers' daily commands for the touched area in each spelling:

  • any: npm --prefix website run test:website, typecheck and lint.
  • posix: npx vitest on a forward-slash spec path, and a read-only git diff with posix separators.
  • windows: the npx.cmd shim with a backslash spec path, and a git diff with backslash separators.

No platform the diff assumes is left unprobed.

[SCOPE-REVIEWED] 483b378

Security scope review -- adjudicated candidates

  • macos-latest (platform posix): 5 classified, 0 newly refused, 5 still allowed.
  • ubuntu-latest (platform posix): 5 classified, 0 newly refused, 5 still allowed.
  • windows-latest (platform windows): 5 classified, 0 newly refused, 5 still allowed.

No candidate flipped from allowed to refused.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 24, 2026
@Pearcekieser
Pearcekieser force-pushed the perf/perf-stream-single-flush branch from 8296dc2 to c79eeb7 Compare October 24, 2026 23:45
@Pearcekieser

Copy link
Copy Markdown
Contributor Author
  • Equal tool refinements let older buffered status overwrite them

issue (non-blocking): fixed in c79eeb7.

fixed: the trigger holds. setSlotStatusDetail skips a refinement equal to the stored tool line, so the entry keeps the identity the buffer recorded for reasoning that arrived before it, and the flush then wrote "thinking" over the newer tool phase.
fixed: onToolCall now calls buffers.supersedePendingStatus(slot) before its status write, so a tool call discards the status implied by events buffered before it, equal write or not. chat_message, steer, segment and turn end already flush the buffer before they write, so the tool call was the only direct writer exposed.
test: useWebSocket.streamFlush.test.ts "keeps a tool refinement equal to the stored line over reasoning buffered before it" fails with the new call removed and passes with it.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

note (non-blocking): two groups of red checks on c79eeb7, neither from this diff.

Bundle Size Gate fails on main too: the ModelEffortDropdown chunk builds at 505.0 KB against its 500.0 KB default budget on this PR's merge-base (1a7190c) with this diff absent. #23866 tracks it and #23867 raises the ceiling; this PR rebases once that lands.

Sensitive Change Review, Dependency License Gate, Internal Content Scan, Opus 5.5 Review and Security Scope Review failed at about 00:01Z before judging anything: "Could not read the PR from the API (CalledProcessError); re-run this job", "could not run", and a scope job that never finished. A re-run of those jobs would clear them; the fork account cannot re-run them, so the rebase onto #23867 restarts them.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 25, 2026
@Pearcekieser
Pearcekieser force-pushed the perf/perf-stream-single-flush branch from c79eeb7 to e28ca26 Compare October 25, 2026 01:46
@Pearcekieser

Copy link
Copy Markdown
Contributor Author

note (non-blocking): rebased on main onto #23867, which raised the ModelEffortDropdown bundle ceiling. No behaviour change; the rebase applied cleanly.

This head restarts the Bundle Size Gate and the five lanes that stopped before judging c79eeb7 (Sensitive Change Review, Dependency License Gate, Internal Content Scan, Opus 5.5 Review, Security Scope Review). tsc is clean and the stream and store tests pass (129 files, 2,382 tests).

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Oct 25, 2026
@Pearcekieser

Copy link
Copy Markdown
Contributor Author

note (non-blocking): PASS, both notes hold as written and need no change here.

rebutted: the hook has three direct setSlotStatusDetail writers. onChatMessage (chatStream.ts:136) calls buffers.flushChunks() before its write, and the chat_done path flushes before turnCompletion.ts:151 writes idle, so a pending status is already landed and cleared when either writes. onToolCall is the one writer that does not flush first, and it calls supersedePendingStatus. A new direct writer would need one of the two, as the note says.
rebutted: the hidden-tab delay for a chat_status label is the trade the PR body states under Backwards compatibility; chunks and reasoning already waited for the same flush on main.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

note (non-blocking): PASS acknowledged, nothing to change. The status-ordering pieces exist only to keep the line in arrival order once status writes move into the flush.

rebutted: PASS verdict, no change needed

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

note (non-blocking): agreed. No control, string or layout changes; a status label can reach the screen one frame later.

rebutted: PASS verdict, no change needed

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 25, 2026

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve. Reviewed at e28ca26b6306812ea85e9dbae64cf5b59dd2362b.

The r124 item is cleared: the First Principles disposition explains why the reconnect and unmount salvage writes the status line and notifies at once. applyStreamFlush keeps arrival order and drops a buffered status once something wrote the entry directly. The equal-refinement gap the GPT lane found is closed by supersedePendingStatus in onToolCall, with a test. I checked the Design rebuttal at this head: the other two direct status writers (onChatMessage and the turn-end idle write) flush the buffer first. Every AI lane on this head is PASS or has no findings, each note has a disposition, and CI is green, including the Bundle Size Gate after the rebase onto #23867.

— crew-pr-reviewer (r125)

@iamwhatever
iamwhatever merged commit 14fc309 into kirodotdev:main Oct 25, 2026
103 checks passed
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf(web): one frame of stream traffic costs one dispatch per session per kind

2 participants