Skip to content

fix(chat): keep a streaming reply and its tables drawn on slow devices - #37414

Merged
KiroCrewCaptain merged 1 commit into
mainfrom
crew/kiro-crew-rust/stream-reply-visible
Nov 15, 2026
Merged

KiroCrewCaptain merged 1 commit into
mainfrom
crew/kiro-crew-rust/stream-reply-visible

Conversation

@buluoray

@buluoray buluoray commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A reply that streams on a slow device shows up as it arrives, and a streaming table costs work per row instead of per character.

On a slow device a reply can stream from start to end with nothing on screen. With the CPU throttled 4x, a 45 s reply into an open 100-row chat never reached the transcript, and neither did the prompt that asked for it: 6 of 6 runs on main. The store held every chunk and the stop button was lit, but the transcript still showed the chat as it was before the send.

While a table streams, the dashboard parses the whole table again and restyles every cell on each animation frame (#37358).

Why it matters

Sending a message and watching the answer arrive is the main thing the chat does. On a slow laptop or phone the user sends, sees nothing, and cannot tell whether the message went out. The table cost grows with the table, because each frame pays for every row drawn so far, and it spends main-thread time that the reply's own text needs on that same slow device.

Not a goal

What changed (motivation → approach → change)

Two changes, one per symptom.

Symptom Cause Change
The prompt and the reply stay off screen on a slow device Rows added to the transcript went through its deferred render, and every streaming flush restarts that render pages/chat/page/transcriptRows.ts: rows added at the tail render in the commit that brings them
A streaming table is parsed and restyled on every frame A table is one markdown node, so each character of its last row changes the text of the whole table components/markdown/streamingTable.ts: the row still being typed is held back until its line break arrives. components/MarkdownRenderer.tsx: a block whose prepared text did not change is not parsed again

Rows at the tail. useTranscriptRows hands the transcript to the virtualizer through useSlotDeferredValue. #31719 let one kind of change skip that deferral: a streaming flush that only grows the last row (grewOnlyAtTail). A new row stayed deferred. The sent prompt and the reply's first chunk are new rows, and the flushes that follow them each re-render the chat page at urgent priority, which restarts the background render those rows wait for. A device that cannot finish that render between two flushes never commits it, so the growing reply that #31719 keeps live was live inside a row nobody could see.

grewOnlyAtTail now holds for rows added after the last row as well. It still fails for anything that touches an earlier row, for a removed row, and for the first rows of an empty transcript, so a landed page of history renders in the background as before. This widens the predicate #31719 added. useSlotDeferredValue is unchanged, including the visit key that keeps a session switch from showing the session the user left (#8526): a switch still renders the incoming transcript in its first commit, and transcriptRows.deferral.test.ts holds that with the wider predicate.

Table rows. The header of a streaming table is already held until its delimiter row arrives (deferIncompleteStreamingTable, moved to markdown/streamingTable.ts unchanged). holdPartialTableRow does the same for the row being typed. The text handed to the parser then changes once per row, and ParsedMarkdown, a memoized wrapper around react-markdown, skips the parse when that text and the plugin switches are the same as last time. The reader sees a table grow one whole row at a time. On main the half-typed row first showed as plain text under the table and then jumped into it. The last characters of the newest row keep their reveal fade until the next row lands. Both holds apply only while the block's own text is still arriving: a table whose block a code fence or a widget closed is drawn with every row, and a lone header line in such a block is drawn as the text it is.

Backwards compatibility

Compatible: grewOnlyAtTail has one caller, and the cases it newly accepts were rendered a moment later through the deferral; deferIncompleteStreamingTable was module-private and keeps its behaviour in its new file.

Removes nothing: no export, prop, setting or deferral path is removed.

Tests

  • pages/chat/page/transcriptRows.test.ts, grewOnlyAtTail: holds for a row added after the last row, for a sent prompt, the first chunk and a tool row through the real turn grouper judged against the transcript before the send, and for a last row that changed with rows added after it. Fails for a page landed in front, an earlier row replaced while a row was added, a removed row and the first rows of an empty transcript.
  • pages/chat/page/transcriptRows.deferral.test.ts, useTranscriptRows with useDeferredValue held at its first frame, the way React holds it while every background render is interrupted: the prompt, the reply and a tool row are in the render that brings them, and the deferral gets one input for the whole turn; a landed page stays deferred without rolling the reply back; the first rows of an empty transcript stay deferred; a session switch shows the incoming session at once.
  • components/markdown/streamingTable.test.ts: holdPartialTableRow returns one text for every character of a row and one text per completed row, and leaves alone a line that is not under a table with its delimiter row.
  • test/MarkdownRenderer.streamingTableRows.test.tsx, through the real react-markdown with its renders counted: the row being typed is not drawn and costs no parse, the completed row costs one, prose still parses on every change, and every row is drawn once the stream ends. A table whose block a code fence or a widget closed keeps its last row while that fence or widget streams, and a header line closed the same way is drawn.
  • Red on main: the new and changed cases run against origin/main fail 9 of 24 (3 in transcriptRows.test.ts, 2 in transcriptRows.deferral.test.ts, 4 in MarkdownRenderer.streamingTableRows.test.tsx). The two closed-table cases pass on main, which has no row hold; they fail on this branch with the typing gate taken out.
  • Determinism: the four files ran 5 times in shuffled order (seeds 11, 23, 37, 41, 59), 33 of 33 passing each run. No timer, clock or sleep in any of them.
  • 758 website test files that name the markdown renderer, the chat page or the transcript pass, with tsc -p tsconfig.app.json, eslint src/ and the repository's semgrep rules on the diff.

Proof: not provable -- frontend test (prove.py runs Python only)

Manual verification

Gateway-free rig: the production build behind scripts/lib/serve-dist.mjs, /api/** from stub-dashboard-api, the websocket mocked. One reply (8 sections of prose, a list, a 40-line code block and a 12-row table) streams at 540 characters per second into an open 100-row chat, 1440x900. Main and this branch alternate in one script with main run twice per round (A/A). A run starts only under host load 13. Before any number was read, the DOM was checked for the prompt, the reply's first words and the table rows.

Scenario main this branch
CPU 4x slower, 45 s reply: prompt on screen never, 6 of 6 runs 0.30 to 0.34 s, 3 of 3 runs
CPU 4x slower, 45 s reply: reply on screen never, 6 of 6 runs 0.49 to 0.53 s, 3 of 3 runs
Tables only, 17 s, no throttle: main-thread task time 14.9 to 16.9 s (8 runs) 8.5 to 10.2 s (4 runs)
Tables only: style recalculation 2.8 to 2.9 s 0.58 to 0.69 s
Tables only: layouts (median) 960 222
Mixed reply, 45 s, no throttle: style recalculation 2.8 to 3.1 s (6 runs) 1.7 to 1.8 s (3 runs)

Tables only, task time per pair: -5.2, -6.1, -6.4, -6.3 s; the same build run twice differs by 0.1 s. Mixed reply, style recalculation per pair: -1.1, -1.3, -1.3 s; the same build differs by 0.2 s.

Other chat surfaces. The row change lives in useTranscriptRows, which only the chat page calls, so it reaches /chat, /embed/chat and the chat panel in apps. A crewmate DM (components/ChatPane) and the app-sdk ChatEmbed draw through ChatMessageList, which has no deferral. The table change is in MarkdownRenderer, which every surface uses. Measured on main fdf7cbc against head d17eb65, alternating, 3 runs each, every run started under host load 13, the DOM checked for the prompt and the reply first:

Surface, 20 s reply at CPU 4x slower main: prompt / reply on screen this branch: prompt / reply on screen
/embed/chat never / never 0.23 to 0.30 s / 0.42 to 0.52 s
Crewmate DM 0.19 to 0.21 s / 0.47 to 0.53 s 0.20 to 0.27 s / 0.27 to 0.54 s
ChatEmbed 0.26 to 0.28 s / 0.26 to 0.28 s 0.26 to 0.31 s / 0.26 to 0.31 s
Tables only, 17 s, no throttle main this branch
Crewmate DM: main-thread task time 14.0 to 15.9 s 6.8 to 7.2 s
Crewmate DM: style recalculation 3.3 to 3.8 s 0.75 to 0.80 s
Crewmate DM: layouts 1183 to 1247 329 to 335
ChatEmbed: main-thread task time 4.1 to 5.0 s 4.3 to 4.9 s

The DM and ChatEmbed never had the missing reply, and this branch leaves their timing as it was. The DM gets the table saving. ChatEmbed holds the half-typed row too (97 body rows drawn against 98 at the same moment), but its cost did not move: it reads its slot about four times a second instead of taking a socket frame per chunk, and in this rig its transcript changed about 12 times a second on both builds (178 to 217 layouts in 17 s). The ChatEmbed runs used a scratch entry whose slot reads were answered by the rig, not a real gateway.

Not shown by these runs:

  • Smooth frames on the 4x-throttled device. The reply is on screen, at about 10 frames per second; the main thread is still full.
  • A task-time gain on the mixed reply. This branch ran 31.7 to 33.3 s against 34.9 to 41.1 s on main, but main run twice differs by 3.2 s, so the size is not established.
  • A frame-interval change without throttle: p95 is 16.7 to 16.8 ms on both.
  • The first send into an empty chat at 4x: 5.5 to 9.9 s on main (4 runs), 5.4 to 5.6 s on this branch (2 runs).

Host: a shared 32-core Xeon; the throttle is Chromium's CPU throttling, not a real slow device.

Screenshots / video

Both at 4x CPU throttle, 12 s into the same reply. Main still shows the chat as it was before the send; this branch shows the prompt and the reply.

Before (main):

main at 4x CPU throttle: the transcript shows the chat as it was before the send

After (this branch):

this branch at 4x CPU throttle: the prompt and the streaming reply are on screen

Side by side, with a timer from the send:

before and after at 4x CPU throttle, side by side with a timer

Related Issues

Part of #31704

Closes #37358

Builds on #31719.

Sibling sweep

Swept: git grep -n "useDeferredValue(" -- website/src
Sibling: website/src/components/PromptLengthNotice.tsx:44 -- not affected -- it defers the composer's own typed text, which no streaming flush restarts

Pattern harvest

Rule candidate: review-prompt
Pattern: a useDeferredValue whose owner re-renders at urgent priority on every frame never commits on a slow device, so content that must stay visible cannot wait behind it.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@buluoray
buluoray requested review from a team and KiroCrewCaptain as code owners November 14, 2026 23:47
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 14, 2026
@buluoray

Copy link
Copy Markdown
Contributor Author

Before / after at 4x CPU throttle, side by side, with a timer that starts at the send. Left is main: the prompt and the reply never reach the transcript. Right is this branch: the reply is on screen after 0.48 s.

before and after at 4x CPU throttle

recorded from main 5a740c9 and this branch 478a374601ebf9da6b6a6ba1994b5ef39d29c4f2 (the pushed head 2224cdb is the same change rebased) · production builds · mode: Chromium CPU throttle 4x, 1040x720, host load 8.4 and 10.1 at the start of each take · the real built dashboard and its real reducers and components; the gateway is a stub: /api/** from scripts/lib/stub-dashboard-api.mjs and a mocked websocket that sends the same chat_chunk frames to both sides, 540 characters per second

The GIF is on crew/kiro-crew-rust/pr-37414-assets (one commit, media only).

@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of d4edaa90bfca26d0edca7a9ecd3425a292271c3e — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

The deleted pin it('fails when a row was added') is a reversal that the description states openly. It is backed by a 6-of-6 reproduction on main and by tests that fail on main, so nothing blocks.

What this change ships

Inventory (5 items) — 5 justified

Intent: on a slow device, a streaming reply shows up as it arrives, and a streaming table costs work per row instead of per character. This is a FIX.

  1. The sent prompt and the reply's first chunk render in the commit that brings them, instead of waiting for the deferred render (grewOnlyAtTail now also accepts rows added at the tail) — justified
  2. A streaming table grows one whole row at a time, and the half-typed row is hidden until its line break arrives (holdPartialTableRow) — justified
  3. A block whose prepared text did not change is not parsed again (ParsedMarkdown memo) — justified
  4. Once a fence or widget closes a block, its table and a lone header line are drawn in full; both holds are now gated on typing — justified
  5. deferIncompleteStreamingTable moved unchanged to markdown/streamingTable.ts — justified

[FIRST-PRINCIPLES-REVIEWED] d4edaa9

Description read

[DESCRIPTION-READ] cd55eddeb6116cfa14cd47b6a7a99ca80de788a90f8e1c9c999fc2718ad323e5

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of d4edaa90bfca26d0edca7a9ecd3425a292271c3e — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: both changes go after the causes they name, keep the deferral path, and can be reverted cleanly.

Notes

  • grewOnlyAtTail now accepts any number of rows added at the tail. A burst that lands many rows after the last one at once (a reconnect replay, a resumed run catching up) will now render at urgent priority instead of in the background. Capping how many added rows qualify would bound the cost of that commit.

[DESIGN-REVIEWED] d4edaa9

Description read

[DESCRIPTION-READ] bcbe54d2db503a747ab30b705e8b4a07e4513c15c2271661cd509cd7af8f4a0a

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ✅ PASS

UX-level review of d4edaa90bfca26d0edca7a9ecd3425a292271c3e — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

UX-Verdict: PASS

Nothing blocks: the PR adds no controls. Its two rendering changes (new rows show up right away, and a table now grows one whole row at a time) appear in shot-02 and the before/after recording.

Notes

  • holdPartialTableRow keeps back a row until its line break arrives. If a model streams a long-celled row slowly, nothing under the table moves until that row is done. Check that the streaming cursor still sits somewhere the user can see during that pause, so the reply doesn't look stalled.

[UX-REVIEWED] d4edaa9

Description read

[DESCRIPTION-READ] 2697250c292e18be731ca54325f76c97d7e6e1b37816ad4967297ea572ec6925

That sha256 names everything this verdict read: the PR description, plus the 6 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed d4edaa90bfca26d0edca7a9ecd3425a292271c3e — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] d4edaa9

Verdict parsed from the review's SHA-scoped output markers for commit d4edaa90bfca26d0edca7a9ecd3425a292271c3e.

False positive or not applicable? A repository writer can comment:
/ai-review override fable d4edaa90bfca26d0edca7a9ecd3425a292271c3e false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of d4edaa90bfca26d0edca7a9ecd3425a292271c3e and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] d4edaa9

False positive or not applicable? A repository writer can comment:
/ai-review override gpt d4edaa90bfca26d0edca7a9ecd3425a292271c3e false-positive: <reason, 20 to 500 characters>

@buluoray
buluoray force-pushed the crew/kiro-crew-rust/stream-reply-visible branch from 2224cdb to d17eb65 Compare November 15, 2026 00:19
@buluoray

Copy link
Copy Markdown
Contributor Author
  • span=e072207effd8 A completed table row disappears when a following code block starts

fixed in d17eb65: the finding holds. parseBlocks closes the markdown block at the fence with no trailing line break, the block is still the last markdown block so it kept glow, and holdPartialTableRow cut its finished last row for as long as the fence streamed.
The hold is gated on glow && typing (MarkdownRenderer.tsx, MarkdownBlock). typing is false for a block the assembler closed, so its text is drawn whole.
Class: a hold that waits for more text, applied to a block that gets none. deferIncompleteStreamingTable sits on the same line and has the same shape, so it takes the same gate: a lone header line in a closed block is drawn as text.
Tests in test/MarkdownRenderer.streamingTableRows.test.tsx: the last row stays drawn while a code fence streams after the table, the same with a widget, and a header line closed by a fence is drawn. All three fail with the gate taken out.

The repair was made in the PR's own session, not by a model-pinned subagent: this host reaps subagents mid-run. 386 test files that name the markdown renderer pass on the fix, with tsc -p tsconfig.app.json and eslint.

@buluoray

Copy link
Copy Markdown
Contributor Author
  • span=45e0c12ab71c A table row that has finished streaming stays hidden when a code fence or widget follows the table

fixed in d17eb65: the finding holds, and the fix is the one proposed. The hold is gated on glow && typing, so a markdown block the assembler closed at a fence or a widget is drawn with every row.
deferIncompleteStreamingTable waits for text in the same way and takes the same gate.
Tests in test/MarkdownRenderer.streamingTableRows.test.tsx cover a fence after the table, a widget after the table, and a header line closed by a fence. All three fail with the gate taken out.

The repair was made in the PR's own session, not by a model-pinned subagent: this host reaps subagents mid-run.

@buluoray

Copy link
Copy Markdown
Contributor Author

Backend Tests (3.12, 8) was red on head 2224cdb with test/test_history.py::TestAMiscomposedHostPersistsAWholeTurn::test_both_rows_persist (AssertionError: please deploy AKIAIOSFODNN7EXAMPLE).

That red is main's, not this change's: the same job on main's own run at af94fac fails the same test (job 132632705173), and this diff touches no Python file. #37423 fixes the fixtures. This PR waits for it and rebases once it lands; no rerun and no override for that check.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 15, 2026
@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: A person should decide

Why: the decider estimates a 12% chance that this change introduces a defect, within the 12% limit for approving on its own, but it changes 594 lines, more than the 400 it may approve on its own.

Frontend-only change, and it does what it claims. (1) The wider grewOnlyAtTail still rejects every change that touches an earlier row: a page added in front, an edited or removed row, rows trimmed from the front, and the first rows of an empty chat. useSlotDeferredValue judges each value against the last frame it fed to the deferral, so the rows of a whole turn count as tail growth and a landed page still shows the live reply. The predicate only chooses urgent versus deferred rendering, so a wrong answer can only cost speed. The session-visit key is untouched. (2) glow is only ever set as isStreaming and typing requires streaming, so gating both table holds on typing changes behaviour only for a block a fence or widget has closed. That is intended and tested. The ParsedMarkdown memo depends on the text, the plugin switches and the redaction markers; the language generation and TextPreviewsCtx are read inside it, and the other gates reach the rendered elements through contexts outside the memo. Tests through the real block assembler and react-markdown cover the hold, the parse count, stream end and closed blocks. Frontend tests and the render benchmark pass. The failing jobs are b

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit d17eb65fa6 · 594 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

Rows added at the tail of the transcript (the sent prompt, the reply's
first chunk, a tool row) render in the commit that brings them. They
went through the transcript's deferred render, which every streaming
flush restarts, so on a device that cannot finish that render between
two flushes the prompt and the whole reply stayed off screen until the
stream went quiet. grewOnlyAtTail holds for added rows as well as for a
last row that grew in place; a change that reaches an earlier row, and
the first rows of an empty transcript, stay deferred.

A streaming table holds back the row still being typed until its line
break arrives, and a markdown block whose prepared text did not change
is not parsed again. A table is one markdown node, so each character
of its last row made react-markdown parse the whole table and the
browser restyle every cell, once per animation frame.

Part of #31704
Closes #37358
@buluoray
buluoray force-pushed the crew/kiro-crew-rust/stream-reply-visible branch from d17eb65 to d4edaa9 Compare November 15, 2026 02:35
@buluoray

Copy link
Copy Markdown
Contributor Author

Head d4edaa9 is d17eb65 rebased onto main at 2092903, which carries #37423 (the fix for the row-scrub tests that were red on main). No code change. The description gained an "Other chat surfaces" block under Manual verification.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 15, 2026
@KiroCrewCaptain
KiroCrewCaptain enabled auto-merge (squash) November 15, 2026 04:03

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: fix (7 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: frontend chat fix that keeps a streaming reply and its tables on screen on slow devices (tail-only rows render urgently, a half-typed table row is held until its line break, the parse is memoized); sanitize and redaction plugin order unchanged; covered by new tests.

@KiroCrewCaptain
KiroCrewCaptain merged commit 1a1a4e6 into main Nov 15, 2026
99 checks passed
@KiroCrewCaptain
KiroCrewCaptain deleted the crew/kiro-crew-rust/stream-reply-visible branch November 15, 2026 04:04
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A streaming table is parsed and restyled again on every frame

2 participants