Repository navigation
fix(chat): keep a streaming reply and its tables drawn on slow devices - #37414
Conversation
|
Before / after at 4x CPU throttle, side by side, with a timer that starts at the send. Left is main: the prompt and the reply never reach the transcript. Right is this branch: the reply is on screen after 0.48 s. recorded from main 5a740c9 and this branch 478a374601ebf9da6b6a6ba1994b5ef39d29c4f2 (the pushed head 2224cdb is the same change rebased) · production builds · mode: Chromium CPU throttle 4x, 1040x720, host load 8.4 and 10.1 at the start of each take · the real built dashboard and its real reducers and components; the gateway is a stub: The GIF is on |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS The deleted pin What this change shipsInventory (5 items) — 5 justifiedIntent: on a slow device, a streaming reply shows up as it arrives, and a streaming table costs work per row instead of per character. This is a FIX.
[FIRST-PRINCIPLES-REVIEWED] d4edaa9 Description read[DESCRIPTION-READ] cd55eddeb6116cfa14cd47b6a7a99ca80de788a90f8e1c9c999fc2718ad323e5 That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: both changes go after the causes they name, keep the deferral path, and can be reverted cleanly. Notes
[DESIGN-REVIEWED] d4edaa9 Description read[DESCRIPTION-READ] bcbe54d2db503a747ab30b705e8b4a07e4513c15c2271661cd509cd7af8f4a0a That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
UX Review (Opus 5.5) — ✅ PASSUX-level review of UX-Verdict: PASS Nothing blocks: the PR adds no controls. Its two rendering changes (new rows show up right away, and a table now grows one whole row at a time) appear in shot-02 and the before/after recording. Notes
[UX-REVIEWED] d4edaa9 Description read[DESCRIPTION-READ] 2697250c292e18be731ca54325f76c97d7e6e1b37816ad4967297ea572ec6925 That sha256 names everything this verdict read: the PR description, plus the 6 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
2224cdb to
d17eb65
Compare
The repair was made in the PR's own session, not by a model-pinned subagent: this host reaps subagents mid-run. 386 test files that name the markdown renderer pass on the fix, with |
The repair was made in the PR's own session, not by a model-pinned subagent: this host reaps subagents mid-run. |
|
That red is main's, not this change's: the same job on main's own run at af94fac fails the same test (job 132632705173), and this diff touches no Python file. #37423 fixes the fixtures. This PR waits for it and rebases once it lands; no rerun and no override for that check. |
Code change approver recommendation: A person should decideWhy: the decider estimates a 12% chance that this change introduces a defect, within the 12% limit for approving on its own, but it changes 594 lines, more than the 400 it may approve on its own. Frontend-only change, and it does what it claims. (1) The wider grewOnlyAtTail still rejects every change that touches an earlier row: a page added in front, an edited or removed row, rows trimmed from the front, and the first rows of an empty chat. useSlotDeferredValue judges each value against the last frame it fed to the deferral, so the rows of a whole turn count as tail growth and a landed page still shows the live reply. The predicate only chooses urgent versus deferred rendering, so a wrong answer can only cost speed. The session-visit key is untouched. (2) glow is only ever set as isStreaming and typing requires streaming, so gating both table holds on An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
Rows added at the tail of the transcript (the sent prompt, the reply's first chunk, a tool row) render in the commit that brings them. They went through the transcript's deferred render, which every streaming flush restarts, so on a device that cannot finish that render between two flushes the prompt and the whole reply stayed off screen until the stream went quiet. grewOnlyAtTail holds for added rows as well as for a last row that grew in place; a change that reaches an earlier row, and the first rows of an empty transcript, stay deferred. A streaming table holds back the row still being typed until its line break arrives, and a markdown block whose prepared text did not change is not parsed again. A table is one markdown node, so each character of its last row made react-markdown parse the whole table and the browser restyle every cell, once per animation frame. Part of #31704 Closes #37358
d17eb65 to
d4edaa9
Compare
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Tier 1 auto-approve: fix (7 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: frontend chat fix that keeps a streaming reply and its tables on screen on slow devices (tail-only rows render urgently, a half-typed table row is held until its line break, the parse is memoized); sanitize and redaction plugin order unchanged; covered by new tests.

Problem / Motivation
Goal: A reply that streams on a slow device shows up as it arrives, and a streaming table costs work per row instead of per character.
On a slow device a reply can stream from start to end with nothing on screen. With the CPU throttled 4x, a 45 s reply into an open 100-row chat never reached the transcript, and neither did the prompt that asked for it: 6 of 6 runs on main. The store held every chunk and the stop button was lit, but the transcript still showed the chat as it was before the send.
While a table streams, the dashboard parses the whole table again and restyles every cell on each animation frame (#37358).
Why it matters
Sending a message and watching the answer arrive is the main thing the chat does. On a slow laptop or phone the user sends, sees nothing, and cannot tell whether the message went out. The table cost grows with the table, because each frame pays for every row drawn so far, and it spends main-thread time that the reply's own text needs on that same slow device.
Not a goal
What changed (motivation → approach → change)
Two changes, one per symptom.
pages/chat/page/transcriptRows.ts: rows added at the tail render in the commit that brings themcomponents/markdown/streamingTable.ts: the row still being typed is held back until its line break arrives.components/MarkdownRenderer.tsx: a block whose prepared text did not change is not parsed againRows at the tail.
useTranscriptRowshands the transcript to the virtualizer throughuseSlotDeferredValue. #31719 let one kind of change skip that deferral: a streaming flush that only grows the last row (grewOnlyAtTail). A new row stayed deferred. The sent prompt and the reply's first chunk are new rows, and the flushes that follow them each re-render the chat page at urgent priority, which restarts the background render those rows wait for. A device that cannot finish that render between two flushes never commits it, so the growing reply that #31719 keeps live was live inside a row nobody could see.grewOnlyAtTailnow holds for rows added after the last row as well. It still fails for anything that touches an earlier row, for a removed row, and for the first rows of an empty transcript, so a landed page of history renders in the background as before. This widens the predicate #31719 added.useSlotDeferredValueis unchanged, including the visit key that keeps a session switch from showing the session the user left (#8526): a switch still renders the incoming transcript in its first commit, andtranscriptRows.deferral.test.tsholds that with the wider predicate.Table rows. The header of a streaming table is already held until its delimiter row arrives (
deferIncompleteStreamingTable, moved tomarkdown/streamingTable.tsunchanged).holdPartialTableRowdoes the same for the row being typed. The text handed to the parser then changes once per row, andParsedMarkdown, a memoized wrapper aroundreact-markdown, skips the parse when that text and the plugin switches are the same as last time. The reader sees a table grow one whole row at a time. On main the half-typed row first showed as plain text under the table and then jumped into it. The last characters of the newest row keep their reveal fade until the next row lands. Both holds apply only while the block's own text is still arriving: a table whose block a code fence or a widget closed is drawn with every row, and a lone header line in such a block is drawn as the text it is.Backwards compatibility
Compatible:
grewOnlyAtTailhas one caller, and the cases it newly accepts were rendered a moment later through the deferral;deferIncompleteStreamingTablewas module-private and keeps its behaviour in its new file.Removes nothing: no export, prop, setting or deferral path is removed.
Tests
pages/chat/page/transcriptRows.test.ts,grewOnlyAtTail: holds for a row added after the last row, for a sent prompt, the first chunk and a tool row through the real turn grouper judged against the transcript before the send, and for a last row that changed with rows added after it. Fails for a page landed in front, an earlier row replaced while a row was added, a removed row and the first rows of an empty transcript.pages/chat/page/transcriptRows.deferral.test.ts,useTranscriptRowswithuseDeferredValueheld at its first frame, the way React holds it while every background render is interrupted: the prompt, the reply and a tool row are in the render that brings them, and the deferral gets one input for the whole turn; a landed page stays deferred without rolling the reply back; the first rows of an empty transcript stay deferred; a session switch shows the incoming session at once.components/markdown/streamingTable.test.ts:holdPartialTableRowreturns one text for every character of a row and one text per completed row, and leaves alone a line that is not under a table with its delimiter row.test/MarkdownRenderer.streamingTableRows.test.tsx, through the realreact-markdownwith its renders counted: the row being typed is not drawn and costs no parse, the completed row costs one, prose still parses on every change, and every row is drawn once the stream ends. A table whose block a code fence or a widget closed keeps its last row while that fence or widget streams, and a header line closed the same way is drawn.origin/mainfail 9 of 24 (3 intranscriptRows.test.ts, 2 intranscriptRows.deferral.test.ts, 4 inMarkdownRenderer.streamingTableRows.test.tsx). The two closed-table cases pass on main, which has no row hold; they fail on this branch with thetypinggate taken out.tsc -p tsconfig.app.json,eslint src/and the repository's semgrep rules on the diff.Proof: not provable -- frontend test (prove.py runs Python only)
Manual verification
Gateway-free rig: the production build behind
scripts/lib/serve-dist.mjs,/api/**fromstub-dashboard-api, the websocket mocked. One reply (8 sections of prose, a list, a 40-line code block and a 12-row table) streams at 540 characters per second into an open 100-row chat, 1440x900. Main and this branch alternate in one script with main run twice per round (A/A). A run starts only under host load 13. Before any number was read, the DOM was checked for the prompt, the reply's first words and the table rows.Tables only, task time per pair: -5.2, -6.1, -6.4, -6.3 s; the same build run twice differs by 0.1 s. Mixed reply, style recalculation per pair: -1.1, -1.3, -1.3 s; the same build differs by 0.2 s.
Other chat surfaces. The row change lives in
useTranscriptRows, which only the chat page calls, so it reaches/chat,/embed/chatand the chat panel in apps. A crewmate DM (components/ChatPane) and the app-sdkChatEmbeddraw throughChatMessageList, which has no deferral. The table change is inMarkdownRenderer, which every surface uses. Measured on main fdf7cbc against head d17eb65, alternating, 3 runs each, every run started under host load 13, the DOM checked for the prompt and the reply first:/embed/chatChatEmbedChatEmbed: main-thread task timeThe DM and
ChatEmbednever had the missing reply, and this branch leaves their timing as it was. The DM gets the table saving.ChatEmbedholds the half-typed row too (97 body rows drawn against 98 at the same moment), but its cost did not move: it reads its slot about four times a second instead of taking a socket frame per chunk, and in this rig its transcript changed about 12 times a second on both builds (178 to 217 layouts in 17 s). TheChatEmbedruns used a scratch entry whose slot reads were answered by the rig, not a real gateway.Not shown by these runs:
Host: a shared 32-core Xeon; the throttle is Chromium's CPU throttling, not a real slow device.
Screenshots / video
Both at 4x CPU throttle, 12 s into the same reply. Main still shows the chat as it was before the send; this branch shows the prompt and the reply.
Before (main):
After (this branch):
Side by side, with a timer from the send:
Related Issues
Part of #31704
Closes #37358
Builds on #31719.
Sibling sweep
Swept: git grep -n "useDeferredValue(" -- website/src
Sibling: website/src/components/PromptLengthNotice.tsx:44 -- not affected -- it defers the composer's own typed text, which no streaming flush restarts
Pattern harvest
Rule candidate: review-prompt
Pattern: a
useDeferredValuewhose owner re-renders at urgent priority on every frame never commits on a slow device, so content that must stay visible cannot wait behind it.Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)