Repository navigation
fix(chat): keep a streaming reply's text live on slow devices - #31719
Conversation
25f767e to
3804469
Compare
|
Rebased: main moved in website/src/hooks/ and website/src/pages/chat/page/ChatTranscriptView.tsx since this head went green (8dbdf04 -> 678769e); CI re-running. The rebase also picks up #31625 and #31803, which fix the two backend tests that were red on main at the old base ( [operator: bolichen97] |
Every streamed chunk made a new transcript, and the whole transcript went through the slot-scoped deferred value, so each chunk scheduled a background render of the chat page. On a slow phone the next chunk interrupted that render every time, so the reply's text almost never updated; on desktop it cost a second page render per chunk. A chunk that only grows the last row renders at urgent priority: useSlotDeferredValue takes an optional sameShape check, and useTranscriptRows passes grewOnlyAtTail. A landed page, a new row or a regroup is still deferred, and while one is pending the hook shows the last value it rendered live, never an older frame. The window while a change of shape is pending is tracked in #31704. Builds on #23487. Closes #23486 Co-authored-by: Pearcekieser <5055971+Pearcekieser@users.noreply.github.com>
3804469 to
78f1d0e
Compare
|
Rebased: main moved in website/src/hooks/ and website/src/pages/chat/page/ since this head went green (678769e -> 464b547); CI re-running. The diff is unchanged. The related frontend tests, The Render Benchmark red on the old head ( [operator: bolichen97] |
UX Review (Opus 5.5) — ✅ PASSUX-level review of UX-Verdict: PASS Nothing to check. The diff adds no control, label or state; it only changes when the streamed text renders. [UX-REVIEWED] 78f1d0e Description read[DESCRIPTION-READ] 80e7bfaa4fe5da4da116e04a4409a815c25ec3b33a8373430ad8304423fbd0c5 That sha256 names everything this verdict read: the PR description, plus the 3 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the root cause is that every chunk went through the deferral, and the fix takes in-place tail growth off that path while new rows, landed pages and regroups stay deferred, with no rollback to an older frame. Notes
[DESIGN-REVIEWED] 78f1d0e Description read[DESCRIPTION-READ] e7913957c986524683f3ea066f6cfa2a7592a6a4e4102acfefef1c141806cbca That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS The fix comes with an issue (#23486) and hook tests that fail on main. Its one wider piece, the Notes
What this change shipsInventory (4 items) — 3 justifiedIntent: on slow devices, a streaming reply's text keeps up as it arrives, without a second chat page render per chunk. This is a FIX.
[FIRST-PRINCIPLES-REVIEWED] 78f1d0e Description read[DESCRIPTION-READ] cce350973344d5f4ff866134421f05066394ec7a577aab24e53cc1534cb3b5e0 That sha256 names the PR description this verdict read. Recompute it with |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Approved for the on-call. Review lanes pass on 78f1d0e, no conflict, no open change request.
Every streamed chunk made a new transcript, and the whole transcript went through the slot-scoped deferred value, so each chunk scheduled a background render of the chat page. On a slow phone the next chunk interrupted that render every time, so the reply's text almost never updated; on desktop it cost a second page render per chunk. A chunk that only grows the last row renders at urgent priority: useSlotDeferredValue takes an optional sameShape check, and useTranscriptRows passes grewOnlyAtTail. A landed page, a new row or a regroup is still deferred, and while one is pending the hook shows the last value it rendered live, never an older frame. The window while a change of shape is pending is tracked in #31704. Builds on #23487. Closes #23486 Co-authored-by: Pearcekieser <5055971+Pearcekieser@users.noreply.github.com> (cherry picked from commit 9d37818)
Problem / Motivation
Goal: A streaming reply's text keeps up on slow devices, and each streamed chunk costs one chat page render instead of two.
On a slow phone a streaming reply's text barely moves. In a 30 s prose stream at 390x844 with 4x CPU throttle, the streaming bubble received new content 2 times while the page kept re-rendering. On desktop each chunk cost a second ChatPage render: 245 of 1048 ChatPage renders in the same stream at 1440x900.
Why it matters
Watching the reply arrive is the main thing a user does while a turn runs. On a phone the text froze for most of the reply even though the page was busy. On desktop about a quarter of ChatPage's renders during a stream were that second pass.
Not a goal
useSmoothStream), ChatPage's per-chunk render from itschat.messagessubscription, and the virtualizer's height-version renders.What changed (motivation → approach → change)
The reply's text now updates in the same commit as each chunk. The whole transcript went through one deferred value, so every chunk asked React for a background render of the chat page. On a slow device the next chunk interrupted that render every time, so the bubble kept its old text; on desktop the render did finish, which made it a second page render per chunk.
A chunk that only grows the reply's last row is cheap to draw, so it skips the deferral. Anything else stays deferred: a page of history landing, a new row, a regrouped turn. While one of those is pending, the transcript keeps showing the last text it drew live, never an older frame, so the reply cannot roll back to its first chunk.
website/src/hooks/useSlotDeferredValue.tssameShape(deferredInput, next): when it holds, the deferral keeps its earlier input and the current value renders now. Its two refs are written in a layout effect, so a render React throws away leaves nothing behindpages/chat/page/transcriptRows.tsuseTranscriptRowspassesgrewOnlyAtTail: same message and row counts, the last message keeps itstsand role, every earlier message and row is the same objectKnown limitation: while a change of shape is pending (the streaming row first appearing, a tool row mid-reply), later tail chunks show the last live text until that background render commits, as every chunk did before. #31704 tracks it.
Backwards compatibility
Compatible:
sameShapeis optional anduseTranscriptRowsis the hook's only caller; without it the hook returns exactly what it returned before.Removes nothing: no export, prop or deferral path is removed.
Tests
useSlotDeferredValue.test.ts,sameShapeblock, with the test's controllableuseDeferredValueheld back the way React holds it under urgent churn:transcriptRows.test.ts,grewOnlyAtTail: holds for an in-place tail replacement and for a streaming flush through the real turn grouper and running state of a two-turn transcript; fails for an added row, a different or role-changed last row, an edited earlier message or row, and an empty transcript.origin/main'suseSlotDeferredValue.tsfail 3 of 9.Proof: not provable -- frontend test (prove.py runs Python only)
Manual verification
The render benchmark replays the same frames each run: a 30 s prose stream into the open chat, gateway-free, 3 runs, median, React profiling build. These numbers were measured on #23487's head; this PR keeps its behaviour (the hook drops one field that never changed the result).
On mobile, ChatPage renders rise because the text reaches the page: row heights change with it, and the virtualizer's height-version renders go up. On this head
npm --prefix website run bench:render -- --fail-on dispatches,selectorCallspasses every scenario.Screenshots / video
Why no screenshot: the transcript draws the same rows; the streamed text reaches them sooner, which a still image cannot show.
Related Issues
Closes #23486
Builds on #23487 by @Pearcekieser.
Follow-up: #31704
Sibling sweep
Swept: git grep -n "useDeferredValue" -- website/src
Sibling: website/src/components/PromptLengthNotice.tsx:44 -- not affected -- it defers the composer's own typed text, which is the intended typing-path deferral, not a streaming transcript
Pattern harvest
Not generalizable: the transcript deferral has one caller, and the shape check is specific to how a streaming transcript grows.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)