Repository navigation
fix(chat): draw the first send into an empty chat at once - #37533
Conversation
|
Before / after at 4x CPU throttle, side by side, with a timer that starts at the send. The chat is empty and this is its first message. Left is main: the transcript stays blank and the prompt and the reply show after 5.37 s. Right is this branch: the prompt shows after 0.35 s and the reply after 0.59 s. The backend is simulated. Recorded from main 932e28e and the first version of this branch, ab65ac9. Three takes of main gave 5.4, 6.9 and 8.5 s; the GIF uses the 5.4 s one. |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. The fix works at the level of the cause: the first rows of an empty chat that has finished loading no longer wait on the starved background render. It reuses What this change shipsInventory (4 items) — 4 justifiedIntent: the first message a user sends into an empty chat, and the reply to it, appear right away on a slow device. FIX.
[FIRST-PRINCIPLES-REVIEWED] 177ae66 Description read[DESCRIPTION-READ] b9dcf72aa8b1880195cbc5cbe4af7f2e6623ea376faf7a0a5b3ed54cca4bc375 That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS No design-level concerns: the change fixes the cause (React starving the deferred render of an empty chat's first rows) by adding one narrow, reversible bypass rule. The tail-deferral contract and the session-switch guard are left alone. Notes
[DESIGN-REVIEWED] 177ae66 Description read[DESCRIPTION-READ] 19f0ab92a460d73f71d8fbe180c7306f9f08c9c792d778b31a77b30d2f0bf26b That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
UX Review (Opus 5.5) — ✅ PASSUX-level review of UX-Verdict: PASS Nothing blocks: no control, label or copy is added or changed. Only when the existing transcript rows render changes, and a before/after recording and stills of both states are attached. [UX-REVIEWED] 177ae66 Description read[DESCRIPTION-READ] e12a450487af25469076c6b7c8e977f3c144dc89df10eeeccec11805b750ca14 That sha256 names everything this verdict read: the PR description, plus the 6 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
Code change approver recommendation: ApproveWhy: the decider estimates a 12% chance that this change introduces a defect, within the 12% limit for approving on its own, and every check passes. The change is narrow.
The new The hook now records every value it renders live, not only the ones the deferral has caught up with. Where the tw An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
ab65ac9 to
aa358b6
Compare
The repair was made in the PR's own session, not by a model-pinned subagent: this host reaps subagents mid-run. |
|
On the earlier head aa358b6, #37545 fixed them on main and merged. This PR is rebased onto main at 2e56b3b as 177ae66, with the six changed files byte-identical to aa358b6. No override was used. |
|
Reply to the code change approver recommendation above, which read commit Both points were true of that commit and are fixed in aa358b6. Tests that fail on |
On a slow device the first prompt sent into an empty chat, and the reply to it, stayed off screen for five seconds or more when the reply began within a second of the send. The store held both and the composer had cleared, but the transcript was blank. The transcript reaches the virtualizer through useSlotDeferredValue. The first rows of an empty transcript are handed to the deferral, because for a chat that is still loading they are its page landing. The first send into an empty chat goes the same way, so the prompt waits for a background render. Each streaming flush re-renders the page at urgent priority and restarts that render. React commits it only when the lane expires, five seconds after the send. The transcript frame now carries whether the session's transcript has loaded, and useSlotDeferredValue takes a second rule, drawnAtOnce. It is judged against the value the deferral last committed and lets a value render in this commit while its deferred render is pending. isFirstTurnOfEmptyChat is that rule for the transcript: the committed transcript is loaded and empty, and the new one holds a single turn. grewOnlyAtTail is unchanged, so the deferral is handed the same frames as before and what it holds moves on with the first rows. A transcript that is still loading keeps its page deferred, so does a page of several turns that lands in an empty chat, and a session switch still renders the incoming session in its first commit. Part of #31704
aa358b6 to
177ae66
Compare
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Approving. On head 177ae66 the review ran the three touched test files: 39 of 39 pass. With main's useSlotDeferredValue.ts and transcriptRows.ts swapped in, 10 of 39 fail. Removing each guard (the loaded check, the second-turn loop, recording only caught-up values) turns tests red. Shuffled runs stayed green. The deferral still gets the same inputs as on main, grewOnlyAtTail is unchanged, and the session-switch visit check is still evaluated first.
Done well: drawnAtOnce only widens the live-render decision, and the new Set(inputs).size === 2 assertion proves the deferral no longer keeps the empty frame.
Optional:
- The timing table and recordings come from
ab65ac9c946, not this head. Please replace them as the body promises, and add the typing-latency check that #31704 asks for. !slotLoading && slotSwitchTarget !== activeSlotnow appears three times inChatPage.tsx(lines 908, 954, 1213). One sharedconstwould keep them from drifting apart.

Problem / Motivation
Goal: The first message a user sends into an empty chat, and the reply to it, show up as they arrive on a slow device.
On a slow device the first send into an empty chat leaves the transcript blank. With the CPU throttled 4x, the prompt reached the screen 3.6 to 8.6 s after Enter in 11 runs on main, and not within the 12 s reply in a twelfth. The reply waits with it. The store holds the prompt about 0.1 s after Enter, the composer has cleared and the stop button is lit, so the user looks at an empty chat that is working.
Those runs use a simulated backend whose reply starts as soon as the request returns, under 1 s after Enter. A real model starts later; Manual verification says what main does then.
#37414 made rows added at the tail of a transcript render at once, and named this case as the one it left.
Why it matters
The first send is the first thing every new chat does. On a slow laptop or phone the user sends, the welcome screen goes away, and nothing replaces it for five seconds or more. They cannot tell whether the message went out.
Not a goal
POST /api/chat.What changed (motivation → approach → change)
Symptom. Blank transcript for 5 s or more after the first send, at 4x CPU throttle, when the reply starts within a second of the send.
Where the time goes. Profiled on main with the React profiling build and the scheduler's own hooks (render started, yielded, committed, each with its lanes), the store subscribed, and the DOM watched. One run on main:
POST /api/chatleavesBetween the send and the prompt reaching the screen React made 264 urgent commits. It started the background render that holds the prompt 19 times and threw it away 18 times. The one that committed, at 5.2 s, ran without yielding: its lane had expired. So the wait is not the network, the request, or the session being set up. It is React's five-second limit on a starved background render. A second run gave 6038 ms with 303 urgent commits and 24 thrown-away renders.
With no reply at all the same build shows the prompt at 0.86 s, after 6 thrown-away renders. With the reply held back 1.5 s the prompt shows at 0.87 s and the reply 0.15 s after its first chunk. So the long wait needs the reply to start before that first background render commits.
Cause.
useTranscriptRowshands the transcript to the virtualizer throughuseSlotDeferredValue. The first rows of an empty transcript go to the deferral, because for a chat that is still loading they are its page landing, which is what the deferral exists for. The first send into an empty chat has the same shape, so it waits too. Every streaming flush then re-renders the chat page at urgent priority and restarts the background render the prompt waits for.Change. Two parts.
useSlotDeferredValuetakes a second rule,drawnAtOnce(committed, next). It is judged against the value the deferral last committed, which is what the deferred path would otherwise keep on screen. When it holds, the value renders in this commit while its deferred render is still pending. It does not hold the deferral's input back, so the deferral is handed the value as before.isFirstTurnOfEmptyChatis that rule for the transcript. It holds when the committed transcript is loaded and empty and the new one holds a single turn. The transcript frame carriesloadedfor this (!slotLoading && slotSwitchTarget !== activeSlot, the test the page already uses for its busy-turn controls).grewOnlyAtTailis the same as on main, so the deferral is handed the same frames as on main and what it holds moves on with the first rows. The first version of this PR changedgrewOnlyAtTailinstead; the deferral then kept the empty frame for the whole visit, which the Opus review blocked. That change is gone.The visit key that keeps a session switch from showing the session the user left (the
#8526note in the hook) is unchanged. A switch still renders the incoming transcript in its first commit, anddrawnAtOnceis only consulted when the committed value belongs to the current visit.While the first turn's deferred render is pending, a value that does not pass the rule (a second turn starting) keeps the last value drawn on screen, not the empty chat: the hook now records every value it renders live, not only the ones the deferral has caught up with.
One thing a maintainer should know, about main and not changed here: in a new chat the frame the deferral holds after the first send is the one-row prompt.
grewOnlyAtTailchecks the rows before the held frame's last row, and a one-row frame has none, so on main a page landing in front or an edit of an earlier row in such a chat already counts as "tail only" (checked on main'sgrewOnlyAtTail: both return true). This PR leaves that as it is.Backwards compatibility
Compatible:
useSlotDeferredValuegains an optional fourth argument and has one caller.grewOnlyAtTailis unchanged.useTranscriptRowsgains a requiredtranscriptLoadedoption and has one caller,ChatPage, which passes it.Removes nothing: no export, prop, setting or deferral path is removed.
Tests
hooks/useSlotDeferredValue.test.ts,drawnAtOnce, withuseDeferredValueheld at its first frame: a value that passes renders at once and is still handed to the deferral; a later value that does not pass keeps the last value drawn at once, not the committed one; the rule is judged against the committed value, not the pending one.pages/chat/page/transcriptRows.test.ts,isFirstTurnOfEmptyChat: holds for the first send, its first chunk and a tool row, and when the running turn takes a steer or a parked send; fails for rows that hold a second turn, while the empty transcript is still loading, when rows are already on screen, and when there is nothing to draw. ThegrewOnlyAtTailcases are main's, unchanged.pages/chat/page/transcriptRows.deferral.test.ts,useTranscriptRowswith the deferral held the same way: the first send and its reply are in the render that brings them, and the deferral is handed the first rows once (two inputs for the turn: the empty chat and the prompt); a chat that loads empty and then gets its first send draws it at once; a page stays deferred when it lands in a chat still loading, when it holds several turns and lands in a loaded empty chat, and when it lands after an empty chat started loading again; a second turn that starts before the first rows committed keeps the first turn on screen and shows once they commit; the first send of a new chat opened from another session never shows that session's rows.origin/main's two source files the new cases fail 10 of 39. Against this PR's first version they also fail 10 of 39, among them the page of several turns landing in a loaded empty chat and the count of inputs handed to the deferral.loadedtest fails 3; dropping the second-turn test fails 3.tsc -p tsconfig.app.jsonandeslint src/ --max-warnings 0.Proof: not provable -- frontend test (prove.py runs Python only)
Manual verification
What these numbers are. The backend is simulated:
/api/**is answered bystub-dashboard-apiand the websocket is mocked. In the table below the reply starts as soon asPOST /api/chatreturns, under 1 s after Enter, where a real model takes longer. The numbers are the time the dashboard takes to draw, not model latency. A reply that starts later is covered under the table.Gateway-free rig, the one #37414 used: the production build behind
scripts/lib/serve-dist.mjs. An empty pinned chat is open, the prompt is typed in the composer and sent with Enter, and a reply streams at 540 characters per second, 1440x900. Main and the branch alternate in one script with main run twice per round (A/A), 6 rounds. Every run started under host load 13. Before any number was read, the DOM was checked for the prompt and the reply's first words in a transcript row.The table was measured on this PR's first version,
ab65ac9c946. The current head draws the first turn in the same commit as that version did and differs in what it hands the deferral. The current head is being measured again, together with replies that start 1.5 s and 3 s after the send, and this section will be replaced with those runs. So far the current head has one run of the profiling build at 4x: the prompt and the reply were on screen (DOM checked) after 7 urgent commits and no background commit; the host was loaded, so no time is quoted from it.Per pair at 4x, prompt on screen: -5.4, -5.3, -5.3, -3.1, -5.3 s (in the other pair main never showed it). Without throttle the median pair is -20 ms for the prompt and +10 ms for the reply; main run twice differs by 5 ms and 1 ms.
A reply that starts later. On the profiling build at 4x, one run each on main: with no reply the prompt shows at 0.86 s; with the reply held back 1.5 s the prompt shows at 0.87 s and the reply 0.15 s after its first chunk. So with a model that takes more than about a second to start, main already shows the prompt in under a second, and what this PR saves is the part of that second the prompt spends waiting. The paired runs for 1.5 s and 3 s will put a number on it.
An open 100-row chat, 3 rounds, was unchanged on the first version: at 4x the prompt shows after 0.28 to 0.33 s on main and 0.29 to 0.30 s on the branch; without throttle 104 to 123 ms and 117 to 133 ms.
Not shown by these runs:
POSTleaves later. At 4x it left 336 to 603 ms after Enter on main and 431 to 616 ms on the first version. Pairs differ by -133 to +261 ms and main run twice differs as much, so no change is established. Without throttle the median pair is +3 ms against +4 ms for main run twice.Host: a shared 32-core Xeon; the throttle is Chromium's CPU throttling, not a real slow device.
Screenshots / video
Recorded against the simulated backend, on this PR's first version
ab65ac9c946: the reply starts under 1 s after the send, so the timer shows the time the dashboard takes to draw, not model latency.Both at 4x CPU throttle, 3 s after the first send into an empty chat. Main shows a blank transcript with the stop button lit; this branch shows the prompt and the reply.
Before (main):
After (this branch):
Side by side, with a timer from the send. Three takes of main gave 5.4, 6.9 and 8.5 s; this is the 5.4 s one.
Related Issues
Part of #31704
Builds on #37414 and #31719.
Sibling sweep
Swept: git grep -n "useDeferredValue(" -- website/src
Sibling: website/src/components/PromptLengthNotice.tsx:44 -- not affected -- it defers the composer's own typed text, which no streaming flush restarts
Pattern harvest
Rule candidate: review-prompt
Pattern: a rule that defers "the first content of an empty list" also catches the first item the user adds themselves, so it needs to know whether the list is still loading.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.