Repository navigation
perf(chat): parse a streaming table once per row, not per character - #37395
jeeshofone wants to merge 1 commit into
Conversation
Hold back a streaming table's half-typed last row until its line break arrives, and reuse the parsed element while a block's prepared text is unchanged, so react-markdown runs once per row. Fixes kirodotdev#37358
|
Benchmark used for the before/after numbers in the description. Copy to import { it } from 'vitest'
import { render } from '@testing-library/react'
import MarkdownRenderer from '../components/MarkdownRenderer'
// Copy to website/src/test/ and run with vitest. 8 tables, 12 rows x 4 cols,
// streamed 9 chars per frame (540 chars/s at 60 fps, as in #37358).
function reply(): string {
const out: string[] = []
for (let t = 0; t < 8; t++) {
out.push(`| Name ${t} | Kind | Size | Note |`, '| --- | --- | --- | --- |')
for (let r = 0; r < 12; r++) out.push(`| row ${r} | item-${t}-${r} | ${r * 37} KB | ok ${r} |`)
out.push('')
}
return out.join('\n')
}
it('bench #37358', () => {
const text = reply()
const runs: number[] = []
for (let k = 0; k < 3; k++) {
let frames = 0
const t0 = performance.now()
const { rerender, unmount } = render(<MarkdownRenderer content="" streaming glow />)
for (let i = 9; i <= text.length; i += 9) {
rerender(<MarkdownRenderer content={text.slice(0, i)} streaming glow />)
frames++
}
runs.push(performance.now() - t0)
unmount()
console.log(`BENCH run ${k}: ${frames} frames, ${runs[k].toFixed(0)} ms`)
}
runs.sort((a, b) => a - b)
console.log(`BENCH median ${runs[1].toFixed(0)} ms, chars ${text.length}`)
}, 600000) |
Design Review (Opus 5.5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS No design-level concerns: holding back the half-typed last row and memoizing on the prepared text fixes the per-character re-parse, and both release paths keep a finished row from staying hidden. Notes
[DESIGN-REVIEWED] 936c5ec Description read[DESCRIPTION-READ] ecdde7eb951c865aa7c5f9e8b7c82c40133d1007479431d437932781ef471c00 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
UX Review (Opus 5.5, fork) — ✅ PASSUX-level review of UX-Verdict: PASS Nothing to check: no control, label or string is added or changed. The one visible change is that a streaming table's last row now appears whole when its line ends, instead of character by character. [UX-REVIEWED] 936c5ec Description read[DESCRIPTION-READ] 5cb2cf354d82e0fae45e4941f8a98f809c7951e8d8f61ca7ab65c316ff003463 That sha256 names everything this verdict read: the PR description, plus the 3 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5, fork) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. Issue #37358 measures the cost being removed, and the existing streaming-table deferral is extended rather than copied. What this change shipsInventory (4 items) — 4 justifiedIntent: keep the dashboard responsive while a reply streams tables, by parsing a table once per row instead of once per character. This is a FIX.
[FIRST-PRINCIPLES-REVIEWED] 936c5ec Description read[DESCRIPTION-READ] 15b6398a24be8078b2d490eb2bd1b37ba8ba24178e633f238be6f4f910d2cc28 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review (fork) — 🔴 changes requested (blocking)Reviewed The memoized markdown element ignores the text-pattern generation, so app text-preview spans no longer render in chat messages. BLOCKING — website/src/components/MarkdownRenderer.tsx:96 |
GPT 6.1 Review (fork) — 🔴 changes requested (blocking)Reviewed Adjudication upheld 1 of 1 blocking finding(s); those must be resolved before merging. BLOCKING -- website/src/components/MarkdownRenderer.tsx:1059 -- Memoization prevents plain-text app previews from appearing
[BLOCK-MERGE] 936c5ec Adjudication (Opus 5.5) — is blocking on each finding proportionate?F1 is real and should keep blocking (harm-warrants-remedy). The new memo breaks plain-text app previews on the normal path, and the fix is one dependency. F1, MEDIUM:
[ADJUDICATION] 936c5ec total=1 uphold=1 downgrade=0 |
Code change approver recommendation: Request changesWhy: the decider found 1 problem and estimates an 85% chance that this change introduces a defect, at or above the 80% level where it asks for changes. Holding back the half-typed last row of a streaming table is reasonable and tested. However, the new useMemo around ReactMarkdown leaves the TextPreviewsCtx generation out of its deps, even though the code relies on that context changing to re-parse blocks once the async text-pattern worker answers. On this head, plain-text link-preview spans would never render, and existing textPatterns tests appear to fail. Request changes. Problems found:
An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
|
This PR and #37414 both close #37358 with the same fix in |
Problem / Motivation
Goal: A streaming table costs work per row, not per character.
While a reply streams a table, the dashboard parses the whole table again and restyles every cell on every frame. A table is one markdown block, so each character typed into its last row changes the block's text, and
MarkdownBlockhands the whole table toreact-markdownagain. The cost grows with the table: every frame pays for all rows drawn so far.Why it matters
Replies made of tables are common, and the reporter measured 11.4 to 12.9 s of main-thread time per 17 s of streamed tables. That time slows the whole dashboard while a reply streams.
Not a goal
useBlockAssembler.What changed (motivation → approach → change)
Symptom: a full parse and restyle of the table on each frame. Cause: the block's text changes on each character, and nothing reuses the parse when the parsed text is the same.
deferIncompleteStreamingTablealready holds back a table header until its delimiter row arrives. It now also holds back the half-typed last row of a real table (a|line after the delimiter with no line break yet), but only while that row is being typed (holdRow).holdRowturns off when the block stops being the text being typed (a fence opened right after the table) or when the text has not changed forREVEAL_IDLE_SETTLE_MS(500 ms, a tool call or an approval wait mid-reply), so a finished row never stays hidden.MarkdownBlockkeeps thereact-markdownelement inuseMemo, keyed on the prepared text and every input of the plugin lists, plus the language generation. While a row is held, the prepared text is the same on each frame, so React reuses the element and skips the parse.Backwards compatibility
Compatible: the final render of every message is unchanged; only a streaming table's half-typed last row appears when its line break arrives instead of character by character.
Removes nothing: the change only defers when one row is drawn during streaming and reuses an identical render.
Tests
website/src/test/MarkdownRenderer.streamingTableRows.test.tsx:react-markdownrender; its line break causes exactly one.Proof: not provable -- frontend test (prove.py runs Python only). The first two tests fail on main, and the pause and fence tests fail on the commit without the
holdRowrelease.Manual verification
Benchmark (vitest + jsdom, same machine, 3 runs each, median): a reply of 8 tables, 12 rows of 4 columns, 4,039 characters, streamed 9 characters per frame (540 characters/s at 60 fps, as in the issue), 448 frames.
That is 73% less render time. The benchmark script is in a PR comment so it can be re-run.
Screenshots / video
Why no screenshot: the finished render is identical (a test pins it), and the only streaming difference is that a table row appears when its line ends instead of character by character.
Related Issues
Fixes #37358
Sibling sweep
Swept: git grep -n "deferIncompleteStreamingTable|<ReactMarkdown" -- website/src/components/
Sibling: website/src/components/MarkdownRenderer.tsx -- changed here
Siblings: none -- MarkdownBlock is the only streaming markdown render; other ReactMarkdown sites render final text once.
Pattern harvest
Not generalizable: the cost comes from this block's per-frame text growth; other streaming surfaces already hold incomplete constructs.
Review dispositions
holdRowwith tests; a verifier confirmed it closed on the current head.markersgets a new identity each frame, so such messages keep main's cost. Output is correct; left as is.Checklist
Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license.