Skip to content

perf(chat): parse a streaming table once per row, not per character - #37395

Closed
jeeshofone wants to merge 1 commit into
kirodotdev:mainfrom
jeeshofone:fix/37358-stream-table-rows
Closed

jeeshofone wants to merge 1 commit into
kirodotdev:mainfrom
jeeshofone:fix/37358-stream-table-rows

Conversation

@jeeshofone

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A streaming table costs work per row, not per character.

While a reply streams a table, the dashboard parses the whole table again and restyles every cell on every frame. A table is one markdown block, so each character typed into its last row changes the block's text, and MarkdownBlock hands the whole table to react-markdown again. The cost grows with the table: every frame pays for all rows drawn so far.

Why it matters

Replies made of tables are common, and the reporter measured 11.4 to 12.9 s of main-thread time per 17 s of streamed tables. That time slows the whole dashboard while a reply streams.

Not a goal

  • Splitting a table into several blocks in useBlockAssembler.
  • Changing how anything other than a table's last row streams.

What changed (motivation → approach → change)

Symptom: a full parse and restyle of the table on each frame. Cause: the block's text changes on each character, and nothing reuses the parse when the parsed text is the same.

  • deferIncompleteStreamingTable already holds back a table header until its delimiter row arrives. It now also holds back the half-typed last row of a real table (a | line after the delimiter with no line break yet), but only while that row is being typed (holdRow).
  • holdRow turns off when the block stops being the text being typed (a fence opened right after the table) or when the text has not changed for REVEAL_IDLE_SETTLE_MS (500 ms, a tool call or an approval wait mid-reply), so a finished row never stays hidden.
  • MarkdownBlock keeps the react-markdown element in useMemo, keyed on the prepared text and every input of the plugin lists, plus the language generation. While a row is held, the prepared text is the same on each frame, so React reuses the element and skips the parse.

Backwards compatibility

Compatible: the final render of every message is unchanged; only a streaming table's half-typed last row appears when its line break arrives instead of character by character.

Removes nothing: the change only defers when one row is drawn during streaming and reuses an identical render.

Tests

website/src/test/MarkdownRenderer.streamingTableRows.test.tsx:

  • typing the last row character by character causes no new react-markdown render; its line break causes exactly one.
  • each streaming frame renders the same HTML as the text up to its last line break (output unchanged).
  • the last row shows once the stream ends, after a 500 ms pause, and when a fence opens right after the table.
  • a prose line after a table stays visible.

Proof: not provable -- frontend test (prove.py runs Python only). The first two tests fail on main, and the pause and fence tests fail on the commit without the holdRow release.

Manual verification

Benchmark (vitest + jsdom, same machine, 3 runs each, median): a reply of 8 tables, 12 rows of 4 columns, 4,039 characters, streamed 9 characters per frame (540 characters/s at 60 fps, as in the issue), 448 frames.

median total render time
main @ 0a27793 6,823 ms (runs 6,884 / 6,823 / 6,642)
this PR 1,861 ms (runs 2,093 / 1,861 / 1,811)

That is 73% less render time. The benchmark script is in a PR comment so it can be re-run.

Screenshots / video

Why no screenshot: the finished render is identical (a test pins it), and the only streaming difference is that a table row appears when its line ends instead of character by character.

Related Issues

Fixes #37358

Sibling sweep

Swept: git grep -n "deferIncompleteStreamingTable|<ReactMarkdown" -- website/src/components/
Sibling: website/src/components/MarkdownRenderer.tsx -- changed here
Siblings: none -- MarkdownBlock is the only streaming markdown render; other ReactMarkdown sites render final text once.

Pattern harvest

Not generalizable: the cost comes from this block's per-frame text growth; other streaming surfaces already hold incomplete constructs.

Review dispositions

  • Opus 5.5 BLOCK on ab04af4be0 (a finished row stayed hidden during a mid-reply pause or before a fence): fixed by holdRow with tests; a verifier confirmed it closed on the current head.
  • Opus low notes: a row typed after a pause longer than 500 ms disappears again until its line break (brief, no content lost), and chunks arriving more than 500 ms apart re-parse on each chunk as main does today. Both are accepted: the hold is tied to the same idle window the reveal settle uses.
  • Opus low note: with redaction markers present, markers gets a new identity each frame, so such messages keep main's cost. Output is correct; left as is.
  • GPT 6.1: no findings.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license.

Hold back a streaming table's half-typed last row until its line break
arrives, and reuse the parsed element while a block's prepared text is
unchanged, so react-markdown runs once per row.

Fixes kirodotdev#37358
@jeeshofone

Copy link
Copy Markdown
Contributor Author

Benchmark used for the before/after numbers in the description. Copy to website/src/test/ and run with npx vitest run <file>.

import { it } from 'vitest'
import { render } from '@testing-library/react'
import MarkdownRenderer from '../components/MarkdownRenderer'

// Copy to website/src/test/ and run with vitest. 8 tables, 12 rows x 4 cols,
// streamed 9 chars per frame (540 chars/s at 60 fps, as in #37358).
function reply(): string {
  const out: string[] = []
  for (let t = 0; t < 8; t++) {
    out.push(`| Name ${t} | Kind | Size | Note |`, '| --- | --- | --- | --- |')
    for (let r = 0; r < 12; r++) out.push(`| row ${r} | item-${t}-${r} | ${r * 37} KB | ok ${r} |`)
    out.push('')
  }
  return out.join('\n')
}

it('bench #37358', () => {
  const text = reply()
  const runs: number[] = []
  for (let k = 0; k < 3; k++) {
    let frames = 0
    const t0 = performance.now()
    const { rerender, unmount } = render(<MarkdownRenderer content="" streaming glow />)
    for (let i = 9; i <= text.length; i += 9) {
      rerender(<MarkdownRenderer content={text.slice(0, i)} streaming glow />)
      frames++
    }
    runs.push(performance.now() - t0)
    unmount()
    console.log(`BENCH run ${k}: ${frames} frames, ${runs[k].toFixed(0)} ms`)
  }
  runs.sort((a, b) => a - b)
  console.log(`BENCH median ${runs[1].toFixed(0)} ms, chars ${text.length}`)
}, 600000)

@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: checking Automated validation is still running labels Nov 14, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of 936c5ec711390fa312e1f64b0765aa485346302e via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

No design-level concerns: holding back the half-typed last row and memoizing on the prepared text fixes the per-character re-parse, and both release paths keep a finished row from staying hidden.

Notes

  • If tables stay costly, the next step is splitting a table into per-row blocks in useBlockAssembler. The PR rules that out as a non-goal; this fix is proportionate until then.
  • The useMemo dependency list repeats by hand what rehypePlugins is built from (eslint-disable exhaustive-deps). A new input added to that builder won't be keyed and would serve a stale render. Memoizing rehypePlugins itself on its real inputs would remove that drift.

[DESIGN-REVIEWED] 936c5ec

Description read

[DESCRIPTION-READ] ecdde7eb951c865aa7c5f9e8b7c82c40133d1007479431d437932781ef471c00

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ✅ PASS

UX-level review of 936c5ec711390fa312e1f64b0765aa485346302e via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

UX-Verdict: PASS

Nothing to check: no control, label or string is added or changed. The one visible change is that a streaming table's last row now appears whole when its line ends, instead of character by character.

[UX-REVIEWED] 936c5ec

Description read

[DESCRIPTION-READ] 5cb2cf354d82e0fae45e4941f8a98f809c7951e8d8f61ca7ab65c316ff003463

That sha256 names everything this verdict read: the PR description, plus the 3 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — ✅ PASS

Premise-level review of 936c5ec711390fa312e1f64b0765aa485346302e via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. Issue #37358 measures the cost being removed, and the existing streaming-table deferral is extended rather than copied.

What this change ships

Inventory (4 items) — 4 justified

Intent: keep the dashboard responsive while a reply streams tables, by parsing a table once per row instead of once per character. This is a FIX.

  1. While a reply streams, a table's half-typed last row now appears when its line ends, not one character at a time — justified
  2. A held row appears after 500 ms with no new text (a tool call or an approval wait mid-reply) — justified
  3. A held row appears as soon as the block is no longer being typed (a fence opens after the table) — justified
  4. A markdown block whose prepared text and plugins did not change reuses its last render and skips the parse — justified

[FIRST-PRINCIPLES-REVIEWED] 936c5ec

Description read

[DESCRIPTION-READ] 15b6398a24be8078b2d490eb2bd1b37ba8ba24178e633f238be6f4f910d2cc28

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — 🔴 changes requested (blocking)

Reviewed 936c5ec711390fa312e1f64b0765aa485346302e via the fork AI-review pipeline; updated in place on each push.

The memoized markdown element ignores the text-pattern generation, so app text-preview spans no longer render in chat messages.

BLOCKING — website/src/components/MarkdownRenderer.tsx:96
[prepared, remarkPlugins, sourcePos, glow, smooth, markers, langGen],
Trigger: A chat message is rendered with textPreviews while an app declares textPatterns. On the first parse, remarkLinkPreviewText calls textPatternHits(value), which queues the text and returns []. When the worker answers, onAnswer calls notify(), which increments generation. That changes TextPreviewsCtx (textPatternGen + 1) and re-renders MarkdownBlock. But textPreviews is reduced to > 0, which stays true, and prepared and remarkPlugins keep the same identity, so useMemo returns the cached element and remark never runs again.
Outcome: Hover spans never appear on a message whose text has not changed since its first parse. After an overrun or a pattern disable, the stale spans stay on screen. Existing tests such as textPatterns.test.tsx:150 (expect(spans(container)).toEqual(['TK-1234']) after the worker answers) depend on this re-parse.
Anchor: defect
Fix: Add the raw useContext(TextPreviewsCtx) value (the generation) to the useMemo dependencies, instead of only the derived boolean.

[BLOCK-MERGE] 936c5ec
[OPUS-REVIEWED] 936c5ec

@github-actions

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — 🔴 changes requested (blocking)

Reviewed 936c5ec711390fa312e1f64b0765aa485346302e via the fork AI-review pipeline; updated in place on each push.

Adjudication upheld 1 of 1 blocking finding(s); those must be resolved before merging.

BLOCKING -- website/src/components/MarkdownRenderer.tsx:1059 -- Memoization prevents plain-text app previews from appearing

[prepared, remarkPlugins, sourcePos, glow, smooth, markers, langGen],
Trigger: A finished chat reply without redaction markers contains text matching an enabled app’s textPatterns. remarkLinkPreviewText queues its initial scan; the worker’s answer calls notify(), changing TextPreviewsCtx and rerendering MarkdownBlock.
Outcome: The context generation is reduced to an unchanged boolean, leaving every memo dependency unchanged. React reuses the markdown element without rerunning the plugin, so matching text never receives preview spans and hovering cannot open the app’s card.
Anchor: defect
Fix: Retain the numeric TextPreviewsCtx generation and include it in the markdown element’s memo dependencies; derive the enablement boolean separately.

[BLOCK-MERGE] 936c5ec
[GPT-REVIEWED] 936c5ec

Adjudication (Opus 5.5) — is blocking on each finding proportionate?

F1 is real and should keep blocking (harm-warrants-remedy). The new memo breaks plain-text app previews on the normal path, and the fix is one dependency.

F1, MEDIUM:

  • The first render of any text never has hits yet. website/src/linkPreview/textPatterns.ts:313-322 returns [] and queues a worker scan. When the answer comes back, onAnswer calls notify() (textPatterns.ts:282), which bumps generation (textPatterns.ts:126-129). That flows through useSyncExternalStore (website/src/components/MarkdownRenderer.tsx:1279) into TextPreviewsCtx (MarkdownRenderer.tsx:1540). But MarkdownBlock only keeps > 0 (MarkdownRenderer.tsx:947), so none of the new useMemo dependencies change (patch line 96). React reuses the ReactMarkdown element and the plugin never runs again.
  • Nothing recovers, short of an unrelated change to the content or plugins. The feature fails silently on every finished reply, and the fix is to put the numeric context value in the memo dependencies.

[ADJUDICATION] 936c5ec total=1 uphold=1 downgrade=0
UPHOLD F1 website/src/components/MarkdownRenderer.tsx:1059 reason=harm-warrants-remedy
[GPT-ADJUDICATED] 936c5ec

@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: Request changes

Why: the decider found 1 problem and estimates an 85% chance that this change introduces a defect, at or above the 80% level where it asks for changes.

Holding back the half-typed last row of a streaming table is reasonable and tested. However, the new useMemo around ReactMarkdown leaves the TextPreviewsCtx generation out of its deps, even though the code relies on that context changing to re-parse blocks once the async text-pattern worker answers. On this head, plain-text link-preview spans would never render, and existing textPatterns tests appear to fail. Request changes.

Problems found:

  • After the PR, a text-pattern generation bump re-renders MarkdownBlock but returns the memoized ReactMarkdown element, so remarkLinkPreviewText never runs again and plain-text preview spans never appear after the async worker answers. The same applies to a pattern set changing or being disabled. This (website/src/components/MarkdownRenderer.tsx, high)

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit 936c5ec711 · 141 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 14, 2026
@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

This PR and #37414 both close #37358 with the same fix in website/src/components/MarkdownRenderer.tsx. Both hold back the half-typed last row of a streaming table next to deferIncompleteStreamingTable, memoize the react-markdown parse while the prepared text is unchanged, and add website/src/test/MarkdownRenderer.streamingTableRows.test.tsx. #37414 also makes the prompt and the reply show up right away on a slow device. Please keep one of #37395 and #37414 and close the other, carrying over anything only this one has.

@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Nov 15, 2026
@jeeshofone

Copy link
Copy Markdown
Contributor Author

Closing as superseded by #37414, which merged with the same fix for #37358. Thanks, @buluoray.

@jeeshofone jeeshofone closed this Nov 15, 2026
@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor) merge conflict Branch has merge conflicts with its base — author must resolve before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A streaming table is parsed and restyled again on every frame

2 participants