Skip to content

fix(chat): bound markdown nesting depth on the parsed tree, not the text - #10753

Merged
iamwhatever merged 1 commit into
mainfrom
fix/markdown-depth-bound-parsed-tree
Sep 14, 2026
Merged

iamwhatever merged 1 commit into
mainfrom
fix/markdown-depth-bound-parsed-tree

Conversation

@iamwhatever

@iamwhatever iamwhatever commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

A chat message containing pathologically nested markdown — thousands of leading > blockquote markers, a list indented hundreds of levels, or a <div><div>… raw-HTML run — parses into a tree whose depth equals the nesting count. Several layers downstream of the parser recurse over that tree: remark-gfm's own post-parse transform (unist-util-visit-parents), remark-rehype, hast-util-from-parse5 inside rehype-raw, react-markdown's hast→JSX conversion, and MarkdownRenderer.tsx's eight self-recursive walkers. Past the engine's call-stack limit any one of them throws RangeError: Maximum call stack size exceeded while rendering one message.

Reproduced at origin/main: 50,000 nested blockquote markers throw through MarkdownRenderer; so do 5,000 nested <div>s in an HTML block, and a 1,200-level indented list does not finish parsing inside a five-minute budget.

Why it matters

In the chat transcript every message is wrapped in MessageErrorBoundary (src/pages/chat/AssistantMessage.tsx, src/pages/chat/ChatPageMessageContent.tsx, src/app-sdk/messageRenderers.tsx), so the pre-fix cost there is one message degraded to raw text plus a multi-second overflow parse repeated on every streaming frame while that message is the live tail. Outside chat, MarkdownRenderer is rendered bare — src/pages/ChannelPage.tsx (channel message bodies) and src/components/notifications/NotificationDetailPanel.tsx (notification bodies) have no per-item boundary — so the same RangeError propagates to the app-level ErrorBoundary and replaces the view. Mochi's ChatPanel runs its own copy of the pipeline over the same content and has no boundary at all. Every one of these inputs is input-controlled: a channel message, a notification body, an agent's output. Nothing upstream caps nesting depth, so the renderer has to.

What changed (motivation → approach → change)

Where to bound. Two places could hold the guard. Before parsing, by rewriting the raw text so the tree comes out shallow — which requires predicting what two parsers (micromark, then parse5 via rehype-raw) will make of the text without running them. Or after parsing, on the tree they produce — where depth is a fact to be read, not predicted.

The pre-parse approach was tried first, in #8916. Its grammar model was bypassed seven times in four review rounds (fake closes inside comments, the code-span mask, <x:y> tag names, empty list items, form-feed-terminated tags, fence-repair ordering, SVG foreign-content voids), and fixing bypasses introduced two more, because every rule the model gets slightly different from the real parser is a hole in the direction that re-opens the crash. That PR is closed with the full record; this one replaces it.

Why the tree is bounded where it is. Verified against the pinned versions in website/node_modules:

  • micromark and mdast-util-from-markdown are iterative. remark-parse alone parses 50,000 nested blockquotes without throwing.
  • The markdown crash lives in mdast-util-gfm-autolink-literal's transforms hook, which recurses via unist-util-visit-parents and runs inside parse(), before any unified transformer. A plain remark plugin is too late. A fromMarkdownExtensions.transforms entry registered before remark-gfm runs first, through the same hook.
  • parse5's tree builder is iterative. The HTML crash lives in hast-util-from-parse5's recursion inside hast-util-raw, whose parser options are a hardcoded constant — no adapter can be injected, but the same parser can be run as an oracle first.
  • The tree bound cannot bound the parser's own cost: micromark re-scans every open container on every line, so a list indented to depth d costs O(lines × d) before a tree exists (1,200 levels: minutes; 600 levels behind a > prefix: 6.8s). That one dimension is decided by whitespace alone, which is lexical.

New website/src/utils/markdownDepthBound.ts, three mechanisms, no grammar modeling:

  1. remarkBoundDepth — a mdast-util-from-markdown transform registered ahead of remark-gfm. Walks the parsed mdast with an explicit stack and folds any subtree past MAX_TREE_DEPTH (100) into one text node carrying its words (image alt included). A test pins both the plugin order and why (a transformer-registered walk provably runs too late).
  2. rehypeBoundRawDepth — ahead of rehype-raw. If the tree holds raw HTML, serializes it with raw admitted and parses that with parse5 as an oracle, measuring the DOM depth the raw HTML will produce. Past the bound, every raw node is downgraded to a text node, so hast-util-from-parse5 never sees a deep tree and the message keeps its words while losing only its HTML rendering. The oracle uses the parser options hast-util-raw uses (scriptingEnabled: false, under which <noscript> content nests rather than being flat RAWTEXT) and counts <template> children on parse5's separate content fragment at the two levels hast-util-from-parse5 gives them. A 47-input differential against hast-util-raw@9.1.0's actual output found no case where the oracle reads ≤ 100 while the real tree exceeds it.
  3. capWhitespaceRuns — truncates any run of spaces and tabs wider than MAX_INDENT_COLS (256), wherever on a line it sits. No CommonMark construct needs more than four columns of whitespace to mean what it means, so 256 means what 300 means everywhere except inside code, where alignment past 256 columns loses the excess — the one visible effect, and only on content no person writes. Identity (same string reference) on any message without such a run.

Below the bounds nothing is rewritten: ordinary content renders byte-identically and data-sourcepos coordinates stay exact.

Wiring. MarkdownRenderer.tsx puts remarkBoundDepth first in REMARK_PLUGINS, rehypeBoundRawDepth first in both rehype lists, and applies the whitespace cap at the MarkdownBlock entry — gated off in sourcePos mode like every other column-shifting pass there (data-sourcepos maps a selection back to source coordinates; a shortened run would anchor a comment to the wrong occurrence). The tree bounds hold in both modes; only the parser-time bound is given up on that surface. Mochi's ChatPanel.tsx adds the same two plugins to MD_REMARK / MD_REHYPE — imported from the core, as its sanitizer already is — and applies the cap per text segment at all five <Markdown> sites, after widget extraction, so an <mcwidget> payload reaches WidgetFrame byte-identical (a pre-parse text clamp could not guarantee that, which is what kept #8916 off this surface).

Dependencies. parse5 and hast-util-to-html become direct dependencies at the versions the lockfile already resolves them to (both are transitive dependencies of rehype-raw); the lockfile change is the two root entries.

Tests

website/src/test/MarkdownRenderer.depthBound.test.tsx (27), website/src/test/markdownDepthBound.oracle.test.ts (22) and website/src/apps/mochi/test/mochiChatPanelDepthBound.test.tsx (5). Every vector review found against the pre-parse approach is here, and none needed a rule:

  • 10,000-deep blockquotes; interleaved > - > -; > > > re-opens; 400-level lists with and without item text, and behind a > prefix
  • raw HTML: deep <div> runs as a block and across paragraphs; <x:y>, <span\f>, NUL; <svg> + <base>; <noscript> and <template> bodies; fake closes inside comments, CDATA, attribute values and code spans; the fence-repair reordering vector
  • plugin-order contract: the bound holds at 30k inside parse(), and a transformer-registered bound would throw first
  • whitespace cap: identity when no run exceeds 256; truncation of only the offending run; tab widths; a run after a blockquote prefix
  • nothing changes below the bound: structure, raw HTML rendering, sourcepos (including past a wide whitespace run, where the cap is gated off), a folded image's alt text
  • oracle parity (markdownDepthBound.oracle.test.ts): for every construct known to change nesting — colon names, \f, NUL, SVG voids, <noscript>, <template>, comment/CDATA/attribute/code-span fake closes, self-closing spelling, implied-end siblings, mixes with blockquotes, lists and math passthrough — the oracle's decision is compared against the tree rehype-raw actually builds, asserting it downgraded whenever the real depth exceeds the bound. This is the drift detector for the design's one assumption: a parse5 or hast-util-raw upgrade, or a lockfile resolving them to different parse5 instances, turns a case red instead of re-opening the overflow (flipping the scriptingEnabled option turns two red today)
  • mochi: Bubble survives the deep vectors; a widget with a 150-deep body and a 300-space <pre> line reaches WidgetFrame byte-identical

Revert-verified per mechanism, not in aggregate: removing the mdast bound turns six red, the raw bound five, the whitespace cap three, and the scripting option (two, in the oracle test), template traversal, alt collection, per-segment widget cap and sourcePos gate one each — each with the exact RangeError or the parser timeout.

Fixtures sit at 10k blockquotes and 400 list levels — far past where the unbounded pipeline throws, far under what the concurrent CI suite parses inside its 15s per-test budget; the 15k parse-level contract carries no React render.

Manual verification

N/A — unit coverage sufficient: the guarantee is "renders without throwing, below-bound content unchanged", and both halves are asserted at the DOM level through the real MarkdownRenderer and mochi Bubble components in the vitest DOM harness.

Related Issues

Fixes #10571 (mochi ChatPanel sibling — covered by construction). Fixes #10607 (pinning copied grammar tables — moot, there are no copied tables). Replaces #8916 (closed; javenciu's problem statement, reproducers and review history are what this design rests on — co-authored). #10620 is deliberately NOT closed by this PR: its two residual crash vectors (fence-repair ordering; SVG foreign-content voids) are tests here and pass by construction, but its third item — a per-item error boundary on the two bare MarkdownRenderer surfaces, which bounds the cost of any future miss to one item regardless of mechanism — stays open as separate work.

Pattern harvest

When a guard must decide "what will the parser make of this input", put the guard after the parser, on its output, and let the parser be the oracle. A guard placed before the parser has to re-implement the grammar, and its failure mode is silent: every rule that differs is a bypass, every fix is a chance to introduce another, and the maintenance obligation is permanent because the upstream parser keeps moving. #8916's record — seven bypasses, two regressions from fixes, four rounds without convergence — is the measurement.

Two corollaries surfaced here. First, "after the parser" means after the whole parse step, including hooks that run inside it: remark-gfm's recursive transform runs inside parse(), so the bound had to register through the same hook, earlier — a plugin that looked like it ran first did not. Second, an oracle is only an oracle under the same options: a one-flag difference (scriptingEnabled) made the oracle read <noscript> content as flat while the real parse nested it. Both are pinned by tests that fail if the placement or the options drift.

Rule candidate: review-prompt — when a change adds a pre-parse rewrite of parser input (markdown, HTML, JSON, a DSL) to defend against a parser-side failure, ask whether the same bound can be applied to the parser's output instead; if it can, the rewrite is a shadow parser and should be replaced.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Why no screenshot: no visual delta -- every mechanism is an identity on content below its bound (pinned by tests), and the only inputs that render differently are ones that crashed the renderer before.

@iamwhatever
iamwhatever requested review from a team and buluoray September 14, 2026 09:30
@iamwhatever

Copy link
Copy Markdown
Contributor Author

🤖 Kiro Crew [operator: iamwhatever#241cc860]:
Intent: A chat message with pathologically deep markdown nesting (blockquote runs, list indent, raw-HTML tag runs) must not crash the renderer with RangeError: Maximum call stack size exceeded. Depth is bounded on the PARSED tree, with the real parsers (micromark, parse5) as the oracle, so there is no grammar model to keep in sync with them; ordinary content stays byte-identical.
Not a goal: Rewriting raw markdown text to predict parser output (the approach #8916 tried and closed); per-item error boundaries on the two bare MarkdownRenderer surfaces (#10620 item 3); capping total message size for parser time on multi-megabyte inputs.

@github-actions

github-actions Bot commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of 68779fe1c55dec5c4339565b9dc0d59757520072 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 68779fe

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 68779fe1c55dec5c4339565b9dc0d59757520072: <one-sentence reason>

@github-actions

github-actions Bot commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of 68779fe1c55dec5c4339565b9dc0d59757520072 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Bounds depth on the parser's own output with the same-options oracle and drift-detecting parity tests — the root-cause shape #8916's pre-parse rewrite could never converge on.

[DESIGN-REVIEWED] 68779fe

@github-actions

github-actions Bot commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Fable 5) — ✅ PASS

UX-level review of 68779fe1c55dec5c4339565b9dc0d59757520072 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

The diff adds no user-visible controls, labels, or strings — it's a rendering-pipeline guard (depth bounding on parsed markdown trees plus a whitespace-run cap). The only visible deltas occur on pathological input (nesting depth >100, whitespace runs >256 columns) that previously crashed the renderer with a RangeError; below the bounds the PR pins byte-identical rendering with tests. With zero added or changed controls, the absent blind read produces zero evidence gaps, and no lens-13 element transformation exists. The degraded paths (deep subtrees fold to their text, raw HTML downgrades to escaped text) strictly improve on the prior behavior of a crashed message or a replaced view.

UX-Verdict: PASS

No user-visible control changes; pathological input that used to crash the renderer now degrades to readable text, byte-identical below the bounds.

[UX-REVIEWED] 68779fe

@github-actions

github-actions Bot commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — ✅ PASS

Premise-level review of 68779fe1c55dec5c4339565b9dc0d59757520072 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

All checks pass. The sourcePos surface (ContentRenderer.tsx:193) is the one declared residual. Writing the final review.

First-Principles-Verdict: PASS

One residual to verify: sourcePos surfaces (ContentRenderer.tsx:193) skip capWhitespaceRuns, so the O(lines×depth) parser cost stays open there — confirm that surface tolerates it.

What this change ships

Inventory (8 items) — 8 justified

Intent: FIX — a chat, channel, or notification message with pathologically nested markdown no longer crashes the renderer or blanks the view.

  1. A message with thousands of nested blockquotes/lists now renders its words instead of throwing — justified
  2. Raw HTML nested past the bound now shows as escaped text instead of crashing — justified
  3. Whitespace runs wider than 256 columns are shortened before parsing; code alignment past 256 cols loses the excess — justified
  4. Inline-comment (sourcePos) surfaces keep exact columns by skipping the cap, retaining the parser-time cost — justified
  5. Mochi chat bubbles gain the same bounds — justified
  6. Mochi widget payloads still reach WidgetFrame byte-identical — justified
  7. parse5 and hast-util-to-html become direct dependencies at the versions the lockfile already resolves — justified
  8. A shared util module exports the two plugins, the cap, and their constants — justified

Provenance is derived throughout: the defect is pinned by tests this PR adds that fail on base (the plugin-order test shows the base pipeline throws RangeError), and the input sits at the EXTERNAL CONTENT boundary. No pre-existing depth-bound mechanism exists (grepped MAX_.*DEPTH|maxDepth|RangeError in website/src, non-test: only unrelated tree caps). Sibling count: grepped remark-gfm/react-markdown across website/src — the only two non-test parse pipelines are MarkdownRenderer.tsx and Mochi's ChatPanel.tsx, both fixed; sessionRefs.ts/tableClipboard.ts don't parse raw text. Consumer counts: remarkBoundDepth/rehypeBoundRawDepth 2 each, capWhitespaceRuns 6 call sites across both surfaces.

[FIRST-PRINCIPLES-REVIEWED] 68779fe

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 14, 2026
@github-actions

github-actions Bot commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

Opus 4.8 Review — ✅ no blocking findings

Reviewed 68779fe1c55dec5c4339565b9dc0d59757520072 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 68779fe

Verdict parsed from the review's SHA-scoped output markers for commit 68779fe1c55dec5c4339565b9dc0d59757520072.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 68779fe1c55dec5c4339565b9dc0d59757520072: <one-sentence reason>

Chat markdown is input-controlled. A message of thousands of leading `>`
markers, of progressively indented list items, or of `<div><div>...` raw
HTML parses into a tree whose depth equals the nesting count, and several
layers downstream of the parser recurse over it: remark-gfm's post-parse
transform (unist-util-visit-parents), remark-rehype, hast-util-from-parse5
inside rehype-raw, react-markdown's hast->JSX conversion, and this
codebase's own tree walkers. Past the engine's call-stack limit any one of
them throws `RangeError: Maximum call stack size exceeded` on a single
message. In the chat transcript MessageErrorBoundary degrades that message
to raw text; ChannelPage and NotificationDetailPanel render MarkdownRenderer
bare, so there the whole view is replaced.

The parsers themselves are iterative. micromark and mdast-util-from-markdown
parse a 50,000-deep blockquote run without throwing, and parse5 keeps an
explicit open-element stack. So the depth of the tree is a fact that can be
read off the parser's output, and it is bounded there:

- `remarkBoundDepth` registers a mdast-util-from-markdown `transforms` entry
  that walks the parsed mdast with an explicit stack and folds any subtree
  past MAX_TREE_DEPTH into one text node carrying its words. That hook runs
  INSIDE parse(), in registration order -- and remark-gfm's autolink-literal
  extension registers a recursive transform through the same hook, which is
  where the markdown crash actually lives. A unified transformer would be
  too late, so the plugin is `use()`d ahead of remark-gfm; a test pins both
  the order and the reason.
- `rehypeBoundRawDepth` runs ahead of rehype-raw. If the tree holds raw
  HTML it serializes the tree with raw admitted and parses that with parse5
  as an oracle (iterative), measuring the DOM depth the raw HTML will
  produce. Past the bound every raw node is downgraded to a text node, so
  hast-util-from-parse5 never sees a deep tree and the message keeps its
  words while losing only its HTML rendering. The oracle is only an oracle
  if it reads the same tree the real parse builds, so it uses the parser
  options hast-util-raw uses (`scriptingEnabled: false`, under which a
  `<noscript>` body nests rather than being flat RAWTEXT) and it counts a
  `<template>`'s children, which parse5 parks on a separate `content`
  fragment that hast-util-from-parse5 also recurses into.
- `capWhitespaceRuns` truncates any run of spaces and tabs wider than
  MAX_INDENT_COLS, wherever on a line it sits. The tree bound cannot bound
  the PARSER'S OWN COST: micromark re-scans every open container on every
  line, so a list indented to depth d costs O(lines * d) before a tree
  exists (a 1,200-level list parsed for minutes; a 600-level one behind a
  `> ` prefix took 6.8s, 1.0s capped). Indentation is decided by whitespace
  alone, and a run after a blockquote prefix indents exactly as a leading
  run does, so the cap is over every run: lexical, recognizing no
  construct. No CommonMark construct needs more than four columns of
  whitespace to mean what it means, so 256 means the same as 300 everywhere
  except inside code, where alignment past 256 columns loses the excess.

Below the bounds nothing is rewritten, so ordinary content renders
byte-identically and data-sourcepos coordinates stay exact. A folded
subtree keeps its words, including an image's alt text.

The alternative -- rewriting the raw text before parsing so the tree comes
out shallow -- requires predicting what two parsers will make of the text
without running them. Every rule that model gets slightly different from the
real parser is a bypass in the direction that re-opens the crash, and the
model has to be maintained against both upstreams. That approach was tried
first; review found seven bypass spellings in four rounds (fake closes
inside comments, the code-span mask, `<x:y>` tag names, empty list items,
form-feed-terminated tags, fence-repair ordering, SVG foreign-content voids)
and fixing them introduced two more. Every one of those spellings is a test
here, and none needed a rule: the parsers already know.

Both pipelines adopt the bound. MarkdownRenderer puts remarkBoundDepth first
in REMARK_PLUGINS and rehypeBoundRawDepth first in both rehype lists, and
caps whitespace runs at the MarkdownBlock entry -- gated off in sourcePos
mode like every other column-shifting pass there, since data-sourcepos maps
a selection back to source coordinates and a shortened run would anchor a
comment to the wrong occurrence; the tree bounds hold either way, so only
the parser-time bound is given up on that surface. Mochi's ChatPanel adds the same
two plugins to MD_REMARK / MD_REHYPE and applies the cap per TEXT segment,
after widget extraction, so an <mcwidget> payload reaches WidgetFrame
byte-identical -- its <pre> indentation included -- which a pre-parse text
clamp could not guarantee.

parse5 and hast-util-to-html become direct dependencies at the versions the
lockfile already resolves them to (both are transitive dependencies of
rehype-raw); the lockfile change is the two root entries.

Tests: `MarkdownRenderer.depthBound.test.tsx` (27),
`markdownDepthBound.oracle.test.ts` (22) and
`mochi/test/mochiChatPanelDepthBound.test.tsx` (5). The oracle test is the
drift detector for the design's one load-bearing assumption -- that the
oracle reads the same tree rehype-raw builds: it runs both on every
construct known to change nesting and asserts the oracle downgraded
whenever the real tree exceeds the bound, so a parse5 or hast-util-raw
upgrade (or a lockfile resolving them to different parse5 instances) turns
a case red instead of re-opening the overflow. Reverting the mdast
bound turns six red, the raw bound five, the whitespace cap three, the
scripting option (two, in the oracle test), the template traversal, the
alt collection, the per-segment widget cap and the sourcePos gate one
each, with the exact RangeError or the parser
timeout; ordinary content, raw HTML below
the bound, and sourcepos are pinned unchanged. Fixtures sit at 10k
blockquotes and 400 list levels -- far past where the unbounded pipeline
throws, far under what the concurrent suite can parse inside CI's 15s
per-test budget; the 15k parse-level contract needs no React render.

Out of scope, tracked in #10620: a per-item error boundary on the two bare
MarkdownRenderer surfaces, which bounds the cost of any future miss to one
item regardless of mechanism.

Co-authored-by: javenciu <129346815+javenciu@users.noreply.github.com>
@iamwhatever
iamwhatever force-pushed the fix/markdown-depth-bound-parsed-tree branch from 3e248f3 to 68779fe Compare September 14, 2026 09:50
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 14, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor Author

🤖 Kiro Crew [operator: iamwhatever#241cc860]:

  • fixed — preserve source coordinates when capping whitespace (span=e072207effd8, MarkdownBlock whitespace cap in sourcePos mode)

Fixed at 68779fe by gating the cap off in sourcePos mode — if (!sourcePos) clean = capWhitespaceRuns(clean) — the same rule the other column-shifting passes in MarkdownBlock (fixCjkAutolinkBoundaries, fixUnencodedLinkDestinations) already follow. data-sourcepos then maps against exactly the text the parser saw. The crash bound does not depend on the cap (both tree bounds run in every mode); only the parser-time bound is given up on the inline-commenting surface, which does not stream.
Chosen over an offset map because it removes the mechanism from the path where it can do harm rather than adding a second mechanism to correct the first. Pinned by a test that renders a 300-column run in sourcePos mode and asserts the paragraph's end column is the original one; revert-verified red with the cap active.
Class-level ruling: no pass that changes column positions runs in sourcePos mode; findings that a column-shifting step is active there are covered by this rule.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

🤖 Kiro Crew [operator: iamwhatever#241cc860]:

  • fixed — pin the oracle-parity invariant with a committed test (span=47ccf6d85154)

Fixed at 68779fe: website/src/test/markdownDepthBound.oracle.test.ts runs the oracle and the real rehype-raw pass on the same 21 inputs — every construct known to change nesting (colon names, \f, NUL, SVG voids, <noscript>, <template>, comment/CDATA/attribute/code-span fake closes, self-closing spelling, implied-end siblings, mixes with blockquotes, lists and math passthrough) — and asserts the oracle downgraded whenever the real tree exceeds the bound. Inputs sit just past the bound so the real pass can run without overflowing.
It detects both drift modes you name: a parse5 / hast-util-raw upgrade that changes either tree, and a lockfile that resolves the two to different parse5 instances — the comparison is against rehype-raw's actual output, so whatever it resolves to is what is compared. Flipping the oracle's scriptingEnabled option turns two cases red today, which is the revert-verification.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

🤖 Kiro Crew [operator: iamwhatever#241cc860]:

  • fixed — commit the differential harness itself (span=0b9e924029be)

Done at 68779fe, as markdownDepthBound.oracle.test.ts (see the Watch record on this head). The harness is the drift detector; its conclusion in the PR body is now a pointer to it rather than a claim.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

🤖 Kiro Crew [operator: iamwhatever#241cc860]:

  • fixed — drop unconsumed exports on the two bound helpers (span=44f7b0b5b9d9)

At 68779fe boundMdastDepth is module-private (its only caller is the plugin in the same file). boundRawHtmlDepth stays exported because the oracle-parity test added on this head calls it directly against the tree rehype-raw builds, and its comment now says exactly that instead of the earlier untrue "exported for tests".

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Sep 14, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor Author

🤖 Kiro Crew [operator: iamwhatever#241cc860]: Ready for maintainer review at head 68779fe1c5.

@iamwhatever
iamwhatever merged commit fec69c3 into main Sep 14, 2026
115 of 116 checks passed
@iamwhatever
iamwhatever deleted the fix/markdown-depth-bound-parsed-tree branch September 14, 2026 18:22
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Sep 14, 2026
CrysisDeu added a commit that referenced this pull request Sep 16, 2026
…rBoundary

The chat transcript wraps every message in MessageErrorBoundary, so a
markdown render crash degrades one message. ChannelPage message bodies and
the NotificationDetailPanel body rendered MarkdownRenderer bare, so the same
crash there escalated to the app-level ErrorBoundary and replaced the whole
view. Wrap both call sites with rawContent so the fallback offers the raw
text, matching the transcript.

Regression test renders each surface with a renderer that throws on a
marker string and asserts the per-item fallback appears while the
surrounding surface (sibling messages, panel header) still renders. Red on
base, green with the fix.

Residuals 1 and 2 of #10620 landed in #10753.

Closes #10620
iamwhatever pushed a commit that referenced this pull request Sep 16, 2026
…rBoundary (#11135)

The chat transcript wraps every message in MessageErrorBoundary, so a
markdown render crash degrades one message. ChannelPage message bodies and
the NotificationDetailPanel body rendered MarkdownRenderer bare, so the same
crash there escalated to the app-level ErrorBoundary and replaced the whole
view. Wrap both call sites with rawContent so the fallback offers the raw
text, matching the transcript.

Regression test renders each surface with a renderer that throws on a
marker string and asserts the per-item fallback appears while the
surrounding surface (sibling messages, panel header) still renders. Red on
base, green with the fix.

Residuals 1 and 2 of #10620 landed in #10753.

Closes #10620
amadsalmon pushed a commit to amadsalmon/KiroCrew that referenced this pull request Sep 22, 2026
…ext (kirodotdev#10753)

Chat markdown is input-controlled. A message of thousands of leading `>`
markers, of progressively indented list items, or of `<div><div>...` raw
HTML parses into a tree whose depth equals the nesting count, and several
layers downstream of the parser recurse over it: remark-gfm's post-parse
transform (unist-util-visit-parents), remark-rehype, hast-util-from-parse5
inside rehype-raw, react-markdown's hast->JSX conversion, and this
codebase's own tree walkers. Past the engine's call-stack limit any one of
them throws `RangeError: Maximum call stack size exceeded` on a single
message. In the chat transcript MessageErrorBoundary degrades that message
to raw text; ChannelPage and NotificationDetailPanel render MarkdownRenderer
bare, so there the whole view is replaced.

The parsers themselves are iterative. micromark and mdast-util-from-markdown
parse a 50,000-deep blockquote run without throwing, and parse5 keeps an
explicit open-element stack. So the depth of the tree is a fact that can be
read off the parser's output, and it is bounded there:

- `remarkBoundDepth` registers a mdast-util-from-markdown `transforms` entry
  that walks the parsed mdast with an explicit stack and folds any subtree
  past MAX_TREE_DEPTH into one text node carrying its words. That hook runs
  INSIDE parse(), in registration order -- and remark-gfm's autolink-literal
  extension registers a recursive transform through the same hook, which is
  where the markdown crash actually lives. A unified transformer would be
  too late, so the plugin is `use()`d ahead of remark-gfm; a test pins both
  the order and the reason.
- `rehypeBoundRawDepth` runs ahead of rehype-raw. If the tree holds raw
  HTML it serializes the tree with raw admitted and parses that with parse5
  as an oracle (iterative), measuring the DOM depth the raw HTML will
  produce. Past the bound every raw node is downgraded to a text node, so
  hast-util-from-parse5 never sees a deep tree and the message keeps its
  words while losing only its HTML rendering. The oracle is only an oracle
  if it reads the same tree the real parse builds, so it uses the parser
  options hast-util-raw uses (`scriptingEnabled: false`, under which a
  `<noscript>` body nests rather than being flat RAWTEXT) and it counts a
  `<template>`'s children, which parse5 parks on a separate `content`
  fragment that hast-util-from-parse5 also recurses into.
- `capWhitespaceRuns` truncates any run of spaces and tabs wider than
  MAX_INDENT_COLS, wherever on a line it sits. The tree bound cannot bound
  the PARSER'S OWN COST: micromark re-scans every open container on every
  line, so a list indented to depth d costs O(lines * d) before a tree
  exists (a 1,200-level list parsed for minutes; a 600-level one behind a
  `> ` prefix took 6.8s, 1.0s capped). Indentation is decided by whitespace
  alone, and a run after a blockquote prefix indents exactly as a leading
  run does, so the cap is over every run: lexical, recognizing no
  construct. No CommonMark construct needs more than four columns of
  whitespace to mean what it means, so 256 means the same as 300 everywhere
  except inside code, where alignment past 256 columns loses the excess.

Below the bounds nothing is rewritten, so ordinary content renders
byte-identically and data-sourcepos coordinates stay exact. A folded
subtree keeps its words, including an image's alt text.

The alternative -- rewriting the raw text before parsing so the tree comes
out shallow -- requires predicting what two parsers will make of the text
without running them. Every rule that model gets slightly different from the
real parser is a bypass in the direction that re-opens the crash, and the
model has to be maintained against both upstreams. That approach was tried
first; review found seven bypass spellings in four rounds (fake closes
inside comments, the code-span mask, `<x:y>` tag names, empty list items,
form-feed-terminated tags, fence-repair ordering, SVG foreign-content voids)
and fixing them introduced two more. Every one of those spellings is a test
here, and none needed a rule: the parsers already know.

Both pipelines adopt the bound. MarkdownRenderer puts remarkBoundDepth first
in REMARK_PLUGINS and rehypeBoundRawDepth first in both rehype lists, and
caps whitespace runs at the MarkdownBlock entry -- gated off in sourcePos
mode like every other column-shifting pass there, since data-sourcepos maps
a selection back to source coordinates and a shortened run would anchor a
comment to the wrong occurrence; the tree bounds hold either way, so only
the parser-time bound is given up on that surface. Mochi's ChatPanel adds the same
two plugins to MD_REMARK / MD_REHYPE and applies the cap per TEXT segment,
after widget extraction, so an <mcwidget> payload reaches WidgetFrame
byte-identical -- its <pre> indentation included -- which a pre-parse text
clamp could not guarantee.

parse5 and hast-util-to-html become direct dependencies at the versions the
lockfile already resolves them to (both are transitive dependencies of
rehype-raw); the lockfile change is the two root entries.

Tests: `MarkdownRenderer.depthBound.test.tsx` (27),
`markdownDepthBound.oracle.test.ts` (22) and
`mochi/test/mochiChatPanelDepthBound.test.tsx` (5). The oracle test is the
drift detector for the design's one load-bearing assumption -- that the
oracle reads the same tree rehype-raw builds: it runs both on every
construct known to change nesting and asserts the oracle downgraded
whenever the real tree exceeds the bound, so a parse5 or hast-util-raw
upgrade (or a lockfile resolving them to different parse5 instances) turns
a case red instead of re-opening the overflow. Reverting the mdast
bound turns six red, the raw bound five, the whitespace cap three, the
scripting option (two, in the oracle test), the template traversal, the
alt collection, the per-segment widget cap and the sourcePos gate one
each, with the exact RangeError or the parser
timeout; ordinary content, raw HTML below
the bound, and sourcepos are pinned unchanged. Fixtures sit at 10k
blockquotes and 400 list levels -- far past where the unbounded pipeline
throws, far under what the concurrent suite can parse inside CI's 15s
per-test budget; the 15k parse-level contract needs no React render.

Out of scope, tracked in kirodotdev#10620: a per-item error boundary on the two bare
MarkdownRenderer surfaces, which bounds the cost of any future miss to one
item regardless of mechanism.

Co-authored-by: Joe Guo <zejiangg@amazon.com>
Co-authored-by: javenciu <129346815+javenciu@users.noreply.github.com>
amadsalmon pushed a commit to amadsalmon/KiroCrew that referenced this pull request Sep 22, 2026
…rBoundary (kirodotdev#11135)

The chat transcript wraps every message in MessageErrorBoundary, so a
markdown render crash degrades one message. ChannelPage message bodies and
the NotificationDetailPanel body rendered MarkdownRenderer bare, so the same
crash there escalated to the app-level ErrorBoundary and replaced the whole
view. Wrap both call sites with rawContent so the fallback offers the raw
text, matching the transcript.

Regression test renders each surface with a renderer that throws on a
marker string and asserts the per-item fallback appears while the
surrounding surface (sibling messages, panel header) still renders. Red on
base, green with the fix.

Residuals 1 and 2 of kirodotdev#10620 landed in kirodotdev#10753.

Closes kirodotdev#10620
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants