Skip to content

fix(shiki): tokenize an unchanged fence once while a document streams - #616

Merged
farnabaz merged 1 commit into
comarkdown:mainfrom
onmax:fix/shiki-stream-token-cache
Nov 19, 2026
Merged

farnabaz merged 1 commit into
comarkdown:mainfrom
onmax:fix/shiki-stream-token-cache

Conversation

@onmax

@onmax onmax commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

What

The Shiki plugin now keeps each fence's tokens by its raw node, the same way pierre-highlights does since #612, so a fence the streaming parser hands back unchanged is tokenized once. Output nodes are still rebuilt on every update. Blocks with transformers go through codeToHast as before.

Why

While a document streams, the parser reuses its previous, unhighlighted nodes, and the plugin highlighted all of them again on every update. Streaming 20 short TypeScript fences 64 characters at a time called codeToTokens 182 times, once for each fence in each update. With this change it's 75, most of them for the fence that is still arriving.

I found it while profiling a streamed chat in comark-ui. The new test fails on main (2 calls instead of 1) and passes here. pnpm --filter comark run test, pnpm typecheck, oxlint and oxfmt on the changed files all pass.

Summary by CodeRabbit

  • Performance
    • Improved code highlighting during streaming updates by reusing results for unchanged code blocks, reducing repeated processing.
  • Bug Fixes
    • Updated code blocks are reprocessed when their content or language changes, and edits to previously rendered output no longer affect subsequent renders.

Read this PR faster    Follow each request

@onmax
onmax requested a review from farnabaz as a code owner November 19, 2026 09:51
@coldtea-pr-lens

coldtea-pr-lens Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Architecture

Walkthrough of Cache Shiki tokenization across streaming updates, in three steps

Read this PR faster


Data flow

Data flow diagram for comarkdown/comark at c92cafb

Follow each request


Tip

The CLI's render reads .github/pr-lens.yml and applies your renames, exclusions and lane pins at draw time

🪧 More tips
  • Run npx skills add coldteadotai/pr-lens, then tell your coding agent: "Diagram the change you just made with PR Lens and attach it to the pull request."
  • Run npx @coldtea/pr-lens-cli analyze --base origin/main on a branch, then npx @coldtea/pr-lens-cli render .pr-lens/graph.json. Same lenses, your own model key, before the pull request exists
  • Click the link under each diagram to open it on a canvas you can zoom, pan and step through
  • The diagrams are links. Click one to open it on the canvas, then press W or click play to walk through the change
  • Open a diagram on the canvas, then press W or click play to walk through the change one step at a time
  • Set github.comment.collapsed: true in .github/pr-lens.yml to fold the comment behind one View architecture… row. Drawing still runs as before
  • Set github.draw: on-demand in .github/pr-lens.yml and PR Lens stops drawing pull requests on its own. Comment @pr-lens draw on a pull request when you want that one drawn
  • Add .github/workflows/pr-lens.yml with coldteadotai/pr-lens/packages/action@v0 and your model provider's key as its api-key to run PR Lens from your own CI. Any /chat/completions endpoint works
  • After a push, tick Redraw to draw the latest commit
  • Switch GitHub to dark mode and the diagrams follow. The moving dots are this pull request's data in motion
  • Set github.description: false in .github/pr-lens.yml to stop PR Lens adding buttons to your PR descriptions

Thanks for using PR Lens! It's free for OSS. If it helped, a star helps other teams find it.  Star PR Lens on GitHub

@chatgpt-codex-connector

Copy link
Copy Markdown
Codex Review Summary: ❌ Failed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ❌ Failed 2026-11-19T09:51:30.296291Z
We couldn’t complete this review. Comment @codex review to try again.
c92cafb PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@vercel

vercel Bot commented Nov 19, 2026

Copy link
Copy Markdown

@onmax is attempting to deploy a commit to the NuxtLabs Team on Vercel.

A member of the Team first needs to authorize it.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@coderabbitai

coderabbitai Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

Walkthrough

Shiki code-block highlighting now accepts a token cache and reuses cached tokenization results when the code and language match. Each plugin instance maintains its own cache. A streaming test checks reuse across parses and distinct output nodes.

Changes: Shiki token caching

Layer / File(s) Summary
Cache contract and token reuse
packages/comark/src/internal/shiki.ts
highlightCodeBlocks accepts an optional cache. The tokenization path reuses cached results when both the code and language match; otherwise, it tokenizes and updates the cache.
Plugin integration and streaming test
packages/comark/src/internal/shiki.ts, packages/comark/test/plugins/shiki-streaming.test.ts
Each plugin instance passes its cache to post-processing. The streaming test checks that unchanged code is tokenized once and that successive parses return distinct nodes.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Suggested reviewers: atinux

Merge Risk: Low · up to c92ca

Streaming Shiki fences are now tokenized once per unchanged block. A stale-color edge case exists only if a caller mutates theme options in place mid-stream, which is unlikely. The change is low risk to merge.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title says the change makes an unchanged fence tokenize once while a document streams. The raw summary confirms a per-fence token cache keyed by the raw code-block node, reused only when code and …
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title says the change makes an unchanged fence tokenize once while a document streams. The raw summary confirms a per-fence token cache keyed by the raw code-block node, reused only when code and language match, plus a streaming test for that behavior. The PR objectives describe the same goal. The title is specific and matches the main change. The 'fix(shiki)' scope and conventional-commit form are acceptable.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
Fix failing CI checks

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@pkg-pr-new

pkg-pr-new Bot commented Nov 19, 2026

Copy link
Copy Markdown

Open in StackBlitz

comark

npm i https://pkg.pr.new/comark@616

@comark/angular

npm i https://pkg.pr.new/@comark/angular@616

@comark/ansi

npm i https://pkg.pr.new/@comark/ansi@616

@comark/html

npm i https://pkg.pr.new/@comark/html@616

@comark/nuxt

npm i https://pkg.pr.new/@comark/nuxt@616

@comark/react

npm i https://pkg.pr.new/@comark/react@616

@comark/svelte

npm i https://pkg.pr.new/@comark/svelte@616

@comark/vue

npm i https://pkg.pr.new/@comark/vue@616

commit: c92cafb

@codspeed

codspeed Bot commented Nov 19, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

1 warning

⚠️ 1 benchmark is dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

✅ 44 untouched benchmarks


Comparing onmax:fix/shiki-stream-token-cache (c92cafb) with main (a320fbf)

Open in CodSpeed

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional comments: 1

packages/comark/src/internal/shiki.ts-407-407: 1

407-407: Minor · Functional Correctness · Quick win

Unverified finding
Verification ran but could not confirm this finding. It is shown for review, not as a verified issue.

Invalidate cached tokens when the highlighting configuration changes.

If a caller replaces the retained options.themes between streaming updates, this condition still accepts the previous theme's result. getHighlighter and themeOptions use the new configuration, but the rebuilt block retains the previous token colors and themeName. Previously, each update tokenized with the current themes. Shiki derives both values from the selected themes. (raw.githubusercontent.com)

Include the effective light/dark theme selection and highlighter identity in cache validity checks. Keep cache hits when that context is unchanged.

Review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: aefeed1f-1f5c-4a61-ba2c-1d58e64aed7e
Commits

Reviewing files that changed from the base of the PR and between a320fbf and c92cafb.

Files selected for processing: 2

  • packages/comark/src/internal/shiki.ts
  • packages/comark/test/plugins/shiki-streaming.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.

@farnabaz farnabaz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, Thanks

@farnabaz
farnabaz merged commit f1a70c8 into comarkdown:main Nov 19, 2026
8 of 17 checks passed
@github-actions github-actions Bot mentioned this pull request Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants