Skip to content

perf(ts): number-only small ints, inline from_be_bytes, and indexed byte patterns - #15

Merged
kixelated merged 10 commits into
mainfrom
quest/m2/ts-runtime-perf
Oct 24, 2026
Merged

kixelated merged 10 commits into
mainfrom
quest/m2/ts-runtime-perf

Conversation

@kixelated

@kixelated kixelated commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

The first benchmark (#7) put the translated codec's decode at 20-140x slower than moq's hand-written TS, with costs at every integer width. This runs before the wide integer spike so that spike measures only the 64-bit representation.

Approach

Each change was kept only where just ts bench measured a win:

  1. Number-only small ints (rt.ts). u8-u32 checks, div, rem, shifts, the checked/saturating/wrapping families, leading_ones/leading_zeros (Math.clz32), and from_be_bytes/to_be_bytes stay in numbers rather than converting to bigint. 64- and 128-bit byte conversions go a 32-bit word at a time, still bigint. ByteVec.reserve copies its whole storage instead of viewing it.
  2. Inline from_be_bytes (library.rs). An array written in place, u32::from_be_bytes([0, 0, a & 0x3f, b]), lowers to ((a & 0x3f) << 8) | b, with no Uint8Array.of. Any other argument is read uncopied, which drops rt.copy from the varint path.
  3. Indexed byte patterns (body.rs). let [a, b] = chunk on bytes reads chunk[0] and chunk[1] instead of destructuring a Uint8Array through its iterator.

Split out: pooled byte storage (rt.bytes) is now draft #24. It changes what a consumer sees, since a view's buffer may hold other bytes, so it waits on the maintainer.

Measured and dropped:

  • A non-allocating Result. As an upper bound, a single mutable Ok cell made no Ok objects at all. It measured no faster in Node and 0-5% in Bun, so Result keeps its shape. The README's Design section says so.
  • One frozen object per fieldless variant ({ tag: "Short" }). Neutral in Bun, slower in Node (8-byte varint decode went from 124 to 185 ns).
  • ? returning the incoming Err only changes error paths, and no workload takes one. Not done.
  • A typed rt.copy per Copy type. No benchmarked path copies after change 2: the only rt.copy left is in version negotiation. Not done.

Before and after

ns per item: the median of 15 interleaved samples. Every number below comes from one run, with main's translation loaded as the A/A route. The host has 32 threads shared with other agents' builds, and the load average was 13-16 during the run. A control run with a true A/A put the noise at 3.8% (Node), 6.9% (Bun), and 4.1% (Chrome) at the median workload, and at 19%, 26%, and 29% at most.

decode Node before → after (÷ hand-written) Bun Chromium
varint/quic/1B 72 → 64 (17) 78 → 66 (16) 64 → 53 (18)
varint/quic/2B 354 → 111 (25) 376 → 136 (28) 325 → 84 (24)
varint/quic/4B 410 → 114 (19) 439 → 135 (22) 312 → 80 (17)
varint/quic/8B 673 → 154 (11) 713 → 235 (18) 543 → 111 (11)
varint/leading-ones/1B 127 → 61 (16) 127 → 65 (15) 99 → 49 (17)
varint/leading-ones/2B 495 → 111 (25) 511 → 137 (28) 363 → 81 (23)
varint/leading-ones/4B 638 → 119 (18) 689 → 145 (23) 512 → 83 (17)
varint/leading-ones/8B 1130 → 187 (10) 1186 → 230 (17) 910 → 111 (8.3)
frames/video/lite-06 855 → 271 (1.7) 983 → 377 (2) 718 → 212 (1.8)
frames/audio/lite-06 831 → 261 (1.9) 957 → 383 (2.1) 702 → 200 (1.9)
group/lite-06 3199 → 1188 (0.48) 2885 → 1068 (0.42) 2700 → 906 (0.49)
frames/video/lite-07 1183 → 272 (1.7) 1317 → 391 (2.1) 951 → 203 (1.9)
frames/audio/lite-07 1141 → 289 (1.9) 1278 → 393 (2.1) 908 → 209 (1.9)
group/lite-07 3295 → 1156 (0.47) 3068 → 1048 (0.41) 2921 → 904 (0.51)
encode Node before → after (÷ hand-written) Bun Chromium
varint/quic/1B 62 → 50 (1.3) 59 → 59 (0.74) 48 → 48 (1.1)
varint/quic/2B 197 → 90 (2.4) 159 → 81 (1.2) 121 → 82 (1.9)
varint/quic/4B 199 → 97 (1.7) 223 → 83 (1.2) 174 → 87 (1.6)
varint/quic/8B 529 → 198 (4.4) 559 → 170 (2.1) 375 → 178 (3.7)
varint/leading-ones/1B 168 → 87 (2.2) 191 → 68 (1.1) 140 → 77 (1.9)
varint/leading-ones/2B 260 → 137 (2.7) 265 → 106 (1.7) 216 → 125 (2.4)
varint/leading-ones/4B 276 → 169 (2.9) 318 → 108 (1.7) 243 → 123 (2.9)
varint/leading-ones/8B 441 → 200 (3.1) 510 → 167 (2.4) 403 → 250 (5.9)
frames/video/lite-06 628 → 344 (0.14) 646 → 299 (0.29) 540 → 333 (0.046)
frames/audio/lite-06 657 → 367 (0.17) 592 → 287 (0.28) 568 → 319 (0.05)
group/lite-06 3032 → 2243 (0.18) 2823 → 1832 (0.25) 3412 → 1952 (0.066)
frames/video/lite-07 743 → 409 (0.25) 801 → 353 (0.35) 655 → 461 (0.084)
frames/audio/lite-07 666 → 422 (0.2) 739 → 348 (0.33) 734 → 339 (0.058)
group/lite-07 4229 → 2435 (0.23) 3515 → 1977 (0.26) 3225 → 2871 (0.098)

Each change against the one before it, from A/B runs (the translation before the change loaded as A/A), measured before the split and unaffected by it:

  • 1, number-only ints: 2-4x on every workload in every engine. Node varint/leading-ones/8B decode went from 1140 to 237, and group/lite-06 decode from 2719 to 1216.
  • 2, inline from_be_bytes: Node varint/quic/2B decode 170 → 103, frames lite-07 decode 480 → 291; Chrome 144 → 113 and 439 → 255. In Bun the varints stayed within noise.
  • 3, indexed patterns: Bun frames decode 513 → 430, 518 → 444, 500 → 427, 514 → 421, and Bun leading-ones/8B encode 265 → 170. Neutral in V8.

The remaining decode gap

Gap = translated − hand-written, in ns per item, split by two ablations of the final translation, each run against the unmodified translation as A/A:

  • views makes every byte view twice; the slowdown is what the views cost.
  • bigint decodes varints into numbers, which is exact for these values; the speedup is what bigint costs.

rest is what is left. When it is negative, moq's own route pays that much overhead that the translation does not. The ablations ran at load averages of 11-23, so a column can be off by the noise above. Chromium, moq's browser target:

decode hand-written translated gap views bigint rest
varint/quic/1B 3 53 50 19 16 15
varint/quic/2B 4 84 80 71 -2 11
varint/quic/4B 5 80 75 75 0 1
varint/quic/8B 10 111 101 87 0 13
varint/leading-ones/1B 3 49 46 26 16 4
varint/leading-ones/2B 4 81 78 81 -4 1
varint/leading-ones/4B 5 83 78 67 -2 13
varint/leading-ones/8B 13 111 98 90 0 8
frames/video/lite-06 116 212 96 165 32 -101
frames/audio/lite-06 106 200 94 146 32 -84
group/lite-06 1851 906 -945 298 44 -1287
frames/video/lite-07 109 203 94 148 28 -82
frames/audio/lite-07 112 209 98 143 29 -74
group/lite-07 1786 904 -882 437 33 -1352
Node and Bun

Node v24.21.0

decode hand-written translated gap views bigint rest
varint/quic/1B 4 64 60 36 18 6
varint/quic/2B 4 111 106 72 5 29
varint/quic/4B 6 114 108 63 2 43
varint/quic/8B 14 154 140 89 -2 53
varint/leading-ones/1B 4 61 57 26 13 18
varint/leading-ones/2B 5 111 107 64 7 36
varint/leading-ones/4B 7 119 112 74 4 34
varint/leading-ones/8B 19 187 168 94 47 28
frames/video/lite-06 157 271 114 139 49 -74
frames/audio/lite-06 137 261 124 135 50 -61
group/lite-06 2491 1188 -1303 251 29 -1584
frames/video/lite-07 157 272 115 158 62 -104
frames/audio/lite-07 156 289 134 155 62 -84
group/lite-07 2463 1156 -1307 233 49 -1589

Bun 1.4.2

decode hand-written translated gap views bigint rest
varint/quic/1B 4 66 62 47 4 11
varint/quic/2B 5 136 131 128 -6 9
varint/quic/4B 6 135 129 127 -4 6
varint/quic/8B 13 235 221 178 55 -12
varint/leading-ones/1B 4 65 60 48 4 8
varint/leading-ones/2B 5 137 133 130 -3 5
varint/leading-ones/4B 6 145 138 136 -4 6
varint/leading-ones/8B 13 230 217 169 35 13
frames/video/lite-06 187 377 190 190 134 -134
frames/audio/lite-06 181 383 202 206 135 -139
group/lite-06 2524 1068 -1456 449 27 -1932
frames/video/lite-07 184 391 207 180 140 -114
frames/audio/lite-07 189 393 204 198 133 -127
group/lite-07 2584 1048 -1536 360 20 -1916

Workload by workload:

  • Varints (8-28x hand-written). Most of the gap is byte views. A varint makes one view at 1 byte, three at 2-4 bytes, and four at 8 bytes, through split_first's rest, split_first_chunk's chunk and rest, and lo @ ... Each costs about 20-25 ns in V8 and 40-50 in JavaScriptCore. moq's Cursor reads through an offset and makes none. bigint costs 13-18 ns at 1 byte in V8, and 35-55 ns at 8 bytes in Node's leading-ones form and in Bun. What is left is within noise in Chrome and Bun; Node leaves 30-50 ns at 2-8 bytes in this run.
  • Frames (1.7-2.1x). Views (6 per frame header, about 135-205 ns) and bigint (30-140 ns) more than cover the gap. Each frame header costs moq's Reader.tryDecode another 60-140 ns.
  • Group (0.41-0.51x, already faster). moq's route awaits its async Reader per message, which costs more than the translation's 10 views and bigint.

Impact

  • Translator output: from_be_bytes of an array literal is inline shifts, and of anything else reads it uncopied. [u8; N] patterns index rather than destructure.
  • rt.ts: no exported signature changed. Int keeps its interface.
  • The new lang cases cover from_be_bytes on array literals for each narrow width (sign bytes, constant zeros, the u32 top bit) and through a reference. The existing differential cases cover the number-only runtime.

Alternatives

  • Exceptions or bare Ok values for Result: dropped, since making no Ok objects at all measured no faster.
  • Uint8Array.of for byte literals in encode (as rt.bytesOf with stores): 10-25% in Bun on the paths that use it, mixed in Node. Left out as encode-only and not clearly a win.

Follow-ups

  • Draft perf(ts): pool small byte storage in shared ArrayBuffers #24: pooled byte storage, with its decision and numbers.
  • New quest Cheaper byte slices: views are now most of decode's gap.
  • Wide integer spike is unblocked. The bigint column above is its baseline.
  • just ts bench hung once in Chrome: the renderer sat idle and never posted a result, while a rerun passed. Recommend a timeout in main.ts's chrome() so a hang fails loudly.
  • An error-path workload, such as a frame split across reads that retries on Short, would price ? reusing the incoming Err.

🤖 Generated with Claude Code

https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn

(written by Opus 5.5)

kixelated and others added 8 commits October 23, 2026 22:53
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
@kixelated kixelated changed the title Faster translator runtime perf(ts): number-only small ints, direct byte reads, and pooled byte storage Oct 24, 2026
kixelated and others added 2 commits October 24, 2026 00:53
Pooling makes a byte view's buffer hold other bytes, which consumers can see,
so it moves to its own PR for the maintainer to decide. This keeps the README
note that a non-allocating Result measured no faster.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
@kixelated kixelated changed the title perf(ts): number-only small ints, direct byte reads, and pooled byte storage perf(ts): number-only small ints, inline from_be_bytes, and indexed byte patterns Oct 24, 2026
@kixelated
kixelated marked this pull request as ready for review October 24, 2026 08:27
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-24T08:29:06.201392Z c6cd134 Draft marked ready
🔒 Security Review 🔄 Running since 2026-10-24T08:27:20.119512Z c6cd134 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@kixelated

Copy link
Copy Markdown
Collaborator Author

Merging at c6cd134.

  • Pooled byte storage (rt.bytes) was split out by a revert commit and is now draft perf(ts): pool small byte storage in shared ArrayBuffers #24, with its decision and numbers. This PR keeps number-only small ints, inline from_be_bytes, and indexed byte patterns.
  • Reran just check, just test, and the bench: the before/after and decode-gap tables in the description are from the post-split code. quest/m2/ts-byte-views.md no longer names rt.bytes.
  • Review: Codex's code review of c6cd134 completed with no findings. CI passed. quest check --base origin/main --branch quest/m2/ts-runtime-perf passed. A trial merge with main (feat(ts): translate loops that only return leaves #14) was clean and just ts test passed.

(written by Opus 5.5)

@kixelated
kixelated merged commit b160d40 into main Oct 24, 2026
1 check passed
kixelated added a commit that referenced this pull request Oct 24, 2026
* chore(quest): claim quest/m2/ts-runtime-perf.md

Quest-Claim: e73e630002303297

* perf(ts): number-only small ints, inline from_be_bytes, indexed byte patterns

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn

* perf(ts): pool small byte storage in shared ArrayBuffers

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn

* quest: finish the runtime quest, plan cheaper byte slices

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn

* test(ts): refresh the fmt snapshot for inline from_be_bytes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn

* revert(ts): split pooled byte storage out of this PR

Pooling makes a byte view's buffer hold other bytes, which consumers can see,
so it moves to its own PR for the maintainer to decide. This keeps the README
note that a non-allocating Result measured no faster.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn

* quest: byte views no longer name rt.bytes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn

* perf(ts): pool small byte storage in shared ArrayBuffers

Restores the pooling split out of #15 for the maintainer's decision: Vec<u8>
storage and [x; N] byte arrays up to 1 KiB share 8 KiB ArrayBuffers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn

* perf(ts): pool only ByteVec storage, not [u8; N] arrays

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn

* perf(ts): pool [u8; N] byte arrays again, as ByteVec storage is

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn

* test(ts): regenerate snapshots for pooled byte arrays

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn

---------

Co-authored-by: Luke Curley <432854+kixelated@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated deleted the quest/m2/ts-runtime-perf branch October 27, 2026 20:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant