Skip to content

perf(ts): pool small byte storage in shared ArrayBuffers - #24

Merged
kixelated merged 17 commits into
mainfrom
quest/m2/ts-byte-pool
Oct 24, 2026
Merged

kixelated merged 17 commits into
mainfrom
quest/m2/ts-byte-pool

Conversation

@kixelated

@kixelated kixelated commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

V8 gives a lone small byte array its own ArrayBuffer, about a microsecond apiece: past 64 bytes when the array is made, and below that the first time something views it. Encoding a size-prefixed message pays that twice. This was change 4 of #15, split out because it changes what a consumer sees.

Approach

ByteVec's storage and [u8; N] arrays come from bytes(n) in rt.ts. Up to 1 KiB, it hands out views of 8 KiB ArrayBuffer slabs, 8-byte aligned, as Node's Buffer pool does. No byte is handed out twice. A [x; N] of u8 lowers to rt.bytes(N), per decision 2.

Decisions

  1. Keep pooled byte storage?

    • (a) Keep, pooling ByteVec and [x; N] arrays.
    • (b) Revert to one buffer per array: close this PR.
    • ✅ (c) Pool only ByteVec, leaving [x; N] arrays unpooled. Superseded by decision 2.
  2. Merge (c) now that it is measured? Pooling only ByteVec keeps the whole group decode cost but loses most of the encode win: the [u8; 9] size prefix in Encoder was carrying it. The earlier "Bun varint decode 10-30% slower" was the bench's route slot, not pooling: Bun runs identical code 17-38% slower as translated than as A/A. With that bias removed, both variants cost group decode about the same.

    change vs main, slot bias removed ByteVec only (c) full pool (a)
    group encode, Node -5%, -1% (noise) -46%, -53%
    group encode, Bun -5%, -9% -43%, -43%
    group encode, Chrome -28%, -22% -64%, -53%
    group decode, Node -7%, 0% 0%, +2%
    group decode, Bun +12%, +16% +8%, +14%
    group decode, Chrome +4%, +8% +4%, +5%
    varint and frames decode, Bun and Chrome within ±5%, up to +11% in Bun (noise) within ±5%

    Each pair is lite-06, lite-07.

    • ✅ (a) Restore full pooling and merge (recommended): it pays the same group decode cost, and it halves group encode in every engine.
    • (b) Merge (c) as is: about a quarter off Chrome's group encode, for Bun's 12-16% group decode cost.
    • (c) Close the PR.

    The maintainer chose (a).

Measured

ns per item: the median of 15 interleaved samples from just ts bench on a 32-thread host shared with other builds and another agent's bench, at load averages of 13-33, with one spike to 63. Main's translation was loaded in place of A/A, once forward (pooled as translated, main as A/A) and once reversed. A slot bias b makes forward f = p·b and reversed r = b/p, so the change is p = √(f/r), with the two forward runs of (c) averaged geometrically. A true A/A control put the noise at 6.8% (Node), 12% (Bun), and 3.0% (Chrome) at the median workload, and 22%, 73%, and 53% at most. In Chrome and Bun, pooled group decode was slower in all six measurements of (c).

ByteVec only (c), main → pooled:

engine workload op run 1 run 2 reversed A/A control corrected
Node group/lite-06 encode 3390 → 3100 1520 → 2050 2080 → 1680 -6% -5%
Node group/lite-06 decode 2510 → 2010 1570 → 1520 1240 → 1210 -7% -7%
Node group/lite-07 encode 3530 → 3200 2910 → 2080 2110 → 2550 -18% -1%
Node group/lite-07 decode 2170 → 2300 1430 → 1460 1290 → 1240 +8% -0%
Bun varint/quic/1B decode 85.3 → 97.5 65.4 → 86.5 87.1 → 78 +38% +5%
Bun varint/quic/2B decode 167 → 225 145 → 173 161 → 150 +21% +9%
Bun varint/quic/4B decode 190 → 233 141 → 184 178 → 148 +18% +3%
Bun varint/quic/8B decode 310 → 344 244 → 330 304 → 243 +17% -1%
Bun varint/leading-ones/1B decode 78.2 → 105 68.7 → 75.3 94.8 → 71.8 +18% -4%
Bun varint/leading-ones/2B decode 175 → 219 149 → 167 181 → 146 +21% -2%
Bun varint/leading-ones/4B decode 187 → 236 142 → 197 176 → 163 +32% +11%
Bun varint/leading-ones/8B decode 294 → 329 245 → 283 285 → 252 -2% +0%
Bun frames/video/lite-06 decode 560 → 572 437 → 455 466 → 443 +6% -1%
Bun frames/audio/lite-06 decode 502 → 589 458 → 451 454 → 428 +14% +1%
Bun group/lite-06 encode 3000 → 2670 2470 → 2410 2650 → 2580 +2% -5%
Bun group/lite-06 decode 1570 → 1710 1180 → 1450 1300 → 1410 +35% +12%
Bun frames/video/lite-07 decode 489 → 541 464 → 438 478 → 435 +2% -4%
Bun frames/audio/lite-07 decode 525 → 573 432 → 425 460 → 460 +1% +2%
Bun group/lite-07 encode 3390 → 3020 2110 → 1920 2750 → 2520 -25% -9%
Bun group/lite-07 decode 1580 → 2320 1170 → 1420 1310 → 1330 +13% +16%
Chromium varint/quic/1B decode 52.6 → 53.6 54.4 → 53.2 59.7 → 54.4 +113% -5%
Chromium varint/quic/2B decode 89.9 → 98.1 86.1 → 87.2 90.8 → 87.5 -5% +1%
Chromium varint/quic/4B decode 93.9 → 92.9 80.6 → 81.6 86.3 → 84.6 -0% -1%
Chromium varint/quic/8B decode 124 → 127 111 → 114 112 → 114 -1% +2%
Chromium varint/leading-ones/1B decode 54.7 → 66.5 53 → 51.7 52.2 → 50.2 +12% +2%
Chromium varint/leading-ones/2B decode 98.2 → 96.3 83.3 → 84.6 91.5 → 83.8 +2% -4%
Chromium varint/leading-ones/4B decode 100 → 103 86.8 → 86.1 92.8 → 90.6 +1% -1%
Chromium varint/leading-ones/8B decode 133 → 131 113 → 111 118 → 113 +0% -3%
Chromium frames/video/lite-06 decode 216 → 215 217 → 201 220 → 213 -1% -4%
Chromium frames/audio/lite-06 decode 218 → 220 219 → 204 219 → 222 +0% -1%
Chromium group/lite-06 encode 2520 → 1870 2240 → 1830 2620 → 1730 +18% -28%
Chromium group/lite-06 decode 950 → 999 948 → 1000 957 → 984 +1% +4%
Chromium frames/video/lite-07 decode 248 → 241 206 → 205 217 → 220 +1% -0%
Chromium frames/audio/lite-07 decode 240 → 243 215 → 205 214 → 232 +1% +3%
Chromium group/lite-07 encode 3300 → 2480 2440 → 1950 2720 → 2160 +4% -22%
Chromium group/lite-07 decode 979 → 1090 922 → 964 908 → 984 +2% +8%

Full pool (a), main → pooled:

engine workload op forward reversed corrected
Node group/lite-06 encode 1480 → 790 1500 → 814 -46%
Node group/lite-06 decode 1130 → 1130 1160 → 1160 +0%
Node group/lite-07 encode 1880 → 942 2090 → 926 -53%
Node group/lite-07 decode 1130 → 1150 1130 → 1150 +2%
Bun varint/quic/1B decode 61 → 75.9 96.9 → 71.7 -4%
Bun varint/quic/2B decode 133 → 158 180 → 167 +5%
Bun varint/quic/4B decode 132 → 157 204 → 157 -4%
Bun varint/quic/8B decode 235 → 257 293 → 254 -3%
Bun varint/leading-ones/1B decode 60.4 → 73.2 87.6 → 73.7 +1%
Bun varint/leading-ones/2B decode 129 → 157 186 → 157 +1%
Bun varint/leading-ones/4B decode 135 → 149 174 → 156 -1%
Bun varint/leading-ones/8B decode 231 → 251 274 → 238 -3%
Bun frames/video/lite-06 decode 392 → 404 469 → 470 +2%
Bun frames/audio/lite-06 decode 404 → 409 545 → 501 -4%
Bun group/lite-06 encode 2030 → 1170 2520 → 1420 -43%
Bun group/lite-06 decode 1140 → 1260 1360 → 1430 +8%
Bun frames/video/lite-07 decode 401 → 417 467 → 469 +2%
Bun frames/audio/lite-07 decode 395 → 411 472 → 474 +2%
Bun group/lite-07 encode 2050 → 1230 2740 → 1460 -43%
Bun group/lite-07 decode 1110 → 1280 1280 → 1440 +14%
Chromium varint/quic/1B decode 49.7 → 50 52.8 → 51.8 -1%
Chromium varint/quic/2B decode 78.3 → 83.6 87 → 87.2 +3%
Chromium varint/quic/4B decode 73.8 → 74.8 84.5 → 84.8 +1%
Chromium varint/quic/8B decode 103 → 103 115 → 113 -1%
Chromium varint/leading-ones/1B decode 47 → 50.4 52.1 → 53.5 +5%
Chromium varint/leading-ones/2B decode 75.4 → 80 81.9 → 82.3 +3%
Chromium varint/leading-ones/4B decode 83.9 → 83.3 88.5 → 88.2 -1%
Chromium varint/leading-ones/8B decode 105 → 109 112 → 116 +4%
Chromium frames/video/lite-06 decode 193 → 200 213 → 215 +2%
Chromium frames/audio/lite-06 decode 194 → 198 215 → 219 +2%
Chromium group/lite-06 encode 1910 → 721 2480 → 861 -64%
Chromium group/lite-06 decode 931 → 945 936 → 994 +4%
Chromium frames/video/lite-07 decode 198 → 212 228 → 215 +0%
Chromium frames/audio/lite-07 decode 199 → 207 232 → 213 -2%
Chromium group/lite-07 encode 2220 → 922 2120 → 1140 -53%
Chromium group/lite-07 decode 883 → 923 921 → 972 +5%

Sanity run of the merged head (05807ac), with main's translation as a --route (#37), forward only, at load average 27. Translated → main ns per item, lite-06 and lite-07:

engine group encode group decode
Node 1170 / 2580, 1170 / 2320 1510 / 1440, 1300 / 1610
Bun 2510 / 4780, 2200 / 4690 2760 / 2070, 3220 / 2520 (±33-66%)
Chromium 798 / 2760, 1030 / 3620 924 / 811, 792 / 746

Encode still halves or better everywhere, and decode still costs some in Bun and Chrome.

Impact

  • Behavior at the API edge: a ByteVec's storage or a [u8; N] up to 1 KiB may share an ArrayBuffer with other bytes. A consumer reads its view through byteOffset and byteLength, and copies it before transferring its buffer. The README says so. rust2-ts has no MIGRATE.md.
  • rt.ts exports bytes(n), which generated [u8; N] code calls.

Alternatives

  • Allocating ByteVec off V8's heap from the start: any lone buffer costs V8 the same microsecond, so only sharing one avoids it.
  • Copying a small chunk instead of viewing it: neutral in V8, faster in JavaScriptCore, and risks the same microsecond on any later view.

Follow-ups

  • The bench's translated slot runs 17-38% slower than A/A in Bun on identical code, so any A/B through A/A needs a reversed run. Alternating which route runs first would remove the bias; another PR is fixing it.
  • One Chrome run hung for 80 minutes with no result: #35 now fails the bench when Chrome stalls.
  • Find what group decode pays for pooled storage in JavaScriptCore and V8, since decode allocates little.

🤖 Generated with Claude Code

https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn

(written by Opus 5.5)

kixelated and others added 11 commits October 23, 2026 22:53
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
Pooling makes a byte view's buffer hold other bytes, which consumers can see,
so it moves to its own PR for the maintainer to decide. This keeps the README
note that a non-allocating Result measured no faster.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
Restores the pooling split out of #15 for the maintainer's decision: Vec<u8>
storage and [x; N] byte arrays up to 1 KiB share 8 KiB ArrayBuffers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
@kixelated kixelated changed the title perf(ts): pool small byte storage in shared ArrayBuffers perf(ts): pool small ByteVec storage in shared ArrayBuffers Oct 24, 2026
@kixelated kixelated changed the title perf(ts): pool small ByteVec storage in shared ArrayBuffers perf(ts): pool small byte storage in shared ArrayBuffers Oct 24, 2026
@kixelated
kixelated marked this pull request as ready for review October 24, 2026 19:45
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-24T19:46:23.969139Z 05807ac Draft marked ready
🔒 Security Review ✅ Completed 2026-10-24T19:48:12.265064Z 05807ac Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@kixelated

Copy link
Copy Markdown
Collaborator Author

Summary before merge, at 05807ac:

  • Restores full pooling, per the maintainer's decision (a): [x; N] of u8 lowers to rt.bytes(N) again, so byte arrays up to 1 KiB share the 8 KiB slabs with ByteVec storage. bytes is exported from rt.ts again, and the README notes that a view's .buffer holds other bytes.
  • Merged main first (no rebase). The source change and the regenerated snapshots are separate commits.
  • A sanity run of just ts bench --route main=... confirms the direction: group encode drops by 47-72% in Node, Bun, and Chromium. Group decode is still slower in Bun and Chrome. The numbers are in the body.
  • just check, just test, CI, and quest check --base origin/main pass. Codex found nothing.
  • Follow-up: find what group decode pays for pooled storage in JavaScriptCore and V8.

(written by Opus 5.5)

@kixelated
kixelated merged commit 56e38a4 into main Oct 24, 2026
1 check passed
@kixelated
kixelated deleted the quest/m2/ts-byte-pool branch October 27, 2026 20:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant