Skip to content

feat(ts): time another run's translations with just ts bench --route - #37

Merged
kixelated merged 2 commits into
mainfrom
quest/m2/ts-bench-routes
Oct 24, 2026
Merged

kixelated merged 2 commits into
mainfrom
quest/m2/ts-bench-routes

Conversation

@kixelated

@kixelated kixelated commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

An A/B run of two translator versions meant patching bench/main.ts by hand: #15 and #26 both did, so neither comparison can be rerun.

Approach

just ts bench --route name=dir times the translations in another run's output directory (its target/bench) as one more route:

  • main.ts checks the name (not empty, not an existing route) and that dir/translated/bench.ts and dir/task/model.ts exist, before any work.
  • The route runs both fixtures: the codec's workloads from translated/, and the async handoff from task/, polled like translated.
  • It gets a column in every engine's tables and a row in both bundle-size tables, and the report names it.
  • dir is absolute or relative to ts/, like the moq checkout argument.

The README's Bench section shows the A/B command. quest/m2/ts-wide-fast.md now points at it.

Verified with just ts bench --route self=target/bench, which loads this run's own translations as the route: self differs from translated by 4.3%, 7.0%, and 6.0% at the median workload in Node, Bun, and Chrome, against A/A's 4.2%, 8.3%, and 5.0%. Bad --route values fail at once with a clear message: foo, =x, x=, translated=…, and a directory without the translations.

Consumer: the tree-shaking spike (quest/m2/ts-shake-spike.md) and the wide-integer fast paths.

Impact

  • just ts bench gains a --route name=dir option (just's [arg] long option). The positional moq checkout is unchanged.
  • bench/main.ts gains --route.

Alternatives

  • dir as a single translation directory (bench.ts only): simpler, but the async handoff could not be A/B'd. A run's output directory covers every fixture, now and later.
  • dir as another rust2 checkout, translated by its own driver: truly one command, but that driver must build on this checkout's pinned nightly, which breaks whenever the pins differ.
  • Several --routes: just takes a long option once, and one extra route is all an A/B run needs.

Follow-ups

None.

🤖 Generated with Claude Code

https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn

kixelated and others added 2 commits October 24, 2026 11:40
`--route name=dir` loads the codec and task translations in another run's
target/bench as one more route in every table and size, so an A/B run of two
translator versions is one command.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
@kixelated kixelated changed the title A/B routes in the TS benchmark feat(ts): time another run's translations with just ts bench --route Oct 24, 2026
@kixelated
kixelated marked this pull request as ready for review October 24, 2026 19:07
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-24T19:08:58.807637Z a23cf3c Draft marked ready
🔒 Security Review ✅ Completed 2026-10-24T19:09:35.298408Z a23cf3c Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@kixelated

Copy link
Copy Markdown
Collaborator Author

Merging. Summary:

  • just ts bench --route name=dir times another run's target/bench translations (codec and task) as one more route, with a column in every table and a row in both size tables.
  • Bad values fail before any work. A self-route run stayed within A/A noise.
  • Checks: just check, just test, and just ts bench --route self=target/bench passed locally. CI passed, Codex review had no findings, and quest check --base origin/main is clean.

(written by Opus 5.5)

@kixelated
kixelated merged commit b5ce393 into main Oct 24, 2026
1 check passed
@kixelated
kixelated deleted the quest/m2/ts-bench-routes branch October 27, 2026 20:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant