Repository navigation
chore(bench): move the binding benchmark to rust2 - #6629
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
# Conflicts: # bench/ffi-gen/src/client.rs
rust2 measures them only at its pinned moq revision, so moq's Nightly runs each runner at a token budget and fails on a crash or a missing report: `just gh nightly bindings` on Linux, and `just swift bench` in the macOS Platform job. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 31 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed commit: bc38052
No actionable bugs found in harness removal, remaining runner integration and Nightly/Platform changes. Consolidating measurement tooling while retaining lightweight runner coverage here is a sensible separation; the new group preserves failure reporting and checks each expected Linux output.
Verification: static review only; no builds, tests, benchmarks or macOS execution. The external rust2 migration and its landing prerequisite were not verified within this repository-only review.
# Conflicts: # justfile
Takes main's BENCH_BASE and BENCH_RESULTS names (#6633) in the bindings nightly group. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
Problem
The custom FFI tooling and its benchmarks are leaving moq for kixelated/rust2, where the UniFFI fork and its perf work now live. The binding benchmark has to measure fork changes there, so moq's copy of the harness and its BoltFFI comparison would only drift.
Approach
Lands after kixelated/rust2#19, which adds
just bench-ffito rust2: it drives these runners at a pinned moq revision, with moq-ffi built against the fork, and runs nightly there. Until it merges, nothing measures the bindings and this PR's links to rust2'sbench/ffi/README.mddo not resolve.Deleted here:
just bench-ffi,bench/ffi.sh,bench/ffi.py, and its test.just bench-ffi-genandbench/ffi-gen(the BoltFFI comparison). Its measured tables are archived in rust2'sbench/ffi/README.md; this repository's history holds the harness.Kept: every wrapper's runner and recipe (
just {go,py,kt,cpp,dart,swift} bench) and the Rust baselines inrs/moq-ffi/src/bench.rs, which rust2 drives. rust2 runs them only at its pinned moq revision, so moq's Nightly still runs each one at head at a 0.05 s budget, unmeasured:just gh nightly bindings(Rust baselines, Go, Python, Kotlin, C++, Dart; fails on a crash or a missing report) replaces the Binding benchmark job, and the macOS Platform job'sbenchmarksinput now smoke-runsjust swift bench.bench/README.mddescribes the runners and points at rust2 for the harness. Docs and quests that named the deleted paths now point at rust2.Choices made without the maintainer (sub-agent takeover), open to review:
Impact
just gh nightly bindings; the Platform workflow'sbenchmarksinput now means a Swift runner smoke run.Alternatives
just bench-ffi: two harnesses would drift, and a fork change could not be measured before moq pins it.Follow-ups
None in moq. Bumping rust2's moq pin picks up runner changes made here.
Checks
just check: everything it selected passed exceptcheck flake, whosegithub-runnerimage build needs user namespaces this host refuses (unrelated); thers testandkt checkit cancelled then passed on their own (just rs test -p moq-ffi,just kt check).just gh nightly bindingslocally: every Linux runner wrote its report, about 35 minutes with cold release builds.actionlint,shellcheck sh/gh/nightly.sh,python3 sh/gh/nightly.test.py,quest check.🤖 Generated with Claude Code
(Written by Claude Opus 5.5)