Skip to content

chore(bench): move the binding benchmark to rust2 - #6629

Merged
kixelated merged 5 commits into
mainfrom
ffi-bench-rust2
Oct 25, 2026
Merged

kixelated merged 5 commits into
mainfrom
ffi-bench-rust2

Conversation

@kixelated

@kixelated kixelated commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

The custom FFI tooling and its benchmarks are leaving moq for kixelated/rust2, where the UniFFI fork and its perf work now live. The binding benchmark has to measure fork changes there, so moq's copy of the harness and its BoltFFI comparison would only drift.

Approach

Lands after kixelated/rust2#19, which adds just bench-ffi to rust2: it drives these runners at a pinned moq revision, with moq-ffi built against the fork, and runs nightly there. Until it merges, nothing measures the bindings and this PR's links to rust2's bench/ffi/README.md do not resolve.

Deleted here:

  • just bench-ffi, bench/ffi.sh, bench/ffi.py, and its test.
  • just bench-ffi-gen and bench/ffi-gen (the BoltFFI comparison). Its measured tables are archived in rust2's bench/ffi/README.md; this repository's history holds the harness.
  • The nightly's measured Binding benchmark job (its series leave the trend table) and the Generator comparison job.

Kept: every wrapper's runner and recipe (just {go,py,kt,cpp,dart,swift} bench) and the Rust baselines in rs/moq-ffi/src/bench.rs, which rust2 drives. rust2 runs them only at its pinned moq revision, so moq's Nightly still runs each one at head at a 0.05 s budget, unmeasured: just gh nightly bindings (Rust baselines, Go, Python, Kotlin, C++, Dart; fails on a crash or a missing report) replaces the Binding benchmark job, and the macOS Platform job's benchmarks input now smoke-runs just swift bench. bench/README.md describes the runners and points at rust2 for the harness. Docs and quests that named the deleted paths now point at rust2.

Choices made without the maintainer (sub-agent takeover), open to review:

  • Keep the runners running nightly in moq at a token budget, rather than relying on rust2's pinned runs, so a wrapper change that breaks a runner fails here the next night instead of at rust2's next pin bump. The job's cost is mostly the release builds the old Binding benchmark job already paid (52 minutes on 2026-10-22).

Impact

  • No public API or wire change.
  • The binding series leave the nightly trend table at git.moq.dev/bench; rust2's report replaces them.
  • New Nightly group just gh nightly bindings; the Platform workflow's benchmarks input now means a Swift runner smoke run.

Alternatives

  • Keep the harness in moq and have rust2 call just bench-ffi: two harnesses would drift, and a fork change could not be measured before moq pins it.
  • Drop the runners from moq's CI entirely: they would break silently until rust2 bumps its pin.

Follow-ups

None in moq. Bumping rust2's moq pin picks up runner changes made here.

Checks

  • just check: everything it selected passed except check flake, whose github-runner image build needs user namespaces this host refuses (unrelated); the rs test and kt check it cancelled then passed on their own (just rs test -p moq-ffi, just kt check).
  • just gh nightly bindings locally: every Linux runner wrote its report, about 35 minutes with cold release builds.
  • actionlint, shellcheck sh/gh/nightly.sh, python3 sh/gh/nightly.test.py, quest check.
  • Not run locally: the Swift runner (macOS only).

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
kixelated and others added 2 commits October 24, 2026 21:53
# Conflicts:
#	bench/ffi-gen/src/client.rs
rust2 measures them only at its pinned moq revision, so moq's Nightly
runs each runner at a token budget and fails on a crash or a missing
report: `just gh nightly bindings` on Linux, and `just swift bench` in
the macOS Platform job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review October 25, 2026 05:42
@coderabbitai

coderabbitai Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 31 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: f0dc8fdc-3336-47c6-aaf1-c28c6a120d82
Commits

Reviewing files that changed from the base of the PR and between fd9cc8b and 552d1d9.

Files ignored due to path filters: 1
  • bench/ffi-gen/Cargo.lock is excluded by !**/*.lock
Files selected for processing: 38
  • .github/justfile
  • .github/workflows/nightly.yml
  • .github/workflows/platform.yml
  • bench/README.md
  • bench/ffi-gen/Cargo.toml
  • bench/ffi-gen/README.md
  • bench/ffi-gen/boltffi.toml
  • bench/ffi-gen/examples/bench.rs
  • bench/ffi-gen/kt/boltffi/src/Gen.kt
  • bench/ffi-gen/kt/build.gradle.kts
  • bench/ffi-gen/kt/common/Bench.kt
  • bench/ffi-gen/kt/settings.gradle.kts
  • bench/ffi-gen/kt/uniffi/src/Gen.kt
  • bench/ffi-gen/py/bench.py
  • bench/ffi-gen/report.py
  • bench/ffi-gen/run.sh
  • bench/ffi-gen/size.py
  • bench/ffi-gen/src/client.rs
  • bench/ffi-gen/src/lib.rs
  • bench/ffi-gen/uniffi-shim/Cargo.toml
  • bench/ffi-gen/uniffi-shim/src/lib.rs
  • bench/ffi.py
  • bench/ffi.sh
  • bench/ffi.test.py
  • cpp/moq/bench/main.cpp
  • dart/moq/bench/ffi.dart
  • justfile
  • kt/moq/src/jvmTest/kotlin/dev/moq/FfiBench.kt
  • py/moq-rs/benches/ffi.py
  • quest/m2/audit-ci.md
  • quest/m2/audit-deps.md
  • quest/m2/ffi-size-profile.md
  • quest/m2/uniffi-fork.md
  • rs/moq-ffi/src/bench.rs
  • rs/moq-ffi/src/lib.rs
  • sh/gh/nightly.sh
  • sh/gh/nightly.test.py
  • test/wasm/generator.md
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-25T15:06:11.474926Z 552d1d9 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: bc38052

No actionable bugs found in harness removal, remaining runner integration and Nightly/Platform changes. Consolidating measurement tooling while retaining lightweight runner coverage here is a sensible separation; the new group preserves failure reporting and checks each expected Linux output.

Verification: static review only; no builds, tests, benchmarks or macOS execution. The external rust2 migration and its landing prerequisite were not verified within this repository-only review.

Takes main's BENCH_BASE and BENCH_RESULTS names (#6633) in the bindings nightly group.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
@kixelated
kixelated added this pull request to the merge queue Oct 25, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 25, 2026
@kixelated
kixelated added this pull request to the merge queue Oct 25, 2026
Merged via the queue into main with commit a87425e Oct 25, 2026
9 checks passed
@kixelated
kixelated deleted the ffi-bench-rust2 branch October 25, 2026 19:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant