diff --git a/quest/m0/ci/README.md b/quest/m0/ci/README.md index 6dd85ee8fcf..4597b0025ae 100644 --- a/quest/m0/ci/README.md +++ b/quest/m0/ci/README.md @@ -94,10 +94,13 @@ only what its diff can affect, measured in runner CPU time. Wall clock for its own sake and replacing cargo are non-goals. mbx already caches each compilation, so the work is making builds identical enough to hit it (flags in `.cargo/config.toml`, cargo-hakari), and auditing the package selection `just ci` -already does. Nix builds run sandboxed and cannot reach R2; decided 2026-10-26 -from kixelated/rust2's -[Nix build benchmark](https://github.com/kixelated/rust2/pull/63), they move to -crate2nix, whose per-crate derivations `nix.moq.dev` caches. +already does. Nix builds run sandboxed and cannot reach R2, so they stay on crane +for packaging only; CI builds with cargo and mbx. Decided 2026-10-26 from +kixelated/rust2's [Nix build benchmark](https://github.com/kixelated/rust2/pull/63), +in billions of instructions over a clean build, a second package, the moq-net edit, +and a lockfile bump: cargo with mbx totals 9,692 and crane 14,756. crate2nix built +to the release profile totals 14,538: it wins only the lockfile bump (1,255 against +crane's 3,681) and costs 11-27% more on the rest. Decided 2026-10-25 (maintainer), from a profile of moq-net: monomorphized generics dominate its build (53% of the library's CPU), and its test target @@ -117,7 +120,6 @@ The historical samples locate candidates; they are not controlled benchmarks. - [Bump to the manifest fix](/quest/m0/ci/mbx-bump.md) - condition: an mbx release contains jdx/mr-boxington#798; then bump the Rust cache's pin - [Verify the R2 Rust cache](/quest/m0/ci/mbx-r2-verify.md) - measure the hosted Nightly jobs' hits, restore time, and expiry once the bump lands - [Verify the Nix cache](/quest/m0/ci/nix-cache-verify.md) - confirm CI and releases use `nix.moq.dev` and retire Cachix -- [Build Nix packages with crate2nix](/quest/m0/ci/crate2nix.md) - one derivation per crate cached in nix.moq.dev, with `Cargo.nix` regenerated by CI so contributors never need Nix - [Unify dependency features](/quest/m0/ci/hakari.md) - a cargo-hakari workspace-hack so every job compiles each dependency alike and mbx hits across jobs - [Replay full-run failures against the selected gate](/quest/m0/ci/select-audit.md) - every dependency edge `just ci` misses is found and fixed with a test - [First shared FFI release](/quest/m0/ci/nightly-ffi-reuse.md) - condition: the backport merges and the next `moq-ffi-v*` tag runs `Release FFI`; verify the wrappers diff --git a/quest/m0/ci/crate2nix.md b/quest/m0/ci/crate2nix.md deleted file mode 100644 index 7a1fdc2aaee..00000000000 --- a/quest/m0/ci/crate2nix.md +++ /dev/null @@ -1,64 +0,0 @@ -# [M] Build moq's Nix packages with crate2nix - -## Goal - -moq's Nix packages build one derivation per crate through crate2nix instead of crane, -so `nix.moq.dev` serves every compiled crate to any runner and a lockfile bump or a -workspace edit rebuilds only the crates it changes. Contributors without Nix are -unaffected: cargo stays the build they use, and nothing they run needs Nix. Record -in the PR the instructions or CPU time for a clean build, a second package, the -moq-net edit, and a lockfile bump, before and after. - -## Plan - -Decided with the maintainer on 2026-10-26, from kixelated/rust2's -[Nix build benchmark](https://github.com/kixelated/rust2/pull/63), in billions of -instructions. Each scenario builds moq-relay, except the second package (moq-cli -after moq-relay); the total sums the four. - -| | crane | crane with mbx | crate2nix | crate2nix, 1 codegen unit | cargo with mbx | -| --- | ---: | ---: | ---: | ---: | ---: | -| clean | 3,681 | 3,377 | 4,736 | 3,695 | 3,378 | -| second package | 5,926 | 3,945 | 5,951 | 4,902 | 3,924 | -| moq-net edit | 1,365 | 1,351 | 1,008 | 753 | 1,276 | -| lockfile bump | 3,681 | 1,123 | 672 | 496 | 1,113 | -| total | 14,756 | 9,894 | 12,461 | 9,846 | 9,692 | - -Neither crate2nix column matches moq's `[profile.release]`, which every other -column builds with: `codegen-units = 1` and fat LTO. buildRustCrate reads no Cargo -profile. The benchmark gave it 16 codegen units, and the separate run used -buildRustCrate's default of 1; neither ran LTO, which every crane and cargo build -pays at the final link, edits and bumps included. The 29% cold-build penalty -vanishes at one codegen unit, so it came from that mismatch. - -crate2nix was still cheapest on the edits and lockfile bumps CI repeats, and it -caches through the Nix store with no builder configuration. crane with mbx needs -`extra-sandbox-paths` and a persistent store on every builder. - -- **Match the release profile first.** crate2nix's builds match `[profile.release]` - before any comparison: one codegen unit, fat LTO on the final link, stripped - symbols, the per-package overrides, and cargo's opt-level 0 for build scripts and - proc macros where buildRustCrate uses 3. The before and after numbers come from - those matched builds; LTO will raise crate2nix's edit and bump costs above the - table's. -- **Explore first.** Prototype crate2nix on moq-relay and moq-cli before switching - every package, and keep crane until the matched numbers hold on moq's own CI. -- **No Nix required.** The generated `Cargo.nix` is committed so Nix users build - without generating it. Any change to a manifest, the lockfile, or a crate's - targets can make it stale, so CI regenerates it on every PR and fails when it - differs. A bot commits the regenerated file to the PR's branch whenever it can - push there, Renovate's included; for a fork that blocks maintainer edits, the - failure carries the update as a patch to `git apply`. The PR's manifests are - untrusted, so generation runs without write access, and the job that pushes - takes only `Cargo.nix` from it. -- **Flags.** Move moq-relay's `-C force-frame-pointers=yes` into - `.cargo/config.toml` so Nix and cargo builds use the same flags; measure aarch64. -- Replaces the crane layering quest, which only improved the second-package case: - with crane, a lockfile bump still rebuilds every dependency. - -Public API: none. Release artifacts: built by crate2nix, with frame pointers on -every target. - -## Related - -- [Unify dependency features](/quest/m0/ci/hakari.md) - lets packages share more crate derivations diff --git a/quest/m0/ci/hakari.md b/quest/m0/ci/hakari.md index 24afd699c2b..14f000c680a 100644 --- a/quest/m0/ci/hakari.md +++ b/quest/m0/ci/hakari.md @@ -30,8 +30,7 @@ feature set in another job is a miss, however warm the cache. (`cargo hakari disable`), and show each still fails on a planted violation. - Unifying features means some packages build features they don't use. Measure whether that costs more CPU than the cache hits save; target, profile, and - `RUSTFLAGS` differences still split keys, and Nix's flags are unified in - [crate2nix](/quest/m0/ci/crate2nix.md). + `RUSTFLAGS` differences still split keys. - Start from the hit report in [Verify the R2 Rust cache](/quest/m0/ci/mbx-r2-verify.md), which lists the compilations that miss only because their features or flags differ.