diff --git a/quest/m0/ci/README.md b/quest/m0/ci/README.md index 881b908829a..609dac871aa 100644 --- a/quest/m0/ci/README.md +++ b/quest/m0/ci/README.md @@ -91,13 +91,12 @@ Decided 2026-10-25 (maintainer): across moq's many CI and release builds compiles once and is reused through the cache, and each run builds and tests only what its diff can affect, measured in runner CPU time. Wall clock for its own sake and replacing cargo are non-goals. mbx already caches each compilation, -so the work is making builds identical enough to hit it (one crane dependency -layer, flags in `.cargo/config.toml`, cargo-hakari), and auditing the package -selection `just ci` already does. Nix builds run sandboxed and cannot reach R2: -kixelated/rust2's -[Nix build benchmark](https://github.com/kixelated/rust2/blob/main/quest/m1/bench-nix.md) -decides between crane with mbx, crate2nix, and cargo with mbx after the crane -fixes land. +so the work is making builds identical enough to hit it (flags in +`.cargo/config.toml`, cargo-hakari), and auditing the package selection `just ci` +already does. Nix builds run sandboxed and cannot reach R2; decided 2026-10-26 +from kixelated/rust2's +[Nix build benchmark](https://github.com/kixelated/rust2/pull/63), they move to +crate2nix, whose per-crate derivations `nix.moq.dev` caches. Decided 2026-10-25 (maintainer), from a profile of moq-net: monomorphized generics dominate its build (53% of the library's CPU), and its test target @@ -118,7 +117,7 @@ The historical samples locate candidates; they are not controlled benchmarks. - [Bump to the manifest fix](/quest/m0/ci/mbx-bump.md) - condition: an mbx release contains jdx/mr-boxington#798; then bump the Rust cache's pin - [Verify the R2 Rust cache](/quest/m0/ci/mbx-r2-verify.md) - measure the hosted Nightly jobs' hits, restore time, and expiry once the bump lands - [Verify the Nix cache](/quest/m0/ci/nix-cache-verify.md) - confirm CI and releases use `nix.moq.dev` and retire Cachix -- [Share crane's dependency layer](/quest/m0/ci/crane-layers.md) - Nix packages build their dependencies once, and a workspace edit rebuilds none of them +- [Build Nix packages with crate2nix](/quest/m0/ci/crate2nix.md) - one derivation per crate cached in nix.moq.dev, with `Cargo.nix` regenerated by CI so contributors never need Nix - [Unify dependency features](/quest/m0/ci/hakari.md) - a cargo-hakari workspace-hack so every job compiles each dependency alike and mbx hits across jobs - [Audit CI's package selection](/quest/m0/ci/select-audit.md) - how often the queue runs everything, and whether each rule that forces it is needed - [First shared FFI release](/quest/m0/ci/nightly-ffi-reuse.md) - condition: the backport merges and the next `moq-ffi-v*` tag runs `Release FFI`; verify the wrappers diff --git a/quest/m0/ci/crane-layers.md b/quest/m0/ci/crane-layers.md deleted file mode 100644 index af44263f843..00000000000 --- a/quest/m0/ci/crane-layers.md +++ /dev/null @@ -1,48 +0,0 @@ -# [S] Share crane's dependency layer - -## Goal - -moq's Nix packages build their dependencies once, in one crane dependency layer that -every package reuses as its `cargoArtifacts`. Editing a workspace crate, kio -included, rebuilds no dependency, and `nix build` of a second package after the -first compiles only that package's own crates plus any dependency whose resolved -features differ from the layer's. Record in the PR how many crates each case -compiles, before and after, and which dependencies still differ. - -## Plan - -Decided with the maintainer on 2026-10-25, as the first step of making each crate -configuration compile once across moq's CI and release builds, measured in runner -CPU time. Today `nix/overlay.nix` builds a separate dependency layer per package, -and any change to that package's features, `cargoExtraArgs`, or `RUSTFLAGS` -rebuilds all of its roughly 330 dependencies. - -- One dependency layer, passed to every package as `cargoArtifacts`. Cargo - fingerprints each dependency by the features its own invocation resolves, and - a `-p` build resolves only that package's, so a union layer is reused only - where a package resolves the same features. Build the layer from the packages' - own invocations (each package's `cargoExtraArgs`, in one derivation) so every - package finds its exact compilations, and count what each package adds. - [Unifying dependency features](/quest/m0/ci/hakari.md) later makes those - resolutions identical. -- Move moq-relay's `-C force-frame-pointers=yes` into `.cargo/config.toml` so - every build, Nix or cargo, compiles with the same flags and shares mbx keys. - Frame pointers cost little on x86_64; measure aarch64 too. -- The dependency layer copies kio's real source into crane's dummy tree, because - `[patch.crates-io] kio` points registry crates at the workspace member. That - makes every kio edit rebuild every dependency. Restructure it so kio builds in - the workspace layer, for example by giving the dummy tree a stub kio that - exposes its API, or by building the crates that depend on the patched kio in - the workspace layer. - -kixelated/rust2's -[Nix build benchmark](https://github.com/kixelated/rust2/blob/main/quest/m1/bench-nix.md) -then compares this layering with crane plus mbx, crate2nix, and cargo with mbx, -and moq adopts the winner in a follow-up. - -Public API: none. Release artifacts: built with frame pointers on every target, -not only moq-relay. - -## Related - -- [Unify dependency features](/quest/m0/ci/hakari.md) - makes every package resolve the layer's features diff --git a/quest/m0/ci/crate2nix.md b/quest/m0/ci/crate2nix.md new file mode 100644 index 00000000000..7a1fdc2aaee --- /dev/null +++ b/quest/m0/ci/crate2nix.md @@ -0,0 +1,64 @@ +# [M] Build moq's Nix packages with crate2nix + +## Goal + +moq's Nix packages build one derivation per crate through crate2nix instead of crane, +so `nix.moq.dev` serves every compiled crate to any runner and a lockfile bump or a +workspace edit rebuilds only the crates it changes. Contributors without Nix are +unaffected: cargo stays the build they use, and nothing they run needs Nix. Record +in the PR the instructions or CPU time for a clean build, a second package, the +moq-net edit, and a lockfile bump, before and after. + +## Plan + +Decided with the maintainer on 2026-10-26, from kixelated/rust2's +[Nix build benchmark](https://github.com/kixelated/rust2/pull/63), in billions of +instructions. Each scenario builds moq-relay, except the second package (moq-cli +after moq-relay); the total sums the four. + +| | crane | crane with mbx | crate2nix | crate2nix, 1 codegen unit | cargo with mbx | +| --- | ---: | ---: | ---: | ---: | ---: | +| clean | 3,681 | 3,377 | 4,736 | 3,695 | 3,378 | +| second package | 5,926 | 3,945 | 5,951 | 4,902 | 3,924 | +| moq-net edit | 1,365 | 1,351 | 1,008 | 753 | 1,276 | +| lockfile bump | 3,681 | 1,123 | 672 | 496 | 1,113 | +| total | 14,756 | 9,894 | 12,461 | 9,846 | 9,692 | + +Neither crate2nix column matches moq's `[profile.release]`, which every other +column builds with: `codegen-units = 1` and fat LTO. buildRustCrate reads no Cargo +profile. The benchmark gave it 16 codegen units, and the separate run used +buildRustCrate's default of 1; neither ran LTO, which every crane and cargo build +pays at the final link, edits and bumps included. The 29% cold-build penalty +vanishes at one codegen unit, so it came from that mismatch. + +crate2nix was still cheapest on the edits and lockfile bumps CI repeats, and it +caches through the Nix store with no builder configuration. crane with mbx needs +`extra-sandbox-paths` and a persistent store on every builder. + +- **Match the release profile first.** crate2nix's builds match `[profile.release]` + before any comparison: one codegen unit, fat LTO on the final link, stripped + symbols, the per-package overrides, and cargo's opt-level 0 for build scripts and + proc macros where buildRustCrate uses 3. The before and after numbers come from + those matched builds; LTO will raise crate2nix's edit and bump costs above the + table's. +- **Explore first.** Prototype crate2nix on moq-relay and moq-cli before switching + every package, and keep crane until the matched numbers hold on moq's own CI. +- **No Nix required.** The generated `Cargo.nix` is committed so Nix users build + without generating it. Any change to a manifest, the lockfile, or a crate's + targets can make it stale, so CI regenerates it on every PR and fails when it + differs. A bot commits the regenerated file to the PR's branch whenever it can + push there, Renovate's included; for a fork that blocks maintainer edits, the + failure carries the update as a patch to `git apply`. The PR's manifests are + untrusted, so generation runs without write access, and the job that pushes + takes only `Cargo.nix` from it. +- **Flags.** Move moq-relay's `-C force-frame-pointers=yes` into + `.cargo/config.toml` so Nix and cargo builds use the same flags; measure aarch64. +- Replaces the crane layering quest, which only improved the second-package case: + with crane, a lockfile bump still rebuilds every dependency. + +Public API: none. Release artifacts: built by crate2nix, with frame pointers on +every target. + +## Related + +- [Unify dependency features](/quest/m0/ci/hakari.md) - lets packages share more crate derivations diff --git a/quest/m0/ci/hakari.md b/quest/m0/ci/hakari.md index a70298dab26..24afd699c2b 100644 --- a/quest/m0/ci/hakari.md +++ b/quest/m0/ci/hakari.md @@ -31,7 +31,7 @@ feature set in another job is a miss, however warm the cache. - Unifying features means some packages build features they don't use. Measure whether that costs more CPU than the cache hits save; target, profile, and `RUSTFLAGS` differences still split keys, and Nix's flags are unified in - [crane-layers](/quest/m0/ci/crane-layers.md). + [crate2nix](/quest/m0/ci/crate2nix.md). - Start from the hit report in [Verify the R2 Rust cache](/quest/m0/ci/mbx-r2-verify.md), which lists the compilations that miss only because their features or flags differ.