From b5b43d4db7f9a2107c16c0282587cbeb8fc17751 Mon Sep 17 00:00:00 2001 From: Luke Curley <432854+kixelated@users.noreply.github.com> Date: Mon, 19 Oct 2026 16:24:29 -0700 Subject: [PATCH 1/4] chore: claim compile-time audit Co-Authored-By: GPT-6.1-Sol From 16645bbc577cc80bfe4817ca4d14db1dc898b930 Mon Sep 17 00:00:00 2001 From: Luke Curley <432854+kixelated@users.noreply.github.com> Date: Mon, 19 Oct 2026 17:03:33 -0700 Subject: [PATCH 2/4] docs: record measured compile-time audit baselines Co-Authored-By: GPT-6.1-Sol --- quest/m0/audit-compile-time.md | 106 ++++++++++++++++++++++++++++++--- 1 file changed, 99 insertions(+), 7 deletions(-) diff --git a/quest/m0/audit-compile-time.md b/quest/m0/audit-compile-time.md index 69d1c8d93d8..d7c1e7a6800 100644 --- a/quest/m0/audit-compile-time.md +++ b/quest/m0/audit-compile-time.md @@ -19,10 +19,102 @@ Only a change that needs a maintainer decision (public API, scope) or is independently landable and too large to review alongside the rest becomes a follow-up through `/quest-plan`. -Starting points (2026-10-19): nothing measures compile time. Record clean -`cargo build --timings` for dev and release, an incremental rebuild of -moq-net, and `just check` wall time. The nightly size job logs release -`Build` step durations without comparing them (`sh/size/README.md`). Look -for heavy proc macros, duplicate dependency versions (`deny.toml` only warns), -monomorphization bloat (`cargo llvm-lines`), and default features that pull -heavy dependencies. +Measured 2026-10-19 on source `cdff3c8f952b1d6bd2fd40c3f64eb0fd53b97e95` +(claim commit `b5b43d4db7f9a2107c16c0282587cbeb8fc17751` adds no source). +The source hash above is recorded in the measurement logs; use +`cdff3c8f9` when selecting this baseline. All experiments used the pinned Nix +shell: Rust 1.98.1, LLVM 22.1.8, Linux x86_64, Ryzen 9 5900XT with 16 physical +cores and 32 logical CPUs. Cargo, native build tools, and Nextest were bounded +to two jobs or threads. These are single observations, not regression limits +or evidence of an optimization. + +Clean Cargo builds disabled mbx explicitly with `MBX_DISABLE=1` and used a new +worktree-local `CARGO_TARGET_DIR=.scratch/cold-target`. The dev and release +profiles both started without compiled artifacts for that profile; registry +sources and the Nix shell were already installed. No shared cache was deleted, +and no compiler, profile, feature, or build-metadata setting was changed. The +dev build had a quiet-host window; release and subsequent runs shared the host +with bounded builds in other worktrees. Do not compare their wall times as a +before/after result. + +| Experiment | Process wall time | Result | +| --- | ---: | --- | +| `cargo build --locked --timings` | 274.015 s | Passed; Cargo reported 4m32s | +| `cargo build --locked --release --timings` | 821.054 s | Passed; Cargo reported 13m40s | +| Unchanged default-member dev build | 0.497 s | Passed | +| Warm exact `cargo build --locked -p moq-net --timings` graph | 21.195 s | Passed; setup for the next two runs | +| Unchanged standalone moq-net build | 0.251 s | Passed | +| Standalone moq-net rebuild after a comment-only edit | 8.025 s | Passed; trivial-edit baseline only | +| Ordinary cache-enabled `just check` on that moq-net edit | 842.849 s | Failed at formatting; not a successful full-check baseline | + +Times exclude Nix startup. The first dev observer polled completion every two +seconds, so its process time includes up to two seconds of observation delay; +the remaining process times use a monotonic clock around process completion. +Sampled one-minute load ranged from 0.66 to 2.65 during dev, and 1.32 to 6.14 +during release. The normal `just check` used mbx 1.23.0 without disabling its +cache; it reported 820 hits, 72 misses, seven units not looked up, and four +bypasses. It is a different cache and package-selection experiment from the +clean builds. + +The default-member dev build compiled 681 Cargo units from 545 distinct +package/version pairs. Longest observed units, in seconds: + +| Cargo unit | Dev | Release | +| --- | ---: | ---: | +| moq-cli binary (`moq`) | 14.87 | 342.30 | +| moq-relay binary | 5.13 | 191.80 | +| aws-lc-sys build-script execution | 87.59 | 102.69 | +| moq-boy binary | 3.74 | 101.72 | +| moq-bench binary | 3.76 | 83.95 | +| moq-tokio library | 17.83 | 50.77 | +| moq-relay library | 9.97 | 44.27 | +| moq-mux library | 8.55 | 30.46 | +| str0m library | 9.39 | 24.35 | +| yuv library | 10.75 | 23.07 | +| moq-net library | 10.01 | 21.06 | + +These are elapsed times for overlapping units, not additive critical-path or +CPU times. Final release binary work dominates the release observations, but +these timings do not separate monomorphization, optimization, LTO, and linking. +The existing release profile uses fat LTO and one codegen unit to keep shipping +artifacts compact; no profile change is justified by this run alone. + +The largest dev proc-macro unit was usage-derive at 6.02 seconds; serde_derive +was 2.61 seconds. Fifty-one compiled dependency names had multiple versions. +The three syn versions took 2.99, 2.41, and 2.53 seconds in separate Cargo +units. Their reverse dependency graph is captured, but compatible versions +cannot be assumed interchangeable. The default build omitted iroh, wgpu, +cpal, and OpenH264; it did compile str0m, object_store, and the NVIDIA crates. +Do not disable supported default functionality merely to make a build smaller. + +The ordinary net-edit check selected 28 Rust packages including downstream C, +FFI, GStreamer, and WASM crates. Test compilation took 7m46s; all 7,608 native +tests passed, with 31 skipped, in 369.431 seconds. It then stopped at +`cargo fmt --all --check`: the disposable comment probe had an extra blank +line after the crate attribute. The probe was fully restored and is absent +from the PR. No SLOW, FAIL, or TIMEOUT test verdict occurred. Rustdoc and later +feature checks were not reached, so this run proves neither a complete +`just check` pass nor its successful wall-time baseline. + +Raw process records, load samples, logs, probe patch, resolved dependency graph, +and named Cargo HTML timing reports are retained in this run's worktree under +`.scratch/`. HTML reports were copied to `.scratch/evidence/timings/` before +discarding the isolated build artifacts. `cargo llvm-lines` was unavailable in +the pinned shell. `mbx explain --last` returned no recorded build with session +events enabled, so this run does not attribute cache misses to `COMMIT_ID` or +any other unverified cause. No global cache configuration was changed. + +Remaining work before completing this run: + +1. Record a successful ordinary net-edit `just check` with a formatted probe, + preserving its exact diff, selected packages, cache counters, and host load. +2. Attribute final release binary work with a supported LLVM-lines tool and + compiler/linker profiling before choosing a refactor. A thin-LTO experiment + must compare shipping sizes and relevant runtime workloads on the same + source; changing the release profile is a maintainer decision. +3. Investigate avoidable repeated aws-lc-sys native compilation with actual + cache-miss evidence. Keep the crypto provider and its behavior unchanged + unless a separately justified decision permits changing them. +4. Fix or settle the measured bottlenecks, then create the recurring m1 + successor as required above. The nightly size job already records release + `Build` durations without comparing noisy hosted times (`sh/size/README.md`). From d26e9a3ae076e983f69245e7738731ba906af30b Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 21:47:22 -0700 Subject: [PATCH 3/4] quest(m0): scope the compile-time audit against the CI audit Job-level CI cost (cache hits, aws-lc-sys recompiles per cold job, both rustls backends in the gate, CI timings) belongs to the CI pipeline audit, and dependency deduplication to the dependency audit. Drop the overlapping aws-lc-sys follow-up, link both quests, and stop pointing at local-only scratch evidence. Co-Authored-By: Claude Opus 5.5 --- quest/m0/audit-compile-time.md | 96 +++++++++++++++------------------- 1 file changed, 43 insertions(+), 53 deletions(-) diff --git a/quest/m0/audit-compile-time.md b/quest/m0/audit-compile-time.md index d7c1e7a6800..d5bfd41f97e 100644 --- a/quest/m0/audit-compile-time.md +++ b/quest/m0/audit-compile-time.md @@ -19,23 +19,24 @@ Only a change that needs a maintainer decision (public API, scope) or is independently landable and too large to review alongside the rest becomes a follow-up through `/quest-plan`. -Measured 2026-10-19 on source `cdff3c8f952b1d6bd2fd40c3f64eb0fd53b97e95` -(claim commit `b5b43d4db7f9a2107c16c0282587cbeb8fc17751` adds no source). -The source hash above is recorded in the measurement logs; use -`cdff3c8f9` when selecting this baseline. All experiments used the pinned Nix -shell: Rust 1.98.1, LLVM 22.1.8, Linux x86_64, Ryzen 9 5900XT with 16 physical -cores and 32 logical CPUs. Cargo, native build tools, and Nextest were bounded -to two jobs or threads. These are single observations, not regression limits -or evidence of an optimization. - -Clean Cargo builds disabled mbx explicitly with `MBX_DISABLE=1` and used a new -worktree-local `CARGO_TARGET_DIR=.scratch/cold-target`. The dev and release -profiles both started without compiled artifacts for that profile; registry -sources and the Nix shell were already installed. No shared cache was deleted, -and no compiler, profile, feature, or build-metadata setting was changed. The -dev build had a quiet-host window; release and subsequent runs shared the host -with bounded builds in other worktrees. Do not compare their wall times as a -before/after result. +Measured 2026-10-19 on `cdff3c8f9` in the pinned Nix shell (Rust 1.98.1, +LLVM 22.1.8, Linux x86_64, Ryzen 9 5900XT, 32 logical CPUs), with Cargo, +native build tools, and Nextest bounded to two jobs. These are single +observations, not regression limits or evidence of an optimization. + +Scope: this audit owns per-crate build cost in one build (dev, release, and +incremental). The [CI pipeline audit](/quest/m0/ci/audit.md) owns job-level CI +cost: cache keys and hits, aws-lc-sys recompiling several times per cold job, +both rustls backends in the gate, and `--timings` output in CI. Deduplicating +or upgrading dependencies belongs to the +[dependency audit](/quest/m0/audit-deps.md). + +Clean builds used `MBX_DISABLE=1` and a fresh `CARGO_TARGET_DIR`, with registry +sources and the Nix shell already present; no cache, compiler, profile, or +feature setting was changed. The dev build ran on a quiet host (one-minute load +0.66 to 2.65); release and later runs shared it with bounded builds in other +worktrees (load up to 6.14). Do not compare their wall times as a before/after +result. Times exclude Nix startup. | Experiment | Process wall time | Result | | --- | ---: | --- | @@ -47,15 +48,6 @@ before/after result. | Standalone moq-net rebuild after a comment-only edit | 8.025 s | Passed; trivial-edit baseline only | | Ordinary cache-enabled `just check` on that moq-net edit | 842.849 s | Failed at formatting; not a successful full-check baseline | -Times exclude Nix startup. The first dev observer polled completion every two -seconds, so its process time includes up to two seconds of observation delay; -the remaining process times use a monotonic clock around process completion. -Sampled one-minute load ranged from 0.66 to 2.65 during dev, and 1.32 to 6.14 -during release. The normal `just check` used mbx 1.23.0 without disabling its -cache; it reported 820 hits, 72 misses, seven units not looked up, and four -bypasses. It is a different cache and package-selection experiment from the -clean builds. - The default-member dev build compiled 681 Cargo units from 545 distinct package/version pairs. Longest observed units, in seconds: @@ -78,31 +70,27 @@ CPU times. Final release binary work dominates the release observations, but these timings do not separate monomorphization, optimization, LTO, and linking. The existing release profile uses fat LTO and one codegen unit to keep shipping artifacts compact; no profile change is justified by this run alone. +aws-lc-sys's build script is the largest dev unit; keep the crypto provider +unchanged unless a separately justified decision permits it. The largest dev proc-macro unit was usage-derive at 6.02 seconds; serde_derive -was 2.61 seconds. Fifty-one compiled dependency names had multiple versions. -The three syn versions took 2.99, 2.41, and 2.53 seconds in separate Cargo -units. Their reverse dependency graph is captured, but compatible versions -cannot be assumed interchangeable. The default build omitted iroh, wgpu, -cpal, and OpenH264; it did compile str0m, object_store, and the NVIDIA crates. -Do not disable supported default functionality merely to make a build smaller. - -The ordinary net-edit check selected 28 Rust packages including downstream C, -FFI, GStreamer, and WASM crates. Test compilation took 7m46s; all 7,608 native -tests passed, with 31 skipped, in 369.431 seconds. It then stopped at -`cargo fmt --all --check`: the disposable comment probe had an extra blank -line after the crate attribute. The probe was fully restored and is absent -from the PR. No SLOW, FAIL, or TIMEOUT test verdict occurred. Rustdoc and later -feature checks were not reached, so this run proves neither a complete -`just check` pass nor its successful wall-time baseline. - -Raw process records, load samples, logs, probe patch, resolved dependency graph, -and named Cargo HTML timing reports are retained in this run's worktree under -`.scratch/`. HTML reports were copied to `.scratch/evidence/timings/` before -discarding the isolated build artifacts. `cargo llvm-lines` was unavailable in -the pinned shell. `mbx explain --last` returned no recorded build with session -events enabled, so this run does not attribute cache misses to `COMMIT_ID` or -any other unverified cause. No global cache configuration was changed. +was 2.61 seconds. Fifty-one compiled dependency names had multiple versions; +the three syn versions took 2.99, 2.41, and 2.53 seconds. The default build +omitted iroh, wgpu, cpal, and OpenH264; it did compile str0m, object_store, and +the NVIDIA crates. Do not disable supported default functionality merely to +make a build smaller. + +The cache-enabled net-edit `just check` used mbx 1.23.0 (820 hits, 72 misses, +seven units not looked up, four bypasses) and selected 28 Rust packages, +including downstream C, FFI, GStreamer, and WASM crates. Test compilation took +7m46s; all 7,608 native tests passed, with 31 skipped, in 369.431 seconds. It +then failed `cargo fmt --all --check` on the disposable probe, so rustdoc and +later feature checks never ran. `mbx explain --last` had no recorded build, so +those misses are unattributed. + +Raw logs and Cargo timing reports were local only; re-measure rather than +trusting anything beyond these tables. `cargo llvm-lines` was unavailable in +the pinned shell. Remaining work before completing this run: @@ -112,9 +100,11 @@ Remaining work before completing this run: compiler/linker profiling before choosing a refactor. A thin-LTO experiment must compare shipping sizes and relevant runtime workloads on the same source; changing the release profile is a maintainer decision. -3. Investigate avoidable repeated aws-lc-sys native compilation with actual - cache-miss evidence. Keep the crypto provider and its behavior unchanged - unless a separately justified decision permits changing them. -4. Fix or settle the measured bottlenecks, then create the recurring m1 +3. Fix or settle the measured bottlenecks, then create the recurring m1 successor as required above. The nightly size job already records release `Build` durations without comparing noisy hosted times (`sh/size/README.md`). + +## Related + +- [CI pipeline audit](/quest/m0/ci/audit.md) - owns job-level CI build cost and caching +- [Dependency audit](/quest/m0/audit-deps.md) - owns deduplicating and upgrading dependencies From fddd367c678b9048cdf0c0fb1a2f1fa6574f4c38 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 20 Oct 2026 13:05:24 -0700 Subject: [PATCH 4/4] quest(m0): point the compile-time audit at the m1 dependency audit The m0 dependency audit finished in #6053; its recurring successor lives in m1. Co-Authored-By: Claude Opus 5.5 --- quest/m0/audit-compile-time.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/quest/m0/audit-compile-time.md b/quest/m0/audit-compile-time.md index d5bfd41f97e..b7b9a88ef2c 100644 --- a/quest/m0/audit-compile-time.md +++ b/quest/m0/audit-compile-time.md @@ -29,7 +29,7 @@ incremental). The [CI pipeline audit](/quest/m0/ci/audit.md) owns job-level CI cost: cache keys and hits, aws-lc-sys recompiling several times per cold job, both rustls backends in the gate, and `--timings` output in CI. Deduplicating or upgrading dependencies belongs to the -[dependency audit](/quest/m0/audit-deps.md). +[dependency audit](/quest/m1/audit-deps.md). Clean builds used `MBX_DISABLE=1` and a fresh `CARGO_TARGET_DIR`, with registry sources and the Nix shell already present; no cache, compiler, profile, or @@ -107,4 +107,4 @@ Remaining work before completing this run: ## Related - [CI pipeline audit](/quest/m0/ci/audit.md) - owns job-level CI build cost and caching -- [Dependency audit](/quest/m0/audit-deps.md) - owns deduplicating and upgrading dependencies +- [Dependency audit](/quest/m1/audit-deps.md) - owns deduplicating and upgrading dependencies