From 32469ac1b0fe8800cf2095fd2810ad135b37bcb0 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 7 Nov 2026 08:15:45 -0800 Subject: [PATCH 1/4] chore(quest): claim quest/m0/ci/hakari-followups.md Quest-Claim: ae460ed919d6e046 From 5d415850603afd3b1dddbed3571549711e1c8280 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 7 Nov 2026 08:16:30 -0800 Subject: [PATCH 2/4] docs(quest): record the 2026-11-07 Nightly as incomplete cache evidence Co-Authored-By: Claude Opus 5.5 --- quest/m0/ci/hakari-followups.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/quest/m0/ci/hakari-followups.md b/quest/m0/ci/hakari-followups.md index e28b8895483..60964a1eafb 100644 --- a/quest/m0/ci/hakari-followups.md +++ b/quest/m0/ci/hakari-followups.md @@ -73,6 +73,16 @@ bypassed, 5 remote failures, about 12m0s avoided; several builds hit the 5 GiB download budget. It failed only on the `moq-uring` socket-queue benchmark's 60 s test timeout, as the baseline did. +Checked 2026-11-07: the next scheduled run, +[42377867523](https://github.com/moq-dev/moq/actions/runs/42377867523/job/128419087851) +(2026-11-07), cancelled the job at the 60-minute timeout while compiling +`moq-ffi` for its tenth workload, so it is incomplete evidence. Its nine +printed summaries total 343 hits and 572 misses (37.49%), 126 not looked up, +3 bypassed, 1 remote failure, about 8m49s avoided; four builds hit the 5 GiB +download budget. The baseline took 57m24s and the 2026-11-06 run 58m38s, so +the job already sat near its limit before the merge. The two post-merge +timeouts are not attributable to the exclusions on their own. + Public API: none. Wire: none. ## Waiting From bfaf0543cbc5a1e3a9835caac17711ca2600f87b Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 11 Nov 2026 22:58:34 -0800 Subject: [PATCH 3/4] quest: hakari cache measurement waits on the maintainer's call after #8522 Co-Authored-By: Claude Opus 5.5 --- quest/m0/ci/hakari-followups.md | 27 ++++++++++++++++----------- 1 file changed, 16 insertions(+), 11 deletions(-) diff --git a/quest/m0/ci/hakari-followups.md b/quest/m0/ci/hakari-followups.md index 3e119afd04d..b7591d7b7f1 100644 --- a/quest/m0/ci/hakari-followups.md +++ b/quest/m0/ci/hakari-followups.md @@ -87,14 +87,19 @@ Public API: none. Wire: none. ## Waiting -- moq-dev/moq: collect a second complete post-merge Nightly's mbx hit rate and - compiler-time fields for `Nightly (Rust benchmark tests)`, beside the - baseline and 2026-11-06 numbers in the Plan. If the cache - cost is material, present the measured tradeoff to the maintainer; recommend - keeping the narrow final exclusions unless the measurements justify another - design. That job timed out on five of the six scheduled runs from - 2026-11-05 to 11-10 and on every one since 11-07. It is now two jobs, - `Nightly (Rust benchmark tests)` and `Nightly (Rust application benchmark - tests)`, each one release build grouping several crates, so none of their - builds match a baseline build's crates and features; compare only builds - that do. +- Maintainer: decide how to finish without a second complete like-for-like + sample. `Nightly (Rust benchmark tests)` timed out on every scheduled run + from 2026-11-07 to 11-11 (runs 42377867523, 42487375545, 42659882487, + 42842827718, 43041909115). [moq-dev/moq#8522](https://github.com/moq-dev/moq/pull/8522) + (merged 2026-11-12) splits it into `Nightly (Rust benchmark tests)` and + `Nightly (Rust application benchmark tests)`, each one release build + grouping several crates, so no later build matches a baseline build's + crates and features. Options: settle on the one complete sample plus the + 2026-11-07 partial (recommended: both show a 7 to 11 point conditional + hit-rate drop with avoided compiler time near the baseline, which is not a + material cost against a ~58 minute job; keep the narrow exclusions and + delete this quest); take a fresh before/after on the new jobs by reverting + the exclusions on a branch and comparing its mbx summaries with a `main` + run of the same commit; or compare per-build summaries that the cancelled + runs printed for builds matching the baseline's, if their logs are still + retrievable (the 2026-11-11 run's log failed to download). From 4853ebbe1b9da50b4ea7a880396adc6ee5458924 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 11 Nov 2026 23:09:13 -0800 Subject: [PATCH 4/4] quest: finish hakari follow-ups on the evidence in hand Co-Authored-By: Claude Opus 5.5 --- quest/m0/ci/hakari-followups.md | 105 -------------------------------- quest/m3/audit-cost.md | 5 +- 2 files changed, 3 insertions(+), 107 deletions(-) delete mode 100644 quest/m0/ci/hakari-followups.md diff --git a/quest/m0/ci/hakari-followups.md b/quest/m0/ci/hakari-followups.md deleted file mode 100644 index b7591d7b7f1..00000000000 --- a/quest/m0/ci/hakari-followups.md +++ /dev/null @@ -1,105 +0,0 @@ -# [S] Measure the Linux cache effect of native media exclusions - -## Goal - -Scheduled Linux jobs record the cache hit rate and build-time effect of -removing native media requirements from unrelated test builds. Bring a -material cache regression to the maintainer before declaring the tradeoff -settled. - -## Plan - -The implementation in -[moq-dev/moq#7713](https://github.com/moq-dev/moq/pull/7713) (merged -2026-11-05) adds the derived trybuild-edge guard and excludes `pipewire`, -`libspa`, and `clang-sys` from the workspace-hack's final output. Other -transitive features remain unified. Linux x86_64 and ARM test graphs of -`moq-json` no longer reach PipeWire or libclang. An x86_64 nextest run with -shared compiler caching bypassed passes with native media pkg-config probes -refused and an invalid libclang path. The original manifests fail under the -same restriction. - -Local Cargo 1.98.1 unit graphs for `moq-json`, `moq-video --features pipewire`, -and `moq-audio --features pipewire`, each built with `--tests`, have 531 distinct -third-party recursive compilation identities both before and after. The JSON -build loses 30 units; video retains 501 total units. Four media identities -change. These are graph identities, not mbx hit rates or CPU measurements. - -Additional Linux x86_64 unit graphs at the merged main baseline -`ac0f04e5dcca9a9a07d15135dc2e904215c494f3` compare `moq-video --features vaapi` -and `moq-video --features pipewire`, each with `--tests`. VAAPI-only drops from -478 to 456 total units; PipeWire stays at 491. Recursive third-party identities -shared by those builds drop from 453 to 428, and their union grows from 483 to -486. Excluding `clang-sys` lets VAAPI's bindgen 0.70 request `clang_6_0`, while -PipeWire's bindgen 0.72 requests `clang_11_0`; the library and its two build-script -units split into three extra compilation identities. The earlier three-build -sample does not cover this cost. Record this VAAPI/PipeWire split when interpreting -the scheduled cache results. These graphs measure compilation identities, not -CPU cost or cache hit rates; the post-merge measurements still decide whether -the tradeoff is material. - -Use the same scheduled Linux job before and after this PR's merge, beginning -with `Nightly (Rust benchmark tests)`, and retain its mbx cache summaries and -available compiler-time fields. Compare the last scheduled `main` run before -the merge with the first two after it. Failed or cancelled builds and missing summaries -are incomplete evidence, not zero cache hits. Record source/build differences -that prevent attributing a change to these exclusions. - -The 2026-11-02 scheduled run was the first pre-merge candidate: -[run 41483654426](https://github.com/moq-dev/moq/actions/runs/41483654426), -[Rust benchmark tests](https://github.com/moq-dev/moq/actions/runs/41483654426/job/125408205554). -Its 12 mbx object-cache summaries total 986 hits, 721 misses, 323 not -looked up, 8 bypassed, and 11 remote failures. The conditional hit rate is -57.76% (`986 / (986 + 721)`); this excludes not-looked-up and bypassed units. -The summaries report estimated compiler time avoided, not measured CPU totals. -The 5 GiB download budget was exhausted during the first build; remote -failures and mixed build phases limit attribution. - -Checked 2026-11-05 (/quest-audit): the last scheduled run before the merge, -[run 41852517544](https://github.com/moq-dev/moq/actions/runs/41852517544) -(2026-11-04), replaces that candidate as the baseline. Its -[Rust benchmark tests](https://github.com/moq-dev/moq/actions/runs/41852517544/job/126652296572) -job failed (test failures and a denied `moq-ffi` test warning) but printed -its mbx summaries. The first post-merge run, -[42050683248](https://github.com/moq-dev/moq/actions/runs/42050683248/job/127325640293), -cancelled that job at the 60-minute timeout, so it is incomplete evidence. - -Checked 2026-11-06 (/quest-unblock): the baseline's 10 summaries total 333 hits and 359 misses -(48.12%), 401 not looked up, 12 bypassed, 2 remote failures, about 12m28s of -compiler time avoided. The first complete post-merge run, -[42236107954](https://github.com/moq-dev/moq/actions/runs/42236107954/job/127954016736) -(2026-11-06), totals 427 hits and 615 misses (40.98%), 20 not looked up, 4 -bypassed, 5 remote failures, about 12m0s avoided; several builds hit the 5 GiB -download budget. It failed only on the `moq-uring` socket-queue benchmark's 60 s -test timeout, as the baseline did. - -Checked 2026-11-07: the next scheduled run, -[42377867523](https://github.com/moq-dev/moq/actions/runs/42377867523/job/128419087851) -(2026-11-07), cancelled the job at the 60-minute timeout while compiling -`moq-ffi` for its tenth workload, so it is incomplete evidence. Its nine -printed summaries total 343 hits and 572 misses (37.49%), 126 not looked up, -3 bypassed, 1 remote failure, about 8m49s avoided; four builds hit the 5 GiB -download budget. The baseline took 57m24s and the 2026-11-06 run 58m38s, so -the job already sat near its limit before the merge. The two post-merge -timeouts are not attributable to the exclusions on their own. - -Public API: none. Wire: none. - -## Waiting - -- Maintainer: decide how to finish without a second complete like-for-like - sample. `Nightly (Rust benchmark tests)` timed out on every scheduled run - from 2026-11-07 to 11-11 (runs 42377867523, 42487375545, 42659882487, - 42842827718, 43041909115). [moq-dev/moq#8522](https://github.com/moq-dev/moq/pull/8522) - (merged 2026-11-12) splits it into `Nightly (Rust benchmark tests)` and - `Nightly (Rust application benchmark tests)`, each one release build - grouping several crates, so no later build matches a baseline build's - crates and features. Options: settle on the one complete sample plus the - 2026-11-07 partial (recommended: both show a 7 to 11 point conditional - hit-rate drop with avoided compiler time near the baseline, which is not a - material cost against a ~58 minute job; keep the narrow exclusions and - delete this quest); take a fresh before/after on the new jobs by reverting - the exclusions on a branch and comparing its mbx summaries with a `main` - run of the same commit; or compare per-build summaries that the cancelled - runs printed for builds matching the baseline's, if their logs are still - retrievable (the 2026-11-11 run's log failed to download). diff --git a/quest/m3/audit-cost.md b/quest/m3/audit-cost.md index d38c5a371e3..66b47e5d504 100644 --- a/quest/m3/audit-cost.md +++ b/quest/m3/audit-cost.md @@ -83,8 +83,9 @@ tests [moq-dev/moq#7600](https://github.com/moq-dev/moq/pull/7600), build time incremental state. In a warm edit the final binaries are about half the unit time. On the hybrid host use `perf stat --no-scale`. - Rows built with dev-dependencies shift across the cargo-hakari - workspace-hack ([moq-dev/moq#6985](https://github.com/moq-dev/moq/pull/6985)). - Feature unification belongs to [hakari follow-ups](/quest/m0/ci/hakari-followups.md). + workspace-hack ([moq-dev/moq#6985](https://github.com/moq-dev/moq/pull/6985)), + which excludes native media crates from unification + ([moq-dev/moq#7713](https://github.com/moq-dev/moq/pull/7713)). ### Check first: artifact size