Repository navigation
quest: settle macOS capture modes and plan the Nightly benchmark timeouts #8490
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
b650697
quest: settle macOS capture modes and plan the Nightly benchmark time…
kixelated 1dc3c89
Merge remote-tracking branch 'origin/main' into quest-unblock/2026-11-10
kixelated 8b58607
quest: fix the dead audit link and Nightly timeout evidence
kixelated 20ee3ba
quest: compare only matching builds for the hakari cache measurement
kixelated f66b352
quest: the relay sweep completed on 11-06 and 11-07
kixelated File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| # [S] Nightly's relay session benchmarks fit their hour | ||
|
|
||
| ## Goal | ||
|
|
||
| `Nightly (Relay session benchmarks)` (`just gh nightly rust-relay`, a paired | ||
| group that measures this commit and the previous Nightly's) finishes inside | ||
| its 60-minute `timeout-minutes` on a scheduled run and keeps its paired | ||
| regression gate. | ||
|
|
||
| ## Plan | ||
|
|
||
| The job was cancelled at the timeout on the 2026-11-08, 11-09, and 11-10 | ||
| scheduled runs. Unlike the other cancelled jobs it does not hang: | ||
| [the 11-10 run](https://github.com/moq-dev/moq/actions/runs/42842827718/job/129922278379) | ||
| was still finishing `relay_session_join_broadcasts` cells seconds before the | ||
| cancel. The scheduled runs from 11-01 to 11-05 failed inside the workload after | ||
| 14 to 20 minutes, the | ||
| [11-01 job](https://github.com/moq-dev/moq/actions/runs/41313466339/job/124868517800) | ||
| with `Evicted`. The | ||
| [11-06](https://github.com/moq-dev/moq/actions/runs/42236107954) and | ||
| [11-07](https://github.com/moq-dev/moq/actions/runs/42377867523) runs | ||
| completed the sweep in about 25 and 23 minutes (their jobs failed later, at | ||
| `Fail on paired regressions`). Find what grew between those sweeps and the | ||
| 60-minute cancellations from 11-08, then trim it or split the group across | ||
| matrix entries, as | ||
| [the Rust benchmark tests quest](/quest/m0/ci/nightly-rust-test-hour.md) does | ||
| for its job. The sweep still covers both fan-out axes. | ||
|
|
||
| ## Related | ||
|
|
||
| - [Rust benchmark tests fit their hour](/quest/m0/ci/nightly-rust-test-hour.md) - the same timeout, caused by builds |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| # [M] Nightly's Rust benchmark tests fit their hour | ||
|
|
||
| ## Goal | ||
|
|
||
| `Nightly (Rust benchmark tests)` (`just gh nightly rust-test`) finishes well | ||
| inside its 60-minute `timeout-minutes` on a scheduled run, with every benchmark | ||
| it runs today still measured in a release build, one at a time, and reported. | ||
| Splitting it across jobs and cutting what it builds are both in scope. | ||
|
|
||
| ## Plan | ||
|
|
||
| Decided 2026-11-10 (maintainer, /quest-unblock): ✅ split the job and reduce | ||
| its size (rejected: finish the hakari measurement with one run; raise the | ||
| timeout). | ||
|
|
||
| Evidence: the job was cancelled at the timeout on the 2026-11-03, 11-05, | ||
| 11-07, 11-08, 11-09, and 11-10 scheduled runs; the 11-04 and 11-06 runs | ||
| finished in 57 to 59 minutes, and the last to pass, on 11-02, took 34. In | ||
| [the 11-10 run](https://github.com/moq-dev/moq/actions/runs/42842827718/job/129922278502) | ||
| the benchmarks themselves ran for about two minutes in all. The hour went to | ||
| `sh/gh/nightly.sh`'s `rust-test` group compiling a separate release build per | ||
| `nextest` invocation (3 to 5.5 minutes each, about ten of them), and the job | ||
| was still compiling `moq-ffi` when it was cancelled. | ||
|
|
||
| Start from the builds: fewer `nextest` invocations sharing one release build | ||
| where the feature sets allow it, then split what remains across matrix entries | ||
| so no entry nears the hour. Keep `sh/gh/nightly.test.py`'s guarantees (release | ||
| builds, one JUnit report per run, the report publish step) or update them with | ||
| the change. | ||
|
|
||
| ## Related | ||
|
|
||
| - [hakari follow-ups](/quest/m0/ci/hakari-followups.md) - its mbx measurement needs a complete run of this job | ||
| - [Relay session benchmarks fit their hour](/quest/m0/ci/nightly-relay-session-hour.md) - the same timeout in another Nightly job | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| # [S] The JS subscribe-drop benchmark finishes | ||
|
|
||
| ## Goal | ||
|
|
||
| `just js bench subscribe-drop` (`js/net/bench/subscribe-drop.ts`) finishes and | ||
| prints its rows, and the root cause of its hang is fixed, so | ||
| `Nightly (JS benchmarks)` finishes again. | ||
|
|
||
| ## Plan | ||
|
|
||
| `Nightly (JS benchmarks)` was cancelled at its 60-minute timeout on every | ||
| scheduled run from 2026-11-04 to 11-10; it last passed on 11-02, in about | ||
| eight minutes. In | ||
| [the 11-10 run](https://github.com/moq-dev/moq/actions/runs/42842827718/job/129922278808) | ||
| `subscribe-drop.ts` printed its CSV header | ||
| (`publishers,subscribers,groups,drop_us`) and nothing for the job's last 57 | ||
| minutes, and the benchmarks after it never ran. The | ||
| [11-04 run](https://github.com/moq-dev/moq/actions/runs/41852517544/job/126652296690) | ||
| stalled at the same header. Reproduce it locally; if the | ||
| hang is in `@moq/net` rather than the benchmark, a test in the merge gate | ||
| covers it. | ||
|
|
||
| ## Related | ||
|
|
||
| - [session_delivery_compression finishes on lite-07](/quest/m0/lite07-compression-bench-hang.md) - another Nightly benchmark that hangs |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| # [M] session_delivery_compression finishes on moq-lite-07 | ||
|
|
||
| ## Goal | ||
|
|
||
| The `session_delivery_compression` Criterion group (moq-net's `session` | ||
| bench target, `rs/moq-net/benches/session/mod.rs`) finishes every | ||
| `moq-lite-07-wip` cell, and the root cause of the hang it shows is fixed. If | ||
| the hang is in moq-net rather than the benchmark harness, a test reproduces it | ||
| in the merge gate. | ||
|
|
||
| ## Plan | ||
|
|
||
| `Nightly (Rust bench targets)` was cancelled at its 60-minute timeout on the | ||
| 2026-11-02, 11-05, 11-06, 11-09, and 11-10 scheduled runs, and failed without | ||
| timing out on 11-03, 11-04, 11-07, and 11-08. | ||
| In [the 11-10 run](https://github.com/moq-dev/moq/actions/runs/42842827718/job/129922278472) | ||
| every `moq-lite-06` cell finished, then | ||
| `session_delivery_compression/moq-lite-07-wip/relays=1/publishers=1/broadcasts=1/viewers=1` | ||
| printed `Benchmarking` and nothing for the job's last 34 minutes, so the hang | ||
| starts with the first lite-07 cell, and every later bench in the job goes | ||
| unmeasured. Reproduce it locally, bisect against the lite-07 changes | ||
| if it is a regression, and fix it at the root. | ||
|
|
||
| ## Related | ||
|
|
||
| - [JS subscribe-drop finishes](/quest/m0/js-subscribe-drop-hang.md) - another Nightly benchmark that hangs |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.