Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions quest/m0/ci/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,8 @@ The historical samples locate candidates; they are not controlled benchmarks.
- [Build queue entries concurrently](/quest/m0/ci/queue-concurrency.md) - the queue builds two or more entries at once instead of one
- [Run the queue's modules in parallel](/quest/m0/ci/parallel-modules.md) - `just ci` overlaps its selected modules
- [One nextest build in the queue](/quest/m0/ci/one-test-build.md) - `test rs` compiles once instead of three times
- [Share Python check and test preparation](/quest/m0/ci/python-setup.md) - prepare editable bindings once per combined CI pass while preserving standalone commands
- [Parallelize OBS source checks](/quest/m0/ci/obs-parallel.md) - check independent translation units concurrently when OBS is on the critical path
- [Measure nightly compiler levers](/quest/m0/ci/nightly-levers.md) - the parallel frontend and Cranelift, adopted only if the numbers beat their risks
- [Bump to the manifest fix](/quest/m0/ci/mbx-bump.md) - condition: an mbx release contains jdx/mr-boxington#798; then bump the Rust cache's pin
- [Verify the R2 Rust cache](/quest/m0/ci/mbx-r2-verify.md) - measure the hosted Nightly jobs' hits, restore time, and expiry once the bump lands
Expand Down
40 changes: 40 additions & 0 deletions quest/m0/ci/obs-parallel.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# [S] Parallelize OBS source checks

## Goal

When OBS's syntax checks are on a queue entry's critical path after modules run
in parallel, they check independent translation units concurrently after
preparing their shared headers, without losing diagnostics or exceeding the
fanout bound modules share. Failure and cancellation leave no child checks
running.

## Plan

Decided 2026-10-23: module-level parallelism lands first, and this adds
translation-unit fanout inside `obs_compile` only where it still shortens the
entry.

- `sh/obs/compile.sh` checks 20 source and test translation units sequentially.
The complete OBS compile module took 46 seconds in
[run 39996587979](https://github.com/moq-dev/moq/actions/runs/39996587979)
and 92 seconds in
[run 39993252909](https://github.com/moq-dev/moq/actions/runs/39993252909),
including header preparation, which these samples do not separate.
- Once modules overlap, measure whether `obs_compile` ends an entry, such as
one touching only `cpp/obs/`. If it never does, delete this quest.
- Finish `sh/obs/includes.sh` and its generated C++ package preparation before
starting concurrent syntax checks. Preserve all source/test coverage,
compiler flags, per-source diagnostics, and the aggregate failure result.
- Take concurrency from the bound parallel modules and local admission share;
do not create a second scheduling policy or unbounded fanout.
- Verify a failing source makes the command fail, all sources are checked, and
interruption reaps children. Compare serial and parallel runs on the same
inputs, reporting header preparation separately from compilation.

## Required

- [Run the queue's modules in parallel](/quest/m0/ci/parallel-modules.md) - decides whether OBS stays on the critical path, and owns the fanout bound

## Related

- [Share local check capacity](/quest/m0/ci/local-gate.md) - coordinate internal fanout with the shared local command budget
9 changes: 8 additions & 1 deletion quest/m0/ci/parallel-modules.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,14 @@ overlap the Rust build, an estimated 45% off those entries.

- Bound concurrency so modules do not starve each other, and keep each
module's output readable when they interleave.
- A failure still fails the run and names its module.
- A failure still fails the run and names its module, and cancels the
modules still running, so a cheap check that fails ends the entry in its own
time. In [queue run 39996587979](https://github.com/moq-dev/moq/actions/runs/39996587979)
the tooling tests failed after 18 s, but only after about 16 minutes of
modules ahead of them. Verify an injected `just gh test` failure ends the run
before the Rust build finishes, and that cancellation leaves no module
running. Decided 2026-10-23: this replaces a separate quest to move cheap
checks ahead of builds in the serial loop.

Public API: none. Wire: none.

Expand Down
32 changes: 32 additions & 0 deletions quest/m0/ci/python-setup.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# [S] Share Python check and test preparation

## Goal

A CI pass that selects Python prepares its editable bindings and environment
once for both checks and tests. Direct `just py check` and `just py test`
remain self-contained and validate the current checkout.

## Plan

Decided 2026-10-23: remove repeated Python preparation. Keep existing coverage
and the `testing,openh264` feature contract.

- `py/justfile` currently synchronizes the environment and runs `_develop`
in both recipes. `sh/dispatch.sh` separates them with other binding checks
that also build moq-ffi through Cargo with other features.
- In [queue run 39996587979](https://github.com/moq-dev/moq/actions/runs/39996587979),
the second preparation rebuilt the UniFFI generator for 43.70 seconds,
in addition to native build and installation work. Repeated compilation
is observed; the exact cache-invalidation cause is not yet established.
- Share preparation within one invocation rather than trusting an old
environment or a persistent success marker. Keep standalone recipes safe
after Rust, binding, feature, or dependency changes, and stop on preparation
failure. Prefer the smallest recipe/dispatch refactor that achieves this.
- Other languages use different features and build environments. This quest
does not unify every native artifact; running modules concurrently belongs
to [parallel modules](/quest/m0/ci/parallel-modules.md).
- Verify one preparation in the combined pass, standalone checks and tests
from a fresh environment, and rebuilds after relevant input changes.
Measure combined Python and full selected-gate time before and after on
matched inputs and cache state; report build counts and memory use. The
observed 44 seconds is a candidate saving, not a guaranteed improvement.
Loading