From ae90d793c097018cd23d5b528ab1e5407b84be7e Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 31 Oct 2026 13:41:01 -0700 Subject: [PATCH 1/2] quest: plan the land gate's CI follow-ups The 2026-10-31 host-gate cutover for moq.pro and rust2 left three moq items: the gate's just ci stops at the first failure, a nix.moq.dev 5xx is a cache miss, and cheap checks run through quest gate --quick. Co-Authored-By: Claude Opus 5.5 --- quest/m0/ci/README.md | 3 +++ quest/m0/ci/ci-fail-fast.md | 23 +++++++++++++++++++++++ quest/m0/ci/gate-quick.md | 20 ++++++++++++++++++++ quest/m0/ci/nix-cache-5xx.md | 26 ++++++++++++++++++++++++++ 4 files changed, 72 insertions(+) create mode 100644 quest/m0/ci/ci-fail-fast.md create mode 100644 quest/m0/ci/gate-quick.md create mode 100644 quest/m0/ci/nix-cache-5xx.md diff --git a/quest/m0/ci/README.md b/quest/m0/ci/README.md index a9de7b9f6d9..49357b4b895 100644 --- a/quest/m0/ci/README.md +++ b/quest/m0/ci/README.md @@ -129,6 +129,9 @@ The historical samples locate candidates; they are not controlled benchmarks. - [Read-only pull-request CI](/quest/m0/ci/read-only-secrets.md) - write secrets and publisher identities reach only protected branches and tags, never a pushed branch - [Land main through quest land](/quest/m0/ci/land.md) - a host daemon tests small batches of queued PRs once and lands each as its tested squash commit +- [The gate stops at the first failure](/quest/m0/ci/ci-fail-fast.md) - a failing batch frees the land gate in seconds; local `just ci` still runs every lane +- [A cache error is a miss](/quest/m0/ci/nix-cache-5xx.md) - a transient `nix.moq.dev` 5xx no longer fails `just ci` +- [Cheap checks skip the queue](/quest/m0/ci/gate-quick.md) - condition: quest ships `quest gate --quick`; then cheap modules wait at most a minute - [Gate each diff in under five minutes](/quest/m0/ci/five-minutes.md) - `just check` and `just test` each finish within a worker's 5-minute prompt cache, admission wait included - [Measure the parallel frontend](/quest/m0/ci/nightly-levers.md) - a pinned nightly's parallel frontend, adopted only if the numbers beat its risks; Cranelift is ruled out - [Verify the R2 Rust cache](/quest/m0/ci/mbx-r2-verify.md) - confirm the hosted Nightly jobs reuse the previous Nightly, then measure restore time and expiry diff --git a/quest/m0/ci/ci-fail-fast.md b/quest/m0/ci/ci-fail-fast.md new file mode 100644 index 00000000000..785f1e4cb96 --- /dev/null +++ b/quest/m0/ci/ci-fail-fast.md @@ -0,0 +1,23 @@ +# [S] The land gate's just ci stops at the first failure + +## Goal + +When `quest land serve` checks a batch, `just ci` stops every lane as soon as +one module fails and reports that module, so a failing batch frees the gate +in seconds instead of running the remaining modules. Without the gate's +opt-in, `just ci` still runs every lane to the end. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from moq-dev/quest's host-gate cutover for moq.pro and rust2 and its fixer repairs). + +- On 2026-10-31 two moq batches failed `check gh` in 2 seconds (`alert.yml`'s + workflow list named the removed Pullfrog) and still ran 32-40 minutes each + before ejecting. +- `sh/dispatch.sh` deliberately completes every lane "so one unavailable + check cannot hide unrelated failures". Keep that default for local and + nightly runs; add an opt-in the land check sets (an environment variable + or `just ci` flag), and have kixelated/nixos's moq check command pass it. +- On the first failed module, signal the other lanes' process groups the way + the existing timeout path does (`kill -TERM -- -$pid`), then print the + seconds-per-module summary as today. diff --git a/quest/m0/ci/gate-quick.md b/quest/m0/ci/gate-quick.md new file mode 100644 index 00000000000..2e231a3431e --- /dev/null +++ b/quest/m0/ci/gate-quick.md @@ -0,0 +1,20 @@ +# [XS] Cheap checks run through quest gate --quick + +## Goal + +moq's cheap checks, such as `check quest`, the formatters, and markdown, +wait at most a short bounded time for `quest gate` admission instead of +queuing behind agent builds. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from moq-dev/quest's host-gate cutover for moq.pro and rust2 and its fixer repairs). + +- Condition: [moq-dev/quest#324](https://github.com/moq-dev/quest/pull/324) + lands, adding `quest gate --quick` (at most 60 seconds of admission wait), + and moq's flake pins a quest that includes it. +- Cheap checks waited up to an hour behind load + ([#7250](https://github.com/moq-dev/moq/pull/7250), + [#7257](https://github.com/moq-dev/moq/pull/7257)). Mark them in the + dispatcher's module table, not by command name, and leave builds and + test suites under normal admission. diff --git a/quest/m0/ci/nix-cache-5xx.md b/quest/m0/ci/nix-cache-5xx.md new file mode 100644 index 00000000000..52c5841c9f1 --- /dev/null +++ b/quest/m0/ci/nix-cache-5xx.md @@ -0,0 +1,26 @@ +# [S] A nix.moq.dev error does not fail a check + +## Goal + +A transient 5xx from `nix.moq.dev` costs a cache miss, not a failed +`just ci`: the dev shell still evaluates and builds what it could not +substitute. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from moq-dev/quest's host-gate cutover for moq.pro and rust2 and its fixer repairs). + +- On 2026-10-31 one HTTP 500 for a `.narinfo` failed `check flake` and + ejected a seven-PR batch in the host land gate. Nix stops on a failed + path-info lookup even with `fallback = true`, which covers only failed + downloads. +- Prefer fixing it at the source: find what made the cache Worker return 500 + for a lookup and have it answer 404 (a miss) when its bucket read fails. + If Nix still sees 5xx from the edge, retry the lookup in the Worker or + through Nix's `download-attempts`. +- The [Nix cache verification](/quest/m0/ci/nix-cache-verify.md) owns the + release side of the cache. + +## Related + +- [Verify the Nix cache](/quest/m0/ci/nix-cache-verify.md) - releases and bucket sizes From f5d0e227d1f62a6e6f23331e8262ba2acd79fec0 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 31 Oct 2026 21:07:05 -0700 Subject: [PATCH 2/2] quest: drop the fail-fast quest #7466 already finished Co-Authored-By: Claude Opus 5.5 --- quest/m0/ci/README.md | 1 - quest/m0/ci/ci-fail-fast.md | 23 ----------------------- 2 files changed, 24 deletions(-) delete mode 100644 quest/m0/ci/ci-fail-fast.md diff --git a/quest/m0/ci/README.md b/quest/m0/ci/README.md index 49357b4b895..3f4b5e4a3c6 100644 --- a/quest/m0/ci/README.md +++ b/quest/m0/ci/README.md @@ -129,7 +129,6 @@ The historical samples locate candidates; they are not controlled benchmarks. - [Read-only pull-request CI](/quest/m0/ci/read-only-secrets.md) - write secrets and publisher identities reach only protected branches and tags, never a pushed branch - [Land main through quest land](/quest/m0/ci/land.md) - a host daemon tests small batches of queued PRs once and lands each as its tested squash commit -- [The gate stops at the first failure](/quest/m0/ci/ci-fail-fast.md) - a failing batch frees the land gate in seconds; local `just ci` still runs every lane - [A cache error is a miss](/quest/m0/ci/nix-cache-5xx.md) - a transient `nix.moq.dev` 5xx no longer fails `just ci` - [Cheap checks skip the queue](/quest/m0/ci/gate-quick.md) - condition: quest ships `quest gate --quick`; then cheap modules wait at most a minute - [Gate each diff in under five minutes](/quest/m0/ci/five-minutes.md) - `just check` and `just test` each finish within a worker's 5-minute prompt cache, admission wait included diff --git a/quest/m0/ci/ci-fail-fast.md b/quest/m0/ci/ci-fail-fast.md deleted file mode 100644 index 785f1e4cb96..00000000000 --- a/quest/m0/ci/ci-fail-fast.md +++ /dev/null @@ -1,23 +0,0 @@ -# [S] The land gate's just ci stops at the first failure - -## Goal - -When `quest land serve` checks a batch, `just ci` stops every lane as soon as -one module fails and reports that module, so a failing batch frees the gate -in seconds instead of running the remaining modules. Without the gate's -opt-in, `just ci` still runs every lane to the end. - -## Plan - -Decided 2026-10-31 (user, /quest-plan of the follow-ups from moq-dev/quest's host-gate cutover for moq.pro and rust2 and its fixer repairs). - -- On 2026-10-31 two moq batches failed `check gh` in 2 seconds (`alert.yml`'s - workflow list named the removed Pullfrog) and still ran 32-40 minutes each - before ejecting. -- `sh/dispatch.sh` deliberately completes every lane "so one unavailable - check cannot hide unrelated failures". Keep that default for local and - nightly runs; add an opt-in the land check sets (an environment variable - or `just ci` flag), and have kixelated/nixos's moq check command pass it. -- On the first failed module, signal the other lanes' process groups the way - the existing timeout path does (`kill -TERM -- -$pid`), then print the - seconds-per-module summary as today.