Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions quest/m0/ci/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,7 @@ The historical samples locate candidates; they are not controlled benchmarks.

- [Read-only pull-request CI](/quest/m0/ci/read-only-secrets.md) - write secrets and publisher identities reach only protected branches and tags, never a pushed branch
- [Land main through quest land](/quest/m0/ci/land.md) - one self-hosted runner tests every queued PR as a squash chain and fast-forwards `main` to the longest passing prefix
- [Gate each diff in under five minutes](/quest/m0/ci/five-minutes.md) - `just check` and `just test` each finish within a worker's 5-minute prompt cache, admission wait included
- [Measure the parallel frontend](/quest/m0/ci/nightly-levers.md) - a pinned nightly's parallel frontend, adopted only if the numbers beat its risks; Cranelift is ruled out
- [Verify the R2 Rust cache](/quest/m0/ci/mbx-r2-verify.md) - measure the hosted Nightly jobs' hits, restore time, and expiry once the bump lands
- [Verify the Nix cache](/quest/m0/ci/nix-cache-verify.md) - confirm CI and releases use `nix.moq.dev` and retire Cachix
Expand Down
50 changes: 50 additions & 0 deletions quest/m0/ci/five-minutes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# [L] Gate each diff in under five minutes

## Goal

`just check` and `just test` each finish in under 5 minutes of wall time for
every diff-scoped run on the desktop with warm caches, counting time queued in
`quest gate` admission. A worker agent then runs its own gate inside its
5-minute prompt-cache window. Explicit full runs (`--all`, nightly, release)
are exempt; work moved out of the diff-scoped gate goes to the merge queue or
nightly with its coverage kept.

## Plan

Decided 2026-10-29 (user): sub-agents run with a 5-minute prompt cache and the
coordinating session with a 1-hour one
([kixelated/quest#276](https://github.com/kixelated/quest/pull/276)). A worker
that waits longer than 5 minutes on one command pays to rewrite its whole
context when it resumes. This refines the [epic](/quest/m0/ci/README.md)'s
2026-10-25 decision that wall clock for its own sake is a non-goal: the bound
comes from the agent cache, and CPU time stays the measure for everything else.

Decided 2026-10-29 (user): time queued in `quest gate` admission counts.
kixelated/quest redesigns the gate so admission no longer outlasts a worker's
cache; this quest owns the gated commands' own run time.

Baseline, 2026-10-22 to 10-29: successful agent-run gates on the desktop, from
Claude Code transcripts. Times include admission wait and load from concurrent
agents, so this is an observation, not a controlled benchmark.

| Command | Runs | Median | p75 | Over 5 min |
| --- | ---: | ---: | ---: | ---: |
| `just check` | 1,961 | 0.9 min | 4.5 min | 24% |
| `just test` | 459 | 2.2 min | 8.5 min | 40% |

Every column leaves out the 177 check and 44 test runs that reached the
agent's 10-minute command limit, so the percentiles and shares over 5 minutes
understate the tail.

- First find which routes and steps exceed 5 minutes on representative diffs,
separating admission wait, compile, and test time.
- Prefer the epic's levers, such as [slow tests](/quest/m0/ci/slow-tests.md),
[hakari](/quest/m0/ci/hakari.md), and cache hits, and tighter diff selection,
over moving coverage. Move a check to the queue or nightly only under the
epic's rule: evidence and an explicit coverage tradeoff for each.
- Split any large single lever into its own quest.

## Related

- [Halve the slowest tests' CPU](/quest/m0/ci/slow-tests.md) - the largest test-time lever
- [Land main through quest land](/quest/m0/ci/land.md) - the queue gate that keeps checks moved out of the local gate
Loading