diff --git a/quest/m0/ci/tlc-cost.md b/quest/m0/ci/tlc-cost.md new file mode 100644 index 00000000000..0728361d3b7 --- /dev/null +++ b/quest/m0/ci/tlc-cost.md @@ -0,0 +1,93 @@ +# [M] TLC runs cost fewer core-seconds and share the host + +## Goal + +The `tla` recipes and their callers check every TLA+ config with the same +verdict as before for measurably fewer core-seconds, a TLC run leaves room on +the host for other gated commands, and no run, killed ones included, leaves +its working directory behind. Agents check the one config a change touches; +whole directories are Nightly's. + +## Plan + +Measured 2026-11-12 on the maintainer's agent hosts: + +- An ungated `just test tla rs/moq-net` matched one agent session's whole + usage: 21.4 CPU-hours over 1.8 hours. +- A gated `just test tla rs/moq-net/verify/route` took about 24 CPU-hours in + 53 minutes at about 31 cores, while 35 other gated commands queued behind + it, the oldest for 53 minutes. `quest gate` admits by load average, so one + run that takes every core stalls everything else. +- Another session ran TLC by hand, outside the recipes, for 2.5 hours: + 15.6 CPU-hours and 17.8 GB, on nixpkgs `tlaplus` 1.7.4 and a Java 8 JRE. +- Nightly splits these configs across five runners at about an hour each on + 4 workers. +- A killed run's `-metadir` under `/tmp` still held 26 GB. + +Decided 2026-11-12 (maintainer, /quest-plan): + +- ✅ Cap TLC's workers in `rs/justfile` and `js/justfile`, which hard-code + `-workers auto`. Recommended: 4, which the `verify-*` skills' times + already assume and Nightly's runners have; the + [native checker quest](/quest/m1/tla-native-checker.md) measured + core-seconds near flat from 2 to 24 workers, so the cap costs wall time, + not CPU. +- ✅ TLC 1.8 on a modern JDK. The recipes moved to `nixpkgs#tlaplus18` in + [moq-dev/moq#8499](https://github.com/moq-dev/moq/pull/8499); the 1.7.4 run + above bypassed them. Leave no doc, skill, or script pointing at the 1.7.4 + package, `nixpkgs#tlaplus`. +- ✅ Replace `SumOver` in `rs/moq-net/verify/route/RouteHandoff.tla`, a + recursive function over `SUBSET S` that TLC re-evaluates on every use, with + the Community Modules' `FoldSet` (`FiniteSetsExt`), folding `f(x)` over + `S`. Not `SumSet({f(x) : x \in S})`, which sums each value once and so drops + two nodes' equal counts. TLC evaluates `FoldSet` natively only through the + Java override in `CommunityModules-deps.jar`, which nixpkgs does not + package: take it from a pinned release, put it on TLC's classpath in both + recipes, and confirm TLC's startup log reports the override loaded. + [moq-dev/moq#8564](https://github.com/moq-dev/moq/pull/8564) rewrites + `SumOver` as an Apalache fold; whichever lands second keeps one fold for the + model. +- ✅ Add `SYMMETRY` where sound. No config checks a `PROPERTY`, so every one + checks safety only, which symmetry preserves. A permutation is sound only if + it fixes every constant the config gives, such as `Links`, `Origin`, or + `Start`, and the spec never orders those values or does arithmetic on them; + so turn + a fully interchangeable set, such as `Relays` or `Endpoints`, into model + values and permute it, or the members no constant singles out. That can cut + a state space by up to n!, though `Endpoints = {"a", "b"}` with + `Start = "a"` gains nothing. Say per config whether it applies and why. +- ✅ Pass `-checkpoint 0`, and make sure no metadir outlives its run. TLC never + removes one it was killed in, and a signal handler cannot run on SIGKILL. + Recommended: put the metadir under the checkout's `.tmp/`, remove it when + the recipe exits, and at start remove the metadirs earlier runs left there + that no live run owns, so a checkout that outlives killed runs does not + collect them. Mark ownership so a concurrent run's metadir survives, such + as a lock each run holds on a file in its metadir, which the kernel drops + when the run dies. +- ✅ Agents check one config locally: the `verify-*` skills, `REVIEW.md`, and + the `js/*/verify/` READMEs name single configs, and only Nightly runs + `just test tla` over a directory. +- Rejected: Apalache, which is no cheaper here and ran out of memory on the + route model ([moq-dev/moq#8564](https://github.com/moq-dev/moq/pull/8564)); + waiting for the Rust checker, which the m1 quest + spikes and this complements. + +Not decided: the route-inductive work in progress adds a refinement check, +`RefinesClaims`, to the simulated `Blind.cfg`, where every sampled trace +re-checks it. Recommended: check a refinement on the smallest exhaustive config +that exercises it, where each distinct state is checked once, unless measuring +shows the simulated config is cheaper. + +Verification: every config keeps its verdict, and every exhaustive config +without `SYMMETRY` keeps its distinct-state count (the `verify-*` skills list them). +Measure each config's core-seconds through `quest gate` at equal worker counts +before and after, report the table in the PR, and update the times the skills +and Nightly's `tla` matrix comment quote; rebalance that matrix if its legs +change a lot. + +Public API: none. Wire: none. + +## Related + +- [A Rust TLA+ checker runs the models locally if it agrees with TLC](/quest/m1/tla-native-checker.md) - a faster engine for the same specs; its TLC baseline moves with this +- [The route rules hold from any state, not only the runs TLC reaches](/quest/m1/route-inductive.md) - rewrites `SumOver` and adds the refinement check named above diff --git a/quest/m1/api-reference-cache.md b/quest/m1/api-reference-cache.md new file mode 100644 index 00000000000..c50aa47656e --- /dev/null +++ b/quest/m1/api-reference-cache.md @@ -0,0 +1,33 @@ +# [XS] The JS API reference regenerates only when its inputs change + +## Goal + +`doc/.vitepress/api/index.ts` skips TypeDoc when a hash of everything the +reference is built from matches the last run's, in the docs build and the dev +server alike, so a JS-module check that changes no input spends no TypeDoc +time. + +## Plan + +Measured 2026-11-12 on one of the maintainer's agent hosts: the reference +regenerated on every JS-module check (`doc`'s `check` script ends in +`vitepress build`, which never reuses it), 50 runs in 4.8 hours at a median +of 74 CPU-seconds, about 6 CPU-hours a day. + +Decided 2026-11-12 (maintainer, /quest-plan): ✅ skip when a hash of the +inputs is unchanged. + +- The hash replaces the `--cached` modification-time stamp, which every fresh + checkout defeats; one rule serves build and dev server. Keep + `MOQ_API_REFERENCE=regenerate` only if something still needs it. +- Inputs are what can change the output: the published packages' sources and + `package.json`s, their tsconfigs and every config those reach through + `extends`, such as `js/tsconfig.json`, the generator under + `doc/.vitepress/api/`, and the TypeDoc versions and patches in `bun.lock` + and `patches/`. A missed input serves a stale reference, so derive the + list from what `generate` reads rather than by hand where possible. +- Recommended: keep the output keyed by its hash in a cache outside the + checkout, written atomically, so each new agent worktree reuses it rather + than regenerating once per worktree. Measure the hit rate across worktrees. + +Public API: none. Wire: none. diff --git a/quest/m1/tla-native-checker.md b/quest/m1/tla-native-checker.md index 79c749fff31..1b105446786 100644 --- a/quest/m1/tla-native-checker.md +++ b/quest/m1/tla-native-checker.md @@ -63,4 +63,5 @@ Public API: none. Wire: none. ## Related +- [TLC runs cost fewer core-seconds and share the host](/quest/m0/ci/tlc-cost.md) - makes TLC itself cheaper, which moves this quest's baseline - [Every delivery order of a small mesh runs on the real route layer](/quest/m2/cluster-routes/route-exhaustive-sim.md) - the other spike, which checks the Rust instead of a faster model