Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 93 additions & 0 deletions quest/m0/ci/tlc-cost.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# [M] TLC runs cost fewer core-seconds and share the host

## Goal

The `tla` recipes and their callers check every TLA+ config with the same
verdict as before for measurably fewer core-seconds, a TLC run leaves room on
the host for other gated commands, and no run, killed ones included, leaves
its working directory behind. Agents check the one config a change touches;
whole directories are Nightly's.

## Plan

Measured 2026-11-12 on the maintainer's agent hosts:

- An ungated `just test tla rs/moq-net` matched one agent session's whole
usage: 21.4 CPU-hours over 1.8 hours.
- A gated `just test tla rs/moq-net/verify/route` took about 24 CPU-hours in
53 minutes at about 31 cores, while 35 other gated commands queued behind
it, the oldest for 53 minutes. `quest gate` admits by load average, so one
run that takes every core stalls everything else.
- Another session ran TLC by hand, outside the recipes, for 2.5 hours:
15.6 CPU-hours and 17.8 GB, on nixpkgs `tlaplus` 1.7.4 and a Java 8 JRE.
- Nightly splits these configs across five runners at about an hour each on
4 workers.
- A killed run's `-metadir` under `/tmp` still held 26 GB.

Decided 2026-11-12 (maintainer, /quest-plan):

- ✅ Cap TLC's workers in `rs/justfile` and `js/justfile`, which hard-code
`-workers auto`. Recommended: 4, which the `verify-*` skills' times
already assume and Nightly's runners have; the
[native checker quest](/quest/m1/tla-native-checker.md) measured
core-seconds near flat from 2 to 24 workers, so the cap costs wall time,
not CPU.
- ✅ TLC 1.8 on a modern JDK. The recipes moved to `nixpkgs#tlaplus18` in
[moq-dev/moq#8499](https://github.com/moq-dev/moq/pull/8499); the 1.7.4 run
above bypassed them. Leave no doc, skill, or script pointing at the 1.7.4
package, `nixpkgs#tlaplus`.
- ✅ Replace `SumOver` in `rs/moq-net/verify/route/RouteHandoff.tla`, a
recursive function over `SUBSET S` that TLC re-evaluates on every use, with
the Community Modules' `FoldSet` (`FiniteSetsExt`), folding `f(x)` over
`S`. Not `SumSet({f(x) : x \in S})`, which sums each value once and so drops
two nodes' equal counts. TLC evaluates `FoldSet` natively only through the
Java override in `CommunityModules-deps.jar`, which nixpkgs does not
package: take it from a pinned release, put it on TLC's classpath in both
recipes, and confirm TLC's startup log reports the override loaded.
[moq-dev/moq#8564](https://github.com/moq-dev/moq/pull/8564) rewrites
`SumOver` as an Apalache fold; whichever lands second keeps one fold for the
model.
- ✅ Add `SYMMETRY` where sound. No config checks a `PROPERTY`, so every one
checks safety only, which symmetry preserves. A permutation is sound only if
it fixes every constant the config gives, such as `Links`, `Origin`, or
`Start`, and the spec never orders those values or does arithmetic on them;
so turn
a fully interchangeable set, such as `Relays` or `Endpoints`, into model
values and permute it, or the members no constant singles out. That can cut
a state space by up to n!, though `Endpoints = {"a", "b"}` with
`Start = "a"` gains nothing. Say per config whether it applies and why.
- ✅ Pass `-checkpoint 0`, and make sure no metadir outlives its run. TLC never
removes one it was killed in, and a signal handler cannot run on SIGKILL.
Recommended: put the metadir under the checkout's `.tmp/`, remove it when
the recipe exits, and at start remove the metadirs earlier runs left there
that no live run owns, so a checkout that outlives killed runs does not
collect them. Mark ownership so a concurrent run's metadir survives, such
as a lock each run holds on a file in its metadir, which the kernel drops
when the run dies.
- ✅ Agents check one config locally: the `verify-*` skills, `REVIEW.md`, and
the `js/*/verify/` READMEs name single configs, and only Nightly runs
`just test tla` over a directory.
- Rejected: Apalache, which is no cheaper here and ran out of memory on the
route model ([moq-dev/moq#8564](https://github.com/moq-dev/moq/pull/8564));
waiting for the Rust checker, which the m1 quest
spikes and this complements.

Not decided: the route-inductive work in progress adds a refinement check,
`RefinesClaims`, to the simulated `Blind.cfg`, where every sampled trace
re-checks it. Recommended: check a refinement on the smallest exhaustive config
that exercises it, where each distinct state is checked once, unless measuring
shows the simulated config is cheaper.

Verification: every config keeps its verdict, and every exhaustive config
without `SYMMETRY` keeps its distinct-state count (the `verify-*` skills list them).
Measure each config's core-seconds through `quest gate` at equal worker counts
before and after, report the table in the PR, and update the times the skills
and Nightly's `tla` matrix comment quote; rebalance that matrix if its legs
change a lot.

Public API: none. Wire: none.

## Related

- [A Rust TLA+ checker runs the models locally if it agrees with TLC](/quest/m1/tla-native-checker.md) - a faster engine for the same specs; its TLC baseline moves with this
- [The route rules hold from any state, not only the runs TLC reaches](/quest/m1/route-inductive.md) - rewrites `SumOver` and adds the refinement check named above
33 changes: 33 additions & 0 deletions quest/m1/api-reference-cache.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# [XS] The JS API reference regenerates only when its inputs change

## Goal

`doc/.vitepress/api/index.ts` skips TypeDoc when a hash of everything the
reference is built from matches the last run's, in the docs build and the dev
server alike, so a JS-module check that changes no input spends no TypeDoc
time.

## Plan

Measured 2026-11-12 on one of the maintainer's agent hosts: the reference
regenerated on every JS-module check (`doc`'s `check` script ends in
`vitepress build`, which never reuses it), 50 runs in 4.8 hours at a median
of 74 CPU-seconds, about 6 CPU-hours a day.

Decided 2026-11-12 (maintainer, /quest-plan): ✅ skip when a hash of the
inputs is unchanged.

- The hash replaces the `--cached` modification-time stamp, which every fresh
checkout defeats; one rule serves build and dev server. Keep
`MOQ_API_REFERENCE=regenerate` only if something still needs it.
- Inputs are what can change the output: the published packages' sources and
`package.json`s, their tsconfigs and every config those reach through
`extends`, such as `js/tsconfig.json`, the generator under
`doc/.vitepress/api/`, and the TypeDoc versions and patches in `bun.lock`
and `patches/`. A missed input serves a stale reference, so derive the
list from what `generate` reads rather than by hand where possible.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
- Recommended: keep the output keyed by its hash in a cache outside the
checkout, written atomically, so each new agent worktree reuses it rather
than regenerating once per worktree. Measure the hit rate across worktrees.

Public API: none. Wire: none.
1 change: 1 addition & 0 deletions quest/m1/tla-native-checker.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,4 +63,5 @@ Public API: none. Wire: none.

## Related

- [TLC runs cost fewer core-seconds and share the host](/quest/m0/ci/tlc-cost.md) - makes TLC itself cheaper, which moves this quest's baseline
- [Every delivery order of a small mesh runs on the real route layer](/quest/m2/cluster-routes/route-exhaustive-sim.md) - the other spike, which checks the Rust instead of a faster model