Repository navigation
tri mutate census: one table and one exit code for a directory of specs #7433
Description
Problem
A mutation census of a spec directory is a hand-written loop today. The 2026-10-07 census of specs/queen/ (#6971 comment) was a 7-line sh script on the lab: run tri mutate spec --file F per spec, write each log, collect exit codes, then read 11 logs by hand to build one table. Every number in that table was copied out of a log, and the totals were summed by a separate script.
Two things the loop needs are missing from tri:
- One table for a directory. Per spec: mutants, killed, survived, hung, unviable, or "the unmutated spec does not pass" (brain_summaries.t27 and lotus.t27 today), and the totals with a percentage.
- One exit code for the directory, so a scheduled run can tell "nothing regressed" from "a new survivor appeared" without reading logs.
tri mutate spec --accepted FILEalready gives this per spec (tri mutate spec: an opt-in non-zero exit when mutants survive #7303), and an accepted file already filters its lines by spec path, so one file can serve a whole directory.
Plan
specs/tri/mutate/census.t27: the rules, with tests and mutation-checked:- the directory's exit from the specs' exits: any 1 (a spec the tool could not judge) -> 1, else any 2 (gate failed) -> 2, else 0;
- the killed share in tenths of a percent (
killed * 1000 / ran, 0 when nothing ran), so the table prints92.8without floats; - which specs a row counts in the totals (a spec whose unmutated form fails is listed, not summed).
tri mutate census --dir D [--accepted F] [--jobs N --timeout S --zig-threads N] [--lab]: thin glue that calls the existingtri mutate specpath perD/*.t27, then the generated rules. Hand-written Rust stays under the own-language budget (Only-t27 gate: a foreign-line budget the label cannot lift; local hook reads exceptions from origin/master #7371: 40 lines per file, 80 per diff); the rules reach the binary throught27c gen-rust, not through a hand copy.--lab: one lab run for the directory (astri mutate spec --lab, tri mutate spec --lab: run the mutation pass on the Railway lab from a Mac worktree #7050), not one per spec. Moved to tri mutate census --lab: one lab job for a directory of specs #7471.
Acceptance criteria
t27c test-report specs/tri/mutate/census.t27:FAIL 0,vacuous passes 0 of N;tri mutate spec --file specs/tri/mutate/census.t27leaves only named equivalents.tri mutate census --dir specs/queen --accepted <file>on master prints the same per-spec counts astri mutate specrun on each file, and its exit is the rule above (checked against the 2026-10-07 hand census: 9 specs judged, 2 not runnable).- Negative controls: remove one accepted line -> exit 2 and that line printed; a spec that does not compile -> exit 1 and the spec named.
Boundary
specs/tri/mutate/census.t27
cli/tri/src/mutate.rs
gen/rust/tri/mutate/census.rs
specs/tools/tri/mutate.t27
docs/now/2026-10-07-mutate-census-rules.md
Activity
gHashTag commentedon Oct 7, 2026
Slice 1 is written and checked:
specs/tri/mutate/census.t27, local commit de33ba7 on branchclaude/tri-mutate-census-7433. No PR yet: the Actions queue held 200+ queued runs at 09:28Z, so it waits (one PR at a time).Two changes from the plan above, made before any Rust:
census_exitholds that.What the spec has: 6 functions (
verdict_known,census_exit,judged,killed_tenths,tenths_whole_part,tenths_digit), 4 constants, 10 tests, 1 invariant. Judged = killed + survived + hung. An unviable mutant is left out, as Stryker leaves out compile and runtime errors. A hung mutant counts as not killed, assurvivors.t27counts it (Stryker counts a timeout as detected). A code no rule names, such as the lab's 3, 4 and 5, has no verdict, so the directory exits 1.Checked on the lab with the release
t27cof master 05e633d:t27c test-report: 10 of 10 tests pass, 0 vacuous;tri mutate spec: 25 of 25 mutants killed;TENTHS_WHOLE1000 -> 100).Gated census of
specs/queen/, same lab and master, with an accepted file of 21 lines (14 equivalent survivors, 7 hung drop-steps):census_exitfolds those to 1, and the spec's first test asserts it. Negative control for the gate: withpriority.t27:72removed from the accepted file,tri mutate spec --file specs/queen/priority.t27 --fn effective_levelexits 2 and prints that line.Next slice:
tri mutate census --dir D, with the rules reaching the binary throught27c gen-rust(bootstrap/gen/rust/tri/mutate/census.rs) and hand-written glue kept under the #7371 budget.gHashTag commentedon Oct 7, 2026
Slice 2a, local commit 67ee20f on
claude/tri-mutate-census-7433(on de33ba7; no PR while the Actions queue is over 50).Reading a spec's summary line is now a rule in
specs/tri/mutate/census.t27, so the census command's Rust only has to split lines and call it:is_summary_line(line): the wholeK of N killed (T by a failing test, I by an invariant at compile time); S survived, H hung, U unviable.line, 7 counts of 1 to 18 digits (19 is refused, so nothing it accepts overflows u64);count_at(line, n), withCOUNT_KILLED0 toCOUNT_UNVIABLE6;summary_adds_up(line): killed = by a test + by an invariant, and mutants = killed + survived + hung + unviable.The fixtures are the 9 summary lines of the 2026-10-07 census of
specs/queen/, copied from the lab logs. Summed withcount_atthey give 783 killed, 54 survived, 7 hung, and 92.7.On the lab (release t27c of 05e633d):
tri mutate spec --fnon each new function:after_count27 of 29 (2 hung: dropped loop steps),is_summary_line2 of 2,count_at13 of 13,summary_adds_up8 of 8.count_atrun was a dead early return, now removed..lenis not bound, Design decision: what a string field and a string comparison mean in generated hardware #2433). The elab ratchet covers only the fpga set.Next is slice 2b:
gen/rust/tri/mutate/census.rsfromt27c gen-rust, and thetri mutate census --dir Dglue within the #7371 budget.Merge origin/master into feat/tri-test-runs-tests (Refs #7369)
refactor(tri): mutate loads the survivor gate and lab rules from gen-…