You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
tri t27b: honest in-reference ratio, compile-only passes, frontend disagreements, per-spec delta between lab runs #6184
On 2026-10-04 tri t27b status printed t27b 70 / reference 648, and the steward posted that to the epic. The honest figure is 56 of 648: 14 of the 70 passes are specs the reference itself fails. Two more facts in the lab's per-spec results show up in no summary number:
29 of the 56 in-reference passes have 0 tests and 0 invariants. They compiled and checked nothing.
specs/port/scripts/gen_w378.t27: t27b's frontend rejects it (cannot assign I32 to ?i32) while the reference passes it. That is a disagreement, and mismatch does not count it.
None of it is visible between two runs either: the lab keeps runs/<sha>.json, but nothing compares them per spec.
Change
tri t27b status prints the in-reference ratio, how many of those passes ran a check and how many are compile-only, and the out-of-reference passes separately.
tri t27b doctor gains LAB-FRONTEND-DISAGREES.
New tri t27b delta [--from SHA] [--to SHA], a per-spec ratchet over two lab runs: REGRESSED, NEW-MISMATCH, CHECK-LOST (exit 1), GAINED and denominator moves (reported).
Activity