Skip to content

igla race: delete 290 vacuous wave-loop tests from ternary_mac and ternary_gemm, assert exact values instead (Closes #10377) - #10379

Merged
gHashTag merged 3 commits into
masterfrom
igla-race-vacuous-tests-purge
Oct 18, 2026
Merged

gHashTag merged 3 commits into
masterfrom
igla-race-vacuous-tests-purge

Conversation

@gHashTag

Copy link
Copy Markdown
Owner

Closes #10377

Follow-up to #10375. Deletes the wave-loop placeholders that cannot fail from the two RACE reference specs and replaces them with tests that assert exact values.

What changed

  • ternary_mac.t27: 146 placeholder tests removed (144 the ratchet counted, plus the two assert(true) ones it does not see), with their paired invariant ..._depth: true lines and wave header comments. The wave tail was nothing but that boilerplate; the real tests and invariants are untouched.
  • ternary_gemm.t27: 144 removed the same way (142 counted, plus two assert(true)). The two real tests that sat at the end of the file (ternary_gemm_8x8_min_activations_do_not_wrap, ..._4x4_...) are kept.
  • Replacements, each with a hand-checkable value:
    • MAC: ternary_mul_wide(127, +1) = 127; ternary_mac_wide(-50, 127, -1) = -177; ternary_mac_wide(0, -128, +1) = -128; ternary_mac_wide(128, -128, -1) = 256 (two min negations leave i8).
    • GEMM: the 2x2 case of ternary_gemm_gpu.t27 ([5,3,7,2] against codes [2,1,1,2] gives [-2,2,-5,5]); a 2x2 case with zero weights ([-3,5,-2,7]); the 4x4 all-ones case (every element 4).
  • tools/assertionless_spec_tests_baseline.txt re-blessed with --bless in the same commit: 20 files / 2464 declarations -> 18 files / 2178; both files are gone from it.
  • Both seals regenerated with t27c seal --save on the lab (second commit, after the spec change).

Measured on the lab

t27c test-report t27b trap t27b wrap
ternary_mac 199 pass, 0 fail, 0 blocked, 0 vacuous 199/199 199/199
ternary_gemm 129 pass, 0 fail, 0 blocked, 0 vacuous 129/129 129/129
  • Before: 341 / 270 tests, of which 146 / 144 could not fail. After: every test executes at least one runtime assert (vacuous passes 0 of 199, 0 of 129).
  • t27c seal --verify: all hashes MATCH for both specs.
  • tools/verify_igla_race.py --require OK; tools/check_assertionless_spec_tests.py OK.
  • Negative control (lab, reverted): one expected value in the new 2x2 test mutated; the compiler fails it (and the pre-existing test that shares the same expectation line), and nothing else.

Everything ran on the Railway lab (t27c from master, t27b under qemu-aarch64); nothing was built or tested on the Mac. The change is .t27 plus tool-written data (seals, ratchet baseline).

🤖 Generated with Claude Code

@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-18 10:21:47 UTC

Summary

Status Count
Total Open PRs 50
PRs with Failing Checks 48
PRs with All Checks Green 2
READY 0
FAILING 48
PENDING 0
NO CHECKS YET 1

These columns do not partition: 0 + 48 + 0 + 1 = 49, and there are 50 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=fad12492937a != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

…res 1141, this PR lowers it by one; refs #10377)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-18 10:28:30 UTC

Summary

Status Count
Total Open PRs 50
PRs with Failing Checks 49
PRs with All Checks Green 1
READY 0
FAILING 49
PENDING 0
NO CHECKS YET 0

These columns do not partition: 0 + 49 + 0 + 0 = 49, and there are 50 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=fad12492937a != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@gHashTag
gHashTag merged commit eb425e6 into master Oct 18, 2026
30 of 33 checks passed
gHashTag added a commit that referenced this pull request Oct 18, 2026
…loses #10416) (#10445)

* ratchet ledgers: refresh suite_expectations and t27b_expectations against master (Closes #10416)

Corpus ratchet: drop the 7 pins whose specs now consume every token (bootrom_tb,
low_bit_ternary, isa/registers, isa/ternary_memory, positional_enc, nn/gla, vsa/ops),
cap 73 -> 66, re-pin the 4 improved discards, and re-pin igla/race/ternary_mac
1139 -> 1140 with its cause (spec rewrites #10375/#10379, parser unchanged).

t27b native: add the 171 specs the reference passes that the ledger did not name
(110 t27b pass, 61 blocked); max_not_pass 77 -> 138, the 61 new blocked entries.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* t27b ledger: re-bless at the rebased tip -- 3 now-passing, 2 moved blockers, 5 new specs; max_not_pass 138 -> 135 (Refs #10416)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

igla race: purge the 286 vacuous wave-loop tests from ternary_mac and ternary_gemm, assert real values instead

1 participant