Repository navigation
seals: 15 specs fail their own test blocks (class c of the 2026-10-02 triage, 30 seals) #5577
Closed as not planned
Description
Class (c) of the seal triage on master a26af0ad5: 30 seals across 15 specs whose own test blocks FAIL under t27c test-report <spec> (t27c built from the same master). These were NOT resealed: a seal over a failing spec certifies a broken contract.
Per line: spec (number of seals) -- failing tests [status]. Each needs the spec or its generated body fixed, then t27c seal <spec> --save.
Note: t27c test-report names its work dir after the spec stem (t27c-test-report-<stem>), so parallel runs on specs sharing a stem (schema.t27, config.t27) clobber each other and report false results. These runs used one TMPDIR per spec.
Full evidence (rules, method, per-seal table): docs/SEAL_TRIAGE_2026-10-02.md in the triage PR. Refs #5453.
tests-fail -- 30 seals, 15 specs
-
specs/fpga/testbench/timing_tb.t27(2) -- 1 of 10 tests fail: test_fmax_computation [FAIL] -
specs/fpga/verification/build_verify.t27(2) -- 1 of 11 tests fail: test_module_count [FAIL] -
specs/ml/layers/residual_connection.t27(2) -- 2 of 2 tests fail: forward_basic_case; forward_residual_connection [FAIL] -
specs/ml/loss/huber_loss.t27(2) -- 1 of 1 tests fail: forward_basic_case [FAIL] -
specs/ml/loss/kl_divergence.t27(2) -- 1 of 1 tests fail: forward_basic_case [FAIL] -
specs/ml/loss/mse_loss.t27(2) -- 3 of 3 tests fail: forward_basic_case; forward_empty_input; forward_negative_values [FAIL] -
specs/ml/optimizer/rmsprop.t27(2) -- 1 of 1 tests fail: step_basic_case [FAIL] -
specs/ml/transformer/multi_head_attention.t27(2) -- 1 of 2 tests fail: forward_basic_case [FAIL] -
specs/tri/encoding/html.t27(2) -- 2 of 4 tests fail: parse_basic_case; query_selector_basic_case [FAIL] -
specs/tri/encoding/xml.t27(2) -- 1 of 2 tests fail: parse_basic_case [FAIL] -
specs/tri/graph/bellman_ford.t27(2) -- 1 of 1 tests fail: shortest_path_basic_case [FAIL] -
specs/tri/search/pattern.t27(2) -- 1 of 11 tests fail: wildcard_match_no_match [FAIL] -
specs/tri/sort/merge_sort.t27(2) -- 2 of 2 tests fail: sort_basic_case; sort_in_place_basic_case [FAIL] -
specs/tri/utils/template.t27(2) -- 2 of 2 tests fail: compile_basic_case; render_basic_case [FAIL] -
specs/tri/utils/terminal.t27(2) -- 1 of 3 tests fail: reset_returns_ansi_sequence [FAIL]
Boundary
specs/fpga/testbench/timing_tb.t27specs/fpga/verification/build_verify.t27specs/ml/layers/residual_connection.t27specs/ml/loss/huber_loss.t27specs/ml/loss/kl_divergence.t27specs/ml/loss/mse_loss.t27specs/ml/optimizer/rmsprop.t27specs/ml/transformer/multi_head_attention.t27specs/tri/encoding/html.t27specs/tri/encoding/xml.t27specs/tri/graph/bellman_ford.t27specs/tri/search/pattern.t27specs/tri/sort/merge_sort.t27specs/tri/utils/template.t27specs/tri/utils/terminal.t27
User Scenarios
- Given a spec with failing test blocks under
t27c test-report, when the spec or its generated body is corrected and then resealed witht27c seal <spec> --save, then subsequentt27c test-report <spec>runs exit 0 with all tests passing. - Given the 15 listed specs each have seals over failing tests, when all failing tests are fixed and each spec is resealed, then the seal triage class (c) count drops to zero for these specs.
Requirements
- FR-001: Each of the 15 cited specs MUST have its failing test(s) corrected so that
t27c test-report <spec>reports zero failures. - FR-002: After test corrections, each spec MUST be resealed using
t27c seal <spec> --saveto update its seal file. - FR-003: The fix MUST address the root cause in either the
.t27spec source or the generated implementation body, not the test expectations alone. - FR-004: Verification MUST use isolated
TMPDIRper spec to avoid work-dir clobbering from shared stem names.
Success Criteria
t27c test-report specs/fpga/testbench/timing_tb.t27exits 0 with 10 tests passingt27c test-report specs/fpga/verification/build_verify.t27exits 0 with 11 tests passingt27c test-report specs/ml/layers/residual_connection.t27exits 0 with 2 tests passingt27c test-report specs/ml/loss/huber_loss.t27exits 0 with 1 test passingt27c test-report specs/ml/loss/kl_divergence.t27exits 0 with 1 test passingt27c test-report specs/ml/loss/mse_loss.t27exits 0 with 3 tests passingt27c test-report specs/ml/optimizer/rmsprop.t27exits 0 with 1 test passingt27c test-report specs/ml/transformer/multi_head_attention.t27exits 0 with 2 tests passingt27c test-report specs/tri/encoding/html.t27exits 0 with 4 tests passingt27c test-report specs/tri/encoding/xml.t27exits 0 with 2 tests passingt27c test-report specs/tri/graph/bellman_ford.t27exits 0 with 1 test passingt27c test-report specs/tri/search/pattern.t27exits 0 with 11 tests passingt27c test-report specs/tri/sort/merge_sort.t27exits 0 with 2 tests passingt27c test-report specs/tri/utils/template.t27exits 0 with 2 tests passingt27c test-report specs/tri/utils/terminal.t27exits 0 with 3 tests passingt27c seal specs/fpga/testbench/timing_tb.t27 --saveexits 0t27c seal specs/fpga/verification/build_verify.t27 --saveexits 0t27c seal specs/ml/layers/residual_connection.t27 --saveexits 0t27c seal specs/ml/loss/huber_loss.t27 --saveexits 0t27c seal specs/ml/loss/kl_divergence.t27 --saveexits 0t27c seal specs/ml/loss/mse_loss.t27 --saveexits 0t27c seal specs/ml/optimizer/rmsprop.t27 --saveexits 0t27c seal specs/ml/transformer/multi_head_attention.t27 --saveexits 0t27c seal specs/tri/encoding/html.t27 --saveexits 0t27c seal specs/tri/encoding/xml.t27 --saveexits 0t27c seal specs/tri/graph/bellman_ford.t27 --saveexits 0t27c seal specs/tri/search/pattern.t27 --saveexits 0t27c seal specs/tri/sort/merge_sort.t27 --saveexits 0t27c seal specs/tri/utils/template.t27 --saveexits 0t27c seal specs/tri/utils/terminal.t27 --saveexits 0
Activity
gHashTag commentedon Oct 2, 2026
#5578 resealed 13 of these 15 specs while their tests still fail
#5578 (merged 2026-10-02 11:23Z) resealed 336 specs with
t27c seal --save. Its diff contains both seal files (bare and directory-prefixed) for 13 of the 15 class (c) specs below. Every one excepttri/encoding/htmlandtri/encoding/xmlis in it.t27c seal --saverefuses only when a backend fails to generate. It never runs the spec's tests. The seal-coverage gate compares hashes, so it now reports these seals as holding.Reproduction on current master (reviewer's re-run, t27c built from
d04bf14)OK: 1410 seals, 1316 hold, 94 known-broken.t27c seal --verifyreports every hash MATCH.t27c test-reporton resealed specs:fpga/testbench/timing_tb: 9/10ml/loss/mse_loss: 0/3tri/sort/merge_sort: 0/2tri/graph/bellman_ford: 0/1tri/utils/terminal: 2/3As a result, master carries seals that certify specs whose own test blocks fail.
Class (c) table (snapshot at master
a26af0ad5, fromdocs/SEAL_TRIAGE_2026-10-02.mdon the #5579 branch, commit 8517bea)Rule:
t27c test-report <spec>reports at least oneFAIL. Each spec was run with its ownTMPDIR.specs/fpga/testbench/timing_tb.t27Timing_Testbench,testbench_Timing_Testbenchspecs/fpga/verification/build_verify.t27BuildVerify,verification_BuildVerifyspecs/ml/layers/residual_connection.t27Residual,layers_Residualspecs/ml/loss/huber_loss.t27HuberLoss,loss_HuberLossspecs/ml/loss/kl_divergence.t27KlDivergence,loss_KlDivergencespecs/ml/loss/mse_loss.t27MseLoss,loss_MseLossspecs/ml/optimizer/rmsprop.t27Rmsprop,optimizer_Rmspropspecs/ml/transformer/multi_head_attention.t27MultiHeadAttn,transformer_MultiHeadAttnspecs/tri/encoding/html.t27TriHtml,encoding_TriHtmlspecs/tri/encoding/xml.t27TriXml,encoding_TriXmlspecs/tri/graph/bellman_ford.t27TriBellmanFord,graph_TriBellmanFordspecs/tri/search/pattern.t27TriPattern,search_TriPatternspecs/tri/sort/merge_sort.t27TriMergeSort,sort_TriMergeSortspecs/tri/utils/template.t27TriTemplate,utils_TriTemplatespecs/tri/utils/terminal.t27TriTerminal,utils_TriTerminalThe "gate" column shows the status before #5578. After #5578 these specs read as holding.
Ask
Make the reseal path (
t27c seal --save, and any bulk reseal built on it) refuse to seal unlesst27c test-report <spec>reports no FAIL. Report BLOCKED (generated Zig does not compile) separately, as a column, the way the triage does. The repo's rule is that blocked is not failing, so BLOCKED should not stop a seal. A FAIL should, unless the reseal is forced, and a forced reseal should be on the record the same way hollow seals are today. Until that lands, the 13 seals above should be treated as unverified, and the fixes tracked here should reseal them.Refs #5578, #5579 (closed, superseded), #5453.
docs(now): record the unseal of the fifteen failing specs (Refs #5577)
fix(seals): unseal the 15 specs whose own tests fail (Refs #5577) (#5580
docs(now): the ledger lines cite #5577 rather than name the tests
fix(seal): seal --save runs the spec's tests and refuses on FAIL (refs …
fix(seals): baseline the four html/xml seals #5580 unsealed (Refs #5577…
fix(gft): sadd orders before it subtracts, underflow is zero; seal cu…
gHashTag commentedon Oct 7, 2026
A boundary drafted from the paths this issue already names, so a bee can take it. Nothing is reserved until this is in the issue body - the Queen reads the body, not comments. Check the paths first: a wrong boundary reserves files the work does not own.
Boundary
specs/fpga/testbench/timing_tb.t27specs/fpga/verification/build_verify.t27specs/ml/layers/residual_connection.t27specs/ml/loss/huber_loss.t27specs/ml/loss/kl_divergence.t27specs/ml/loss/mse_loss.t27specs/ml/optimizer/rmsprop.t27specs/ml/transformer/multi_head_attention.t27specs/tri/encoding/html.t27specs/tri/encoding/xml.t27specs/tri/graph/bellman_ford.t27specs/tri/search/pattern.t27specs/tri/sort/merge_sort.t27specs/tri/utils/template.t27specs/tri/utils/terminal.t27docs/SEAL_TRIAGE_2026-10-02.mdfix(seals): unseal the 15 specs whose own tests fail (Refs #5577)
docs(now): record the unseal of the fifteen failing specs (Refs #5577)
github-actions commentedon Oct 10, 2026
Re-filed as #8510.
An attempt claimed this issue on 2026-10-07 and left an empty branch. A dispatch row that has spent its retry ceiling is released once and then keeps its issue for good, which is the right rule for a third identical failure and the wrong outcome for an issue nobody ever attempted twice. The new issue carries the same work, the same boundary, and today's brief.
Closing this one so the swarm can take the work again.