Skip to content

duplicate-bodies: share tree helpers across eight gen_w bench ports #9633

Description

@dmitrii-f-t27

Title: duplicate-bodies: the eight gen_w bench ports copy indent, build_level, flat_index and is_ascii; write them once

Summary

Master (cd8a656) fails python3 tools/dupe_scan.py with exit 1 (12 violations; master CI run 38757535577 prints the same list).
Four of the groups come from eight specs/port/scripts/gen_w*.t27 bench ports that each carry the same helper block: indent, build_level, flat_index, is_ascii
(gen_w698 spells indent as a table; its outputs are the same). AGENTS.md "DRY" forbids new duplicate bodies, and the ledger only moves down.
The same fix was already made for gen_w310..gen_w322 (specs/port/scripts/gen_w_blocks.t27).

This issue covers only the gen_w part. The trinity_block STEP1/STEP2 groups are tracked in #9631 and are not touched here; I am not taking #9631.
The rust_convert and ratchet work is tracked in #9632 and #9600 and is not touched here.

Boundary

Exactly these eleven paths and no others:

specs/port/scripts/gen_w_tree.t27
specs/port/scripts/gen_w698.t27
specs/port/scripts/gen_w709.t27
specs/port/scripts/gen_w713.t27
specs/port/scripts/gen_w716.t27
specs/port/scripts/gen_w721.t27
specs/port/scripts/gen_w728.t27
specs/port/scripts/gen_w729.t27
specs/port/scripts/gen_w732.t27
tools/duplicate_bodies_baseline.txt
docs/reports/suite_expectations.json

gen_w_tree.t27 is new. The baseline change is two removed lines (flat_index 2, is_ascii 2) and nothing else.
The suite expectations change is a CONDITIONAL CANDIDATE, not a proven improvement. It removes exactly one entry, specs/port/scripts/gen_w698.t27 phase gen-verilog, and lowers max_entries from 83 to 82.
What was observed: the corpus ratchet of the first commit reported an unexpected pass (job 116524835401 of run 38764724462 on PR #9636: ledger 83 of 83, UNEXPECTED FAILURES 0, UNEXPECTED PASSES 1:
specs/port/scripts/gen_w698.t27 [gen-verilog], "fixed -- remove from ledger"). The entry recorded W469 (a table of strings in indent, issue #1959), and the table is gone.
What it does NOT prove: see "Blocked" below. gen-verilog does not carry imported function bodies, so the spec generates with its helpers silently missing. The entry may be removed for real only after #8769 is fixed
and the helper definitions are shown in the generated Verilog. No other entry, discard cap or expiry changes.
No gen/ file, no seal, no workflow, no Rust, no foreign-exceptions entry, no port file of #9622 or #9630.

User Scenarios

Scenario 1: a maintainer runs the duplicate gate after the change

Scenario 2: a gen_w port still behaves the same

  • Given any one of the eight ports
  • When t27c test-report runs on it
  • Then it reports the same 13 tests as on master, all passing, with 0 vacuous, and the helper functions resolve through use.

Scenario 3: the shared helpers have their own tests

  • Given specs/port/scripts/gen_w_tree.t27
  • When it is tested by the reference and natively
  • Then 7 of 7 tests pass in both, with runtime asserts.

Scenario 5: the Verilog output keeps the helpers (blocked by #8769)

  • Given gen_w698 and a base port such as gen_w709, both using or defining the helpers
  • When t27c gen-verilog runs on each
  • Then both outputs define function indent, elem_x, elem_y, build_level and build_tree. On the base gen_w709 they exist; on gen_w698 after the move to use only function generate remains, so this scenario FAILS today.

Scenario 4: the full gate after #9631

Requirements

  • FR-001: The gen_w tree helpers (TreeBuild, indent, elem_x, elem_y, build_level, build_tree, flat_index, is_ascii) MUST exist once, in specs/port/scripts/gen_w_tree.t27, with tests.
  • FR-002: Each of the eight ports MUST drop its local copies of those items and MUST use port::scripts::gen_w_tree;.
  • FR-003: Each port's header constants, generate() and every existing test MUST be left unchanged.
  • FR-004: The behavior of every helper MUST be unchanged; indent MUST give the same string for every level, including levels at or below 0 and above 7.
  • FR-005: tools/duplicate_bodies_baseline.txt MUST only move down: exactly the two lines flat_index 2 and is_ascii 2 are removed.
  • FR-006: The change MUST add no foreign code and MUST NOT edit a generated file by hand.
  • FR-007: The change MUST NOT touch the STEP1/STEP2 specs, rust_convert.t27, the t27b ledger or any workflow.
  • FR-008: docs/reports/suite_expectations.json MAY lose exactly the entry for specs/port/scripts/gen_w698.t27 (gen-verilog) with max_entries going from 83 to 82 only as a conditional candidate; it MUST NOT be treated as justified until gen-verilog: a function brought in by use is called but never defined, and gen-verilog exits 0 #8769 is fixed and FR-009 holds. No other entry, cap or expiry MUST change and no broad bless MUST be run.
  • FR-009: The Verilog generated for every touched port MUST still define every helper function it defined before the move (indent, elem_x, elem_y, build_level, build_tree, and flat_index/is_ascii where the backend emitted them). An exit code 0 of any gate MUST NOT be read as proof of this.

Success Criteria

Run from the repository root with the repository's t27c.

  • ./target/release/t27c test-report specs/port/scripts/gen_w_tree.t27 --specs-dir specs exits 0 and reports 7 of 7 tests.
  • ./target/release/t27c test-report specs/port/scripts/gen_w698.t27 --specs-dir specs exits 0 and reports 13 of 13 tests with 0 vacuous; the same command for gen_w709, gen_w713, gen_w716, gen_w721, gen_w728, gen_w729 and gen_w732 (verified together as 104 of 104 on the prepared branch).
  • ./target/release/t27b test specs/port/scripts/gen_w_tree.t27 --check exits 0 (7 of 7, 33 asserts on the prepared branch). The eight ports stay blocked ConstDecl natively, as they were in the master artifact; this issue does not claim they are native green.
  • python3 tools/dupe_scan.py --self-test exits 0.
  • ! python3 tools/dupe_scan.py 2>&1 | grep -E 'build_level|indent|flat_index|is_ascii' exits 0 (none of the four names remains).
  • git diff --numstat origin/master -- tools/duplicate_bodies_baseline.txt shows 0 added lines and 2 deleted lines.
  • git diff --numstat origin/master -- docs/reports/suite_expectations.json shows 1 added line and 8 deleted lines (the max_entries line and the seven-line gen_w698 entry).
  • Conditional on gen-verilog: a function brought in by use is called but never defined, and gen-verilog exits 0 #8769: the gen-verilog output of specs/port/scripts/gen_w698.t27 contains function indent, function elem_x, function elem_y, function build_level and function build_tree (checked on the generated text, as for the base gen_w709, whose output has them).
  • Conditional on that: grep -n '"max_entries": 82' docs/reports/suite_expectations.json exits 0 and the corpus ratchet job reports UNEXPECTED PASSES 0. These two are not proof of FR-009.
  • After ci: remove copied helpers from STEP1/STEP2 t27 ports #9631 is integrated: python3 tools/dupe_scan.py exits 0.

Blocked: #8769 (gen-verilog drops imported helper functions)

Found on the prepared change: gen-verilog imports enum and struct types from a used spec but not functions. Evidence (Codex, same base t27c):

  • /tmp/t27-ci-gen-w709-base.v, gen-verilog of the unchanged base gen_w709: defines function indent, elem_x, elem_y, build_level, build_tree and the rest.
  • /tmp/t27-ci-gen-w698-after.v, gen-verilog of gen_w698 after its helpers moved behind use: only function generate; every helper definition is gone, and the test names appear only as comments.
    So for the Verilog backend this move loses code, even though the reference (Zig) and Rust targets and t27c test-report pass. The suite ratchet's UNEXPECTED PASS for gen_w698 is a symptom of that loss, not evidence that W469 is fixed;
    the corpus ratchet check does not catch missing definitions (clean ratchet, UNEXPECTED FAIL 0 and PASS 0 after the expectation is removed). Exit 0 does not prove preservation.
    The root is the existing issue gen-verilog: a function brought in by use is called but never defined, and gen-verilog exits 0 #8769. Until it is fixed this change MUST NOT merge, even with every check green. Other lanes' work on gen-verilog: a function brought in by use is called but never defined, and gen-verilog exits 0 #8769 is not touched here.

Not green until

The full duplicate-bodies check stays red on any branch that has only this change, because the STEP1/STEP2 groups (#9631) remain. PRs #9622 and #9630 cannot be called green from this issue alone.

Dmitrii Fedorov (@dmitrii-f-t27)

Activity

  1. added a commit that references this issue on Oct 17, 2026
    c4bb3b6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions