validate-vacuity never looks inside a braceless test, so an assert true test written without braces goes uncounted #4848

scan_vacuity treats a line as a test only when the test line itself contains {. Most of specs/ writes tests without braces, and the scanner handles those in one of two ways:

• a braceless test that looks like BDD (given/when/then) is counted, but its body is never examined;
• any other braceless test is not counted at all.

Reproduce

Two files in one directory:

// specs/zz_vac/braced.t27
module VacBraced {
    fn one() -> u32 {
        return 1;
    }

    test vacuous_braced {
        assert true;
    }
}

// specs/zz_vac/braceless.t27
module VacBraceless {
    fn one() -> u32 {
        return 1;
    }

    test vacuous_braceless
        assert true
}

$ t27c validate-vacuity --specs-dir specs/zz_vac
spec                                                        tests  vacuous       %    inv    vac
specs/zz_vac/braced.t27                                         1        1  100.0%      0      0

TOTAL over 1 specs: tests=1 vacuous=1 (100.0%)  invariants=0 vacuous=0 (0.0%)

The braced test is caught. The braceless file does not appear in the report at all, and the total counts 1 spec.

Why

In bootstrap/src/main.rs at ddaab2a, scan_vacuity is at line 6626:

• is_block requires line.contains('{') on the test line. Only those tests reach tests_total and the is_vacuous_stmt check.
• A braceless test whose next four lines start with given/when/then goes into tests_bdd. It is counted, but its body is never examined.
• Any other braceless test is counted nowhere.

The scanner's comment predates W559: "Braceless test <name> with a given/when/then body: counted by the parser, discarded by every backend." Since W559 (and #2735 / #2742), braceless bodies lower to real assertions. They can now be vacuous in exactly the way a braced body can.

Invariants have the same gap. Only the one-line invariant name: expr form is counted. A braceless invariant name block is not, for example the five invariants from line 326 of specs/base/ternary_encoding.t27.

Size, over specs/ at ddaab2a

• There are 15,375 test declarations. 8,509 of them, in 347 files, are braceless.
• validate-vacuity --specs-dir specs reports tests=7288 and BDD-form tests: 8273. The BDD-form tests are counted, but their bodies are never checked.
• 236 braceless tests in 49 files have no given/when/then in the next four lines, so they are counted nowhere. All 10 tests in specs/base/ternary_encoding.t27 are among them.
• Under the tool's own rule (every statement is assert true), no braceless test is vacuous today. So this is a hole in the check, not hidden debt.
• The rule also does not recognise then true. 50 braceless tests have then true as their only check. Each one calls something first (for example specs/isa/ternary_control_flow.t27:296 and specs/nn/hslm.t27:714), so whether they count as vacuous is a policy question.

These counts come from a line scan written for this issue, not from t27c. The scan uses a test-declaration regex, a brace check, and an indentation-bounded body, so treat the figures as approximate.

Why it matters

The issue briefs generated by tools/queen/feed_untested.py tell agents that assert true is "counted as vacuous by t27c validate-vacuity, and the corpus ratchet fails a file that gains one". #4613 and #4695 both say so. That does not hold for a braceless test, and most of the corpus is braceless.

A paired agent run on #4613 noticed that validate-vacuity does not scan that file's tests at all, and judged the agent's new test non-vacuous by reading it (report).``

I found no workflow under .github/workflows that runs validate-vacuity; suite.rs runs it in Phase 6 as reporting only. If the ratchet the briefs name lives outside this repository, it has the same blind spot.

Suggested fix

1. Walk a braceless test body by indentation: take the lines indented deeper than the test line, up to the next line at its level or shallower.
2. Run that body through the same is_vacuous_stmt check.
3. Count braceless tests in tests_total, and keep tests_bdd as a sub-count.
4. Do the same for braceless invariant blocks.

Adding a rule that makes then true vacuous is a separate decision. Counting braceless tests will move the corpus totals, so any stored baseline moves with this fix.

Acceptance: for the two-file directory above, the report lists 2 specs with tests=2 vacuous=2.

Boundary

• bootstrap/src/main.rs

User Scenarios & Testing

• Given master, where scan_vacuity treats a line as a test only when the test line itself contains {, when braceless test and invariant bodies are walked by indentation and run through the same is_vacuous_stmt check, then a braceless assert true test is counted as vacuous exactly as a braced one is.
• Given master, where t27c validate-vacuity --specs-dir specs/base omits specs/base/ternary_encoding.t27 (10 braceless tests, 5 braceless invariants, none of them counted), when braceless tests are counted in tests_total, then that file appears in the report and the TOTAL tests figure includes the braceless tests.

Requirements

• FR-001: scan_vacuity MUST walk a braceless test body by indentation and run it through the same is_vacuous_stmt check as a braced body.
• FR-002: Braceless tests MUST be counted in tests_total, with tests_bdd kept as a sub-count.
• FR-003: Braceless invariant blocks MUST be counted and checked the same way.

Success Criteria

• t27c validate-vacuity --specs-dir specs/base | grep -c "specs/base/ternary_encoding.t27" prints 1
• t27c validate-vacuity --specs-dir specs/base | grep "^TOTAL" | cut -d "=" -f 2 | cut -d " " -f 1 prints more than 248
• For the two-file directory in this issue (braced.t27 and braceless.t27, each with one assert true test), t27c validate-vacuity lists 2 specs with tests=2 vacuous=2, and the command exits 0.