validate-vacuity never looks inside a braceless test, so an assert true test written without braces goes uncounted #9506

Re-filed from #6772. An attempt claimed that issue on 2026-10-06 and left an empty branch, and a dispatch row that has spent its retry ceiling keeps its issue for good - so no bee could take it again, however much the brief improved. The work below is carried over unchanged.

Re-filed from #4848. An attempt claimed that issue on 2026-09-27 and left an empty branch, and a dispatch row that has spent its retry ceiling keeps its issue for good - so no bee could take it again, however much the brief improved. The work below is carried over unchanged.

scan_vacuity treats a line as a test only when the test line itself contains {. Most of specs/ writes tests without braces, and the scanner handles those in one of two ways:

• a braceless test that looks like BDD (given/when/then) is counted, but its body is never examined;
• any other braceless test is not counted at all.

Reproduce

Two files in one directory:

// specs/zz_vac/braced.t27
module VacBraced {
    fn one() -> u32 {
        return 1;
    }

    test vacuous_braced {
        assert true;
    }
}

// specs/zz_vac/braceless.t27
module VacBraceless {
    fn one() -> u32 {
        return 1;
    }

    test vacuous_braceless
        assert true
}

$ t27c validate-vacuity --specs-dir specs/zz_vac
spec                                                        tests  vacuous       %    inv    vac
specs/zz_vac/braced.t27                                         1        1  100.0%      0      0

TOTAL over 1 specs: tests=1 vacuous=1 (100.0%)  invariants=0 vacuous=0 (0.0%)

The braced test is caught. The braceless file does not appear in the report at all, and the total counts 1 spec.

Why

In bootstrap/src/main.rs at ddaab2a, scan_vacuity is at line 6626:

• is_block requires line.contains('{') on the test line. Only those tests reach tests_total and the is_vacuous_stmt check.
• A braceless test whose next four lines start with given/when/then goes into tests_bdd. It is counted, but its body is never examined.
• Any other braceless test is counted nowhere.

The scanner's comment predates W559: "Braceless test <name> with a given/when/then body: counted by the parser, discarded by every backend." Since W559 (and #2735 / #2742), braceless bodies lower to real assertions. They can now be vacuous in exactly the way a braced body can.

Invariants have the same gap. Only the one-line invariant name: expr form is counted. A braceless invariant name block is not, for example the five invariants from line 326 of specs/base/ternary_encoding.t27.

Size, over specs/ at ddaab2a

• There are 15,375 test declarations. 8,509 of them, in 347 files, are braceless.
• validate-vacuity --specs-dir specs reports tests=7288 and BDD-form tests: 8273. The BDD-form tests are counted, but their bodies are never checked.
• 236 braceless tests in 49 files have no given/when/then in the next four lines, so they are counted nowhere. All 10 tests in specs/base/ternary_encoding.t27 are among them.
• Under the tool's own rule (every statement is assert true), no braceless test is vacuous today. So this is a hole in the check, not hidden debt.
• The rule also does not recognise then true. 50 braceless tests have then true as their only check. Each one calls something first (for example specs/isa/ternary_control_flow.t27:296 and specs/nn/hslm.t27:714), so whether they count as vacuous is a policy question.

These counts come from a line scan written for this issue, not from t27c. The scan uses a test-declaration regex, a brace check, and an indentation-bounded body, so treat the figures as approximate.

Why it matters

The issue briefs generated by tools/queen/feed_untested.py tell agents that assert true is "counted as vacuous by t27c validate-vacuity, and the corpus ratchet fails a file that gains one". #4613 and #4695 both say so. That does not hold for a braceless test, and most of the corpus is braceless.

A paired agent run on #4613 noticed that validate-vacuity does not scan that file's tests at all, and judged the agent's new test non-vacuous by reading it (report).``

I found no workflow under .github/workflows that runs validate-vacuity; suite.rs runs it in Phase 6 as reporting only. If the ratchet the briefs name lives outside this repository, it has the same blind spot.

Suggested fix

1. Walk a braceless test body by indentation: take the lines indented deeper than the test line, up to the next line at its level or shallower.
2. Run that body through the same is_vacuous_stmt check.
3. Count braceless tests in tests_total, and keep tests_bdd as a sub-count.
4. Do the same for braceless invariant blocks.

Adding a rule that makes then true vacuous is a separate decision. Counting braceless tests will move the corpus totals, so any stored baseline moves with this fix.

Acceptance: for the two-file directory above, the report lists 2 specs with tests=2 vacuous=2.

Boundary

• bootstrap/src/main.rs

User Scenarios & Testing

• Given master, where scan_vacuity treats a line as a test only when the test line itself contains {, when braceless test and invariant bodies are walked by indentation and run through the same is_vacuous_stmt check, then a braceless assert true test is counted as vacuous exactly as a braced one is.
• Given master, where t27c validate-vacuity --specs-dir specs/base omits specs/base/ternary_encoding.t27 (10 braceless tests, 5 braceless invariants, none of them counted), when braceless tests are counted in tests_total, then that file appears in the report and the TOTAL tests figure includes the braceless tests.

Requirements

• FR-001: scan_vacuity MUST walk a braceless test body by indentation and run it through the same is_vacuous_stmt check as a braced body.
• FR-002: Braceless tests MUST be counted in tests_total, with tests_bdd kept as a sub-count.
• FR-003: Braceless invariant blocks MUST be counted and checked the same way.

Success Criteria

• t27c validate-vacuity --specs-dir specs/base | grep -c "specs/base/ternary_encoding.t27" prints 1
• t27c validate-vacuity --specs-dir specs/base | grep "^TOTAL" | cut -d "=" -f 2 | cut -d " " -f 1 prints more than 248
• For the two-file directory in this issue (braced.t27 and braceless.t27, each with one assert true test), t27c validate-vacuity lists 2 specs with tests=2 vacuous=2, and the command exits 0.

The instruments

Every one of these reads and prints; none of them writes. Run them from the repository root, with <spec> replaced by the file in ## Boundary:

.t27 is not Rust and not Zig, and a bee fluent in either writes one by
accident. Measured 2026-09-20: a bee filled eight bodies in
specs/file/watcher.t27 with return Ok(());, Err(FileError::WatcherNotFound)
and for i in 0..watchers.length. The parse ratchet refused the whole file --
Unexpected token in expression: RParen, line 123 -- and a spec that does not
parse generates nothing, so every test it already carried stopped running.

At the top level the parser accepts exactly these eight forms, with an optional
pub, and no others:

const   var   fn   enum   struct   test   invariant   bench

There is no trait, no impl, no type X = ..., no generics, no macro. What
a bee reaches for from another language, and what happens:

• Rust's result sugar (Ok(()), Err(E::V), ?) is not a construct here.
• Rust macros (println!, format!, vec!) are not constructs here.
• A range loop (for i in 0..n) is Rust and Zig, not this language.
• Zig builtins (catch unreachable, @intCast, @constCast) are not constructs here.
• Cross-module reuse (use other::fn;) parses and then generates a comment and an unqualified call, so the Zig fails with use of undeclared identifier (use a::b; generates no import, so no spec can reuse a function from another spec #4298). It does not work yet.

t27c parse <spec> answers in one line whether what you wrote is the language.
Run it before you report, every time.

• t27c spec-status <spec> - the compiler's one-word verdict: IMPLEMENTED, PARTIAL, UNWRITTEN, NOPARSE, NOFN. Exit code is 0 whatever it says, so read the word.
• t27c symbols <spec> - every name the file declares, with its kind. Answers "does this already exist here?" before you add it.
• t27c outline <spec> - per function: its locals, what it calls, what it returns. The contract you are implementing against.
• t27c coverage <spec> - which functions have a test and which do not. The issue asks for tests; this is how you check you wrote them.
• t27c lint <spec> - style and shape warnings. It OVER-REPORTS has no test or invariant: measured 2026-09-20 over 60 specs it printed 677 of those where coverage found 190 untested functions, disagreeing on 59 of the 60 - it warns about bit_to_trit_pair in specs/base/ternary_encoding.t27, which test bit_to_trit_pair_zero calls on line 249. Read it as a hint; coverage is the answer.
• t27c typecheck <spec> - types, before generation. Prints Typecheck OK (0 errors, 0 warnings) or the errors.
• t27c test-report <spec> - builds this spec and runs its own tests, which is what the oracle does. BLOCKED means the generated Zig does not compile, with the error beside it.
• python3 tools/dupe_scan.py --name <function> - where that function already lives, if it does. 576 of 4021 bodies here are byte-identical copies.

The rest of the toolbelt is in docs/BEE_TOOLBELT.md.

The instruments

Every one of these reads and prints; none of them writes. Run them from the repository root, with <spec> replaced by the file in ## Boundary:

.t27 is not Rust and not Zig, and a bee fluent in either writes one by
accident. Measured 2026-09-20: a bee filled eight bodies in
specs/file/watcher.t27 with return Ok(());, Err(FileError::WatcherNotFound)
and for i in 0..watchers.length. The parse ratchet refused the whole file --
Unexpected token in expression: RParen, line 123 -- and a spec that does not
parse generates nothing, so every test it already carried stopped running.

At the top level the parser accepts exactly these eight forms, with an optional
pub, and no others:

const   var   fn   enum   struct   test   invariant   bench

There is no trait, no impl, no type X = ..., no generics, no macro. What
a bee reaches for from another language, and what happens:

• Rust's result sugar (Ok(()), Err(E::V), ?) is not a construct here.
• Rust macros (println!, format!, vec!) are not constructs here.
• A range loop (for i in 0..n) is Rust and Zig, not this language.
• Zig builtins (catch unreachable, @intCast, @constCast) are not constructs here.
• Cross-module reuse (use other::fn;) parses and then generates a comment and an unqualified call, so the Zig fails with use of undeclared identifier (use a::b; generates no import, so no spec can reuse a function from another spec #4298). It does not work yet.

t27c parse <spec> answers in one line whether what you wrote is the language.
Run it before you report, every time.

• t27c spec-status <spec> - the compiler's one-word verdict: IMPLEMENTED, PARTIAL, UNWRITTEN, NOPARSE, NOFN. Exit code is 0 whatever it says, so read the word.
• t27c symbols <spec> - every name the file declares, with its kind. Answers "does this already exist here?" before you add it.
• t27c outline <spec> - per function: its locals, what it calls, what it returns. The contract you are implementing against.
• t27c coverage <spec> - which functions have a test and which do not. The issue asks for tests; this is how you check you wrote them.
• t27c lint <spec> - style and shape warnings. It OVER-REPORTS has no test or invariant: measured 2026-09-20 over 60 specs it printed 677 of those where coverage found 190 untested functions, disagreeing on 59 of the 60 - it warns about bit_to_trit_pair in specs/base/ternary_encoding.t27, which test bit_to_trit_pair_zero calls on line 249. Read it as a hint; coverage is the answer.
• t27c typecheck <spec> - types, before generation. Prints Typecheck OK (0 errors, 0 warnings) or the errors.
• t27c test-report <spec> - builds this spec and runs its own tests, which is what the oracle does. BLOCKED means the generated Zig does not compile, with the error beside it.
• python3 tools/dupe_scan.py --name <function> - where that function already lives, if it does. 576 of 4021 bodies here are byte-identical copies.

The rest of the toolbelt is in docs/BEE_TOOLBELT.md.