Repository navigation
default_input has no author: the converter generated 169 specs' worth of hollow tests #5166
Description
Re-filed from #2623. An attempt claimed that issue on 2026-09-24 and left an empty branch, and a dispatch row that has spent its retry ceiling keeps its issue for good - so no bee could take it again, however much the brief improved. The work below is carried over unchanged.
default_input is the single largest undeclared identifier in the corpus — 154
of the 202 corpus-side names, across 169 specs. I have been reporting it for
three iterations as "the spec authors reference something they never defined,
and what it should return is the owner's call."
That was wrong. No spec author wrote it.
tools/converter/src/main.rs:684:
output.push_str(&format!(" given input = default_input()\n"));
The generator is the sole author of a name it never provides. The full
template it emitted for every function in every spec:
test {name}_basic_case
given input = default_input()
when result = {fn}(input)
then result != undefined
And for every constraint:
invariant {name}_constraint_{i}
given input = valid_input()
then true // <the constraint, as prose, in a comment>
These tests could never have passed, and never should have
Even with default_input supplied, then result != undefined asserts nothing —
and Zig cannot compare against undefined at all. then true is an invariant
that cannot fail.
So the generator was manufacturing hollow tests: they make a spec look
covered while checking nothing. That is exactly the failure mode valid AND asserting was added in #2595 to catch, and it was being produced mechanically,
at scale, by a tool.
The gate hole and this defect are the same story
tools/converter is in the workspace exclude list, so cargo check --workspace has never covered it. It is the one crate the gate cannot see, and
it is the crate that wrote a broken clause into 169 specs.
That is not a coincidence worth glossing: the uncovered crate is where the
damage came from. Noting it as still open rather than fixing the hook here —
changing a shared pre-commit hook affects everyone's commits and is not mine to
do unilaterally.
Changed
The generator now says what it knows and stops inventing what it does not:
// TODO: behaviour for encode() -- write given/when/then by hand.
// Not generated: a signature does not imply a behaviour.
Verified with cargo check --manifest-path tools/converter/Cargo.toml — clean,
and the unused-variable warnings the change exposed are removed rather than
silenced.
What this does NOT do
The 169 existing specs are untouched. This stops the manufacture; it does
not undo it. Regenerating them is a corpus-wide change and a decision for the
owner — and worth pairing with the observation that the tests being removed
never asserted anything, so nothing is lost by deleting them outright.
Boundary
tools/converter/Cargo.tomltools/converter/src/main.rs
The instruments
Every one of these reads and prints; none of them writes. Run them from the repository root, with <spec> replaced by the file in ## Boundary:
.t27 is not Rust and not Zig, and a bee fluent in either writes one by
accident. Measured 2026-09-20: a bee filled eight bodies in
specs/file/watcher.t27 with return Ok(());, Err(FileError::WatcherNotFound)
and for i in 0..watchers.length. The parse ratchet refused the whole file --
Unexpected token in expression: RParen, line 123 -- and a spec that does not
parse generates nothing, so every test it already carried stopped running.
At the top level the parser accepts exactly these eight forms, with an optional
pub, and no others:
const var fn enum struct test invariant bench
There is no trait, no impl, no type X = ..., no generics, no macro. What
a bee reaches for from another language, and what happens:
- Rust's result sugar (
Ok(()),Err(E::V),?) is not a construct here. - Rust macros (
println!,format!,vec!) are not constructs here. - A range loop (
for i in 0..n) is Rust and Zig, not this language. - Zig builtins (
catch unreachable,@intCast,@constCast) are not constructs here. - Cross-module reuse (
use other::fn;) parses and then generates a comment and an unqualified call, so the Zig fails withuse of undeclared identifier(use a::b; generates no import, so no spec can reuse a function from another spec #4298). It does not work yet.
t27c parse <spec> answers in one line whether what you wrote is the language.
Run it before you report, every time.
t27c spec-status <spec>- the compiler's one-word verdict: IMPLEMENTED, PARTIAL, UNWRITTEN, NOPARSE, NOFN. Exit code is 0 whatever it says, so read the word.t27c symbols <spec>- every name the file declares, with its kind. Answers "does this already exist here?" before you add it.t27c outline <spec>- per function: its locals, what it calls, what it returns. The contract you are implementing against.t27c coverage <spec>- which functions have a test and which do not. The issue asks for tests; this is how you check you wrote them.t27c lint <spec>- style and shape warnings. It OVER-REPORTShas no test or invariant: measured 2026-09-20 over 60 specs it printed 677 of those wherecoveragefound 190 untested functions, disagreeing on 59 of the 60 - it warns aboutbit_to_trit_pairinspecs/base/ternary_encoding.t27, whichtest bit_to_trit_pair_zerocalls on line 249. Read it as a hint;coverageis the answer.t27c typecheck <spec>- types, before generation. PrintsTypecheck OK (0 errors, 0 warnings)or the errors.t27c test-report <spec>- builds this spec and runs its own tests, which is what the oracle does.BLOCKEDmeans the generated Zig does not compile, with the error beside it.python3 tools/dupe_scan.py --name <function>- where that function already lives, if it does. 576 of 4021 bodies here are byte-identical copies.
The rest of the toolbelt is in docs/BEE_TOOLBELT.md.
User Scenarios & Testing
- Given master, where 85d67a9 (Drop 572 generated stub tests: valid 136 to 237, asserting 96 to 197, on its own branch #4746) already removed the
default_input()test template butgenerate_t27still writesinvariant ..._constraint_Nblocks withgiven input = valid_input()andthen true, when the change lands, then the converter no longer invents an input name it never provides or an invariant that cannot fail, and says in a comment that the behaviour was not generated.
Requirements
- FR-001: The converter MUST stop inventing what it does not know:
generate_t27intools/converter/src/main.rsMUST NOT emitvalid_input()or athen trueinvariant for a constraint. - FR-002: Where it emits nothing, the generator MUST say so in a comment, in the issue's words:
Not generated: a signature does not imply a behaviour. - FR-003: The existing specs MUST stay untouched; regenerating them is an owner decision, as the issue states.
Success Criteria
grep -c valid_input tools/converter/src/main.rsprints0grep -c 'Not generated' tools/converter/src/main.rsprints at least1
Activity
salvage(queen-5166): commit what the turn left uncommitted
t27c test-report: a test that outruns its limit is stopped and counte…
self-host: t27core accepts compound assignment, byte-identical to gen…
t27core: numeric literal forms and pub var, byte-identical to gen-c (C…