You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Found while writing specs/automation/claim-guard-bar.t27 (PR #6662) in the overnight improvement loop the owner asked for on 2026-10-06.
t27c test-report --verbose printed FAIL rows for four deliberately false tests (1 > 2, a wrong digest accepted, a false string equality, a bar check at 89% recall) and still exited with code 0. Any gate that trusts the exit code passes a spec whose tests fail; only the printed table tells the truth.
Expected: a non-zero exit when at least one test fails. If the current behaviour is relied on, put it behind a flag such as --no-fail.
Acceptance: a fixture spec with one false test block makes the command exit non-zero. A mutant that restores exit 0 makes that check fail.
Boundary
specs/automation/claim-guard-bar.t27
User Scenarios
Given a .t27 spec containing a test block with a false assertion (e.g., 1 > 2), when running t27c test-report on that spec, then the command exits with a non-zero code.
Given a .t27 spec where all test blocks pass, when running t27c test-report, then the command exits with code 0.
Given a .t27 spec with multiple test blocks where at least one fails, when running t27c test-report --verbose, then the printed table shows FAIL rows and the process exits non-zero.
Requirements
FR-001: t27c test-report MUST exit with a non-zero code when any test in the spec fails.
FR-002: t27c test-report MUST exit with code 0 only when all tests in the spec pass.
FR-003: If existing tooling relies on the current always-zero exit, the legacy behaviour MUST be opt-in via a flag such as --no-fail.
Activity
docs: the coordination entry this branch needs to land