Skip to content

story-038: A test module is named for what it checks - #36

Merged
jerodw merged 3 commits into
mainfrom
story/story-038
Aug 15, 2026
Merged

story-038: A test module is named for what it checks#36
jerodw merged 3 commits into
mainfrom
story/story-038

Conversation

@jerodw

@jerodw jerodw commented Aug 15, 2026

Copy link
Copy Markdown
Owner

Thirty-two modules under tests/ were named test_story_NNN_validation.py. The number is meaningful for exactly as long as the story is in flight; once it merges the name says only that somebody once worked on a numbered thing. A reader looking for the tests covering the revert check, the resume guard or the retry routing had no way to find them but to grep.

Zero story-numbered modules remain.

The rename was the easy half

The shared baseline resolution identified a story by the commit that added its validation file, so git mv silently repoints every renamed module's baseline at the rename commit — each differential assertion then compares almost nothing and passes. Vacuously green, which is the defect class stories 015, 016, 026, 029 and 031 all existed to remove.

The story replaced path-inference with a declared STORY_ORIGINS map: a module names the file that identifies the story it validates, and the resolution reads that rather than inferring it from its own path. Two modules that merged the coverage of two stories declare both origins and raise a message naming the module and both origins when called without one; a stale origin is refused.

Verified independently after the run, against the merged history rather than a fixture — each renamed module still resolves to its own story and not to the rename commit 929ba58:

test_retry_routing.py        baseline=7e9e12aa2  endpoint=6ae93a152  story-028
test_revert_check.py         baseline=95afa16c8  endpoint=fe565a379  story-017
test_resume_guard.py         baseline=75e8acf7c  endpoint=4e3ca4898  story-034
test_stage_baseline.py       baseline=fdb7cfd25  endpoint=8c5ac264e  story-037
test_branch_base.py          baseline=cbb941b7d  endpoint=4af09a681  story-030

The retry is worth reading

Verification failed once and routed to implementation. Both findings were real and neither was about the renames themselves:

  • The suite did not pass with the story committed. Four failures in test_contract_assertions_bite.py, all one cause: a helper walked git log for the old filename, and since the rename the newest revision touching it is the rename commit. Exactly the trap above, in the one place the declared-origins map did not reach.
  • The recorded test count did not match the tree. test-results.json recorded 2077; the committed tree collected 2074. The verifier settled it with an AST census of every module before and after — 1344 → 1388 test functions, +44 — establishing that no assertion was lost in the rename and that the discrepancy was in the count, not the coverage.

The second is the kind of finding a green suite hides, and it was produced by driving the comparison rather than reasoning about it.

Also in this branch

The verifier's self-route budget (426fe1f, already on main) is present here. This story's first attempt escalated on 2026-08-14 when the verifier ended its turn without writing verification-result.json — it started the suite, needed ~7 minutes, and reported prose at the 9-minute mark. That is row two of story-036's conversion table with no budget to spend at the verifier; the grant closes it.

Verification

  • Tester recorded 2111 passed, 37 written
  • Verification passed on the second iteration; one retry, routed to implementation
  • Clean-clone suite green with the story committed

One thing noted, not introduced here

test_self_routing_retry.py resolves its endpoint to story-036's escalation commit rather than its completion commit, because the run commit is the oldest commit that added the identifying file and story-036's escalation committed the tester's file before the story finished. That is a property of the resolution's design for any story that escalated and resumed, and it predates this story. Worth knowing; not a reason to hold this.

🤖 Generated with Claude Code

jerodw added 3 commits August 14, 2026 20:49
The run escalated and this commit is a holding place for what it left in the working tree, so the work survives a checkout of another branch. It is not a decision about that work: the story did not finish and nothing here has been accepted or reviewed.

Escalation reason: verifier did not produce required artifacts: verification-result.json

To put these changes back in the working tree:
    git reset --mixed HEAD~2
The run escalated and this commit is a holding place for what it left in the working tree, so the work survives a checkout of another branch. It is not a decision about that work: the story did not finish and nothing here has been accepted or reviewed.

Escalation reason: verifier did not produce required artifacts: verification-result.json

To put these changes back in the working tree:
    git reset --mixed HEAD~2
Implemented by the l5 harness story workflow.
@jerodw
jerodw merged commit 0f558db into main Aug 15, 2026
3 checks passed
@jerodw
jerodw deleted the story/story-038 branch August 15, 2026 13:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant