Skip to content

test(corpus): gate on node kinds, because a grammar is not a bag of words - #78

Merged
rmichaelthomas merged 1 commit into
mainfrom
test/require-each-corpus-case
Sep 12, 2026
Merged

rmichaelthomas merged 1 commit into
mainfrom
test/require-each-corpus-case

Conversation

@rmichaelthomas

Copy link
Copy Markdown
Owner

The hole

require each (v8a §49) had no corpus case and is not ported to TypeScript — and the reserved-word gate was green the whole time, because require and each both appear in other programs. The port's divergence list was empty and read as complete coverage. It was measuring a different thing.

This is the Calendar Era lesson one level up. v29 changed behaviour while adding no word, so a word-list gate couldn't see it, and the answer was a behavioural corpus. But a corpus indexed by words has the identical hole: it cannot see a construct whose words are already spoken elsewhere.

The gate

A second gate, over the AST node kinds the parser can produce. Running it found require each and four more the word gate had been silently satisfied about:

Node kind The construct Why the word gate missed it
RequireEachNode require each n in nums is above 0 both words used elsewhere
EachNode each the numbers show each appeared only as the pronoun in a where clause — a different thing
FieldAccessNode require total of o1 is above 50 a different operand path from show total of o1, which produces no such node
CompositionCallNode greet, and greet-name from "bob" the call form was never exercised, only the definition

Eight cases added. Node coverage 39 of 43.

The four that remain

Excused with a stated reason, on the same contract the word list already uses — no entry without a reason, and a test that fails if a reason stops being true:

  • AboutNode, WhenNode — validate()-level, never seen by the per-line pipeline
  • FinishNode — finish outside an event handler is refused by the analyzer, and the corpus has no handler
  • PackVerbNode — needs a pack the generator does not load

Three tests hold that contract: every node kind covered or excused; no excuse that has stopped being true; no excuse for a node the parser no longer defines.

Verified by deletion

Removing the require each cases fails the gate with assert not ['RequireEachNode']. A gate that has never been seen to fail isn't known to be a gate.

Consequence for the port

@liminate/ts-validator now diverges on require each, which is correct — it doesn't implement it. That is the gate reporting a real gap instead of an empty list. The port PR follows.

Verification

1902 passed, 0 failed.

🤖 Generated with Claude Code

https://claude.ai/code/session_01E6qcDj1e1dEYvc8jLnpGZy

…ords

`require each` (v8a §49) had no corpus case and is not ported to TypeScript,
and the reserved-word gate was green the whole time — because `require` and
`each` both appear in other programs. The port's divergence list was empty
and read as complete coverage. It was measuring a different thing.

That is the same blind spot the Calendar Era found, one level up. v29 changed
behaviour while adding no word, so a word-list gate could not see it, and the
answer was a behavioural corpus. A corpus indexed by words has the identical
hole: it cannot see a construct whose words are already spoken elsewhere.

So the corpus now has a second gate, over the AST node kinds the parser can
produce. Running it found `require each` and four more constructs the word
gate had been silently satisfied about:

  RequireEachNode        require each n in nums is above 0
  EachNode               the iteration verb — `each` appeared only as the
                         pronoun inside a `where` clause, a different thing
  FieldAccessNode        `total of o1` in a condition, a different operand
                         path from `show total of o1`, which produces no
                         such node
  CompositionCallNode    a named composition, called — and called with a
                         literal argument

Eight cases added, bringing node coverage to 39 of 43. The remaining four are
excused with a stated reason, on the same contract as the word list's: no
entry without a reason, and a test that fails if a reason stops being true.
`AboutNode` and `WhenNode` are `validate()`-level; `finish` outside a handler
is refused by the analyzer; `PackVerbNode` needs a pack the generator does
not load.

The gate was verified by deletion: removing the `require each` cases fails it
with `assert not ['RequireEachNode']`.

The port now diverges on `require each`, which is correct — it does not
implement it. That is the gate reporting a real gap instead of an empty list.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E6qcDj1e1dEYvc8jLnpGZy
@rmichaelthomas
rmichaelthomas merged commit d4c72fd into main Sep 12, 2026
3 checks passed
@rmichaelthomas
rmichaelthomas deleted the test/require-each-corpus-case branch September 12, 2026 08:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant