test(corpus): gate on node kinds, because a grammar is not a bag of words - #78
Merged
Merged
Conversation
…ords
`require each` (v8a §49) had no corpus case and is not ported to TypeScript,
and the reserved-word gate was green the whole time — because `require` and
`each` both appear in other programs. The port's divergence list was empty
and read as complete coverage. It was measuring a different thing.
That is the same blind spot the Calendar Era found, one level up. v29 changed
behaviour while adding no word, so a word-list gate could not see it, and the
answer was a behavioural corpus. A corpus indexed by words has the identical
hole: it cannot see a construct whose words are already spoken elsewhere.
So the corpus now has a second gate, over the AST node kinds the parser can
produce. Running it found `require each` and four more constructs the word
gate had been silently satisfied about:
RequireEachNode require each n in nums is above 0
EachNode the iteration verb — `each` appeared only as the
pronoun inside a `where` clause, a different thing
FieldAccessNode `total of o1` in a condition, a different operand
path from `show total of o1`, which produces no
such node
CompositionCallNode a named composition, called — and called with a
literal argument
Eight cases added, bringing node coverage to 39 of 43. The remaining four are
excused with a stated reason, on the same contract as the word list's: no
entry without a reason, and a test that fails if a reason stops being true.
`AboutNode` and `WhenNode` are `validate()`-level; `finish` outside a handler
is refused by the analyzer; `PackVerbNode` needs a pack the generator does
not load.
The gate was verified by deletion: removing the `require each` cases fails it
with `assert not ['RequireEachNode']`.
The port now diverges on `require each`, which is correct — it does not
implement it. That is the gate reporting a real gap instead of an empty list.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E6qcDj1e1dEYvc8jLnpGZy
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The hole
require each(v8a §49) had no corpus case and is not ported to TypeScript — and the reserved-word gate was green the whole time, becauserequireandeachboth appear in other programs. The port's divergence list was empty and read as complete coverage. It was measuring a different thing.This is the Calendar Era lesson one level up. v29 changed behaviour while adding no word, so a word-list gate couldn't see it, and the answer was a behavioural corpus. But a corpus indexed by words has the identical hole: it cannot see a construct whose words are already spoken elsewhere.
The gate
A second gate, over the AST node kinds the parser can produce. Running it found
require eachand four more the word gate had been silently satisfied about:RequireEachNoderequire each n in nums is above 0EachNodeeach the numbers showeachappeared only as the pronoun in awhereclause — a different thingFieldAccessNoderequire total of o1 is above 50show total of o1, which produces no such nodeCompositionCallNodegreet, andgreet-name from "bob"Eight cases added. Node coverage 39 of 43.
The four that remain
Excused with a stated reason, on the same contract the word list already uses — no entry without a reason, and a test that fails if a reason stops being true:
AboutNode,WhenNode—validate()-level, never seen by the per-line pipelineFinishNode—finishoutside an event handler is refused by the analyzer, and the corpus has no handlerPackVerbNode— needs a pack the generator does not loadThree tests hold that contract: every node kind covered or excused; no excuse that has stopped being true; no excuse for a node the parser no longer defines.
Verified by deletion
Removing the
require eachcases fails the gate withassert not ['RequireEachNode']. A gate that has never been seen to fail isn't known to be a gate.Consequence for the port
@liminate/ts-validatornow diverges onrequire each, which is correct — it doesn't implement it. That is the gate reporting a real gap instead of an empty list. The port PR follows.Verification
1902 passed, 0 failed.
🤖 Generated with Claude Code
https://claude.ai/code/session_01E6qcDj1e1dEYvc8jLnpGZy