Skip to content

A fired rule reaches the exit status, and includes reaches the parser - #72

Merged
rmichaelthomas merged 2 commits into
mainfrom
fix/deontic-exit-status-and-includes-in-where
Sep 12, 2026
Merged

rmichaelthomas merged 2 commits into
mainfrom
fix/deontic-exit-status-and-includes-in-where

Conversation

@rmichaelthomas

Copy link
Copy Markdown
Owner

Found while a downstream consumer (commongage) was string-matching Liminate's stdout to learn a verdict. Three things, all small, all in one area.

1. A violated forbid exited 0

record_result classified ERROR_PARSE, ERROR_SEMANTIC, ERROR_RUNTIME and PACK_VERB_FAILURE as had_any_error. PROHIBITION_VIOLATED and REQUIREMENT_NOT_MET were in neither list.

Measured before:

forbid violated →  Error: Prohibition violated: anchor is no.
                   this line still runs          exit=0
cite failed     →  Error: The text 'absent text' was not found in 's'.
                   this line still runs          exit=1

A pack verb was more consequential to a shell than the language's own deontic core. So every consumer matched the string Prohibition violated to get the verdict — and one that forgot reported a denial as an admission.

Exit status only. Execution still continues past a fired rule, so a program reports every violation rather than the first. That is the more useful behaviour — the case that prompted this reports two prohibitions — and is why this is not a halt.

2. where <field> includes <value> never reached the parser

The parser already builds op="includes" / op="not_includes" for that form, and analyzer.py accepts them explicitly as list-membership. Only the reorderer's two-token gate refused it:

second_ok = (second.type is TokenType.OPERATOR and second.value == "is")

includes is a CONNECTIVE, so the one stage that does no parsing reported the condition as unparseable. The gate had already been widened once, for the v25 extrema head; this is the same shape of change.

What it unlocks works and was unreachable — filtering records by a field that holds a list:

remember an order called order1 with total as 75 and tags as roof-tags
filter the orders where tags includes "urgent"    → 1 of 2 records remain

The boundary is tested, not left to be found later. each is an item, and includes over a non-list operand is false by decision (test_includes_with_scalar_left_operand_is_false), so where each includes "x" empties the list. This does not make includes a substring test — text-contains is still not in the language, and the new test says so in place.

3. A comment claimed a halt that does not happen

run.py justified running contradiction detection up front "even if a later require/forbid halts the run before reaching the conflicting statement." Neither halts. Running it up front is still right; the reason was not.

Verified

  • 1742 passed, 2 skipped, 0 failures
  • Reverting the gate fails five of the new tests
  • forbid now exits 1, a satisfied forbid/require still exits 0

🤖 Generated with Claude Code

https://claude.ai/code/session_01E6qcDj1e1dEYvc8jLnpGZy

rmichaelthomas and others added 2 commits September 11, 2026 22:49
… parser

Three things found while a downstream consumer was string-matching our
output to learn a verdict.

1. A violated `forbid` or unmet `require` exited 0.

`record_result` classified ERROR_PARSE, ERROR_SEMANTIC, ERROR_RUNTIME and
PACK_VERB_FAILURE as `had_any_error`, and the deontic statuses were in
neither list. So a failed `cite` — a pack verb — exited 1 while the
language's own prohibition exited 0, and every shell consumer had to
match "Prohibition violated" in the output to find out. One that forgot
reported a denial as an admission.

The exit status only. Execution still continues past a fired rule, so a
program reports every violation rather than the first, which is the more
useful behaviour and is why this is not a halt.

2. `where <field> includes <value>` was refused by the reorderer.

The parser builds `op="includes"` and `op="not_includes"` for that form
and the analyzer accepts them explicitly as list-membership. Only the
reorderer's two-token gate stood in front, requiring the token after the
field to be the operator `is`. So the one stage that does no parsing
reported the condition as unparseable.

The case this unlocks works and was unreachable: filtering a list of
records by a field that holds a list. Test added with real records, and
the gate had already been widened once for the v25 extrema head.

The boundary is tested too, not left to be discovered: `each` is an item,
`includes` over a non-list operand is false by decision, so
`where each includes "x"` empties the list. This does not make `includes`
a substring test, and text-contains remains absent from the language.

3. A comment claimed a halt that does not happen.

`run.py` said the contradiction check runs up front "even if a later
require/forbid halts the run". Neither halts. Running it up front is
still right; the reason given was not true.

1742 passed, 2 skipped. Reverting the gate fails five of the new tests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E6qcDj1e1dEYvc8jLnpGZy
…t see it

`@liminate/ts-validator` is a hand-written port of five of this package's
stages, and its parity gate compares the two implementations' reserved-word
lists against a fixture frozen at 0.14.1. It has been green throughout.

It could not have been anything else. What diverged changed no word: v29
added `date` to `_require_comparable`, so date ranges became expressible
and the vocabulary did not move. Downstream, `commongage` carried a code
comment reading "a DATE RANGE cannot be written in a Liminate sentence at
this language version" for two months after it could, and a note to
re-probe before trusting it — which is the right instinct and not a
mechanism.

This is the mechanism. `scripts/gen_conformance_corpus.py` runs the same
five stages the port assembles in `_run_line` — tokenize, reorder, parse,
analyze, render — over a corpus of programs and records what each line
does: status, canonical rendering, error kind, error message. Shaped as
the port's own `ValidationResult` so cases compare field for field with
neither side reshaping the other's output.

Execution is used only to advance the symbol table between a case's lines,
never recorded. The port has no interpreter, so a runtime status is a field
one side can never match, and a parity file must not contain one.

Three things keep it from rotting:

- every reserved word must appear in a corpus program, or be listed as
  unreachable with a reason — `about` and `when` are, both being
  `validate()`-level rather than per-line
- an unreachable entry that turns out to be reachable fails too, because a
  stale excuse is the same defect as a stale silence
- regenerating must be a no-op, and the fixture must name this tree's
  version

40 cases, 92 lines, 0.18.1. Adding a verb to the vocabulary fails the
coverage gate until a program uses it. 1746 passed, 2 skipped.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E6qcDj1e1dEYvc8jLnpGZy
@rmichaelthomas
rmichaelthomas merged commit 8c26a1c into main Sep 12, 2026
3 checks passed
@rmichaelthomas
rmichaelthomas deleted the fix/deontic-exit-status-and-includes-in-where branch September 12, 2026 06:05
rmichaelthomas added a commit that referenced this pull request Sep 12, 2026
`includes` is a list-membership probe, and a non-list operand evaluates
to false at runtime by decision (`test_includes_with_scalar_left_operand
_is_false`). That is right where the type is whatever the value turned
out to be. It is wrong at analysis time when the record schema has
already said the field holds text:

    filter the orders where title includes "roof"

validated clean, emptied a two-record list, and reported success.

Reachable only since #72 admitted `includes` after `where`. The refusal
that change replaced was safe and described the wrong problem — "I
couldn't parse the condition after 'where'" — and what replaced it
described nothing and was wrong. A misleading refusal beats a silent
wrong answer, so this is the one I introduced and the one to fix.

The analyzer now answers where the type is known and defers where it is
not. A list-valued field is `unknown` statically, so it still reaches
runtime and nothing that worked before stops working — the list-field
case is tested alongside.

The message says what is absent rather than what is malformed:
"'includes' tests whether a list holds a value, and 'title' is text.
Liminate has no text-contains test." That is the honest answer to a
downstream consumer asking for `contains`, and v27 §56's locked
meta-finding — the era of new verbs is ending — is why the answer is not
to add one.

A corpus case pins it, so the TypeScript port inherits the refusal rather
than the silence. 1748 passed, 2 skipped; grammar projections regenerated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E6qcDj1e1dEYvc8jLnpGZy
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant