Repository navigation
Make generated filter schemas a narrowing again - #10
Merged
Merged
Conversation
The generator restated the published constraint object's dependency rules
instead of reading them, and had only one of the two: $flags → $regex was
carried, $unknownAs → "needs something to modify" was not. So a generated
schema accepted {"microchip": {"$unknownAs": false}} — a modifier with no
predicate under it — where query-language-schema.json rejects it, which is
the one direction generation may never go. A server on the documented path
then had to evaluate a filter with nothing in it; the SQL compiler emitted
coalesce((), FALSE) and the database answered 500 to a filter its own tool
schema had declared valid.
Both keywords now come off $defs/ConstraintObject, so a rule added there
arrives in generated schemas with the version that introduces it. The
$comment is left behind: a validator never reads it, and one copy per field
is charged by the token to whoever inlines the schema in a tool definition.
The property this breaks is universally quantified, and it sat under a list
of fifteen filters — which can only re-check the leaks someone already
thought of, and did not contain this one. tests/fuzz.mjs samples filters out
of a generated schema's own vocabulary, deliberately blind to the keywords
that bound the shape of a value rather than its type, and the soundness test
now runs 5000 samples against three generated schemas: every one the
generated schema accepts must be valid JQL. It also asserts the run was not
vacuous — a fraction of samples accepted, every offered operator reached.
Against the schema this commit replaces it reports 205 leaks. A second test
pins what the generator does with each instance-constraining keyword of
$defs/ConstraintObject, so adding one there fails the suite until it is
handled deliberately.
experiments/filter-to-sql rejects a constraint object with no predicate in
it as malformed-query, pointing at the clause, rather than compiling an
empty expression. Both schemas reject those now, so this only matters for a
compiler reached another way — but the failure it replaces was a database
syntax error.
Fixes #8
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #8.
The leak
tools/generate-filter-schema.mjsrestated the published constraint object's dependency rules instead of reading them, and had one of the two:dependentRequired($flags→$regex) was carried,dependentSchemas($unknownAsneeds something to modify) was not. So the generated schema accepted a filter the grammar rejects:That is the one direction generation may never go. A lone
$unknownAshas no predicate under it, so a server on the documented path — generated schema as the MCPinputSchema, published semantics behind it — had nothing to compile:experiments/filter-to-sqlemittedSELECT id FROM pets WHERE coalesce((), FALSE)and SQLite answered with a syntax error, i.e. a 500 for a filter the tool's own schema had declared valid.The fix
tools/generate-filter-schema.mjs— both dependency keywords are now read off$defs/ConstraintObjectand carried onto every generated constraint object whose trigger operator survived, so a rule added to the published definition reaches generated schemas with the version that introduces it. The$commentjustifying a rule is deliberately not copied: a validator never reads it, and one copy per field is charged by the token to whoever inlines the schema in a tool definition.examples/pet.filter.json— regenerated. 12 of its 18 constraint objects gain the rule: exactly the fields that can be absent or null, which are the ones offering$unknownAs.query-language-schema.json. The grammar was right; the generator disagreed with it.Testing the property instead of listing cases
tests/generator.test.mjsasserted narrowing over a hand-written list of fifteen filters. The property is quantified over every filter, so a list can only re-check the leaks someone already thought of — and it did not contain this one, for as long as it existed.tests/fuzz.mjs(new) samples filters out of a generated schema's own vocabulary: mostly conformant, because a random JSON value is rejected by both schemas and tells you nothing, but deliberately blind to the keywords that bound the shape of a value rather than its type —minProperties,minItems/maxItems,uniqueItems, and the dependency keywords. Seeded, so a failure reproduces.coreonly, a recursive resource) and asserts every filter the generated schema accepts is valid JQL. It also asserts the run was not vacuous: a fraction of samples must be accepted, and every operator the schema offers must be reached by an accepted one.$defs/ConstraintObject, so a keyword added there fails the suite until it is handled deliberately. This is what the issue asked be checked —dependentSchemaswas the only one missing.Downstream
experiments/filter-to-sqlnow rejects a constraint object with no predicate in it asmalformed-query, with a pointer at the clause, rather than compiling an empty expression. Both schemas reject those now, so this only matters for a compiler reached another way — but the failure it replaces was a database syntax error. Casef05bcovers it, and the counts in the experiment's README are brought back in line with what--metricsprints (some were already stale by one corpus).npm test— 119 pass.npm run test:experiment— 152 pass.npm run example:mcp:demounchanged: the server readspet.filter.jsonat startup, so it picks the rule up.Generated with Claude Code