Make generated filter schemas a narrowing again

CHANGELOG.md

@@ -48,6 +48,37 @@ No change to the grammar or to the semantics of evaluation. The only edit to
   and the `type` URIs, the `pointer` member and the recovery members (`queryableFields`,
   `accepted`) are unchanged as its encoding. This relaxes a requirement, so nothing that
   conformed before stops conforming.
+
+### Fixed
+
+- **Generated filter schemas were not a narrowing** ([#8](https://github.com/christosgkoros/json-query-language/issues/8)).
+  `tools/generate-filter-schema.mjs` carried the published constraint object's
+  `dependentRequired` rule but not its `dependentSchemas` one, so `{"microchip": {"$unknownAs":
+  false}}` — a modifier with nothing to modify — passed a generated schema while
+  `query-language-schema.json` rejected it. A server following the documented path (generated
+  schema as the tool's `inputSchema`, published semantics behind it) then had to evaluate a filter
+  with no predicate in it; the SQL compiler in `experiments/filter-to-sql` emitted
+  `coalesce((), FALSE)` and the database answered with a syntax error. The generator now reads
+  both dependency keywords off `$defs/ConstraintObject` instead of restating either, so a rule
+  added there reaches generated schemas with the version that introduces it, and
+  `examples/pet.filter.json` is regenerated: 12 of its 18 constraint objects gain the rule — the
+  ones offering `$unknownAs`, which is every field that can be absent or null. The `$comment`
+  justifying the rule is deliberately not copied along with it: a validator never reads it, and
+  one copy per field is charged by the token to whoever inlines the schema in a tool definition.
+  No change to the grammar — this is the generator agreeing with it.
+- **The narrowing property is now tested as a property.** `tests/generator.test.mjs` asserted it
+  over a hand-written list of fifteen filters, which can only re-check the leaks someone already
+  thought of — the keyword above was dropped for as long as the list existed. It now samples
+  filters out of each generated schema's own vocabulary (`tests/fuzz.mjs`, seeded, deterministic)
+  and asserts that every one the generated schema accepts is valid JQL, over three generated
+  schemas; the run is checked for not being vacuous, in that it must accept a fraction of its
+  samples and must reach every operator the schema offers. A second test pins what the generator
+  does with each instance-constraining keyword of `$defs/ConstraintObject`, so adding one there
+  fails the suite until it is handled.
+- **`experiments/filter-to-sql` rejects a constraint object with no predicate in it** rather than
+  emitting an empty expression — `malformed-query`, at the pointer of the offending clause. Both
+  schemas already reject these, so this only matters for a compiler reached another way, but the
+  failure it replaces was a `500` from the database.
 - The prose in [README §Errors](./README.md#errors), [COMPARISON.md §4](./COMPARISON.md), the
   OpenAPI examples and `experiments/filter-to-sql` follows: they now describe Problem Details as
   the recommended shape rather than the required one, and name the failing *condition* where they

README.md

@@ -321,7 +321,7 @@ What it decides, and why:
 - **Operators follow the type.** Ordering and ranges go to numbers and to `date`/`date-time`/`time` strings; pattern matching goes to free text but not to enums or opaque formats like `uuid`; `$some`/`$every` go to arrays and recurse into arrays of objects, with `$hasAll` added for arrays of scalars. `$exists` is omitted where the property is required all the way up, `$isNull` where the type does not admit null, and `$unknownAs` where the field can be neither absent nor null — all three would be constants.
 - **Operands follow the value domain.** `$eq`, `$in` and friends carry the field's `enum`, `pattern` and bounds. The ordering operators deliberately do not: `{"$gt": 0}` against a field whose `minimum` is 1 is a sensible predicate.
 - **Prose comes from the grammar**, not from the generator, so operator descriptions stay in one place. `--descriptions brief` (the default) keeps them for the operators people get wrong and drops them for `$eq` and `$gt`, which matters when the output goes into an MCP tool definition.
-- **Narrowing only.** Every filter the generated schema accepts is also valid against the published grammar, so a server implementing the published semantics evaluates it unchanged. `tests/generator.test.mjs` asserts this.
+- **Narrowing only.** Every filter the generated schema accepts is also valid against the published grammar, so a server implementing the published semantics evaluates it unchanged. The rule that makes this hold: the constraint object's own keywords are copied from the grammar, and only its *operator set* and *operand schemas* are narrowed. `tests/generator.test.mjs` asserts the property by sampling filters out of each generated schema's vocabulary — a list of examples can only re-check the leaks someone already thought of.
 
 Opt a property out, or override its operators, from the resource schema itself:
 

examples/pet.filter.json

@@ -304,6 +304,11 @@
       "description": "Taxon, coarse-grained.",
       "type": "object",
       "minProperties": 1,
+      "dependentSchemas": {
+        "$unknownAs": {
+          "minProperties": 2
+        }
+      },
       "properties": {
         "$eq": {
           "title": "$eq",
@@ -423,6 +428,11 @@
       "description": "Date of birth, RFC 3339 full-date.",
       "type": "object",
       "minProperties": 1,
+      "dependentSchemas": {
+        "$unknownAs": {
+          "minProperties": 2
+        }
+      },
       "properties": {
         "$eq": {
           "title": "$eq",
@@ -518,6 +528,11 @@
       "description": "Last recorded weight.",
       "type": "object",
       "minProperties": 1,
+      "dependentSchemas": {
+        "$unknownAs": {
+          "minProperties": 2
+        }
+      },
       "properties": {
         "$eq": {
           "title": "$eq",
@@ -608,6 +623,11 @@
       "description": "Whether the pet has been neutered.",
       "type": "object",
       "minProperties": 1,
+      "dependentSchemas": {
+        "$unknownAs": {
+          "minProperties": 2
+        }
+      },
       "properties": {
         "$eq": {
           "title": "$eq",
@@ -646,6 +666,11 @@
       "description": "Chip number, null if unchipped.",
       "type": "object",
       "minProperties": 1,
+      "dependentSchemas": {
+        "$unknownAs": {
+          "minProperties": 2
+        }
+      },
       "properties": {
         "$eq": {
           "title": "$eq",
@@ -869,6 +894,11 @@
       "description": "Free-form labels used for browsing.",
       "type": "object",
       "minProperties": 1,
+      "dependentSchemas": {
+        "$unknownAs": {
+          "minProperties": 2
+        }
+      },
       "properties": {
         "$eq": {
           "title": "$eq",
@@ -932,6 +962,11 @@
       "description": "Where the pet is housed.",
       "type": "object",
       "minProperties": 1,
+      "dependentSchemas": {
+        "$unknownAs": {
+          "minProperties": 2
+        }
+      },
       "properties": {
         "$exists": {
           "title": "$exists",
@@ -954,6 +989,11 @@
       "description": "Shelter name.",
       "type": "object",
       "minProperties": 1,
+      "dependentSchemas": {
+        "$unknownAs": {
+          "minProperties": 2
+        }
+      },
       "properties": {
         "$eq": {
           "title": "$eq",
@@ -1045,6 +1085,11 @@
       "description": "Shelter city.",
       "type": "object",
       "minProperties": 1,
+      "dependentSchemas": {
+        "$unknownAs": {
+          "minProperties": 2
+        }
+      },
       "properties": {
         "$eq": {
           "title": "$eq",
@@ -1140,6 +1185,11 @@
       "description": "Total kennels.",
       "type": "object",
       "minProperties": 1,
+      "dependentSchemas": {
+        "$unknownAs": {
+          "minProperties": 2
+        }
+      },
       "properties": {
         "$eq": {
           "title": "$eq",
@@ -1373,6 +1423,11 @@
       "description": "When the booster is due.",
       "type": "object",
       "minProperties": 1,
+      "dependentSchemas": {
+        "$unknownAs": {
+          "minProperties": 2
+        }
+      },
       "properties": {
         "$eq": {
           "title": "$eq",
@@ -1529,6 +1584,11 @@
       "description": "Vaccination history.",
       "type": "object",
       "minProperties": 1,
+      "dependentSchemas": {
+        "$unknownAs": {
+          "minProperties": 2
+        }
+      },
       "properties": {
         "$eq": {
           "title": "$eq",

experiments/filter-to-sql/README.md

@@ -32,13 +32,13 @@ the numbers and findings here are measured against v0.4.0.
 | --- | --- |
 | `compile.mjs` | the compiler: filter + binding → `{ sql, params, warnings }`, dialects `sqlite` and `postgres` |
 | `dataset.mjs` | 10 fixture records, and the two bindings the corpus is compiled under |
-| `cases.mjs` | 72 use cases, each with the row set SPEC.md says it should return |
+| `cases.mjs` | 73 use cases, each with the row set SPEC.md says it should return |
 | `harness.mjs` | an in-memory SQLite database, plus the ajv validation a server does first |
-| `compile.test.mjs` | the corpus, executed: 150 assertions |
+| `compile.test.mjs` | the corpus, executed: 152 assertions |
 | `run.mjs` | command line — compile a filter, print the SQL, run the corpus, print the measurements |
 
 ```bash
-npm run test:experiment                # the corpus, executed: 150 assertions
+npm run test:experiment                # the corpus, executed: 152 assertions
 npm run experiment -- --corpus         # every case, both bindings, side by side
 npm run experiment -- --metrics        # the numbers quoted below
 npm run experiment -- --sql b02        # one case, both bindings, both dialects
@@ -116,29 +116,29 @@ Treat every Postgres claim below as reasoned, not tested.
 ## Results
 
 ```
-130/130 case-binding pairs match the hand-derived expectation.
-150 assertions pass (65 cases × 2 bindings + 6 structural tests).
+146/146 case-binding pairs match the hand-derived expectation.
+152 assertions pass (73 cases × 2 bindings + 6 structural tests).
 ```
 
 | | |
 | --- | --- |
 | Operator names in `x-profiles` | 34 |
 | Compiled | 33 — every one except `$search` |
 | Faithful on both dialects | 31 — `$regex`/`$flags` are faithful on SQLite only |
-| Cases that compile | 63 of 72 |
-| Cases rejected at compile time, with the right §8 problem type | 8, plus `c09` in the column binding only |
+| Cases that compile | 63 of 73 |
+| Cases rejected at compile time, with the right §8 problem type | 9, plus `c09` in the column binding only |
 | Cases whose answer is legitimately binding-dependent | 1 (`c10`) |
 
 ## Where the cost went
 
-`compile.mjs` is 1003 lines, 743 of them code, against v0.4.0 of the grammar.
+`compile.mjs` is 1015 lines, 749 of them code, against v0.4.0 of the grammar.
 By section:
 
 | Section | Code lines | What it does |
 | --- | --- | --- |
 | Operators | 215 | one function per operator family |
 | Dialects | 104 | the SQLite and Postgres primitives |
-| Constraints and filters | 91 | the dispatch: 33 `case` labels, the AND/OR/NOT plumbing and `$unknownAs` |
+| Constraints and filters | 97 | the dispatch: 33 `case` labels, the AND/OR/NOT plumbing and `$unknownAs` |
 | Compilation context | 64 | parameter interning, §7 limits, §2.1 profile gating |
 | Field paths | 60 | the §3.2 path grammar |
 | Field resolution | 54 | path → binding → accessor |

experiments/filter-to-sql/cases.mjs

@@ -583,6 +583,14 @@ export const CASES = [
     expect: { problem: "malformed-query" },
     note: "Caught by the schema, not the compiler: additionalProperties: false on the constraint object. §3.3 is what makes this an error rather than a field named $eqq.",
   },
+  {
+    group: "F. What has to be rejected",
+    id: "f05b",
+    title: "A modifier with nothing to modify",
+    filter: { microchip: { $unknownAs: false } },
+    expect: { problem: "malformed-query" },
+    note: "$unknownAs resolves UNKNOWN for its siblings, so alone it has no predicate to wrap. Rejected by the schema (ConstraintObject.dependentSchemas) — and by the compiler too, because emitting coalesce((), FALSE) turns a 400 into a database syntax error.",
+  },
   {
     group: "F. What has to be rejected",
     id: "f06",

experiments/filter-to-sql/compile.mjs

@@ -921,6 +921,18 @@ function constraintObjectToSql(ctx, scope, acc, constraint, pointer, level) {
         throw new QueryProblem("malformed-query", `"${op}" is not an operator of this language`, ptr);
     }
   }
+  if (terms.length === 0) {
+    // Nothing here is a predicate: $unknownAs is a modifier and $flags is an
+    // argument to $regex, so there is no expression to emit. The grammar
+    // rejects both cases (ConstraintObject.dependentSchemas and
+    // .dependentRequired), and so does a generated schema — but a compiler
+    // reached anyway must say malformed-query rather than emit "()" and let
+    // the database answer with a syntax error.
+    const detail = keys.length === 0
+      ? "a constraint object needs at least one operator"
+      : `"${keys.join('", "')}" needs an operator to modify`;
+    throw new QueryProblem("malformed-query", detail, pointer);
+  }
   const conjunction = terms.length === 1 ? terms[0] : `(${terms.join(" AND ")})`;
 
   // SPEC §4.6: $unknownAs applies last, to the conjunction of its siblings and