diff --git a/CHANGELOG.md b/CHANGELOG.md index f1796a4..69e199a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -48,6 +48,37 @@ No change to the grammar or to the semantics of evaluation. The only edit to and the `type` URIs, the `pointer` member and the recovery members (`queryableFields`, `accepted`) are unchanged as its encoding. This relaxes a requirement, so nothing that conformed before stops conforming. + +### Fixed + +- **Generated filter schemas were not a narrowing** ([#8](https://github.com/christosgkoros/json-query-language/issues/8)). + `tools/generate-filter-schema.mjs` carried the published constraint object's + `dependentRequired` rule but not its `dependentSchemas` one, so `{"microchip": {"$unknownAs": + false}}` — a modifier with nothing to modify — passed a generated schema while + `query-language-schema.json` rejected it. A server following the documented path (generated + schema as the tool's `inputSchema`, published semantics behind it) then had to evaluate a filter + with no predicate in it; the SQL compiler in `experiments/filter-to-sql` emitted + `coalesce((), FALSE)` and the database answered with a syntax error. The generator now reads + both dependency keywords off `$defs/ConstraintObject` instead of restating either, so a rule + added there reaches generated schemas with the version that introduces it, and + `examples/pet.filter.json` is regenerated: 12 of its 18 constraint objects gain the rule — the + ones offering `$unknownAs`, which is every field that can be absent or null. The `$comment` + justifying the rule is deliberately not copied along with it: a validator never reads it, and + one copy per field is charged by the token to whoever inlines the schema in a tool definition. + No change to the grammar — this is the generator agreeing with it. +- **The narrowing property is now tested as a property.** `tests/generator.test.mjs` asserted it + over a hand-written list of fifteen filters, which can only re-check the leaks someone already + thought of — the keyword above was dropped for as long as the list existed. It now samples + filters out of each generated schema's own vocabulary (`tests/fuzz.mjs`, seeded, deterministic) + and asserts that every one the generated schema accepts is valid JQL, over three generated + schemas; the run is checked for not being vacuous, in that it must accept a fraction of its + samples and must reach every operator the schema offers. A second test pins what the generator + does with each instance-constraining keyword of `$defs/ConstraintObject`, so adding one there + fails the suite until it is handled. +- **`experiments/filter-to-sql` rejects a constraint object with no predicate in it** rather than + emitting an empty expression — `malformed-query`, at the pointer of the offending clause. Both + schemas already reject these, so this only matters for a compiler reached another way, but the + failure it replaces was a `500` from the database. - The prose in [README §Errors](./README.md#errors), [COMPARISON.md §4](./COMPARISON.md), the OpenAPI examples and `experiments/filter-to-sql` follows: they now describe Problem Details as the recommended shape rather than the required one, and name the failing *condition* where they diff --git a/README.md b/README.md index 16ed94a..51fb88d 100644 --- a/README.md +++ b/README.md @@ -321,7 +321,7 @@ What it decides, and why: - **Operators follow the type.** Ordering and ranges go to numbers and to `date`/`date-time`/`time` strings; pattern matching goes to free text but not to enums or opaque formats like `uuid`; `$some`/`$every` go to arrays and recurse into arrays of objects, with `$hasAll` added for arrays of scalars. `$exists` is omitted where the property is required all the way up, `$isNull` where the type does not admit null, and `$unknownAs` where the field can be neither absent nor null — all three would be constants. - **Operands follow the value domain.** `$eq`, `$in` and friends carry the field's `enum`, `pattern` and bounds. The ordering operators deliberately do not: `{"$gt": 0}` against a field whose `minimum` is 1 is a sensible predicate. - **Prose comes from the grammar**, not from the generator, so operator descriptions stay in one place. `--descriptions brief` (the default) keeps them for the operators people get wrong and drops them for `$eq` and `$gt`, which matters when the output goes into an MCP tool definition. -- **Narrowing only.** Every filter the generated schema accepts is also valid against the published grammar, so a server implementing the published semantics evaluates it unchanged. `tests/generator.test.mjs` asserts this. +- **Narrowing only.** Every filter the generated schema accepts is also valid against the published grammar, so a server implementing the published semantics evaluates it unchanged. The rule that makes this hold: the constraint object's own keywords are copied from the grammar, and only its *operator set* and *operand schemas* are narrowed. `tests/generator.test.mjs` asserts the property by sampling filters out of each generated schema's vocabulary — a list of examples can only re-check the leaks someone already thought of. Opt a property out, or override its operators, from the resource schema itself: diff --git a/examples/pet.filter.json b/examples/pet.filter.json index acbe9d6..0bf8e16 100644 --- a/examples/pet.filter.json +++ b/examples/pet.filter.json @@ -304,6 +304,11 @@ "description": "Taxon, coarse-grained.", "type": "object", "minProperties": 1, + "dependentSchemas": { + "$unknownAs": { + "minProperties": 2 + } + }, "properties": { "$eq": { "title": "$eq", @@ -423,6 +428,11 @@ "description": "Date of birth, RFC 3339 full-date.", "type": "object", "minProperties": 1, + "dependentSchemas": { + "$unknownAs": { + "minProperties": 2 + } + }, "properties": { "$eq": { "title": "$eq", @@ -518,6 +528,11 @@ "description": "Last recorded weight.", "type": "object", "minProperties": 1, + "dependentSchemas": { + "$unknownAs": { + "minProperties": 2 + } + }, "properties": { "$eq": { "title": "$eq", @@ -608,6 +623,11 @@ "description": "Whether the pet has been neutered.", "type": "object", "minProperties": 1, + "dependentSchemas": { + "$unknownAs": { + "minProperties": 2 + } + }, "properties": { "$eq": { "title": "$eq", @@ -646,6 +666,11 @@ "description": "Chip number, null if unchipped.", "type": "object", "minProperties": 1, + "dependentSchemas": { + "$unknownAs": { + "minProperties": 2 + } + }, "properties": { "$eq": { "title": "$eq", @@ -869,6 +894,11 @@ "description": "Free-form labels used for browsing.", "type": "object", "minProperties": 1, + "dependentSchemas": { + "$unknownAs": { + "minProperties": 2 + } + }, "properties": { "$eq": { "title": "$eq", @@ -932,6 +962,11 @@ "description": "Where the pet is housed.", "type": "object", "minProperties": 1, + "dependentSchemas": { + "$unknownAs": { + "minProperties": 2 + } + }, "properties": { "$exists": { "title": "$exists", @@ -954,6 +989,11 @@ "description": "Shelter name.", "type": "object", "minProperties": 1, + "dependentSchemas": { + "$unknownAs": { + "minProperties": 2 + } + }, "properties": { "$eq": { "title": "$eq", @@ -1045,6 +1085,11 @@ "description": "Shelter city.", "type": "object", "minProperties": 1, + "dependentSchemas": { + "$unknownAs": { + "minProperties": 2 + } + }, "properties": { "$eq": { "title": "$eq", @@ -1140,6 +1185,11 @@ "description": "Total kennels.", "type": "object", "minProperties": 1, + "dependentSchemas": { + "$unknownAs": { + "minProperties": 2 + } + }, "properties": { "$eq": { "title": "$eq", @@ -1373,6 +1423,11 @@ "description": "When the booster is due.", "type": "object", "minProperties": 1, + "dependentSchemas": { + "$unknownAs": { + "minProperties": 2 + } + }, "properties": { "$eq": { "title": "$eq", @@ -1529,6 +1584,11 @@ "description": "Vaccination history.", "type": "object", "minProperties": 1, + "dependentSchemas": { + "$unknownAs": { + "minProperties": 2 + } + }, "properties": { "$eq": { "title": "$eq", diff --git a/experiments/filter-to-sql/README.md b/experiments/filter-to-sql/README.md index 078fc76..56cfde9 100644 --- a/experiments/filter-to-sql/README.md +++ b/experiments/filter-to-sql/README.md @@ -32,13 +32,13 @@ the numbers and findings here are measured against v0.4.0. | --- | --- | | `compile.mjs` | the compiler: filter + binding → `{ sql, params, warnings }`, dialects `sqlite` and `postgres` | | `dataset.mjs` | 10 fixture records, and the two bindings the corpus is compiled under | -| `cases.mjs` | 72 use cases, each with the row set SPEC.md says it should return | +| `cases.mjs` | 73 use cases, each with the row set SPEC.md says it should return | | `harness.mjs` | an in-memory SQLite database, plus the ajv validation a server does first | -| `compile.test.mjs` | the corpus, executed: 150 assertions | +| `compile.test.mjs` | the corpus, executed: 152 assertions | | `run.mjs` | command line — compile a filter, print the SQL, run the corpus, print the measurements | ```bash -npm run test:experiment # the corpus, executed: 150 assertions +npm run test:experiment # the corpus, executed: 152 assertions npm run experiment -- --corpus # every case, both bindings, side by side npm run experiment -- --metrics # the numbers quoted below npm run experiment -- --sql b02 # one case, both bindings, both dialects @@ -116,8 +116,8 @@ Treat every Postgres claim below as reasoned, not tested. ## Results ``` -130/130 case-binding pairs match the hand-derived expectation. -150 assertions pass (65 cases × 2 bindings + 6 structural tests). +146/146 case-binding pairs match the hand-derived expectation. +152 assertions pass (73 cases × 2 bindings + 6 structural tests). ``` | | | @@ -125,20 +125,20 @@ Treat every Postgres claim below as reasoned, not tested. | Operator names in `x-profiles` | 34 | | Compiled | 33 — every one except `$search` | | Faithful on both dialects | 31 — `$regex`/`$flags` are faithful on SQLite only | -| Cases that compile | 63 of 72 | -| Cases rejected at compile time, with the right §8 problem type | 8, plus `c09` in the column binding only | +| Cases that compile | 63 of 73 | +| Cases rejected at compile time, with the right §8 problem type | 9, plus `c09` in the column binding only | | Cases whose answer is legitimately binding-dependent | 1 (`c10`) | ## Where the cost went -`compile.mjs` is 1003 lines, 743 of them code, against v0.4.0 of the grammar. +`compile.mjs` is 1015 lines, 749 of them code, against v0.4.0 of the grammar. By section: | Section | Code lines | What it does | | --- | --- | --- | | Operators | 215 | one function per operator family | | Dialects | 104 | the SQLite and Postgres primitives | -| Constraints and filters | 91 | the dispatch: 33 `case` labels, the AND/OR/NOT plumbing and `$unknownAs` | +| Constraints and filters | 97 | the dispatch: 33 `case` labels, the AND/OR/NOT plumbing and `$unknownAs` | | Compilation context | 64 | parameter interning, §7 limits, §2.1 profile gating | | Field paths | 60 | the §3.2 path grammar | | Field resolution | 54 | path → binding → accessor | diff --git a/experiments/filter-to-sql/cases.mjs b/experiments/filter-to-sql/cases.mjs index db6d12e..015fef2 100644 --- a/experiments/filter-to-sql/cases.mjs +++ b/experiments/filter-to-sql/cases.mjs @@ -583,6 +583,14 @@ export const CASES = [ expect: { problem: "malformed-query" }, note: "Caught by the schema, not the compiler: additionalProperties: false on the constraint object. §3.3 is what makes this an error rather than a field named $eqq.", }, + { + group: "F. What has to be rejected", + id: "f05b", + title: "A modifier with nothing to modify", + filter: { microchip: { $unknownAs: false } }, + expect: { problem: "malformed-query" }, + note: "$unknownAs resolves UNKNOWN for its siblings, so alone it has no predicate to wrap. Rejected by the schema (ConstraintObject.dependentSchemas) — and by the compiler too, because emitting coalesce((), FALSE) turns a 400 into a database syntax error.", + }, { group: "F. What has to be rejected", id: "f06", diff --git a/experiments/filter-to-sql/compile.mjs b/experiments/filter-to-sql/compile.mjs index 81b5be5..2d70e65 100644 --- a/experiments/filter-to-sql/compile.mjs +++ b/experiments/filter-to-sql/compile.mjs @@ -921,6 +921,18 @@ function constraintObjectToSql(ctx, scope, acc, constraint, pointer, level) { throw new QueryProblem("malformed-query", `"${op}" is not an operator of this language`, ptr); } } + if (terms.length === 0) { + // Nothing here is a predicate: $unknownAs is a modifier and $flags is an + // argument to $regex, so there is no expression to emit. The grammar + // rejects both cases (ConstraintObject.dependentSchemas and + // .dependentRequired), and so does a generated schema — but a compiler + // reached anyway must say malformed-query rather than emit "()" and let + // the database answer with a syntax error. + const detail = keys.length === 0 + ? "a constraint object needs at least one operator" + : `"${keys.join('", "')}" needs an operator to modify`; + throw new QueryProblem("malformed-query", detail, pointer); + } const conjunction = terms.length === 1 ? terms[0] : `(${terms.join(" AND ")})`; // SPEC §4.6: $unknownAs applies last, to the conjunction of its siblings and diff --git a/tests/fuzz.mjs b/tests/fuzz.mjs new file mode 100644 index 0000000..d8dcbcb --- /dev/null +++ b/tests/fuzz.mjs @@ -0,0 +1,173 @@ +/** + * fuzz.mjs — a seeded sampler of candidate filters for a generated filter + * schema. + * + * The narrowing property ("every filter the generated schema accepts is also + * valid against the published grammar") is universally quantified over filters, + * so a list of examples cannot establish it: it can only re-check the cases + * someone already thought of. This walks a generated schema and builds filters + * out of its own vocabulary instead, which puts enough of them inside the + * accepted set for the property to be worth testing. + * + * Sampling is *mostly* conformant — a uniformly random JSON value is rejected + * by both schemas and tells us nothing. Where a keyword bounds the shape of a + * value rather than its type (minProperties, minItems/maxItems, uniqueItems, + * and the dependency keywords, which are ignored outright) the sampler steps + * over the line on purpose: those are the keywords a generator drops silently, + * and a dropped one only shows up as a filter this schema accepts and the + * grammar does not. + * + * Deterministic: same seed, same filters, so a failure reproduces. + */ + +/** mulberry32 — small, seeded, and stable across Node versions. */ +export function rngFrom(seed) { + let a = seed >>> 0; + return () => { + a = (a + 0x6d2b79f5) | 0; + let t = Math.imul(a ^ (a >>> 15), 1 | a); + t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t; + return ((t ^ (t >>> 14)) >>> 0) / 4294967296; + }; +} + +/** Values by format, so a format-constrained operand is usually satisfiable. */ +const BY_FORMAT = { + date: ["2020-01-01", "2024-06-15", "1999-12-31", "not-a-date"], + "date-time": ["2024-01-01T00:00:00Z", "2020-06-15T12:30:00Z", "nope"], + time: ["12:00:00", "23:59:59"], + uuid: ["6f9619ff-8b86-d011-b42d-00c04fc964ff", "00000000-0000-0000-0000-000000000000", "abc"], + email: ["vet@example.com"], + uri: ["https://example.com/a"], + duration: ["P1Y"], +}; + +const STRINGS = [ + "Fido", "rabies", "cat", "available", "adopt-", "%amsterdam%", "^Fi", "x", "", + // Short alphabetic values so a pattern-constrained operand ($flags is + // "^[ims]{0,3}$") has something in the pool that matches it. + "i", "m", "s", "im", "ims", +]; + +const FOREIGN = ["Available", "unicorn", 42, true, null]; + +/** Keys whose value recurses into another filter or constraint object. */ +const RECURSIVE = new Set(["$and", "$or", "$nor", "$not", "$some", "$every"]); + +/** + * Builds a sampler over one generated schema. `maxDepth` bounds the nesting the + * logical operators would otherwise pursue forever, since $not and $some point + * back at the definition that offers them. + */ +export function sampler(schema, seed, { maxDepth = 4 } = {}) { + const rand = rngFrom(seed); + const pick = (list) => list[Math.floor(rand() * list.length)]; + const chance = (p) => rand() < p; + + const deref = (node) => { + let cur = node; + for (let hops = 0; cur && typeof cur === "object" && typeof cur.$ref === "string"; hops++) { + if (hops > 8) return null; + cur = cur.$ref === "#" ? schema : schema.$defs?.[cur.$ref.replace("#/$defs/", "")]; + } + return cur; + }; + + function sample(node, depth) { + const s = deref(node); + if (!s || typeof s !== "object") return pick(FOREIGN); + if ("const" in s) return s.const; + if (Array.isArray(s.enum)) return chance(0.9) ? pick(s.enum) : pick(FOREIGN); + if (Array.isArray(s.anyOf)) { + const { anyOf, ...rest } = s; + return sample({ ...rest, ...deref(pick(anyOf)) }, depth); + } + + const types = Array.isArray(s.type) ? s.type : s.type ? [s.type] : ["object"]; + const type = pick(types); + switch (type) { + case "null": + return null; + case "boolean": + return chance(0.5); + case "integer": + case "number": { + const lo = s.minimum ?? s.exclusiveMinimum ?? 0; + const hi = s.maximum ?? s.exclusiveMaximum ?? lo + 100; + const v = pick([lo, hi, (lo + hi) / 2, lo - 1, hi + 1, 3.5]); + return type === "integer" ? Math.round(v) : v; + } + case "string": { + let pool = s.format ? BY_FORMAT[s.format] ?? STRINGS : STRINGS; + if (s.pattern) { + const rx = new RegExp(s.pattern); + const matching = pool.filter((v) => rx.test(v)); + if (matching.length > 0) pool = matching; + } + return pick(pool); + } + case "array": { + const min = s.minItems ?? 0; + const max = Math.max(s.maxItems ?? min + 2, min); + let n = min + Math.floor(rand() * (max - min + 1)); + // Step outside the declared cardinality now and then: if the generator + // dropped minItems or maxItems, this is what finds it. + if (chance(0.08)) n = Math.max(0, n + (chance(0.5) ? 1 : -1)); + const items = Array.from({ length: n }, () => sample(s.items ?? {}, depth + 1)); + // Same for uniqueItems, which $in and $hasAll both carry. + if (items.length > 1 && chance(0.15)) items[1] = items[0]; + return items; + } + default: { + const names = Object.keys(s.properties ?? {}); + if (names.length === 0) return chance(0.5) ? {} : { vaccine: "rabies" }; + const usable = depth >= maxDepth ? names.filter((k) => !RECURSIVE.has(k)) : names; + const from = usable.length > 0 ? usable : names.filter((k) => k === "$not"); + if (from.length === 0) return {}; + // Deliberately blind to minProperties, dependentRequired and + // dependentSchemas: an operator subset is chosen freely, so a lone + // $unknownAs or a lone $flags is a filter this sampler will produce. + let k = 1 + Math.floor(rand() * Math.min(3, from.length)); + if (chance(0.05)) k = 0; + const shuffled = [...from]; + for (let i = shuffled.length - 1; i > 0; i--) { + const j = Math.floor(rand() * (i + 1)); + [shuffled[i], shuffled[j]] = [shuffled[j], shuffled[i]]; + } + const out = {}; + for (const key of shuffled.slice(0, k)) out[key] = sample(s.properties[key], depth + 1); + return out; + } + } + } + + return () => sample(schema, 0); +} + +/** Every operator named anywhere in a generated schema's constraint objects. */ +export function operatorsIn(schema) { + const found = new Set(); + const walk = (node) => { + if (!node || typeof node !== "object") return; + for (const key of Object.keys(node.properties ?? {})) { + if (key.startsWith("$") && !key.startsWith("$$")) found.add(key); + } + for (const value of Object.values(node)) walk(value); + }; + walk(schema); + return found; +} + +/** Every operator used by a filter, so a sample run can report its coverage. */ +export function operatorsUsed(filter, into = new Set()) { + if (Array.isArray(filter)) { + for (const item of filter) operatorsUsed(item, into); + return into; + } + if (!filter || typeof filter !== "object") return into; + for (const [key, value] of Object.entries(filter)) { + if (key.startsWith("$") && !key.startsWith("$$")) into.add(key); + operatorsUsed(value, into); + } + return into; +} diff --git a/tests/generator.test.mjs b/tests/generator.test.mjs index a13a388..7ab7987 100644 --- a/tests/generator.test.mjs +++ b/tests/generator.test.mjs @@ -8,6 +8,7 @@ import _Ajv2020 from "ajv/dist/2020.js"; import _addFormats from "ajv-formats"; import { generateFilterSchema } from "../tools/generate-filter-schema.mjs"; +import { sampler, operatorsIn, operatorsUsed } from "./fuzz.mjs"; const Ajv2020 = _Ajv2020.default ?? _Ajv2020; const addFormats = _addFormats.default ?? _addFormats; @@ -90,15 +91,128 @@ const ACCEPTED = [ for (const [name, filter] of ACCEPTED) { test(`accepts: ${name}`, () => { assert.ok(validate(filter), errs(validate)); + // Narrowing, on the cases README shows: each of these is legal JQL too. + assert.ok(validateGrammar(filter), errs(validateGrammar)); }); } +const SEED = 20260909; +const SAMPLES = 5000; + test("everything the generated schema accepts, the published grammar also accepts", () => { // The soundness property that makes generation safe: narrowing only. A filter // written against a generated schema is always a legal JQL filter, so a // server implementing the published semantics can evaluate it unchanged. - for (const [name, filter] of ACCEPTED) { - assert.ok(validateGrammar(filter), `${name}: ${errs(validateGrammar)}`); + // + // The property is quantified over every filter, so it is checked by sampling + // the generated schema's own vocabulary rather than by listing cases: a list + // can only re-check the leaks someone already thought of. The leak in #8 — + // a dependency keyword the generator dropped, so a lone $unknownAs passed — + // sat under a list of fifteen filters that could never have found it. + const targets = [ + ["pet", generated], + ["pet, core profile only", generateFilterSchema(pet, { profiles: ["core"] }).schema], + ["a recursive resource", generateFilterSchema(RECURSIVE_RESOURCE, { maxDepth: 2 }).schema], + ]; + + for (const [label, schema] of targets) { + const accepts = makeAjv().compile(schema); + const next = sampler(schema, SEED); + const covered = new Set(); + let accepted = 0; + + for (let i = 0; i < SAMPLES; i++) { + const filter = next(); + if (!accepts(filter)) continue; + accepted++; + operatorsUsed(filter, covered); + assert.ok( + validateGrammar(filter), + `${label}: accepted by the generated schema, rejected by the grammar:\n` + + `${JSON.stringify(filter)}\n${errs(validateGrammar)}`, + ); + } + + // A sampler that lands outside the schema every time would pass the loop + // above without testing anything, and so would one that never reaches an + // operator. Both are asserted rather than assumed. + assert.ok(accepted > SAMPLES / 10, `${label}: only ${accepted}/${SAMPLES} samples were accepted`); + const unreached = [...operatorsIn(schema)].filter((op) => !covered.has(op)).sort(); + assert.deepEqual(unreached, [], `${label}: no accepted sample exercised ${unreached.join(", ")}`); + } +}); + +test("a modifier cannot stand alone in a generated constraint object", () => { + // The regression from #8, at both levels it can occur: $unknownAs has nothing + // to modify, so the published grammar rejects it and a generated schema that + // accepted it would be wider than the grammar. `$flags` is the same shape of + // rule, on the operator that already had it. + for (const filter of [ + { microchip: { $unknownAs: false } }, + { microchip: { $unknownAs: true } }, + { microchip: { $not: { $unknownAs: true } } }, + { shelter: { $unknownAs: true } }, + { vaccinations: { $some: { boosterDue: { $unknownAs: true } } } }, + ]) { + assert.equal(validate(filter), false, `expected rejection: ${JSON.stringify(filter)}`); + assert.equal(validateGrammar(filter), false, `expected the grammar to reject it too: ${JSON.stringify(filter)}`); + } + // With something to modify, it is accepted again. + assert.ok(validate({ microchip: { $ne: "X", $unknownAs: true } }), errs(validate)); +}); + +test("no instance-constraining keyword of the published constraint object is dropped", () => { + // #8 was one keyword of $defs/ConstraintObject missing from every generated + // constraint object. This says what the generator does with each of them, so + // a keyword added to the published definition fails here — rather than in a + // filter someone's agent emits — until it is handled deliberately. + const source = grammar.$defs.ConstraintObject; + const carriesTriggers = (constraint, keyword) => { + for (const [trigger, rule] of Object.entries(source[keyword])) { + if (!(trigger in constraint.properties)) continue; + // Everything that constrains an instance has to survive; $comment is the + // one part that may be dropped, being prose a validator never reads. + const expected = Array.isArray(rule) + ? rule + : Object.fromEntries(Object.entries(rule).filter(([k]) => k !== "$comment")); + assert.deepEqual(constraint[keyword]?.[trigger], expected, `${constraint.title}: ${keyword}.${trigger}`); + } + return true; + }; + const handled = { + type: (c) => c.type === "object", + minProperties: (c) => c.minProperties >= source.minProperties, + additionalProperties: (c) => c.additionalProperties === false, + dependentSchemas: (c) => carriesTriggers(c, "dependentSchemas"), + dependentRequired: (c) => carriesTriggers(c, "dependentRequired"), + }; + + const ANNOTATIONS = new Set(["title", "description", "examples", "$comment", "properties"]); + assert.deepEqual( + Object.keys(source).filter((k) => !ANNOTATIONS.has(k)).sort(), + Object.keys(handled).sort(), + "an instance-constraining keyword of $defs/ConstraintObject has no rule here", + ); + + // A generated constraint object is one whose properties are all operators; + // a generated Filter carries field paths beside them. + const withFlags = generateFilterSchema(pet, { + profiles: ["core", "strings", "ranges", "collections", "regex"], + }).schema; + const constraints = [generated, withFlags].flatMap((schema) => + Object.values(schema.$defs).filter((def) => { + const names = Object.keys(def?.properties ?? {}); + return names.length > 0 && names.every((n) => n.startsWith("$")) && !("$and" in def.properties); + }), + ); + assert.ok(constraints.length > 10, `expected the pet schema to yield constraint objects, got ${constraints.length}`); + assert.ok(constraints.some((c) => "$flags" in c.properties), "no constraint object offering $flags was checked"); + assert.ok(constraints.some((c) => "$unknownAs" in c.properties), "no constraint object offering $unknownAs was checked"); + + for (const constraint of constraints) { + for (const [keyword, holds] of Object.entries(handled)) { + assert.ok(holds(constraint), `${constraint.title}: ${keyword} is not carried or narrowed`); + } } }); @@ -180,20 +294,21 @@ test("exclude and max-depth bound the surface", () => { assert.ok(shallow.warnings.some((w) => w.includes("max-depth"))); }); +const RECURSIVE_RESOURCE = { + $schema: "https://json-schema.org/draft/2020-12/schema", + $id: "https://example.com/node.json", + title: "Node", + type: "object", + required: ["id"], + properties: { + id: { type: "string" }, + parent: { $ref: "#" }, + children: { type: "array", items: { $ref: "#" } }, + }, +}; + test("a recursive resource schema terminates", () => { - const recursive = { - $schema: "https://json-schema.org/draft/2020-12/schema", - $id: "https://example.com/node.json", - title: "Node", - type: "object", - required: ["id"], - properties: { - id: { type: "string" }, - parent: { $ref: "#" }, - children: { type: "array", items: { $ref: "#" } }, - }, - }; - const { schema } = generateFilterSchema(recursive, { maxDepth: 2 }); + const { schema } = generateFilterSchema(RECURSIVE_RESOURCE, { maxDepth: 2 }); assert.ok(makeAjv().validateSchema(schema)); assert.ok("parent.id" in schema.properties, Object.keys(schema.properties).join(", ")); }); diff --git a/tools/generate-filter-schema.mjs b/tools/generate-filter-schema.mjs index 37b4279..13ac486 100644 --- a/tools/generate-filter-schema.mjs +++ b/tools/generate-filter-schema.mjs @@ -440,6 +440,35 @@ function emitSizeDef(ctx) { return { $ref: "#/$defs/Size" }; } +/** + * Carries the published constraint object's dependency keywords over to a + * generated one, for the triggers whose operator survived: $flags needs $regex + * beside it, and $unknownAs needs something to modify. + * + * These rules are part of what the grammar rejects, so a generated schema that + * drops one is *wider* than the grammar there — which is the one thing a + * generated schema may never be (README §"Narrowing only"). Reading them off + * the grammar rather than restating them here means a rule added to + * $defs/ConstraintObject reaches generated schemas with the version that + * introduced it. + */ +function dependencyRules(ctx, props) { + const source = ctx.grammar.$defs.ConstraintObject; + // The rule is carried; the $comment justifying it is not. That prose is + // written for someone reading the grammar, and here it would be one copy per + // field, charged by the token to whoever inlines this in a tool definition. + const constraining = ({ $comment, ...rest }) => structuredClone(rest); + const rules = {}; + for (const keyword of ["dependentSchemas", "dependentRequired"]) { + const kept = {}; + for (const [trigger, rule] of Object.entries(source[keyword] ?? {})) { + if (trigger in props) kept[trigger] = Array.isArray(rule) ? [...rule] : constraining(rule); + } + if (Object.keys(kept).length > 0) rules[keyword] = kept; + } + return rules; +} + /** Builds the constraint-object subschema for one field, and registers it. */ function emitConstraint(ctx, field, prefix) { const ops = operatorsFor(field, ctx); @@ -521,10 +550,10 @@ function emitConstraint(ctx, field, prefix) { ...(field.schema.description ? { description: field.schema.description } : {}), type: "object", minProperties: 1, + ...dependencyRules(ctx, props), properties: props, additionalProperties: false, }; - if (props.$flags) constraint.dependentRequired = { $flags: ["$regex"] }; ctx.defs[name] = constraint; field.constraintDef = name;