diff --git a/CHANGELOG.md b/CHANGELOG.md index e94e448..6b9bed9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,15 @@ # Changelog +## Unreleased source - 2026-10-18 + +- Extend `reference_exists` with optional exact metadata bindings, such as a + citation's digest and revision, while preserving existing normalized contracts. +- Share reference-index capacity across IDs and metadata: 100,000 indexed cells + and 64 MiB of hashed values. Release partial indexes on exhaustion and retain + only metadata hashes and row positions, without raw reference values. +- Add offline report review actions and an original synthetic citation example. + Public release artifacts and runtime capabilities are unchanged. + ## 0.1.7 - 2026-10-16 - Publish the verified GitHub CLI package with offline `verify --format html`, diff --git a/README.md b/README.md index d178254..3cea3a0 100644 --- a/README.md +++ b/README.md @@ -71,6 +71,12 @@ The command above verifies the included sample Skill. Replace the fixture path w ## What you get +- Current source can bind a citation to its exact declared source metadata with + `reference_exists.matches`. An existing ID with a different digest or revision + fails offline inspection without exposing record values. See the + [synthetic citation example](fixtures/product/citation-metadata/README.md). + This feature requires a source build and is not in GitHub v0.1.7 or npm 0.1.0. + - v0.1.7 produces an offline acceptance report with `artifacts --format html`. Reviewers see file binding, failed or incomplete identity checks, affected metadata and concrete review actions in a browser. The report includes no raw diff --git a/docs/artifact-delivery.md b/docs/artifact-delivery.md index dfe24c4..018107d 100644 --- a/docs/artifact-delivery.md +++ b/docs/artifact-delivery.md @@ -292,3 +292,49 @@ runs a local summarizer, preserves a SkillTape bundle, passes this checker and rejects corruption and an internally inconsistent summary in copies. None of those checks establishes customer-data truth, model/Agent acceptance or a live production/business integration. + +## Bind citations to exact source metadata + +Status: current source, not included in the published GitHub v0.1.7 package. + +A valid source ID alone does not establish that a citation uses the reviewed +revision or digest. Add an optional `matches` array to `reference_exists`: + +```json +{ + "type": "reference_exists", + "source": { "path": "citations.csv", "field": "source" }, + "target": { "path": "sources.csv", "field": "id" }, + "matches": [ + { "source": "digest", "target": "sha256" }, + { "source": "version", "target": "revision" } + ] +} +``` + +Declare each mapped column as a required nonempty string in its CSV rule. The +existing target ID must still be a unique nonempty string. One to eight distinct +column pairs are accepted. Matching preserves exact strings and pair order; +there is no trimming, case folding or separator-based tuple concatenation. +Existing contracts without `matches` keep their normalized digest and behavior. + +After all physical files are inspected, a matching ID with different metadata +produces `artifact.reference-value-mismatch`; missing valid metadata produces +`artifact.reference-value-unavailable`. Missing IDs retain the existing +`artifact.reference-missing` finding. Reports identify only declared paths, +column names and CSV data-record positions. Raw IDs, digest cell values and +version cell values do not enter JSON, text or HTML findings. + +ID and metadata indexes share a 100,000-cell budget and a 64 MiB aggregate +hash-input budget. Tuple values are streamed with byte-length framing rather +than joined into a large copied string. Exhaustion clears retained indexes and +fails the check without accepting partial reference evidence. Existing physical +file, row, UTF-8, formula, path and snapshot limits continue to apply. These are +input/work bounds; CPU and process RSS were not measured. + +Try the [original synthetic example](../fixtures/product/citation-metadata/README.md). +The interface is informed by source-linked citation review in +[WeKnora](https://github.com/Tencent/WeKnora), inspected on 2026-10-18. No upstream +code or dependencies were copied. This local check proves consistency between +observed metadata only. It does not fetch the cited source, authenticate a +publisher, or establish that the underlying source bytes or claims are true. diff --git a/fixtures/product/citation-metadata/README.md b/fixtures/product/citation-metadata/README.md new file mode 100644 index 0000000..9f43b9d --- /dev/null +++ b/fixtures/product/citation-metadata/README.md @@ -0,0 +1,20 @@ +# Citation metadata example + +Status: current source; not in the published v0.1.7 package. + +This original synthetic catalog binds each citation ID to the declared source +row and checks the exact digest and revision strings. It does not fetch a +source, authenticate an author, or validate a teaching rule. + +```sh +skillsync artifacts --contract fixtures/product/citation-metadata/contract.json \ + --path fixtures/product/citation-metadata/artifacts --format html > review.html +``` + +A source build is required. In a copied artifact directory, change a citation's +revision from `r1` to `r2` while keeping its source ID. The current-source check +reports `artifact.reference-value-mismatch`, the CSV data-record position and +the declared metadata columns. It does not copy the ID or cell values into +JSON, text or HTML. Preserve the original example when trying a correction. + +See the [contract guide](../../../docs/artifact-delivery.md#bind-citations-to-exact-source-metadata). diff --git a/fixtures/product/citation-metadata/artifacts/citations.csv b/fixtures/product/citation-metadata/artifacts/citations.csv new file mode 100644 index 0000000..d67e4f1 --- /dev/null +++ b/fixtures/product/citation-metadata/artifacts/citations.csv @@ -0,0 +1,2 @@ +source,digest,version +lesson-01,c01102f76ce762de460e7257b579800e99d38ea1832e3c62bbf89d684f307406,r1 diff --git a/fixtures/product/citation-metadata/artifacts/sources.csv b/fixtures/product/citation-metadata/artifacts/sources.csv new file mode 100644 index 0000000..af46192 --- /dev/null +++ b/fixtures/product/citation-metadata/artifacts/sources.csv @@ -0,0 +1,2 @@ +id,sha256,revision +lesson-01,c01102f76ce762de460e7257b579800e99d38ea1832e3c62bbf89d684f307406,r1 diff --git a/fixtures/product/citation-metadata/contract.json b/fixtures/product/citation-metadata/contract.json new file mode 100644 index 0000000..438c0a7 --- /dev/null +++ b/fixtures/product/citation-metadata/contract.json @@ -0,0 +1,74 @@ +{ + "schema": "skillsync.artifacts/v1", + "limits": { + "max_files": 2, + "max_file_bytes": 4096, + "max_total_bytes": 8192 + }, + "files": [ + { + "path": "citations.csv", + "format": "csv", + "columns": [ + { + "name": "source", + "type": "string" + }, + { + "name": "digest", + "type": "string" + }, + { + "name": "version", + "type": "string" + } + ], + "max_rows": 10 + }, + { + "path": "sources.csv", + "format": "csv", + "columns": [ + { + "name": "id", + "type": "string" + }, + { + "name": "sha256", + "type": "string" + }, + { + "name": "revision", + "type": "string" + } + ], + "unique_by": [ + "id" + ], + "max_rows": 10 + } + ], + "checks": [ + { + "type": "reference_exists", + "source": { + "path": "citations.csv", + "field": "source" + }, + "target": { + "path": "sources.csv", + "field": "id" + }, + "matches": [ + { + "source": "digest", + "target": "sha256" + }, + { + "source": "version", + "target": "revision" + } + ] + } + ] +} diff --git a/src/cli/commands/artifacts.ts b/src/cli/commands/artifacts.ts index d0d9792..9ebcfdd 100644 --- a/src/cli/commands/artifacts.ts +++ b/src/cli/commands/artifacts.ts @@ -6,7 +6,7 @@ import { z } from "zod"; import { parseArtifactContract } from "../../domain/artifact-contract.js"; import { parseStrictJson } from "../../domain/strict-json.js"; -import { inspectArtifactContent, type ArtifactFacts, type ArtifactFinding } from "../../domain/artifact-content.js"; +import { inspectArtifactContent, type ArtifactFacts, type ArtifactFinding, type ArtifactRowObserver } from "../../domain/artifact-content.js"; import { createArtifactReferenceIndex } from "../../domain/artifact-references.js"; import { createArtifactReconciliationIndex, type ReconciliationSummary } from "../../domain/artifact-reconciliation.js"; import { artifactSetHash, compareArtifactReports, parseArtifactReport, type ArtifactComparison } from "../../domain/artifact-report-comparison.js"; @@ -117,7 +117,11 @@ async function inspectArtifacts(options: ArtifactsOptions): Promise { + referenceRows?.(values, row); reconciliationRows?.(values, row); + } : undefined; + const inspected = inspectArtifactContent(rule, content, references.observer(path), observeRow); if (inspected.findings.length) reconciliation.invalidate(path); facts.set(path, inspected.facts); for (const finding of inspected.findings) if (report.findings.length < 256) report.findings.push(finding); diff --git a/src/domain/artifact-contract.ts b/src/domain/artifact-contract.ts index 3d868dc..7b93034 100644 --- a/src/domain/artifact-contract.ts +++ b/src/domain/artifact-contract.ts @@ -40,6 +40,7 @@ const comparison = z.object({ type: z.literal("csv_summary"), csv: safePath, jso const referenceField = z.object({ path: safePath, field: z.string().min(1).max(128) }).strict(); const reference = z.object({ type: z.literal("reference_exists"), source: referenceField, target: referenceField, require_all_targets: z.boolean().optional(), + matches: z.array(z.object({ source: z.string().min(1).max(128), target: z.string().min(1).max(128) }).strict()).min(1).max(8).optional(), }).strict(); const keyColumn = z.string().min(1).max(128); const keyedSide = z.object({ path: safePath, key: z.union([keyColumn, z.array(keyColumn).min(2).max(4)]), value: z.string().min(1).max(128) }).strict(); @@ -109,6 +110,14 @@ export const artifactContractSchema = z.object({ schema: z.literal("skillsync.ar if (sourceColumn?.type !== "string" || targetColumn?.type !== "string" || sourceColumn.allow_empty || targetColumn.allow_empty || target.unique_by.length !== 1 || target.unique_by[0] !== check.target.field) fail(); + if (check.matches) { + if (new Set(check.matches.map(pair => JSON.stringify(pair))).size !== check.matches.length) fail(); + for (const pair of check.matches) { + const left = source.columns.find(column => column.name === pair.source); + const right = target.columns.find(column => column.name === pair.target); + if (left?.type !== "string" || right?.type !== "string" || left.allow_empty || right.allow_empty) fail(); + } + } continue; } const csv = files.get(check.csv), json = files.get(check.json); diff --git a/src/domain/artifact-references.ts b/src/domain/artifact-references.ts index 2630968..7d5c388 100644 --- a/src/domain/artifact-references.ts +++ b/src/domain/artifact-references.ts @@ -1,14 +1,17 @@ import { createHash } from "node:crypto"; import type { ArtifactContract } from "./artifact-contract.js"; -import type { ArtifactCellObserver, ArtifactFinding } from "./artifact-content.js"; +import type { ArtifactCellObserver, ArtifactFinding, ArtifactRowObserver } from "./artifact-content.js"; export const MAX_REFERENCE_CELLS = 100_000; +const MAX_REFERENCE_HASH_BYTES = 64 * 1024 * 1024; type Index = { targets?: Map; sources?: Array<{ digest: string; row: number }>; referenced?: Set }; /** One invocation-local index: declared fields only, hashed IDs, fixed aggregate ceiling. */ export function createArtifactReferenceIndex(contract: ArtifactContract) { const checks = contract.checks.filter(check => check.type === "reference_exists"); const fields = new Map>(); + const metadata = checks.map(() => ({ source: new Map(), target: new Map() })); + const bindings = new Map>(); const index = (path: string, field: string): Index => { let columns = fields.get(path); if (!columns) { columns = new Map(); fields.set(path, columns); } @@ -21,7 +24,25 @@ export function createArtifactReferenceIndex(contract: ArtifactContract) { index(check.target.path, check.target.field).targets ??= new Map(); if (check.require_all_targets) index(check.source.path, check.source.field).referenced ??= new Set(); } - let cells = 0, exhausted = false; + checks.forEach((check, position) => { + if (!check.matches) return; + for (const side of ["source", "target"] as const) { + const path = check[side].path; + const roles = bindings.get(path) ?? []; + roles.push({ check: position, side }); bindings.set(path, roles); + } + }); + let cells = 0, hashBytes = 0, exhausted = false; + const charge = (count: number, bytes = 0): boolean => { + cells += count; hashBytes += bytes; + if (cells <= MAX_REFERENCE_CELLS && hashBytes <= MAX_REFERENCE_HASH_BYTES) return true; + exhausted = true; + for (const columns of fields.values()) for (const entry of columns.values()) { + entry.targets?.clear(); entry.referenced?.clear(); if (entry.sources) entry.sources.length = 0; + } + for (const entry of metadata) { entry.source.clear(); entry.target.clear(); } + return false; + }; return { observer(path: string): ArtifactCellObserver | undefined { const columns = fields.get(path); @@ -30,28 +51,52 @@ export function createArtifactReferenceIndex(contract: ArtifactContract) { const entry = columns.get(field); if (!entry || exhausted) return; // Repeated IDs count too. Stop hashing and release retained indexes at the ceiling. - cells += 1; - if (cells > MAX_REFERENCE_CELLS) { - exhausted = true; - for (const columns of fields.values()) for (const entry of columns.values()) { - entry.targets?.clear(); entry.referenced?.clear(); if (entry.sources) entry.sources.length = 0; - } - return; - } + if (!charge(1, Buffer.byteLength(value))) return; const digest = createHash("sha256").update("skillsync.references/v1\0string\0").update(value).digest("hex"); if (entry.targets && !entry.targets.has(digest)) entry.targets.set(digest, row); entry.sources?.push({ digest, row }); entry.referenced?.add(digest); }; }, + rowObserver(path: string): ArtifactRowObserver | undefined { + const roles = bindings.get(path); + if (!roles) return undefined; + return (values, row) => { + if (exhausted || row === undefined) return; + for (const role of roles) { + const pairs = checks[role.check].matches!; + if (!charge(pairs.length)) return; + const tuple = pairs.map(pair => values.get(pair[role.side])); + // Invalid cells never become a fabricated matching value. + if (!tuple.every((value): value is string => typeof value === "string" && value.length > 0)) continue; + const hasher = createHash("sha256").update("skillsync.reference-values/v1\0"); + for (const value of tuple) { + const bytes = Buffer.byteLength(value); + if (!charge(0, bytes)) return; + // Length framing keeps tuples exact without copying repeated large values. + hasher.update(String(bytes)).update("\0").update(value); + } + metadata[role.check][role.side].set(row, hasher.digest("hex")); + } + }; + }, findings(): ArtifactFinding[] { if (exhausted) return [{ code: "artifact.reference-capacity" }]; const findings: ArtifactFinding[] = []; - for (const check of checks) { + for (const [position, check] of checks.entries()) { const targets = index(check.target.path, check.target.field).targets!; for (const source of index(check.source.path, check.source.field).sources!) { if (!targets.has(source.digest)) { findings.push({ code: "artifact.reference-missing", path: check.source.path, field: check.source.field, row: source.row }); if (findings.length >= 256) return findings; + } else if (check.matches) { + const sourceValue = metadata[position].source.get(source.row); + const targetValue = metadata[position].target.get(targets.get(source.digest)!); + const unavailable = sourceValue === undefined || targetValue === undefined; + if (unavailable || sourceValue !== targetValue) { + findings.push({ code: unavailable ? "artifact.reference-value-unavailable" : "artifact.reference-value-mismatch", + path: check.source.path, field: check.matches.map(pair => pair.source), row: source.row }); + if (findings.length >= 256) return findings; + } } } if (check.require_all_targets) { diff --git a/src/reporters/artifacts-html.ts b/src/reporters/artifacts-html.ts index a977cd8..b7374d2 100644 --- a/src/reporters/artifacts-html.ts +++ b/src/reporters/artifacts-html.ts @@ -21,6 +21,8 @@ function columns(value: string | string[]): string { } function action(code: string): string { + if (code.startsWith("artifact.reference-value-")) return "Review the cited source record and declared metadata columns, such as digest or revision. Keep the original copy; do not accept an existing ID as proof that its metadata matches."; + if (code === "artifact.reference-missing") return "Review the source ID column and its declared target table. The cited record was not found; no source is fetched or authenticated."; if (code === "artifact.reconciliation-keys" || code === "artifact.reconciliation-sum") return "Compare expected and delivered records using the declared key columns. Preserve the original delivery while reviewing a corrected copy."; if (code.startsWith("artifact.reconciliation-")) return "No partial result is accepted. Review the declared types, integer range and selected-cell capacity before checking again."; if (code === "artifact.contract-invalid") return "Review the contract declaration and directory selection before inspecting files. No file acceptance is established."; diff --git a/tests/integration/reference-values.test.ts b/tests/integration/reference-values.test.ts new file mode 100644 index 0000000..667c8cb --- /dev/null +++ b/tests/integration/reference-values.test.ts @@ -0,0 +1,87 @@ +import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, describe, expect, it } from "vitest"; +import { renderArtifacts, runArtifacts } from "../../src/cli/commands/artifacts"; +import { parseArtifactContract } from "../../src/domain/artifact-contract"; +import { createArtifactReferenceIndex } from "../../src/domain/artifact-references"; + +const dirs: string[] = []; +afterEach(async () => { await Promise.all(dirs.splice(0).map(path => rm(path, { recursive: true, force: true }))); }); +function declaration() { + return { schema: "skillsync.artifacts/v1", limits: { max_files: 2, max_file_bytes: 4096, max_total_bytes: 8192 }, + files: [ + { path: "citations.csv", format: "csv", columns: ["source", "digest", "version"].map(name => ({ name, type: "string" })), max_rows: 10 }, + { path: "sources.csv", format: "csv", columns: ["id", "sha256", "revision"].map(name => ({ name, type: "string" })), unique_by: ["id"], max_rows: 10 }, + ], checks: [{ type: "reference_exists", source: { path: "citations.csv", field: "source" }, target: { path: "sources.csv", field: "id" }, + matches: [{ source: "digest", target: "sha256" }, { source: "version", target: "revision" }] }] }; +} +async function fixture(source = "source,digest,version\nPRIVATE_ID,known,v1\n", target = "id,sha256,revision\nPRIVATE_ID,known,v1\n", contract = declaration()) { + const root = await mkdtemp(join(tmpdir(), "reference-values-")); dirs.push(root); + const path = join(root, "files"); await mkdir(path); + await writeFile(join(path, "citations.csv"), source); + await writeFile(join(path, "sources.csv"), target); + const file = join(root, "contract.json"); await writeFile(file, JSON.stringify(contract)); + return { root, path, contract: file }; +} + +describe("reference metadata binding", () => { + it("binds different column names on repeated references without executing or fetching anything", async () => { + const report = await runArtifacts(await fixture("source,digest,version\nPRIVATE_ID,known,v1\nPRIVATE_ID,known,v1\n")); + expect(report.status).toBe("passed"); + expect(report.execution).toBe("not-run"); expect(report.provenance).toBe("not-authenticated"); + }); + it("rejects an existing citation ID with a changed digest or revision without disclosing values", async () => { + for (const row of ["PRIVATE_ID,PRIVATE_DIGEST,v1", "PRIVATE_ID,known,PRIVATE_VERSION"]) { + const report = await runArtifacts(await fixture(`source,digest,version\n${row}\n`)); + expect(report.exitCode).toBe(1); + expect(report.findings).toEqual([{ code: "artifact.reference-value-mismatch", path: "citations.csv", field: ["digest", "version"], row: 1 }]); + const rendered = ["json", "text", "html"].map(format => renderArtifacts(report, format)).join(""); + for (const value of ["PRIVATE_ID", "PRIVATE_DIGEST", "PRIVATE_VERSION"]) expect(rendered).not.toContain(value); + } + }); + it("uses unambiguous ordered tuples instead of concatenating metadata values", async () => { + const report = await runArtifacts(await fixture("source,digest,version\nPRIVATE_ID,a|b,c\n", "id,sha256,revision\nPRIVATE_ID,a,b|c\n")); + expect(report.findings[0].code).toBe("artifact.reference-value-mismatch"); + }); + it("keeps missing IDs separate from metadata mismatches", async () => { + const report = await runArtifacts(await fixture("source,digest,version\nMISSING,known,v1\n")); + expect(report.findings).toEqual([{ code: "artifact.reference-missing", path: "citations.csv", field: "source", row: 1 }]); + }); + it("never accepts missing metadata or duplicate target identities", async () => { + const missing = await runArtifacts(await fixture("source,digest,version\nPRIVATE_ID,,v1\n")); + expect(missing.exitCode).toBe(1); + expect(missing.findings.some(finding => finding.code === "artifact.reference-value-unavailable")).toBe(true); + const duplicate = await runArtifacts(await fixture(undefined, "id,sha256,revision\nPRIVATE_ID,known,v1\nPRIVATE_ID,other,v2\n")); + expect(duplicate.exitCode).toBe(1); + expect(duplicate.findings.some(finding => finding.code === "artifact.csv-duplicate")).toBe(true); + }); + it("rejects malformed bindings before inspecting the selected directory", async () => { + for (const mutation of ["missing", "integer", "empty", "duplicate", "too-many"] as const) { + const contract = declaration(); + if (mutation === "missing") contract.checks[0].matches[0].source = "undeclared"; + if (mutation === "integer") contract.files[0].columns[1].type = "integer"; + if (mutation === "empty") contract.checks[0].matches = []; + if (mutation === "duplicate") contract.checks[0].matches.push(contract.checks[0].matches[0]); + if (mutation === "too-many") contract.checks[0].matches = Array.from({ length: 9 }, () => contract.checks[0].matches[0]); + const options = await fixture(undefined, undefined, contract); + const report = await runArtifacts({ ...options, path: join(options.root, "absent") }); + expect(report.exitCode).toBe(2); expect(report.findings[0].code).toBe("artifact.contract-invalid"); + } + }); + it("charges metadata indexes against the existing shared reference budget", async () => { + const contract = declaration(); + contract.limits = { max_files: 2, max_file_bytes: 1_000_000, max_total_bytes: 2_000_000 }; + for (const file of contract.files) file.max_rows = 20_000; + const rows = Array.from({ length: 17_000 }, (_, i) => `ID${i},known,v1\n`).join(""); + const report = await runArtifacts(await fixture("source,digest,version\n" + rows, "id,sha256,revision\n" + rows, contract)); + expect(report.findings).toEqual([{ code: "artifact.reference-capacity" }]); expect(report.exitCode).toBe(1); + }); + it("bounds repeated metadata hashing even when the cell count stays small", () => { + const index = createArtifactReferenceIndex(parseArtifactContract(declaration())); + const observe = index.rowObserver("citations.csv")!; + const values = new Map([["digest", "x".repeat(8 * 1024 * 1024)], ["version", "v1"]]); + for (let row = 1; row <= 9; row += 1) observe(values, row); + expect(index.findings()).toEqual([{ code: "artifact.reference-capacity" }]); + }); +});