diff --git a/CHANGELOG.md b/CHANGELOG.md index 18c7e19..700263c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,12 @@ # Changelog +## 0.3.0 - 2026-10-20 + +- Review all validated cross-file declarations and inspection limits in offline HTML, text and JSON reports, including CSV summaries, reference metadata and keyed integer allocations. +- Keep declarations visible when inputs are missing, retain legacy baselines without declaration metadata, and omit expected scalar values, raw identities and cell contents. +- Bound the checklist to 128 declarations and eight visible mappings per declaration; preserve complete mappings in the validated JSON report. + + ## 0.2.1 - 2026-10-19 - Locate citation metadata failures in both the delivered table and referenced table, including data-record positions and declared column mappings. Raw IDs and cell values remain omitted. diff --git a/README.md b/README.md index 1a0b464..846f671 100644 --- a/README.md +++ b/README.md @@ -2,10 +2,10 @@ **Verify Agent Skills before you trust them.** SkillSync checks a local Skill's provenance, compatibility, and changes without executing it. -> **Alpha · GitHub release v0.2.1 · Node.js 20+** +> **Alpha · GitHub release v0.3.0 · Node.js 20+** > SkillSync performs offline checks of local Skill content. It does not execute Skill scripts, does not read credentials, and does not enable live provider, remote-worker, or runtime capabilities. -GitHub v0.2.1 locates citation metadata failures by file, data record and compared columns. It checks exact digests and revisions without copying record values into reports. The offline review, correction workflow and bounded composite-key reconciliation remain available. The npm registry stays at 0.1.0; use the verified GitHub archive for current features. +GitHub v0.3.0 adds a declared cross-file checklist to HTML, text and JSON reports, including paths, field mappings and capacity limits. Declarations remain separate from actual results, and expected values are omitted. [Review the declaration list](docs/declared-checks.md). It also locates citation metadata failures by file, data record and compared columns. It checks exact digests and revisions without copying record values into reports. The offline review, correction workflow and bounded composite-key reconciliation remain available. The npm registry stays at 0.1.0; use the verified GitHub archive for current features. [![Terminal demo](https://raw.githubusercontent.com/Chumaniac/skillsync/main/docs/assets/verify-demo.svg)](https://github.com/Chumaniac/skillsync/blob/main/docs/assets/verify-demo.svg) @@ -15,7 +15,7 @@ GitHub v0.2.1 locates citation metadata failures by file, data record and compar Tagged builds now publish a CLI tarball, checksums, a validated CycloneDX SBOM and a source manifest on [GitHub Releases](https://github.com/Chumaniac/skillsync/releases). -[GitHub v0.2.1](https://github.com/Chumaniac/skillsync/releases/tag/v0.2.1) +[GitHub v0.3.0](https://github.com/Chumaniac/skillsync/releases/tag/v0.3.0) is distributed as a versioned CLI tarball with checksums, SBOM, source manifest and GitHub provenance. Its release workflow requires the exact immutable tag and an already successful main check before packaging. Earlier v0.1.7 assets @@ -24,12 +24,12 @@ Installation uses Node.js 20+ and fetches the package's public dependencies; it needs no npm login. ```bash -release_base="https://github.com/Chumaniac/skillsync/releases/download/v0.2.1" -for asset in chumanic-skillsync-0.2.1.tgz checksums.txt sbom.cyclonedx.json release-manifest.json; do +release_base="https://github.com/Chumaniac/skillsync/releases/download/v0.3.0" +for asset in chumanic-skillsync-0.3.0.tgz checksums.txt sbom.cyclonedx.json release-manifest.json; do curl --fail --location --output "$asset" "$release_base/$asset" done shasum -a 256 -c checksums.txt -npm install --prefix ./skillsync-tools ./chumanic-skillsync-0.2.1.tgz +npm install --prefix ./skillsync-tools ./chumanic-skillsync-0.3.0.tgz ./skillsync-tools/node_modules/.bin/skillsync --version ``` @@ -47,7 +47,7 @@ npm install -g @chumanic/skillsync@0.1.0 skillsync verify --path . --target codex ``` -**From source (0.2.1):** +**From source (0.3.0):** ```bash git clone https://github.com/Chumaniac/skillsync.git diff --git a/docs/declared-checks.md b/docs/declared-checks.md new file mode 100644 index 0000000..246e923 --- /dev/null +++ b/docs/declared-checks.md @@ -0,0 +1,17 @@ +# Review declared cross-file checks + +Artifact text, JSON and standalone HTML reports now include the validated cross-file declarations and file/byte limits. The three supported kinds are CSV summaries, reference existence/metadata bindings, and integer reconciliation by identity. This lets a reviewer see the selected paths, fields, identity columns and mappings without reopening the contract. + +Declarations describe expectations; they are separate from completed observations and findings. They remain visible when a required file is missing. Invalid contracts have no inferred declaration list, and legacy JSON baselines without this optional metadata remain accepted. Per-file type, digest and scalar-value rules remain in the original contract: expected values, enum choices, raw IDs and cell values are not copied into the list. + +The contract/parser limit is 128 cross-file checks. HTML shows all 128 and up to 8 mappings per check, clips metadata at 160 characters and has no scripts or external requests. JSON retains complete declared mappings; report input remains bounded and validated when used as a baseline. + +Method reference: [CLI-Anything's structured and verifiable output](https://github.com/HKUDS/CLI-Anything/blob/34f519533bc175d2fe287ab8316b0dd99bb9cc43/README.md), Apache-2.0, reviewed from the 2026-10-19 Trending snapshot. Only the output-review method is borrowed; no upstream executable or plugin is installed. + +From a source checkout, try the original synthetic fixture: + +```sh +node dist/cli/index.js artifacts --contract fixtures/product/declared-checks/contract.json --path fixtures/product/declared-checks/artifacts --format html > declared-checks.html +``` + +The packaged CLI includes the same fixture under its package directory. Open the resulting HTML and select **Declared checks** to review all three kinds before inspecting outcomes. diff --git a/fixtures/product/declared-checks/artifacts/actual.csv b/fixtures/product/declared-checks/artifacts/actual.csv new file mode 100644 index 0000000..c4eb6c0 --- /dev/null +++ b/fixtures/product/declared-checks/artifacts/actual.csv @@ -0,0 +1,2 @@ +id,cents,revision +synthetic-order-a,100,fixture-v1 diff --git a/fixtures/product/declared-checks/artifacts/expected.csv b/fixtures/product/declared-checks/artifacts/expected.csv new file mode 100644 index 0000000..c4eb6c0 --- /dev/null +++ b/fixtures/product/declared-checks/artifacts/expected.csv @@ -0,0 +1,2 @@ +id,cents,revision +synthetic-order-a,100,fixture-v1 diff --git a/fixtures/product/declared-checks/artifacts/summary.json b/fixtures/product/declared-checks/artifacts/summary.json new file mode 100644 index 0000000..2a31548 --- /dev/null +++ b/fixtures/product/declared-checks/artifacts/summary.json @@ -0,0 +1 @@ +{"rows":1,"cents":100} diff --git a/fixtures/product/declared-checks/contract.json b/fixtures/product/declared-checks/contract.json new file mode 100644 index 0000000..26775ee --- /dev/null +++ b/fixtures/product/declared-checks/contract.json @@ -0,0 +1,112 @@ +{ + "schema": "skillsync.artifacts/v1", + "limits": { + "max_files": 3, + "max_file_bytes": 4096, + "max_total_bytes": 8192 + }, + "files": [ + { + "path": "expected.csv", + "format": "csv", + "columns": [ + { + "name": "id", + "type": "string" + }, + { + "name": "cents", + "type": "integer" + }, + { + "name": "revision", + "type": "string" + } + ], + "unique_by": [ + "id" + ], + "max_rows": 10 + }, + { + "path": "actual.csv", + "format": "csv", + "columns": [ + { + "name": "id", + "type": "string" + }, + { + "name": "cents", + "type": "integer" + }, + { + "name": "revision", + "type": "string" + } + ], + "unique_by": [ + "id" + ], + "max_rows": 10 + }, + { + "path": "summary.json", + "format": "json", + "fields": [ + { + "name": "rows", + "type": "integer" + }, + { + "name": "cents", + "type": "integer" + } + ] + } + ], + "checks": [ + { + "type": "csv_summary", + "csv": "actual.csv", + "json": "summary.json", + "row_count_field": "rows", + "sums": [ + { + "column": "cents", + "field": "cents" + } + ] + }, + { + "type": "reference_exists", + "source": { + "path": "actual.csv", + "field": "id" + }, + "target": { + "path": "expected.csv", + "field": "id" + }, + "matches": [ + { + "source": "revision", + "target": "revision" + } + ] + }, + { + "type": "keyed_integer_sum_equals", + "source": { + "path": "expected.csv", + "key": "id", + "value": "cents" + }, + "target": { + "path": "actual.csv", + "key": "id", + "value": "cents" + } + } + ] +} diff --git a/package-lock.json b/package-lock.json index e7aabcc..a0363a9 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "@chumanic/skillsync", - "version": "0.2.1", + "version": "0.3.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@chumanic/skillsync", - "version": "0.2.1", + "version": "0.3.0", "license": "MIT", "dependencies": { "commander": "^13.1.0", diff --git a/package.json b/package.json index 3967b0e..a4df2e8 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@chumanic/skillsync", - "version": "0.2.1", + "version": "0.3.0", "private": false, "publishConfig": { "access": "public" diff --git a/src/cli/commands/artifacts.ts b/src/cli/commands/artifacts.ts index 36c0a4a..51cbfc8 100644 --- a/src/cli/commands/artifacts.ts +++ b/src/cli/commands/artifacts.ts @@ -4,7 +4,7 @@ import { join } from "node:path"; import { parse as parseYaml } from "yaml"; import { z } from "zod"; -import { parseArtifactContract } from "../../domain/artifact-contract.js"; +import { parseArtifactContract, type ArtifactDeclarations } from "../../domain/artifact-contract.js"; import { parseStrictJson } from "../../domain/strict-json.js"; import { inspectArtifactContent, type ArtifactFacts, type ArtifactFinding, type ArtifactRowObserver } from "../../domain/artifact-content.js"; import { createArtifactReferenceIndex } from "../../domain/artifact-references.js"; @@ -30,6 +30,7 @@ export type ArtifactReport = { files: Array<{ path: string; bytes: number; sha256: string; rows?: number }>; findings: ArtifactFinding[]; reconciliations?: ReconciliationSummary[]; + declarations?: ArtifactDeclarations; comparison?: ArtifactComparison; }; @@ -70,6 +71,8 @@ async function inspectArtifacts(options: ArtifactsOptions): Promise ({ ordinal: index + 1, check })) }; } catch { issue("artifact.contract-invalid"); report.exitCode = 2; return report; } let root = options.path ?? join(options.delivery!, "artifacts"); try { @@ -175,6 +178,7 @@ export function renderArtifacts(report: ArtifactReport, format = "text"): string ...(safe.comparison ? [`Baseline comparison: ${safe.comparison.status}${safe.comparison.reason ? `; ${safe.comparison.reason}` : ""}`, ...(safe.comparison.findings ? [`Finding categories: ${safe.comparison.findings.added} new; ${safe.comparison.findings.ongoing} ongoing; ${safe.comparison.findings.resolved} resolved`] : [])] : []), ...(safe.comparison?.checks ? [`Record check changes: ${safe.comparison.checks.filter(check => check.change === "improved").length} improved; ${safe.comparison.checks.filter(check => check.change === "regressed").length} regressed; ${safe.comparison.checks.filter(check => check.change === "changed").length} mixed changes; ${safe.comparison.checks.filter(check => check.change === "unchanged").length} unchanged`] : []), + ...(safe.declarations ? [`Declared cross-file checks: ${safe.declarations.checks.length}`, `Inspection limits: ${JSON.stringify(safe.declarations.limits)}`, ...safe.declarations.checks.map(item => `Declaration ${item.ordinal}: ${JSON.stringify(item.check)}`)] : ["Declaration metadata: unavailable in this snapshot"]), `Files: ${safe.files.length}`, ...(safe.reconciliations ?? []).map(item => `Record reconciliation: ${item.status}${item.status !== "unknown" ? `; source keys=${item.sourceKeys}; target keys=${item.targetKeys}; missing=${item.missingKeys}; unexpected=${item.unexpectedKeys}; mismatched values=${item.valueMismatches}` : ""}`), ...safe.findings.map(finding => `${finding.code}${finding.path ? ` file=${JSON.stringify(finding.path)}` : ""}${finding.field ? ` field=${JSON.stringify(finding.field)}` : ""}${finding.row !== undefined ? ` row=${finding.row}` : ""}${finding.reference ? ` reference=${JSON.stringify(finding.reference)}` : ""}${finding.message ? ` ${finding.message}` : ""}`)].join("\n") + "\n"; } diff --git a/src/domain/artifact-contract.ts b/src/domain/artifact-contract.ts index 7b93034..2fe0d0c 100644 --- a/src/domain/artifact-contract.ts +++ b/src/domain/artifact-contract.ts @@ -46,14 +46,25 @@ const keyColumn = z.string().min(1).max(128); const keyedSide = z.object({ path: safePath, key: z.union([keyColumn, z.array(keyColumn).min(2).max(4)]), value: z.string().min(1).max(128) }).strict(); const reconciliation = z.object({ type: z.literal("keyed_integer_sum_equals"), source: keyedSide, target: keyedSide }).strict(); -export const artifactContractSchema = z.object({ schema: z.literal("skillsync.artifacts/v1"), - limits: z.object({ +const checkSchema = z.discriminatedUnion("type", [comparison, reference, reconciliation]); +const limitsSchema = z.object({ max_files: z.number().int().positive().max(WORKSPACE_TREE_LIMITS.maxFiles), max_file_bytes: z.number().int().positive().max(WORKSPACE_TREE_LIMITS.maxFileBytes), max_total_bytes: z.number().int().positive().max(WORKSPACE_TREE_LIMITS.maxTotalBytes), - }).strict(), + }).strict(); + +export const artifactDeclarationsSchema = z.object({ + limits: limitsSchema, + checks: z.array(z.object({ ordinal: z.number().int().min(1).max(128), check: checkSchema }).strict()).max(128), +}).strict().superRefine((value, context) => { + if (value.checks.some((item, index) => item.ordinal !== index + 1)) context.addIssue({ code: z.ZodIssueCode.custom, message: "declaration ordinals must be consecutive" }); +}); +export type ArtifactDeclarations = z.infer; + +export const artifactContractSchema = z.object({ schema: z.literal("skillsync.artifacts/v1"), + limits: limitsSchema, files: z.array(z.discriminatedUnion("format", [bytes, csv, json])).max(WORKSPACE_TREE_LIMITS.maxFiles), - checks: z.array(z.discriminatedUnion("type", [comparison, reference, reconciliation])).max(128).default([]), + checks: z.array(checkSchema).max(128).default([]), }).strict().superRefine((contract, context) => { const fail = () => context.addIssue({ code: z.ZodIssueCode.custom, message: "inconsistent artifact declarations" }); const files = new Map(contract.files.map(file => [file.path, file])); diff --git a/src/domain/artifact-report-comparison.ts b/src/domain/artifact-report-comparison.ts index 0efcb38..507a69a 100644 --- a/src/domain/artifact-report-comparison.ts +++ b/src/domain/artifact-report-comparison.ts @@ -1,5 +1,6 @@ import { createHash } from "node:crypto"; import { z } from "zod"; +import { artifactDeclarationsSchema } from "./artifact-contract.js"; import type { ArtifactReport } from "../cli/commands/artifacts.js"; const sha = z.string().regex(/^[0-9a-f]{64}$/); @@ -54,6 +55,7 @@ const reportSchema = z.object({ schema: z.literal("skillsync.artifacts-report/v1 }).strict().optional(), }).strict()).max(256), reconciliations: z.array(summary).max(128).optional(), comparison: artifactComparisonSchema.optional(), + declarations: artifactDeclarationsSchema.optional(), }).strict().superRefine((report, context) => { if (new Set(report.files.map(file => file.path)).size !== report.files.length || report.files.reduce((total, file) => total + file.bytes, 0) > 64 * 1024 * 1024 || diff --git a/src/reporters/artifacts-html.ts b/src/reporters/artifacts-html.ts index a1f8bd2..c23389e 100644 --- a/src/reporters/artifacts-html.ts +++ b/src/reporters/artifacts-html.ts @@ -87,6 +87,29 @@ function comparisonSection(report: ArtifactReport): string {
Baseline report identity${text(value.baseline_report_sha256)}

SHA-256 binds the supplied JSON bytes; it does not establish their author or truth.

`; } +function declarationSection(report: ArtifactReport): string { + const value = report.declarations; + if (!value) return '

Declared cross-file checks

Declaration metadata is unavailable in this snapshot. Consult its original contract; no expectations are inferred.

'; + return `

Declared cross-file checks

+

These are declarations, not proof of a completed check. Per-file type, digest and scalar-value rules remain in the original contract; expected values and raw cells are omitted here.

+

Limits: ${text(value.limits.max_files)} files · ${text(value.limits.max_file_bytes)} bytes per file · ${text(value.limits.max_total_bytes)} total bytes.

+

Showing ${Math.min(128, value.checks.length)} of ${value.checks.length} declarations. Up to 8 mappings per declaration are shown; JSON retains the complete declared mappings.

+ ${value.checks.slice(0, 128).map(({ ordinal, check }) => { + let source: string, target: string, fields: string; + if (check.type === "csv_summary") { + source = check.csv; target = check.json; + fields = `Row count → ${text(check.row_count_field)}
    ${check.sums.slice(0, 8).map(pair => `
  • ${text(pair.column)} → ${text(pair.field)}
    • `).join("")}
${Math.min(8, check.sums.length)} of ${check.sums.length} sum mappings shown.`; + } else if (check.type === "reference_exists") { + source = check.source.path; target = check.target.path; + fields = `Lookup: ${text(check.source.field)} → ${text(check.target.field)}

Require all targets: ${check.require_all_targets === true ? "yes" : "no"}

    ${(check.matches ?? []).slice(0, 8).map(pair => `
  • ${text(pair.source)} → ${text(pair.target)}
    • `).join("")}
`; + } else { + source = check.source.path; target = check.target.path; + fields = `Identity: ${columns(check.source.key)} → ${columns(check.target.key)}

Integer value: ${text(check.source.value)} → ${text(check.target.value)}

`; + } + return `

Declaration ${text(ordinal)} · ${text(check.type)}

Source: ${text(source)} → Target: ${text(target)}

${fields}
`; + }).join("") || '

No cross-file check was declared. Consult the original contract for file-level requirements.

'}
`; +} + /** Use the same path policy as text/JSON, including when called directly. */ export function renderArtifactsHtml(rawReport: ArtifactReport): string { const report = redactLocalPaths(rawReport); @@ -113,13 +136,14 @@ export function renderArtifactsHtml(rawReport: ArtifactReport): string { .review-summary{display:flex;flex-wrap:wrap;align-items:center;justify-content:space-between;gap:18px;padding-top:18px;margin-top:18px;border-top:1px solid #d6e0e5}.review-summary p{margin:0;font-variant-numeric:tabular-nums}.review-link{display:inline-flex;align-items:center;min-height:44px;padding:8px 14px;background:#075f70;color:white;border-radius:6px;text-decoration:none;font-weight:650}.review-link:hover{background:#064956}.reference-location{border-top:1px solid #d6e0e5;margin-top:14px;padding-top:12px;font-size:14px}.reference-location strong,.reference-location span,.reference-location small{display:block}.reference-location ul{list-style:none;padding:0;margin:8px 0}.reference-location li{margin:5px 0}.reference-location code{overflow-wrap:anywhere}h2[tabindex="-1"]{scroll-margin-top:24px}h2[tabindex="-1"]:focus{outline:3px solid #207d94;outline-offset:5px} -
SkillSync
Offline delivery review
+
SkillSync
Offline delivery review
${passed ? "Passed" : "Not accepted"}

${title}

${passed ? "The inspected files match this contract. Review the expectations and responsible approval before using the result." : "Start with the reported file and field locations below. Keep the original delivery while reviewing corrections."}

${report.files.length} inspected ${report.files.length === 1 ? "file" : "files"} · ${report.findings.length} reported ${report.findings.length === 1 ? "finding" : "findings"}${checks.length ? ` · ${failedChecks} failed and ${incomplete} incomplete allocation checks` : ""}

${report.findings.length ? `Review ${report.findings.length} reported ${report.findings.length === 1 ? "finding" : "findings"}` : "Review inspected files"}
Receipt / file binding${text(report.receipt_binding)}Hash consistency with supplied local evidence; author identity is not authenticated.
${comparisonSection(report)} +${declarationSection(report)}

Allocation and identity checks

Showing ${Math.min(CHECK_DETAILS, checks.length)} of ${checks.length} reconciliation checks. Failed and incomplete checks appear first.

${ordered.slice(0, CHECK_DETAILS).map(check => reconciliationCard(check, rowBudget)).join("") || "

No record reconciliation was declared. Other declared file checks are reflected in the result.

"}

Review actions

${referenceCount > REFERENCE_DETAILS ? `

Reference locations: first ${REFERENCE_DETAILS} of ${referenceCount} reported findings shown. The JSON report retains all location metadata.

` : ""}${report.findings.length ? `
${report.findings.slice(0, 256).map(finding => findingRow(finding, referenceBudget)).join("")}
FindingAffected metadataNext review step
` : "

No finding was reported for the declared scope. This does not authenticate source truth or establish business approval.

"}
diff --git a/tests/cli/help.test.ts b/tests/cli/help.test.ts index 4147a68..aad007b 100644 --- a/tests/cli/help.test.ts +++ b/tests/cli/help.test.ts @@ -12,7 +12,7 @@ describe("skillsync CLI", () => { it("reports the installed source version and lists the verification commands", async () => { const version = await runCli(["--version"]); expect(version.exitCode).toBe(0); - expect(version.stdout).toBe("0.2.1\n"); + expect(version.stdout).toBe("0.3.0\n"); const result = await runCli(["--help"]); diff --git a/tests/cli/report.test.ts b/tests/cli/report.test.ts index d0fd606..1bae851 100644 --- a/tests/cli/report.test.ts +++ b/tests/cli/report.test.ts @@ -255,7 +255,7 @@ describe("skillsync report", () => { const json = await runCli(["report", "--before", beforePath, "--after", afterPath, "--format", "json"]); expect(JSON.parse(json.stdout).conclusion).toBe("verified"); - expect(JSON.parse(json.stdout).toolVersion).toBe("0.2.1"); + expect(JSON.parse(json.stdout).toolVersion).toBe("0.3.0"); const sarif = await runCli(["report", "--before", beforePath, "--after", afterPath, "--format", "sarif"]); const parsedSarif = JSON.parse(sarif.stdout) as { @@ -266,7 +266,7 @@ describe("skillsync report", () => { }>; }; expect(parsedSarif.version).toBe("2.1.0"); - expect(parsedSarif.runs[0]?.tool.driver.version).toBe("0.2.1"); + expect(parsedSarif.runs[0]?.tool.driver.version).toBe("0.3.0"); expect(parsedSarif.runs[0]?.results[0]?.properties.issueId).toMatch(/^iss_/); }); diff --git a/tests/docs/documentation.test.ts b/tests/docs/documentation.test.ts index dcdd041..2dd2f62 100644 --- a/tests/docs/documentation.test.ts +++ b/tests/docs/documentation.test.ts @@ -39,10 +39,10 @@ describe("release documentation", () => { }; expect(readme).toContain("Verify Agent Skills before you trust them."); - expect(readme).toContain("Alpha · GitHub release v0.2.1 · Node.js 20+"); + expect(readme).toContain("Alpha · GitHub release v0.3.0 · Node.js 20+"); expect(readme).toContain("GitHub Releases"); - expect(readme).toContain("download/v0.2.1"); - expect(readme).toContain("chumanic-skillsync-0.2.1.tgz"); + expect(readme).toContain("download/v0.3.0"); + expect(readme).toContain("chumanic-skillsync-0.3.0.tgz"); expect(readme).toContain("## Install"); expect(readme).toContain("npx --yes @chumanic/skillsync@0.1.0 verify"); expect(readme).toContain([ diff --git a/tests/integration/artifact-declarations.test.ts b/tests/integration/artifact-declarations.test.ts new file mode 100644 index 0000000..1db5f01 --- /dev/null +++ b/tests/integration/artifact-declarations.test.ts @@ -0,0 +1,54 @@ +import { mkdtemp, mkdir, writeFile, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, describe, expect, it } from "vitest"; +import { renderArtifacts, runArtifacts } from "../../src/cli/commands/artifacts.js"; +import { parseArtifactReport } from "../../src/domain/artifact-report-comparison.js"; + +const directories: string[] = []; +afterEach(async () => { await Promise.all(directories.splice(0).map(path => rm(path, { recursive: true, force: true }))); }); +async function fixture() { + const root = await mkdtemp(join(tmpdir(), "declaration-checks-")); directories.push(root); + const path = join(root, "files"); await mkdir(path); + for (const name of ["expected.csv", "actual.csv"]) await writeFile(join(path, name), "id,cents,digest\nPRIVATE_ID,100,PRIVATE_EXPECTED_VALUE\n"); + await writeFile(join(path, "summary.json"), '{"rows":1,"cents":100}'); + const contract = { schema: "skillsync.artifacts/v1", limits: { max_files: 3, max_file_bytes: 4096, max_total_bytes: 8192 }, files: [ + ...["expected.csv", "actual.csv"].map(path => ({ path, format: "csv", max_rows: 10, unique_by: ["id"], columns: [{ name: "id", type: "string", one_of: ["PRIVATE_ID"] }, { name: "cents", type: "integer" }, { name: "digest", type: "string", equals: "PRIVATE_EXPECTED_VALUE" }] })), + { path: "summary.json", format: "json", fields: [{ name: "rows", type: "integer" }, { name: "cents", type: "integer" }] }, + ], checks: [ + { type: "csv_summary", csv: "actual.csv", json: "summary.json", row_count_field: "rows", sums: [{ column: "cents", field: "cents" }] }, + { type: "reference_exists", source: { path: "actual.csv", field: "id" }, target: { path: "expected.csv", field: "id" }, matches: [{ source: "digest", target: "digest" }] }, + { type: "keyed_integer_sum_equals", source: { path: "expected.csv", key: "id", value: "cents" }, target: { path: "actual.csv", key: "id", value: "cents" } }, + ] }; + const file = join(root, "contract.json"); await writeFile(file, JSON.stringify(contract)); + return { path, contract: file, declaration: contract }; +} + +describe("declared cross-file review", () => { + it("retains all three validated check kinds and limits without copying expected values or raw cells", async () => { + const input = await fixture(); const report = await runArtifacts(input); + expect(report.status).toBe("passed"); + expect(report.declarations?.checks.map(item => item.check.type)).toEqual(["csv_summary", "reference_exists", "keyed_integer_sum_equals"]); + expect(report.declarations?.checks.map(item => item.ordinal)).toEqual([1, 2, 3]); + expect(report.declarations?.limits.max_files).toBe(3); + expect(parseArtifactReport(report).declarations).toEqual(report.declarations); + for (const format of ["text", "json", "html"]) { + const rendered = renderArtifacts(report, format); + expect(rendered).not.toContain("PRIVATE_ID"); expect(rendered).not.toContain("PRIVATE_EXPECTED_VALUE"); + expect(rendered).toContain("csv_summary"); expect(rendered).toContain("reference_exists"); expect(rendered).toContain("keyed_integer_sum_equals"); + } + const legacy = { ...report }; delete legacy.declarations; + expect(parseArtifactReport(legacy).declarations).toBeUndefined(); + expect(() => parseArtifactReport({ ...report, declarations: { ...report.declarations, checks: [{ ...report.declarations!.checks[0], ordinal: 2 }] } })).toThrow(); + expect(() => parseArtifactReport({ ...report, declarations: { ...report.declarations, raw: "PRIVATE" } })).toThrow(); + }); + it("keeps declarations distinct from missing files and invalid setup", async () => { + const input = await fixture(); await rm(join(input.path, "actual.csv")); + const missing = await runArtifacts(input); + expect(missing.status).toBe("failed"); expect(missing.declarations?.checks).toHaveLength(3); + expect(renderArtifacts(missing, "html")).toContain("These are declarations, not proof of a completed check"); + await writeFile(input.contract, '{"schema":"invalid"}'); + const invalid = await runArtifacts(input); + expect(invalid.exitCode).toBe(2); expect(invalid.declarations).toBeUndefined(); + }); +}); diff --git a/tests/reporters/artifacts-html.test.ts b/tests/reporters/artifacts-html.test.ts index e0a2c66..1249534 100644 --- a/tests/reporters/artifacts-html.test.ts +++ b/tests/reporters/artifacts-html.test.ts @@ -13,6 +13,17 @@ function report(): ArtifactReport { } describe("offline artifact acceptance HTML", () => { + it("escapes and bounds declaration metadata with no scripts or external destinations", () => { + const value = report(); + value.declarations = { limits: { max_files: 3, max_file_bytes: 4096, max_total_bytes: 8192 }, checks: Array.from({ length: 128 }, (_, index) => ({ ordinal: index + 1, check: { type: "csv_summary" as const, csv: "", json: "/private/tmp/PRIVATE_DECLARATION", row_count_field: "rows", sums: Array.from({ length: 128 }, () => ({ column: "&".repeat(128), field: "&".repeat(128) })) } })) }; + const html = renderArtifacts(value, "html"); + expect(html).not.toMatch(/ { const html = renderArtifacts(report(), "html"); expect(html).toMatch(/^/iu); diff --git a/tests/reporters/sarif.test.ts b/tests/reporters/sarif.test.ts index a2599b6..2012377 100644 --- a/tests/reporters/sarif.test.ts +++ b/tests/reporters/sarif.test.ts @@ -44,7 +44,7 @@ describe("renderSarif", () => { expect(sarif.version).toBe("2.1.0"); expect(sarif.runs[0]?.tool.driver.name).toBe("skillsync"); - expect(sarif.runs[0]?.tool.driver.version).toBe("0.2.1"); + expect(sarif.runs[0]?.tool.driver.version).toBe("0.3.0"); expect(sarif.runs[0]?.tool.driver.rules[0]?.id).toBe("structure.missing-reference"); expect(sarif.runs[0]?.tool.driver.rules[0]?.help?.text).toBe("Add the referenced file."); expect(sarif.runs[0]?.results[0]?.ruleId).toBe("structure.missing-reference");