Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,24 @@ Format: [Semantic Versioning](https://semver.org/). Spec versions follow `MAJOR.
Keep the first execution outcome unknown after a reported successful retry.
Uses an example-local format with a v0.2 record, not a v0.3 implementation.

- **`docs/schema.md` names both digest algorithms wherever the schema admits
both.** All thirteen digest fields in `schema/trace-claim.json` carry the same
pattern shape and take `sha256:` or `sha384:`. Seven of the thirteen rows on
the schema reference page described their field more narrowly than that:
`model.weights_digest`, `policy.bundle_hash`, `tool_transcript.hash` and
`build_provenance.digest` named `sha256:` alone, and the two
`appraisal.re_execution` digests and
`appraisal.platform_measurement.measurement` named no algorithm at all, while
the other six named both: four as `sha256:` or `sha384:`, one as
`sha256:`/`sha384:`, and `runtime.measurement` by naming the platform each
belongs to. A producer reading one of the narrower rows writes the narrower
record, and a verifier author reading one may refuse a record the schema
accepts. `tests/test_schema_doc_coverage.py` now fails when a row does not
name every algorithm its field's pattern admits. It asks the pattern with a
probe value rather than searching its text, because the schema factors the
prefix out as `^sha(256:[0-9a-f]{64}|384:[0-9a-f]{96})$` and so contains
neither name. Editorial: no schema, wire-format or normative change.

## [0.11.0] - 2026-09-25

- **Breaking:** `TraceSandboxAdapter` and `TraceAGTAdapter` require `enforcement_mode`;
Expand Down
14 changes: 7 additions & 7 deletions docs/schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ Which AI model the run used. These fields tie the record to one exact model arti
| `provider` | string | **yes** | Model provider (e.g., `example-provider`) |
| `model_id` | string | **yes** | Model identifier (e.g., `example-model-1`) |
| `version` | string | **yes** | Model version or date stamp |
| `weights_digest` | string | no | SHA-256 digest of model weights artifact |
| `weights_digest` | string | no | `sha256:` or `sha384:` digest of model weights artifact |
| `aibom_uri` | string | no | URI to the AI Bill of Materials (SPDX/CycloneDX) |

<a id="runtime"></a>
Expand All @@ -80,7 +80,7 @@ Which rule set (the governance policy) was in force during the run, pinned by it

| Field | Type | Required | Description |
|---|---|---|---|
| `bundle_hash` | string | **yes** | `sha256:` digest of the Cedar policy bundle bytes |
| `bundle_hash` | string | **yes** | `sha256:` or `sha384:` digest of the Cedar policy bundle bytes |
| `enforcement_mode` | string | **yes** | One of: `enforce` (evaluated, blocked on deny), `advisory` (evaluated, logged, allowed), `silent` (evaluated and allowed, including on deny, with operational logs suppressed; the audit chain still records every would-have-denied decision), `declared` (the policy is named and bound into the signed record and nothing evaluated it: the honest value for a producer with no policy engine, never a default, and not evidence that any rule was checked). Defaults to `enforce`. Section 4.3 of the spec defines `enforce`, `silent` and `declared`; `advisory` is in the schema's closed set and its one-line meaning is the schema's own description, not spec text |
| `version` | string | no | Policy bundle version string |
| `policy_uri` | string | no | URI to the policy bundle for inspection |
Expand All @@ -103,7 +103,7 @@ A summary of the tool calls the agent made during the run, with a hash that comm

| Field | Type | Required | Description |
|---|---|---|---|
| `hash` | string | **yes** | `sha256:` of the canonical JSON of the full `AuditEntry` list |
| `hash` | string | **yes** | `sha256:` or `sha384:` digest of the canonical JSON of the full `AuditEntry` list |
| `call_count` | integer | **yes** | Number of tool invocations recorded |
| `transcript_uri` | string | no | URI to the full per-call transcript (may be encrypted) |

Expand Down Expand Up @@ -195,7 +195,7 @@ Where the deployed software came from: which build system produced it and the di
|---|---|---|---|
| `slsa_level` | integer | **yes** | SLSA provenance level (0 to 3) |
| `builder` | string | no | Builder identity URI (e.g., GitHub Actions SLSA generator) |
| `digest` | string | **yes** | `sha256:` digest of the built artifact |
| `digest` | string | **yes** | `sha256:` or `sha384:` digest of the built artifact |
| `provenance_uri` | string | no | URI to the SLSA provenance document (e.g., Rekor entry) |
| `provenance_depth` | string | no | Depth the issuer claims: `surface`, `builder` or `transitive`. Absent is read as `surface` |

Expand Down Expand Up @@ -223,17 +223,17 @@ The result of re-running a `reproducibility` claim, attributed to the party name
| Field | Type | Required | Description |
|---|---|---|---|
| `outcome` | string | **yes** | `reproduced`: the re-run completed on the closure alone and its transcript digest equals `transcript_digest`. `diverged`: it completed on the closure alone and the digests differ. `not-attempted`: a closure blob could not be resolved, `code_identity` could not be obtained, the function read beyond the closure, the re-run did not complete, or the verifier could not establish that it used the closure alone |
| `observed_digest` | string | when `outcome` is `diverged` | The verifier's digest of the transcript its re-run produced. Divergence localises nothing by itself, so the two transcripts have to be comparable by a third party |
| `observed_digest` | string | when `outcome` is `diverged` | The verifier's `sha256:` or `sha384:` digest of the transcript its re-run produced. Divergence localises nothing by itself, so the two transcripts have to be comparable by a third party |
| `reason` | string | when `outcome` is `not-attempted` | Why no outcome could be reported. A named absence and a generic one are different findings |
| `verifier_code_identity` | string | no | Digest of the verifier's own implementation. Self-asserted and of no weight singly; a correlation key across results, since two verifiers at different implementations disagreeing over one closure is verifier drift |
| `verifier_code_identity` | string | no | `sha256:` or `sha384:` digest of the verifier's own implementation. Self-asserted and of no weight singly; a correlation key across results, since two verifiers at different implementations disagreeing over one closure is verifier drift |

### `appraisal.platform_measurement` members {#trace-field-appraisal-platform-measurement}

What a matching `runtime.measurement` covers, layer by layer, attributed to the party named as `verifier`. A composite that matches its reference does not say which layers recorded anything, which were appraised before they ran, or whether the evidence describes one boot; this block says it. `status` is untouched by it, except that a verifier does not report `affirming` while a layer its policy requires is not established or not listed, and a record carrying this block does not report `none`. Spec section 3.1.5.

| Field | Type | Required | Description |
|---|---|---|---|
| `measurement` | string | **yes** | The digest this appraisal is about. Must equal `runtime.measurement`: a result about another measurement is not about this record |
| `measurement` | string | **yes** | `sha256:` or `sha384:` digest this appraisal is about. Must equal `runtime.measurement`: a result about another measurement is not about this record |
| `layers` | object | **yes** | At least one member, keyed by the platform's name for the layer. On `tpm2` the key is `pcr:` and a register number from 0 to 23 in decimal without leading zeros (`pcr:0`, `pcr:23`). Each member carries `outcome` and, when not established, `reason`. A layer not listed is not established, and a record without this block establishes no layer |

Each member of `layers`:
Expand Down
153 changes: 150 additions & 3 deletions tests/test_schema_doc_coverage.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,17 @@
from its own row while it survives in another. Each mutation reds the guard and
names exactly the member it removed.

One thing a description says *is* decided here, and the boundary is worth stating
exactly. Where a field's ``pattern`` admits more than one digest algorithm, the row
has to name each one. That is not a judgement about whether prose agrees with the
spec; it is a comparison against the field's own machine-readable constraint, which
is why it can be checked at all and why the failure names the field and the missing
algorithm rather than asking a reader to adjudicate wording.

What this file does not decide, and is not scoped to: whether the description
attached to a field agrees with the spec, or whether an example is correct.
Neither is mechanically decidable, and a guard that reaches past what it can
decide produces failures nobody can act on. The wider corpus under ``docs/`` is
attached to a field agrees with the spec in any wider sense, or whether an example
is correct. Neither is mechanically decidable, and a guard that reaches past what
it can decide produces failures nobody can act on. The wider corpus under ``docs/`` is
not consulted: a mention on a tutorial page is not documentation of a field, and
a corpus-wide token check is a strict subset of this page-level one, so it could
never be the only failure.
Expand Down Expand Up @@ -171,6 +178,86 @@ def undocumented(schema: dict[str, Any], page: str) -> tuple[list[Field], list[V
return missing_fields, missing_values


#: Every digest algorithm the project spells, with the hex length that follows it.
#: A probe value is built from each and offered to the field's own `pattern`, so
#: "admits sha384" is answered by the pattern rather than by reading it: the
#: alternation `^sha(256:[0-9a-f]{64}|384:[0-9a-f]{96})$` factors the prefix out, so it
#: contains neither `sha256` nor `sha384` as text.
_DIGEST_ALGORITHMS: tuple[tuple[str, int], ...] = (
("sha256", 64),
("sha384", 96),
("sha512", 128),
)


def _subschema(schema: dict[str, Any], path: str, name: str) -> dict[str, Any]:
"""The subschema for a field, addressed the way `SECTION_OF_PATH` addresses it.

This follows `properties` and `items` only, while `_walk` also descends `if`,
`then` and `allOf`. A member declared solely inside a conditional branch is
therefore reachable by `_walk` and not by this, and raises `KeyError` here rather
than being passed over, so the day one appears this fails loudly instead of
exempting the field from the algorithm check without saying so.
"""
node: Any = schema
for part in filter(None, path.split(".")):
array = part.endswith("[]")
node = node["properties"][part[:-2] if array else part]
if array:
node = node["items"]
member: dict[str, Any] = node["properties"][name]
return member


def admitted_algorithms(subschema: dict[str, Any]) -> set[str]:
"""The digest algorithms a field's `pattern` actually accepts.

`re.search` rather than `fullmatch`, because an unanchored JSON Schema `pattern`
matches anywhere and these are anchored in the schema itself.
"""
pattern = subschema.get("pattern")
if not isinstance(pattern, str) or "sha" not in pattern:
return set()
matcher = re.compile(pattern)
return {
label for label, width in _DIGEST_ALGORITHMS if matcher.search(f"{label}:{'a' * width}")
}


def undocumented_algorithms(schema: dict[str, Any], page: str) -> list[tuple[Field, set[str]]]:
"""Fields whose `pattern` admits more than one digest algorithm, paired with the
ones their row does not name.

A row that documents a field into existence can still misdescribe it, and the
coverage guard above cannot see that: it asks whether a row exists, never what the
row claims. A producer reading `sha256:` where the schema also takes `sha384:`
writes the narrower record, and a verifier author reading it may refuse a valid one.

The field set comes from `schema_fields_and_values`, not from a second enumeration,
so a field this file cannot reach is equally invisible to both guards rather than to
one of them. Resolving each field's subschema is a separate lookup, `_subschema`,
which reaches less of the schema than that walk does and says so.
"""
fields, _ = schema_fields_and_values(schema)
sections = _sections(page)
tables = {path: _rows(sections.get(heading, "")) for path, heading in SECTION_OF_PATH.items()}

gaps: list[tuple[Field, set[str]]] = []
for path, name in sorted(fields):
admitted = admitted_algorithms(_subschema(schema, path, name))
if len(admitted) < 2:
continue
description = tables.get(path, {}).get(name, "")
spelled = {
label
for label in admitted
if re.search(label.replace("sha", "sha-?"), description, re.IGNORECASE)
}
if spelled != admitted:
gaps.append(((path, name), admitted - spelled))
return gaps


@pytest.fixture(scope="module")
def schema() -> dict[str, Any]:
loaded: dict[str, Any] = json.loads(CANONICAL_SCHEMA_PATH.read_text(encoding="utf-8"))
Expand Down Expand Up @@ -234,6 +321,23 @@ def test_every_schema_field_and_value_is_documented_in_its_place(
assert not missing_fields and not missing_values, _report(missing_fields, missing_values)


def test_every_digest_field_names_each_algorithm_its_pattern_admits(
schema: dict[str, Any], page: str
) -> None:
"""A row that names one algorithm where the pattern takes two sends a producer to
the narrower record and a verifier author to the narrower rule."""
gaps = undocumented_algorithms(schema, page)
assert not gaps, "\n".join(
["docs/schema.md describes these fields more narrowly than the schema accepts:"]
+ [
f" {path + '.' if path else ''}{name}: the row does not name "
f"{', '.join(sorted(unnamed))}, which the pattern admits"
for (path, name), unnamed in gaps
]
+ ["Name every algorithm the pattern takes, as the other digest rows do."]
)


# --------------------------------------------------------------------- it can go red


Expand Down Expand Up @@ -413,3 +517,46 @@ def test_the_guard_fails_when_a_value_is_removed_from_its_fields_row(
missing_fields, missing_values = undocumented(schema, stripped)
assert missing_fields == []
assert missing_values == [expected]


def test_admitted_algorithms_reads_the_alternation_the_schema_actually_uses() -> None:
"""The schema factors the prefix out: `^sha(256:...|384:...)$` contains neither the
literal `sha256` nor `sha384`. A check that searched the pattern text for either
would find no algorithm at all, conclude the field constrains none, and skip it, so
the probe offers the pattern a value and believes the answer. The reading version of
this check was written first and reported no gaps anywhere, which is what this pins."""
both = r"^sha(256:[0-9a-f]{64}|384:[0-9a-f]{96})$"
assert admitted_algorithms({"pattern": both}) == {"sha256", "sha384"}
assert admitted_algorithms({"pattern": "^sha256:[0-9a-f]{64}$"}) == {"sha256"}
assert admitted_algorithms({"type": "string"}) == set()


def test_the_guard_fails_when_a_row_stops_naming_an_algorithm_its_pattern_admits(
schema: dict[str, Any], page: str
) -> None:
"""Narrowing a corrected row back to its previous wording reports exactly that
field, and only it."""
sections = _sections(page)
row = next(
line for line in sections["## `policy`"].splitlines() if line.startswith("| `bundle_hash` ")
)
assert page.count(row) == 1
narrowed = row.replace("`sha256:` or `sha384:`", "`sha256:`")
assert narrowed != row, "the row no longer carries the form this test narrows"
gaps = undocumented_algorithms(schema, page.replace(row, narrowed))
assert gaps == [(("policy", "bundle_hash"), {"sha384"})]


def test_the_guard_ignores_a_field_whose_pattern_admits_one_algorithm(
schema: dict[str, Any], page: str
) -> None:
"""The rule is that a row names what its own pattern takes, not that every digest row
lists every algorithm the project spells. A field narrowed to one is not a gap even
though its row still names two."""
narrowed = copy.deepcopy(schema)
narrowed["properties"]["policy"]["properties"]["bundle_hash"]["pattern"] = (
"^sha256:[0-9a-f]{64}$"
)
assert admitted_algorithms(_subschema(narrowed, "policy", "bundle_hash")) == {"sha256"}
gaps = undocumented_algorithms(narrowed, page)
assert not [gap for gap in gaps if gap[0][1] == "bundle_hash"]
Loading