Repository navigation
docs/spec: close Go verifier findings on Anchor Format §1 escaping rules, verifier-compatibility README, and Acta crosswalk (#536) - #537
Conversation
…verifier-compatibility README, and Acta crosswalk (agentrust-io#536) - State the five language-independent canonical JSON rules in spec/registry-anchor-v1.md §1 (recursive Unicode code-point key sorting, ',' and ':' separators without whitespace, exact string and surrogate-pair escaping rules including \u007f for U+007F, integer 0 without minus sign, and no trailing newline). - Update examples/verifier-compatibility/README.md to reflect that Obligations 2 and 3 of agentrust-io#116 are now normative in spec/trace-v0.2.md §3.3 and documented in docs/verification.md ("Step 4: Check the EAT profile"), while Obligations 1 and 4 remain deferred. - Update the two stale v0.1 section 3.3.2 citations in docs/crosswalks/acta-decision-receipts.md to 3.3.3. - Add tests/test_registry_anchor_canonical_rules.py pinning all five rules across agentrust_trace.sign.anchor_bytes and trace_tests.inclusion.canonical_claim_bytes plus the documentation fixes. Closes agentrust-io#536 Refs agentrust-io#448 Signed-off-by: Raghuvar Durga <177662415+drk-ram@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…gentrust-io#536) The root CI workflow installs only the root agentrust-trace package and does not place conformance/src on PYTHONPATH. Load conformance/src/trace_tests/inclusion.py via importlib.util.spec_from_file_location so test_registry_anchor_canonical_rules.py collects and runs in the root CI environment. Signed-off-by: Raghuvar Durga <177662415+drk-ram@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
imran-siddique
left a comment
imran-siddique
left a comment
There was a problem hiding this comment.
Checked 92ef2fd. The new test file passes 32 of 32 here, also on a trial merge with #533, and turning ensure_ascii off in canonical_claim_bytes fails 4 of them.
One normative gap. #536 item 4 said lone surrogates are rejected. They are not: anchor_bytes({"s": "\ud800"}) and canonical_claim_bytes both return {"s":"\ud800"}, and the new bullet folds them into U+007F to U+FFFF. A Go or Rust peer cannot hold that string, so its leaf hash differs. Please put unpaired surrogates outside the profile next to non-integer numbers, and either reject them in both serializers here or open a follow-up issue for the code.
| ``` | ||
|
|
||
| Claims MUST be JSON objects. TRACE Trust Records contain only strings, integers, booleans, nulls, arrays, and objects. Claims containing non-integer numbers are outside this profile, because cross-language float serialization is not canonical, and claims containing an integer outside -9007199254740991 to 9007199254740991 are outside it for the same reason: an implementation of the four rules above in a language whose only number type is the IEEE 754 double writes one value for two distinct integers. TRACE v0.2 §3.2.2 already holds a Trust Record to that range, so a schema-valid record cannot reach this; a claim that is not a Trust Record can, and this is where it is excluded. | ||
| Claims MUST be JSON objects. TRACE Trust Records contain only strings, integers, booleans, nulls, arrays, and objects. Claims containing non-integer numbers are outside this profile, because cross-language float serialization is not canonical, and claims containing an integer outside -9007199254740991 to 9007199254740991 are outside it for the same reason: an implementation of the five rules above in a language whose only number type is the IEEE 754 double writes one value for two distinct integers. TRACE v0.2 §3.2.2 already holds a Trust Record to that range, so a schema-valid record cannot reach this; a claim that is not a Trust Record can, and this is where it is excluded. |
There was a problem hiding this comment.
Add unpaired surrogates (U+D800 to U+DFFF not part of a pair) to the out-of-profile list here.
| @pytest.mark.parametrize("serialize", SERIALIZERS) | ||
| def test_rule_4_integer_zero_is_written_without_minus_sign(serialize) -> None: | ||
| """Integer zero (-0) is serialized as 0, never -0.""" | ||
| assert serialize({"n": -0}) == b'{"n":0}' |
There was a problem hiding this comment.
-0 is the int 0 before it reaches the serializer, and json.loads("-0") is 0 too, so this cannot fail. Keep the spec bullet for implementations with a signed zero; drop or relabel the test.
…serializers (agentrust-io#536) - Exclude U+D800 to U+DFFF from the BMP escape range in spec/registry-anchor-v1.md §1 (scoping BMP \uXXXX escapes to U+007F to U+D7FF and U+E000 to U+FFFF) and state that unpaired UTF-16 surrogates in strings or object keys are outside the anchor-leaf profile. - Reject unpaired UTF-16 surrogates in agentrust_trace.sign.anchor_bytes (raising UnanchorableValue) and trace_tests.inclusion.canonical_claim_bytes (raising InclusionError). - Add test_unpaired_surrogates_in_values_and_keys_are_rejected in tests/test_registry_anchor_canonical_rules.py. Signed-off-by: Raghuvar Durga <177662415+drk-ram@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Addressed in 563a8c3:
|
dinakarjs
left a comment
dinakarjs
left a comment
There was a problem hiding this comment.
Independent technical review of 563a8c3, including the surrogate refusal update. No blocking defect found in the six-file change.
Validation in a fresh Python 3.12.14 environment installed from the repository's hashed root and conformance test locks, with both packages installed editable:
- Reference suite: 4,070 passed, 48 skipped; 97% aggregate coverage.
- Conformance suite: 778 passed, 5 xpassed.
- New canonical-rules module: 37 passed.
- Ruff passes for root src/tests and the changed conformance inclusion module; mypy passes all 15 reference source files. Diff whitespace and dash checks pass. A broader root-config Ruff scan of conformance reports 45 diagnostics, all in 16 files unchanged by this PR; that broader scan is not a clean lint result.
Independent byte controls: I wrote a separate Python encoder directly from the five stated rules, without json.dumps or rfc8785 for expected bytes. Both serializers exactly match its 12,963,355-byte output for a single string containing every one of the 1,112,064 Unicode scalar values, and match additional recursive key-order, escaped-key, safe-integer-boundary, boolean/null, slash, and signature-field cases. The alphabet-output SHA-256 is b48cf55e7ca0ce46a0ea7ee015e657c15aee5c6d8f1de05908cb8a7af82a3f6f.
All 2,048 surrogate code points were independently exercised as a value, object key, and nested value against both serializers: 12,288 expected refusals, with the documented exception types. This confirms the newest fix covers the original gap on both surfaces.
Behavioral weakening checks retained symbols and valid ASCII controls: disabling each serializer's string refusal or key refusal separately produces 5 failing refusal cases and 2 passing positive controls (four mutations); disabling ensure_ascii in the conformance serializer produces 1 failing BMP-escaping case and 3 passing controls. All temporary mutations were restored and the checkout is clean.
Scope: this is source/runtime evidence from a separately written Python oracle, not an independent-language interoperability result or hardware acceptance claim. The integer-zero case is a positive serialization control; Python integer -0 is already 0 before serialization. This COMMENT records technical findings and does not replace maintainer approval or the project's normative-change process.
This branch has not been deployed
3 participants
What this changes
Closes the three remaining verified findings from the independent Go verifier report (#448 items 3, 16, and 19; tracked in #536):
spec/registry-anchor-v1.md§1,Canonical claim bytes,src/agentrust_trace/sign.py, andconformance/src/trace_tests/inclusion.py): Replaces the prose summary that deferred byte-level string/number escaping details to Python'sjson.dumpswith the five explicit, language-independent canonical JSON rules, and excludes unpaired UTF-16 surrogates (U+D800toU+DFFF) in strings and object keys from the anchor-leaf profile:","and":", no whitespace."(U+0022),\(U+005C), and the five control charactersU+0008,U+0009,U+000A,U+000C, andU+000Duse the two-character escapes\",\\,\b,\t,\n,\f, and\r;/(U+002F) is not escaped; all other control characters (U+0000toU+001F) and all BMP characters inU+007FtoU+D7FFandU+E000toU+FFFF(includingU+007F, which is escaped as\u007f) use lowercase four-hex-digit\uXXXXescapes; supplementary-plane characters (U+10000toU+10FFFF) use lowercase UTF-16 surrogate pairs (a high surrogate inU+D800toU+DBFFfollowed by a low surrogate inU+DC00toU+DFFF, e.g.\ud8xx\udcxx); output is encoded as ASCII bytes.0(never-0).U+D800toU+DFFF) in strings or object keys are placed outside the profile and rejected by bothagentrust_trace.sign.anchor_bytes(UnanchorableValue) andtrace_tests.inclusion.canonical_claim_bytes(InclusionError).examples/verifier-compatibility/README.md): Updates the opening note, introductory paragraphs, andfailurediscussion so they accurately state that Obligations 2 and 3 of verification.md: format-versioning and verifier-compatibility obligations (evidence outlives verifiers) #116 (nonemptyaccepted_profilesdeclaration and reporting bothprofileandaccepted_profileson the verification result) are now normative inspec/trace-v0.2.md§3.3 and documented indocs/verification.md("Step 4: Check the EAT profile"), while Obligations 1 (emitter version declarations) and 4 (downgrade disclosure) remain deferred.docs/crosswalks/acta-decision-receipts.md): Updates the two stale v0.1 section3.3.2references (lines 13 and 60) to3.3.3to matchspec/trace-v0.2.md§3.3.3.tests/test_registry_anchor_canonical_rules.py): Pins all five serialization rules and unpaired-surrogate rejection across bothagentrust_trace.sign.anchor_bytesandtrace_tests.inclusion.canonical_claim_bytes(including key and value escaping) as well as the documentation fixes inspec/registry-anchor-v1.md,examples/verifier-compatibility/README.md, anddocs/crosswalks/acta-decision-receipts.md.Cold-reader review:
\ud8xx\udcxxfor supplementary-plane surrogate pairs could be read literally as restricting high surrogates toU+D800–U+D8FFand low surrogates toU+DC00–U+DCFF, excluding code points such asU+10FFFF(\udbff\udfff). Updatedspec/registry-anchor-v1.md§1 andtests/test_registry_anchor_canonical_rules.pyto specify the full surrogate ranges (a high surrogate in U+D800 to U+DBFF followed by a low surrogate in U+DC00 to U+DFFF, e.g. \ud8xx\udcxx).What failure is and is notsection ofexamples/verifier-compatibility/README.mdstill referred to "The draft normative text for verification.md: format-versioning and verifier-compatibility obligations (evidence outlives verifiers) #116, held for a maintainer to carry". Updated that paragraph to reference verification.md: format-versioning and verifier-compatibility obligations (evidence outlives verifiers) #116 andspec/trace-v0.2.md§3.3 consistently with the updated opening section.tests/test_registry_anchor_canonical_rules.pyexercised string values but not object member names for control/short/hex escapes. Addedtest_rule_3_escaping_applies_to_object_keys_as_well_as_values.@imran-siddiquepointed out thatU+007FtoU+FFFFincluded unpaired UTF-16 surrogates (U+D800toU+DFFF) and thatanchor_bytesandcanonical_claim_bytesaccepted and escaped them (\ud800), whereas UTF-8 peers cannot represent unpaired surrogates. Updatedspec/registry-anchor-v1.md§1 to scope BMP escapes toU+007FtoU+D7FFandU+E000toU+FFFF, put unpaired UTF-16 surrogates (U+D800toU+DFFF) outside the profile, and rejected them in bothanchor_bytesandcanonical_claim_byteswithtest_unpaired_surrogates_in_values_and_keys_are_rejected.Closes #536
Refs #448
Contributor: Raghuvar Durga (@drk-ram)
Type of change
Spec section
Companion specification
spec/registry-anchor-v1.md§0 and §1 (Canonical claim bytes),examples/verifier-compatibility/README.md(referencingspec/trace-v0.2.md§3.3), anddocs/crosswalks/acta-decision-receipts.md(referencingspec/trace-v0.2.md§3.3.3).Tests for a fix
tests/test_registry_anchor_canonical_rules.py::test_unpaired_surrogates_in_values_and_keys_are_rejectedfails withDID NOT RAISE UnanchorableValue(foranchor_bytes) andDID NOT RAISE InclusionError(forcanonical_claim_bytes) when the unpaired-surrogate rejection insrc/agentrust_trace/sign.pyandconformance/src/trace_tests/inclusion.pyis switched off (python tools/review_bar.py --base origin/main --head HEAD: 7 fail on behaviour, 0 fail on a missing name or file).Validation commands run and results:
pytest tests/test_registry_anchor_canonical_rules.py tests/test_requirement_keywords_are_attributable.py tests/test_safe_integer_range.py tests/test_public_functions_raise_what_they_document.py tests/test_untrusted_input_stays_on_the_documented_error.py -q— passed (348 passed)pytest conformance/tests -q— passed (778 passed, 5 xpassed)python tools/review_bar.py --base origin/main --head HEAD— passed (7 fail on behaviour, 0 fail on a missing name or file)ruff check tests/test_registry_anchor_canonical_rules.py src/agentrust_trace/sign.py conformance/src/trace_tests/inclusion.py— passedpython tools/check_dashes.py— passed (no dashes outside examples/, spec/trace-v0.1.md)mypy src/agentrust_trace— passed (Success: no issues found in 15 source files)Checklist
git commit -s)CHANGELOG.mdupdated (for any normative change)<!-- CHANGED: #NNN: description -->in spec text