Repository navigation
Fix auditor append-only bypass in batch node insertion - #495
Merged
Merged
Conversation
kevinlewi
marked this pull request as draft
August 12, 2026 00:36
The auditor verifies an epoch transition by inserting a proof's `unchanged_nodes`, then its `unchanged_nodes + inserted` nodes, checking each against the start/end root hash. The append-only guarantee relies on re-inserting an unchanged node being a no-op. However, `AzksElementSet::partition` silently drops any node whose label is not strictly beneath the pivot, so an unchanged interior node whose label is a strict prefix of an inserted leaf has its committed value discarded. A malicious server could exploit this to rewrite an existing label's value while still producing a valid append-only proof, defeating the auditor's append-only guarantee. Enforce that no committed value is dropped during an `InsertMode::Auditor` batch insert, turning the silent drop into a verification error. The check is count-based (every input element must land in the left or right subtree) so it catches a drop regardless of label shape, with a narrow carve-out for the single element that legitimately becomes a leaf. Scoped to `InsertMode::Auditor`; the directory publish path inserts full-length leaves only and is unaffected. Adds regression tests that reproduce the bypass end-to-end through `audit_verify` and directly at the batch-insert guard (strict-prefix and duplicate-label collisions), plus a positive control. Reported by Sanjit Bhat.
kevinlewi
force-pushed
the
auditor-append-only-fix
branch
from
August 12, 2026 00:43
4631968 to
685e2a7
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #495 +/- ##
==========================================
+ Coverage 88.61% 89.44% +0.82%
==========================================
Files 39 38 -1
Lines 9109 7579 -1530
==========================================
- Hits 8072 6779 -1293
+ Misses 1037 800 -237 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
|
The updated logic makes sense to me, thanks for fixing this! |
kevinlewi
marked this pull request as ready for review
August 12, 2026 16:28
artemkorneev
approved these changes
Aug 12, 2026
This was referenced Aug 23, 2026
Closed
skyl
added a commit
to free2z/zuu
that referenced
this pull request
Aug 24, 2026
#557) Records the outcome of spike #544 and specifies the key-transparency protocol. Docs only — no crates, no manifests, no workflows. ADR 0013 — adopt `akd` (Meta's Auditable Key Directory, the SEEMless/Parakeet-lineage zero-knowledge set behind WhatsApp Key Transparency) rather than hand-rolling the Merkle machinery. Records the hard floor `>= 0.13.0` and, more importantly, *why it cannot be enforced by advisory tooling*: meta/akd#495 fixed an auditor append-only bypass letting a malicious server rewrite a label's value while still producing a valid append-only proof, and no RustSec/OSV advisory exists for it — an OSV query returns `{}` — so `cargo audit` passes on a stale 0.12.0 pin silently and forever. The floor is a `[[bans.deny]]` entry in `rs/deny.toml`, landing with the code. Also records the NCC Group audit (Aug 2023, 3 consultants / 20 person-days, public 40-page report, scope v0.9.0, 1 Medium / 8 Low / 6 Info, all fixed into v0.11.0), what `akd` is not (the ZKS only — no STHs, no cosigning, no gossip, in-memory `Database`), the general soundness-asymmetry rule, and the rejected alternatives. ARCHITECTURE §9.2 — the consistency-proof bullet is false as written. `akd`'s `AppendOnlyProof` is O(entries added), not O(log n): 3.9 MB and 1–3 s for five epochs. Corrected with a dated correction block in the style §5.4 and ADR 0007 established, leaving the wrong sentence visible rather than rewriting it, plus a table of what each party can actually verify and the consequence the merged text under-stated: a client cannot substitute its own consistency check for a witness's, which makes the witness set more load-bearing, not less. THREAT-MODEL §3.1 gains the matching correction (and the ADR 0014 reset annotation that ADR asked for), §3.9 gains the lazy-witness weakness, §5's summary row is qualified. KT.md — the wire-level KT specification, WIRE.md's analogue for the directory: `DirectoryEntry` and its authorization under ADR 0014, `SignedTreeHead` with a domain-separated transcript, monotonicity and log-key rotation (a successor key is accepted only if the announcement was itself cosigned, otherwise a key change is indistinguishable from a fork), epoch cadence plus a published maximum merge delay and a submission receipt so an unpublished-but-accepted entry is a detectable fault with evidence, the witness poll → verify → refuse-and-record → cosign → atomic-state-update protocol, `WitnessCosignature` signing contents rather than the log's signature, the proof-serving API, where `akd`'s types sit underneath, and the threshold rule with fail-closed behaviour per the owner decision on #311. Invented material is marked as invented; open questions are listed (§13-P/Q/R added) rather than answered. Also records the upstreamable wart: `akd::auditor::audit_verify` traps under wasm because `verify_append_only_hash` hardcodes `AzksParallelismConfig::default()`, reaching `tokio::task::spawn`. It is in the server crate and the auditor is the native witness role, so the client verifier reaches the browser cleanly at ~40 KB brotli. Closes #554 Claude-Session: https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku Co-authored-by: Claude <noreply@anthropic.com>
skyl
added a commit
to free2z/zuu
that referenced
this pull request
Aug 24, 2026
…#571) rs/deny.toml gains ADR 0013's three [[bans.deny]] entries verbatim — akd and akd_core floored at 0.13.0, akd_mysql banned outright — with the comment naming meta/akd#495 and the reason no advisory will ever back it up. Mutation-tested with a scratch member depending on akd = "0.12.0": advisories green, bans red; at akd = "0.13" both green. That asymmetry is the whole point. Also: - rs/README.md no longer claims all four gate scripts discover crates. Three do; check-rust-toolchain.sh reads a hardcoded list plus --manifest flags, so a second crate under rs/crates/ passes with a wrong MSRV or none. Proved with a throwaway crate and tabled, pointed at #553. - rs/README.md argues the #341 workspace question from the accepted ADR (docs/architecture/CARGO-WORKSPACE.md) rather than the closed issue, and drops the "if #341 later chooses one repo-wide workspace" hedge the ADR rules out. - rs/deny.toml's [graph] comment now sits on `targets = []`, the knob that actually decides target coverage, instead of on the feature knob below it. - wallet/zuuli/crypto-target-spike/Cargo.toml is registered in check-rust-toolchain.sh's MANIFESTS; its rust-version was unchecked. Closes #565 Closes #566 Claude-Session: https://claude.ai/code/session_01NMwYnLDGotB9Ph4NgDFNeD
skyl
pushed a commit
to free2z/zuu
that referenced
this pull request
Aug 24, 2026
The one crate the log server, the witness and the client all link (KT.md §11.4). A witness that verified with a different implementation than the log builds with would produce cosignatures that mean nothing, and §7.4's only structural defence against a lazy witness is that there are not two implementations to disagree. ADR 0013 buys the append-only zero-knowledge set from `akd` and nothing else. Everything between that library and a deployment is here: - `DirectoryEntry` / `DirectoryEntryTBS` and §4.4's entry authorization — `same_key` / `key_change` / `platform_reset`, `RotationProof`, `ResetAuthorization`. A key change carrying only one of the two required signatures cannot even be represented, let alone accepted. - `SignedTreeHead`, the §6.2 signing transcript with its closed domain-separation label set, §6.3's eight monotonicity rules, and log-key rotation gated on a cosigned announcement (§6.4). - `WitnessCosignature` over the CONTENTS — log id, epoch, tree size, root hash — never over the log's signature, plus the two-cosignature contradiction test. - Client verification (§8.1, §8.2) wrapping `akd_core`'s lookup and key-history verifiers, and the witness-side append-only path (§7.1 step 4) wrapping `akd::auditor`. - The threshold rule (§8.3): >= t cosignatures from a CALLER-SUPPLIED witness set, failing closed with no partial result and no override. - `SubmissionReceipt`, `LogDescriptor`, `FaultReport`, §9.5's error codes. The single most important thing: **akd enforces none of our authorization rules.** It will commit any bytes to any label, and a log that skips §4.4 produces proofs that verify perfectly for entries nobody authorized. So the crate is built out of choke points, not helpers. `validate_submission` is the only entry point and the only producer of `AcceptedSubmission`, which is the only carrier of the AkdLabel/AkdValue pair `publish()` needs. `AcceptedRoot` (threshold), `AppendOnlyVerified` (auditor) and `LogView` (monotonicity) work the same way: none has a public constructor, none has a bypass flag, and the next step's argument is the previous step's proof. Four rules are enforced that §4.4's numbered list does not state but ADR 0014's case analysis requires; each is marked ADDITIONAL RULE at its call site in `submit.rs`, and the first is a real gap in KT.md: 1. A `same_key` entry MUST NOT change `identity_pk`. `same_key` is authorized by the PREVIOUS `directory_auth_pk` alone, so without this a party holding only that key installs a new identity key — a key change with one signature, which ADR 0014 says the log MUST reject. 2. A `key_change` or `platform_reset` MUST change `identity_pk`. 3. A `platform_reset` MUST be refused when the previous entry set `no_reset`. 4. A `ResetAuthorization` is bound to its handle, log, version and outgoing key, as §4.4 rule 6 already binds a `RotationProof`. `rs/deny.toml` gets ADR 0013's `[[bans.deny]]` stanza for akd/akd_core < 0.13.0 and akd_mysql. meta/akd#495 has no RustSec or OSV advisory and never will, so the floor cannot live in a manifest or in `cargo audit`. Verified firing: a 0.12.0 downgrade is rejected by `check bans` while `check advisories` reports "advisories ok". The `verifier` feature reaches wasm32-unknown-unknown and is added to the rs_wasm CI job as `--no-default-features --features verifier`; `auditor` pulls the server crate and must never reach a browser (KT.md §11.3). Claude-Session: https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku
skyl
added a commit
to free2z/zuu
that referenced
this pull request
Aug 24, 2026
…1) (#629) * feat(kt): add f2z-kt-core, the key-transparency crate (KT.md v1) The one crate the log server, the witness and the client all link (KT.md §11.4). A witness that verified with a different implementation than the log builds with would produce cosignatures that mean nothing, and §7.4's only structural defence against a lazy witness is that there are not two implementations to disagree. ADR 0013 buys the append-only zero-knowledge set from `akd` and nothing else. Everything between that library and a deployment is here: - `DirectoryEntry` / `DirectoryEntryTBS` and §4.4's entry authorization — `same_key` / `key_change` / `platform_reset`, `RotationProof`, `ResetAuthorization`. A key change carrying only one of the two required signatures cannot even be represented, let alone accepted. - `SignedTreeHead`, the §6.2 signing transcript with its closed domain-separation label set, §6.3's eight monotonicity rules, and log-key rotation gated on a cosigned announcement (§6.4). - `WitnessCosignature` over the CONTENTS — log id, epoch, tree size, root hash — never over the log's signature, plus the two-cosignature contradiction test. - Client verification (§8.1, §8.2) wrapping `akd_core`'s lookup and key-history verifiers, and the witness-side append-only path (§7.1 step 4) wrapping `akd::auditor`. - The threshold rule (§8.3): >= t cosignatures from a CALLER-SUPPLIED witness set, failing closed with no partial result and no override. - `SubmissionReceipt`, `LogDescriptor`, `FaultReport`, §9.5's error codes. The single most important thing: **akd enforces none of our authorization rules.** It will commit any bytes to any label, and a log that skips §4.4 produces proofs that verify perfectly for entries nobody authorized. So the crate is built out of choke points, not helpers. `validate_submission` is the only entry point and the only producer of `AcceptedSubmission`, which is the only carrier of the AkdLabel/AkdValue pair `publish()` needs. `AcceptedRoot` (threshold), `AppendOnlyVerified` (auditor) and `LogView` (monotonicity) work the same way: none has a public constructor, none has a bypass flag, and the next step's argument is the previous step's proof. Four rules are enforced that §4.4's numbered list does not state but ADR 0014's case analysis requires; each is marked ADDITIONAL RULE at its call site in `submit.rs`, and the first is a real gap in KT.md: 1. A `same_key` entry MUST NOT change `identity_pk`. `same_key` is authorized by the PREVIOUS `directory_auth_pk` alone, so without this a party holding only that key installs a new identity key — a key change with one signature, which ADR 0014 says the log MUST reject. 2. A `key_change` or `platform_reset` MUST change `identity_pk`. 3. A `platform_reset` MUST be refused when the previous entry set `no_reset`. 4. A `ResetAuthorization` is bound to its handle, log, version and outgoing key, as §4.4 rule 6 already binds a `RotationProof`. `rs/deny.toml` gets ADR 0013's `[[bans.deny]]` stanza for akd/akd_core < 0.13.0 and akd_mysql. meta/akd#495 has no RustSec or OSV advisory and never will, so the floor cannot live in a manifest or in `cargo audit`. Verified firing: a 0.12.0 downgrade is rejected by `check bans` while `check advisories` reports "advisories ok". The `verifier` feature reaches wasm32-unknown-unknown and is added to the rs_wasm CI job as `--no-default-features --features verifier`; `auditor` pulls the server crate and must never reach a browser (KT.md §11.3). Claude-Session: https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku * feat(kt): add f2z-authority, the handle-ownership assertion layer The directory can prove that @alice's key was published, never changed behind anyone's back, and was authorized by whoever held the handle before. It cannot prove that @alice is Alice. Something has to say who owns a handle, no cryptography derives it, and that something is a non-cryptographic trust root. This is the whole of that root, isolated so a self-hoster swapping it — or running with none — reads one short crate instead of grepping a server. - `HandleAssertion` / `HandleAssertionTBS`: the tls_codec structure an authority signs, with the label field as its domain-separated signing prefix. - `AssertionBindingTBS`: what the identity key the assertion is *about* signs back. An assertion is a bearer document; without this, a stolen copy is a handle takeover. `AuthorityConfig::admit` is the only public verification path and takes both on every branch, so there is no route to a half-checked assertion. - `AuthorityConfig::admit`: seventeen rules in order — authority membership and signature, log_id, handle_id, charset, timestamps, a validity cap the LOG holds rather than the issuer, intent against the entry sequence, account_epoch monotonicity, the identity binding, and last of all the nonce, so a failed submission never burns one. - `AuthoritySet`: multiple (kid, pubkey) entries, so rotation is a set-membership change; plus an explicit, reported, no-authority mode for a log with no user directory. - `f2z-assert`: an example issuer, so a log with no web application in front of it can issue by hand. MIT, no_std + alloc, #![forbid(unsafe_code)], and it builds for wasm32-unknown-unknown — clients verify assertions too. Registered with scripts/check-rust-toolchain.sh and rs.yml's rs_wasm job. https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku * chore(rs): reconcile Cargo.lock after merging f2z-authority into f2z-kt-core The two branches each resolved ed25519-dalek unambiguously; together the graph carries both major versions, so f2z-authority's entry needs the disambiguating version suffix. https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku * wip(kt): f2z-kt log server — choke point, storage, API, rate limits The submission path routes through f2z-kt-core's validate_submission and f2z-authority's admit; AdmittedSubmission has no public constructor and is the only thing LogService will publish. Not yet: f2z-witness, the adversarial and acceptance tests, CI registration. https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku * refactor(kt): move KT.md §9.2's envelopes into f2z-kt-core §11.4 is explicit that the log, the witness and the client link one crate. A witness that decoded a tree-head bundle differently from the log that encoded it would cosign a root it did not actually read, so these composites cannot have two implementations — and the witness needs them at runtime without linking the server. https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku * wip(kt): f2z-witness — poll, verify, refuse and record, cosign State is one file replaced atomically; the halt survives a restart; healthz is a subcommand because the distroless image has no shell and §9.3 promises no inbound listener. https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku * test(kt): adversarial submission tests and the witness acceptance suite 15 adversarial submissions, each well-formed with one thing wrong, each asserting both the KT.md §9.5 code and that the tree did not move. The witness suite drives a real f2z-kt log — §7.4's argument is that there are not two implementations, so a mock would defeat it — and stages the equivocation as two real logs sharing one signing key. https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku * chore(kt): register both binaries with the gates and cross the licence boundary - scripts/check-rust-toolchain.sh --manifest for f2z-kt and f2z-witness, in rs.yml and rs/README.md. - rs/deny.toml: AGPL-3.0-only exceptions by name for the two server binaries and nothing else; ISC and CDLA-Permissive-2.0 added for the witness's rustls stack with the reasoning at the entry. - Durability tests: a restart re-derives every root the log signed, and an edited journal refuses to start. fmt, clippy -D warnings, cargo-deny and the full test suite are green on the pinned 1.97.1 toolchain; both wasm builds still pass. https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku * docs(rs): document both binaries in rs/README.md; add f2z-kt redaction tests The redaction tests check base16 in both cases, a decimal byte list and base64url — a hex-only assertion sails straight past Debug for [u8; 32], which prints decimals. https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku * feat(kt): rework onto main's reviewed f2z-authority API #626 landed f2z-authority on main with a reviewed API: admit became check_assertion_layer, Submission gained EntryKind and previous_vouch, and the result is explicitly not an authorization-to-publish token. Every submission now routes through the assertion layer, not just a first entry — the reviewed API covers all four cases, refuses a platform assertion on a user-authorized entry as a category error, and requires the predecessor's vouch and account_epoch back on every non-initial entry. The log retains and feeds both forward, and the journal now stores the whole envelope so a restart re-runs the entire admission rather than half of it. Also satisfies two repo-wide invariants added on main that the unmerged f2z-kt-core base never saw: - workspace_debug_scan: hand-written Debug for AcceptedSubmission, VerifiedEntry and the log's Pending, which held a submission between acceptance and publication — the one window in which an entry is not public. - workspace_strict_verification_scan: all thirteen verify functions registered. Two new Strictness categories were needed and are documented as such: DelegatesToFn, because f2z-kt-core's Ed25519 entry point is a free function rather than a method on a key newtype, and NotASignatureCheck for the four that verify akd proofs or count already-verified cosignatures. Each still names the call it makes, so a row cannot go vacuous. https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku * fix(kt): make the KT label set prefix-free (zuu#602 forward-port) check-hash-domain-labels.mjs, added on main, is right about all three: - f2z-kt-core still spelled the tree-head chain-hash label `sth-hash`, which makes the SignedTreeHeadTBS signing label a proper prefix of it. H() has no separator, so the two domains were not separated at all. That is #602, already fixed in KT.md on main; this forward-ports the fix into the unmerged crate and renames LABEL_STH_HASH to LABEL_TREE_HEAD_HASH. - The log's heartbeat label was a proper prefix of its value label — the same defect in miniature, in new code, caught before it shipped. It now ends in ':' like KT.md §3.3's handle prefix. - A kt-core test asserted the composed AkdLabel as one literal, which mints a token that is not prefix-free against the prefix constant. Composed from the constant instead. https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku --------- Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes an append-only bypass in the AKD auditor.
Problem
The auditor verifies an epoch transition by inserting a proof's
unchanged_nodes, then itsunchanged_nodes + insertednodes, checking each against the start/end root hash. This relies on re-inserting an unchanged node being a no-op. ButAzksElementSet::partitionsilently drops any node whose label is not strictly beneath the pivot, so an unchanged interior node whose label is a strict prefix of an inserted leaf has its committed value discarded.A malicious server can exploit this to rewrite an existing label's value while still producing a valid append-only proof, defeating the auditor's append-only guarantee.
Fix
Enforce that no committed value is dropped during an
InsertMode::Auditorbatch insert, turning the silent drop into a verification error. The check is count-based (every input element must land in the left or right subtree), so it catches a drop regardless of label shape, with a narrow carve-out for the single element that legitimately becomes a leaf.Scoped to
InsertMode::Auditor; the directory publish path inserts full-length leaves only and is unaffected.Tests
test_auditor_rejects_prefix_collision_value_rewrite(auditor.rs) — reproduces the bypass end-to-end throughaudit_verify: both membership proofs verify against their respective roots withval1 != val2, and the transition must now be rejected.test_auditor_insert_rejects_dropped_nodes(append_only_zks.rs) — exercises the guard directly: strict-prefix collision, duplicate-label collision, plus a positive control confirming disjoint leaves still insert cleanly.Both verified to fail without the guard and pass with it, under both
whatsapp_v1andexperimentalconfigurations. Fullakdlib suite green (106 tests).Reported by @sanjit-bhat