Skip to content

Fix auditor append-only bypass in batch node insertion - #495

Merged
kevinlewi merged 1 commit into
meta:mainfrom
kevinlewi:auditor-append-only-fix
Aug 12, 2026
Merged

kevinlewi merged 1 commit into
meta:mainfrom
kevinlewi:auditor-append-only-fix

Conversation

@kevinlewi

@kevinlewi kevinlewi commented Aug 12, 2026 •

Copy link
Copy Markdown
Contributor

Fixes an append-only bypass in the AKD auditor.

Problem

The auditor verifies an epoch transition by inserting a proof's unchanged_nodes, then its unchanged_nodes + inserted nodes, checking each against the start/end root hash. This relies on re-inserting an unchanged node being a no-op. But AzksElementSet::partition silently drops any node whose label is not strictly beneath the pivot, so an unchanged interior node whose label is a strict prefix of an inserted leaf has its committed value discarded.

A malicious server can exploit this to rewrite an existing label's value while still producing a valid append-only proof, defeating the auditor's append-only guarantee.

Fix

Enforce that no committed value is dropped during an InsertMode::Auditor batch insert, turning the silent drop into a verification error. The check is count-based (every input element must land in the left or right subtree), so it catches a drop regardless of label shape, with a narrow carve-out for the single element that legitimately becomes a leaf.

Scoped to InsertMode::Auditor; the directory publish path inserts full-length leaves only and is unaffected.

Tests

  • test_auditor_rejects_prefix_collision_value_rewrite (auditor.rs) — reproduces the bypass end-to-end through audit_verify: both membership proofs verify against their respective roots with val1 != val2, and the transition must now be rejected.
  • test_auditor_insert_rejects_dropped_nodes (append_only_zks.rs) — exercises the guard directly: strict-prefix collision, duplicate-label collision, plus a positive control confirming disjoint leaves still insert cleanly.

Both verified to fail without the guard and pass with it, under both whatsapp_v1 and experimental configurations. Full akd lib suite green (106 tests).

Reported by @sanjit-bhat

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 12, 2026
@kevinlewi
kevinlewi marked this pull request as draft August 12, 2026 00:36
The auditor verifies an epoch transition by inserting a proof's
`unchanged_nodes`, then its `unchanged_nodes + inserted` nodes, checking
each against the start/end root hash. The append-only guarantee relies on
re-inserting an unchanged node being a no-op. However,
`AzksElementSet::partition` silently drops any node whose label is not
strictly beneath the pivot, so an unchanged interior node whose label is a
strict prefix of an inserted leaf has its committed value discarded. A
malicious server could exploit this to rewrite an existing label's value
while still producing a valid append-only proof, defeating the auditor's
append-only guarantee.

Enforce that no committed value is dropped during an `InsertMode::Auditor`
batch insert, turning the silent drop into a verification error. The check
is count-based (every input element must land in the left or right subtree)
so it catches a drop regardless of label shape, with a narrow carve-out for
the single element that legitimately becomes a leaf. Scoped to
`InsertMode::Auditor`; the directory publish path inserts full-length leaves
only and is unaffected.

Adds regression tests that reproduce the bypass end-to-end through
`audit_verify` and directly at the batch-insert guard (strict-prefix and
duplicate-label collisions), plus a positive control.

Reported by Sanjit Bhat.
@kevinlewi
kevinlewi force-pushed the auditor-append-only-fix branch from 4631968 to 685e2a7 Compare August 12, 2026 00:43
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.35036% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.44%. Comparing base (3ce5335) to head (685e2a7).
⚠️ Report is 38 commits behind head on main.

Files with missing lines Patch % Lines
akd/src/append_only_zks.rs 97.18% 2 Missing ⚠️
akd/src/errors.rs 0.00% 2 Missing ⚠️
akd/src/auditor.rs 98.43% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #495      +/-   ##
==========================================
+ Coverage   88.61%   89.44%   +0.82%     
==========================================
  Files          39       38       -1     
  Lines        9109     7579    -1530     
==========================================
- Hits         8072     6779    -1293     
+ Misses       1037      800     -237     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@sanjit-bhat

Copy link
Copy Markdown
Contributor

The updated logic makes sense to me, thanks for fixing this!

@kevinlewi
kevinlewi marked this pull request as ready for review August 12, 2026 16:28
@kevinlewi
kevinlewi merged commit 2e99b03 into meta:main Aug 12, 2026
14 checks passed
skyl added a commit to free2z/zuu that referenced this pull request Aug 24, 2026
#557)

Records the outcome of spike #544 and specifies the key-transparency
protocol. Docs only — no crates, no manifests, no workflows.

ADR 0013 — adopt `akd` (Meta's Auditable Key Directory, the
SEEMless/Parakeet-lineage zero-knowledge set behind WhatsApp Key
Transparency) rather than hand-rolling the Merkle machinery. Records the
hard floor `>= 0.13.0` and, more importantly, *why it cannot be enforced
by advisory tooling*: meta/akd#495 fixed an auditor append-only
bypass letting a malicious server rewrite a label's value while still
producing a valid append-only proof, and no RustSec/OSV advisory exists
for it — an OSV query returns `{}` — so `cargo audit` passes on a stale
0.12.0 pin silently and forever. The floor is a `[[bans.deny]]` entry in
`rs/deny.toml`, landing with the code. Also records the NCC Group audit
(Aug 2023, 3 consultants / 20 person-days, public 40-page report, scope
v0.9.0, 1 Medium / 8 Low / 6 Info, all fixed into v0.11.0), what `akd`
is not (the ZKS only — no STHs, no cosigning, no gossip, in-memory
`Database`), the general soundness-asymmetry rule, and the rejected
alternatives.

ARCHITECTURE §9.2 — the consistency-proof bullet is false as written.
`akd`'s `AppendOnlyProof` is O(entries added), not O(log n): 3.9 MB and
1–3 s for five epochs. Corrected with a dated correction block in the
style §5.4 and ADR 0007 established, leaving the wrong sentence visible
rather than rewriting it, plus a table of what each party can actually
verify and the consequence the merged text under-stated: a client cannot
substitute its own consistency check for a witness's, which makes the
witness set more load-bearing, not less. THREAT-MODEL §3.1 gains the
matching correction (and the ADR 0014 reset annotation that ADR asked
for), §3.9 gains the lazy-witness weakness, §5's summary row is
qualified.

KT.md — the wire-level KT specification, WIRE.md's analogue for the
directory: `DirectoryEntry` and its authorization under ADR 0014,
`SignedTreeHead` with a domain-separated transcript, monotonicity and
log-key rotation (a successor key is accepted only if the announcement
was itself cosigned, otherwise a key change is indistinguishable from a
fork), epoch cadence plus a published maximum merge delay and a
submission receipt so an unpublished-but-accepted entry is a detectable
fault with evidence, the witness poll → verify → refuse-and-record →
cosign → atomic-state-update protocol, `WitnessCosignature` signing
contents rather than the log's signature, the proof-serving API, where
`akd`'s types sit underneath, and the threshold rule with fail-closed
behaviour per the owner decision on #311. Invented material is marked as
invented; open questions are listed (§13-P/Q/R added) rather than
answered.

Also records the upstreamable wart: `akd::auditor::audit_verify` traps
under wasm because `verify_append_only_hash` hardcodes
`AzksParallelismConfig::default()`, reaching `tokio::task::spawn`. It is
in the server crate and the auditor is the native witness role, so the
client verifier reaches the browser cleanly at ~40 KB brotli.

Closes #554

Claude-Session: https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku

Co-authored-by: Claude <noreply@anthropic.com>
skyl added a commit to free2z/zuu that referenced this pull request Aug 24, 2026
…#571)

rs/deny.toml gains ADR 0013's three [[bans.deny]] entries verbatim — akd and
akd_core floored at 0.13.0, akd_mysql banned outright — with the comment naming
meta/akd#495 and the reason no advisory will ever back it up.

Mutation-tested with a scratch member depending on akd = "0.12.0": advisories
green, bans red; at akd = "0.13" both green. That asymmetry is the whole point.

Also:
- rs/README.md no longer claims all four gate scripts discover crates. Three do;
  check-rust-toolchain.sh reads a hardcoded list plus --manifest flags, so a
  second crate under rs/crates/ passes with a wrong MSRV or none. Proved with a
  throwaway crate and tabled, pointed at #553.
- rs/README.md argues the #341 workspace question from the accepted ADR
  (docs/architecture/CARGO-WORKSPACE.md) rather than the closed issue, and drops
  the "if #341 later chooses one repo-wide workspace" hedge the ADR rules out.
- rs/deny.toml's [graph] comment now sits on `targets = []`, the knob that
  actually decides target coverage, instead of on the feature knob below it.
- wallet/zuuli/crypto-target-spike/Cargo.toml is registered in
  check-rust-toolchain.sh's MANIFESTS; its rust-version was unchecked.

Closes #565
Closes #566

Claude-Session: https://claude.ai/code/session_01NMwYnLDGotB9Ph4NgDFNeD
skyl pushed a commit to free2z/zuu that referenced this pull request Aug 24, 2026
The one crate the log server, the witness and the client all link (KT.md
§11.4). A witness that verified with a different implementation than the
log builds with would produce cosignatures that mean nothing, and §7.4's
only structural defence against a lazy witness is that there are not two
implementations to disagree.

ADR 0013 buys the append-only zero-knowledge set from `akd` and nothing
else. Everything between that library and a deployment is here:

- `DirectoryEntry` / `DirectoryEntryTBS` and §4.4's entry authorization —
  `same_key` / `key_change` / `platform_reset`, `RotationProof`,
  `ResetAuthorization`. A key change carrying only one of the two
  required signatures cannot even be represented, let alone accepted.
- `SignedTreeHead`, the §6.2 signing transcript with its closed
  domain-separation label set, §6.3's eight monotonicity rules, and
  log-key rotation gated on a cosigned announcement (§6.4).
- `WitnessCosignature` over the CONTENTS — log id, epoch, tree size, root
  hash — never over the log's signature, plus the two-cosignature
  contradiction test.
- Client verification (§8.1, §8.2) wrapping `akd_core`'s lookup and
  key-history verifiers, and the witness-side append-only path (§7.1
  step 4) wrapping `akd::auditor`.
- The threshold rule (§8.3): >= t cosignatures from a CALLER-SUPPLIED
  witness set, failing closed with no partial result and no override.
- `SubmissionReceipt`, `LogDescriptor`, `FaultReport`, §9.5's error codes.

The single most important thing: **akd enforces none of our
authorization rules.** It will commit any bytes to any label, and a log
that skips §4.4 produces proofs that verify perfectly for entries nobody
authorized. So the crate is built out of choke points, not helpers.
`validate_submission` is the only entry point and the only producer of
`AcceptedSubmission`, which is the only carrier of the AkdLabel/AkdValue
pair `publish()` needs. `AcceptedRoot` (threshold), `AppendOnlyVerified`
(auditor) and `LogView` (monotonicity) work the same way: none has a
public constructor, none has a bypass flag, and the next step's argument
is the previous step's proof.

Four rules are enforced that §4.4's numbered list does not state but
ADR 0014's case analysis requires; each is marked ADDITIONAL RULE at its
call site in `submit.rs`, and the first is a real gap in KT.md:

1. A `same_key` entry MUST NOT change `identity_pk`. `same_key` is
   authorized by the PREVIOUS `directory_auth_pk` alone, so without this
   a party holding only that key installs a new identity key — a key
   change with one signature, which ADR 0014 says the log MUST reject.
2. A `key_change` or `platform_reset` MUST change `identity_pk`.
3. A `platform_reset` MUST be refused when the previous entry set
   `no_reset`.
4. A `ResetAuthorization` is bound to its handle, log, version and
   outgoing key, as §4.4 rule 6 already binds a `RotationProof`.

`rs/deny.toml` gets ADR 0013's `[[bans.deny]]` stanza for
akd/akd_core < 0.13.0 and akd_mysql. meta/akd#495 has no RustSec or
OSV advisory and never will, so the floor cannot live in a manifest or in
`cargo audit`. Verified firing: a 0.12.0 downgrade is rejected by
`check bans` while `check advisories` reports "advisories ok".

The `verifier` feature reaches wasm32-unknown-unknown and is added to the
rs_wasm CI job as `--no-default-features --features verifier`; `auditor`
pulls the server crate and must never reach a browser (KT.md §11.3).

Claude-Session: https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku
skyl added a commit to free2z/zuu that referenced this pull request Aug 24, 2026
…1) (#629)

* feat(kt): add f2z-kt-core, the key-transparency crate (KT.md v1)

The one crate the log server, the witness and the client all link (KT.md
§11.4). A witness that verified with a different implementation than the
log builds with would produce cosignatures that mean nothing, and §7.4's
only structural defence against a lazy witness is that there are not two
implementations to disagree.

ADR 0013 buys the append-only zero-knowledge set from `akd` and nothing
else. Everything between that library and a deployment is here:

- `DirectoryEntry` / `DirectoryEntryTBS` and §4.4's entry authorization —
  `same_key` / `key_change` / `platform_reset`, `RotationProof`,
  `ResetAuthorization`. A key change carrying only one of the two
  required signatures cannot even be represented, let alone accepted.
- `SignedTreeHead`, the §6.2 signing transcript with its closed
  domain-separation label set, §6.3's eight monotonicity rules, and
  log-key rotation gated on a cosigned announcement (§6.4).
- `WitnessCosignature` over the CONTENTS — log id, epoch, tree size, root
  hash — never over the log's signature, plus the two-cosignature
  contradiction test.
- Client verification (§8.1, §8.2) wrapping `akd_core`'s lookup and
  key-history verifiers, and the witness-side append-only path (§7.1
  step 4) wrapping `akd::auditor`.
- The threshold rule (§8.3): >= t cosignatures from a CALLER-SUPPLIED
  witness set, failing closed with no partial result and no override.
- `SubmissionReceipt`, `LogDescriptor`, `FaultReport`, §9.5's error codes.

The single most important thing: **akd enforces none of our
authorization rules.** It will commit any bytes to any label, and a log
that skips §4.4 produces proofs that verify perfectly for entries nobody
authorized. So the crate is built out of choke points, not helpers.
`validate_submission` is the only entry point and the only producer of
`AcceptedSubmission`, which is the only carrier of the AkdLabel/AkdValue
pair `publish()` needs. `AcceptedRoot` (threshold), `AppendOnlyVerified`
(auditor) and `LogView` (monotonicity) work the same way: none has a
public constructor, none has a bypass flag, and the next step's argument
is the previous step's proof.

Four rules are enforced that §4.4's numbered list does not state but
ADR 0014's case analysis requires; each is marked ADDITIONAL RULE at its
call site in `submit.rs`, and the first is a real gap in KT.md:

1. A `same_key` entry MUST NOT change `identity_pk`. `same_key` is
   authorized by the PREVIOUS `directory_auth_pk` alone, so without this
   a party holding only that key installs a new identity key — a key
   change with one signature, which ADR 0014 says the log MUST reject.
2. A `key_change` or `platform_reset` MUST change `identity_pk`.
3. A `platform_reset` MUST be refused when the previous entry set
   `no_reset`.
4. A `ResetAuthorization` is bound to its handle, log, version and
   outgoing key, as §4.4 rule 6 already binds a `RotationProof`.

`rs/deny.toml` gets ADR 0013's `[[bans.deny]]` stanza for
akd/akd_core < 0.13.0 and akd_mysql. meta/akd#495 has no RustSec or
OSV advisory and never will, so the floor cannot live in a manifest or in
`cargo audit`. Verified firing: a 0.12.0 downgrade is rejected by
`check bans` while `check advisories` reports "advisories ok".

The `verifier` feature reaches wasm32-unknown-unknown and is added to the
rs_wasm CI job as `--no-default-features --features verifier`; `auditor`
pulls the server crate and must never reach a browser (KT.md §11.3).

Claude-Session: https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku

* feat(kt): add f2z-authority, the handle-ownership assertion layer

The directory can prove that @alice's key was published, never changed
behind anyone's back, and was authorized by whoever held the handle
before. It cannot prove that @alice is Alice. Something has to say who
owns a handle, no cryptography derives it, and that something is a
non-cryptographic trust root.

This is the whole of that root, isolated so a self-hoster swapping it —
or running with none — reads one short crate instead of grepping a
server.

- `HandleAssertion` / `HandleAssertionTBS`: the tls_codec structure an
  authority signs, with the label field as its domain-separated signing
  prefix.
- `AssertionBindingTBS`: what the identity key the assertion is *about*
  signs back. An assertion is a bearer document; without this, a stolen
  copy is a handle takeover. `AuthorityConfig::admit` is the only public
  verification path and takes both on every branch, so there is no route
  to a half-checked assertion.
- `AuthorityConfig::admit`: seventeen rules in order — authority
  membership and signature, log_id, handle_id, charset, timestamps, a
  validity cap the LOG holds rather than the issuer, intent against the
  entry sequence, account_epoch monotonicity, the identity binding, and
  last of all the nonce, so a failed submission never burns one.
- `AuthoritySet`: multiple (kid, pubkey) entries, so rotation is a
  set-membership change; plus an explicit, reported, no-authority mode
  for a log with no user directory.
- `f2z-assert`: an example issuer, so a log with no web application in
  front of it can issue by hand.

MIT, no_std + alloc, #![forbid(unsafe_code)], and it builds for
wasm32-unknown-unknown — clients verify assertions too. Registered with
scripts/check-rust-toolchain.sh and rs.yml's rs_wasm job.

https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku

* chore(rs): reconcile Cargo.lock after merging f2z-authority into f2z-kt-core

The two branches each resolved ed25519-dalek unambiguously; together the
graph carries both major versions, so f2z-authority's entry needs the
disambiguating version suffix.

https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku

* wip(kt): f2z-kt log server — choke point, storage, API, rate limits

The submission path routes through f2z-kt-core's validate_submission and
f2z-authority's admit; AdmittedSubmission has no public constructor and is
the only thing LogService will publish.

Not yet: f2z-witness, the adversarial and acceptance tests, CI registration.

https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku

* refactor(kt): move KT.md §9.2's envelopes into f2z-kt-core

§11.4 is explicit that the log, the witness and the client link one crate.
A witness that decoded a tree-head bundle differently from the log that
encoded it would cosign a root it did not actually read, so these
composites cannot have two implementations — and the witness needs them
at runtime without linking the server.

https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku

* wip(kt): f2z-witness — poll, verify, refuse and record, cosign

State is one file replaced atomically; the halt survives a restart; healthz
is a subcommand because the distroless image has no shell and §9.3 promises
no inbound listener.

https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku

* test(kt): adversarial submission tests and the witness acceptance suite

15 adversarial submissions, each well-formed with one thing wrong, each
asserting both the KT.md §9.5 code and that the tree did not move.

The witness suite drives a real f2z-kt log — §7.4's argument is that there
are not two implementations, so a mock would defeat it — and stages the
equivocation as two real logs sharing one signing key.

https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku

* chore(kt): register both binaries with the gates and cross the licence boundary

- scripts/check-rust-toolchain.sh --manifest for f2z-kt and f2z-witness,
  in rs.yml and rs/README.md.
- rs/deny.toml: AGPL-3.0-only exceptions by name for the two server
  binaries and nothing else; ISC and CDLA-Permissive-2.0 added for the
  witness's rustls stack with the reasoning at the entry.
- Durability tests: a restart re-derives every root the log signed, and an
  edited journal refuses to start.

fmt, clippy -D warnings, cargo-deny and the full test suite are green on
the pinned 1.97.1 toolchain; both wasm builds still pass.

https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku

* docs(rs): document both binaries in rs/README.md; add f2z-kt redaction tests

The redaction tests check base16 in both cases, a decimal byte list and
base64url — a hex-only assertion sails straight past Debug for [u8; 32],
which prints decimals.

https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku

* feat(kt): rework onto main's reviewed f2z-authority API

#626 landed f2z-authority on main with a reviewed API: admit became
check_assertion_layer, Submission gained EntryKind and previous_vouch, and
the result is explicitly not an authorization-to-publish token.

Every submission now routes through the assertion layer, not just a first
entry — the reviewed API covers all four cases, refuses a platform
assertion on a user-authorized entry as a category error, and requires the
predecessor's vouch and account_epoch back on every non-initial entry. The
log retains and feeds both forward, and the journal now stores the whole
envelope so a restart re-runs the entire admission rather than half of it.

Also satisfies two repo-wide invariants added on main that the unmerged
f2z-kt-core base never saw:

- workspace_debug_scan: hand-written Debug for AcceptedSubmission,
  VerifiedEntry and the log's Pending, which held a submission between
  acceptance and publication — the one window in which an entry is not
  public.
- workspace_strict_verification_scan: all thirteen verify functions
  registered. Two new Strictness categories were needed and are documented
  as such: DelegatesToFn, because f2z-kt-core's Ed25519 entry point is a
  free function rather than a method on a key newtype, and
  NotASignatureCheck for the four that verify akd proofs or count
  already-verified cosignatures. Each still names the call it makes, so a
  row cannot go vacuous.

https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku

* fix(kt): make the KT label set prefix-free (zuu#602 forward-port)

check-hash-domain-labels.mjs, added on main, is right about all three:

- f2z-kt-core still spelled the tree-head chain-hash label `sth-hash`,
  which makes the SignedTreeHeadTBS signing label a proper prefix of it.
  H() has no separator, so the two domains were not separated at all. That
  is #602, already fixed in KT.md on main; this forward-ports the fix into
  the unmerged crate and renames LABEL_STH_HASH to LABEL_TREE_HEAD_HASH.
- The log's heartbeat label was a proper prefix of its value label — the
  same defect in miniature, in new code, caught before it shipped. It now
  ends in ':' like KT.md §3.3's handle prefix.
- A kt-core test asserted the composed AkdLabel as one literal, which mints
  a token that is not prefix-free against the prefix constant. Composed
  from the constant instead.

https://claude.ai/code/session_01EF5qmzNm91R75ujoFuaTku

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants