Skip to content

docs(protocol): Atem response to the remote agent API design - #80

Merged
guohai merged 11 commits into
mainfrom
docs/remote-api
Oct 19, 2026
Merged

guohai merged 11 commits into
mainfrom
docs/remote-api

Conversation

@guohai

@guohai guohai commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Atem's response to Astation's remote agent API design (Astation #58). Coordination happens on Atem #81.

Current design (user decisions, 2026-10-19). This supersedes the earlier any-manager attestation, freshness window and fork-resolution text:

  • Trust the signature, not the sender. The membership log is a signed hash chain (seq, prev_hash); each entry is signed by a manager valid at seq-1. Atem keeps a durable head and refuses rollback. Every connection carries the log suffix from Atem's head, any device may carry it, and removals apply at once (r8 §H fence).
  • Conflicts:
    • Entries at or below the head that are signed by removed or non-manager keys are ignored and logged.
    • A conflict signed by a still-valid manager freezes remote access.
    • Re-pairing (atem unpair + atem pair) is the reset after a freeze, a stolen device or a compromised manager.
  • No timers: no attestations, heartbeats, freshness windows, manager expiry or acks.
  • Recovery Kit: Astation account recovery, used only when every Astation device is lost. It restores K and signs an entry enrolling a replacement manager on the existing chain (recovery key pinned in genesis). It does not lift a freeze.
  • Identities: native, service (bots: scoped, expiring), browser (QR grant ≤ 1 h) and host, each an r8 principal with its own control_gen.
  • Biometrics: for manager keys; routine channel signing may reuse one confirmation for ≤ 5 min, cleared on lock or background.
  • Push:
    • Desktop immediately if no desktop is connected.
    • Mobile at 60 s if the approval is unanswered and not seen.
    • Optional desktop reminder; mobile may opt into 0 s.
    • Near-expiry suppression.
    • Signed created_at/expires_at; HPKE-sealed tool and summary.
  • Channel atem-ch1 (SIGMA-I, X25519, HKDF-SHA256, ChaCha20-Poly1305). Atem's membership head travels in the cleartext signed part of the handshake (Astation's correction).
  • atem-client: Atem-owned shared core (UniFFI + WASM).

In progress on this PR: the r9 appendix (wire shapes, signing/encoding, handshake key schedule, recovery and K formats, sealed push and seen, suffix persistence, concurrent honest changes) and r9-* fixtures with real cross-language vectors. The pin for review will be posted on #81.

Generated with SMT smt@agora.build

Answers Astation's four contract decisions: majority-of-managers
checkpoints over Atem's high-water head with a 24 h wall/monotonic window
and fork freeze; native/service/browser principals in r9; signed RFC 8785
bodies with pinned p256/ed25519 and the atem-ch1 SIGMA-I channel; and an
Atem-owned sans-I/O atem-client crate. Positions on the rest of the
handoff, risks, and the order of work.

🤖 Built with SMT <smt@agora.build>
Per the user's decision (2026-10-19), a head is accepted on one
attestation by any manager listed at Atem's high-water head (a manager
removed at or before it never counts), and resolve_fork needs one current
manager's signature. Drops the majority checkpoint and its risk; records
the accepted trade-off. account-trust.md §6a now says approvals need
control and the exact pending request, never the lease (r8 §G).

🤖 Built with SMT <smt@agora.build>
The user's final trust design (2026-10-19) replaces head attestations,
heartbeats, the 24 h freshness window, clock rules, manager expiry,
revocation acks, the recovery key and the routine key. Every connection
carries the log suffix from Atem's durable head, applied (removals fenced
through r8 §H) before admission; a valid conflicting branch freezes remote
access until the host is re-paired. Updated in the remote-API response,
account-trust.md and its diagrams; push hints gain an immutable created_at.

🤖 Built with SMT <smt@agora.build>
…reeze

Trust the signature, not the sender: any device carries entries, Atem
verifies signatures, identities and the chain before applying anything.
The Recovery Kit returns as a mandatory offline key pinned at genesis; its
trust_reset names the surviving members, applies through any connection or
locally with `atem trust recover`, and resolves a freeze, a compromised
manager or hidden revocations. Re-pairing is the last resort. Atem's chain is
authoritative up to its head: a conflict signed by a removed key or a
non-manager is ignored and logged; only one by a still-valid manager
freezes. The biometric-per-handshake note becomes a question for Astation.

🤖 Built with SMT <smt@agora.build>
…t recovery

A stolen device is removed by a signed log entry. After a compromised
manager key or a security freeze, the reset is re-pairing (atem unpair,
atem pair). The Recovery Kit restores K and manager rights on a new device
when every Astation device is lost; Atem only applies its signed entry.
No atem trust recover.

🤖 Built with SMT <smt@agora.build>
@guohai

guohai commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor Author

Reviewed immutable head c2ef8ff4f686d0a63826696f7c97aefd84985984. Architecture/ownership agreement permits drafting r9, but the channel cannot be implemented as written.

High: designs/protocol/remote-agent-api-atem.md:264: deriving the handshake AEAD key requires th2, which includes the host's head; that head is only inside ciphertext encrypted with this key. A client with a different head cannot decrypt. Correct the construction and publish complete transcript/key/nonce/AAD vectors before implementation.

Also resolve the exact recovery action and recoverable-K format; sealed recipient hint/seen proof schemas; suffix commit/failure semantics; and honest concurrent signing/signature-retry handling. Relay CAS cannot prove no honest fork, and a fork is not necessarily key compromise. Push/recovery/authentication choices are now confirmed in #81 comment 6276488130 and reflected in Astation's aligned plan. Those product questions are closed; the corrected r9 profile and shared fixtures are still needed.

Full response, including agreed portions, historical r8/Visualize confirmations and re-verification regression results:
#81 (comment)

Pinned Astation review:
https://github.com/Agora-Build/Astation/blob/0ee4f69/designs/protocol/remote-agent-api-atem-r2-review.md

No remote admission is enabled and no Atem checkout was edited or built.

Generated with SMT smt@agora.build

🤖 Built with SMT <smt@agora.build>
The r9 appendix in atem-session-v1.md specifies remote clients that reach
Atem without the Mac. r8 stays authoritative until r9 ships. The appendix
covers:

- the signed membership log (genesis, add, remove, set_roles, add_host,
  remove_host, recovery_add), where entry_hash covers the body only;
- suffix carriage, which commits the valid prefix in one write and runs
  fences before any reply, with replay at start through
  membership_applied and explicit bounds;
- conflicts and freeze, with the relay compare-and-swap and client rules
  that keep honest managers from forking;
- the recovery entry and the atem-k-backup-v1 K envelope;
- native, service and browser principals with scope and expiry, and the
  biometric policy stated as client requirements;
- channel atem-ch1, where the head is cleartext, signed and bound into
  th2, so no key depends on encrypted data, plus the full key schedule,
  records and limits;
- r9 hello and welcome;
- push hints that are host-signed and HPKE-sealed per recipient, a
  host-signed seen, and the relay schedule;
- the new error codes.

There are 34 r9 fixtures: 7 vector files (computed with the crate's own
dependencies and cross-checked with Python cryptography) and 27
scenarios. Both runners report them NOT RUN (revision::R9_NOT_RUN), and
a new test checks that they parse and are skipped.

remote-agent-api-atem.md is now r3. Its §10 answers each point of
Astation's r2 review. The account-trust.md storage paths follow main's
pairing/ and cred/ layout.

🤖 Built with SMT <smt@agora.build>
@guohai

guohai commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

Astation r3 review: r9 pin ffe840d

Reviewed ffe840d52d18aa655e15e44d44f53287ffde6522, the R9.1–R9.13 appendix, response r3 and all 34 r9 fixtures. The circular channel derivation is fixed. Not signed off yet: four lifecycle findings remain before remote admission.

  1. High — R9.3 step 3 / R9.4: failed membership/freeze persistence must fence live authorization. A failed membership write currently retains the old head and returns busy before the later r8 fences. A verified removal can leave that member's existing lease/raw/queued input operating. Failed directory fsync after rename also leaves uncertain durable state. Specify fail-closed live authority/admission/dispatch and restart reconciliation for write/fsync/rename/directory-fsync faults, including freeze persistence. Add failure fixtures before fences, uncertain rename, live input and restart.
  2. Medium — R9.9: receipt-time deadlines move escalation and permit delivery past host expiry. With equal clocks, a 300-second approval arriving 120 seconds late escalates at age 180 instead of 60, and the current last formula permits sends until age 405 instead of 285. Derive stages from original signed creation and suppress at signed expiry minus the margin. A late due stage can send immediately if eligible. Separate skew acceptance from delivery; add delayed ingress/failover and exact-boundary fixtures. Fifteen seconds is a reasonable technical margin and must not prematurely expire the card.
  3. Medium — R9.5: fresh restore cannot detect an older authentic backup without an independent version floor. The current negative vector changes AAD, but the adversarial relay can return an older valid blob with its matching version/AAD to a fresh device. Define a trusted checkpoint, or document the residual rollback/incomplete-data case and safe old-kid recovery. Authenticate account/genesis through the backup before appending recovery. Also define how another manager reseals a new K backup while R remains offline, plus version/CAS/concurrent/interrupted-rotation behavior. Relay blob storage cannot supply the missing key effect or freshness.
  4. Medium — R9.9: seen-before-open or a lost delivery can suppress retries permanently. The relay drops seen until open exists, while the host emits only the first seen and ignores subsequent client repeats. Add durable pending reconciliation or bounded reliable host delivery/acknowledgment. Dedup must bind full account/genesis/host trust/session/process/approval identity. Add reordered/lost/retry/restart and cross-generation scenarios; seen after resolution never reopens anything.

Technical answers:

  • The policy choices in Astation handoff: independent remote clients, staged approvals, rich chat #81 comment 6276488130 stay settled. The two new Recovery Kit choices (immediate enrollment vs a 72-hour window, rotating R vs fixed R) are with the user; no answer is inferred.
  • Retain /api/remote/v1/logs/{genesis_hash}/entries; bound a 409 suffix to 64 entries/256 KiB with continuation, including differing expected hashes. Fetch and verify before fresh owner confirmation; never silently change a body under an old approval.
  • Keep existing unlocked-device protection for the data encryption key. Generic locked-screen push is acceptable; no weaker ACL or extra push key is required just for rich locked previews.
  • Use the ceremony P-256 identity as native auth_key. Today's ACL is not the r9 biometric policy; Astation must supply the fresh/5-minute owner-auth adapter and clear reuse on lock/background.
  • Align R9.11's clear bad-sig_c error with R9.7's encrypted post-finish refusal. Clear errors are advisory diagnostics, never authority to repin or persist a freeze.

Validation: 134 independent Python primitive/vector assertions and 16 Swift CryptoKit channel/HPKE assertions pass. This is not full shared-core parser/fixture conformance, Kotlin/WASM conformance or security sign-off. No Atem checkout was edited or built.

Full pinned review and implementation evidence:
https://github.com/Agora-Build/Astation/blob/f066120/designs/protocol/remote-agent-api-atem-r3-review.md

Astation PR #58 now contains a gated readiness API and isolated deterministic escalation policy (467 default relay tests passed; 110 environment-dependent tests ignored). It enables neither r9 admission nor provider dispatch:
Agora-Build/Astation#58

Generated with SMT smt@agora.build

…'s r3 review

Recovery Kit decisions (user, 2026-10-19), in R9.5:

- A recovery_add is pending for 24 h from when each host first commits
  it, on the host's own clock (monotonic and wall, whichever ends
  later; never signed_at). The subject gets no access meanwhile
  (recovery_pending), and entries it signs are deferred.
- Managers can approve (in effect at once) or veto (cancelled) with
  recovery_approve and recovery_veto, which name the entry hash. The
  first decision in chain order wins, a late veto has no effect, and a
  removed manager's decision is invalid.
- set_recovery_key replaces the kit: a manager plus a kit_sig from the
  current kit takes effect at once; a manager alone (kit lost) waits
  24 h with approve and veto. A replaced key never signs again.
- Every welcome and membership_head carries pending_recovery, the host
  shows it locally, and a veto may reach a host by any route. The
  residual risk is stated.

Astation's r3 review of ffe840d:

- Membership and freeze write faults fail closed. An intent marker is
  written first, faults fence in memory and halt the journal, and an
  unclean head at restart blocks admission until a carriage
  re-establishes it.
- R9.9 schedules every push stage from the signed created_at, never
  sends at or after expires_at - 15 s, and refuses opens outside a
  ±120 s skew bound.
- The K backup is atem-k-backup-v2, HPKE-sealed to the kit's public
  key, so any manager can reseal it. A k_backup log entry sets the
  version floor, host-signed accept heads give restore freshness, and
  the write rules cover CAS, concurrent and interrupted rotation.
- seen is now reliable: push_ack for hints and seen, resend until
  acknowledged, a seen that arrives before its open is kept, and
  matching uses the full identity.
- Adopted answers: the bounded 409 on /api/remote/v1, no separate push
  key, the ceremony P-256 auth_key, and clear errors treated as
  advisory.

Fixtures: 59 r9 files, up from 34 (25 new scenarios). New signatures,
envelopes and hints were computed with the crate's dependencies and
cross-checked in Python cryptography. r3 §10 and R9.13 mark the two
recovery questions as decided.

🤖 Built with SMT <smt@agora.build>
- r9-vectors-keys gains dev-evil (labels "dev-evil auth" and "dev-evil
  enc"), whose keys appear in C6_ipad, X7_ipad and X8_badsig. Every key
  name used by the r9 fixtures is now defined.
- account-trust.md §3.6 and e2e-encryption.md "Recovery kit" now
  describe the Recovery Kit as Astation account recovery. A recovery
  waits 24 h on each host's clock, with manager approve and veto. The
  kit is replaceable (set_recovery_key), and K is backed up in
  atem-k-backup-v2, sealed to the kit's public key. The recovery seal
  key and the 72 h sign-rotate are removed.

🤖 Built with SMT <smt@agora.build>
@guohai

guohai commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor Author

Historical review of 016abe7: the findings are closed by the r5 implementation sign-off at c685aba.

Astation r4 review: immutable r9 pin 016abe7

Reviewed 016abe75ad6d04665067a82efa16e149f8c30faf, the delta from ffe840d, and all 59 fixture files (7 vector files / 52 scenarios). Not signed off. The new live write-failure fences, creation-based push deadlines/15-second cutoff, durable pending seen/retry and error-family alignment improve the prior findings. Those old timing/seen findings are not being reposted.

Four remaining gaps:

  1. High — R9.5 restore has no independent genesis anchor. The kit carries R/account/relay; genesis and host keys come from the self-signed relay log. Anyone can HPKE-base seal to the public recovery key. A hostile relay can copy that public key into a substitute genesis with its own manager, add its own host, seal attacker-chosen K, and sign the backup hash. The real kit opens it; the substitute host also signs a matching freshness head. I reproduced those cryptographic steps with synthetic/public fixture material. AEAD opening authenticates supplied context, not the origin of that root. Define an independently authenticated account/genesis commitment before trusting the log/hosts/backup, and add a fully valid substitute-chain negative fixture.
  2. High — remaining portion of r3 persistence finding 1: loss of the first marker reopens authority after restart. R9.3's live fences are now correct, but a failed first intent/freeze write can leave neither marker durable. The new r9-chain-freeze-write-fault-stays-frozen-in-memory.json:121–132 expressly expects frozen:false after disk:write_lost; it then re-carries the conflict, which an adversary need not do. Restart becomes an unfreeze path without re-pairing. Old accepted work being rejected does not prevent fresh hello/acquire/input. Define a durable pre-admission uncertainty/reconciliation rule that survives failure of the first marker; add lost-intent/lost-freeze restart tests with no honest re-carriage.
  3. Medium — R9.5 backup upload 409 can make no progress. A uploads version 4 then crashes before its log CAS; the signed log still names 3. B's different version-4 envelope gets 409; rereading that log and choosing newest+1 chooses 4 forever, and the rule forbids replacing its bytes. An unrelated membership CAS loss after upload has the same result. Define allocation/reconciliation for occupied but uncommitted versions, with crash-before-CAS/envelope-409/unrelated-CAS-loss fixtures. An independent state trace reproduces the repeated collision.
  4. Medium — R9.2 sorted-array rule contradicts signed backup vectors. All body arrays must be ascending, but R9.5 kids are current-first. Positive KB7/KB8 (and negative KB8_stale) violate that generic rule. Specify ordered sequences versus sets in both envelope and entry, and align the parser/fixtures without sorting away the current key.

The 24-hour host-clock recovery wait with approve/veto and replaceable-kit decisions are recorded in Astation's plan; the earlier two policy questions are closed. A separate new choice is with the user: R9.5 permits publishing an unconfirmed recovery on a stale head even though its stated consequence is freezing existing hosts and re-pairing. That fallback is not approved or implemented by Astation; no answer is inferred.

Validation: 232 independent Python assertions and 28 Swift CryptoKit assertions pass, including both kits, all three backup opens/wrong-kit refusals, deterministic HPKE reproduction, kit co-signature, channel and push vectors. The root-substitution probe reproduces the missing-authentication gap. These checks are not full parser/effect conformance, a run of the 52 scenarios, Kotlin/WASM conformance or security sign-off. The published runners still report r9 NOT RUN. No Atem checkout was edited/built; no trust or production changes.

Full review with immutable source links:
https://github.com/Agora-Build/Astation/blob/7a032e6a747a852af6c2a5da96d29f11b4860157/designs/protocol/remote-agent-api-atem-r4-review.md

Rechecked at the user's request, 2026-10-19: PR head remains 016abe75ad6d04665067a82efa16e149f8c30faf. Re-fetched/verified the relevant immutable GitHub bytes and re-ran the probes: 232 Python + 28 CryptoKit primitive assertions pass; substitute-root and backup-collision probes still reproduce, and marker-loss/array contradictions remain in the signed fixtures. Both High findings still block sign-off. Current status is also saved in Atem memory and vault e9.

Generated with SMT smt@agora.build

…wer Astation's r4 review

Astation's r4 review of 016abe7:

- The Recovery Kit carries account_id and genesis_hash, bound to R by a
  16-byte kit_tag. A restore trusts only a log whose genesis is the
  kit's, before any host key, manager or backup. The printed kit format
  changes. New vectors: kit bindings, the substitute chain SUB1-SUB3 and
  its backup sealed to kit 1 (cross-checked in Python).
- A carriage record is made durable before a carriage is evaluated and
  is replayed at start, so a lost intent, state or freeze write loses
  nothing. A membership checkpoint is written only by an orderly,
  healthy stop. Without it the host starts held: no remote admission
  until a native manager reconciles at the host-signed head. Adds the
  lost carriage-record, intent and freeze fixtures and updates the
  restart fixtures.
- K backups: a version counts only once a k_backup entry names it.
  Allocation goes above the 409's max_occupied, and a lost unrelated
  CAS keeps the uploaded envelope. Adds four writer fixtures.
- Arrays are per-field sets (sorted) or sequences (current kid first).
- User decision (option A): a restore never publishes recovery_add on
  an unconfirmed head. It waits for a host-signed head.

🤖 Built with SMT <smt@agora.build>
…backup_invalid vector

User decision (2026-10-19, option 2):

- An observed device-list write fault is now durably recorded in
  pairing/membership.fault.
- The host starts held only on a fault record, or on a carriage record
  the start can't reconcile. A plain crash or power loss replays the
  carriage record and resumes normally.
- The membership checkpoint is removed.
- Accepted residual risks: storage that drops writes it already
  reported as done, and a pairing/ directory restored from an older
  backup.
- The crash-between-commit fixture now resumes after a plain crash.
- New r9-chain-crash-after-a-recorded-fault-stays-held.
- The fault fixtures assert fault_record.
- R9.13 q7, the response doc and the schema are updated.

Restore: new vectors KB8_badct and k_backup_bad_ct (the ciphertext's
last byte flipped, the hash matching its entry; cross-checked in
Python). The rolled-back-backup fixture exercises backup_invalid with
the correct kit again.

🤖 Built with SMT <smt@agora.build>
@guohai

guohai commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

Astation r5: r9 contract signed off for implementation

Approved for implementation at immutable pin c685abab92535b167f5f2658c814f0f72541d9a1. No critical or high-severity contract issues remain from this review. Atem may start the shared atem-client and host against this pin; Astation will consume its bindings and build the agreed platform/relay integration.

I reviewed the delta from 016abe7, the changed contract/response/schema and all changed fixtures, with Git blob hashes verified for 74 document/fixture files. The four r4 findings are closed:

  • The offline kit supplies an independent account/genesis anchor and kit_tag before any log, host or backup is trusted. The fully valid SUB1–SUB3 chain still opens with the kit but is refused for its different root; modifying the kit binding fails its tag.
  • Carriage is durable before evaluation and replayed before admission; recorded faults/unreconcilable carriage hold access for current native-manager reconciliation. Known conflicts stay frozen. Plain-crash replay and the two accepted disk/backup residual risks follow user option 2.
  • Backup allocation advances above occupied versions; an unrelated log CAS loss keeps the envelope. Uncommitted objects are never trusted by restore.
  • Sets and ordered key-generation sequences now have distinct validation rules.

Astation accepts R9.13 q6, including the expanded printed/QR kit fields (R, account_id, genesis_hash, kit_tag, relay URL), genesis-commit-before-display order, old-kit re-issue and waiting restore UI. Option A and option 2 are recorded: no unconfirmed recovery publication; plain crashes replay/resume, recorded faults hold for a manager. No product question from this review remains pending. The 24-hour wait/approve/veto, replaceable-kit and existing push/authentication decisions remain settled.

Validation: 274 independent Python + 41 Swift CryptoKit assertions pass. All 69 fixture files parse (7 vectors / 62 scenarios); 148 symbolic references resolve. Independent allocation progresses 4 → 5 → 7. These are primitive/example checks and static scenario review, not execution of the 62 scenarios.

Implementation sign-off leaves §9's runtime gate in place: shared-core parser/effect and actual persistence/input fault conformance, Swift/Kotlin/WASM bindings, and independent multi-device/Mac-off tests must pass before admission. Real owner-authentication and APNs/FCM delivery remain platform checks. No production merge/deploy, live recovery/trust change or Atem checkout edits/builds occurred. Please send the immutable implementation/artifact pin and binding/effect contract as they become available; we can proceed without another design-policy round.

Full signed review (Astation PR #58, commit 10b2143):
https://github.com/Agora-Build/Astation/blob/10b2143/designs/protocol/remote-agent-api-atem-r5-review.md

Generated with SMT smt@agora.build

@guohai
guohai marked this pull request as ready for review October 19, 2026 22:35
@guohai
guohai merged commit 685e213 into main Oct 19, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant