diff --git a/apps/backend/docs/concepts-e2ee-key-management.md b/apps/backend/docs/concepts-e2ee-key-management.md new file mode 100644 index 0000000..9af475d --- /dev/null +++ b/apps/backend/docs/concepts-e2ee-key-management.md @@ -0,0 +1,309 @@ +# E2EE device identity & key lifecycle (backend) + +This document explains the backend's role in the end-to-end-encryption key +lifecycle: how a device gets an identity, how it publishes key material, +how other clients fetch and consume that material, and how a device is +retired. It covers the full lifecycle as one coherent story and names the +exact routes and tables involved at each stage. + +**Scope.** This is a backend concepts document. It does not describe any +client-side cryptography (X3DH session establishment, Double Ratchet, MLS +epoch secrets) — only what the server stores, serves, validates, and +deletes along the way. See +[`../../apps/web/docs/concepts-e2ee-architecture.md`](../../apps/web/docs/concepts-e2ee-architecture.md) +for the client-side key model. + +## What the server can and cannot see + +Stated explicitly, because every section below depends on it: + +- **The server never sees a private key, of any kind.** Not an identity + private key, not a signed-prekey private key, not a one-time-prekey + private key, not an MLS HPKE init private key, not a session/ratchet key. + Every key-shaped column in `devices` and `device_prekeys` (and + `mls_key_packages`) stores **public** material only — + `identityPublicKey`, `publicKey` (signed and one-time), `keyPackage`. This + is a property of what the schema is defined to hold, not an access-control + claim about what a query *could* return if misused. +- **The server verifies signatures without needing the signer's private + key** — Ed25519 signature verification is a public-key operation. The + signed-prekey upload route (`POST /devices/:id/prekeys`) verifies that + `signedPreKey.signature` is a valid signature over `signedPreKey.publicKey` + made by the device's own `identityPublicKey`, entirely with public data. + This proves the signed prekey really was published by the device that owns + the identity key, without the server ever holding a signing key of its own + for that device. +- **The server cannot decrypt message ciphertext.** `messages.ciphertext` + and `message_envelopes.ciphertext` are opaque blobs to every backend code + path — nothing in `routes/` or `services/` ever attempts to decrypt them. + (See [`contracts-database-schema.md`](contracts-database-schema.md#messages) + for the column, and + [`../../apps/web/docs/concepts-e2ee-architecture.md`](../../apps/web/docs/concepts-e2ee-architecture.md) + for where decryption actually happens — exclusively on-device.) +- **What the server *can* see and act on**: which device belongs to which + user, when a device was last active, how many one-time prekeys a device + has left, which device fetched which other device's bundle and when + (`key_bundle_drained` audit events — see + [Audit trail](#audit-trail-what-gets-logged-and-why) below), and the + device capability set a device advertises (which protocols/ciphersuites it + supports — see + [`concepts-protocol-negotiation.md`](concepts-protocol-negotiation.md)). + All of this is metadata about key *distribution*, never about message + *content*. + +## The lifecycle, stage by stage + +### 1. Device identity & registration + +A device's identity is its Ed25519 `identityPublicKey`, stored once per +device in `devices.identity_public_key`. There are **two independent code +paths** that can create or resolve a `devices` row for an existing account, +and they behave differently on a revoked identity key — this distinction +matters and is easy to miss: + +| | `POST /auth/challenge` → `POST /auth/verify` (`routes/auth.ts`) | `POST /devices/link/challenge` → `POST /devices/link/verify` (`routes/devices.ts`) | +|---|---|---| +| Purpose | Sign in (first device, or any later device, via wallet proof) | Explicitly link an *additional* device to an already-authenticated account | +| Caller authentication | None required beforehand — this *is* the login step | Requires a valid JWT already (the caller must already be signed in on some device) | +| Proof required | Wallet signature over the sign-in challenge message | A **second, distinct** wallet signature over a device-link-specific message (`"Link device to Clicked\nUser: ...\nNonce: ..."`) — deliberately different wording from the login message so one signature can never be replayed as the other (see [`concepts-nonce-lifecycle.md`](concepts-nonce-lifecycle.md)) | +| Lookup key | `(userId, identityPublicKey)` | `(userId, identityPublicKey)` | +| New identity key, no existing row | Inserts a new `devices` row | Inserts a new `devices` row | +| Existing row, **not** revoked | Reuses it, updates `lastSeenAt`/name/platform/`registrationId`/capabilities | `409 Device already registered for this user` | +| Existing row, **revoked** | **Hard refusal**: `401 Device has been revoked` — sign-in is blocked outright, no reactivation | **Reactivates** the row in place (`revokedAt` cleared, name/platform/`registrationId` updated) — the comment in the source calls this out as a deliberately distinct audit event from a fresh link | +| Audit action recorded | `auth_failed` with `reason: device_revoked` (failure case only; success records nothing here) | `device_linked`, with `metadata.reactivatedRevokedDevice: true/false` | + +`POST /devices` itself (the bare, JWT-only registration endpoint) is +**retired** — it now unconditionally returns `403` directing the caller to +the link-challenge flow (tracked in-source as issue #333, closing a gap +where a stolen JWT alone was sufficient to attach an attacker-controlled +device). If you are reading +[`api-devices.md`](api-devices.md) and its `POST /devices` section describes +reactivation-on-registration as a single unauthenticated-by-wallet step, +that section predates #333 and no longer matches `routes/devices.ts` — this +document reflects the current two-step link flow; `api-devices.md` should be +read for the routes it still accurately describes (listing, revocation, +prekey upload). + +Either path writes the same columns: `identityPublicKey` (immutable per +row — there is no "update my identity key in place" operation; a client that +rotates its long-term identity key produces a **new** `devices` row, not a +mutated existing one — see [Key rotation](#7-key-rotation-and-the-unwritten-devicekeyhistory-table) +below), plus `deviceName`, `platform`, `registrationId` (the X3DH +registration id from the prekey bundle spec), and `capabilities` (the +protocol/ciphersuite/file-transfer versions the device advertises, defaulted +via `normalizeCapabilities` if omitted — see +[`concepts-protocol-negotiation.md`](concepts-protocol-negotiation.md)). + +### 2. Prekey publication + +`POST /devices/:id/prekeys` (ownership- and revocation-checked — see +[`api-devices.md`](api-devices.md#post-devicesidprekeys) for the full +request/response contract) writes into `device_prekeys`, discriminated by +`keyType`: + +- **Signed prekey** (`key_type = 'signed'`) — exactly one live row per + device, **upserted** on every upload (`ON CONFLICT` on the partial unique + index over `deviceId` where `keyType = 'signed'`). The server verifies + `signature` is a valid Ed25519 signature over `publicKey`, signed by the + device's `identityPublicKey`, **before** writing anything — an invalid + signature means nothing is persisted. This is the one point in the whole + lifecycle where the server actively validates a cryptographic property of + uploaded key material, rather than just storing it. +- **One-time prekeys** (`key_type = 'one_time'`) — many rows, added to the + existing pool (`ON CONFLICT ... DO NOTHING` on `(deviceId, keyType, + keyId)`, so a retried upload of the same `keyId` is a silent no-op, not an + error). Capped at **200 unconsumed** per device (`OTP_CAP`); a batch that + would exceed the cap is trimmed, not rejected outright, as long as some + headroom remains — see [`api-devices.md`](api-devices.md#the-200-key-cap-and-trimming-behavior) + for the exact trimming arithmetic. + +Nothing here is decrypted or re-derived — the server stores the public +key/signature bytes exactly as submitted and never generates key material on +a device's behalf. + +### 3. Key-bundle fetch & one-time-prekey consumption + +`GET /users/:userId/devices/:deviceId/key-bundle` (`routes/users.ts`) is how +a sender obtains a recipient device's X3DH bundle to start a session. Three +things happen atomically inside one DB transaction: + +1. The device's current **signed prekey** is read (required — `409` if the + device has never uploaded one). +2. **At most one** unconsumed one-time prekey row is selected + (`FOR UPDATE SKIP LOCKED`, oldest first) and flipped to `consumed = true` + — flipped, never deleted, so the fetch history stays auditable. Two + concurrent fetches for the same device can never be handed the same + one-time prekey, because the row lock excludes it from the other + transaction's candidate set. +3. The post-consumption remaining count is computed **inside the same + transaction**, so it can never race a concurrent fetch into reporting a + stale number. + +If no one-time prekey remains, the bundle is still returned — with +`oneTimePreKey: null` — rather than erroring, so the sender can fall back to +a 3-DH session using only the signed prekey. See +[`e2ee-onboarding.md`](e2ee-onboarding.md#c-low-prekey-warning-before-exhaustion) +for the exact JSON shapes of both the normal and exhausted responses, and +for the `prekeys_low` socket event that fires (debounced, once per threshold +crossing) to warn the *owning* device to replenish before it actually runs +dry. + +Every successful one-time-prekey consumption also increments a metric +(`prekeyConsumedTotal`) and records a `key_bundle_drained` audit event (see +[Audit trail](#audit-trail-what-gets-logged-and-why)) — the subject is the +device **owner** (whose prekey supply just shrank), the actor is whoever +fetched the bundle. + +### 4. Replenishment signal and background pruning + +Two independent mechanisms keep the one-time-prekey pool healthy without any +cross-service coordination: + +- **Low-supply push**: a `prekeys_low` event to the owning device's socket + room (`device:{deviceId}`) once a fetch drops the remaining count below a + threshold (default `20`, `PREKEY_LOW_THRESHOLD`). Re-arms once the device + tops back up at or above the threshold, so a later crossing warns again. +- **Background GC** (`services/deviceGc.ts`, hourly by default): prunes + `device_prekeys` rows where `key_type = 'one_time'` and either `consumed = + true` and older than `PREKEY_CONSUMED_RETENTION_DAYS` (default 30d), or + unconsumed and older than `PREKEY_UNCONSUMED_MAX_AGE_DAYS` (default 90d). + Signed prekeys are never touched by this pass — there is exactly one live + signed prekey per device, replaced in place on upload, not garbage + collected. See [`concepts-gc-jobs.md`](concepts-gc-jobs.md) for the full + job reference, including the MLS KeyPackage pass that mirrors this policy. + +### 5. Revocation + +A device is revoked via `DELETE /devices/:id` (single device, caller-owned +only, idempotent, refuses to revoke the caller's last active device) or +`POST /devices/logout-everywhere` (every other active device on the +account). Both route through one shared helper with **identical** per-device +side effects: + +1. `devices.revokedAt` is set (row kept, not deleted — revocation history is + permanent and queryable). +2. Every row in `device_prekeys` for that device — signed **and** one-time, + consumed or not — is **deleted outright**. A revoked device has no + prekeys left to hand out; re-registering the same identity key later (via + the link-verify reactivation path above) starts prekey-less and must + re-upload. +3. Live sockets for that device are force-disconnected, same-node + immediately and cross-node via a `device_revoked:{deviceId}` Redis + pub/sub publish (best-effort; a publish failure does not fail the HTTP + request). +4. A `device_revoked` system event is emitted into every conversation the + user is a member of, so other clients learn "this user's device set + changed" and know to refresh key bundles before encrypting to them again. + +Full request/response detail and the exact ownership/idempotency rules are +in [`api-devices.md`](api-devices.md#revocation-side-effects) — that section +of the doc is still accurate (only the *registration* section is affected by +the #333 change above). + +### 6. Post-revocation staleness flagging + +`services/deviceGc.ts`'s third pass flags (never deletes) devices that have +been revoked longer than `DEVICE_STALE_AFTER_DAYS` (default 180d), by +setting `devices.staleFlaggedAt`. This is purely informational — it marks a +row eligible for whatever downstream archival process an operator chooses +to run; revocation audit history is preserved indefinitely regardless. + +### 7. Key rotation and the unwritten `deviceKeyHistory` table + +`device_key_history` exists in the schema specifically to let clients detect +a silent identity-key swap and show a safety-number-style warning: it +records `(deviceId, userId, previousKey, newKey, changeReason, recordedAt)`, +and `GET /users/:id/key-history` (`routes/users.ts`) reads it back, ordered +oldest-first. + +**Verified against the current source: nothing writes to this table.** +There is no `db.insert(deviceKeyHistory, ...)` call anywhere in +`apps/backend/src`, and no test exercises one. The schema comment's claim +that it is "written whenever a device's `identityPublicKey` changes" does +not correspond to any implemented write path today — because, per +[stage 1](#1-device-identity--registration), **there is also no operation +that changes an existing device row's `identityPublicKey` in place.** A +client that rotates its long-term identity key does so by registering what +the server sees as a brand-new, unrelated `devices` row (a new `(userId, +identityPublicKey)` pair) — the old row is left exactly as it was, not +updated and not linked to the new one. `GET /users/:id/key-history` is +therefore a read endpoint over a table that the current implementation never +populates; treat it as reading back an empty history today, not as a working +key-transparency feed. Closing this gap would mean either (a) adding an +explicit identity-key-rotation endpoint that updates a row in place and logs +the transition, or (b) deriving history from the existing +add-new-row/revoke-old-row pattern after the fact — neither exists yet. + +### MLS key packages — a parallel, separate mechanism + +Group conversations run MLS (RFC 9420) instead of per-recipient X3DH +sessions, and MLS has its own public-key-material table, +`mls_key_packages`, populated by `POST /devices/:id/mls-key-packages` under +the same "public material only, idempotent upload, capped pool" shape as +`device_prekeys` — but it is a genuinely separate mechanism with its own +single-use-per-Add semantics, not an alternate code path over the same +table. This document's scope is the `devices`/`device_prekeys` pair +specifically; see [`mls-key-packages.md`](mls-key-packages.md) and +[`mls-group-membership.md`](mls-group-membership.md) for the MLS-side +lifecycle. + +## `devices` / `device_prekeys` at a glance + +Full column-by-column documentation (types, defaults, indexes, FKs) is in +[`contracts-database-schema.md`](contracts-database-schema.md#devices) — this +is just the subset load-bearing for the lifecycle above: + +| Column | Table | Role in the lifecycle | +|---|---|---| +| `identityPublicKey` | `devices` | The device's long-term public identity; immutable per row (see [stage 7](#7-key-rotation-and-the-unwritten-devicekeyhistory-table)); lookup key for both registration paths. | +| `registrationId` | `devices` | X3DH registration id published alongside the bundle. | +| `capabilities` | `devices` | Advertised protocol/ciphersuite support; defaults applied for pre-existing rows. | +| `revokedAt` | `devices` | `NULL` while active; set (never cleared except by link-verify reactivation) on revocation. | +| `staleFlaggedAt` | `devices` | Set by GC once revoked past the retention window; informational only. | +| `keyType` | `device_prekeys` | `'signed'` (one live row, upserted) or `'one_time'` (many rows, pooled and capped). | +| `signature` | `device_prekeys` | Required (DB check constraint) and server-verified only for `keyType = 'signed'`; never set for one-time keys. | +| `consumed` | `device_prekeys` | Flips `true` on bundle-fetch claim; one-time keys only; never un-set. | + +## Audit trail: what gets logged, and why + +Every stage above that touches a device's security posture writes to +`audit_logs` (see +[`../../../docs/security/audit-logging.md`](../../../docs/security/audit-logging.md) +for the full catalog and what is deliberately excluded from it): + +| Action | Fired by | Subject vs. actor | +|---|---|---| +| `device_linked` | `POST /devices/link/verify` | Actor = the authenticated caller; target = the new/reactivated device | +| `device_revoked` | `DELETE /devices/:id`, `logout-everywhere` (once per call, not per device) | Subject = device owner | +| `logout_everywhere` | `POST /devices/logout-everywhere` | Subject = the caller (revoking their own other devices) | +| `auth_failed` (`reason: device_revoked`) | `POST /auth/verify` rejecting a revoked device | Subject = the account being signed into | +| `key_bundle_drained` | `GET /users/:userId/devices/:deviceId/key-bundle` consuming the last (or any) one-time prekey | Subject = bundle/device **owner**; actor = the fetching caller | + +These rows never contain key material or message content — only +identifiers, counts, and outcomes, per the append-only audit log's design +constraint. + +## Related documents + +- [`api-devices.md`](api-devices.md) — full request/response reference for + listing, revocation, and prekey upload (registration section is + superseded by [stage 1](#1-device-identity--registration) above). +- [`e2ee-onboarding.md`](e2ee-onboarding.md) — the exact JSON shapes and + sequencing for first-time onboarding and first-DM bundle fetch, including + the offline-recipient and prekey-exhausted paths. +- [`contracts-database-schema.md`](contracts-database-schema.md) — full + column/index/FK reference for `devices`, `device_prekeys`, and every other + table. +- [`concepts-protocol-negotiation.md`](concepts-protocol-negotiation.md) — + how `capabilities` drives sender/recipient protocol agreement. +- [`concepts-nonce-lifecycle.md`](concepts-nonce-lifecycle.md) — the + sign-in and device-link nonce stores referenced in stage 1. +- [`concepts-gc-jobs.md`](concepts-gc-jobs.md) — the full background-job + reference, including the prekey/MLS-KeyPackage pruning and device + stale-flagging passes. +- [`mls-key-packages.md`](mls-key-packages.md) — the parallel MLS key + material lifecycle. +- [`../../../docs/security/audit-logging.md`](../../../docs/security/audit-logging.md) — + the audit log format and full action catalog. +- [`../../apps/web/docs/concepts-e2ee-architecture.md`](../../apps/web/docs/concepts-e2ee-architecture.md) — + the client-side key model this document deliberately does not cover. diff --git a/apps/backend/docs/concepts-presence-heartbeat.md b/apps/backend/docs/concepts-presence-heartbeat.md new file mode 100644 index 0000000..536ce27 --- /dev/null +++ b/apps/backend/docs/concepts-presence-heartbeat.md @@ -0,0 +1,317 @@ +# Presence, heartbeat & connection liveness + +This document explains how the backend knows whether a user is online: the +Redis-backed per-device/per-user presence model in +[`services/presence.ts`](../src/services/presence.ts), the heartbeat watchdog +in [`services/heartbeat.ts`](../src/services/heartbeat.ts), and how the two +are wired together in the Socket.IO connection handler in +[`index.ts`](../src/index.ts). Verified against the current implementation of +all three files. + +For the WebSocket event contract (`heartbeat`, `presence_update`, +`user_online`/`user_offline`) from a client's point of view, see +[`api-websocket-events.md`](api-websocket-events.md). For the connection +lifecycle these services plug into, see +[`concepts-gateway-architecture.md`](concepts-gateway-architecture.md). For +the `users`/`devices` table columns referenced throughout (`presenceVisible`, +`lastSeenVisible`, `devices.lastSeenAt`, `devices.revokedAt`), see +[`contracts-database-schema.md`](contracts-database-schema.md). + +## Table of contents + +- [Two independent signals of "online"](#two-independent-signals-of-online) +- [Redis key structure](#redis-key-structure) +- [The device → user presence model](#the-device--user-presence-model) +- [Heartbeat timing contract](#heartbeat-timing-contract) +- [Connection lifecycle, step by step](#connection-lifecycle-step-by-step) +- [Multi-device aggregation](#multi-device-aggregation) +- [Transition-broadcast behavior](#transition-broadcast-behavior) +- [Privacy gating](#privacy-gating) +- [Known caveats, verified against the current code](#known-caveats-verified-against-the-current-code) + +## Two independent signals of "online" + +The backend actually has **two** ways to decide if a user is online, and it +prefers the first but falls back to the second: + +1. **Redis presence registry** (`services/presence.ts`) — live, socket-driven, + second-granularity. This is the fast path and the one that drives + real-time `presence_update`/`user_online`/`user_offline` broadcasts. +2. **Postgres `devices.lastSeenAt`** (`deriveDevicePresence`) — a + database-backed fallback with a 90-second recency window, used when Redis + is unreachable, and as the source for the `lastSeen` timestamp shown once + a user is confirmed offline. + +`GET /users/:id/presence` (`src/routes/users.ts`) tries Redis first +(`isOnline`) and only calls `deriveDevicePresence` if Redis says the user is +not online (or Redis is not configured at all) — see +[Known caveats](#known-caveats-verified-against-the-current-code) for what +this ordering does and does not guarantee. + +## Redis key structure + +Every key below is per-process-shared (visible to every gateway node), so +presence is correct in a horizontally-scaled deployment without any +in-process state. `PRESENCE_TTL = 90` (seconds) is the single TTL constant +used throughout `presence.ts`. + +| Key pattern | Type | Holds | TTL | Set/refreshed by | +|---|---|---|---|---| +| `presence:user:{userId}` | hash | `{ [deviceId]: lastSeenEpochMs }` — one field per currently-online device | **none** — not expired directly; see [caveat](#known-caveats-verified-against-the-current-code) | `setOnline`, `refreshPresence` (hset); `setOffline`/`markDeviceOffline` (hdel field, delete key once empty) | +| `presence:user:{userId}:device:{deviceId}` | hash | `{ lastSeen: epochMs }` | `PRESENCE_TTL` (90s), refreshed on every `setOnline`/`refreshPresence` | `setOnline`, `refreshPresence` | +| `presence:sockets:{userId}` | set | socket ids currently open for this user, across all their devices | `PRESENCE_TTL` (90s), refreshed on register | `registerPresenceSocket`/`refreshPresenceSocket`; drained by `unregisterPresenceSocket` | +| `presence:device_sockets:{userId}:{deviceId}` | set | socket ids for this specific `(user, device)` pair | `PRESENCE_TTL` (90s) | `registerPresenceSocket`; drained by `unregisterPresenceSocket` | +| `presence:device_sockets_by_device:{deviceId}` | set | socket ids for this device, keyed by **device id alone** (no `userId`) | `PRESENCE_TTL` (90s) | `registerPresenceSocket`; drained by `unregisterPresenceSocket` | +| `presence:socket:{socketId}` | hash | `{ userId, deviceId }` — reverse lookup from a raw socket id | `PRESENCE_TTL` (90s) | `registerPresenceSocket`; deleted by `unregisterPresenceSocket` | + +Why `presence:device_sockets_by_device:{deviceId}` exists as a *separate* +index from `presence:device_sockets:{userId}:{deviceId}`: some callers only +ever have a `deviceId` and no `userId` in hand — the device-revocation +pub/sub listener (`services/deviceRevocation.ts`) and the push-notification +online check are the two cited in the source comments — and this index lets +them resolve connected sockets without an extra DB round trip to look up the +owning user first. It replaced an in-process `Map` that was only visible to +sockets connected to the local gateway node (issue #341 in the source +comments). + +`cleanupStaleSockets` and `reconcileBoot` (called on gateway boot, after the +Redis adapter attaches) walk `presence:sockets:*`/`presence:sockets:{userId}` +and use `io.in(socketId).fetchSockets()` to detect socket ids that are no +longer connected anywhere in the cluster, removing them via the same +`removeStaleSocketMapping` helper used by normal unregistration. + +## The device → user presence model + +Presence is tracked **per device first**, then aggregated to a user: + +- `presence:user:{userId}` is a hash, not a boolean — one field per device + that is currently considered online, keyed by `deviceId`. +- `isOnline(redis, userId)` is simply `HLEN presence:user:{userId} > 0` — a + user is online if **any** device entry exists in that hash, regardless of + which device or how recently (within whatever the hash currently holds). +- A single device going offline (`setOffline`/`markDeviceOffline`) removes + only that device's field; the user stays online as long as at least one + other device's field remains. The user transitions to fully offline only + once the hash becomes empty, at which point the whole key is deleted. + +This is why the module is organized around **device-keyed** operations +(`setOnline(redis, userId, deviceId, ...)`, not `setOnline(redis, userId, +...)`) even though the externally-visible presence state (`isOnline`) is +per-user. + +## Heartbeat timing contract + +Two constants in `services/heartbeat.ts` define the contract, plus one in +`services/presence.ts`: + +| Constant | File | Value | Purpose | +|---|---|---|---| +| `HEARTBEAT_TIMEOUT_MS` | `heartbeat.ts` | **90,000 ms (90s)** | Server-side watchdog: if no `heartbeat` event arrives from a socket within this window, the server force-disconnects it and marks the device offline. | +| `LAST_SEEN_THROTTLE_MS` | `heartbeat.ts` | **30,000 ms (30s)** | Minimum spacing between `devices.lastSeenAt` DB writes for the same device — every heartbeat refreshes Redis, but the Postgres write is throttled to at most once per 30s per device. | +| `PRESENCE_TTL` | `presence.ts` | **90 seconds** | TTL on every per-device/per-socket Redis key (see table above); matches the heartbeat timeout so a device's Redis footprint expires at roughly the same time the watchdog would have fired anyway. | + +The client-side recommended send interval is **30 seconds** (documented in +[`api-websocket-events.md`](api-websocket-events.md#heartbeat)), giving a 3× +margin before the 90-second server timeout fires — a single missed or +delayed heartbeat does not disconnect the socket. + +> The module-level comment block at the top of `presence.ts` describes the +> per-device key's TTL as "60s" in prose. The actual constant in the same +> file is `PRESENCE_TTL = 90`. Treat the constant, not the comment, as +> authoritative — this documentation uses 90s throughout because that is +> what the code enforces. + +### What one `heartbeat` event does, server-side + +`handleHeartbeat(socket, userId, deviceId, redis)` runs, in order: + +1. Clears and does **not** immediately reschedule the existing disconnect + timer for this socket (the reschedule happens at the end, step 4). +2. If Redis is configured: `refreshPresence` (bumps the per-device hash entry + and its TTL) and `refreshPresenceSocket` (re-registers the socket mapping, + refreshing all four socket-related key TTLs from the table above). +3. Throttled DB write: if at least `LAST_SEEN_THROTTLE_MS` (30s) has elapsed + since the last write for this `deviceId` (tracked in an in-process `Map`, + per gateway node), updates `devices.lastSeenAt` and `devices.updatedAt`. + A failed write is swallowed (`catch { /* non-critical */ }`) — a heartbeat + is never allowed to fail the DB write path visibly to the client. +4. Reschedules the 90-second disconnect timer via the stored `schedule()` + closure (set up by `startHeartbeatTimer` at connect time). + +### What a timeout does, server-side + +If no heartbeat (and no other activity — the timer is reset by `schedule()`, +which is only called from `startHeartbeatTimer` and `handleHeartbeat`, so an +otherwise-active socket that never sends `heartbeat` will still time out) +arrives within 90 seconds, the scheduled callback: + +1. Logs the timeout. +2. If Redis is configured: `unregisterPresenceSocket` (remove this socket + from all socket-mapping keys) and, only if that was the device's **last** + remaining socket, `markDeviceOffline` (remove the device's field from + `presence:user:{userId}`). +3. If the device became fully offline and the socket is still marked + connected, emits `user_offline` and `presence_update` (volatile, i.e. + best-effort/no-buffering) to every room the socket was in. +4. Force-disconnects the socket (`socket.disconnect(true)`) if still + connected. + +## Connection lifecycle, step by step + +On a new Socket.IO connection (`index.ts`'s `io.on('connection', ...)`): + +1. `startHeartbeatTimer(socket, userId, deviceId, redis, io)` — arms the + 90-second watchdog described above. +2. `devices.lastSeenAt` is updated immediately (unthrottled — this is a + one-time write at connect, separate from the heartbeat throttle). +3. The socket joins its device room (`device:{deviceId}`), user room, and + every conversation room it's a member of. +4. If Redis is configured: + - `registerPresenceSocket` — writes the socket-mapping keys. + - `cleanupStaleSockets` — prunes any dead socket ids left over from a + previous session for this user. + - `cancelPendingOfflineBroadcast(userId)` — see + [Transition-broadcast behavior](#transition-broadcast-behavior). + - `setOnline(redis, userId, deviceId)` — returns `true` only if the user's + hash was empty before this call, i.e. this connection is what brought + the user online (as opposed to e.g. a second device connecting while + the user was already online via a first device). + - If the user *became* online (and the pending-offline broadcast wasn't + already cancelling a flap — see below) and `presenceVisible` is true, + emits `user_online`/`presence_update` to every conversation room the + user belongs to, and records the transition on co-members' resume + streams for offline replay (`recordPresenceForCoMembers`). + +On disconnect (`socket.on('disconnect', ...)`): + +1. `clearHeartbeatTimer(socket.id)` — cancels the watchdog; a clean + disconnect does not need the 90-second timeout to fire separately. +2. `devices.lastSeenAt` is updated again (unthrottled, same as connect). +3. **Gateway-shutdown short-circuit**: if the process is shutting down + (`SIGTERM`/`SIGINT` received, or Socket.IO's own shutdown reasons), the + handler returns immediately *without* touching presence state at all. The + comment in `index.ts` is explicit about why: "we must NOT wipe presence — + surviving devices re-assert via heartbeat and Redis TTLs." A rolling + restart should not cause every connected user to flicker offline. +4. Otherwise: `unregisterPresenceSocket`, `cleanupStaleSockets`, and — only + if this was the device's last socket — `setOffline`. If that leaves the + user's hash empty (`fullyOffline`), the offline broadcast is scheduled + (not fired immediately — see next section). + +## Multi-device aggregation + +Because `presence:user:{userId}` is a per-device hash, a user with two +connected devices (e.g. phone + laptop) has two fields in that one key. Each +device's connect/heartbeat/disconnect only ever touches its own field (and +its own `presence:user:{userId}:device:{deviceId}` key and own socket-mapping +keys) — devices never read or write each other's entries. + +The aggregation rule is purely "any field present ⇒ online": there is no +weighting, no "most recent device wins," and no per-device online/offline +event exposed externally — only the user-level transition (hash +empty↔non-empty) is ever broadcast. A client reconnecting a second device +while the first is already connected does **not** re-trigger a `user_online` +broadcast, since `setOnline` returns `false` (`wasOnline` was already `true`) +in that case. + +The Postgres fallback (`deriveDevicePresence`) aggregates differently: it +checks whether **any** non-revoked device for the user has `lastSeenAt` +within the last `DEVICE_PRESENCE_WINDOW_MS` (90,000 ms — the same 90-second +figure as `PRESENCE_TTL`, defined as its own local constant in `presence.ts` +rather than shared), and if none qualify, returns the single most recent +`lastSeenAt` across all of that user's non-revoked devices as `lastSeen`. + +## Transition-broadcast behavior + +Going offline is **debounced**, going online is not. This asymmetry exists +specifically to avoid a disconnect-then-immediately-reconnect blip (page +refresh, brief network hiccup, mobile app backgrounding) producing a visible +`user_offline` followed by `user_online` pair to every other member of a +shared conversation. + +- `scheduleOfflineBroadcast(userId, broadcastFn)` — called only once the + caller has already confirmed zero remaining live devices/sockets for the + user (the underlying Redis state removal via `setOffline` has already + happened synchronously; only the *broadcast* is deferred). It sets a timer + for `PRESENCE_OFFLINE_GRACE_MS` (env-configurable, **default 5,000 ms**, + falling back to the default on a missing/invalid/negative value), replacing + any previous pending timer for the same user. The timer is `.unref()`'d so + it never keeps the process alive on its own. +- `cancelPendingOfflineBroadcast(userId)` — called at the **start** of the + connect handler, before the new connection's own online-check runs. If a + reconnect lands within the grace window of a very recent disconnect (same + user, any device — not necessarily the same device that disconnected), the + pending offline broadcast is cancelled and the reconnect produces **no** + `presence_update` at all: nothing was ever announced offline, so there is + nothing to announce online again either (`becameOnline && !cancelledPendingOffline` + gates the online broadcast specifically to suppress this). + +When the offline broadcast does fire (grace window elapsed with no +reconnect), it re-reads conversation memberships fresh (not the stale +membership list captured at disconnect time) and calls +`deriveDevicePresence` to compute the `lastSeen` value to report, rather than +reusing any value cached from the live connection. + +## Privacy gating + +Two per-user boolean columns (`users.presenceVisible`, +`users.lastSeenVisible` — see +[`contracts-database-schema.md`](contracts-database-schema.md#users)) gate +what's ever broadcast or returned, independent of the Redis/DB mechanics +above: + +- `presenceVisible` (default `false`) gates **all** presence broadcasting for + that user — both the connect-time `user_online`/`presence_update` emission + and the disconnect-time scheduled offline broadcast check this flag before + emitting anything. `GET /users/:id/presence` returns `{ online: "unknown" }` + for a user with `presenceVisible: false`, regardless of actual connection + state. +- `lastSeenVisible` (default `false`) gates only whether a `lastSeen` + timestamp is **included** in an otherwise-permitted presence payload — + a user can be visibly online/offline without ever disclosing exactly when + they were last active. + +Both checks read the *current* value of these flags at the moment of +broadcast/query, not at connect time — a user who flips `presenceVisible` off +mid-session stops appearing in future presence broadcasts without needing to +reconnect (see the dedicated visibility-change broadcast in +`routes/users.ts`'s profile-update handler, which emits a synthetic +online/offline transition when `presenceVisible` itself changes). + +## Known caveats, verified against the current code + +- **`presence:user:{userId}` has no TTL of its own.** Every other + presence-related Redis key in the table above carries `PRESENCE_TTL`. + This one does not — it is only ever shrunk/deleted by an explicit + `hdel`/`del` inside `setOffline`, `markDeviceOffline`, or + `cleanupStaleSockets`'s indirect effects. If a gateway process crashes (as + opposed to a graceful shutdown or a normal socket disconnect) between + `setOnline` and any of those cleanup calls, that device's field can persist + in the hash indefinitely — there is no background sweep that expires it by + age. `isOnline()` would then report the user online based on a stale field + with no live socket or process behind it, until some other event + (`reconcileBoot` on the next gateway restart, a later `setOffline` call for + the same device, or a device revocation) happens to clear it. + `reconcileBoot` itself only rebuilds room membership from + `presence:sockets:*` and prunes *socket-mapping* keys for dead sockets — it + does not independently audit or repair the `presence:user:{userId}` + device-presence hash. +- **The per-device TTL (`PRESENCE_TTL`) is the only thing that would + eventually make a truly-dead device's own hash key disappear**, but that + key (`presence:user:{userId}:device:{deviceId}`) expiring does **not** + cascade into removing the corresponding field from + `presence:user:{userId}` — the two are updated independently by the same + call sites, not linked by Redis expiry notifications. A production + deployment that cares about this edge case exactly would need either a + periodic reconciliation pass or Redis keyspace-notification-driven cleanup, + neither of which exists in the current code. +- **`isOnline()` does not re-validate recency.** It is `HLEN > 0`, not a + check that any field's value is within `PRESENCE_TTL` of now — recency + enforcement is delegated entirely to the per-device key's TTL and the + explicit cleanup paths, not re-checked at read time. +- **The in-file header comment block in `presence.ts` is stale** — it + describes a 60-second TTL and a more simplified "add socketId to a set" + model closer to an earlier version of this module. The actual + implementation (device-hash + per-device key + three tiers of + socket-mapping keys, 90-second TTL) supersedes it; this document reflects + the code, not the comment. diff --git a/apps/backend/docs/contracts-database-schema.md b/apps/backend/docs/contracts-database-schema.md new file mode 100644 index 0000000..17e7b87 --- /dev/null +++ b/apps/backend/docs/contracts-database-schema.md @@ -0,0 +1,734 @@ +# Database schema reference + +Every table, column, index, check constraint, and relation defined in +[`db/schema.ts`](../src/db/schema.ts), the single source of truth for the +backend's Postgres schema (Drizzle ORM). Generated directly from the current +`schema.ts`; if you suspect it has drifted from the applied database, diff it +against a fresh `pnpm db:generate` output (see +[`migrations.md`](migrations.md) for that workflow) — this document does not +read `drizzle/` migration files directly. + +## Table of contents + +- [Enums](#enums) +- [Entity-relationship diagram](#entity-relationship-diagram) +- [Core identity & messaging](#core-identity--messaging) + - [`users`](#users) + - [`wallets`](#wallets) + - [`conversations`](#conversations) + - [`conversation_members`](#conversation_members) + - [`files`](#files) + - [`messages`](#messages) + - [`message_envelopes`](#message_envelopes) +- [Devices & E2EE key material](#devices--e2ee-key-material) + - [`devices`](#devices) + - [`device_prekeys`](#device_prekeys) + - [`device_key_history`](#device_key_history) +- [MLS group state](#mls-group-state) + - [`mls_groups`](#mls_groups) + - [`mls_group_members`](#mls_group_members) + - [`mls_commits`](#mls_commits) + - [`mls_welcomes`](#mls_welcomes) + - [`mls_key_packages`](#mls_key_packages) +- [Chain-synced data](#chain-synced-data) + - [`token_transfers`](#token_transfers) + - [`treasury_proposals`](#treasury_proposals) + - [`proposal_votes`](#proposal_votes) +- [Push & audit](#push--audit) + - [`push_subscriptions`](#push_subscriptions) + - [`audit_logs`](#audit_logs) + - [`group_control_events`](#group_control_events) +- [Drizzle `relations()` vs. actual foreign keys](#drizzle-relations-vs-actual-foreign-keys) +- [Notable schema observations](#notable-schema-observations) + +## Enums + +Every `pgEnum` defined in `schema.ts`, in declaration order: + +| Enum (Postgres type name) | Values | Used by | +|---|---|---| +| `conversation_type` | `dm`, `group` | `conversations.type` | +| `content_type` | `text`, `file`, `image`, `video`, `audio`, `system` | **Not currently applied to any column** — see [Notable schema observations](#notable-schema-observations) | +| `file_status` | `pending`, `ready`, `deleted` | `files.status` | +| `e2ee_protocol` | `sealed_box`, `signal`, `mls` | `message_envelopes.protocol` | +| `device_platform` | `web`, `ios`, `android` | `devices.platform` | +| `prekey_type` | `signed`, `one_time` | `device_prekeys.key_type` | +| `treasury_proposal_status` | `active`, `approved`, `rejected`, `executed`, `expired` | `treasury_proposals.status` | +| `proposal_vote_type` | `approve`, `reject` | `proposal_votes.vote` | +| `audit_action` | `device_linked`, `device_revoked`, `logout_everywhere`, `key_bundle_drained`, `auth_failed`, `file_access_denied`, `group_member_added`, `group_member_removed` | `audit_logs.action` | +| `group_control_event_type` | `member_added`, `member_removed`, `member_left`, `commit` | `group_control_events.event_type` | + +## Entity-relationship diagram + +Cardinalities reflect the actual `.references()` foreign keys in `schema.ts` +(not the separate `relations()` blocks — see +[below](#drizzle-relations-vs-actual-foreign-keys)). Split into two diagrams +for legibility; `devices` and `conversations` are the shared anchors between +them. + +**Core identity, messaging, devices, and chain-synced data:** + +```mermaid +erDiagram + users ||--o{ wallets : "owns" + users ||--o{ conversation_members : "joins" + users ||--o{ devices : "owns" + users ||--o{ messages : "sends" + users ||--o{ files : "uploads" + users ||--o{ token_transfers : "sends" + users ||--o{ proposal_votes : "casts" + users ||--o{ device_key_history : "has history for" + users ||--o{ group_control_events : "acts in" + + conversations ||--o{ conversation_members : "has" + conversations ||--o{ messages : "contains" + conversations ||--o{ files : "scopes" + conversations ||--o{ token_transfers : "scopes" + conversations |o--o{ treasury_proposals : "scopes (nullable)" + conversations ||--o| mls_groups : "runs (1:1)" + conversations ||--o{ group_control_events : "logs" + + messages |o--o{ conversation_members : "last-read ref (nullable, SET NULL)" + messages ||--o{ message_envelopes : "fans out to" + messages |o--o| messages : "edits (self-ref, nullable, SET NULL)" + files |o--o{ messages : "attached to (nullable, SET NULL)" + devices |o--o{ messages : "sender device (nullable, SET NULL)" + + devices ||--o{ device_prekeys : "publishes" + devices ||--o{ device_key_history : "rotates" + devices ||--o{ push_subscriptions : "registers" + devices ||--o{ message_envelopes : "receives" + + treasury_proposals ||--o{ proposal_votes : "receives" +``` + +**MLS group state** (all keyed through `mls_groups`, one row per +`conversations` row running MLS): + +```mermaid +erDiagram + conversations ||--o| mls_groups : "1:1" + mls_groups ||--o{ mls_group_members : "has leaves" + mls_groups ||--o{ mls_commits : "has commit log" + mls_groups ||--o{ mls_welcomes : "has pending welcomes" + devices ||--o{ mls_group_members : "holds leaf in" + devices |o--o{ mls_commits : "commits (nullable, SET NULL)" + devices ||--o{ mls_welcomes : "addressed to" + devices ||--o{ mls_key_packages : "publishes" +``` + +--- + +## Core identity & messaging + +### `users` + +| Column | DB name | Type | Nullable | Default | Notes | +|---|---|---|---|---|---| +| `id` | `id` | `uuid` | no | `gen_random_uuid()` | Primary key. | +| `username` | `username` | `text` | yes | — | Unique. | +| `avatarUrl` | `avatar_url` | `text` | yes | — | | +| `presenceVisible` | `presence_visible` | `boolean` | no | `false` | Gates presence broadcast — see [`concepts-presence-heartbeat.md`](concepts-presence-heartbeat.md#privacy-gating). | +| `lastSeenVisible` | `last_seen_visible` | `boolean` | no | `false` | Gates whether a `lastSeen` timestamp is ever disclosed. | +| `createdAt` | `created_at` | `timestamp` | no | `now()` | | +| `updatedAt` | `updated_at` | `timestamp` | no | `now()` | Not auto-updated by a DB trigger — the application must set it explicitly on update. | +| `sendReadReceipts` | `send_read_receipts` | `boolean` | no | `false` | Privacy setting: whether this user sends read receipts to others. | +| `allowDirectMessages` | `allow_direct_messages` | `boolean` | no | `true` | | +| `allowGroupInvites` | `allow_group_invites` | `boolean` | no | `false` | | + +**Indexes/constraints:** primary key on `id`; unique constraint on `username` +(no named index given — Drizzle's `.unique()` shorthand). No composite or +partial indexes. + +**Relations:** referenced by `wallets`, `conversation_members`, `devices`, +`messages.senderId`, `files.uploaderId`, `token_transfers.senderId`, +`proposal_votes`, `device_key_history`, `group_control_events` +(`actorUserId`/`targetUserId`) — all `ON DELETE CASCADE` except +`group_control_events`, which uses `ON DELETE SET NULL` for both user +references (see that table below). + +--- + +### `wallets` + +| Column | DB name | Type | Nullable | Default | Notes | +|---|---|---|---|---|---| +| `id` | `id` | `uuid` | no | `gen_random_uuid()` | PK. | +| `userId` | `user_id` | `uuid` | no | — | FK → `users.id`, `ON DELETE CASCADE`. | +| `address` | `address` | `text` | no | — | Unique. Stellar wallet address. | +| `isPrimary` | `is_primary` | `boolean` | no | `false` | No DB-level constraint enforcing at most one primary wallet per user — enforced by application logic only (see `resolveAccountWallet` in `routes/devices.ts`, which falls back to the first row if none is flagged primary). | +| `createdAt` | `created_at` | `timestamp` | no | `now()` | | + +**Indexes/constraints:** PK on `id`; unique constraint on `address`. + +--- + +### `conversations` + +| Column | DB name | Type | Nullable | Default | Notes | +|---|---|---|---|---|---| +| `id` | `id` | `uuid` | no | `gen_random_uuid()` | PK. | +| `type` | `type` | `conversation_type` enum | no | `'dm'` | `dm` or `group`. | +| `name` | `name` | `text` | yes | — | | +| `avatarUrl` | `avatar_url` | `text` | yes | — | | +| `epoch` | `epoch` | `integer` | no | `0` | Incremented by every group-control event; this row also serializes event ordering, so a concurrent join and leave can never receive the same sequence number — see `group_control_events` below. | +| `createdAt` | `created_at` | `timestamp` | no | `now()` | | + +**Indexes/constraints:** PK on `id`. No other indexes or constraints. + +**Relations:** referenced (all `ON DELETE CASCADE` unless noted) by +`conversation_members`, `messages`, `files`, `token_transfers`, +`treasury_proposals` (`ON DELETE SET NULL`), `mls_groups` (unique — one MLS +group per conversation), `group_control_events`. + +--- + +### `conversation_members` + +| Column | DB name | Type | Nullable | Default | Notes | +|---|---|---|---|---|---| +| `id` | `id` | `uuid` | no | `gen_random_uuid()` | PK. | +| `conversationId` | `conversation_id` | `uuid` | no | — | FK → `conversations.id`, `ON DELETE CASCADE`. | +| `userId` | `user_id` | `uuid` | no | — | FK → `users.id`, `ON DELETE CASCADE`. | +| `lastReadMessageId` | `last_read_message_id` | `uuid` | yes | — | FK → `messages.id`, `ON DELETE SET NULL`. | +| `isMuted` | `is_muted` | `boolean` | no | `false` | | +| `isArchived` | `is_archived` | `boolean` | no | `false` | | +| `joinedAt` | `joined_at` | `timestamp` | no | `now()` | | + +**Indexes/constraints:** PK on `id` only. **No index at all on +`(conversationId, userId)` or `conversationId` alone** — see +[Notable schema observations](#notable-schema-observations) for what this +means in practice. + +--- + +### `files` + +| Column | DB name | Type | Nullable | Default | Notes | +|---|---|---|---|---|---| +| `id` | `id` | `uuid` | no | `gen_random_uuid()` | PK. | +| `uploaderId` | `uploader_id` | `uuid` | no | — | FK → `users.id`, `ON DELETE CASCADE`. | +| `conversationId` | `conversation_id` | `uuid` | no | — | FK → `conversations.id`, `ON DELETE CASCADE`. | +| `status` | `status` | `file_status` enum | no | `'pending'` | `pending` → `ready` → `deleted` lifecycle. | +| `size` | `size` | `integer` | no | — | | +| `mimeType` | `mime_type` | `text` | no | — | | +| `sha256` | `sha256` | `text` | no | — | Not unique — two uploads of identical bytes get two rows and two storage objects. | +| `storageKey` | `storage_key` | `text` | no | — | Unique. Object-store key. | +| `isThumbnail` | `is_thumbnail` | `boolean` | no | `false` | | +| `deletedAt` | `deleted_at` | `timestamp` | yes | — | Set when every referencing message is retracted (soft delete). | +| `hardDeletedAt` | `hard_deleted_at` | `timestamp` | yes | — | Set once the object-store object is actually deleted — see [`concepts-gc-jobs.md`](concepts-gc-jobs.md#file-cleanup--servicesfilecleanupts). | +| `createdAt` | `created_at` | `timestamp` | no | `now()` | | + +**Indexes/constraints:** PK on `id`; unique constraint on `storageKey`. Only +the `fileId` column *on `messages`* (not here) indexes the relationship back +to a file — there is no index on `files.conversationId` or +`files.uploaderId`. + +The `fileKey` (symmetric encryption key for the file's contents) is +deliberately **never** stored in this table or anywhere server-side — it +lives only inside the E2EE envelope ciphertext. See +[`concepts-e2ee-key-management.md`](concepts-e2ee-key-management.md#what-the-server-can-and-cannot-see). + +--- + +### `messages` + +Ordering is by `(createdAt, id)` — `id` (a `uuid`) is only a tiebreaker for +same-millisecond inserts, not a chronological signal. The source comment +notes a monotonic per-conversation sequence counter was considered (and +briefly implemented) but dropped, since the same counter couldn't coherently +serve both per-conversation ordering and the cross-conversation offline-sync +cursor (`/sync`). + +| Column | DB name | Type | Nullable | Default | Notes | +|---|---|---|---|---|---| +| `id` | `id` | `uuid` | no | `gen_random_uuid()` | PK. | +| `conversationId` | `conversation_id` | `uuid` | no | — | FK → `conversations.id`, `ON DELETE CASCADE`. | +| `senderId` | `sender_id` | `uuid` | no | — | FK → `users.id`, `ON DELETE CASCADE`. | +| `senderDeviceId` | `sender_device_id` | `uuid` | yes | — | FK → `devices.id`, `ON DELETE SET NULL`. | +| `contentType` | `content_type` | **plain `text`** | no | `'text'` | **Not** typed as the `content_type` enum despite that enum existing — see [Notable schema observations](#notable-schema-observations). Values used in practice: `text`, `file`, `image`, `video`, `audio`, `system`. | +| `ciphertext` | `ciphertext` | `text` | yes | — | Opaque E2EE ciphertext; `NULL` for `system` rows (enforced by the check constraint below). | +| `systemPayload` | `system_payload` | `jsonb` | yes | — | Typed as `{ userId: string; change: string } \| null`. Structured, server-generated metadata for `content_type = 'system'` rows only. | +| `fileId` | `file_id` | `uuid` | yes | — | FK → `files.id`, `ON DELETE SET NULL`. | +| `editsMessageId` | `edits_message_id` | `uuid` | yes | — | Self-referencing FK → `messages.id`, `ON DELETE SET NULL`. | +| `mlsEpoch` | `mls_epoch` | `bigint` (number mode) | yes | — | MLS epoch whose secrets encrypted `ciphertext`; `NULL` for non-MLS-group messages. | +| `createdAt` | `created_at` | `timestamp` | no | `now()` | | +| `deletedAt` | `deleted_at` | `timestamp` | yes | — | Retraction marker. | + +**Indexes:** + +| Name | Columns | Notes | +|---|---|---| +| `messages_conversation_created_idx` | `(conversation_id, created_at)` | The primary read-path index — paginated history per conversation. | + +**Check constraints:** + +| Name | Expression | Enforces | +|---|---|---| +| `messages_system_payload_check` | `content_type <> 'system' OR (ciphertext IS NULL AND system_payload IS NOT NULL)` | A `system` row always carries structured metadata and never ciphertext; every other row never carries a system payload. The source comment notes this **supersedes** an earlier, looser constraint (`messages_system_payload_only_on_system_type`) that only forbade a payload on non-system rows without requiring a system row to actually have one. | + +--- + +### `message_envelopes` + +One row per **(message, recipient device)** — the actual per-device delivery +unit. + +| Column | DB name | Type | Nullable | Default | Notes | +|---|---|---|---|---|---| +| `id` | `id` | `uuid` | no | `gen_random_uuid()` | PK. | +| `messageId` | `message_id` | `uuid` | no | — | FK → `messages.id`, `ON DELETE CASCADE`. | +| `recipientDeviceId` | `recipient_device_id` | `uuid` | no | — | FK → `devices.id`, `ON DELETE CASCADE`. | +| `recipientUserId` | `recipient_user_id` | `uuid` | no | — | FK → `users.id`, `ON DELETE CASCADE`. | +| `ciphertext` | `ciphertext` | `text` | no | — | Per-device-encrypted ciphertext — opaque to the server. | +| `protocol` | `protocol` | `e2ee_protocol` enum | no | `'sealed_box'` | Which construction produced this envelope's ciphertext. Defaults to `sealed_box` so every pre-existing row is correctly labelled by the migration's backfill. Recorded per envelope (not derived from the device's current capabilities) because a device's capability set changes over time but an envelope written months ago must still decrypt on whatever path it was actually encrypted with. | +| `deliveredAt` | `delivered_at` | `timestamp` | yes | — | | +| `readAt` | `read_at` | `timestamp` | yes | — | | +| `createdAt` | `created_at` | `timestamp` | no | `now()` | | + +**Indexes:** + +| Name | Columns | Notes | +|---|---|---| +| `me_recipient_device_created_idx` | `(recipient_device_id, created_at)` | Per-device delivery/sync queries. | +| `me_message_idx` | `(message_id)` | Fan-out lookups by message. | + +This table is the target of the tightest-retention background job in the +system — see +[`concepts-gc-jobs.md`](concepts-gc-jobs.md#envelope-gc--servicesenvelopegcts). + +--- + +## Devices & E2EE key material + +### `devices` + +The single canonical device registry — see +[`concepts-e2ee-key-management.md`](concepts-e2ee-key-management.md) for the +full lifecycle this table models. + +| Column | DB name | Type | Nullable | Default | Notes | +|---|---|---|---|---|---| +| `id` | `id` | `uuid` | no | `gen_random_uuid()` | PK. | +| `userId` | `user_id` | `uuid` | no | — | FK → `users.id`, `ON DELETE CASCADE`. | +| `identityPublicKey` | `identity_public_key` | `text` | no | — | Base64 Ed25519 public key. Immutable per row — see [`concepts-e2ee-key-management.md`](concepts-e2ee-key-management.md#7-key-rotation-and-the-unwritten-devicekeyhistory-table). | +| `registrationId` | `registration_id` | `integer` | yes | — | X3DH/Signal registration id. | +| `deviceName` | `device_name` | `text` | yes | — | | +| `platform` | `platform` | `device_platform` enum | yes | — | `web`, `ios`, or `android`. | +| `lastSeenAt` | `last_seen_at` | `timestamp` | yes | — | Updated on auth, connect, heartbeat, and disconnect — see [`concepts-presence-heartbeat.md`](concepts-presence-heartbeat.md). | +| `pushEnabled` | `push_enabled` | `boolean` | no | `true` | | +| `revokedAt` | `revoked_at` | `timestamp` | yes | — | `NULL` while active. Row is never deleted on revocation — preserves audit history. | +| `staleFlaggedAt` | `stale_flagged_at` | `timestamp` | yes | — | Set by the device-GC job once revoked past the retention window; informational only. | +| `capabilities` | `capabilities` | `jsonb` | no | `DEFAULT_CAPABILITIES` | Typed as `DeviceCapabilities`; defaults to the sealed-box-only baseline so pre-existing rows (or clients that omit it) negotiate correctly — see [`concepts-protocol-negotiation.md`](concepts-protocol-negotiation.md). | +| `createdAt` | `created_at` | `timestamp` | no | `now()` | | +| `updatedAt` | `updated_at` | `timestamp` | no | `now()` | | + +**Indexes:** + +| Name | Columns | Type | Enforces | +|---|---|---|---| +| `devices_user_identity_idx` | `(user_id, identity_public_key)` | unique | At most one device row per `(user, identity key)` pair — the lookup key both device-creation paths (`POST /auth/verify`, `POST /devices/link/verify`) key off of. | +| `devices_user_id_active_idx` | `(user_id)` **WHERE `revoked_at IS NULL`** | partial, non-unique | Fast "this user's active devices" scans without touching revoked history. | + +--- + +### `device_prekeys` + +Signed and one-time prekeys in a single table, discriminated by `key_type`. + +| Column | DB name | Type | Nullable | Default | Notes | +|---|---|---|---|---|---| +| `id` | `id` | `uuid` | no | `gen_random_uuid()` | PK. | +| `deviceId` | `device_id` | `uuid` | no | — | FK → `devices.id`, `ON DELETE CASCADE`. | +| `keyType` | `key_type` | `prekey_type` enum | no | — | `signed` or `one_time`. | +| `keyId` | `key_id` | `integer` | no | — | Application-assigned, unique per `(device, type)`. | +| `publicKey` | `public_key` | `text` | no | — | Base64 public key. | +| `signature` | `signature` | `text` | yes | — | Base64 Ed25519 signature over `publicKey`, signed by the device's `identityPublicKey`. Required when `keyType = 'signed'` (enforced below). | +| `consumed` | `consumed` | `boolean` | no | `false` | One-time keys only in practice; flips to `true` on bundle-fetch claim rather than deleting the row, so fetch history stays auditable. | +| `createdAt` | `created_at` | `timestamp` | no | `now()` | | + +**Indexes:** + +| Name | Columns | Type | Enforces | +|---|---|---|---| +| `device_prekeys_device_type_keyid_idx` | `(device_id, key_type, key_id)` | unique | No duplicate `keyId` within the same device + key type; also the `ON CONFLICT DO NOTHING` target for idempotent one-time-prekey re-uploads. | +| `device_prekeys_signed_device_idx` | `(device_id)` **WHERE `key_type = 'signed'`** | partial unique | At most one active signed prekey per device — the `ON CONFLICT` target the upload route upserts against. | +| `device_prekeys_one_time_available_idx` | `(device_id)` **WHERE `key_type = 'one_time' AND consumed = false`** | partial, non-unique | Fast bundle-assembly lookup of a device's unconsumed one-time pool. | + +**Check constraints:** + +| Name | Expression | Enforces | +|---|---|---| +| `device_prekeys_signed_requires_signature` | `key_type <> 'signed' OR signature IS NOT NULL` | A signed prekey row must carry a signature; one-time rows are unconstrained on this column. | + +--- + +### `device_key_history` + +Append-only log of identity-key changes per device — intended to let clients +detect silent key swaps. **Verified: no code path in `apps/backend/src` +currently writes to this table** — see +[`concepts-e2ee-key-management.md`](concepts-e2ee-key-management.md#7-key-rotation-and-the-unwritten-devicekeyhistory-table) +for the full explanation. Only `GET /users/:id/key-history` reads it. + +| Column | DB name | Type | Nullable | Default | Notes | +|---|---|---|---|---|---| +| `id` | `id` | `uuid` | no | `gen_random_uuid()` | PK. | +| `deviceId` | `device_id` | `uuid` | no | — | FK → `devices.id`, `ON DELETE CASCADE`. | +| `userId` | `user_id` | `uuid` | no | — | FK → `users.id`, `ON DELETE CASCADE`. | +| `previousKey` | `previous_key` | `text` | yes | — | | +| `newKey` | `new_key` | `text` | no | — | | +| `changeReason` | `change_reason` | `text` | yes | — | | +| `recordedAt` | `recorded_at` | `timestamp` | no | `now()` | | + +**Indexes:** + +| Name | Columns | +|---|---| +| `device_key_history_device_idx` | `(device_id, recorded_at)` | +| `device_key_history_user_idx` | `(user_id, recorded_at)` | + +--- + +## MLS group state + +Group conversations run MLS (RFC 9420). The server is a transport and +ordering service — it never holds group secrets, only the public ledger +every member needs to agree on. See +[`mls-group-membership.md`](mls-group-membership.md) and +[`mls-key-packages.md`](mls-key-packages.md) for the protocol-level detail +this section's tables support. + +### `mls_groups` + +| Column | DB name | Type | Nullable | Default | Notes | +|---|---|---|---|---|---| +| `id` | `id` | `uuid` | no | `gen_random_uuid()` | PK. | +| `conversationId` | `conversation_id` | `uuid` | no | — | FK → `conversations.id`, `ON DELETE CASCADE`. | +| `groupId` | `group_id` | `text` | no | — | Base64 of the MLS group id chosen by the founding client. | +| `cipherSuite` | `cipher_suite` | `integer` | no | — | | +| `currentEpoch` | `current_epoch` | `bigint` (number mode) | no | `0` | Epoch of the most recent accepted commit. | +| `createdAt` | `created_at` | `timestamp` | no | `now()` | | +| `updatedAt` | `updated_at` | `timestamp` | no | `now()` | | + +**Indexes:** + +| Name | Columns | Type | Enforces | +|---|---|---|---| +| `mls_groups_conversation_idx` | `(conversation_id)` | unique | One MLS group per conversation — the conversation *is* the group. | +| `mls_groups_group_id_idx` | `(group_id)` | unique | | + +--- + +### `mls_group_members` + +Device membership recorded as an **epoch interval**, not a boolean, so a +device's decryption window is derivable: it can read epochs in +`[joinedAtEpoch, removedAtEpoch)`. + +| Column | DB name | Type | Nullable | Default | Notes | +|---|---|---|---|---|---| +| `id` | `id` | `uuid` | no | `gen_random_uuid()` | PK. | +| `mlsGroupId` | `mls_group_id` | `uuid` | no | — | FK → `mls_groups.id`, `ON DELETE CASCADE`. | +| `deviceId` | `device_id` | `uuid` | no | — | FK → `devices.id`, `ON DELETE CASCADE`. | +| `userId` | `user_id` | `uuid` | no | — | FK → `users.id`, `ON DELETE CASCADE`. | +| `joinedAtEpoch` | `joined_at_epoch` | `bigint` (number mode) | no | — | | +| `removedAtEpoch` | `removed_at_epoch` | `bigint` (number mode) | yes | — | `NULL` while the device is still in the group. | +| `createdAt` | `created_at` | `timestamp` | no | `now()` | | + +**Indexes:** + +| Name | Columns | Type | Enforces | +|---|---|---|---| +| `mls_group_members_active_idx` | `(mls_group_id, device_id)` **WHERE `removed_at_epoch IS NULL`** | partial unique | A device can hold at most one *active* leaf in a group at a time. A device removed and later re-added gets a **second row** with a later `joinedAtEpoch` rather than reusing the first — this partial unique index is what makes that legal (the old row's `removedAtEpoch` is non-null, so it no longer competes for uniqueness). | +| `mls_group_members_device_idx` | `(device_id)` | non-unique | Cross-group lookups by device. | + +--- + +### `mls_commits` + +Append-only commit log. A device that was offline replays from its last +known epoch to catch up. + +| Column | DB name | Type | Nullable | Default | Notes | +|---|---|---|---|---|---| +| `id` | `id` | `uuid` | no | `gen_random_uuid()` | PK. | +| `mlsGroupId` | `mls_group_id` | `uuid` | no | — | FK → `mls_groups.id`, `ON DELETE CASCADE`. | +| `epoch` | `epoch` | `bigint` (number mode) | no | — | The epoch this commit *produces*. | +| `committerDeviceId` | `committer_device_id` | `uuid` | yes | — | FK → `devices.id`, `ON DELETE SET NULL`. | +| `commit` | `commit` | `text` | no | — | Base64 of the TLS-serialised MLS `Commit` message. | +| `createdAt` | `created_at` | `timestamp` | no | `now()` | | + +**Indexes:** + +| Name | Columns | Type | Enforces | +|---|---|---|---| +| `mls_commits_group_epoch_idx` | `(mls_group_id, epoch)` | unique | Two members committing concurrently cannot both win — the epoch race resolves in the database via this constraint. | + +--- + +### `mls_welcomes` + +Welcome messages addressed to a device being added, held until it comes +online and claims them. + +| Column | DB name | Type | Nullable | Default | Notes | +|---|---|---|---|---|---| +| `id` | `id` | `uuid` | no | `gen_random_uuid()` | PK. | +| `mlsGroupId` | `mls_group_id` | `uuid` | no | — | FK → `mls_groups.id`, `ON DELETE CASCADE`. | +| `deviceId` | `device_id` | `uuid` | no | — | FK → `devices.id`, `ON DELETE CASCADE`. | +| `epoch` | `epoch` | `bigint` (number mode) | no | — | The epoch the recipient joins at. | +| `welcome` | `welcome` | `text` | no | — | Base64 of the TLS-serialised MLS `Welcome` message. | +| `claimedAt` | `claimed_at` | `timestamp` | yes | — | `NULL` while pending. | +| `createdAt` | `created_at` | `timestamp` | no | `now()` | | + +**Indexes:** + +| Name | Columns | Type | Enforces | +|---|---|---|---| +| `mls_welcomes_group_device_epoch_idx` | `(mls_group_id, device_id, epoch)` | unique | | +| `mls_welcomes_pending_idx` | `(device_id)` **WHERE `claimed_at IS NULL`** | partial, non-unique | Fast "what's waiting for this device" lookup on reconnect. | + +--- + +### `mls_key_packages` + +Each device publishes a stock of MLS KeyPackages so any group member can add +it without the device being online. Public key material only. + +| Column | DB name | Type | Nullable | Default | Notes | +|---|---|---|---|---|---| +| `id` | `id` | `uuid` | no | `gen_random_uuid()` | PK. | +| `deviceId` | `device_id` | `uuid` | no | — | FK → `devices.id`, `ON DELETE CASCADE`. | +| `cipherSuite` | `cipher_suite` | `integer` | no | — | IANA MLS cipher suite id; a device may publish packages for several suites. | +| `keyPackage` | `key_package` | `text` | no | — | Base64 of the TLS-serialised MLS `KeyPackage`. Up to ~4 KiB. | +| `packageHash` | `package_hash` | `text` | no | — | SHA-256 (hex) of `keyPackage` — exists specifically because the package itself is too large for a btree unique index, but idempotent re-uploads still need a conflict target. | +| `expiresAt` | `expires_at` | `timestamp` | yes | — | | +| `consumed` | `consumed` | `boolean` | no | `false` | Single-use by spec — flips to `true` inside the same transaction that hands the package out (never deleted, preserving audit history), mirroring `device_prekeys.consumed`. | +| `consumedAt` | `consumed_at` | `timestamp` | yes | — | | +| `createdAt` | `created_at` | `timestamp` | no | `now()` | | + +**Indexes:** + +| Name | Columns | Type | Enforces | +|---|---|---|---| +| `mls_key_packages_device_hash_idx` | `(device_id, package_hash)` | unique | Dedupe target for idempotent re-uploads. | +| `mls_key_packages_available_idx` | `(device_id, cipher_suite, created_at)` **WHERE `consumed = false`** | partial, non-unique | Fast claim of the next available package for a device + cipher suite. | + +--- + +## Chain-synced data + +### `token_transfers` + +One row per Soroban `transfer` event the listener +(`services/stellarListener.ts`) pulls off the chain. + +| Column | DB name | Type | Nullable | Default | Notes | +|---|---|---|---|---|---| +| `id` | `id` | `uuid` | no | `gen_random_uuid()` | PK. | +| `conversationId` | `conversation_id` | `uuid` | no | — | FK → `conversations.id`, `ON DELETE CASCADE`. | +| `senderId` | `sender_id` | `uuid` | no | — | FK → `users.id`, `ON DELETE CASCADE`. | +| `recipientAddress` | `recipient_address` | `text` | no | — | Raw Stellar address — not a FK to `wallets`, since the recipient need not be a Clicked user. | +| `amount` | `amount` | `text` | no | — | Stored as text, not a numeric type — avoids floating-point/precision loss for on-chain integer amounts. | +| `tokenContractId` | `token_contract_id` | `text` | no | — | | +| `txHash` | `tx_hash` | `text` | no | — | Unique — the upsert target that makes reconnects and replayed event pages idempotent. | +| `memo` | `memo` | `text` | yes | — | Correlates an on-chain transfer with a chat message. | +| `createdAt` | `created_at` | `timestamp` | no | `now()` | | + +**Indexes/constraints:** PK on `id`; unique constraint on `txHash`. + +--- + +### `treasury_proposals` + +Synced from `GROUP_TREASURY_CONTRACT_ID` events by the Stellar listener. +Idempotent upsert on `(contractId, proposalId)`. + +| Column | DB name | Type | Nullable | Default | Notes | +|---|---|---|---|---|---| +| `id` | `id` | `uuid` | no | `gen_random_uuid()` | PK. | +| `contractId` | `contract_id` | `text` | no | — | The on-chain `group_treasury` contract address. | +| `proposalId` | `proposal_id` | `text` | no | — | The on-chain proposal id (`u32`, stored as text). | +| `conversationId` | `conversation_id` | `uuid` | yes | — | FK → `conversations.id`, `ON DELETE SET NULL`. | +| `status` | `status` | `treasury_proposal_status` enum | no | `'active'` | Mirrors the on-chain `ProposalStatus` — see [`../../../contracts/docs/contracts-group-treasury-storage.md`](../../../contracts/docs/contracts-group-treasury-storage.md) for the contract-side state machine this tracks. | +| `approvalsCount` | `approvals_count` | `integer` | no | `0` | | +| `rejectionsCount` | `rejections_count` | `integer` | no | `0` | | +| `recipient` | `recipient` | `text` | yes | — | | +| `amount` | `amount` | `text` | yes | — | Text, same precision rationale as `token_transfers.amount`. | +| `token` | `token` | `text` | yes | — | | +| `threshold` | `threshold` | `integer` | no | `3` | Mirrors the on-chain threshold at sync time — a default used only until the real event arrives. | +| `createdAt` | `created_at` | `timestamp` | no | `now()` | | +| `updatedAt` | `updated_at` | `timestamp` | no | `now()` | | + +**Indexes:** + +| Name | Columns | Type | Enforces | +|---|---|---|---| +| `treasury_proposals_contract_proposal_idx` | `(contract_id, proposal_id)` | unique | The upsert target for idempotent chain-event syncing. | + +--- + +### `proposal_votes` + +| Column | DB name | Type | Nullable | Default | Notes | +|---|---|---|---|---|---| +| `id` | `id` | `uuid` | no | `gen_random_uuid()` | PK. | +| `treasuryProposalId` | `treasury_proposal_id` | `uuid` | no | — | FK → `treasury_proposals.id`, `ON DELETE CASCADE`. | +| `userId` | `user_id` | `uuid` | no | — | FK → `users.id`, `ON DELETE CASCADE`. | +| `vote` | `vote` | `proposal_vote_type` enum | no | — | `approve` or `reject`. | +| `signature` | `signature` | `text` | yes | — | | +| `createdAt` | `created_at` | `timestamp` | no | `now()` | | + +**Indexes:** + +| Name | Columns | Type | Enforces | +|---|---|---|---| +| `proposal_votes_proposal_user_unique` | `(treasury_proposal_id, user_id)` | unique | One recorded vote per user per proposal. | + +--- + +## Push & audit + +### `push_subscriptions` + +| Column | DB name | Type | Nullable | Default | Notes | +|---|---|---|---|---|---| +| `id` | `id` | `uuid` | no | `gen_random_uuid()` | PK. | +| `deviceId` | `device_id` | `uuid` | no | — | FK → `devices.id`, `ON DELETE CASCADE`. | +| `endpoint` | `endpoint` | `text` | no | — | Unique. The Web Push endpoint URL. | +| `p256dh` | `p256dh` | `text` | no | — | Web Push encryption key. | +| `auth` | `auth` | `text` | no | — | Web Push auth secret. | +| `lastUsedAt` | `last_used_at` | `timestamp` | yes | — | | +| `disabledAt` | `disabled_at` | `timestamp` | yes | — | Backoff marker set on a transient send failure; cleared by the re-enable pass once the backoff window elapses — see [`concepts-gc-jobs.md`](concepts-gc-jobs.md#push-subscription-backoff-re-enable). | +| `createdAt` | `created_at` | `timestamp` | no | `now()` | | + +**Indexes/constraints:** PK on `id`; unique constraint on `endpoint`. + +--- + +### `audit_logs` + +Append-only record of security-relevant events. Nothing here may contain +message content — rows carry identifiers, counts, and outcomes only; the +writing service (`services/auditLog.ts`) strips anything content-shaped +before it reaches the database. + +| Column | DB name | Type | Nullable | Default | Notes | +|---|---|---|---|---|---| +| `id` | `id` | `uuid` | no | `gen_random_uuid()` | PK. | +| `action` | `action` | `audit_action` enum | no | — | | +| `actorUserId` | `actor_user_id` | `uuid` | yes | — | **Not a foreign key** (deliberately — see below). | +| `actorDeviceId` | `actor_device_id` | `uuid` | yes | — | Not a foreign key. | +| `subjectUserId` | `subject_user_id` | `uuid` | yes | — | Not a foreign key. Nullable because a failed sign-in has no established actor. | +| `targetType` | `target_type` | `text` | yes | — | Free text: `'device'`, `'file'`, `'conversation'`, `'wallet'`, etc. | +| `targetId` | `target_id` | `text` | yes | — | | +| `ipAddress` | `ip_address` | `text` | yes | — | | +| `userAgent` | `user_agent` | `text` | yes | — | | +| `metadata` | `metadata` | `jsonb` | yes | — | Typed as `Record`; sanitised, bounded key/value context — never message content. | +| `createdAt` | `created_at` | `timestamp` | no | `now()` | | + +**Why the actor/subject columns are plain `uuid`, not foreign keys:** an +audit row must record what was true when it was written and stay that way. +A `CASCADE` FK would delete audit history along with the account it +incriminates; `ON DELETE SET NULL` would issue an `UPDATE` that the +append-only trigger (next paragraph) correctly refuses. A responder resolves +these ids (or finds them already gone) at read time instead. + +**Indexes:** + +| Name | Columns | Notes | +|---|---|---| +| `audit_logs_subject_created_idx` | `(subject_user_id, created_at)` | Primary read path: "what happened *to* this account." | +| `audit_logs_actor_created_idx` | `(actor_user_id, created_at)` | "What did this account/device *do*." | +| `audit_logs_action_created_idx` | `(action, created_at)` | "Every failed auth in the last hour," incident-response style queries. | + +**Append-only enforcement is not visible in `schema.ts` at all** — it's a +database-level trigger (`audit_logs_no_mutation`, referenced by the schema +comment as living in the corresponding migration) that rejects `UPDATE`s and +`DELETE`s against this table outright, specifically so the log's integrity +does not depend on the same application credentials an attacker would +already have reached. See +[`../../../docs/security/audit-logging.md`](../../../docs/security/audit-logging.md). + +--- + +### `group_control_events` + +| Column | DB name | Type | Nullable | Default | Notes | +|---|---|---|---|---|---| +| `id` | `id` | `uuid` | no | `gen_random_uuid()` | PK. | +| `conversationId` | `conversation_id` | `uuid` | no | — | FK → `conversations.id`, `ON DELETE CASCADE`. | +| `sequence` | `sequence` | `integer` | no | — | Strictly increasing from 1, gap-free within a conversation. | +| `epoch` | `epoch` | `integer` | no | — | Group epoch after this event was applied — mirrors `conversations.epoch` at the time of writing. | +| `eventType` | `event_type` | `group_control_event_type` enum | no | — | | +| `actorUserId` | `actor_user_id` | `uuid` | yes | — | FK → `users.id`, `ON DELETE SET NULL`. | +| `targetUserId` | `target_user_id` | `uuid` | yes | — | FK → `users.id`, `ON DELETE SET NULL`. | +| `messageId` | `message_id` | `uuid` | yes | — | FK → `messages.id`, `ON DELETE SET NULL`. The system message emitted for this event, when one was created. | +| `payload` | `payload` | `text` | yes | — | Opaque client-supplied MLS commit/welcome material — never parsed server-side. | +| `createdAt` | `created_at` | `timestamp` | no | `now()` | | + +**Indexes:** + +| Name | Columns | Type | Enforces | +|---|---|---|---| +| `group_control_conversation_sequence_idx` | `(conversation_id, sequence)` | unique | Both the gap-free sequencing guarantee and the catch-up read path for a device replaying missed group-control events. | + +--- + +## Drizzle `relations()` vs. actual foreign keys + +`schema.ts` ends with a large block of `relations(table, ({ one, many }) => +...)` calls (`usersRelations`, `conversationsRelations`, +`messagesRelations`, etc.). **These are Drizzle ORM query-time conveniences +for the relational query API (`db.query.x.findMany({ with: {...} })`) — they +are not separate database constraints.** Every cascade/`SET NULL` behavior +documented in the per-table sections above comes from the `.references()` +call on the column itself, not from a `relations()` block. A handful of +`relations()` entries describe a join that has **no corresponding FK on +either side at all** — `users.transfers` (`many(tokenTransfers)`) is backed +by the real `tokenTransfers.senderId` FK, but if a future relation were added +without a matching `.references()`, it would type-check and run queries +successfully while enforcing nothing at the database level. When auditing +referential integrity, read the column definitions in each table's own +`pgTable(...)` block, not the `relations()` section. + +## Notable schema observations + +Verified against the current `schema.ts`, not speculative: + +- **`conversation_members` has no unique or composite index at all** beyond + its own primary key — specifically, nothing prevents two rows from + recording the same `(conversationId, userId)` pair, and there is no index + to accelerate "is this user a member of this conversation" or "list all + members of this conversation" beyond a full-table scan filtered by the FK + columns' implicit (non-indexed) equality. Every other membership-shaped + table in this schema (`mls_group_members`, `proposal_votes`) has an + explicit uniqueness constraint on its natural key; this one does not. +- **`content_type` (the enum) is defined but unused.** `messages.contentType` + is declared as a plain `text('content_type')` column, not + `contentTypeEnum('content_type')`, despite the enum existing immediately + above it in the file with exactly the values used in practice (`text`, + `file`, `image`, `video`, `audio`, `system`). The check constraint + `messages_system_payload_check` relies on string comparison + (`content_type <> 'system'`) rather than an enum comparison, which works + either way, but the column accepts any text value at the database level — + the enum provides no actual validation here. +- **`files.sha256` is not unique.** Two uploads of byte-identical content + produce two separate `files` rows (and two separate object-store objects) + rather than being deduplicated — `storageKey` is the only uniqueness + guarantee on this table. +- **`wallets.isPrimary` has no partial unique index** enforcing "at most one + primary wallet per user" — a user could have zero, one, or several wallets + flagged primary simultaneously as far as the database is concerned; the + single-primary invariant is maintained (if at all) by application code + such as `resolveAccountWallet`. +- **Chain-sourced monetary amounts (`token_transfers.amount`, + `treasury_proposals.amount`/`.token`) are stored as `text`, not a numeric + type** — this is deliberate (avoids floating-point representation of + on-chain integer amounts) but means the database itself performs no + numeric validation or arithmetic on these columns; all math happens in + application code or on-chain. +- **`audit_logs`'s actor/subject columns are plain UUIDs with no foreign + key**, by design — see the dedicated note under + [`audit_logs`](#audit_logs) above. diff --git a/contracts/docs/api-group-treasury.md b/contracts/docs/api-group-treasury.md new file mode 100644 index 0000000..90ebffb --- /dev/null +++ b/contracts/docs/api-group-treasury.md @@ -0,0 +1,804 @@ +# `group_treasury` Contract API Reference + +Every public function exposed by the `group_treasury` Soroban contract: full signature, +parameters, return value, authorization requirements, state mutations, and panic +conditions — verified against the current +[`contracts/contracts/group_treasury/src/lib.rs`](../contracts/group_treasury/src/lib.rs). + +This is the function-by-function reference. For the storage layout behind these functions +(the `DataKey` inventory, the `WithdrawProposal` struct, the proposal state diagram), see +[`contracts-group-treasury-storage.md`](contracts-group-treasury-storage.md). For the +authorization model and known limitations in prose form, see +[`concepts-treasury-multisig-model.md`](concepts-treasury-multisig-model.md). For the +exact panic message catalog across all three contracts, see +[`contracts-errors.md`](contracts-errors.md). + +## Table of contents + +- [Shared auth helper: `require_admin`](#shared-auth-helper-require_admin) +- [Shared vote-validation helper: `require_votable`](#shared-vote-validation-helper-require_votable) +- [Setup & configuration](#setup--configuration) + - [`initialize`](#initialize) + - [`get_threshold`](#get_threshold) +- [Membership](#membership) + - [`add_member`](#add_member) + - [`remove_member`](#remove_member) + - [`is_member`](#is_member) + - [`get_members`](#get_members) +- [Funds movement](#funds-movement) + - [`deposit`](#deposit) + - [`withdraw`](#withdraw) + - [`balance`](#balance) +- [Multisig withdrawal proposals](#multisig-withdrawal-proposals) + - [`propose_withdraw`](#propose_withdraw) + - [`approve_withdraw`](#approve_withdraw) + - [`reject_withdraw`](#reject_withdraw) + - [`get_proposal`](#get_proposal) + - [`list_proposals`](#list_proposals) + - [`get_pending_proposals`](#get_pending_proposals) +- [Worked example: multisig withdrawal flow end-to-end](#worked-example-multisig-withdrawal-flow-end-to-end) + +--- + +## Shared auth helper: `require_admin` + +Not a public contract function, but every admin-gated entrypoint below routes through it, +so its behavior is documented once here instead of repeated per function. + +```rust +fn require_admin(env: &Env) -> Address { + let admin: Address = env + .storage() + .instance() + .get(&DataKey::Admin) + .expect("not initialized"); + admin.require_auth(); + admin +} +``` + +- Loads `DataKey::Admin`. Panics with `"not initialized"` (an `.expect()` panic, not a + `panic!` string literal, but the message is the same string) if the contract has never + been initialized. +- Calls `admin.require_auth()` — the transaction must carry a valid authorization entry for + the stored admin address, or the host aborts the invocation with an authorization failure + (no contract-level message). +- Returns the admin `Address` to the caller for use in event payloads. + +## Shared vote-validation helper: `require_votable` + +Likewise not public, but both `approve_withdraw` and `reject_withdraw` delegate to it, so +their failure conditions are identical and documented once: + +```rust +fn require_votable(env: &Env, voter: &Address, proposal_id: u32) -> WithdrawProposal { + voter.require_auth(); + + if !Self::is_member(env.clone(), voter.clone()) { + panic!("not a member"); + } + + let proposal: WithdrawProposal = env + .storage() + .instance() + .get(&DataKey::Proposal(proposal_id)) + .expect("proposal not found"); + + if proposal.status != ProposalStatus::Active { + panic!("proposal is not pending"); + } + if proposal.status == ProposalStatus::Expired + || env.ledger().timestamp() >= proposal.expires_at + { + panic!("proposal expired"); + } + if env + .storage() + .instance() + .has(&DataKey::Vote(proposal_id, voter.clone())) + { + panic!("already voted"); + } + + proposal +} +``` + +Checks run in this order: `require_auth()` → membership → proposal existence → status → +expiry → duplicate-vote. The `status != Active` check runs before the expiry check, so once +a proposal's `status` field would read as anything other than `Active`, the caller sees +`"proposal is not pending"` rather than `"proposal expired"` even on a proposal whose +deadline has also passed. + +--- + +## Setup & configuration + +### `initialize` + +One-time initialization. Sets the admin, the approval threshold, and sets up the empty +balances map and members list. + +**Full signature:** + +```rust +pub fn initialize(env: Env, admin: Address, _token: Address, threshold: u32) +``` + +**Parameters:** + +- `admin: Address` — the address granted admin rights (`add_member`, `remove_member`, + `withdraw`). +- `_token: Address` — accepted but unused by the function body; the contract tracks + balances per-token in a `Map`, not a single fixed token, so this parameter has no effect + on behavior. Kept for interface/ABI compatibility. +- `threshold: u32` — number of approvals a withdraw proposal needs to pass. Must be `>= 1`. + +**Authorization requirements:** **none.** `initialize` does not call `require_auth()` on +`admin` or any other address — anyone can call it on a freshly deployed, uninitialized +contract and name themselves admin. Deployment and initialization must happen atomically +(see [`api-deployment-invocation.md`](api-deployment-invocation.md)); the `"already +initialized"` guard below is the only protection against a takeover after the fact. + +**State mutations:** + +- `DataKey::Admin` ← `admin` +- `DataKey::Threshold` ← `threshold` +- `DataKey::ProposalCount` ← `0u32` +- `DataKey::Balances` ← empty `Map` +- `DataKey::Members` ← empty `Vec
` + +**Panics:** + +| Condition | Message | +|---|---| +| `DataKey::Admin` already present in instance storage | `"already initialized"` | +| `threshold == 0` | `"threshold must be at least 1"` | + +There is no upper bound and no cross-check against member count — a treasury can be +initialized with a threshold higher than it will ever have members (members don't exist yet +at init time, since they're added afterward via `add_member`). See +[`concepts-treasury-multisig-model.md`](concepts-treasury-multisig-model.md) for the +consequences. + +--- + +### `get_threshold` + +Returns the configured approval threshold. + +**Full signature:** + +```rust +pub fn get_threshold(env: Env) -> u32 +``` + +**Return value:** `u32` — the value stored at `DataKey::Threshold`. + +**Authorization requirements:** none (read-only). + +**Panics:** + +| Condition | Message | +|---|---| +| Contract not initialized | `"not initialized"` | + +This is the only accessor that reliably detects an uninitialized contract — `is_member` and +`get_members` both return harmless defaults (`false` / empty vector) instead of panicking. + +--- + +## Membership + +### `add_member` + +Admin-only: add a new member to the treasury. + +**Full signature:** + +```rust +pub fn add_member(env: Env, member: Address) +``` + +**Parameters:** `member: Address` — the address to add. + +**Authorization requirements:** admin-only, via `require_admin` (`admin.require_auth()`). + +**State mutations:** + +- Appends `member` to `DataKey::Members`. +- Publishes event topic `"member_added"` with a `MemberAddedEvent { member, added_by: admin }`. + +**Panics:** + +| Condition | Message | +|---|---| +| Contract not initialized | `"not initialized"` | +| Caller is not the admin | host auth failure (no contract message) | +| `member` is already in `DataKey::Members` | `"member already exists"` | + +Duplicate detection is a linear scan over the current members vector before insertion — +there is no faster membership index. + +--- + +### `remove_member` + +Admin-only: remove a member from the treasury. + +**Full signature:** + +```rust +pub fn remove_member(env: Env, member: Address) +``` + +**Parameters:** `member: Address` — the address to remove. + +**Authorization requirements:** admin-only, via `require_admin`. + +**State mutations:** + +- Rebuilds `DataKey::Members` without `member` (not swap-remove — relative order of the + remaining members is preserved). +- Publishes event topic `"member_removed"` with a `MemberRemovedEvent { member, removed_by: admin }`. + +**Panics:** + +| Condition | Message | +|---|---| +| Contract not initialized | `"not initialized"` | +| Caller is not the admin | host auth failure | +| `member` is not currently in `DataKey::Members` | `"member not found"` | + +**Important side effect not visible from the signature:** removal touches only +`DataKey::Members`. It does **not** touch any `DataKey::Proposal` or `DataKey::Vote` entry — +a removed member's prior votes stay counted in `approvals`/`rejections` on any proposal they +already voted on, they just can no longer cast *new* votes (`require_votable`'s `is_member` +check uses current membership, not membership at proposal-creation time). Removing a member +also shrinks `member_count`, which changes the blocking-minority arithmetic in +`reject_withdraw` for every currently-open proposal. + +--- + +### `is_member` + +Check whether an address is a current member. + +**Full signature:** + +```rust +pub fn is_member(env: Env, member: Address) -> bool +``` + +**Return value:** `bool` — `true` if `member` is present in `DataKey::Members`. + +**Authorization requirements:** none (read-only). + +**Panics:** none. Uses `unwrap_or_else(|| Vec::new(&env))` when reading `DataKey::Members`, +so it returns `false` on an uninitialized contract rather than panicking — **this function +cannot be used to detect initialization state.** + +--- + +### `get_members` + +Return the full current membership list. + +**Full signature:** + +```rust +pub fn get_members(env: Env) -> Vec
+``` + +**Return value:** `Vec
` — the current contents of `DataKey::Members`, in insertion +order (minus any removed addresses). + +**Authorization requirements:** none (read-only). + +**Panics:** none. Returns an empty vector on an uninitialized contract (same +`unwrap_or_else` pattern as `is_member`). + +--- + +## Funds movement + +### `deposit` + +Transfer `amount` tokens from `from` into the treasury. + +**Full signature:** + +```rust +pub fn deposit(env: Env, from: Address, token: Address, amount: i128) +``` + +**Parameters:** + +- `from: Address` — the depositor; **not required to be a treasury member.** +- `token: Address` — the SEP-41 token contract address being deposited. +- `amount: i128` — amount to deposit, must be `> 0`. + +**Authorization requirements:** `from.require_auth()` only. Deposit is deliberately **not** +member-gated — any address may deposit into the treasury. + +**State mutations:** + +- Calls `TokenClient::new(&env, &token).transfer(&from, &env.current_contract_address(), &amount)` + — an external cross-contract call that moves the actual tokens. +- Increments the treasury's internally tracked balance for `token` in `DataKey::Balances`. +- Publishes event topic `"deposit"` with a `DepositEvent { from, amount }`. + +**Panics:** + +| Condition | Message | +|---|---| +| `amount <= 0` | `"amount must be positive"` | +| `from` did not authorize the call | host auth failure | +| The underlying token's `transfer` call fails (e.g. insufficient balance) | delegated — the token contract's own failure, not a `group_treasury` message | + +The amount check runs **before** `from.require_auth()`, so an invalid amount fails without +ever prompting a signature. + +--- + +### `withdraw` + +Admin-only: transfer `amount` tokens from the treasury to `to`. + +**Full signature:** + +```rust +pub fn withdraw(env: Env, to: Address, token: Address, amount: i128) +``` + +**Parameters:** + +- `to: Address` — withdrawal recipient. +- `token: Address` — the SEP-41 token contract address. +- `amount: i128` — amount to withdraw, must be `> 0`. + +**Authorization requirements:** admin-only, via `require_admin`. **This function does not +consult the proposal/voting system at all** — no threshold, no approvals, no member vote. +It is a separate, parallel path to moving funds alongside `propose_withdraw` → +`approve_withdraw`. See +[`concepts-treasury-multisig-model.md`](concepts-treasury-multisig-model.md#two-independent-authorization-surfaces) +for how the two relate (they don't — reaching `Passed` via the proposal system does not +itself call this function; there is no execute entrypoint today). + +**State mutations:** + +- Calls `TokenClient::new(&env, &token).transfer(&env.current_contract_address(), &to, &amount)`. +- Decrements the treasury's tracked balance for `token`. +- Publishes event topic `"withdraw"` with a `WithdrawEvent { to, amount }`. + +**Panics:** + +| Condition | Message | +|---|---| +| `amount <= 0` | `"amount must be positive"` | +| Contract not initialized | `"not initialized"` | +| Caller is not the admin | host auth failure | +| Tracked balance for `token` is less than `amount` | `"insufficient funds"` | +| The underlying token's `transfer` call fails | delegated | + +Order: amount check → `require_admin` → balance check → token transfer. The `insufficient +funds` check reads the **internally tracked** `Balances` map, not the token contract's +actual on-chain balance — tokens sent directly to the treasury address without going +through `deposit` are invisible to this check. + +--- + +### `balance` + +Returns the token balance currently held by this treasury, as tracked internally. + +**Full signature:** + +```rust +pub fn balance(env: Env, token: Address) -> i128 +``` + +**Parameters:** `token: Address` — the token contract to query. + +**Return value:** `i128` — the tracked balance for `token`, or `0` if the token has never +been deposited or the contract is uninitialized (`unwrap_or(0)`). + +**Authorization requirements:** none (read-only). + +**Panics:** none. A `0` return is ambiguous between "no funds", "unknown token", and +"uninitialized contract" — call `get_threshold` first if that distinction matters. + +--- + +## Multisig withdrawal proposals + +### `propose_withdraw` + +Member-only: create a new withdraw proposal. The proposer's own approval is recorded +automatically at creation time. + +**Full signature:** + +```rust +pub fn propose_withdraw( + env: Env, + proposer: Address, + to: Address, + token: Address, + amount: i128, + ttl_ledgers: u32, +) -> u32 +``` + +**Parameters:** + +- `proposer: Address` — must be a current member. +- `to: Address` — intended withdrawal recipient. +- `token: Address` — token contract to withdraw from. +- `amount: i128` — amount requested, must be `> 0`. +- `ttl_ledgers: u32` — caller-supplied lifetime for the proposal, converted to an absolute + expiry timestamp as `env.ledger().timestamp() + (ttl_ledgers as u64 * 5)` (an + approximate 5-seconds-per-ledger conversion). Despite the parameter name, `expires_at` is + stored and checked as a **timestamp**, not a ledger count. + +**Return value:** `u32` — the new proposal's id (the pre-increment value of +`DataKey::ProposalCount`; ids start at `0`). + +**Authorization requirements:** `proposer.require_auth()`, plus a membership check +(`is_member`). + +**State mutations:** + +- Increments `DataKey::ProposalCount`. +- Writes a new `WithdrawProposal` to `DataKey::Proposal(id)` with `approvals: 1`, + `rejections: 0`, `status: Active`. +- Writes `DataKey::Vote(id, proposer)` ← `true` — **the proposer auto-approves their own + proposal** at creation; they do not call `approve_withdraw` separately (doing so + afterward hits the "already voted" panic). +- Publishes event topic `"proposal_created"` with a `ProposalCreatedEvent { id, proposer, to, token, amount, expires_at }`. + +**Panics:** + +| Condition | Message | +|---|---| +| `proposer` did not authorize | host auth failure | +| `proposer` is not a current member | `"proposer is not a member"` | +| `amount <= 0` | `"amount must be positive"` | +| Tracked balance for `token` is less than `amount` | `"insufficient funds"` | + +Order: `require_auth` → membership → amount → balance. The signing prompt appears **before** +the membership check, so a non-member is asked to sign a transaction that then fails. The +balance check happens at proposal-creation time only — funds are not escrowed, so a proposal +that was fundable when created may not be when a later execution step (if one existed) +attempted to move funds. + +--- + +### `approve_withdraw` + +Member-only: approve a pending withdraw proposal. Transitions the proposal to `Passed` once +the approval count reaches `threshold`. + +**Full signature:** + +```rust +pub fn approve_withdraw(env: Env, approver: Address, proposal_id: u32) +``` + +**Parameters:** + +- `approver: Address` — must be a current member who has not yet voted on this proposal. +- `proposal_id: u32` — target proposal. + +**Authorization requirements:** delegated to [`require_votable`](#shared-vote-validation-helper-require_votable): +`approver.require_auth()` + current membership + proposal must be `Active`, unexpired, and +not already voted on by `approver`. + +**State mutations:** + +- Writes `DataKey::Vote(proposal_id, approver)` ← `true`. +- Increments `proposal.approvals`. +- If `approvals >= threshold` (read from `DataKey::Threshold`): sets `status` to `Passed` + and publishes event topic `"proposal_approved"` with a + `ProposalApprovedEvent { id, approvals, threshold }`. +- Writes the updated proposal back to `DataKey::Proposal(proposal_id)`. +- Always publishes event topic `"withdraw_vote"` with a + `WithdrawVoteCastEvent { id, voter: approver, approve: true }`, regardless of whether + threshold was reached. + +**Panics:** (all via `require_votable`, plus one of its own) + +| Condition | Message | +|---|---| +| `approver` did not authorize | host auth failure | +| `approver` is not a current member | `"not a member"` | +| No proposal with `proposal_id` | `"proposal not found"` | +| Proposal status is not `Active` | `"proposal is not pending"` | +| `env.ledger().timestamp() >= proposal.expires_at` | `"proposal expired"` | +| `approver` already has a recorded vote on this proposal | `"already voted"` | +| `DataKey::Threshold` unset | `"not initialized"` | + +Reaching `Passed` is a **terminal state as far as this contract's own functions are +concerned** — there is no execute entrypoint in `lib.rs` that reads a `Passed` proposal and +moves funds. See +[`concepts-treasury-multisig-model.md`](concepts-treasury-multisig-model.md#withdrawal-authorization-why-proposal--approvals--threshold-not-one-signature). + +--- + +### `reject_withdraw` + +Member-only: reject a pending withdraw proposal. Transitions the proposal to `Rejected` once +rejections reach the blocking minority. + +**Full signature:** + +```rust +pub fn reject_withdraw(env: Env, rejecter: Address, proposal_id: u32) +``` + +**Parameters:** + +- `rejecter: Address` — must be a current member who has not yet voted on this proposal. +- `proposal_id: u32` — target proposal. + +**Authorization requirements:** same as `approve_withdraw`, via `require_votable`. + +**State mutations:** + +- Writes `DataKey::Vote(proposal_id, rejecter)` ← `false`. +- Increments `proposal.rejections`. +- Computes `blocking_minority = member_count.saturating_sub(threshold) + 1` — the smallest + number of rejections that makes it mathematically impossible for the remaining members to + still supply `threshold` approvals. +- If `rejections >= blocking_minority`: sets `status` to `Rejected` and publishes event + topic `"proposal_rejected"` with a `ProposalRejectedEvent { id, rejections }`. +- Writes the updated proposal back to storage. +- Always publishes event topic `"withdraw_vote"` with + `WithdrawVoteCastEvent { id, voter: rejecter, approve: false }`. + +**Panics:** identical trigger set to `approve_withdraw` (shared `require_votable`): + +| Condition | Message | +|---|---| +| `rejecter` did not authorize | host auth failure | +| `rejecter` is not a current member | `"not a member"` | +| No proposal with `proposal_id` | `"proposal not found"` | +| Proposal status is not `Active` | `"proposal is not pending"` | +| `env.ledger().timestamp() >= proposal.expires_at` | `"proposal expired"` | +| `rejecter` already has a recorded vote on this proposal | `"already voted"` | +| `DataKey::Threshold` unset | `"not initialized"` | + +`Rejected` is terminal — no function reads or acts on a `Rejected` proposal afterward, and +there is no separate "cancel" function. + +--- + +### `get_proposal` + +Returns the withdraw proposal with the given id. + +**Full signature:** + +```rust +pub fn get_proposal(env: Env, proposal_id: u32) -> WithdrawProposal +``` + +**Return value:** the full `WithdrawProposal` struct (see +[`contracts-group-treasury-storage.md`](contracts-group-treasury-storage.md#withdrawproposal) +for field-by-field documentation). + +**Authorization requirements:** none (read-only). + +**Panics:** + +| Condition | Message | +|---|---| +| No proposal exists at `proposal_id` | `"proposal not found"` | + +--- + +### `list_proposals` + +Returns every proposal ever created. + +**Full signature:** + +```rust +pub fn list_proposals(env: Env) -> Vec +``` + +**Return value:** `Vec`, built by iterating ids `1..=count` (where `count` +is `DataKey::ProposalCount`) and collecting whichever ones still have a stored entry. + +**Authorization requirements:** none (read-only). + +**Panics:** none — missing entries are skipped via `if let Some(...)`. + +**Known off-by-one:** ids are assigned starting at `0` (see `propose_withdraw`), but this +function iterates `1..=count`. **Proposal `0` — the very first proposal ever created — is +never included in the result**, while the loop's final iteration probes a nonexistent id +past the end (harmlessly skipped by the `Option` guard). Fetch proposal `0` directly via +`get_proposal(0)` if completeness matters. + +--- + +### `get_pending_proposals` + +Returns every proposal whose status is `Active`. + +**Full signature:** + +```rust +pub fn get_pending_proposals(env: Env) -> Vec +``` + +**Return value:** `Vec`, same `1..=count` iteration as `list_proposals`, +filtered to `status == ProposalStatus::Active`. + +**Authorization requirements:** none (read-only). + +**Panics:** none. + +Shares the same off-by-one as `list_proposals` (proposal `0` excluded). It also filters only +on `status`, not on `expires_at` — a proposal whose deadline has passed but which nothing +has yet voted into a terminal state still shows up here, even though `require_votable` would +reject any further vote on it with `"proposal expired"`. + +--- + +## Worked example: multisig withdrawal flow end-to-end + +This walks `propose_withdraw` → `approve_withdraw` × N through to a `Passed` proposal, then +shows the rejection path, using the same setup pattern as +[`src/test.rs`](../contracts/group_treasury/src/test.rs)'s `voting_setup` helper. All +addresses are illustrative; `amount`s are in a token's smallest unit. + +### Setup + +```rust +// threshold = 2, 3 members +let admin = Address::generate(&env); +let token_id = env.register(mock_token::MockToken, ()); +let contract_id = env.register(GroupTreasuryContract, ()); +let client = GroupTreasuryContractClient::new(&env, &contract_id); + +client.initialize(&admin, &token_id, &2); // threshold = 2 + +let member_a = Address::generate(&env); +let member_b = Address::generate(&env); +let member_c = Address::generate(&env); +client.add_member(&member_a); +client.add_member(&member_b); +client.add_member(&member_c); +``` + +`get_threshold()` now returns `2`; `get_members()` returns `[member_a, member_b, member_c]`. + +### 1. Fund the treasury + +```rust +let token = mock_token::MockTokenClient::new(&env, &token_id); +token.mint(&member_a, &500_000); +client.deposit(&member_a, &token_id, &500_000); +``` + +`balance(token_id)` now returns `500_000`. (Deposit is not member-gated, but `member_a` +happens to be a member here.) + +### 2. Create the proposal + +```rust +let recipient = Address::generate(&env); +let id = client.propose_withdraw(&member_a, &recipient, &token_id, &100_000, &100); +// id == 0 +``` + +Immediately after this call: + +- `get_proposal(0).approvals == 1` (the proposer's auto-approval). +- `get_proposal(0).status == ProposalStatus::Active`. +- `DataKey::Vote(0, member_a) == true` — `member_a` cannot vote again on proposal `0`. +- `expires_at = env.ledger().timestamp() + (100 * 5)` — roughly 500 seconds of voting + window from a `ttl_ledgers` of `100`. + +### 3. First additional approval — still below threshold + +```rust +client.approve_withdraw(&member_b, &0); + +let proposal = client.get_proposal(0); +assert_eq!(proposal.approvals, 2); +assert_eq!(proposal.status, ProposalStatus::Active); // not yet Passed... +``` + +Wait — with `threshold = 2` and the proposer's auto-approval already counted as `1`, a +single additional approval (`member_b`) brings `approvals` to `2`, which **meets** +`threshold`. So in this concrete 2-of-3 setup, `status` flips to `Passed` right here, not +after a third vote: + +```rust +assert_eq!(proposal.status, ProposalStatus::Passed); +``` + +A `proposal_approved` event with `{ id: 0, approvals: 2, threshold: 2 }` is published on +this call. `member_c` never needs to vote for this proposal to pass — exactly `threshold` +distinct approvals are required, not unanimity. + +### 4. Attempting to vote after `Passed` + +```rust +client.approve_withdraw(&member_c, &0); +// panics: "proposal is not pending" +``` + +`require_votable` rejects any further vote once `status != Active`, approval or rejection, +from any member — including one who never voted. + +### 5. What `Passed` does — and does not — do + +```rust +let treasury_balance = client.balance(&token_id); +assert_eq!(treasury_balance, 500_000); // unchanged +``` + +No funds moved. There is no function in `lib.rs` that reads a `Passed` proposal and executes +a transfer — `propose_withdraw` / `approve_withdraw` / `reject_withdraw` is a complete, +self-contained voting record, but this contract's own `Passed` state is not wired to its +`withdraw` function. (The separate `proposals` contract calls `is_member` / `balance` / +`withdraw` on this contract after running its *own* independent proposal cycle — see +[`api-proposals.md`](api-proposals.md) — but that is not triggered by this contract reaching +`Passed`.) If funds are to move for this specific proposal today, the admin must call +`withdraw(to, token, amount)` separately and manually, using the proposal's `to`/`token`/ +`amount` fields as a reference — `withdraw` does not read or require a `Passed` proposal to +exist. + +### 6. The rejection path (a second proposal, same treasury) + +```rust +let id2 = client.propose_withdraw(&member_a, &recipient, &token_id, &50_000, &100); +// id2 == 1, approvals == 1 (member_a's auto-approval), status == Active + +client.reject_withdraw(&member_b, &1); +// rejections == 1; blocking_minority = 3 - 2 + 1 = 2; still Active + +client.reject_withdraw(&member_c, &1); +// rejections == 2 == blocking_minority -> status flips to Rejected +``` + +A `proposal_rejected` event with `{ id: 1, rejections: 2 }` is published on the second +rejection. From here, proposal `1` is terminal — no function acts on a `Rejected` proposal +again, and `member_a`'s own approval cannot be withdrawn or converted. + +### Reading back the full picture + +```rust +let all = client.list_proposals(); // note: starts scanning at id 1 — see caveat above +let pending = client.get_pending_proposals(); // empty here, since both proposals are terminal +let p0 = client.get_proposal(0); // Passed +let p1 = client.get_proposal(1); // Rejected +``` + +Because both proposals in this walkthrough are ids `0` and `1`, and `list_proposals`/ +`get_pending_proposals` both start their scan at `1`, `all` in this example contains only +proposal `1` — proposal `0` is silently excluded by the off-by-one documented under +[`list_proposals`](#list_proposals). Always fetch id `0` directly with `get_proposal(0)` if +your integration cares about the very first proposal a treasury ever created. + +--- + +## Related documents + +- [`contracts-group-treasury-storage.md`](contracts-group-treasury-storage.md) — storage key + inventory, `WithdrawProposal` field reference, and the proposal state diagram. +- [`concepts-treasury-multisig-model.md`](concepts-treasury-multisig-model.md) — the + authorization model in prose, including the two independent ways funds can move and the + documented limitations (unreachable threshold, stale proposals, membership changes mid-vote). +- [`contracts-errors.md`](contracts-errors.md) — panic catalog across all three contracts, + with client-presentation guidance per failure. +- [`api-proposals.md`](api-proposals.md) — the separate `proposals` contract, which calls + `is_member` / `balance` / `withdraw` on this contract via cross-contract invocation. +- [`contracts-events.md`](contracts-events.md) — the full event catalog, including every + event this contract publishes. +- [`testing.md`](testing.md) — how the test suite referenced in the worked example above + exercises these functions, including `#[ignore]`d stubs for not-yet-implemented execution + and expiry-finalization paths. diff --git a/docs/README.md b/docs/README.md index 34fa642..b2f8d94 100644 --- a/docs/README.md +++ b/docs/README.md @@ -62,6 +62,7 @@ listener. | [Device capability and E2EE protocol negotiation](../apps/backend/docs/concepts-protocol-negotiation.md) | What a device advertises at registration, how a sender and recipient agree on a protocol, and why the protocol is recorded per envelope. | | [Replay protection and event idempotency](../apps/backend/docs/concepts-replay-protection.md) | The device-scoped `eventId` dedup, the message-level `messageId` idempotency, the TTL, and the `dispatch_ack` duplicate flag. | | [Backend caching reference](../apps/backend/docs/concepts-caching.md) | The conversation-list cache: key, TTL, payload, every invalidation site, and the behaviour when Redis is down. | +| [Presence, heartbeat & connection liveness](../apps/backend/docs/concepts-presence-heartbeat.md) | The Redis presence key structure, the 30s/90s heartbeat timing contract, multi-device aggregation, and the debounced offline-broadcast behaviour. | | [Testing strategy and conventions](testing.md) | The Drizzle mocking pattern, driving socket handlers through the `dispatch` envelope, and the in-process counters that leak between tests. | | [Backend testing guide](../apps/backend/docs/testing.md) | The standard route-test mock set with a copyable skeleton, the Drizzle chain traps, and the state a suite has to reset between tests. | @@ -87,6 +88,7 @@ listener. | [REST schemas](../apps/backend/docs/contracts-rest-schemas.md) | Request and response body schemas shared across the REST surface. | | [Error code and response catalog](../apps/backend/docs/contracts-error-catalog.md) | Every error the backend can return on either transport: the REST status/`error` table, the socket `error` payload shapes, the rate-limit response, and which errors are retryable. | | [WebSocket payloads](../apps/backend/docs/contracts-websocket-payloads.md) | Payload shapes for each WebSocket event, as validated on the wire. | +| [Database schema reference](../apps/backend/docs/contracts-database-schema.md) | Every table, column, index, check constraint, and relation in `db/schema.ts`, with an ER diagram and notable gaps verified against the current schema. | **Encryption and migrations** @@ -94,6 +96,7 @@ listener. | ------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [Database migration workflow](../apps/backend/docs/migrations.md) | The drizzle-kit loop from `schema.ts` to applied SQL, the `drizzle/` layout, and how to resolve the colliding-migration merge conflict that has already broken this history once. | | [E2EE onboarding](../apps/backend/docs/e2ee-onboarding.md) | Device registration and prekey upload flow for first-contact DM setup. | +| [E2EE key management concepts](../apps/backend/docs/concepts-e2ee-key-management.md) | The full backend-side key lifecycle — registration, prekey upload/consumption, revocation — and an explicit statement of what the server can and cannot see. | | [MLS key packages](../apps/backend/docs/mls-key-packages.md) | Key package publication, consumption, and replenishment. | | [MLS group membership](../apps/backend/docs/mls-group-membership.md) | Adding and removing members from an MLS group and the resulting epoch changes. | | [MLS group files](../apps/backend/docs/mls-group-files.md) | How file keys are distributed to an MLS group. | @@ -148,6 +151,7 @@ The Soroban workspace in `contracts/`: `token_transfer`, `group_treasury`, and ` | [Deployment and invocation](../contracts/docs/api-deployment-invocation.md) | Deploying each contract, initialising it, and invoking it from the CLI, including required environment variables. | | [Contract events reference](../contracts/docs/contracts-events.md) | Every published event across all three contracts, its topic and data shape, the state change it signals, and whether the backend listener consumes it. | | [Proposals API](../contracts/docs/api-proposals.md) | The `proposals` contract surface: creating, voting, finalising, and executing. | +| [Group treasury API](../contracts/docs/api-group-treasury.md) | The `group_treasury` contract surface: membership, deposits/withdrawals, and the propose/approve/reject multisig voting flow, with a worked end-to-end example. | | [Token transfer API](../contracts/docs/api-token-transfer.md) | The `token_transfer` contract surface, including the memo field used to correlate a transfer with a chat message. | | [Proposal lifecycle](../contracts/docs/concepts-proposal-lifecycle.md) | Every proposal status, the transitions between them, and what triggers each one. | | [Token transfer flow](../contracts/docs/concepts-token-transfer-flow.md) | The end-to-end flow of an in-chat payment through the contract. |