diff --git a/AGENTS.md b/AGENTS.md index 94079a17a58..a3c59a50e4c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -38,6 +38,7 @@ PRs target `main`, where breaking API changes are allowed. Wire changes stay backwards compatible with *published* drafts/versions; an unpublished `-wip` version changes freely. moq.pro rolls out with relays on the newest and previous release side by side, so those two versions must interoperate as relays. A temporary migration for that is fine when it is clearly marked and easy to delete at the next version migration. Our libraries prefer moq-lite, so moq-transport compatibility only matters against third-party implementations. +Ask before planning or building work that a large live-media platform would not use, such as mesh networking, robotics, AI agents, P2P, or TCP tunnels: it belongs to Kixel (kixel-ai/kixel.ai), which builds on moq rather than waiting on it. Build output may live outside the checkout and gets wiped, so scratch never goes in `target/`. diff --git a/Cargo.toml b/Cargo.toml index 0f29d2430a7..a74ae70474f 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -142,7 +142,7 @@ moq-archive = { version = "0.0.11", path = "rs/moq-archive" } moq-audio = { version = "0.1.10", path = "rs/moq-audio", default-features = false } moq-auth = { version = "0.2.1", path = "rs/moq-auth" } moq-congestion = { version = "0.1.0", path = "rs/moq-congestion" } -# default-features off: moq-auth takes only the endpoint key and its text codec. +# default-features off: dependents opt into only the pieces they use, such as `cpu`. moq-crypto = { version = "0.0.11", path = "rs/moq-crypto", default-features = false } moq-hls = { version = "0.5.11", path = "rs/moq-hls", default-features = false } moq-json = { version = "0.5.8", path = "rs/moq-json" } diff --git a/doc/.vitepress/config.ts b/doc/.vitepress/config.ts index 32636e95552..93b7c766fe9 100644 --- a/doc/.vitepress/config.ts +++ b/doc/.vitepress/config.ts @@ -132,7 +132,6 @@ export default defineConfig({ { text: "vs RTMP/SRT", link: "/concept/use-case/contribution" }, { text: "vs WebRTC", link: "/concept/use-case/conferencing" }, { text: "For AI", link: "/concept/use-case/ai" }, - { text: "For teleoperation", link: "/concept/use-case/teleoperation" }, ], }, ], diff --git a/doc/bin/demo.md b/doc/bin/demo.md index 7bba2a05e16..0a0bd24ceab 100644 --- a/doc/bin/demo.md +++ b/doc/bin/demo.md @@ -46,8 +46,7 @@ What it demonstrates: Code: [`rs/moq-boy`](https://github.com/moq-dev/moq/tree/main/rs/moq-boy) (emulator and publisher) and [`demo/boy`](https://github.com/moq-dev/moq/tree/main/demo/boy) -(the `` element and page). The same -shape covers teleoperation: swap the emulator for a robot. +(the `` element and page). ## CARLA diff --git a/doc/bin/relay/auth.md b/doc/bin/relay/auth.md index fc0a7c5fe8d..c51b5f6f034 100644 --- a/doc/bin/relay/auth.md +++ b/doc/bin/relay/auth.md @@ -373,7 +373,7 @@ and each field it omits takes the server-wide value. Its patterns replace the shared ones, so a rule with empty lists revokes. A certificate no rule names gets `--mtls-*`. The server watches the file: an edit takes effect at each session's next re-check, and one that fails to load keeps the last valid -config and logs an error. Listener and mesh settings apply on restart. +config and logs an error. Listener settings apply on restart. ```toml # moq auth serve /etc/moq/auth.toml @@ -449,46 +449,6 @@ from its URL, `--auth-url https://relay:@auth.example.com/` (percent-enc reserved characters), through the TLS proxy in front of the server. `--listen-secret` also guards a loopback listener. See `moq auth serve --help` for the rest. -### Mesh enrollment - -`--mesh-url URL` also makes the server a mesh coordinator: a device holding an -Ed25519 endpoint key enrolls and receives the relay URLs to dial and a JWT that -publishes `mesh//**` and subscribes `mesh/**`, where `` is the key in -z-base-32 (52 characters, the form pkarr and iroh's DNS discovery use). It is -verified like any other token, so the verifying keys (`--key`, `--key-dir`, -or `--key-set`) must include `--mesh-key`'s public half. - -```bash -moq auth serve --key-dir /etc/moq/keys \ - --mesh-url https://mesh.example.com --mesh-listen 127.0.0.1:4441 \ - --mesh-key /etc/moq/mesh.jwk --mesh-state /var/lib/moq/mesh.json \ - --mesh-relay https://relay.example.com -moq auth mesh create-key --tag robot --reusable --expires 7d -moq auth mesh revoke -``` - -Every request proves the endpoint key by signing a fresh, single-use -challenge bound to `--mesh-url`, so no token is issued for a key the caller -does not hold. A device joins with an enrollment key, with no human in the -loop (single-use unless `--reusable`; revoked or expired keys enroll -nothing). It refreshes its token -two thirds of the way through `--mesh-token-ttl` (default 1h) by proving -the key again, so a revoked device is cut off within one token lifetime. A -revoked device cannot enroll again. Each enrollment and refresh also returns -every member's unique, DNS-safe name, defaulting to the device's hostname. - -The device-facing API on `--mesh-listen` is meant to be public behind a TLS -proxy, since devices refuse plain `http://` beyond loopback. A challenge holds -no state until a proof uses it, so unauthenticated requests cannot crowd out a -device's; the proxy should still rate-limit each client. The owner's -commands go to `--listen` (`moq auth mesh --url`, `unix:///path` for a -socket), which every relay holding `--listen-secret` could reach, so `--mesh-url` -refuses `--listen-public`. With `--listen-secret` set, the owner presents it -too, as the URL's password: `moq auth mesh --url http://owner:SECRET@127.0.0.1:4440`. A hosted coordinator implements -the same JSON contract, documented in -[`moq_auth::mesh`](https://docs.rs/moq-auth/latest/moq_auth/mesh/); it adds -no token claims. - ### Migrating from the relay flags The 0.14 relay's auth flags moved to `moq auth serve`, and the relay points diff --git a/doc/concept/use-case/index.md b/doc/concept/use-case/index.md index aacba7eaac8..c27a345c0a1 100644 --- a/doc/concept/use-case/index.md +++ b/doc/concept/use-case/index.md @@ -15,4 +15,3 @@ of scale come from. | [Contribution](/concept/use-case/contribution) | RTMP, SRT | Pull-based delivery, optional on-demand encoding, and instance-based failover. | | [Conferencing](/concept/use-case/conferencing) | WebRTC | Bidirectional on one session, browser control of the pipeline, no SDP or TURN. | | [AI](/concept/use-case/ai) | WebRTC, WebSockets | Adjustable reliability, on-demand inference, media and prompts on one connection. | -| [Teleoperation](/concept/use-case/teleoperation) | VPN plus video and control flows | Fresh media, replaceable state, and ordered commands share one session. | diff --git a/doc/concept/use-case/teleoperation.md b/doc/concept/use-case/teleoperation.md deleted file mode 100644 index 618970adc52..00000000000 --- a/doc/concept/use-case/teleoperation.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -title: MoQ for Teleoperation -description: Carry video, current control state, and ordered commands on one session ---- - -# MoQ for Teleoperation - -A remote operator needs fresh video from a machine and a responsive control path -back to it. MoQ carries both directions over one session to a relay. The machine -connects outbound, publishes its video and telemetry, and subscribes to the -operator's control broadcast. Additional viewers subscribe through the relay -without another upload from the machine. - -A VPN plus separate video and control UDP flows can provide reachability, but the -application still has to coordinate their delivery and congestion. With MoQ, -tracks sent in the same direction share a connection and its congestion budget, -so the machine can rank telemetry above its video. Priority only orders data -queued at one sender: the operator's commands travel the other way, so they never -compete with the machine's video, and priority does not guarantee a deadline or -make a command safe to execute. Keep each kind of data on a track with the -framing it needs. - -## Choose the delivery shape - -| Data | Shape | What falling behind means | -| --- | --- | --- | -| Steering, throttle, current position, sensor state | JSON snapshots | Catch up to the newest state; intermediate values are superseded. | -| Arm, stop, a parameter change, mission or file records | Ordered JSON stream | Consume every record in order, or report an error. | -| Camera and microphone | Media groups | Join at a decodable boundary and choose the playback buffer. | - -Use [`moq-json`](https://docs.rs/moq-json) snapshot mode for replaceable state. It -starts a group with a full value and carries changes as deltas. A reader can skip -to a newer group and reconstruct the current value. Stream mode puts an ordered -log in one group; it reports a broken or rolled stream instead of silently -presenting a gap as a complete log. In Rust that group grows for the track's -life, but the cache keeps only its newest 32 MiB, so a reader that falls further -behind or joins late fails with `Evicted`; `@moq/json` still bounds it at 8,192 -records. Either way, open a new command track per control session rather than -replaying one forever. These are framing choices, not a universal -'reliable' switch on the session. - -Transport delivery is not command execution. An application that needs to know -whether a command ran sends a command ID and an acknowledgment, defines duplicate -handling, and rejects expired control state at the machine. It also decides who -may control the machine. A subscriber being connected does not grant operator -ownership. - -## Run a non-media example - -From the repository root: - -```bash -nix develop --command cargo run -p moq-json --example teleop -``` - -The [example source](https://github.com/moq-dev/moq/blob/main/rs/moq-json/examples/teleop.rs) -creates `control.json` and `commands.json` tracks in one in-process broadcast. -It writes several control states before reading and gets only the latest one; -it reads all three commands in order. It uses fixed media timestamps and needs -no network service or robot. - -```text -Current control: Control { speed: 0, turn: 0 } -Command: Arm -Command: Stop -Command: Disarm -``` - -To send these tracks over a network, publish the broadcast through a -[`moq-tokio` origin](https://docs.rs/moq-tokio/latest/moq_tokio/origin/index.html) -and connect that origin to a [relay](/bin/relay/). Publish the machine's media -and telemetry on its broadcast and the operator's control on another. Hang's -[catalog](/concept/hang) advertises JSON and binary tracks beside audio and video -when an application needs to discover them together. Give each machine and -operator the [token permissions](/bin/relay/auth) for their own publish and -subscribe paths. diff --git a/doc/lib/rs/moq-auth.md b/doc/lib/rs/moq-auth.md index c06f997afad..ab8c765e11e 100644 --- a/doc/lib/rs/moq-auth.md +++ b/doc/lib/rs/moq-auth.md @@ -18,7 +18,6 @@ accept loop that decides in process. - **Refusals.** `Refusal` carries a `refusal::Reason` (`Refused`, `Forbidden`, or `Expired`) and optional `root` and `tier`. An auth server returns it as JSON with HTTP 403; `Client` returns `Error::Refused(refusal)`. Legacy plain-text 401/403 bodies and unknown reason strings read as `Refused`. Set `root` only from a verified credential, never the dialed path. Embedded relay deciders pass the same shape in `auth::Error::Refused`. - **Client.** `Client` drives a lease against an auth server over `https://`, `unix://`, or loopback `http://`. It revalidates on cadence and retries through an outage until the grant expires, revokes on a 401, 403, or invalid grant, and reports `end` with the session's byte totals. - **Server** (feature `serve`). `serve::Server` is the reference auth server behind `moq auth serve`: JWTs from the query or the moq-transport SETUP token, certificate grants, anonymous rules, tiers, and session caps. `Server::router` mounts it in your own axum service. -- **Mesh enrollment.** `mesh` holds the [enrollment contract](/bin/relay/auth#mesh-enrollment): a device proves its Ed25519 endpoint key against a fresh challenge and receives relay URLs and a scoped token. `mesh::Client` (feature `client`) enrolls with an enrollment key and renews before expiry; `mesh::Coordinator` (feature `serve`) answers it. - **Keys and claims.** Generate HMAC, RSA, ECDSA, or EdDSA keys as JWKs, sign `Claims` (`root`, `publish`, `subscribe`, their optional `Rate` caps, `exp`), and verify them. `Claims::authorize(path)` scopes verified claims to the path a client dialed, exactly as `moq auth serve` does. Roles hold path pattern unions directly. ```bash diff --git a/nix/Cargo.nix b/nix/Cargo.nix index 1aebfbfaf75..adfbc43fd37 100644 --- a/nix/Cargo.nix +++ b/nix/Cargo.nix @@ -17554,10 +17554,7 @@ rec { name = "moq-crypto"; packageId = "moq-crypto"; usesDefaultFeatures = false; - features = [ - "serde" - "cpu" - ]; + features = [ "cpu" ]; } { name = "moq-pattern"; @@ -17649,7 +17646,6 @@ rec { { name = "url"; packageId = "url"; - features = [ "serde" ]; } ]; devDependencies = [ diff --git a/quest/m2/README.md b/quest/m2/README.md index f0efda2e5a5..200f944be7d 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -37,13 +37,13 @@ Decided 2026-11-12 (maintainer, /quest-audit before the cut): offset goes on the wire version after lite-07 (rejected for each: staying in m2). The padding and JS rung disable quests are parked in m5. - ✅ Joined m2, each a small break the cut would otherwise freeze: the mesh - [grant shape](/quest/m2/mesh-grant-shape.md) and - [key budget](/quest/m2/mesh-device-bounds.md), since `moq_auth::mesh` - publishes for the first time at this cut; + grant shape and key budget, since `moq_auth::mesh` + publishes for the first time at this cut (both dropped 2026-11-14, when + `moq_auth::mesh` left moq for Kixel); auth key validation; the RTMPS connect signature; usage's derived settings; and two - API fences: [mesh enrollment](/quest/m2/api-mesh-enrollment.md) and + API fences: mesh enrollment (dropped with it) and the capture selections. None is on the routes chain, which stays the critical path. - ✅ Under the path rule a route keeps one sticky down-only flag beside its diff --git a/quest/m2/api-mesh-enrollment.md b/quest/m2/api-mesh-enrollment.md deleted file mode 100644 index 3c9722bb737..00000000000 --- a/quest/m2/api-mesh-enrollment.md +++ /dev/null @@ -1,24 +0,0 @@ -# [XS] A mesh enrollment reply can grow - -## Goal - -`moq_auth::mesh::Enrollment` and the other reply and request structs the mesh -module exports are `#[non_exhaustive]` with constructors where a caller builds -one, so a later field (a certificate, a revocation hint) is additive. The cut -publishes the module for the first time, and today `Enrollment` is a plain -struct with a public `token: String`. - -## Plan - -Decided 2026-11-12 (maintainer, /quest-audit before the cut): - -- ✅ Fence the structs now (rejected: publishing them open and breaking - later). -- ✅ The mesh [relay node](/quest/m3/mesh/node.md) takes a credential handle - and the application enrolls however it likes (rejected: the coordinator and - `Enrollment` learning to issue certificates; deferring). So this quest adds - no certificate field; it only keeps one possible. - -Sweep `rs/moq-auth/src/mesh/` for every public struct and enum a caller -matches or builds, and fence each the same way. Public API: breaking for a -caller that builds or exhaustively destructures one. Wire: none. diff --git a/quest/m2/mesh-device-bounds.md b/quest/m2/mesh-device-bounds.md deleted file mode 100644 index 2a9976d684f..00000000000 --- a/quest/m2/mesh-device-bounds.md +++ /dev/null @@ -1,12 +0,0 @@ -# [S] Mesh enrollment is bounded and revocation takes effect - -## Goal - -The mesh coordinator (`rs/moq-auth/src/mesh/coordinator.rs`) stops assuming its device store stays small and its revocations wait for token expiry: - -- A reusable enrollment key carries a use budget set when it is created, so the coordinator holds at most what its keys allow. Today a reusable key enrolls any number of endpoints, and each enroll clones the whole store, rewrites the state file and returns every member under one mutex, so a leaked key can mint thousands. -- A revoked device loses access within a bounded time, not only at its token's `exp` (1h by default): either `decide` checks revocation for mesh tokens, or mesh tokens are short-lived with revalidation on. - -## Plan - -Found by the 2026-11-07 invariant audit (auth-7, auth-8). The budget belongs to the key's owner, not a server-wide constant or rate limit, per REVIEW.md's rule against constant caps. diff --git a/quest/m2/mesh-grant-shape.md b/quest/m2/mesh-grant-shape.md deleted file mode 100644 index 12e234606d1..00000000000 --- a/quest/m2/mesh-grant-shape.md +++ /dev/null @@ -1,43 +0,0 @@ -# [S] Mesh grants scope nodes and services apart - -## Goal - -A mesh member's grant publishes `mesh/node//**` and the two service -patterns `mesh/service/*/.server/` and `mesh/service/*/.client/`, in -place of `mesh//**`. A service named after another member can no longer -put `.server/` under its own grant and answer as that member. The -shape lands before the cut publishes `moq_auth::mesh` for the first time, so -no enrolled device ever holds the old grant from a release. - -## Plan - -Split from the [Endpoint API](/quest/m3/mesh/endpoint.md) on 2026-11-12 -(maintainer, /quest-audit before the cut): ✅ the grant shape lands in m2 on -its own (rejected: landing with the Endpoint after the cut, with the -coordinator issuing both shapes for one release). `rs/moq-auth/src/mesh` -exists on `main` and not on `release`, so there is nothing to migrate and no -dual-grant rollout. - -The shape was decided 2026-11-07 (maintainer, /quest-complete, from the -review of [moq-dev/moq#8214](https://github.com/moq-dev/moq/pull/8214)): -members own `mesh/node//**` and each service lives at -`mesh/service/` (rejected: `mesh/.service/`; keeping -`mesh/` and refusing service names shaped like an endpoint ID). A -service name is one path segment, so `*` in the grant covers it. - -- `moq_auth::mesh`'s coordinator issues the new patterns - (`rs/moq-auth/src/mesh/coordinator.rs`), with its docs - (`rs/moq-auth/src/mesh/mod.rs`) and contract tests. -- `doc/bin/relay/auth.md`'s mesh enrollment section names the new grant. -- moq-crypto's and `@moq/crypto`'s sign examples and tests that use - `mesh//agent` paths move to `mesh/node//agent`. -- A hosted coordinator implementing the JSON contract issues the same grant; - the PR's Impact section says so. Applications built on `main` re-enroll. - -Public API: the grant a mesh coordinator issues changes; no Rust or JS -signature does. Wire: none. - -## Related - -- [Mesh enrollment is bounded](/quest/m2/mesh-device-bounds.md) - the other mesh contract change before the cut -- [Mesh enrollment can grow](/quest/m2/api-mesh-enrollment.md) - fences the enrollment reply diff --git a/quest/m3/README.md b/quest/m3/README.md index bf2918d2af2..1e8543595d7 100644 --- a/quest/m3/README.md +++ b/quest/m3/README.md @@ -26,7 +26,7 @@ a new necessary API break brings that decision to the maintainer before proceeding. Decided 2026-11-03 (maintainer, Kixel's quest audit): byte streams -and the [mesh libraries](/quest/m3/mesh/README.md) rank first here, ahead +and the mesh libraries rank first here, ahead of [release execution](/quest/m3/release/README.md). Kixel's daemon builds on them from `main` through a git submodule, so it needs them merged, not released. They stay out of m1 and m2 so the cut does not wait on them; each @@ -34,6 +34,8 @@ starts as soon as its prerequisites (sockets, fastest-reader pacing, outbound federation) land. Superseded 2026-11-12 (maintainer, /quest-audit): both moved to [m1](/quest/m1/README.md), since the cut no longer waits on m1. The mesh libraries came back 2026-11-13, below; byte streams moved to Kixel 2026-11-14. +The mesh libraries left moq for Kixel on 2026-11-14 (maintainer): moq keeps +what a large live-media platform would use. Decided 2026-11-12 (maintainer, /quest-audit before the cut), from a read of every quest against the code and the downstream quest trees: diff --git a/quest/m3/cache-routing/README.md b/quest/m3/cache-routing/README.md index f5eb60e30eb..08e865e68b5 100644 --- a/quest/m3/cache-routing/README.md +++ b/quest/m3/cache-routing/README.md @@ -12,7 +12,7 @@ stays loop-free under stale state, converges once demand and topology stop changing, and sends no per-broadcast message for an idle broadcast. Non-goals: steering clients to the relay that holds a broadcast; decisions -from measured load or RTT ([Link quality](/quest/m5/link-quality.md)); +from measured load or RTT (link quality, planned downstream in Kixel); browsers serving copies they receive; a relay on neither consumer's cheapest route becoming a shared stop; one copy across a multi-hop sparse overlay rather than among direct neighbors; and creating links, since routing diff --git a/quest/m3/cluster-routing/README.md b/quest/m3/cluster-routing/README.md index 1090caaa9f9..a3ee172a111 100644 --- a/quest/m3/cluster-routing/README.md +++ b/quest/m3/cluster-routing/README.md @@ -160,5 +160,4 @@ setup after the capabilities land. - [Routing guide](/quest/m3/routing-guide.md) - concept page and federation setup - [Route wakes](/quest/m4/route-wakes.md) - local work follows affected fronts rather than the whole covered table -- [Link quality](/quest/m5/link-quality.md) - optional measured link cost, distinct from delivery-health failover - [Cache-aware routing](/quest/m3/cache-routing/README.md) - per-broadcast warm routes ranked against these node routes diff --git a/quest/m3/cluster-routing/federation.md b/quest/m3/cluster-routing/federation.md index 316d8daac8c..97be5261ea1 100644 --- a/quest/m3/cluster-routing/federation.md +++ b/quest/m3/cluster-routing/federation.md @@ -64,8 +64,8 @@ session integration (landed in [moq-dev/moq#8023](https://github.com/moq-dev/moq [route epic](/quest/m2/cluster-routes/README.md). Federation is relay-to-relay, so browser route-key proofs, the Playwright pin they wait on, and the mixed-version follow-up (mount takeover) gates the -lite-07 cut, not this. Its consumer, the [mesh node](/quest/m3/mesh/node.md), -is a Rust relay too. Acceptance still brings the js/net route layer for its +lite-07 cut, not this. Its consumer, Kixel's mesh relay node (downstream, in +kixel-ai/kixel.ai), is a Rust relay too. Acceptance still brings the js/net route layer for its Rust/JS interop checks. Rejected: session integration alone, which drops the proven contract the wire rule below relies on. diff --git a/quest/m3/mesh/README.md b/quest/m3/mesh/README.md deleted file mode 100644 index 1b29e02bb60..00000000000 --- a/quest/m3/mesh/README.md +++ /dev/null @@ -1,135 +0,0 @@ -# Mesh libraries - -## Goal - -A Rust application or a relay node enrolls into a mesh served by a relay -(self-hosted with `moq auth`, or a hosted coordinator implementing the same -contract) and reaches any other node by its Ed25519 endpoint ID, through -relays or directly. Nodes that can dial each other (one LAN, a shared VPC or -Docker network, a public address) talk directly over a sparse, deterministic -overlay that forwards for the rest; everything else, and any partition of -the overlay, goes through relays. Content is signed by default, and each node -publishes a `.status` track describing its links. - -This repository ships the libraries. The mesh binary and its UI live outside -it: Kixel builds a `kixel` daemon on these libraries, and any other binary -can do the same. - -Non-goals: a mesh daemon or CLI here (`moq mesh` is not added to moq-cli), -MagicDNS, agent protocols, a graph UI, internet P2P or NAT traversal in m3 -(remote traffic always crosses a relay; [mesh over ICE](/quest/m5/mesh-ice.md) -is parked in m5), TUN devices or IP routing, encryption (parked in -[mesh keys](/quest/m5/mesh-keys.md)), a member list that a compromised -coordinator cannot extend, and Windows. - -## Plan - -Moved from m1 to m3 2026-11-13 (maintainer, /quest-plan): it waits on m2 -work, and no quest requires one in a later milestone, since a blocker ranks -by its own milestone. ✅ m3 (rejected: m2, where the cut would wait on it; -pulling the m2 blocker into m1). It ranks right after m2, when it can start. - -Decided 2026-11-02 with the maintainer (Kixel's planning interview, which -moved moq.pro's hosted mesh and T3 Code lines to Kixel): **moq for -libraries, Kixel for the binary and UI.** This epic keeps the `Endpoint` API, -address hints, the sparse overlay, and the `.status` track, beside the byte -streams, sockets, federation, and `moq_auth::mesh` contract it builds on. The -daemon, MagicDNS, agents and their MCP server, the graph component, robot -builds and tests, and the user guide moved to Kixel. Why: Kixel is the brand -for non-media work on MoQ, and a mesh node is a moq-relay instance, so any -moq library, media included, can point at one. - -Decided 2026-10-25 with the maintainer (planning interview), still in force: - -- Traffic is app-level byte streams and named - tracks, never IP packets. -- The coordinator contract is generic and lives here: enrollment issues a - relay URL and a token, and self-hosters run it with `moq auth`. A hosted - service implements the same contract with its own accounts and billing. -- Direct paths only to addresses a node can already dial: `[cluster.lan]` - mDNS, plus [address hints](/quest/m3/mesh/addr.md) published on mesh - tracks. No ICE, STUN, or hole punching; the P2P - line is separate. -- An endpoint ID is a persistent Ed25519 device key, iroh-style. It is not - the [routing identity](/quest/m2/cluster-routes/README.md), which each - process declares fresh. -- Addressing is announcement: each node publishes its address and status - tracks under `mesh/node//...`, and reaching an ID is subscribing - there. There is no separate directory. Services are not under a member - subtree: each is one socket endpoint at `mesh/service/` (decided - 2026-11-07, see [Endpoint API](/quest/m3/mesh/endpoint.md)). -- Certificates can stand in for the token (decided 2026-11-07, maintainer, - Kixel's planning): a downstream mesh that must admit and evict devices - with no internet issues long-lived certificates from its own signers, and - [certificate credentials](/quest/m3/mesh/cert-credentials.md) checks them - against a trust store it supplies. The coordinator's token stays the - default for `moq auth`. -- The token is the binding: enrollment proves the device key, then the - coordinator issues a grant to publish `mesh/node//**` and subscribe - `mesh/**`, plus publish on its own [socket](/doc/concept/socket.md) - endpoints, `mesh/service/*/.server/` and `mesh/service/*/.client/`, - so it can accept and dial under its own id (amended 2026-11-01, - 2026-11-03 when sockets became endpoints, 2026-11-06 when endpoints split - into server and client roles, and 2026-11-07 when member and service - paths were scoped apart; the [Endpoint API](/quest/m3/mesh/endpoint.md) - changes the shipped `mesh//**` grant). Relays need no new check. Membership is the coordinator's - token only; a Tailnet-Lock analog was declined. -- Protection is per broadcast, with levels that nest: identity (the token - only), authentication (per-frame Ed25519 signatures, the default), and - later encryption (signed plus moq-crypto's e2ee, waiting on - [mesh keys](/quest/m5/mesh-keys.md)). -- Enrollment is automated, with no human in the loop (decided 2026-10-30): a - device presents an enrollment key the owner created and then refreshes by - proving its key. A hosted coordinator may put its own sign-in in front of - minting that key. -- The API is iroh-shaped (not wire-compatible with iroh). -- It builds on [outbound federation](/quest/m3/cluster-routing/federation.md): - a node's local relay dials its relays as `upstream` links and ranks LAN - peers above them. - -Decided 2026-10-31 with the maintainer (moq.pro's mesh planning interview): - -- mDNS alone is not enough. It fails in Docker bridge and overlay networks, - cloud VPCs, across subnets and VLANs, on isolated Wi-Fi, and for a macOS - launchd agent denied Local Network access, so nodes publish - [address hints](/quest/m3/mesh/addr.md). -- Direct links form a deterministic [sparse overlay](/quest/m3/mesh/overlay.md) - computed locally from the member list, with an optional per-node degree - cap. Every node keeps its relay uplink, which carries traffic across an - overlay partition. -- `.status` carries each link's RTT, and an application embedding `Endpoint` - publishes it too ([status track](/quest/m3/mesh/status.md)). - -This epic owns the end-to-end test: three [mesh relay nodes](/quest/m3/mesh/node.md), -two in network namespaces sharing a LAN segment and a third behind a relay -only, enrolled through `moq auth`; an `Endpoint` on each dials the others' -services, taking the LAN path between the first two and the relay path to -the third, and survives a relay restart. - -Decided 2026-11-12 (maintainer, /quest-audit before the cut): - -- ✅ (Superseded 2026-11-13: this epic is back in m3, above.) This epic and byte streams moved to m1 - from the head of m3 (rejected: staying in m3). The cut no longer waits on - m1, and a downstream application builds on them from `main` now. -- ✅ The epic no longer lists federation and the stream epic under - `Required` (rejected: keeping them). An epic's `Required` holds back every - child, so certificate credentials and the Endpoint waited on federation and - on JS streams they do not use. The relay node and the Endpoint each list - their own blocker. -- ✅ The relay node takes a credential handle and the application enrolls - however it likes (rejected: the coordinator issuing certificates; - deferring). `moq_auth::mesh`'s token enrollment stays one way to get a - handle. [Mesh enrollment can grow](/quest/m2/api-mesh-enrollment.md) - fences the reply before the cut publishes it. -- ✅ The contract changes land in m2, because the cut publishes - `moq_auth::mesh` for the first time: the - [grant shape](/quest/m2/mesh-grant-shape.md) and the - [key budget](/quest/m2/mesh-device-bounds.md). -- The JS enrollment client and the guide are parked in m5: the application - they were planned for enrolls with certificates and documents itself. - -## Related - -- [Outbound federation](/quest/m3/cluster-routing/federation.md) - the relay node requires it; the other children do not -- [Mesh over ICE](/quest/m5/mesh-ice.md) - deferred direct paths between mesh nodes on different networks -- [End-to-end encryption](/quest/m5/e2ee/README.md) - the AEAD the encryption level will reuse diff --git a/quest/m3/mesh/addr.md b/quest/m3/mesh/addr.md deleted file mode 100644 index 23b82172d6d..00000000000 --- a/quest/m3/mesh/addr.md +++ /dev/null @@ -1,48 +0,0 @@ -# [M] Address hints - -## Goal - -Mesh nodes that can reach each other directly but cannot discover each other -over mDNS still connect directly: each node publishes its dialable addresses -(interface addresses, its listen port, its certificate fingerprint), signed by -its endpoint key, on a track under `mesh/node//`, and every node dials peers' -hinted addresses alongside `[cluster.lan]`. Two nodes on a network namespace -with multicast disabled (the stand-in for a Docker bridge or a cloud VPC) find -the direct path and prefer it over the relay. - -## Plan - -Decided 2026-10-31 (maintainer, moq.pro mesh planning interview): mDNS fails -in Docker bridge and overlay networks, cloud VPCs, across subnets and VLANs, -on isolated Wi-Fi, and for macOS launchd agents denied Local Network access. -Hints cover the cases where peers can reach each other. Generic: the -coordinator adds nothing, and self-hosted meshes get it. - -Decided 2026-10-31 (maintainer): a mesh node is a relay, and the rule in -`doc/bin/relay/cluster.md` ("never a URL learned from an announcement") does -not apply when the operator opts in to the mesh and everything is signed. -Dial a hinted address only when the mesh is enabled, the hint verifies against -the endpoint key named by its `mesh/node//` path, and the dial pins the hinted -fingerprint; admit the peer only after it proves its own enrollment, as LAN -peers do. Amend that sentence of the docs with the mesh exception. - -- Settle the track name and schema and record them here; keep it separate - from `.status` so a node can read hints without the status track's payload. -- Publish every non-loopback interface address, public ones included: a host - with a public address is directly dialable from anywhere, with no punching. - A private address that collides with an unrelated host on another network - fails the fingerprint pin, so it costs one failed dial, not a wrong peer. -- Bound and back off failed dials per peer, so a fleet of unreachable hints - does not become a dial storm; record the bound chosen. -- The [sparse overlay](/quest/m3/mesh/overlay.md) decides which reachable - peers to keep; until it lands, dial every hinted peer. - -Tests: namespaces with multicast off take the direct path; a forged hint -(signed by another key) is never dialed; a colliding private address fails the -pin and falls back to the relay; a peer that stops answering is backed off. -Update `doc/bin/relay/cluster.md` and the [guide](/quest/m5/mesh-guide.md) -if it exists. - -## Required - -- [Mesh relay node](/quest/m3/mesh/node.md) - the enrolled relay node that publishes and dials the hints diff --git a/quest/m3/mesh/cert-credentials.md b/quest/m3/mesh/cert-credentials.md deleted file mode 100644 index d2d85838372..00000000000 --- a/quest/m3/mesh/cert-credentials.md +++ /dev/null @@ -1,67 +0,0 @@ -# [M] Certificate credentials - -## Goal - -A client can present an X.509 certificate chain over its Ed25519 key in place -of a JWT: in the `jwt` query, as a moq-transport SETUP token, and on every -transport, browsers included. The credential is the chain plus a proof -signature over the relay's host name, the dialed path, and the current time, -so a captured chain alone admits nothing. `moq-auth` parses and verifies it -against a trust store the embedding application supplies and swaps at -runtime: its anchors, and a check for whether a certificate is revoked. A -relay or node using that store refuses a chain that fails it, and closes -sessions it admitted earlier once their certificate is revoked or the anchors -change, at its next re-check. The application turns a verified chain into a -grant, the way in-process admissions already decide. `moq_auth::mesh::Client` -can present such a credential, and `@moq/auth` builds one from a -non-extractable WebCrypto Ed25519 key. - -## Plan - -Decided 2026-11-07 (maintainer, Kixel's `/quest-plan`): a mesh whose devices -must admit and evict each other with no internet needs credentials that last -longer than a coordinator token and can be checked offline, so Kixel moved its -devices from coordinator-minted JWTs to 30-day certificates from the mesh's -own signers. Presenting the certificate as the token, rather than over mTLS, -gives one path that also crosses relays that accept only their own cluster's -client certificates, WebSocket fallbacks, and browsers, which cannot present -client certificates over WebTransport. The signer log, revocation set, and -their sync stay in Kixel; this quest is the generic format and hook. - -- Verify chains with webpki. Name constraints work only on dNSName (webpki - rejects URI constraints and lacks directoryName), so document scopes - carried as DNS names. -- The trust store is a handle the application updates, not a callback per - connection. Re-check live sessions on the relay's existing `--revalidate` - cadence. -- The time-window proof comes first. Its replay window (a relay that saw a - credential could reuse it at the same host within the skew) closes with the - [challenge proof](/quest/m4/cert-challenge.md). Settle the skew and the - signed encoding before coding, and record them in `doc/bin/relay/auth.md`. -- Browsers: WebCrypto Ed25519 keys sign in current Chrome, Firefox, and - Safari; confirm it before relying on it, and fail loud where it is missing. -- Overlaps [moq-dev/moq#8149](https://github.com/moq-dev/moq/pull/8149)'s - bounded revocation for `moq auth`'s own mesh coordinator, which keeps - JWTs; this quest does not change that coordinator's tokens. - -Tests: a valid chain admits; a revoked or unanchored one, a proof for another -host or path, and a stale proof are refused; a session admitted before its -certificate is revoked closes on the next re-check; swapping the anchors -applies without a restart; a JS client presents a credential from a WebCrypto -key and a Rust relay admits it. - -Public API: new credential types in `moq-auth` and `@moq/auth`, and a trust -store handle on the relay. Wire: a new credential form wherever a JWT is -accepted. - -Added 2026-11-07 (maintainer, /quest-plan from the API sweep of open PRs and remaining quests), before any code: pin the credential encoding, since it is permanent once shipped. - -- A self-describing, versioned encoding (for example a `moqc1.` prefix, or a distinct token type) in a slot named by role. If the query key stays `jwt` for compatibility, the prefix still makes it dispatchable without sniffing, and moq-transport uses its AUTHORIZATION TOKEN type field. -- One proof structure whose signed payload has a `nonce | timestamp` field, so [the challenge](/quest/m4/cert-challenge.md) reuses it instead of adding a second format. -- Revocation is data in the trust-store handle (a revoked SPKI-hash set, or a set of serials qualified by their issuer, which the application replaces), not a per-connection predicate. A bare serial is never enough, since two trusted issuers can reuse one; a test revokes one issuer's serial and still admits the other issuer's certificate with the same serial. -- Chains are 2 to 4 KB in base64, close to common 8 KB URL limits; cache verified chains by fingerprint, and say in the PR how a long chain is carried. -- The shared token corpus planned in [moq-dev/moq#8151](https://github.com/moq-dev/moq/pull/8151) includes this credential form. - -## Related - -- [Mesh relay node](/quest/m3/mesh/node.md) - checks these credentials for its LAN peers, and only uses them, so this lands first diff --git a/quest/m3/mesh/endpoint.md b/quest/m3/mesh/endpoint.md deleted file mode 100644 index d22b6922f29..00000000000 --- a/quest/m3/mesh/endpoint.md +++ /dev/null @@ -1,63 +0,0 @@ -# [M] Endpoint API - -## Goal - -A Rust application builds an `Endpoint` from its secret key and a mesh -enrollment, then `connect(endpoint_id, service)` opens a byte stream to that -endpoint's named service and `accept()` yields incoming ones with the remote -endpoint ID verified. The surface mirrors iroh's (`Endpoint`, `id()`, -`connect`, `accept`, bi streams, `remote_id()`) so an iroh application ports -with small changes, without iroh's wire or discovery. - -## Plan - -Decided 2026-10-25 (maintainer): iroh-shaped, not drop-in compatible; tickets, -`EndpointAddr`, and `Router` are out unless porting proves they are needed. - -- A service is the iroh ALPN analog: a socket endpoint per service name - (below), while each node's address and status tracks stay under its - `mesh/node//`. -- Decided 2026-11-01 (maintainer, socket planning interview), and updated - 2026-11-03 when sockets became endpoints and 2026-11-04 when the segment - became `.socket` with the protocol in the session name: an `Endpoint` wraps - a [socket](/doc/concept/socket.md) endpoint, which since 2026-11-06 is a - server at `.server/`, a client at `.client/`, or both. - `connect` dials with the caller's own endpoint ID, and `accept` reads the - remote ID from the dial, which the socket's answer check proves and the - grant pins (publish `.server/` and `.client/` only). -- Streams are byte streams at the protection - level the service declares: identity reads `moq-net` directly, and - authentication, the mesh default, reads through `moq_crypto::sign` against the - `PublicKey` parsed from the peer's endpoint id, which names the endpoint - broadcast it subscribes to. -- The secret key comes from `moq_crypto::sign::SecretKey::load_or_generate`. -- A mesh daemon (Kixel's, outside this repository, decided 2026-11-02) is - built on this API, so a process either embeds an `Endpoint` or talks to a - local [mesh relay node](/quest/m3/mesh/node.md). -- Write a short porting note into `doc/` beside the Endpoint's own page; the - mesh [guide](/quest/m5/mesh-guide.md) is parked. - -Decided 2026-11-07 (maintainer, /quest-plan from the API sweep of open PRs and remaining quests): one socket endpoint per service, with a grant to publish on it under the member's own id, so `moq_net::socket` ships unchanged and needs no service field. - -✅ Decided 2026-11-07 (maintainer, /quest-complete): scope both -namespaces. Members own `mesh/node//**` (was `mesh//**`) and each -service lives at `mesh/service/` (was `mesh/`), so the -grant adds publish `mesh/service/*/.server/` and -`mesh/service/*/.client/` (rejected: `mesh/.service/`; -keeping `mesh/` and refusing service names shaped like an endpoint -ID). Review of [moq-dev/moq#8214](https://github.com/moq-dev/moq/pull/8214) -found that each member publishing `mesh//**` lets a service named after -member A put `.server/B` under A's own grant, so A could answer as B; the -`node` and `service` segments keep the two apart. A service name is still -one path segment, validated when the `Endpoint` is built, so `*` in the -grant covers it (`video/h264` would fall outside). - -The grant change itself is its own quest, in m2, so it lands before the cut -publishes `moq_auth::mesh` (split 2026-11-12, maintainer, /quest-audit): -[mesh grants scope nodes and services apart](/quest/m2/mesh-grant-shape.md). -This quest builds the `Endpoint` on the new grant, and the dual-grant rollout -question it carried is gone, since no release ever issued the old shape. - -## Required - -- [Mesh grant shape](/quest/m2/mesh-grant-shape.md) - the grant that lets a member publish its service endpoints diff --git a/quest/m3/mesh/node.md b/quest/m3/mesh/node.md deleted file mode 100644 index 3247c82548b..00000000000 --- a/quest/m3/mesh/node.md +++ /dev/null @@ -1,51 +0,0 @@ -# [M] Mesh relay node - -## Goal - -A relay node (the `moq-relay` library, and its binary through a `[mesh]` -config section) joins a mesh: it enrolls with an enrollment key through -`moq_auth::mesh::Client`, signing with its endpoint key, renews before the -token expires, dials the mesh's relays as `upstream` links, and admits a -`[cluster.lan]` peer only after the peer proves its own enrollment. A restart -keeps the endpoint key and enrollment, so a node never re-enrolls on its -own. A mesh daemon outside this repository (Kixel's) embeds this rather than -reimplementing it. - -## Plan - -Decided 2026-11-02 (maintainer, Kixel planning): moq keeps the libraries and -Kixel the binary. This is the library half of the deleted daemon quest; its -CLI verbs, service install, and UI moved to Kixel. Carried over from it: - -- Mesh relays are `upstream` links; LAN peers come from `[cluster.lan]` mDNS - and rank above them, with no extra shared secret to configure. -- Enroll and renew with `moq_auth::mesh::Client`; the contract is in - `doc/bin/relay/auth.md#mesh-enrollment`. Pass the hostname as the default - name. -- Nothing is served unless the embedding application names it. -- A LAN peer proves its enrollment with the coordinator token, or with a - [certificate credential](/quest/m3/mesh/cert-credentials.md) when the - embedding application supplies a trust store (decided 2026-11-07, - maintainer). - -Decided 2026-11-12 (maintainer, /quest-audit before the cut): ✅ the node -takes a credential handle, and the embedding application decides how it -enrolls (rejected: `moq_auth::mesh`'s coordinator and `Enrollment` learning -to issue certificates, which puts one consumer's contract in a public crate; -deferring). The handle yields what the node presents to the mesh's relays -and to LAN peers and says when to renew; `moq_auth::mesh::Client`'s token -enrollment is the implementation this repository ships and the `[mesh]` -config section uses, and a -[certificate credential](/quest/m3/mesh/cert-credentials.md) is another. The -Goal's "enrolls with an enrollment key" describes that default, not the only -path. Keep the handle a small trait or enum in moq-relay's mesh module, private -to the crate until an embedder outside this repository needs to implement it. - -Tests in network namespaces: two relay nodes enroll through `moq auth`, take -the LAN link when they share a segment and the relay path otherwise, refuse -a LAN peer without an enrollment, and survive a relay restart. Update -`doc/bin/relay/cluster.md` inline. - -## Required - -- [Outbound federation](/quest/m3/cluster-routing/federation.md) - upstream links and LAN preference diff --git a/quest/m3/mesh/overlay.md b/quest/m3/mesh/overlay.md deleted file mode 100644 index 02f41c9ed23..00000000000 --- a/quest/m3/mesh/overlay.md +++ /dev/null @@ -1,106 +0,0 @@ -# [L] Sparse overlay - -## Goal - -A mesh relay node keeps direct links to about log N of its directly reachable peers -instead of all of them, chosen deterministically so every node computes the -same overlay from the member list, and forwards for the rest. Any two nodes in -a reachable group reach each other through the overlay in about log N hops, -a departed node's links heal, an optional per-node degree cap bounds a -constrained robot's links, and traffic crosses the relay uplink, which every -relay node keeps, when the overlay partitions. Every node is a core unless -its application marks it an edge: an edge links to the cores it can reach -and its links cost more, so traffic between edges crosses a core while one -is reachable, and edges ring among themselves when none is. An application -embedding `Endpoint` reaches the overlay through a relay node. - -## Plan - -Decided 2026-10-31 (maintainer, moq.pro mesh planning interview): -deterministic and local, not coordinator-assigned, so self-hosted meshes get -it; the coordinator is only the member list (announcements under `mesh/**`) -and the fallback path. - -Decided 2026-11-09 (maintainer, /quest-plan from Kixel's audit of its fleet -plan): ✅ the ring stays the link choice, with no measured cost and no exit -designation (rejected: links chosen by measured proximity; a coordinator -computing the graph), on the working assumption that a peer is either on the -same network or out of reach. Two cases the ring as written does not cover -join this quest: - -- Partial reach. On radios A reaches B and B reaches C while A cannot reach - C, so each node computes its ring over a different set of peers. Test it: - in namespaces with filtered links (a line and a grid), every pair with a - path connects without crossing the relay, a flapping link reroutes without - a loop, and the links and hops are reported. A node with no uplink has no - fallback, so report whether the overlay stays connected whenever the reach - graph is, and record what such a node can know about being cut off. -- ~~Nodes that do not forward.~~ Superseded 2026-11-10 by the roles below: - Kixel dropped its `--leaf` flag, and every node forwards. - -Decided 2026-11-10 (maintainer, /quest-plan on lite-07 routing, asked how a -beefier node should attract traffic once Kixel's non-forwarding leaf is -gone): ✅ **two roles, core and edge, and both link shaping and link -pricing.** In the maintainer's words: "Kinda like how moq.pro wants to have -edges and cores. Edges should prioritize connecting to cores, but if there's -no cores, do a ring of edges? I just want this to work by default, which I -guess means everything is a core unless stated otherwise." Rejected: pricing -alone; shaping alone; ranking links to cheaper nodes by priority, whose -outcome under the feasibility rule depends on which route arrived first. -That reason was written under Babel. Under the path rule -([routes epic](/quest/m2/cluster-routes/README.md) Plan item 27) a -relay takes the cheapest route whose path does not name it, whatever arrived -first. - -- Every node is a core unless the embedding application says it is an edge. - Every node forwards; the role only changes which links it keeps and what - they cost. -- Shaping: cores ring among themselves as below. An edge links to the cores - it can reach, and rings with other edges only when it reaches none, so the - mesh works with no core at all. Settle in the PR what an edge that reaches - a core does for an edge that reaches none (partial reach); recommended: it - accepts that edge's link. -- Pricing: a node declares a cost, 1 for a core and more for an edge, and a - link costs the product of its two ends' costs. A cost added per hop cannot - move traffic onto a core: a direct link is always cheaper than the same - link plus another. With a core at 1 and an edge at 3, edge to edge costs 9 - and edge to core to edge costs 6, whichever route arrived first. This is - local pricing of each peer's declared cost - ([link costs](/doc/bin/relay/cluster.md#link-costs)) and needs no wire - change; confirm that, and the edge's default cost, in the PR. -- Recommended, to confirm in the PR: an edge marks its links to cores - `upstream`, as moq.pro's edges do, so it never carries one core's routes to - another. - -- Ring: order the reachable peers (found through mDNS or - [address hints](/quest/m3/mesh/addr.md)) by a hash of their endpoint ID, and - link to the first peer at or after distance 2^k for each k. Accept incoming - links from peers whose ring picks this node. A target that cannot be dialed - is replaced by the next peer on the ring. -- Degree cap: an optional per-node setting for the maximum direct links, - unset by default; the ring alone decides when it is unset. Over the cap, - drop the longest fingers first. Small meshes come out close to full mesh on - their own. -- Forwarding and route choice are [cluster routing](/quest/m3/cluster-routing/README.md)'s: - overlay links join relay nodes as LAN-cost links, and relay - uplinks stay `upstream` links costed above them. Nothing routes outside the - shared priority/cost rules. -- Membership churn recomputes the ring; hold a change briefly so a flapping - node does not churn every link, and record the hold chosen. - -Tests in network namespaces, on one segment beside the partial-reach layouts -above: each node's direct link count -follows the ring (and never exceeds the cap when set); every pair connects -without crossing the relay; killing a node heals the overlay; splitting the -segment sends cross-half traffic through the relay and healing restores the -direct path. Report the link counts and hop counts the test measures. -Update `doc/bin/relay/cluster.md` and the [guide](/quest/m5/mesh-guide.md) if it exists. - -## Required - -- [Mesh relay node](/quest/m3/mesh/node.md) - the enrolled relay node that dials the overlay - -## Related - -- [Address hints](/quest/m3/mesh/addr.md) - a second source of reachable peers -- [Link quality](/quest/m5/link-quality.md) - measured link cost a fading overlay link could use diff --git a/quest/m3/mesh/status.md b/quest/m3/mesh/status.md deleted file mode 100644 index e0db44aff97..00000000000 --- a/quest/m3/mesh/status.md +++ /dev/null @@ -1,35 +0,0 @@ -# [M] Status track - -## Goal - -Each mesh node, a relay node with the mesh enabled or an application -embedding `Endpoint`, publishes `mesh/node//.status`, signed: its peers, its -links (each with its smoothed and minimum RTT, and for a relay uplink the -relay's identity), its routes, and its services. A subscriber can draw the -whole mesh from these tracks alone. - -## Plan - -Decided 2026-10-25 (maintainer): the data is a track on the node's -member subtree (`mesh/node/` since 2026-11-07), not an -HTTP API. - -Decided 2026-10-31 (maintainer, moq.pro mesh planning interview): per-link -RTTs, and embedded apps publish `.status` so they appear as nodes. RTT comes -from the session's existing connection stats; coalesce updates to a few a -minute. - -Decided 2026-11-02 (maintainer, Kixel planning): the graph component, the -localhost page, and the hosted dashboard moved to Kixel, which draws them -from this track. Keep the schema generic and documented, with no extension -point for a hosting service's own nodes. Settle the schema and record it in -`doc/` with this quest. - -Tests: two nodes in network namespaces publish `.status` naming each other's -link with an RTT, a relay uplink names its relay, and a frame signed by -another key is rejected by a subscriber. - -## Required - -- [Endpoint API](/quest/m3/mesh/endpoint.md) - the application node whose links the track describes -- [Mesh relay node](/quest/m3/mesh/node.md) - the relay node whose links the track describes diff --git a/quest/m4/cert-challenge.md b/quest/m4/cert-challenge.md deleted file mode 100644 index 99bdf521aa4..00000000000 --- a/quest/m4/cert-challenge.md +++ /dev/null @@ -1,28 +0,0 @@ -# [S] Challenge proof for certificate credentials - -## Goal - -A relay can ask a client presenting a [certificate credential](/quest/m3/mesh/cert-credentials.md) -to sign a fresh nonce instead of a timestamp, so a credential replayed within -the time window admits nothing. Clients that support it answer the -challenge; a relay configured to require it refuses time-window proofs. - -## Plan - -Decided 2026-11-07 (maintainer, Kixel's `/quest-plan`: "plan the challenge, -but do the simple thing first"): certificate credentials ship with a -time-window proof, and this closes its replay window later. Settle where the -nonce rides (SETUP parameters on moq-transport and moq-lite, and what a -WebSocket or HTTP admission does) and what one extra round trip costs a -connect, measured, before coding. - -Tests: a replayed time-window proof is refused when the relay requires the -challenge; a client signs the nonce and is admitted; a mismatched nonce is -refused. - -Public API: a challenge option on the relay and the clients. Wire: a nonce -exchanged during setup. - -## Required - -- [Certificate credentials](/quest/m3/mesh/cert-credentials.md) - the credential this hardens diff --git a/quest/m4/crypto-key-load.md b/quest/m4/crypto-key-load.md deleted file mode 100644 index 7237610d6f9..00000000000 --- a/quest/m4/crypto-key-load.md +++ /dev/null @@ -1,20 +0,0 @@ -# [S] @moq/crypto loads a stored signing key - -## Goal - -`@moq/crypto`'s `SecretKey` loads a stored Ed25519 key (raw bytes, and a -non-extractable WebCrypto `CryptoKey` for browsers) and exposes `sign` -publicly, mirroring Rust's `moq_crypto::sign::SecretKey` -(`load_or_generate`, `sign`), so one device key both proves mesh identity -and signs broadcasts. - -## Plan - -Planned 2026-11-03 (maintainer) from Kixel's `@kixel/mesh` -([moq-dev/kixel.ai#16](https://github.com/moq-dev/kixel.ai/pull/16)), which -had to carry its own Ed25519 and z-base-32 code because `SecretKey` cannot -load a stored key and its `sign` is internal. Keep generated keys working, -keep the browser path non-extractable, and test that a key loaded from bytes -signs what Rust verifies. A raw 32-byte seed already loads through -`SecretKey.fromSeed` (`js/crypto/src/sign/key.ts`), but it and `sign` are -marked `@internal`, and no `CryptoKey` path exists. Moved to m4 2026-11-12 (maintainer, /quest-audit): the downstream application it was ranked for made moq-crypto opt-in and plans nothing on it. diff --git a/quest/m5/link-quality.md b/quest/m5/link-quality.md deleted file mode 100644 index 8bc39e2b0f5..00000000000 --- a/quest/m5/link-quality.md +++ /dev/null @@ -1,38 +0,0 @@ -# [M] Link quality - -## Goal - -A link can take its cost from its measured quality instead of configuration, -so a drone mesh steers around a fading radio link without manual tuning, -while a small wobble never flips a route. Static configured cost stays the -default and is what the CDN runs; measured cost is opt-in per link. This is -local policy feeding the one metric on the wire, with no wire change. - -## Plan - -Decided 2026-10-01: the wire carries one additive metric and each node -computes its own link costs. - -Decided 2026-10-08: m2, and Related rather than Required for the -[cluster routing line](/quest/m3/cluster-routing/README.md): static costs are -the default and measured cost is opt-in, so the line ships without it. - -- Derive the cost from what QUIC already measures (smoothed RTT, loss); an - ETX-style estimate (Babel RFC 8966 Appendix A, B.A.T.M.A.N.'s TQ) is the - reference. Pick the formula from the harness, not from intuition. -- Hysteresis: only advertise a cost change past a threshold or after it - holds, so the route layer sees a few changes per real shift. Report how - many ROUTE changes a fading link causes. -- Never mix it into business pricing: an operator's priced topology keeps - RTT out of monetary costs, and that stays true because measured cost is - opt-in on links the operator chooses. -- A lossy-link fixture over the mock transport with mocked time. - -Detecting stalled epoch-bound track delivery is out of scope. - -Public API: a per-link way to ask for measured cost in the peer entry. -Wire: none. - -## Required - -- [Routes and announces](/quest/m2/cluster-routes/README.md) - a cost change costs one ROUTE per origin there, instead of a re-announce per broadcast diff --git a/quest/m5/mesh-guide.md b/quest/m5/mesh-guide.md deleted file mode 100644 index e83d9898445..00000000000 --- a/quest/m5/mesh-guide.md +++ /dev/null @@ -1,21 +0,0 @@ -# [S] Mesh library guide - -## Goal - -doc.moq.dev has a mesh page for developers building on the libraries: -enrolling a node against `moq auth`, the enrollment contract a hosted -coordinator implements, `Endpoint` services and byte streams, LAN discovery -and address hints, the sparse overlay, protection levels, the `.status` -schema, and porting from iroh. It links Kixel as one binary built on them. - -## Plan - -Decided 2026-10-25 (maintainer): a new page, because no feature quest can -carry the whole setup. Each feature quest still updates the docs it makes -stale. Decided 2026-11-02 (maintainer, Kixel planning): installing and -running a mesh daemon is Kixel's documentation, so this page covers only the -libraries. - -## Required - -- [Endpoint API](/quest/m3/mesh/endpoint.md) - the API the page centers on diff --git a/quest/m5/mesh-ice.md b/quest/m5/mesh-ice.md deleted file mode 100644 index ebd19650527..00000000000 --- a/quest/m5/mesh-ice.md +++ /dev/null @@ -1,21 +0,0 @@ -# [L] Mesh over ICE - -## Goal - -Mesh nodes on different networks that cannot dial each other's addresses -connect directly through the P2P line's native -QUIC-over-ICE pairs, with the relay as rendezvous and fallback, and the -direct link joins the [sparse overlay](/quest/m3/mesh/overlay.md) as an -ordinary link. - -## Plan - -Parked 2026-10-31 (maintainer, moq.pro mesh planning interview): remote mesh -traffic is meant to cross relays, so hole punching waits in m5 until a mesh -user needs it. Reuse the P2P line's ICE, signaling under the mesh's own -prefix, and operator-supplied ICE servers; no TURN, since the relay already -carries the fallback route. - -## Required - -- [Sparse overlay](/quest/m3/mesh/overlay.md) - the overlay the direct link joins diff --git a/quest/m5/mesh-js-enroll.md b/quest/m5/mesh-js-enroll.md deleted file mode 100644 index c6a8bb7614a..00000000000 --- a/quest/m5/mesh-js-enroll.md +++ /dev/null @@ -1,26 +0,0 @@ -# [S] JS mesh enrollment client - -## Goal - -`@moq/auth` exports a JS mesh enrollment client matching -`moq_auth::mesh::Client` (`rs/moq-auth/src/mesh/client.rs`): challenge, -enroll with an enrollment key, and refresh by proving the endpoint key, -interoperating with `moq auth serve --mesh-url` and any coordinator that -implements the same contract. - -## Plan - -Planned 2026-11-03 (maintainer) from a downstream application's JS mesh -client, which implemented this contract then. Port its hardening where it is -generic: https except on loopback, no redirects, bounded time and reply -size, and every reply field checked. Test against `moq auth serve` the way -`rs/moq-auth/src/mesh/contract_tests.rs` does. - -Moved to m5 2026-11-12 (maintainer, /quest-audit): that client has since -moved to a certificate enrollment reply, which `moq_auth::mesh::Client`'s -token-only `Enrollment` (`rs/moq-auth/src/mesh/mod.rs`) does not carry, so it -no longer wraps this contract and nothing else asks for a JS client. - -## Required - -- [`@moq/crypto` loads a stored signing key](/quest/m4/crypto-key-load.md) - the endpoint key this proves diff --git a/quest/m5/mesh-keys.md b/quest/m5/mesh-keys.md deleted file mode 100644 index a24ce8ca614..00000000000 --- a/quest/m5/mesh-keys.md +++ /dev/null @@ -1,23 +0,0 @@ -# [L] Mesh keys - -## Goal - -A mesh broadcast can be encrypted for its authorized subscribers, one-to-one -or fanout, and a mesh can authenticate frames with symmetric keys instead of -per-frame signatures. A publisher distributes per-broadcast keys on a -cacheable `.keys` track, wrapped to each subscriber's endpoint key, and rotates -them when a subscriber is removed. - -## Plan - -Parked 2026-10-25 (maintainer): encryption is out of the first mesh; identity -and authentication ship first. Research recorded in the planning PR: SFrame -and IETF secure objects leave key distribution open and give no sender -authenticity; MLS per broadcast is heavy for churning viewers; a sender-keys -style wrapped key track fans out like media. The AEAD stays moq-crypto's e2ee. -Revisit when a mesh user needs confidentiality from relays. - -## Related - -- [Mesh](/quest/m3/mesh/README.md) - the protection levels this completes -- [End-to-end encryption](/quest/m5/e2ee/README.md) - the AEAD and its key contract diff --git a/rs/moq-auth/Cargo.toml b/rs/moq-auth/Cargo.toml index d8cd41e6e1a..86140161a14 100644 --- a/rs/moq-auth/Cargo.toml +++ b/rs/moq-auth/Cargo.toml @@ -36,7 +36,7 @@ axum = { workspace = true, optional = true } base64 = { workspace = true } jsonwebtoken = { workspace = true } kio = { workspace = true, features = ["serde"] } -moq-crypto = { workspace = true, features = ["serde", "cpu"] } +moq-crypto = { workspace = true, features = ["cpu"] } moq-pattern = { workspace = true, features = ["serde"] } pkcs1 = { workspace = true, features = ["std"] } pkcs8 = { workspace = true } @@ -52,7 +52,7 @@ serde_with = { workspace = true, features = ["base64"] } thiserror = { workspace = true } tokio = { workspace = true, features = ["fs", "macros", "rt", "time", "sync"], optional = true } tracing = { workspace = true, optional = true } -url = { workspace = true, features = ["serde"] } +url = { workspace = true } [dev-dependencies] anyhow = { workspace = true } diff --git a/rs/moq-auth/src/error.rs b/rs/moq-auth/src/error.rs index e9018409647..bd0c6a213ff 100644 --- a/rs/moq-auth/src/error.rs +++ b/rs/moq-auth/src/error.rs @@ -196,14 +196,6 @@ pub enum Error { #[error("{0}")] Other(String), - /// A mesh coordinator refused the request. - #[error("mesh coordinator refused: {0}")] - Mesh(#[from] crate::mesh::ErrorReply), - - /// The mesh coordinator's state file could not be read, parsed, or written. - #[error("mesh state: {0}")] - MeshState(String), - /// This CPU lacks a feature the crate's cryptography requires, named in rind's message. #[error(transparent)] UnsupportedCpu(#[from] moq_crypto::cpu::UnsupportedCpu), diff --git a/rs/moq-auth/src/fs.rs b/rs/moq-auth/src/fs.rs index 18ed78b35b2..9ea7bcff02b 100644 --- a/rs/moq-auth/src/fs.rs +++ b/rs/moq-auth/src/fs.rs @@ -20,7 +20,7 @@ pub(crate) fn write(path: &Path, contents: &str, private: bool) -> io::Result<() } #[cfg(unix)] -pub(crate) fn write_private(path: &Path, contents: &str) -> io::Result<()> { +fn write_private(path: &Path, contents: &str) -> io::Result<()> { use std::io::Write; use std::os::unix::fs::{OpenOptionsExt, PermissionsExt}; @@ -40,6 +40,6 @@ pub(crate) fn write_private(path: &Path, contents: &str) -> io::Result<()> { /// Windows has no cheap equivalent of the Unix mode bits, so the file inherits the directory's ACL. #[cfg(not(unix))] -pub(crate) fn write_private(path: &Path, contents: &str) -> io::Result<()> { +fn write_private(path: &Path, contents: &str) -> io::Result<()> { std::fs::write(path, contents) } diff --git a/rs/moq-auth/src/lib.rs b/rs/moq-auth/src/lib.rs index 75c37f79240..e512566fb44 100644 --- a/rs/moq-auth/src/lib.rs +++ b/rs/moq-auth/src/lib.rs @@ -44,7 +44,6 @@ mod set; mod wire; pub mod lease; -pub mod mesh; pub mod refusal; pub use refusal::Refusal; #[cfg(feature = "serve")] diff --git a/rs/moq-auth/src/mesh/client.rs b/rs/moq-auth/src/mesh/client.rs deleted file mode 100644 index 10cdaf8c869..00000000000 --- a/rs/moq-auth/src/mesh/client.rs +++ /dev/null @@ -1,158 +0,0 @@ -//! The device side of the contract: enroll, then keep the token fresh. - -use std::sync::Arc; -use std::time::Duration; - -use url::Url; - -use super::{Challenge, EnrollRequest, Enrollment, ErrorCode, ErrorReply, Proof, Signer}; -use crate::Error; - -/// Every request is bounded so a hung coordinator fails rather than parks the device. -const TIMEOUT: Duration = Duration::from_secs(10); - -/// The largest reply read from the coordinator; a mesh's member list fits easily. -const MAX_REPLY: usize = 4 << 20; - -/// The longest wait between refresh attempts while the coordinator is unreachable. -const MAX_BACKOFF: Duration = Duration::from_secs(60); - -/// A device's client for one coordinator, proving `signer`'s key on every request that -/// needs it. -#[derive(Clone)] -pub struct Client { - http: reqwest::Client, - base: Url, - signer: Arc, -} - -impl Client { - /// A client for the coordinator at `base`, the URL the contract's paths hang off. - /// - /// `https://` is required, except `http://` to a loopback host, since the token rides - /// the reply. - pub fn new(base: Url, signer: Arc) -> crate::Result { - let loopback = match base.host() { - Some(url::Host::Ipv4(ip)) => ip.is_loopback(), - Some(url::Host::Ipv6(ip)) => ip.is_loopback(), - Some(url::Host::Domain(host)) => host == "localhost", - None => false, - }; - match base.scheme() { - "https" => {} - "http" if loopback => {} - "http" => return Err(Error::InsecureUrl(base.to_string())), - _ => return Err(Error::InvalidUrl(base.to_string())), - } - // A redirect would resend a proof or enrollment key to wherever it points. - let http = reqwest::Client::builder() - .redirect(reqwest::redirect::Policy::none()) - .timeout(TIMEOUT) - .use_preconfigured_tls(crate::crypto::tls()?) - .build()?; - Ok(Self { http, base, signer }) - } - - /// Join with a pre-authorized enrollment key, asking for `name`. - /// - /// When the coordinator refuses the key but this endpoint is already enrolled, such as - /// a retry after a single-use key's reply was lost, this refreshes instead, keeping the - /// name the device already has. - pub async fn enroll(&self, key: &str, name: Option<&str>) -> crate::Result { - let proof = self.proof().await?; - let request = EnrollRequest { - proof, - key: key.into(), - name: name.map(Into::into), - }; - match self.post("enroll", &request).await { - Err(Error::Mesh(refused)) if refused.error == ErrorCode::InvalidGrant => match self.refresh().await { - // Not enrolled after all, so the key's refusal is the answer. - Err(Error::Mesh(reply)) if reply.error == ErrorCode::AccessDenied => Err(Error::Mesh(refused)), - res => res, - }, - res => res, - } - } - - /// Prove the key again for a fresh token. Fails with [`ErrorCode::AccessDenied`] once - /// the device is revoked. - pub async fn refresh(&self) -> crate::Result { - let proof = self.proof().await?; - self.post("refresh", &proof).await - } - - /// Wait until `current` is two thirds through its remaining life, then refresh it, - /// retrying through an outage until it expires. A refusal is returned at once, so a - /// revoked device stops rather than retrying, except a transient one: a server error, - /// `slow_down`, or a challenge that expired or outlived a coordinator restart. - pub async fn renew(&self, current: &Enrollment) -> crate::Result { - let remaining = || { - current - .expires - .checked_duration_since(kio::clock::wall()) - .unwrap_or_default() - }; - kio::clock::sleep(remaining() * 2 / 3).await; - - let mut backoff = Duration::from_secs(1); - loop { - match self.refresh().await { - Err(Error::Mesh(reply)) - if !matches!( - reply.error, - ErrorCode::ServerError | ErrorCode::SlowDown | ErrorCode::InvalidChallenge - ) => - { - return Err(Error::Mesh(reply)); - } - Ok(enrollment) => return Ok(enrollment), - Err(err) if remaining().is_zero() => return Err(err), - Err(err) => { - tracing::warn!(%err, "mesh refresh failed; retrying"); - kio::clock::sleep(backoff.min(remaining())).await; - backoff = (backoff * 2).min(MAX_BACKOFF); - } - } - } - } - - async fn proof(&self) -> crate::Result { - let challenge: Challenge = self.post("challenge", &serde_json::Value::Null).await?; - Ok(Proof::sign(self.signer.as_ref(), &self.base, challenge.challenge)) - } - - async fn post( - &self, - path: &str, - body: &T, - ) -> crate::Result { - let mut url = self.base.clone(); - let joined = format!("{}/{path}", url.path().trim_end_matches('/')); - url.set_path(&joined); - - let request = self.http.post(url); - let request = match serde_json::to_value(body)? { - serde_json::Value::Null => request, - body => request.json(&body), - }; - let mut response = request.send().await?; - let status = response.status(); - - let mut bytes = Vec::new(); - while let Some(chunk) = response.chunk().await? { - if bytes.len() + chunk.len() > MAX_REPLY { - return Err(Error::Unavailable("the coordinator's reply is too large".into())); - } - bytes.extend_from_slice(&chunk); - } - - if status.is_success() { - return Ok(serde_json::from_slice(&bytes)?); - } - match serde_json::from_slice::(&bytes) { - Ok(reply) => Err(Error::Mesh(reply)), - Err(_) => Err(Error::Unavailable(format!("the coordinator answered {status}"))), - } - } -} diff --git a/rs/moq-auth/src/mesh/contract_tests.rs b/rs/moq-auth/src/mesh/contract_tests.rs deleted file mode 100644 index 6f3f043fc5b..00000000000 --- a/rs/moq-auth/src/mesh/contract_tests.rs +++ /dev/null @@ -1,317 +0,0 @@ -//! The contract end to end: a [`Client`] against a [`Coordinator`] over HTTP. - -use std::sync::Arc; -use std::time::Duration; - -use rind::signature::Ed25519KeyPair; - -use super::*; -use crate::{Algorithm, Error, Key, KeyId}; - -struct Mesh { - coordinator: Coordinator, - base: url::Url, - key: Key, -} - -impl Mesh { - async fn new() -> Self { - Self::with_ttl(Duration::from_secs(3600)).await - } - - async fn with_ttl(ttl: Duration) -> Self { - let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); - let base: url::Url = format!("http://{}/mesh", listener.local_addr().unwrap()) - .parse() - .unwrap(); - let key = Key::generate(Algorithm::HS256, Some(KeyId::decode("mesh").unwrap())).unwrap(); - let mut config = Config::new(base.clone(), key.clone()); - config.relays = vec!["https://relay.example".parse().unwrap()]; - config.token_ttl = ttl; - let coordinator = Coordinator::new(config).unwrap(); - let router = coordinator.router(); - tokio::spawn(async move { axum::serve(listener, router).await }); - Self { coordinator, base, key } - } - - fn device(&self) -> Client { - let signer = Arc::new(Ed25519KeyPair::generate().unwrap()); - Client::new(self.base.clone(), signer).unwrap() - } - - fn key(&self, reusable: bool) -> String { - let new = NewKey { - reusable, - tags: vec!["robot".into()], - ..Default::default() - }; - self.coordinator.create_key(&new).unwrap().key.unwrap() - } - - /// The code a refusal carries. - fn refused(err: Error) -> ErrorCode { - match err { - Error::Mesh(reply) => reply.error, - other => panic!("expected a refusal, got {other}"), - } - } -} - -#[tokio::test] -async fn an_enrollment_key_grants_the_endpoints_subtree() { - let mesh = Mesh::new().await; - let device = mesh.device(); - let enrollment = device.enroll(&mesh.key(false), Some("Robot 1")).await.unwrap(); - - assert_eq!(enrollment.name, "robot-1"); - assert_eq!( - enrollment.relays, - vec!["https://relay.example".parse::().unwrap()] - ); - assert_eq!(enrollment.members.len(), 1); - assert_eq!(enrollment.members[0].tags, vec!["robot".to_string()]); - - let id = enrollment.members[0].endpoint; - let claims = mesh.key.verify(&enrollment.token).unwrap(); - assert_eq!(claims.expires, Some(enrollment.expires)); - let own = claims.authorize(&format!("mesh/{id}/ssh")).unwrap(); - assert!(!own.publish.is_empty()); - let other = claims.authorize("mesh/someone").unwrap(); - assert!(other.publish.is_empty()); - assert!(!other.subscribe.is_empty()); - assert!(claims.authorize("elsewhere").is_err()); -} - -#[tokio::test] -async fn a_single_use_key_enrolls_once() { - let mesh = Mesh::new().await; - let key = mesh.key(false); - let device = mesh.device(); - let first = device.enroll(&key, Some("pi")).await.unwrap(); - let err = mesh.device().enroll(&key, None).await.unwrap_err(); - assert_eq!(Mesh::refused(err), ErrorCode::InvalidGrant); - - // The device it enrolled can retry, as after a lost reply, and keeps its name. - let retried = device.enroll(&key, Some("other")).await.unwrap(); - assert_eq!(retried.name, first.name); - assert!(mesh.key.verify(&retried.token).is_ok()); - - let reusable = mesh.key(true); - let first = mesh.device().enroll(&reusable, Some("bot")).await.unwrap(); - let second = mesh.device().enroll(&reusable, Some("bot")).await.unwrap(); - assert_eq!(first.name, "bot"); - assert_eq!(second.name, "bot-2"); - assert_eq!(second.members.len(), 3); -} - -#[tokio::test] -async fn a_revoked_or_unknown_key_enrolls_nothing() { - let mesh = Mesh::new().await; - let key = mesh.key(true); - let id = key.split_once('.').unwrap().0; - assert!(mesh.coordinator.revoke_key(id).unwrap()); - let err = mesh.device().enroll(&key, None).await.unwrap_err(); - assert_eq!(Mesh::refused(err), ErrorCode::InvalidGrant); - - let err = mesh.device().enroll(&format!("{id}.wrong"), None).await.unwrap_err(); - assert_eq!(Mesh::refused(err), ErrorCode::InvalidGrant); -} - -#[tokio::test] -async fn a_proof_from_the_wrong_key_is_refused() { - let mesh = Mesh::new().await; - let challenge = mesh.coordinator.challenge(); - let holder = Ed25519KeyPair::generate().unwrap(); - let victim = Ed25519KeyPair::generate().unwrap(); - let mut proof = Proof::sign(&holder, &mesh.base, challenge.challenge); - proof.endpoint = victim.endpoint(); - let request = EnrollRequest { - proof, - key: mesh.key(true), - name: None, - }; - assert_eq!( - mesh.coordinator.enroll(&request).unwrap_err().error, - ErrorCode::InvalidSignature - ); - - // A forged proof spends nothing, so the holder can still use the challenge, once. - let mut request = request; - request.proof = Proof::sign(&holder, &mesh.base, request.proof.challenge); - mesh.coordinator.enroll(&request).unwrap(); - assert_eq!( - mesh.coordinator.enroll(&request).unwrap_err().error, - ErrorCode::InvalidChallenge - ); - - // A proof made for another coordinator does not verify here. - let challenge = mesh.coordinator.challenge(); - request.proof = Proof::sign( - &holder, - &"https://other.example/mesh".parse().unwrap(), - challenge.challenge, - ); - assert_eq!( - mesh.coordinator.enroll(&request).unwrap_err().error, - ErrorCode::InvalidSignature - ); -} - -#[tokio::test] -async fn refresh_issues_a_new_token_until_revoked() { - let mesh = Mesh::new().await; - let device = mesh.device(); - - // Refreshing before enrolling proves a key the mesh does not know. - let err = device.refresh().await.unwrap_err(); - assert_eq!(Mesh::refused(err), ErrorCode::AccessDenied); - - let enrollment = device.enroll(&mesh.key(true), Some("pi")).await.unwrap(); - let refreshed = device.refresh().await.unwrap(); - assert_eq!(refreshed.name, "pi"); - assert!(mesh.key.verify(&refreshed.token).is_ok()); - - let id = enrollment.members[0].endpoint; - assert!(mesh.coordinator.revoke_device(id).unwrap()); - let err = device.refresh().await.unwrap_err(); - assert_eq!(Mesh::refused(err), ErrorCode::AccessDenied); - - // A reusable key cannot bring a revoked device back. - let err = device.enroll(&mesh.key(true), None).await.unwrap_err(); - assert_eq!(Mesh::refused(err), ErrorCode::AccessDenied); -} - -#[tokio::test] -async fn renew_refreshes_before_expiry() { - // A clock only the test moves, which the coordinator reads too; the HTTP between them is real. - let clock = kio::clock::Manual::new().install(&tokio::runtime::Handle::current()); - let ttl = Duration::from_secs(3); - let mesh = Mesh::with_ttl(ttl).await; - let device = mesh.device(); - let enrollment = device.enroll(&mesh.key(true), None).await.unwrap(); - let mut renew = std::pin::pin!(device.renew(&enrollment)); - // The first poll arms the wait for two thirds of the token's life, before any request. - let mut cx = std::task::Context::from_waker(std::task::Waker::noop()); - assert!(std::future::Future::poll(renew.as_mut(), &mut cx).is_pending()); - clock.advance(ttl * 2 / 3); - let renewed = renew.await.unwrap(); - assert!(kio::clock::wall() < enrollment.expires); - assert!(renewed.expires > enrollment.expires); -} - -#[tokio::test] -async fn only_an_issued_challenge_is_accepted() { - let mesh = Mesh::new().await; - let holder = Ed25519KeyPair::generate().unwrap(); - let enroll = |challenge: Vec| { - let request = EnrollRequest { - proof: Proof::sign(&holder, &mesh.base, challenge), - key: mesh.key(true), - name: None, - }; - mesh.coordinator.enroll(&request).unwrap_err().error - }; - - // Made up, tampered with, or issued before a restart (another secret). - assert_eq!(enroll(vec![7; CHALLENGE_LEN]), ErrorCode::InvalidChallenge); - let mut tampered = mesh.coordinator.challenge().challenge; - tampered[0] ^= 1; - assert_eq!(enroll(tampered), ErrorCode::InvalidChallenge); - let mut config = Config::new(mesh.base.clone(), mesh.key.clone()); - config.relays = vec!["https://relay.example".parse().unwrap()]; - let restarted = Coordinator::new(config).unwrap(); - assert_eq!(enroll(restarted.challenge().challenge), ErrorCode::InvalidChallenge); -} - -#[tokio::test] -async fn renew_retries_a_transient_refusal() { - kio::clock::host_time(); // real HTTP, and the backoff is armed only after its refusal - use wiremock::matchers::{method, path}; - use wiremock::{Mock, MockServer, ResponseTemplate}; - - let mesh = Mesh::with_ttl(Duration::from_secs(3)).await; - let signer = Arc::new(Ed25519KeyPair::generate().unwrap()); - let device = Client::new(mesh.base.clone(), signer.clone()).unwrap(); - let enrollment = device.enroll(&mesh.key(true), None).await.unwrap(); - let fresh = device.refresh().await.unwrap(); - - // A coordinator that restarts between issuing the challenge and seeing the proof. - let server = MockServer::start().await; - Mock::given(method("POST")) - .and(path("/mesh/challenge")) - .respond_with(ResponseTemplate::new(200).set_body_json(mesh.coordinator.challenge())) - .mount(&server) - .await; - Mock::given(method("POST")) - .and(path("/mesh/refresh")) - .respond_with( - ResponseTemplate::new(400).set_body_json(ErrorReply::new(ErrorCode::InvalidChallenge, "unknown challenge")), - ) - .up_to_n_times(1) - .with_priority(1) - .mount(&server) - .await; - Mock::given(method("POST")) - .and(path("/mesh/refresh")) - .respond_with(ResponseTemplate::new(200).set_body_json(&fresh)) - .mount(&server) - .await; - - let device = Client::new(format!("{}/mesh", server.uri()).parse().unwrap(), signer).unwrap(); - let renewed = device.renew(&enrollment).await.unwrap(); - assert_eq!(renewed, fresh); -} - -#[test] -fn a_key_that_cannot_sign_refuses_to_start() { - let key = Key::generate(Algorithm::ES256, None).unwrap().to_public().unwrap(); - let config = Config::new("https://mesh.example".parse().unwrap(), key); - assert!(Coordinator::new(config).is_err()); -} - -#[test] -fn a_failed_save_spends_nothing() { - let dir = tempfile::tempdir().unwrap(); - let key = Key::generate(Algorithm::HS256, None).unwrap(); - let mut config = Config::new("https://mesh.example".parse().unwrap(), key); - config.state = Some(dir.path().join("mesh.json")); - let coordinator = Coordinator::new(config).unwrap(); - let secret = coordinator.create_key(&NewKey::default()).unwrap().key.unwrap(); - - // A directory where the temporary file goes makes every save fail. - std::fs::create_dir(dir.path().join("mesh.tmp")).unwrap(); - let signer = Ed25519KeyPair::generate().unwrap(); - let base: url::Url = "https://mesh.example".parse().unwrap(); - let enroll = |coordinator: &Coordinator| EnrollRequest { - proof: Proof::sign(&signer, &base, coordinator.challenge().challenge), - key: secret.clone(), - name: None, - }; - let err = coordinator.enroll(&enroll(&coordinator)).unwrap_err(); - assert_eq!(err.error, ErrorCode::ServerError); - assert!(coordinator.devices().is_empty()); - assert!(!coordinator.keys()[0].used); - - std::fs::remove_dir(dir.path().join("mesh.tmp")).unwrap(); - coordinator.enroll(&enroll(&coordinator)).unwrap(); - assert!(coordinator.keys()[0].used); -} - -#[tokio::test] -async fn a_malformed_body_is_an_oauth_error() { - let mesh = Mesh::new().await; - let url = format!("{}/enroll", mesh.base); - let response = reqwest::Client::builder() - .use_preconfigured_tls(crate::crypto::tls().unwrap()) - .build() - .unwrap() - .post(url) - .header("content-type", "application/json") - .body("{}") - .send() - .await - .unwrap(); - assert_eq!(response.status(), reqwest::StatusCode::BAD_REQUEST); - let reply: ErrorReply = response.json().await.unwrap(); - assert_eq!(reply.error, ErrorCode::InvalidRequest); -} diff --git a/rs/moq-auth/src/mesh/coordinator.rs b/rs/moq-auth/src/mesh/coordinator.rs deleted file mode 100644 index cee6b7e5206..00000000000 --- a/rs/moq-auth/src/mesh/coordinator.rs +++ /dev/null @@ -1,783 +0,0 @@ -//! The reference coordinator behind `moq auth serve --mesh-url`. -//! -//! [`Coordinator::router`] answers the device-facing contract; [`Coordinator::admin_router`] -//! is the owner's API, mounted on the auth server's own listener, which has no -//! authentication and stays on loopback. Enrollment keys and devices persist to a JSON -//! state file. Challenges are sealed with a secret held in memory, so a restart only makes -//! a device fetch a new challenge. -//! -//! A challenge is single-use per endpoint: each carries a sequence number, and an endpoint's -//! proof is accepted only over a challenge newer than the last one it was granted with. The -//! replay state is one number per enrolled device, so no caller can spend another's room. - -use std::collections::HashMap; -use std::path::PathBuf; -use std::sync::atomic::{AtomicU64, Ordering}; -use std::sync::{Arc, Mutex}; -use std::time::Duration; - -use axum::extract::{Path, State}; -use axum::http::StatusCode; -use axum::response::{IntoResponse, Response}; -use axum::routing::{delete, get, post}; -use axum::{Json, Router}; -use moq_crypto::sign::PublicKey; -use serde::{Deserialize, Serialize}; -use serde_with::serde_as; - -use super::{CHALLENGE_LEN, Challenge, EnrollRequest, Enrollment, ErrorCode, ErrorReply, Member, Proof}; -use crate::{Claims, Key, Pattern}; - -/// How long a challenge stays valid. -const CHALLENGE_TTL: Duration = Duration::from_secs(60); - -/// A challenge is its expiry (unix seconds), its sequence number, then a truncated HMAC -/// over both. The HMAC makes it unpredictable; the sequence makes it single-use. -const TAG_LEN: usize = CHALLENGE_LEN - 16; - -/// The longest a device name may be: one DNS label. -const MAX_NAME: usize = 63; - -/// What the coordinator issues, and where it keeps its state. -/// -/// `#[non_exhaustive]`, so build it with [`Config::new`] and set the fields. -#[derive(Clone, Debug)] -#[non_exhaustive] -pub struct Config { - /// The public base URL devices reach the contract at, which every proof is bound to. - pub url: url::Url, - /// The key that signs mesh tokens; the relays must verify it. - pub key: Key, - /// The relays an [`Enrollment`] tells a device to dial. - pub relays: Vec, - /// The JSON file enrollment keys and devices persist to; `None` keeps them in memory. - pub state: Option, - /// How long an issued token lasts. - pub token_ttl: Duration, -} - -impl Config { - /// A coordinator at `url` signing with `key`, with hour-long tokens, no relays, and no - /// state file. - pub fn new(url: url::Url, key: Key) -> Self { - Self { - url, - key, - relays: Vec::new(), - state: None, - token_ttl: Duration::from_secs(3600), - } - } -} - -/// The state that survives a restart. -#[derive(Clone, Default, Serialize, Deserialize)] -struct Store { - #[serde(default)] - keys: Vec, - #[serde(default)] - devices: Vec, -} - -/// An enrollment key, holding only the SHA-256 of its secret. -#[serde_as] -#[derive(Clone, Serialize, Deserialize)] -struct StoredKey { - id: String, - /// Hex, so the file stays readable. - hash: String, - #[serde(default, skip_serializing_if = "Vec::is_empty")] - tags: Vec, - #[serde_as(as = "Option")] - #[serde(default, skip_serializing_if = "Option::is_none")] - expires: Option, - reusable: bool, - #[serde(default)] - used: bool, - #[serde(default)] - revoked: bool, -} - -/// An enrolled device. A revoked one stays recorded so a reusable key cannot re-enroll it. -#[serde_as] -#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] -#[non_exhaustive] -pub struct Device { - /// Its endpoint ID. - pub endpoint: PublicKey, - /// Its unique, DNS-safe name. - pub name: String, - /// The tags of the enrollment key it joined with. - #[serde(default, skip_serializing_if = "Vec::is_empty")] - pub tags: Vec, - /// When it first enrolled, unix seconds. - #[serde_as(as = "kio::clock::serde::UnixSeconds")] - pub enrolled: kio::clock::Wall, - /// Whether the owner revoked it; its next refresh fails. - #[serde(default)] - pub revoked: bool, -} - -/// An enrollment key as the owner sees it, without its secret. -#[serde_as] -#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] -#[non_exhaustive] -pub struct KeyInfo { - /// The ID the owner revokes it by. - pub id: String, - /// The tags every device it enrolls carries. - #[serde(default, skip_serializing_if = "Vec::is_empty")] - pub tags: Vec, - /// When it stops enrolling, unix seconds. - #[serde_as(as = "Option")] - #[serde(default, skip_serializing_if = "Option::is_none")] - pub expires: Option, - /// Whether it enrolls more than one device. - pub reusable: bool, - /// Whether a single-use key has enrolled its device. - pub used: bool, - /// Whether the owner revoked it. - pub revoked: bool, - /// The key itself, returned only when it is created. - #[serde(default, skip_serializing_if = "Option::is_none")] - pub key: Option, -} - -impl From<&StoredKey> for KeyInfo { - fn from(key: &StoredKey) -> Self { - Self { - id: key.id.clone(), - tags: key.tags.clone(), - expires: key.expires, - reusable: key.reusable, - used: key.used, - revoked: key.revoked, - key: None, - } - } -} - -/// `POST /mesh/keys` on the admin API: what the new enrollment key allows. -#[serde_as] -#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)] -#[serde(default)] -#[non_exhaustive] -pub struct NewKey { - /// Tags every device it enrolls carries. - pub tags: Vec, - /// How long until it stops enrolling; `None` never expires. Whole seconds on the wire. - #[serde_as(as = "Option>")] - pub expires_in: Option, - /// Whether it enrolls more than one device. - pub reusable: bool, -} - -#[derive(Default)] -struct Inner { - store: Store, - /// The sequence of the last challenge each endpoint was granted with. Only an - /// enrolled device gets an entry, so it grows with the devices and nothing else. - used: HashMap, -} - -/// The mesh coordinator: issues challenges, enrolls devices, and refreshes their tokens. -#[derive(Clone)] -pub struct Coordinator { - config: Arc, - /// Seals challenges, so an outstanding one needs no state. - secret: Arc, - /// The next challenge's sequence number. It restarts with the secret, which voids - /// every earlier challenge. - sequence: Arc, - inner: Arc>, -} - -type Reply = Result; - -impl Coordinator { - /// A coordinator for `config`, loading its state file if one exists. - pub fn new(config: Config) -> crate::Result { - let store = match &config.state { - Some(path) => match std::fs::read_to_string(path) { - Ok(json) => serde_json::from_str(&json).map_err(|err| crate::Error::MeshState(err.to_string()))?, - Err(err) if err.kind() == std::io::ErrorKind::NotFound => Store::default(), - Err(err) => return Err(crate::Error::MeshState(err.to_string())), - }, - None => Store::default(), - }; - // Sign once now, so a key that cannot sign these claims fails at startup rather - // than after an enrollment key is spent. - config - .key - .sign(&claims(PublicKey::from_bytes([0; 32]), kio::clock::wall(), None))?; - Ok(Self { - config: Arc::new(config), - secret: Arc::new(rind::hmac::Key::new(&rind::hmac::HMAC_SHA256, &random(32))), - sequence: Default::default(), - inner: Arc::new(Mutex::new(Inner { - store, - ..Default::default() - })), - }) - } - - /// Issue a fresh challenge. It holds no state until a proof uses it, so no number of - /// requests can crowd out one a device is still signing. - pub fn challenge(&self) -> Challenge { - let now = kio::clock::wall(); - let expires = - now.checked_duration_since(kio::clock::Wall::UNIX_EPOCH) - .unwrap_or_default() - .as_secs() + CHALLENGE_TTL.as_secs(); - let mut challenge = expires.to_be_bytes().to_vec(); - challenge.extend_from_slice(&self.sequence.fetch_add(1, Ordering::Relaxed).to_be_bytes()); - let tag = self.tag(&challenge); - challenge.extend_from_slice(&tag); - Challenge { - challenge, - expires: kio::clock::Wall::UNIX_EPOCH.saturating_add(Duration::from_secs(expires)), - } - } - - /// Enroll with an enrollment key. - pub fn enroll(&self, request: &EnrollRequest) -> Reply { - let mut inner = self.inner.lock().unwrap(); - let sequence = self.check(&inner, &request.proof)?; - - let (id, secret) = request - .key - .split_once('.') - .ok_or_else(|| ErrorReply::new(ErrorCode::InvalidGrant, "unknown enrollment key"))?; - let hash = hex(&sha256(secret.as_bytes())); - let now = kio::clock::wall(); - let key = inner - .store - .keys - .iter() - .find(|key| key.id == id && constant_eq(key.hash.as_bytes(), hash.as_bytes())) - .ok_or_else(|| ErrorReply::new(ErrorCode::InvalidGrant, "unknown enrollment key"))?; - if key.revoked { - return Err(ErrorReply::new( - ErrorCode::InvalidGrant, - "the enrollment key is revoked", - )); - } - if key.expires.is_some_and(|expires| expires <= now) { - return Err(ErrorReply::new( - ErrorCode::InvalidGrant, - "the enrollment key has expired", - )); - } - if !key.reusable && key.used { - return Err(ErrorReply::new( - ErrorCode::InvalidGrant, - "the enrollment key was already used", - )); - } - let tags = key.tags.clone(); - let reusable = key.reusable; - let id = id.to_string(); - - let endpoint = request.proof.endpoint; - let enrollment = self.commit(&mut inner, |store| { - self.admit(store, endpoint, request.name.as_deref(), tags)?; - if !reusable && let Some(key) = store.keys.iter_mut().find(|key| key.id == id) { - key.used = true; - } - self.issue(store, endpoint) - })?; - inner.used.insert(endpoint, sequence); - Ok(enrollment) - } - - /// Refresh an enrolled device's token, refusing a revoked or unknown one. - pub fn refresh(&self, proof: &Proof) -> Reply { - let mut inner = self.inner.lock().unwrap(); - let sequence = self.check(&inner, proof)?; - let enrollment = match inner - .store - .devices - .iter() - .find(|device| device.endpoint == proof.endpoint) - { - Some(device) if !device.revoked => self.issue(&inner.store, proof.endpoint)?, - Some(_) => return Err(ErrorReply::new(ErrorCode::AccessDenied, "the device is revoked")), - None => return Err(ErrorReply::new(ErrorCode::AccessDenied, "the device is not enrolled")), - }; - inner.used.insert(proof.endpoint, sequence); - Ok(enrollment) - } - - /// Create an enrollment key, returning it with its secret, which is never shown again. - pub fn create_key(&self, new: &NewKey) -> crate::Result { - let mut inner = self.inner.lock().unwrap(); - let id = hex(&random(8)); - let secret = encode(&random(32)); - let stored = StoredKey { - id: id.clone(), - hash: hex(&sha256(secret.as_bytes())), - tags: new.tags.clone(), - expires: new.expires_in.map(|ttl| kio::clock::wall().saturating_add(ttl)), - reusable: new.reusable, - used: false, - revoked: false, - }; - let mut info = KeyInfo::from(&stored); - info.key = Some(format!("{id}.{secret}")); - self.commit(&mut inner, |store| { - store.keys.push(stored); - Ok(()) - })?; - Ok(info) - } - - /// Every enrollment key, without secrets. - pub fn keys(&self) -> Vec { - self.inner - .lock() - .unwrap() - .store - .keys - .iter() - .map(KeyInfo::from) - .collect() - } - - /// Revoke an enrollment key so it enrolls nothing new. Devices it enrolled stay. - /// Returns whether the key exists. - pub fn revoke_key(&self, id: &str) -> crate::Result { - let mut inner = self.inner.lock().unwrap(); - let found = self.commit(&mut inner, |store| { - let key = store.keys.iter_mut().find(|key| key.id == id); - Ok(key.map(|key| key.revoked = true).is_some()) - })?; - Ok(found) - } - - /// Every device, revoked ones included. - pub fn devices(&self) -> Vec { - self.inner.lock().unwrap().store.devices.clone() - } - - /// Revoke a device: its next refresh fails, and it cannot enroll again. - /// Returns whether the device exists. - pub fn revoke_device(&self, endpoint: PublicKey) -> crate::Result { - let mut inner = self.inner.lock().unwrap(); - let found = self.commit(&mut inner, |store| { - let device = store.devices.iter_mut().find(|device| device.endpoint == endpoint); - Ok(device.map(|device| device.revoked = true).is_some()) - })?; - Ok(found) - } - - /// The device-facing contract, mounted at [`Config::url`]'s path. - pub fn router(&self) -> Router { - let router = Router::new() - .route("/challenge", post(challenge_handler)) - .route("/enroll", post(enroll_handler)) - .route("/refresh", post(refresh_handler)) - .with_state(self.clone()); - match self.config.url.path().trim_end_matches('/') { - "" => router, - base => Router::new().nest(base, router), - } - } - - /// The owner's API under `/mesh`, with no authentication of its own: mount it only on - /// a listener the owner alone reaches. - pub fn admin_router(&self) -> Router { - Router::new() - .route("/mesh/keys", get(keys_handler).post(create_key_handler)) - .route("/mesh/keys/{id}", delete(revoke_key_handler)) - .route("/mesh/devices", get(devices_handler)) - .route("/mesh/devices/{endpoint}", delete(revoke_device_handler)) - .with_state(self.clone()) - } - - /// Serve the device-facing contract on a TCP listener until the task is dropped. - pub async fn serve(&self, listener: tokio::net::TcpListener) -> std::io::Result<()> { - axum::serve(listener, self.router()).await - } - - /// Verify the proof's challenge and signature, returning the challenge's sequence. - /// - /// The caller records the sequence as the endpoint's last only once the request is - /// granted, so a refused proof changes nothing. - fn check(&self, inner: &Inner, proof: &Proof) -> Reply { - let invalid = |reason: &str| ErrorReply::new(ErrorCode::InvalidChallenge, reason); - let challenge = &proof.challenge; - if challenge.len() != CHALLENGE_LEN - || !constant_eq( - &self.tag(&challenge[..CHALLENGE_LEN - TAG_LEN]), - &challenge[CHALLENGE_LEN - TAG_LEN..], - ) { - return Err(invalid("unknown challenge")); - } - let expires = u64::from_be_bytes(challenge[..8].try_into().expect("8 bytes")); - let expires = kio::clock::Wall::UNIX_EPOCH.saturating_add(Duration::from_secs(expires)); - if expires <= kio::clock::wall() { - return Err(invalid("the challenge expired")); - } - if !proof.verify(&self.config.url) { - return Err(ErrorReply::new( - ErrorCode::InvalidSignature, - "the signature does not verify against the endpoint ID", - )); - } - let sequence = u64::from_be_bytes(challenge[8..16].try_into().expect("8 bytes")); - if inner.used.get(&proof.endpoint).is_some_and(|last| sequence <= *last) { - return Err(invalid("the endpoint already used this challenge or a newer one")); - } - Ok(sequence) - } - - /// The truncated HMAC sealing a challenge's expiry and nonce. - fn tag(&self, body: &[u8]) -> Vec { - rind::hmac::sign(&self.secret, body).as_ref()[..TAG_LEN].to_vec() - } - - /// Apply `change` to a copy of the store and save it, adopting the copy only once it - /// is on disk, so a failed save leaves no credential spent and no device admitted. - fn commit(&self, inner: &mut Inner, change: impl FnOnce(&mut Store) -> Reply) -> Reply { - let mut store = inner.store.clone(); - let out = change(&mut store)?; - self.save(&store)?; - inner.store = store; - Ok(out) - } - - /// Record `endpoint` as enrolled, or update its name and tags if it already is. - fn admit(&self, store: &mut Store, endpoint: PublicKey, name: Option<&str>, tags: Vec) -> Reply<()> { - refuse_revoked(store, endpoint)?; - let devices = &mut store.devices; - let current = devices.iter().position(|device| device.endpoint == endpoint); - let name = match (name, current) { - (None, Some(index)) => devices[index].name.clone(), - (name, _) => unique_name(devices, endpoint, &sanitize(name.unwrap_or_default())), - }; - match current { - Some(index) => { - devices[index].name = name; - devices[index].tags = tags; - } - None => devices.push(Device { - endpoint, - name, - tags, - enrolled: kio::clock::wall(), - revoked: false, - }), - } - Ok(()) - } - - /// Sign a token for an enrolled `endpoint` and list the mesh. - fn issue(&self, store: &Store, endpoint: PublicKey) -> Reply { - let internal = |err: crate::Error| ErrorReply::new(ErrorCode::ServerError, err.to_string()); - let now = kio::clock::wall(); - // `exp` is whole seconds on the wire; truncate so the reply matches the token. - let secs = now - .checked_duration_since(kio::clock::Wall::UNIX_EPOCH) - .unwrap_or_default() - .as_secs(); - let expires = kio::clock::Wall::UNIX_EPOCH.saturating_add(Duration::from_secs(secs) + self.config.token_ttl); - let token = self - .config - .key - .sign(&claims(endpoint, now, Some(expires))) - .map_err(internal)?; - let device = store - .devices - .iter() - .find(|device| device.endpoint == endpoint) - .expect("admitted before it is issued"); - Ok(Enrollment { - token, - expires, - relays: self.config.relays.clone(), - name: device.name.clone(), - members: store - .devices - .iter() - .filter(|device| !device.revoked) - .map(|device| Member { - endpoint: device.endpoint, - name: device.name.clone(), - tags: device.tags.clone(), - }) - .collect(), - }) - } - - /// Write the store atomically: a temporary file beside it, renamed over it. - fn save(&self, store: &Store) -> Reply<()> { - let Some(path) = &self.config.state else { - return Ok(()); - }; - let failed = |err: std::io::Error| { - tracing::error!(path = %path.display(), %err, "failed to save the mesh state"); - ErrorReply::new(ErrorCode::ServerError, "the coordinator could not save its state") - }; - let json = serde_json::to_string_pretty(store).expect("the store serializes"); - let tmp = path.with_extension("tmp"); - crate::fs::write_private(&tmp, &json).map_err(failed)?; - std::fs::rename(&tmp, path).map_err(failed) - } -} - -fn refuse_revoked(store: &Store, endpoint: PublicKey) -> Reply<()> { - if store - .devices - .iter() - .any(|device| device.endpoint == endpoint && device.revoked) - { - return Err(ErrorReply::new(ErrorCode::AccessDenied, "the device is revoked")); - } - Ok(()) -} - -/// The grant every mesh token carries: publish under its own ID, subscribe to the mesh. -fn claims(endpoint: PublicKey, issued: kio::clock::Wall, expires: Option) -> Claims { - Claims::default() - .with_publish([Pattern::subtree(&format!("mesh/{endpoint}")).expect("a z-base-32 segment is a valid path")]) - .with_subscribe([Pattern::subtree("mesh").expect("a valid path")]) - .with_issued(issued) - .with_expires(expires) -} - -/// `name` as one DNS label: lowercase ASCII letters, digits, and inner hyphens, at most -/// 63 bytes, `device` when nothing is left. -fn sanitize(name: &str) -> String { - let mut out = String::with_capacity(name.len().min(MAX_NAME)); - for c in name.chars() { - let c = c.to_ascii_lowercase(); - let c = if c.is_ascii_alphanumeric() { c } else { '-' }; - if c == '-' && (out.is_empty() || out.ends_with('-')) { - continue; - } - out.push(c); - } - out.truncate(MAX_NAME); - let out = out.trim_end_matches('-'); - if out.is_empty() { "device".into() } else { out.into() } -} - -/// `name`, or `name-2`, `name-3`, ... when another endpoint holds it. -fn unique_name(devices: &[Device], endpoint: PublicKey, name: &str) -> String { - let taken = |candidate: &str| { - devices - .iter() - .any(|device| device.endpoint != endpoint && device.name == candidate) - }; - if !taken(name) { - return name.into(); - } - (2..) - .map(|n| { - let suffix = format!("-{n}"); - let base = &name[..name.len().min(MAX_NAME - suffix.len())]; - format!("{}{suffix}", base.trim_end_matches('-')) - }) - .find(|candidate| !taken(candidate)) - .expect("an unbounded search finds a free name") -} - -/// base64url without padding, the encoding of every random secret in the contract. -fn encode(bytes: &[u8]) -> String { - use base64::Engine; - base64::engine::general_purpose::URL_SAFE_NO_PAD.encode(bytes) -} - -fn random(len: usize) -> Vec { - let mut bytes = vec![0u8; len]; - rind::rand::fill(&mut bytes).expect("system randomness"); - bytes -} - -fn sha256(bytes: &[u8]) -> Vec { - rind::digest::digest(&rind::digest::SHA256, bytes).as_ref().to_vec() -} - -fn constant_eq(a: &[u8], b: &[u8]) -> bool { - rind::constant_time::verify(a, b).is_ok() -} - -fn hex(bytes: &[u8]) -> String { - bytes.iter().map(|byte| format!("{byte:02x}")).collect() -} - -impl IntoResponse for ErrorReply { - fn into_response(self) -> Response { - // RFC 6749 §5.2: 400 for every OAuth error except a failed client authentication. - let status = match self.error { - ErrorCode::ServerError => StatusCode::INTERNAL_SERVER_ERROR, - _ => StatusCode::BAD_REQUEST, - }; - (status, Json(self)).into_response() - } -} - -/// A JSON body that refuses with the contract's error shape rather than axum's text. -struct Body(T); - -impl axum::extract::FromRequest for Body { - type Rejection = ErrorReply; - - async fn from_request(request: axum::extract::Request, state: &S) -> Result { - Json::::from_request(request, state) - .await - .map(|Json(body)| Self(body)) - .map_err(|err| ErrorReply::new(ErrorCode::InvalidRequest, err.body_text())) - } -} - -async fn challenge_handler(State(coordinator): State) -> Json { - Json(coordinator.challenge()) -} - -async fn enroll_handler( - State(coordinator): State, - Body(request): Body, -) -> Reply> { - coordinator.enroll(&request).map(Json) -} - -async fn refresh_handler(State(coordinator): State, Body(proof): Body) -> Reply> { - coordinator.refresh(&proof).map(Json) -} - -fn admin_error(err: crate::Error) -> Response { - (StatusCode::INTERNAL_SERVER_ERROR, err.to_string()).into_response() -} - -async fn keys_handler(State(coordinator): State) -> Json> { - Json(coordinator.keys()) -} - -async fn create_key_handler(State(coordinator): State, Body(new): Body) -> Response { - match coordinator.create_key(&new) { - Ok(info) => (StatusCode::CREATED, Json(info)).into_response(), - Err(err) => admin_error(err), - } -} - -async fn revoke_key_handler(State(coordinator): State, Path(id): Path) -> Response { - match coordinator.revoke_key(&id) { - Ok(true) => StatusCode::NO_CONTENT.into_response(), - Ok(false) => (StatusCode::NOT_FOUND, "not found").into_response(), - Err(err) => admin_error(err), - } -} - -async fn devices_handler(State(coordinator): State) -> Json> { - Json(coordinator.devices()) -} - -async fn revoke_device_handler(State(coordinator): State, Path(endpoint): Path) -> Response { - let Ok(endpoint) = endpoint.parse() else { - return (StatusCode::BAD_REQUEST, "invalid endpoint ID").into_response(); - }; - match coordinator.revoke_device(endpoint) { - Ok(true) => StatusCode::NO_CONTENT.into_response(), - Ok(false) => (StatusCode::NOT_FOUND, "not found").into_response(), - Err(err) => admin_error(err), - } -} - -#[cfg(test)] -mod tests { - use super::*; - use crate::{Algorithm, KeyId}; - - #[test] - fn names_are_dns_labels() { - assert_eq!(sanitize("Luke's MacBook Pro"), "luke-s-macbook-pro"); - assert_eq!(sanitize("--robot_01.local--"), "robot-01-local"); - assert_eq!(sanitize("日本"), "device"); - assert_eq!(sanitize(&"a".repeat(100)).len(), MAX_NAME); - } - - #[test] - fn names_stay_unique() { - let device = |byte, name: &str| Device { - endpoint: PublicKey::from_bytes([byte; 32]), - name: name.into(), - tags: Vec::new(), - enrolled: kio::clock::wall(), - revoked: false, - }; - let devices = vec![device(1, "robot"), device(2, "robot-2")]; - assert_eq!(unique_name(&devices, PublicKey::from_bytes([1; 32]), "robot"), "robot"); - assert_eq!( - unique_name(&devices, PublicKey::from_bytes([3; 32]), "robot"), - "robot-3" - ); - let long = "a".repeat(MAX_NAME); - let devices = vec![device(1, &long)]; - let name = unique_name(&devices, PublicKey::from_bytes([3; 32]), &long); - assert_eq!(name.len(), MAX_NAME); - assert!(name.ends_with("-2")); - } - - #[test] - fn a_challenge_is_single_use_per_endpoint() { - let key = Key::generate(Algorithm::HS256, None).unwrap(); - let base: url::Url = "https://mesh.example".parse().unwrap(); - let coordinator = Coordinator::new(Config::new(base.clone(), key)).unwrap(); - let secret = coordinator - .create_key(&NewKey { - reusable: true, - ..Default::default() - }) - .unwrap() - .key - .unwrap(); - let a = rind::signature::Ed25519KeyPair::generate().unwrap(); - let b = rind::signature::Ed25519KeyPair::generate().unwrap(); - let enroll = |signer: &rind::signature::Ed25519KeyPair, challenge: &Challenge| EnrollRequest { - proof: Proof::sign(signer, &base, challenge.challenge.clone()), - key: secret.clone(), - name: None, - }; - let older = coordinator.challenge(); - let newer = coordinator.challenge(); - - // A refused proof records nothing, so an unknown caller holds no state. - assert_eq!( - coordinator.refresh(&enroll(&a, &older).proof).unwrap_err().error, - ErrorCode::AccessDenied - ); - assert!(coordinator.inner.lock().unwrap().used.is_empty()); - - // Once `a` uses the newer challenge, its own older one is spent too. - coordinator.enroll(&enroll(&a, &newer)).unwrap(); - for challenge in [&newer, &older] { - let err = coordinator.enroll(&enroll(&a, challenge)).unwrap_err(); - assert_eq!(err.error, ErrorCode::InvalidChallenge); - } - - // Another endpoint's state is its own: `a` spending challenges costs `b` nothing. - coordinator.enroll(&enroll(&b, &older)).unwrap(); - coordinator.enroll(&enroll(&b, &newer)).unwrap(); - assert_eq!(coordinator.inner.lock().unwrap().used.len(), 2); - } - - #[test] - fn state_survives_a_restart() { - let dir = tempfile::tempdir().unwrap(); - let mut config = Config::new( - "https://mesh.example".parse().unwrap(), - Key::generate(Algorithm::HS256, Some(KeyId::decode("mesh").unwrap())).unwrap(), - ); - config.state = Some(dir.path().join("mesh.json")); - let coordinator = Coordinator::new(config.clone()).unwrap(); - let key = coordinator.create_key(&NewKey::default()).unwrap(); - assert!(coordinator.revoke_key(&key.id).unwrap()); - - let restarted = Coordinator::new(config).unwrap(); - let keys = restarted.keys(); - assert_eq!(keys.len(), 1); - assert!(keys[0].revoked); - assert_eq!(keys[0].key, None); - } -} diff --git a/rs/moq-auth/src/mesh/mod.rs b/rs/moq-auth/src/mesh/mod.rs deleted file mode 100644 index ae832bbb42f..00000000000 --- a/rs/moq-auth/src/mesh/mod.rs +++ /dev/null @@ -1,301 +0,0 @@ -//! Mesh enrollment: how a device with an endpoint key joins a mesh. -//! -//! A device proves it holds its Ed25519 endpoint key by signing a fresh challenge from -//! the coordinator, and presents an enrollment key the mesh owner created, so enrollment -//! needs no human in the loop. The coordinator answers with an -//! [`Enrollment`]: the relays to dial and a JWT granting publish on `mesh//**` and -//! subscribe on `mesh/**`. The device refreshes it before it expires by proving its key -//! again, so revoking a device makes its next refresh fail. -//! -//! These types are the JSON contract. [`Coordinator`] (feature `serve`) answers it behind -//! `moq auth serve`; [`Client`] (feature `client`) drives it from a device. A hosted -//! service implements the same contract with its own accounts. -//! -//! | Request | Body | Reply | -//! |---|---|---| -//! | `POST {base}/challenge` | none | [`Challenge`] | -//! | `POST {base}/enroll` | [`EnrollRequest`] | [`Enrollment`] | -//! | `POST {base}/refresh` | [`Proof`] | [`Enrollment`] | -//! -//! Every refusal is a 4xx with an [`ErrorReply`]. - -use std::fmt; - -use moq_crypto::sign::PublicKey; -use serde::{Deserialize, Serialize}; -use serde_with::serde_as; - -#[cfg(feature = "client")] -mod client; -#[cfg(all(test, feature = "client", feature = "serve"))] -mod contract_tests; -#[cfg(feature = "serve")] -mod coordinator; - -#[cfg(feature = "client")] -pub use client::*; -#[cfg(feature = "serve")] -pub use coordinator::*; - -/// The context string every [`Proof`] signature starts with, so a signature made for -/// enrollment can never be replayed as anything else the endpoint key signs. -const PROOF_CONTEXT: &[u8] = b"moq-mesh-enroll-00"; - -/// The length of a challenge in bytes. -#[cfg_attr( - not(feature = "serve"), - allow(dead_code, reason = "only the coordinator issues challenges") -)] -pub(crate) const CHALLENGE_LEN: usize = 32; - -const SIGNATURE_LEN: usize = 64; - -/// The message a [`Proof`] signs: [`PROOF_CONTEXT`], a zero byte, the coordinator's base -/// URL as the device addressed it, a zero byte, then the raw challenge. -/// -/// The URL binds the proof to one coordinator, so a coordinator the device trusts cannot -/// relay another's challenge and enroll the device's key there. -fn proof_message(base: &url::Url, challenge: &[u8]) -> Vec { - let base = audience(base); - let mut message = Vec::with_capacity(PROOF_CONTEXT.len() + base.len() + challenge.len() + 2); - message.extend_from_slice(PROOF_CONTEXT); - message.push(0); - message.extend_from_slice(base.as_bytes()); - message.push(0); - message.extend_from_slice(challenge); - message -} - -/// The base URL without a trailing slash, the form both sides sign over. -fn audience(base: &url::Url) -> &str { - base.as_str().trim_end_matches('/') -} - -/// A fresh, single-use challenge from `POST {base}/challenge`. -#[serde_as] -#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] -pub struct Challenge { - /// 32 bytes, opaque to the device, base64url without padding. - #[serde_as(as = "serde_with::base64::Base64")] - pub challenge: Vec, - /// When the coordinator stops accepting a proof over it, unix seconds. - #[serde_as(as = "kio::clock::serde::UnixSeconds")] - pub expires: kio::clock::Wall, -} - -/// Proof that the caller holds `endpoint`'s secret key: its signature over a challenge the -/// coordinator issued and has not yet seen used. -/// -/// The signed message is `moq-mesh-enroll-00`, a zero byte, the coordinator's base URL as -/// the device addressed it without a trailing slash, a zero byte, then the raw challenge. -#[serde_as] -#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] -pub struct Proof { - /// The endpoint ID, which is the key that signed. - pub endpoint: PublicKey, - /// The challenge, as issued. - #[serde_as(as = "serde_with::base64::Base64")] - pub challenge: Vec, - /// The Ed25519 signature over the challenge, bound to the coordinator's base URL. - #[serde_as(as = "serde_with::base64::Base64")] - pub signature: Vec, -} - -impl Proof { - /// A proof for `challenge` at the coordinator `base`, signed by `signer`. - pub fn sign(signer: &dyn Signer, base: &url::Url, challenge: Vec) -> Self { - let signature = signer.sign(&proof_message(base, &challenge)).to_vec(); - Self { - endpoint: signer.endpoint(), - challenge, - signature, - } - } - - /// Whether the signature verifies for the coordinator `base`. The challenge's - /// freshness is the coordinator's to check. - pub fn verify(&self, base: &url::Url) -> bool { - use rind::signature::{PublicKey, verify}; - self.signature.len() == SIGNATURE_LEN - && PublicKey::new(&verify::ED25519, self.endpoint.as_bytes()).is_ok_and(|key| { - key.verify(&proof_message(base, &self.challenge), &self.signature) - .is_ok() - }) - } -} - -/// An endpoint's secret key, held by whatever stores it. -/// -/// Implemented for [`rind::signature::Ed25519KeyPair`]; a daemon implements it for -/// its own key type. -pub trait Signer: Send + Sync { - /// The endpoint ID: this key's public half. - fn endpoint(&self) -> PublicKey; - /// The Ed25519 signature over `message`. - fn sign(&self, message: &[u8]) -> [u8; SIGNATURE_LEN]; -} - -impl Signer for rind::signature::Ed25519KeyPair { - fn endpoint(&self) -> PublicKey { - PublicKey::from_bytes( - self.public_key() - .as_ref() - .try_into() - .expect("a 32-byte Ed25519 public key"), - ) - } - - fn sign(&self, message: &[u8]) -> [u8; SIGNATURE_LEN] { - rind::signature::Ed25519KeyPair::sign(self, message) - .as_ref() - .try_into() - .expect("a 64-byte Ed25519 signature") - } -} - -/// `POST {base}/enroll`: join with a pre-authorized enrollment key. -#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] -pub struct EnrollRequest { - /// Proof of the endpoint key. - #[serde(flatten)] - pub proof: Proof, - /// The enrollment key the mesh owner created. - pub key: String, - /// The device's requested name, usually its hostname. The coordinator may change it to - /// keep names unique and DNS-safe; the [`Enrollment`] carries the one it assigned. - #[serde(default, skip_serializing_if = "Option::is_none")] - pub name: Option, -} - -/// What a device needs to join: the relays to dial and the token to present. -#[serde_as] -#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] -pub struct Enrollment { - /// A JWT granting publish on `mesh//**` and subscribe on `mesh/**`, rooted at `/`. - pub token: String, - /// When the token expires, unix seconds. Refresh before then. - #[serde_as(as = "kio::clock::serde::UnixSeconds")] - pub expires: kio::clock::Wall, - /// The relays to dial, presenting the token. - pub relays: Vec, - /// The name the coordinator assigned this device. - pub name: String, - /// Every enrolled member of the mesh, this device included. - pub members: Vec, -} - -/// One enrolled device, as listed in an [`Enrollment`]. -#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] -pub struct Member { - /// Its endpoint ID. - pub endpoint: PublicKey, - /// Its unique, DNS-safe name. - pub name: String, - /// The tags of the enrollment key it joined with. - #[serde(default, skip_serializing_if = "Vec::is_empty")] - pub tags: Vec, -} - -/// Why the coordinator refused, in the shape of an OAuth error response (RFC 6749 §5.2). -#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, thiserror::Error)] -#[error("{error}{}", description.as_deref().map(|d| format!(": {d}")).unwrap_or_default())] -#[non_exhaustive] -pub struct ErrorReply { - /// The machine-readable reason. - pub error: ErrorCode, - /// A human-readable detail. - #[serde(rename = "error_description", default, skip_serializing_if = "Option::is_none")] - pub description: Option, -} - -impl ErrorReply { - /// A refusal for `error` with a human-readable detail. - pub fn new(error: ErrorCode, description: impl Into) -> Self { - Self { - error, - description: Some(description.into()), - } - } -} - -/// The reasons a coordinator refuses. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)] -#[serde(rename_all = "snake_case")] -#[non_exhaustive] -pub enum ErrorCode { - /// The body did not parse. - InvalidRequest, - /// The challenge is unknown, used, or expired; fetch a new one. - InvalidChallenge, - /// The signature does not verify against the endpoint ID. - InvalidSignature, - /// The enrollment key is unknown, used, revoked, or expired. - InvalidGrant, - /// The coordinator is busy; retry shortly. - SlowDown, - /// The device is revoked or not enrolled. - AccessDenied, - /// The coordinator failed on its side; retry later. - ServerError, - /// A reason this version does not know. - #[serde(other)] - Unknown, -} - -impl fmt::Display for ErrorCode { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - let value = serde_json::to_value(self).expect("an enum serializes"); - f.write_str(value.as_str().unwrap_or("unknown")) - } -} - -#[cfg(test)] -mod tests { - use super::*; - - fn signer() -> rind::signature::Ed25519KeyPair { - rind::signature::Ed25519KeyPair::generate().unwrap() - } - - #[test] - fn endpoint_vectors() { - let vectors: serde_json::Value = - serde_json::from_str(include_str!("../../../../drafts/moq-sign-00.json")).unwrap(); - for row in vectors["keys"].as_array().unwrap() { - let text = row["endpoint"].as_str().unwrap(); - let member: Member = serde_json::from_value(serde_json::json!({"endpoint": text, "name": "node"})).unwrap(); - assert_eq!(member.endpoint.to_string(), text, "{}", row["id"]); - assert_eq!(serde_json::to_value(&member).unwrap()["endpoint"], text); - } - for row in vectors["invalid_ids"].as_array().unwrap() { - assert!( - serde_json::from_value::(serde_json::json!({"endpoint": row["text"], "name": "node"})).is_err(), - "{}", - row["id"] - ); - } - } - - #[test] - fn a_proof_is_bound_to_its_coordinator() { - let key = signer(); - let base: url::Url = "https://a.example/mesh".parse().unwrap(); - let proof = Proof::sign(&key, &base, vec![7; CHALLENGE_LEN]); - assert!(proof.verify(&base)); - assert!(proof.verify(&"https://a.example/mesh/".parse().unwrap())); - assert!(!proof.verify(&"https://b.example/mesh".parse().unwrap())); - - let mut forged = proof.clone(); - forged.endpoint = signer().endpoint(); - assert!(!forged.verify(&base)); - } - - #[test] - fn error_replies_read_unknown_codes() { - let reply: ErrorReply = serde_json::from_str(r#"{"error":"access_denied"}"#).unwrap(); - assert_eq!(reply.error, ErrorCode::AccessDenied); - let reply: ErrorReply = serde_json::from_str(r#"{"error":"new_thing","error_description":"x"}"#).unwrap(); - assert_eq!(reply.error, ErrorCode::Unknown); - assert_eq!(ErrorCode::SlowDown.to_string(), "slow_down"); - } -} diff --git a/rs/moq-auth/src/serve.rs b/rs/moq-auth/src/serve.rs index 12c474a6b4c..17a4c819177 100644 --- a/rs/moq-auth/src/serve.rs +++ b/rs/moq-auth/src/serve.rs @@ -675,7 +675,6 @@ fn hash(token: &str) -> u64 { pub struct Server { policy: Arc>>, sessions: Arc>, - mesh: Option, secret: Option>, } @@ -687,7 +686,6 @@ impl Server { Ok(Self { policy: Arc::new(std::sync::RwLock::new(Arc::new(Self::validate(policy)?))), sessions: Default::default(), - mesh: None, secret: None, }) } @@ -737,13 +735,6 @@ impl Server { self } - /// Also answer the mesh owner's API under `/mesh` with `coordinator`, behind the same - /// [`secret`](Self::with_secret) as `POST /`. - pub fn with_mesh(mut self, coordinator: crate::mesh::Coordinator) -> Self { - self.mesh = Some(coordinator); - self - } - /// The cadence the session table ages out by, or `None` when no limit needs a table. fn cadence(policy: &Policy) -> Option { let limits = policy.limits; @@ -784,14 +775,10 @@ impl Server { } } - /// The router: `POST /` reading a [`Request`] and writing a [`Grant`], plus the mesh - /// owner's API when [`with_mesh`](Self::with_mesh) set one, all behind the + /// The router: `POST /` reading a [`Request`] and writing a [`Grant`], behind the /// [`secret`](Self::with_secret) when there is one. pub fn router(&self) -> Router { - let mut router = Router::new().route("/", post(handle)).with_state(self.clone()); - if let Some(mesh) = &self.mesh { - router = router.merge(mesh.admin_router()); - } + let router = Router::new().route("/", post(handle)).with_state(self.clone()); match &self.secret { Some(secret) => router.layer(axum::middleware::from_fn_with_state(secret.clone(), guard)), None => router, @@ -1067,15 +1054,12 @@ mod tests { } #[tokio::test] - async fn the_secret_guards_every_route() { - let key = Key::generate(Algorithm::HS256, None).unwrap(); - let config = crate::mesh::Config::new("https://mesh.example".parse().unwrap(), key); + async fn the_secret_guards_the_route() { let server = Server::new(Policy { public: rules(&["**"], &[]), ..Default::default() }) .unwrap() - .with_mesh(crate::mesh::Coordinator::new(config).unwrap()) .with_secret("hunter2:x"); let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); let addr = listener.local_addr().unwrap(); @@ -1092,25 +1076,14 @@ mod tests { ("relay:hunter2%3Ax@", true), (":hunter2%3Ax@", true), ] { - let status = |path: &'static str| { - let url = format!("http://{userinfo}{addr}{path}"); - let http = http.clone(); - async move { - match path { - "/" => http.post(url).json(&request("/demo")).send().await, - _ => http.get(url).send().await, - } - .unwrap() - .status() - } - }; - for path in ["/", "/mesh/keys"] { - assert_eq!( - status(path).await == StatusCode::UNAUTHORIZED, - !admitted, - "{userinfo}{path}" - ); - } + let status = http + .post(format!("http://{userinfo}{addr}/")) + .json(&request("/demo")) + .send() + .await + .unwrap() + .status(); + assert_eq!(status == StatusCode::UNAUTHORIZED, !admitted, "{userinfo}"); } // The relay's client sends the secret from its URL's userinfo. @@ -1124,7 +1097,8 @@ mod tests { let credentials = base64::engine::general_purpose::STANDARD.encode("relay:hunter2:x"); for scheme in ["basic", "BASIC", "Basic", "Bearer"] { let status = http - .get(format!("http://{addr}/mesh/keys")) + .post(format!("http://{addr}/")) + .json(&request("/demo")) .header(reqwest::header::AUTHORIZATION, format!("{scheme} {credentials}")) .send() .await diff --git a/rs/moq-cli/src/auth.rs b/rs/moq-cli/src/auth.rs index 63f1a03c8d6..c2c47eab3e0 100644 --- a/rs/moq-cli/src/auth.rs +++ b/rs/moq-cli/src/auth.rs @@ -129,7 +129,6 @@ impl Args { println!("{payload:#?}"); } - Command::Mesh(mesh) => mesh.run().await?, Command::Sessions(filter) => filter.get("/sessions").await?, Command::Revalidate(filter) => filter.post("/sessions/revalidate").await?, } @@ -220,9 +219,6 @@ enum Command { /// Answer a relay's auth requests with keys, public rules, an mTLS grant, tiers, and session limits. Serve(Serve), - /// Manage a mesh served by `moq auth serve --mesh-url`: enrollment keys and devices. - Mesh(Mesh), - /// List live sessions on a relay's internal listener. Sessions(SessionFilter), @@ -453,10 +449,6 @@ pub struct Serve { #[usage(flatten)] #[serde(default)] limit: SessionLimits, - - #[usage(flatten)] - #[serde(default)] - mesh: MeshServe, } /// What an anonymous session may do. @@ -519,64 +511,6 @@ struct SessionLimits { remote: Option, } -/// The mesh coordinator served beside the auth server. -#[derive(usage_rs::Args, Clone, Debug, PartialEq, serde::Serialize, serde::Deserialize)] -#[usage(unknown_flags = "error", args_override_self = false)] -#[serde(deny_unknown_fields)] -struct MeshServe { - /// Coordinate a mesh: the public base URL devices enroll at, which every enrollment - /// proof is bound to. Serve it behind TLS; devices refuse plain `http://` beyond loopback. - #[usage(name = "mesh-url", long = "mesh-url", setting = "mesh.url")] - #[serde(default, skip_serializing_if = "Option::is_none")] - url: Option, - - /// Where to answer devices at `--mesh-url`, usually behind a TLS-terminating proxy. - #[usage( - name = "mesh-listen", - long = "mesh-listen", - default = "127.0.0.1:4441", - setting = "mesh.listen" - )] - listen: SocketAddr, - - /// The private key that signs mesh tokens; the relays verify it like any other. - #[usage(name = "mesh-key", long = "mesh-key", value_hint = usage_rs::ValueHint::FilePath, extensions("jwk", "json"), setting = "mesh.key")] - #[serde(default, skip_serializing_if = "Option::is_none")] - key: Option, - - /// A relay URL enrolled devices dial (repeatable). - #[usage(name = "mesh-relay", long = "mesh-relay", setting = "mesh.relay")] - #[serde(default)] - relay: Vec, - - /// The JSON file enrollment keys and devices persist to. Without one, a restart forgets them. - #[usage(name = "mesh-state", long = "mesh-state", value_hint = usage_rs::ValueHint::FilePath, setting = "mesh.state")] - #[serde(default, skip_serializing_if = "Option::is_none")] - state: Option, - - /// How long a mesh token lasts; devices refresh it two thirds of the way through. - #[usage( - name = "mesh-token-ttl", - long = "mesh-token-ttl", - default = "1h", - setting = "mesh.token_ttl" - )] - token_ttl: moq_tokio::cli::Duration, -} - -impl Default for MeshServe { - fn default() -> Self { - Self { - url: None, - listen: SocketAddr::from(([127, 0, 0, 1], 4441)), - key: None, - relay: Vec::new(), - state: None, - token_ttl: std::time::Duration::from_secs(3600).into(), - } - } -} - /// Every setting `moq auth serve` merges from its file, declared for the merge's registry. #[allow( dead_code, @@ -608,8 +542,6 @@ struct ServeSettings { expires: Option, #[usage(flatten)] limit: LimitSettings, - #[usage(flatten)] - mesh: MeshSettings, } #[allow( @@ -653,32 +585,8 @@ struct LimitSettings { remote: Option, } -#[allow( - dead_code, - reason = "declares the settings for the usage registry; nothing reads the fields" -)] -#[derive(usage_rs::Config)] -#[usage(prefix = "mesh")] -struct MeshSettings { - #[usage(cli("--mesh-url"))] - url: Option, - #[usage(cli("--mesh-listen"))] - listen: Option, - #[usage(cli("--mesh-key"))] - key: Option, - #[usage(cli("--mesh-relay"))] - relay: Option>, - #[usage(cli("--mesh-state"))] - state: Option, - #[usage(cli("--mesh-token-ttl"))] - token_ttl: Option, -} - impl Serve { /// These flags over the settings in [`file`](Self::file), read from disk now. - /// - /// A mesh setting without a mesh URL is refused here rather than by the parser, so - /// the URL can come from the file and the rest from the command line. fn merged(&self, cli: &usage_rs::config::CliLayer, env: &usage_rs::config::EnvLayer) -> anyhow::Result { let value = self .file @@ -694,7 +602,7 @@ impl Serve { .as_deref() .zip(value.as_ref()) .map(|(path, value)| moq_tokio::cli::FileSource { path, value }); - let (mut merged, resolved) = moq_tokio::cli::Merge { + let (mut merged, _) = moq_tokio::cli::Merge { registry: ServeSettings::SETTINGS_REGISTRY, cli, env, @@ -703,14 +611,6 @@ impl Serve { .apply(self.clone()) .map_err(|err| anyhow::anyhow!("{err}"))?; merged.file = self.file.clone(); - - if merged.mesh.url.is_none() { - for key in ["mesh.listen", "mesh.key", "mesh.relay", "mesh.state", "mesh.token_ttl"] { - if let Some(origin) = resolved.origin_key(key) { - anyhow::bail!("{} needs --mesh-url (mesh.url)", origin.describe()); - } - } - } Ok(merged) } @@ -823,33 +723,6 @@ impl Serve { Ok(Listen::Tcp(addr)) } - /// The mesh coordinator these settings describe, if `--mesh-url` asks for one. - fn coordinator(&self) -> anyhow::Result> { - let Some(url) = &self.mesh.url else { - return Ok(None); - }; - if self.listen_public { - anyhow::bail!( - "--mesh-url serves the mesh owner's API on --listen, which every relay holding --listen-secret could reach, so it cannot be combined with --listen-public" - ); - } - let path = self - .mesh - .key - .as_ref() - .context("--mesh-url needs --mesh-key to sign tokens")?; - let key = moq_auth::Key::from_file(path).with_context(|| format!("failed to read {}", path.display()))?; - let ttl = std::time::Duration::from(self.mesh.token_ttl); - if ttl.is_zero() { - anyhow::bail!("--mesh-token-ttl must be longer than 0s"); - } - let mut config = moq_auth::mesh::Config::new(url.clone(), key); - config.relays = self.mesh.relay.clone(); - config.state = self.mesh.state.clone(); - config.token_ttl = ttl; - Ok(Some(moq_auth::mesh::Coordinator::new(config)?)) - } - /// Re-read the file and answer with its policy, leaving `server` as it was when /// either is refused. fn apply( @@ -863,14 +736,9 @@ impl Serve { Ok(serve) } - /// The settings only a restart applies: the listeners, their secret, and the mesh. - fn restart_only(&self) -> (&str, bool, Option<&PathBuf>, &MeshServe) { - ( - &self.listen, - self.listen_public, - self.listen_secret.as_ref(), - &self.mesh, - ) + /// The settings only a restart applies: the listener and its secret. + fn restart_only(&self) -> (&str, bool, Option<&PathBuf>) { + (&self.listen, self.listen_public, self.listen_secret.as_ref()) } /// The secret in [`listen_secret`](Self::listen_secret), without its trailing newline. @@ -902,7 +770,7 @@ impl Serve { let reload = |quiet: bool| match self.apply(server, cli, &usage_rs::config::EnvLayer::from_process()) { Ok(serve) if serve.restart_only() != running.restart_only() => tracing::error!( file = %path.display(), - "reloaded the policy; listener and mesh changes take effect on restart" + "reloaded the policy; listener changes take effect on restart" ), Ok(_) if quiet => {} Ok(_) => tracing::info!(file = %path.display(), "reloaded the policy"), @@ -925,25 +793,7 @@ impl Serve { if let Some(secret) = serve.secret()? { server = server.with_secret(secret); } - // The device-facing contract runs beside the auth server; either stopping ends both. - let mut mesh = None; - if let Some(coordinator) = serve.coordinator()? { - let addr = serve.mesh.listen; - let listener = tokio::net::TcpListener::bind(addr) - .await - .with_context(|| format!("failed to bind {addr}"))?; - let addr = listener.local_addr().context("failed to read the bound address")?; - tracing::info!(%addr, url = ?serve.mesh.url, "mesh coordinator listening"); - mesh = Some((coordinator.clone(), listener)); - server = server.with_mesh(coordinator); - } let reload = self.reload(&serve, &server, cli); - let mesh = async move { - match mesh { - Some((coordinator, listener)) => coordinator.serve(listener).await, - None => std::future::pending().await, - } - }; let auth = async { match listen { Listen::Tcp(addr) => { @@ -970,133 +820,11 @@ impl Serve { }; tokio::select! { res = auth => res, - res = mesh => res.context("the mesh coordinator stopped"), res = reload => res, } } } -/// `moq auth mesh`: the mesh owner's commands, sent to the auth server's own listener. -#[derive(usage_rs::Args, Clone, Debug)] -#[usage(unknown_flags = "error", args_override_self = false)] -pub struct Mesh { - /// The auth server's listener, which serves the owner's API alongside `POST /`: - /// `http://` for a TCP `--listen`, or `unix:///path` for a unix socket. - /// With `--listen-secret`, put the secret in the URL as its password: `http://owner:SECRET@127.0.0.1:4440`. - #[usage(long, default = "http://127.0.0.1:4440")] - url: url::Url, - - #[usage(subcommand)] - command: MeshCommand, -} - -#[derive(usage_rs::Subcommands, Clone, Debug)] -enum MeshCommand { - /// Create an enrollment key, printing it once. - CreateKey { - /// A tag every device it enrolls carries (repeatable). - #[usage(long)] - tag: Vec, - - /// How long until it stops enrolling. Never, by default. - #[usage(long)] - expires: Option, - - /// Enroll any number of devices, instead of one. - #[usage(long)] - reusable: bool, - }, - - /// List enrollment keys, without their secrets. - Keys, - - /// Revoke an enrollment key so it enrolls nothing new. Its devices stay. - RevokeKey { - /// The key's ID: the part before the dot. - id: String, - }, - - /// List enrolled devices. - Devices, - - /// Revoke a device: its next refresh fails, and it cannot enroll again. - Revoke { - /// The device's endpoint ID. - endpoint: String, - }, -} - -impl Mesh { - async fn run(self) -> anyhow::Result<()> { - let (client, base) = self.client()?; - let url = |path: &str, segments: &[&str]| { - let mut url = base.clone(); - let prefix = url.path().trim_end_matches('/').to_string(); - url.set_path(&format!("{prefix}{path}")); - if let Ok(mut parts) = url.path_segments_mut() { - parts.extend(segments); - } - url - }; - let request = match &self.command { - MeshCommand::CreateKey { tag, expires, reusable } => { - let mut new = moq_auth::mesh::NewKey::default(); - new.tags = tag.clone(); - new.expires_in = expires.map(std::time::Duration::from); - new.reusable = *reusable; - client.post(url("/mesh/keys", &[])).json(&new) - } - MeshCommand::Keys => client.get(url("/mesh/keys", &[])), - MeshCommand::RevokeKey { id } => client.delete(url("/mesh/keys", &[id])), - MeshCommand::Devices => client.get(url("/mesh/devices", &[])), - MeshCommand::Revoke { endpoint } => client.delete(url("/mesh/devices", &[endpoint])), - }; - let response = request.send().await.with_context(|| { - let mut shown = self.url.clone(); - let _ = shown.set_password(None); - format!("failed to reach {shown}") - })?; - let status = response.status(); - let body = response.text().await.context("failed to read the reply")?; - if status == reqwest::StatusCode::NOT_FOUND { - // The routes exist only with `--mesh-url`, so a miss there is the likelier cause. - anyhow::bail!(match self.command { - MeshCommand::RevokeKey { .. } if !body.is_empty() => "no enrollment key has that ID", - MeshCommand::Revoke { .. } if !body.is_empty() => "no device has that endpoint ID", - _ => "not found; is the server running with --mesh-url?", - }); - } - if !status.is_success() { - anyhow::bail!("{status}: {body}"); - } - match serde_json::from_str::(&body) { - Ok(value) => println!("{}", serde_json::to_string_pretty(&value)?), - Err(_) if body.is_empty() => println!("ok"), - Err(_) => println!("{body}"), - } - Ok(()) - } - - /// The HTTP client for `--url` and the URL its requests target: a `unix://` URL names - /// the socket, and the requests go to the server's root over it. - fn client(&self) -> anyhow::Result<(reqwest::Client, url::Url)> { - let builder = reqwest::Client::builder(); - match self.url.scheme() { - "http" | "https" => Ok((builder.build()?, self.url.clone())), - #[cfg(unix)] - "unix" => { - let path = self - .url - .to_file_path() - .map_err(|()| anyhow::anyhow!("--url {} names no socket path", self.url))?; - let target = url::Url::parse("http://localhost/").expect("a constant URL parses"); - Ok((builder.unix_socket(path).build()?, target)) - } - _ => anyhow::bail!("--url expects http://, https://, or unix://, got {}", self.url), - } - } -} - #[derive(Debug)] enum Listen { Tcp(SocketAddr), @@ -1289,102 +1017,6 @@ mod tests { assert_eq!(merged.key, Some(key)); } - #[test] - fn mesh_flags_need_a_mesh_url_and_key() { - let err = merged(&["moq", "auth", "serve", "--mesh-relay", "https://relay.example"]) - .err() - .unwrap(); - assert!(err.to_string().contains("--mesh-relay needs --mesh-url"), "{err}"); - - let flags = serve(&["moq", "auth", "serve", "--mesh-url", "https://mesh.example"]); - let err = flags.coordinator().err().expect("no signing key"); - assert!(err.to_string().contains("--mesh-key"), "{err}"); - - let dir = tempfile::tempdir().unwrap(); - let key = dir.path().join("mesh.jwk"); - moq_auth::Key::generate(Algorithm::HS256, None) - .unwrap() - .to_file(&key) - .unwrap(); - let flags = serve(&[ - "moq", - "auth", - "serve", - "--mesh-url", - "https://mesh.example", - "--mesh-key", - key.to_str().unwrap(), - "--mesh-relay", - "https://a.example", - "--mesh-relay", - "https://b.example", - ]); - assert_eq!(flags.mesh.relay.len(), 2); - assert!(flags.mesh.listen.ip().is_loopback()); - assert!(flags.coordinator().unwrap().is_some()); - assert!(serve(&["moq", "auth", "serve"]).coordinator().unwrap().is_none()); - - // The owner's API rides --listen, so a public one would hand out enrollment keys. - let flags = serve(&[ - "moq", - "auth", - "serve", - "--listen", - "0.0.0.0:4440", - "--listen-public", - "--mesh-url", - "https://mesh.example", - "--mesh-key", - key.to_str().unwrap(), - ]); - let err = flags.coordinator().err().expect("a public owner API"); - assert!(err.to_string().contains("--listen-public"), "{err}"); - } - - #[cfg(unix)] - #[tokio::test] - async fn mesh_commands_reach_a_unix_listener() { - let dir = tempfile::tempdir().unwrap(); - let key = moq_auth::Key::generate(Algorithm::HS256, None).unwrap(); - let config = moq_auth::mesh::Config::new("https://mesh.example".parse().unwrap(), key); - let coordinator = moq_auth::mesh::Coordinator::new(config).unwrap(); - let server = moq_auth::serve::Server::new(Policy::default()) - .unwrap() - .with_mesh(coordinator.clone()); - let path = dir.path().join("auth.sock"); - let listener = tokio::net::UnixListener::bind(&path).unwrap(); - tokio::spawn(async move { server.serve_unix(listener).await }); - - let url = format!("unix://{}", path.display()); - run(&["moq", "auth", "mesh", "--url", &url, "create-key", "--expires", "1d"]) - .await - .unwrap(); - let keys = coordinator.keys(); - assert_eq!(keys.len(), 1); - assert!(keys[0].expires.is_some()); - } - - #[test] - fn mesh_commands_parse() { - for args in [ - &[ - "moq", - "auth", - "mesh", - "create-key", - "--tag", - "robot", - "--reusable", - "--expires", - "1d", - ][..], - &["moq", "auth", "mesh", "devices"], - &["moq", "auth", "mesh", "--url", "http://127.0.0.1:9000", "revoke", "ab"], - ] { - assert!(matches!(parse(args).unwrap().command, Command::Mesh(_)), "{args:?}"); - } - } - #[test] fn serve_refuses_a_public_bind_without_saying_so() { let err = serve(&["moq", "auth", "serve", "--listen", "0.0.0.0:4440"]) @@ -1547,35 +1179,18 @@ link = "peer" publish = ["alice/**"] link = "none" publish_rate = 1000000 - -[mesh] -relay = ["https://relay.example"] "# ), ) .unwrap(); let path = file.to_str().unwrap(); - let err = merged(&["moq", "auth", "serve", path]).err().unwrap(); - assert!(err.to_string().contains("mesh.relay needs --mesh-url"), "{err}"); - - let (serve, _) = merged(&[ - "moq", - "auth", - "serve", - path, - "--tier", - "from-flag", - "--mesh-url", - "https://mesh.example", - ]) - .unwrap(); + let (serve, _) = merged(&["moq", "auth", "serve", path, "--tier", "from-flag"]).unwrap(); assert!(matches!(serve.listener(), Ok(Listen::Tcp(addr)) if !addr.ip().is_loopback())); assert_eq!( serve.listen_secret.as_deref(), Some(std::path::Path::new("/etc/moq/auth.secret")) ); - assert_eq!(serve.mesh.relay.len(), 1); let policy = serve.policy().unwrap(); assert_eq!(policy.tier.as_deref(), Some("from-flag")); assert_eq!(policy.revalidate, Some(std::time::Duration::from_secs(30))); @@ -1588,17 +1203,7 @@ relay = ["https://relay.example"] assert_eq!(rule.publish_rate, moq_auth::Rate::from_bps(1_000_000).ok()); // A flag replaces the file's list rather than adding to it. - let (serve, _) = merged(&[ - "moq", - "auth", - "serve", - path, - "--mtls-subscribe", - "room/**", - "--mesh-url", - "https://mesh.example", - ]) - .unwrap(); + let (serve, _) = merged(&["moq", "auth", "serve", path, "--mtls-subscribe", "room/**"]).unwrap(); assert_eq!(serve.mtls.subscribe, vec!["room/**".parse::().unwrap()]); for (body, needs) in [ diff --git a/rs/moq-json/Cargo.toml b/rs/moq-json/Cargo.toml index 253f71ccea0..f47346a10d4 100644 --- a/rs/moq-json/Cargo.toml +++ b/rs/moq-json/Cargo.toml @@ -60,10 +60,5 @@ harness = false name = "window" harness = false -# Run the documented example's inline test in the normal nextest gate. -[[example]] -name = "teleop" -test = true - [lints] workspace = true diff --git a/rs/moq-json/examples/teleop.rs b/rs/moq-json/examples/teleop.rs deleted file mode 100644 index c55e5b891b8..00000000000 --- a/rs/moq-json/examples/teleop.rs +++ /dev/null @@ -1,80 +0,0 @@ -//! Carry replaceable control state and ordered commands on separate MoQ tracks. -//! -//! Run `cargo run -p moq-json --example teleop`. This in-process example teaches the -//! framing; attach the same broadcast to a moq-tokio origin to carry it over a session. -use std::task::Poll; - -use moq_json::Timed; -use moq_json::{snapshot, stream}; -use moq_net::{Timescale, Timestamp, broadcast, track}; -use serde::{Deserialize, Serialize}; - -#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] -struct Control { - speed: i32, - turn: i32, -} - -#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] -enum Command { - Arm, - Stop, - Disarm, -} - -fn main() -> Result<(), Box> { - let broadcast = broadcast::Info::new().produce(); - let info = track::Info::default().with_timescale(Timescale::MILLI); - let state = broadcast.create_track("control.json", info.clone())?; - let commands = broadcast.create_track("commands.json", info)?; - let mut state_reader = snapshot::Consumer::::new(state.consume(), Default::default()); - let mut command_reader = stream::Consumer::::new(commands.subscribe(None), Default::default()); - let mut state = snapshot::Producer::new(state, Default::default()); - let mut commands = stream::Producer::new(commands, Default::default()); - - // The reader falls behind. Replaying the old steering values would be harmful. - for (tick, control) in [ - Control { speed: 1, turn: -1 }, - Control { speed: 2, turn: 1 }, - Control { speed: 0, turn: 0 }, - ] - .iter() - .enumerate() - { - state.update(Timed::new(control, Timestamp::from_millis(tick as u64)?))?; - } - let _state = state.finish()?; - - // Every discrete command is meaningful, including Stop between Arm and Disarm. - let expected = [Command::Arm, Command::Stop, Command::Disarm]; - for (tick, command) in expected.iter().enumerate() { - commands.append(Timed::new(command, Timestamp::from_millis(tick as u64)?))?; - } - let _commands = commands.finish()?; - - let waiter = kio::Waiter::noop(); - // `latest` skips the stale steering values; `next` would replay them in order. - let Poll::Ready(Ok(Some(latest))) = state_reader.poll_latest(&waiter) else { - return Err("finished control track did not yield its latest state".into()); - }; - assert_eq!(latest.value, Control { speed: 0, turn: 0 }); - assert_eq!(latest.at, Timestamp::from_millis(2)?); - println!("Current control: {:?} at {:?}", latest.value, latest.at); - assert!(matches!(state_reader.poll_latest(&waiter), Poll::Ready(Ok(None)))); - for (tick, expected) in expected.into_iter().enumerate() { - let Poll::Ready(Ok(Some(command))) = command_reader.poll_next(&waiter) else { - return Err("finished command track lost an ordered command".into()); - }; - assert_eq!(command.value, expected); - assert_eq!(command.at, Timestamp::from_millis(tick as u64)?); - println!("Command: {:?} at {:?}", command.value, command.at); - } - assert!(matches!(command_reader.poll_next(&waiter), Poll::Ready(Ok(None)))); - Ok(()) -} - -#[cfg(test)] -#[test] -fn latest_state_and_every_command() { - main().unwrap(); -} diff --git a/rs/moq-json/src/snapshot/producer.rs b/rs/moq-json/src/snapshot/producer.rs index 0b840024fc4..72c1327d824 100644 --- a/rs/moq-json/src/snapshot/producer.rs +++ b/rs/moq-json/src/snapshot/producer.rs @@ -125,7 +125,7 @@ impl Producer { /// # fn example(producer: &mut moq_json::snapshot::Producer) -> moq_json::Result<()> { /// producer.mutate(|catalog| { /// catalog["scte35"] = serde_json::json!({ "id": 1 }); - /// }, None) + /// }, moq_net::Timestamp::from_millis(100).unwrap()) /// # } /// ``` ///