diff --git a/quest/m1/3046-fold-moq-token-into-moq-token-via-a-usage-executable-view.md b/quest/m1/3046-fold-moq-token-into-moq-token-via-a-usage-executable-view.md deleted file mode 100644 index 9f0afd20ad6..00000000000 --- a/quest/m1/3046-fold-moq-token-into-moq-token-via-a-usage-executable-view.md +++ /dev/null @@ -1,41 +0,0 @@ -# [M] Retire the standalone moq-token binary - -## Goal - -`moq token` is the only spelling of the token CLI: one binary, one help tree, -one completion tree, one release artifact. After one deprecation release the -`moq-token-cli` crate, its release workflow, and its packaging are deleted. -The `moq-token` library is untouched. - -## Plan - -`moq-token-cli` is a lib+bin. Its `Args` is nested by `moq-cli` as `moq token` -(rs/moq-cli/src/main.rs:197-199, the only consumer, rs/moq-cli/Cargo.toml:101) -and wrapped by the standalone binary's own `Root` (rs/moq-token-cli/src/lib.rs: -12-22). The logic is shared; the surface is duplicated: two spec roots, two -help renderings, two completion trees, two release artifacts, and two places -for a flag to drift. The docs already point at `moq token` (#3557). - -usage-rs 6.3.0 can dispatch a second root off argv0 (executable views, -`executable_views_emit_and_dispatch_from_argv0` in its tests/facade.rs), so -`moq-token` could survive as a renamed copy of `moq`. That keeps a name nobody -depends on and makes it an alias of the full media router, so the binary goes -instead. - -1. Deprecation release: `moq-token` prints a notice naming `moq token` on - every run and keeps working. Ship it as an ordinary `moq-token-cli` patch. -2. Delete the crate. Move `Args` and its subcommands into `moq-cli`, and - remove `.github/workflows/moq-token-cli.yml`, - `packaging/moq-token-cli/nfpm.yaml`, - `.github/homebrew/Formula/moq-token-cli.rb.tmpl`, and the - `moq-token-cli` triggers and entries in `.github/workflows/docker.yml:8`, - `cachix.yml:7,47`, `alert.yml:37`, `release-brew.yml:22,152`, and - `release-winget.yml:28,92`. -3. Grep the repo for `moq-token ` invocations and `moq-token-cli`, and fix - every remaining doc, demo recipe, and install page. - -Branch from `dev`, where the CLI lives on `usage`. - -## Closes - -- [#3046](https://github.com/moq-dev/moq/issues/3046) - close this issue when the quest finishes diff --git a/quest/m1/README.md b/quest/m1/README.md index de21a29b9c2..25a5b623173 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -45,7 +45,8 @@ with the dev tree. - [A/V clock](/quest/m1/plan-av-clock.md) - the audio playhead drives Sync.reference while audio plays, through per-track sync handles - [Config provenance](/quest/m1/config-provenance.md) - the merge records which source set a value, so an empty TOML list survives the environment and env outranks the file - [Cluster construction](/quest/m1/cluster-construction.md) - construct one stable origin after its cache settings are known, deleting the rebuilding builder -- [#3046](/quest/m1/3046-fold-moq-token-into-moq-token-via-a-usage-executable-view.md) - retire the standalone moq-token binary after one deprecation release; `moq token` is the only spelling +- [LAN discovery app id](/quest/m1/lan-app.md) - every advertisement names an application as a DNS-SD subtype bound into the proofs, so unrelated apps on one network never meet +- [One LAN mesh](/quest/m1/lan-mesh.md) - moq-cli drives the relay's Cluster, LAN peers authenticate by mDNS credential, and the two binaries mesh with each other - [Native Go context](/quest/m1/go-native-context.md) - the Go generator emits context.Context itself, retiring the hand-rolled cancellation token - [#2152](/quest/m1/2152-libmoq-c-abi-catch-up-with-the-moq-ffi-surface.md) - libmoq serves tracks on demand and accepts sessions, the two moq-ffi calls C still lacks - [Transport feature](/quest/m1/tokio-transport-feature.md) - moq-tokio has one `_transport` gate and its backend-less build passes `-D warnings` diff --git a/quest/m1/lan-app.md b/quest/m1/lan-app.md new file mode 100644 index 00000000000..d9824e1f395 --- /dev/null +++ b/quest/m1/lan-app.md @@ -0,0 +1,57 @@ +# [S] LAN discovery app id + +## Goal + +Two unrelated MoQ applications on one network never discover each other. Every +LAN advertisement names an application; a peer from another application is +never resolved or reported, and the membership proofs are bound to the +application, so one secret reused across two applications still yields two +meshes. moq-cli and moq-relay share one default, so they find each other with +no configuration. + +Non-goals: hiding the record (mDNS is multicast in the clear either way) and +any change to how a discovered peer is dialed. + +## Plan + +`rs/moq-tokio/src/mdns.rs` registers every process under one DNS-SD type, +`_moq._udp.local.`, and tells groups apart only after resolving, by whether the +HMAC proof verifies. Two applications with no secret therefore mesh with each +other, and two with different secrets still resolve and log every foreign +record. + +- `mdns::App`: a newtype over 1..=62 lowercase ASCII letters, digits, and + hyphens, with `FromStr` and `Display`, refusing anything else at parse time. + 62, not 63: the subtype label is `_`, and a DNS label holds 63 octets, + so the boundary test refuses a 63-character app rather than letting + registration fail later. + `impl Default` returns `default`, the name both binaries advertise under. + `Config::new(app, port)` replaces `Config::new(port)`; moq-tokio is + unpublished, so no shim. +- Partition: register the service under the DNS-SD subtype + `_._sub._moq._udp.local.` (RFC 6763 section 7.1) and browse that + subtype, so the daemon only ever resolves this application's records. + mdns-sd 0.20 supports it: the subtype rides `ServiceInfo::new`'s type + argument and `browse` takes the same string. `_moq._udp` stays the parent + type, so `dns-sd -B _moq._udp` still lists every MoQ process on the network. +- Bind: `Identity` gains the app and both proofs cover it, so a record cannot + be replayed under another application's subtype and the same secret under + two apps never verifies. Extend `proofs_are_bound_to_one_listener`. +- Surfaces: moq-cli gains `--cluster-lan-app` (`MOQ_CLUSTER_LAN_APP`) beside + `--cluster-lan-secret` in `rs/moq-cli/src/cluster.rs`; moq-relay gains + `[cluster.lan] app` (`--cluster-lan-app`) in `LanConfig`. Both default to + `default` and document that an application built on the library picks its + own name. Update `doc/bin/relay/cluster.md` and `doc/bin/relay/config.md`; + `doc/bin/cli.md` does not mention `--cluster-lan` at all, so add the flags + there while reconciling against `--help`. +- Test: there is no mDNS test today. Add one that runs two `Discovery` + instances on the host under different apps and asserts neither reports the + other, then two under the same app and secret that do. Skip cleanly when no + interface can announce, the way `ANNOUNCE_TIMEOUT` already fails, so CI + without multicast stays green rather than silently passing. + +Branch from `dev`; mDNS exists only there. + +## Related + +- [One LAN mesh](/quest/m1/lan-mesh.md) - consolidates the flag surface this adds into the relay's `ClusterConfig` diff --git a/quest/m1/lan-mesh.md b/quest/m1/lan-mesh.md new file mode 100644 index 00000000000..b025b18fabb --- /dev/null +++ b/quest/m1/lan-mesh.md @@ -0,0 +1,67 @@ +# [L] One LAN mesh + +## Goal + +A `moq --cluster-lan` process and a `moq-relay` with `[cluster.lan]` on the +same network mesh with each other, in both directions, through one +implementation. A LAN peer authenticates with its mDNS credential and is never +handed `cluster.token`; the secret is optional for both, and without one the +mesh is open and documented as such. + +## Plan + +Two meshes share the `_moq._udp` service today and cannot connect. The CLI +(`rs/moq-cli/src/cluster.rs`) advertises a fingerprint and no node, dials +`/.cluster/` with the fingerprint pinned, and attaches one +unauthenticated origin. The relay (`rs/moq-relay/src/cluster.rs`) always +advertises a node, skips any peer without one, dials the node URL with +`?jwt=` under ordinary TLS trust, and never reads +`Peer::credential` or `Peer::fingerprint`. A CLI dialing a relay lands as an +anonymous client at the node URL and is refused unless `--auth-public` happens +to cover it. + +One implementation: the relay's `Cluster`. + +- moq-cli depends on the moq-relay library; its `cluster-lan` feature becomes + `moq-relay/cluster-lan`. `MoqSide.cluster` nests `moq_relay::ClusterConfig`, + so the CLI's `--cluster-*` flags are the relay's, LAN and WAN alike, and the + stages publish and subscribe on the cluster's origin (the one + [cluster construction](/quest/m1/cluster-construction.md) builds once). + Delete `rs/moq-cli/src/cluster.rs`. +- Advertise: the listener's fingerprint whenever its certificate was + generated, the node URL when one is configured, and at least one of them + (the `UnboundSecret` rule). A relay without `--cluster-node` advertises its + listen port, the way the CLI does, so the `--cluster-lan` needs + `--cluster-node` start-up check goes away. +- Dial: `Peer::urls()` in order, node first, on the path + `/.cluster/`, with the advertised fingerprint pinned. The lower + id dials. Never `?jwt=`; `cluster.token` is for static and gossip peers only. + The request path needs moq-lite-05 or any moq-transport version, so the + CLI's `validate_versions` moves with the dial. +- Accept: `Connection::authenticate` recognizes `/.cluster/`, + verifies it with `Discovery::verify_credential` (constant time), and grants + the cluster-peer scope a `cluster.token` would, recording the hop. Every + other path stays on the JWT, mTLS, and public-prefix path. A `/.cluster` + request on a relay without LAN discovery is refused, not routed. +- Secret: optional for the relay too, since a LAN dial no longer carries the + token; `doc/bin/relay/cluster.md` inherits the CLI's warning that an open + mesh admits anyone who can reach the listener. Drop the relay's + secret-required start-up check and reconcile + [cluster flags](/quest/m2/cluster-flags.md), which lists both removed checks. +- Docs: `doc/bin/relay/cluster.md`, `doc/bin/relay/config.md`, and the CLI + page describe one mesh; the CLI page names the WAN flags it gains. +- Test: an in-process test meshes a node-advertising cluster with a + fingerprint-advertising one and asserts a broadcast crosses in each + direction. A smoke recipe runs `moq import --cluster-lan` beside `moq-relay` + with `[cluster.lan]` and watches from the relay. + +Branch from `dev`. + +## Required + +- [LAN discovery app id](/quest/m1/lan-app.md) - the flag surface this consolidates includes `app` + +## Related + +- [Cluster construction](/quest/m1/cluster-construction.md) - the origin the CLI's stages attach to +- [`moq relay`](/quest/m2/moq-relay-subcommand.md) - the second place the CLI hosts the relay library diff --git a/quest/m2/README.md b/quest/m2/README.md index 461e8bb5735..2d735c82466 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -100,6 +100,8 @@ targeting a `0.0.x` crate. - [#2907](/quest/m2/2907-bind-the-browser-through-moq-ffi-uniffi-instead-of-a.md) - Bind the browser through moq-ffi/UniFFI instead of a second hand-written wasm API - [#2850](/quest/m2/2850-js-net-give-reader-a-synchronous-decode-so-the-publisher.md) - js/net: decode messages synchronously from buffered bytes and delete the publisher read-ahead queue (dev) - [Cluster flags](/quest/m2/cluster-flags.md) - a discovery mechanism carries its own prerequisites, so an incomplete cluster config cannot be expressed +- [`moq relay`](/quest/m2/moq-relay-subcommand.md) - the relay runs under a `moq` verb with its own flags and TOML, while `moq-relay` stays a minimal binary +- [`moq` serves like a relay](/quest/m2/cli-serve.md) - a `moq --listen` session is authenticated, scoped, counted, and drained like a relay's; the relay is `moq` listening by default - [Revalidation updates](/quest/m2/revalidation-updates.md) - an auth re-check moves the tier in place and names an alias change when it closes the session - [#3137](/quest/m2/3137-moqsrc-bound-the-pending-rendition-subscriptions-a.md) - moqsrc: bound the pending rendition subscriptions a catalog can open - [#3115](/quest/m2/3115-moqsink-the-publication-has-no-generation-so-a-flush.md) - moqsink: a flushing restart after EOS opens a new publication generation diff --git a/quest/m2/cli-serve.md b/quest/m2/cli-serve.md new file mode 100644 index 00000000000..6ce83264224 --- /dev/null +++ b/quest/m2/cli-serve.md @@ -0,0 +1,46 @@ +# [L] The CLI serves like a relay + +## Goal + +The only difference between `moq` and `moq-relay` is that the relay listens by +default. A `moq --listen` session is authenticated (JWT, mTLS, or public +prefixes), scoped by its token, recorded as a hop, counted in stats, and +drained on shutdown, exactly as on a relay. The CLI's stages are a local +publisher and subscriber on that same origin. + +## Plan + +Today `spawn_server`, `route_server`, and `spawn_serve` +(`rs/moq-cli/src/main.rs`) accept every request with `request.ok()`, attach +the process's single origin in whichever directions the stages use, and never +look at a path or token. The relay's `serve` and `Connection::run` +(`rs/moq-relay/src/relay.rs`, `connection.rs`) do everything listed in the +goal. Once [one LAN mesh](/quest/m1/lan-mesh.md) lands the CLI's origin is the +cluster's, so the relay's `serve` can take over the listener directly. + +- Replace the CLI's three server helpers with `moq_relay::serve` over the + cluster, the auth, and the shutdown handle. `MoqSide` nests + `moq_relay::AuthConfig` and `StatsConfig`, so `--auth-*` and `--stats-*` + read the same on both binaries. +- Default grant: with no auth configured the CLI keeps what it gives today, an + open listener over the whole origin, spelled as the relay's public prefixes + covering the root. The relay's own default stays a refusal, since that is + its `no auth-key, auth-key-dir, auth-api, or public path configured` check; + the difference is the listening default this quest is about, and both defaults + are documented side by side. +- The Unix peer-credential gate on `--listen-unix-bind` and the certificate + endpoint `web::run_web` spawns for an explicit `--listen` survive, rehomed + on the relay's web server or kept as the CLI's, whichever leaves one copy. +- Drain: `moq import --listen` on SIGTERM sends GOAWAY and waits + `drain_timeout` like the relay, instead of dropping sessions. +- Docs: `doc/bin/cli.md` describes serving in the relay's terms and links the + relay's auth page rather than restating it. +- Test: an authenticated viewer with a scoped token sees only its root on a + `moq import --listen`, and an unscoped one is refused when a key is + configured; the existing unauthenticated import-to-export smoke keeps + passing with the default grant. + +## Required + +- [One LAN mesh](/quest/m1/lan-mesh.md) - the CLI's origin becomes the cluster's +- [`moq relay`](/quest/m2/moq-relay-subcommand.md) - the CLI hosts the whole relay library diff --git a/quest/m2/cluster-flags.md b/quest/m2/cluster-flags.md index e0737ecb3ad..4df359fa3f8 100644 --- a/quest/m2/cluster-flags.md +++ b/quest/m2/cluster-flags.md @@ -20,10 +20,16 @@ it cannot work without: unauthenticated and any advertiser on the network would otherwise be dialed and handed `--cluster-token`. +[One LAN mesh](/quest/m1/lan-mesh.md) deletes the last two: a LAN peer +advertises its listener and fingerprint when it has no node URL, and is dialed +with its mDNS credential rather than the token, so neither flag is a +prerequisite any more and the secret is optional. What remains is the mesh +rule, plus the LAN fields (`secret`, `app`) that belong to the LAN mechanism. + Each is a real rule with a good error message, and each exists because the flag shape allows a state that has no meaning. Make the mechanism's value -carry its prerequisite instead: mesh and LAN both need this relay's own URL, -and LAN needs its secret, so those belong to the mechanism rather than sitting +carry its prerequisite instead: mesh needs this relay's own URL, and LAN owns +its secret and app, so those belong to the mechanism rather than sitting beside it as an independent flag another mechanism might or might not want. A compile error beats a runtime check, and an operator finding out at startup that a flag needed a companion is the failure mode this removes. @@ -35,9 +41,13 @@ so they belong where they are. Every rename is user-facing, so old spellings stay as hidden aliases per the deprecation rules: no `--help` entry, no "deprecated, use X" note, and `/doc` examples updated to the new names only. The flags are TOML keys too, so this -is a config-file migration as well as a CLI one, and `--cluster-node` is read -by `moq-relay` alone. Grep the binary name repo-wide and reconcile every -sample invocation under `doc/bin/`, `doc/setup/`, and `demo/` against -`--help`. +is a config-file migration as well as a CLI one, and once moq-cli nests +`ClusterConfig` the same flags are read by both binaries. Grep both binary +names repo-wide and reconcile every sample invocation under `doc/bin/`, +`doc/setup/`, and `demo/` against `--help`. Leave `--cluster-linger` alone: it is already a hidden no-op. + +## Required + +- [One LAN mesh](/quest/m1/lan-mesh.md) - settles which LAN checks survive and moves the flags into moq-cli too diff --git a/quest/m2/moq-relay-subcommand.md b/quest/m2/moq-relay-subcommand.md new file mode 100644 index 00000000000..4ec22f46907 --- /dev/null +++ b/quest/m2/moq-relay-subcommand.md @@ -0,0 +1,49 @@ +# [S] Relay verb + +## Goal + +`moq relay` runs the relay with exactly the flags, environment, and TOML +`moq-relay` accepts, and `moq-relay` stays published as its own minimal +binary: someone who only wants a relay does not download the capture, encode, +decode, and render stack. + +## Plan + +The same shape `moq token` already uses: `moq-token-cli` is lib+bin, its +`Args` nested by moq-cli (`rs/moq-cli/src/main.rs`), its binary untouched. +`moq-relay` is lib+bin as well (`rs/moq-relay/Cargo.toml`), with a 15-line +`main.rs` that installs the aws-lc-rs provider, optionally jemalloc, and calls +`Relay::load(Config::load()?).await?.run()`. + +- moq-cli gains a default-on `relay` feature depending on the moq-relay + library and a `Command::Relay(moq_relay::Config)` verb, answered before any + transport binds like `token` and `completion`, with `MoqSide::reject` + covering it. The relay's own features (`iroh`, `cluster-lan`, backends, + `websocket`) forward from the CLI's features of the same name. +- `Config::load` parses argv, reads the TOML `file` names, and merges the + layers in one step (`rs/moq-relay/src/config.rs`, `parse_and_merge`), so a + nested `Config` would carry only the flags and ignore its TOML. Split the + file read and precedence merge into a step that takes an already-parsed + `Config`, used by both `Config::load` and the verb, and test the same TOML + invocation through both binaries. +- Move the provider install out of both mains into `Relay::load`, so an + embedder and the two binaries share one startup. jemalloc stays a + binary-level global allocator. +- `moq relay --help` renders the relay's whole tree under the verb; check the + spec dumps both binaries produce agree. +- Docs: `doc/bin/relay/` names both spellings once, `doc/bin/cli.md` gains the + verb, and the demo recipes keep using `moq-relay`. +- Test: `rs/moq-relay/tests/released_cli.rs` drives the binary; add a case + that runs the same smoke through `moq relay`. + +Starts on `main` after the merge: the CLI's verbs live on `usage`, which only +`dev` has. + +## Required + +- [Merge dev](/quest/m1/merge-dev.md) - builds on dev-only code that reaches `main` with the merge + +## Related + +- [One LAN mesh](/quest/m1/lan-mesh.md) - the CLI already hosts the relay library for its cluster +- [`moq` serves like a relay](/quest/m2/cli-serve.md) - the end state where the relay is `moq` with listening on by default