From 0f34777102d2db5e72fbd24ad2132ad8ed94c454 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 14 Nov 2026 10:56:32 -0800 Subject: [PATCH 1/5] feat(moq-auth)!: remove mesh enrollment Mesh work moves to the downstream Kixel product, which has its own certificate-based enrollment contract, so the JWT mesh coordinator here has no consumer. Public API: `moq_auth::mesh` (Coordinator, Client, Config, and the contract types), `serve::Server::with_mesh`, and `Error::Mesh` and `Error::MeshState` are removed. The CLI loses `moq auth mesh` and every `moq auth serve --mesh-*` flag and `[mesh]` file setting. None of it shipped on `release`, so no migration is needed. The unreleased "Mesh endpoints use the signing public key" note in rs/moq-auth/MIGRATE.md describes removed API; only the release cut edits MIGRATE.md, so the cut should drop it. moq-auth no longer enables moq-crypto's `serde` or url's `serde` features, which only the mesh used. Wire: none. Co-Authored-By: Claude Opus 5.5 --- Cargo.toml | 2 +- doc/bin/relay/auth.md | 42 +- doc/lib/rs/moq-auth.md | 1 - nix/Cargo.nix | 6 +- rs/moq-auth/Cargo.toml | 4 +- rs/moq-auth/src/error.rs | 8 - rs/moq-auth/src/fs.rs | 4 +- rs/moq-auth/src/lib.rs | 1 - rs/moq-auth/src/mesh/client.rs | 158 ----- rs/moq-auth/src/mesh/contract_tests.rs | 317 ---------- rs/moq-auth/src/mesh/coordinator.rs | 783 ------------------------- rs/moq-auth/src/mesh/mod.rs | 301 ---------- rs/moq-auth/src/serve.rs | 52 +- rs/moq-cli/src/auth.rs | 409 +------------ 14 files changed, 27 insertions(+), 2061 deletions(-) delete mode 100644 rs/moq-auth/src/mesh/client.rs delete mode 100644 rs/moq-auth/src/mesh/contract_tests.rs delete mode 100644 rs/moq-auth/src/mesh/coordinator.rs delete mode 100644 rs/moq-auth/src/mesh/mod.rs diff --git a/Cargo.toml b/Cargo.toml index 84ac9d1a16b..59fb6c61e91 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -135,7 +135,7 @@ moq-archive = { version = "0.0.11", path = "rs/moq-archive" } moq-audio = { version = "0.1.10", path = "rs/moq-audio", default-features = false } moq-auth = { version = "0.2.1", path = "rs/moq-auth" } moq-congestion = { version = "0.1.0", path = "rs/moq-congestion" } -# default-features off: moq-auth takes only the endpoint key and its text codec. +# default-features off: dependents opt into only the pieces they use, such as `cpu`. moq-crypto = { version = "0.0.11", path = "rs/moq-crypto", default-features = false } moq-hls = { version = "0.5.11", path = "rs/moq-hls", default-features = false } moq-json = { version = "0.5.8", path = "rs/moq-json" } diff --git a/doc/bin/relay/auth.md b/doc/bin/relay/auth.md index 80022a41215..a5bd897b9ec 100644 --- a/doc/bin/relay/auth.md +++ b/doc/bin/relay/auth.md @@ -372,7 +372,7 @@ and each field it omits takes the server-wide value. Its patterns replace the shared ones, so a rule with empty lists revokes. A certificate no rule names gets `--mtls-*`. The server watches the file: an edit takes effect at each session's next re-check, and one that fails to load keeps the last valid -config and logs an error. Listener and mesh settings apply on restart. +config and logs an error. Listener settings apply on restart. ```toml # moq auth serve /etc/moq/auth.toml @@ -448,46 +448,6 @@ from its URL, `--auth-url https://relay:@auth.example.com/` (percent-enc reserved characters), through the TLS proxy in front of the server. `--listen-secret` also guards a loopback listener. See `moq auth serve --help` for the rest. -### Mesh enrollment - -`--mesh-url URL` also makes the server a mesh coordinator: a device holding an -Ed25519 endpoint key enrolls and receives the relay URLs to dial and a JWT that -publishes `mesh//**` and subscribes `mesh/**`, where `` is the key in -z-base-32 (52 characters, the form pkarr and iroh's DNS discovery use). It is -verified like any other token, so the verifying keys (`--key`, `--key-dir`, -or `--key-set`) must include `--mesh-key`'s public half. - -```bash -moq auth serve --key-dir /etc/moq/keys \ - --mesh-url https://mesh.example.com --mesh-listen 127.0.0.1:4441 \ - --mesh-key /etc/moq/mesh.jwk --mesh-state /var/lib/moq/mesh.json \ - --mesh-relay https://relay.example.com -moq auth mesh create-key --tag robot --reusable --expires 7d -moq auth mesh revoke -``` - -Every request proves the endpoint key by signing a fresh, single-use -challenge bound to `--mesh-url`, so no token is issued for a key the caller -does not hold. A device joins with an enrollment key, with no human in the -loop (single-use unless `--reusable`; revoked or expired keys enroll -nothing). It refreshes its token -two thirds of the way through `--mesh-token-ttl` (default 1h) by proving -the key again, so a revoked device is cut off within one token lifetime. A -revoked device cannot enroll again. Each enrollment and refresh also returns -every member's unique, DNS-safe name, defaulting to the device's hostname. - -The device-facing API on `--mesh-listen` is meant to be public behind a TLS -proxy, since devices refuse plain `http://` beyond loopback. A challenge holds -no state until a proof uses it, so unauthenticated requests cannot crowd out a -device's; the proxy should still rate-limit each client. The owner's -commands go to `--listen` (`moq auth mesh --url`, `unix:///path` for a -socket), which every relay holding `--listen-secret` could reach, so `--mesh-url` -refuses `--listen-public`. With `--listen-secret` set, the owner presents it -too, as the URL's password: `moq auth mesh --url http://owner:SECRET@127.0.0.1:4440`. A hosted coordinator implements -the same JSON contract, documented in -[`moq_auth::mesh`](https://docs.rs/moq-auth/latest/moq_auth/mesh/); it adds -no token claims. - ### Migrating from the relay flags The 0.14 relay's auth flags moved to `moq auth serve`, and the relay points diff --git a/doc/lib/rs/moq-auth.md b/doc/lib/rs/moq-auth.md index c06f997afad..ab8c765e11e 100644 --- a/doc/lib/rs/moq-auth.md +++ b/doc/lib/rs/moq-auth.md @@ -18,7 +18,6 @@ accept loop that decides in process. - **Refusals.** `Refusal` carries a `refusal::Reason` (`Refused`, `Forbidden`, or `Expired`) and optional `root` and `tier`. An auth server returns it as JSON with HTTP 403; `Client` returns `Error::Refused(refusal)`. Legacy plain-text 401/403 bodies and unknown reason strings read as `Refused`. Set `root` only from a verified credential, never the dialed path. Embedded relay deciders pass the same shape in `auth::Error::Refused`. - **Client.** `Client` drives a lease against an auth server over `https://`, `unix://`, or loopback `http://`. It revalidates on cadence and retries through an outage until the grant expires, revokes on a 401, 403, or invalid grant, and reports `end` with the session's byte totals. - **Server** (feature `serve`). `serve::Server` is the reference auth server behind `moq auth serve`: JWTs from the query or the moq-transport SETUP token, certificate grants, anonymous rules, tiers, and session caps. `Server::router` mounts it in your own axum service. -- **Mesh enrollment.** `mesh` holds the [enrollment contract](/bin/relay/auth#mesh-enrollment): a device proves its Ed25519 endpoint key against a fresh challenge and receives relay URLs and a scoped token. `mesh::Client` (feature `client`) enrolls with an enrollment key and renews before expiry; `mesh::Coordinator` (feature `serve`) answers it. - **Keys and claims.** Generate HMAC, RSA, ECDSA, or EdDSA keys as JWKs, sign `Claims` (`root`, `publish`, `subscribe`, their optional `Rate` caps, `exp`), and verify them. `Claims::authorize(path)` scopes verified claims to the path a client dialed, exactly as `moq auth serve` does. Roles hold path pattern unions directly. ```bash diff --git a/nix/Cargo.nix b/nix/Cargo.nix index e083b7f46bc..d2c22ba379c 100644 --- a/nix/Cargo.nix +++ b/nix/Cargo.nix @@ -17516,10 +17516,7 @@ rec { name = "moq-crypto"; packageId = "moq-crypto"; usesDefaultFeatures = false; - features = [ - "serde" - "cpu" - ]; + features = [ "cpu" ]; } { name = "moq-pattern"; @@ -17611,7 +17608,6 @@ rec { { name = "url"; packageId = "url"; - features = [ "serde" ]; } ]; devDependencies = [ diff --git a/rs/moq-auth/Cargo.toml b/rs/moq-auth/Cargo.toml index eeddd18b0a9..efac83073a9 100644 --- a/rs/moq-auth/Cargo.toml +++ b/rs/moq-auth/Cargo.toml @@ -34,7 +34,7 @@ axum = { workspace = true, optional = true } base64 = { workspace = true } jsonwebtoken = { workspace = true } kio = { workspace = true, features = ["serde"] } -moq-crypto = { workspace = true, features = ["serde", "cpu"] } +moq-crypto = { workspace = true, features = ["cpu"] } moq-pattern = { workspace = true, features = ["serde"] } pkcs1 = { workspace = true, features = ["std"] } pkcs8 = { workspace = true } @@ -50,7 +50,7 @@ serde_with = { workspace = true, features = ["base64"] } thiserror = { workspace = true } tokio = { workspace = true, features = ["fs", "macros", "rt", "time", "sync"], optional = true } tracing = { workspace = true, optional = true } -url = { workspace = true, features = ["serde"] } +url = { workspace = true } [dev-dependencies] anyhow = { workspace = true } diff --git a/rs/moq-auth/src/error.rs b/rs/moq-auth/src/error.rs index e9018409647..bd0c6a213ff 100644 --- a/rs/moq-auth/src/error.rs +++ b/rs/moq-auth/src/error.rs @@ -196,14 +196,6 @@ pub enum Error { #[error("{0}")] Other(String), - /// A mesh coordinator refused the request. - #[error("mesh coordinator refused: {0}")] - Mesh(#[from] crate::mesh::ErrorReply), - - /// The mesh coordinator's state file could not be read, parsed, or written. - #[error("mesh state: {0}")] - MeshState(String), - /// This CPU lacks a feature the crate's cryptography requires, named in rind's message. #[error(transparent)] UnsupportedCpu(#[from] moq_crypto::cpu::UnsupportedCpu), diff --git a/rs/moq-auth/src/fs.rs b/rs/moq-auth/src/fs.rs index 18ed78b35b2..9ea7bcff02b 100644 --- a/rs/moq-auth/src/fs.rs +++ b/rs/moq-auth/src/fs.rs @@ -20,7 +20,7 @@ pub(crate) fn write(path: &Path, contents: &str, private: bool) -> io::Result<() } #[cfg(unix)] -pub(crate) fn write_private(path: &Path, contents: &str) -> io::Result<()> { +fn write_private(path: &Path, contents: &str) -> io::Result<()> { use std::io::Write; use std::os::unix::fs::{OpenOptionsExt, PermissionsExt}; @@ -40,6 +40,6 @@ pub(crate) fn write_private(path: &Path, contents: &str) -> io::Result<()> { /// Windows has no cheap equivalent of the Unix mode bits, so the file inherits the directory's ACL. #[cfg(not(unix))] -pub(crate) fn write_private(path: &Path, contents: &str) -> io::Result<()> { +fn write_private(path: &Path, contents: &str) -> io::Result<()> { std::fs::write(path, contents) } diff --git a/rs/moq-auth/src/lib.rs b/rs/moq-auth/src/lib.rs index 00fda9f732f..a3004c7f17c 100644 --- a/rs/moq-auth/src/lib.rs +++ b/rs/moq-auth/src/lib.rs @@ -41,7 +41,6 @@ mod set; mod wire; pub mod lease; -pub mod mesh; pub mod refusal; pub use refusal::Refusal; #[cfg(feature = "serve")] diff --git a/rs/moq-auth/src/mesh/client.rs b/rs/moq-auth/src/mesh/client.rs deleted file mode 100644 index 10cdaf8c869..00000000000 --- a/rs/moq-auth/src/mesh/client.rs +++ /dev/null @@ -1,158 +0,0 @@ -//! The device side of the contract: enroll, then keep the token fresh. - -use std::sync::Arc; -use std::time::Duration; - -use url::Url; - -use super::{Challenge, EnrollRequest, Enrollment, ErrorCode, ErrorReply, Proof, Signer}; -use crate::Error; - -/// Every request is bounded so a hung coordinator fails rather than parks the device. -const TIMEOUT: Duration = Duration::from_secs(10); - -/// The largest reply read from the coordinator; a mesh's member list fits easily. -const MAX_REPLY: usize = 4 << 20; - -/// The longest wait between refresh attempts while the coordinator is unreachable. -const MAX_BACKOFF: Duration = Duration::from_secs(60); - -/// A device's client for one coordinator, proving `signer`'s key on every request that -/// needs it. -#[derive(Clone)] -pub struct Client { - http: reqwest::Client, - base: Url, - signer: Arc, -} - -impl Client { - /// A client for the coordinator at `base`, the URL the contract's paths hang off. - /// - /// `https://` is required, except `http://` to a loopback host, since the token rides - /// the reply. - pub fn new(base: Url, signer: Arc) -> crate::Result { - let loopback = match base.host() { - Some(url::Host::Ipv4(ip)) => ip.is_loopback(), - Some(url::Host::Ipv6(ip)) => ip.is_loopback(), - Some(url::Host::Domain(host)) => host == "localhost", - None => false, - }; - match base.scheme() { - "https" => {} - "http" if loopback => {} - "http" => return Err(Error::InsecureUrl(base.to_string())), - _ => return Err(Error::InvalidUrl(base.to_string())), - } - // A redirect would resend a proof or enrollment key to wherever it points. - let http = reqwest::Client::builder() - .redirect(reqwest::redirect::Policy::none()) - .timeout(TIMEOUT) - .use_preconfigured_tls(crate::crypto::tls()?) - .build()?; - Ok(Self { http, base, signer }) - } - - /// Join with a pre-authorized enrollment key, asking for `name`. - /// - /// When the coordinator refuses the key but this endpoint is already enrolled, such as - /// a retry after a single-use key's reply was lost, this refreshes instead, keeping the - /// name the device already has. - pub async fn enroll(&self, key: &str, name: Option<&str>) -> crate::Result { - let proof = self.proof().await?; - let request = EnrollRequest { - proof, - key: key.into(), - name: name.map(Into::into), - }; - match self.post("enroll", &request).await { - Err(Error::Mesh(refused)) if refused.error == ErrorCode::InvalidGrant => match self.refresh().await { - // Not enrolled after all, so the key's refusal is the answer. - Err(Error::Mesh(reply)) if reply.error == ErrorCode::AccessDenied => Err(Error::Mesh(refused)), - res => res, - }, - res => res, - } - } - - /// Prove the key again for a fresh token. Fails with [`ErrorCode::AccessDenied`] once - /// the device is revoked. - pub async fn refresh(&self) -> crate::Result { - let proof = self.proof().await?; - self.post("refresh", &proof).await - } - - /// Wait until `current` is two thirds through its remaining life, then refresh it, - /// retrying through an outage until it expires. A refusal is returned at once, so a - /// revoked device stops rather than retrying, except a transient one: a server error, - /// `slow_down`, or a challenge that expired or outlived a coordinator restart. - pub async fn renew(&self, current: &Enrollment) -> crate::Result { - let remaining = || { - current - .expires - .checked_duration_since(kio::clock::wall()) - .unwrap_or_default() - }; - kio::clock::sleep(remaining() * 2 / 3).await; - - let mut backoff = Duration::from_secs(1); - loop { - match self.refresh().await { - Err(Error::Mesh(reply)) - if !matches!( - reply.error, - ErrorCode::ServerError | ErrorCode::SlowDown | ErrorCode::InvalidChallenge - ) => - { - return Err(Error::Mesh(reply)); - } - Ok(enrollment) => return Ok(enrollment), - Err(err) if remaining().is_zero() => return Err(err), - Err(err) => { - tracing::warn!(%err, "mesh refresh failed; retrying"); - kio::clock::sleep(backoff.min(remaining())).await; - backoff = (backoff * 2).min(MAX_BACKOFF); - } - } - } - } - - async fn proof(&self) -> crate::Result { - let challenge: Challenge = self.post("challenge", &serde_json::Value::Null).await?; - Ok(Proof::sign(self.signer.as_ref(), &self.base, challenge.challenge)) - } - - async fn post( - &self, - path: &str, - body: &T, - ) -> crate::Result { - let mut url = self.base.clone(); - let joined = format!("{}/{path}", url.path().trim_end_matches('/')); - url.set_path(&joined); - - let request = self.http.post(url); - let request = match serde_json::to_value(body)? { - serde_json::Value::Null => request, - body => request.json(&body), - }; - let mut response = request.send().await?; - let status = response.status(); - - let mut bytes = Vec::new(); - while let Some(chunk) = response.chunk().await? { - if bytes.len() + chunk.len() > MAX_REPLY { - return Err(Error::Unavailable("the coordinator's reply is too large".into())); - } - bytes.extend_from_slice(&chunk); - } - - if status.is_success() { - return Ok(serde_json::from_slice(&bytes)?); - } - match serde_json::from_slice::(&bytes) { - Ok(reply) => Err(Error::Mesh(reply)), - Err(_) => Err(Error::Unavailable(format!("the coordinator answered {status}"))), - } - } -} diff --git a/rs/moq-auth/src/mesh/contract_tests.rs b/rs/moq-auth/src/mesh/contract_tests.rs deleted file mode 100644 index 6f3f043fc5b..00000000000 --- a/rs/moq-auth/src/mesh/contract_tests.rs +++ /dev/null @@ -1,317 +0,0 @@ -//! The contract end to end: a [`Client`] against a [`Coordinator`] over HTTP. - -use std::sync::Arc; -use std::time::Duration; - -use rind::signature::Ed25519KeyPair; - -use super::*; -use crate::{Algorithm, Error, Key, KeyId}; - -struct Mesh { - coordinator: Coordinator, - base: url::Url, - key: Key, -} - -impl Mesh { - async fn new() -> Self { - Self::with_ttl(Duration::from_secs(3600)).await - } - - async fn with_ttl(ttl: Duration) -> Self { - let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); - let base: url::Url = format!("http://{}/mesh", listener.local_addr().unwrap()) - .parse() - .unwrap(); - let key = Key::generate(Algorithm::HS256, Some(KeyId::decode("mesh").unwrap())).unwrap(); - let mut config = Config::new(base.clone(), key.clone()); - config.relays = vec!["https://relay.example".parse().unwrap()]; - config.token_ttl = ttl; - let coordinator = Coordinator::new(config).unwrap(); - let router = coordinator.router(); - tokio::spawn(async move { axum::serve(listener, router).await }); - Self { coordinator, base, key } - } - - fn device(&self) -> Client { - let signer = Arc::new(Ed25519KeyPair::generate().unwrap()); - Client::new(self.base.clone(), signer).unwrap() - } - - fn key(&self, reusable: bool) -> String { - let new = NewKey { - reusable, - tags: vec!["robot".into()], - ..Default::default() - }; - self.coordinator.create_key(&new).unwrap().key.unwrap() - } - - /// The code a refusal carries. - fn refused(err: Error) -> ErrorCode { - match err { - Error::Mesh(reply) => reply.error, - other => panic!("expected a refusal, got {other}"), - } - } -} - -#[tokio::test] -async fn an_enrollment_key_grants_the_endpoints_subtree() { - let mesh = Mesh::new().await; - let device = mesh.device(); - let enrollment = device.enroll(&mesh.key(false), Some("Robot 1")).await.unwrap(); - - assert_eq!(enrollment.name, "robot-1"); - assert_eq!( - enrollment.relays, - vec!["https://relay.example".parse::().unwrap()] - ); - assert_eq!(enrollment.members.len(), 1); - assert_eq!(enrollment.members[0].tags, vec!["robot".to_string()]); - - let id = enrollment.members[0].endpoint; - let claims = mesh.key.verify(&enrollment.token).unwrap(); - assert_eq!(claims.expires, Some(enrollment.expires)); - let own = claims.authorize(&format!("mesh/{id}/ssh")).unwrap(); - assert!(!own.publish.is_empty()); - let other = claims.authorize("mesh/someone").unwrap(); - assert!(other.publish.is_empty()); - assert!(!other.subscribe.is_empty()); - assert!(claims.authorize("elsewhere").is_err()); -} - -#[tokio::test] -async fn a_single_use_key_enrolls_once() { - let mesh = Mesh::new().await; - let key = mesh.key(false); - let device = mesh.device(); - let first = device.enroll(&key, Some("pi")).await.unwrap(); - let err = mesh.device().enroll(&key, None).await.unwrap_err(); - assert_eq!(Mesh::refused(err), ErrorCode::InvalidGrant); - - // The device it enrolled can retry, as after a lost reply, and keeps its name. - let retried = device.enroll(&key, Some("other")).await.unwrap(); - assert_eq!(retried.name, first.name); - assert!(mesh.key.verify(&retried.token).is_ok()); - - let reusable = mesh.key(true); - let first = mesh.device().enroll(&reusable, Some("bot")).await.unwrap(); - let second = mesh.device().enroll(&reusable, Some("bot")).await.unwrap(); - assert_eq!(first.name, "bot"); - assert_eq!(second.name, "bot-2"); - assert_eq!(second.members.len(), 3); -} - -#[tokio::test] -async fn a_revoked_or_unknown_key_enrolls_nothing() { - let mesh = Mesh::new().await; - let key = mesh.key(true); - let id = key.split_once('.').unwrap().0; - assert!(mesh.coordinator.revoke_key(id).unwrap()); - let err = mesh.device().enroll(&key, None).await.unwrap_err(); - assert_eq!(Mesh::refused(err), ErrorCode::InvalidGrant); - - let err = mesh.device().enroll(&format!("{id}.wrong"), None).await.unwrap_err(); - assert_eq!(Mesh::refused(err), ErrorCode::InvalidGrant); -} - -#[tokio::test] -async fn a_proof_from_the_wrong_key_is_refused() { - let mesh = Mesh::new().await; - let challenge = mesh.coordinator.challenge(); - let holder = Ed25519KeyPair::generate().unwrap(); - let victim = Ed25519KeyPair::generate().unwrap(); - let mut proof = Proof::sign(&holder, &mesh.base, challenge.challenge); - proof.endpoint = victim.endpoint(); - let request = EnrollRequest { - proof, - key: mesh.key(true), - name: None, - }; - assert_eq!( - mesh.coordinator.enroll(&request).unwrap_err().error, - ErrorCode::InvalidSignature - ); - - // A forged proof spends nothing, so the holder can still use the challenge, once. - let mut request = request; - request.proof = Proof::sign(&holder, &mesh.base, request.proof.challenge); - mesh.coordinator.enroll(&request).unwrap(); - assert_eq!( - mesh.coordinator.enroll(&request).unwrap_err().error, - ErrorCode::InvalidChallenge - ); - - // A proof made for another coordinator does not verify here. - let challenge = mesh.coordinator.challenge(); - request.proof = Proof::sign( - &holder, - &"https://other.example/mesh".parse().unwrap(), - challenge.challenge, - ); - assert_eq!( - mesh.coordinator.enroll(&request).unwrap_err().error, - ErrorCode::InvalidSignature - ); -} - -#[tokio::test] -async fn refresh_issues_a_new_token_until_revoked() { - let mesh = Mesh::new().await; - let device = mesh.device(); - - // Refreshing before enrolling proves a key the mesh does not know. - let err = device.refresh().await.unwrap_err(); - assert_eq!(Mesh::refused(err), ErrorCode::AccessDenied); - - let enrollment = device.enroll(&mesh.key(true), Some("pi")).await.unwrap(); - let refreshed = device.refresh().await.unwrap(); - assert_eq!(refreshed.name, "pi"); - assert!(mesh.key.verify(&refreshed.token).is_ok()); - - let id = enrollment.members[0].endpoint; - assert!(mesh.coordinator.revoke_device(id).unwrap()); - let err = device.refresh().await.unwrap_err(); - assert_eq!(Mesh::refused(err), ErrorCode::AccessDenied); - - // A reusable key cannot bring a revoked device back. - let err = device.enroll(&mesh.key(true), None).await.unwrap_err(); - assert_eq!(Mesh::refused(err), ErrorCode::AccessDenied); -} - -#[tokio::test] -async fn renew_refreshes_before_expiry() { - // A clock only the test moves, which the coordinator reads too; the HTTP between them is real. - let clock = kio::clock::Manual::new().install(&tokio::runtime::Handle::current()); - let ttl = Duration::from_secs(3); - let mesh = Mesh::with_ttl(ttl).await; - let device = mesh.device(); - let enrollment = device.enroll(&mesh.key(true), None).await.unwrap(); - let mut renew = std::pin::pin!(device.renew(&enrollment)); - // The first poll arms the wait for two thirds of the token's life, before any request. - let mut cx = std::task::Context::from_waker(std::task::Waker::noop()); - assert!(std::future::Future::poll(renew.as_mut(), &mut cx).is_pending()); - clock.advance(ttl * 2 / 3); - let renewed = renew.await.unwrap(); - assert!(kio::clock::wall() < enrollment.expires); - assert!(renewed.expires > enrollment.expires); -} - -#[tokio::test] -async fn only_an_issued_challenge_is_accepted() { - let mesh = Mesh::new().await; - let holder = Ed25519KeyPair::generate().unwrap(); - let enroll = |challenge: Vec| { - let request = EnrollRequest { - proof: Proof::sign(&holder, &mesh.base, challenge), - key: mesh.key(true), - name: None, - }; - mesh.coordinator.enroll(&request).unwrap_err().error - }; - - // Made up, tampered with, or issued before a restart (another secret). - assert_eq!(enroll(vec![7; CHALLENGE_LEN]), ErrorCode::InvalidChallenge); - let mut tampered = mesh.coordinator.challenge().challenge; - tampered[0] ^= 1; - assert_eq!(enroll(tampered), ErrorCode::InvalidChallenge); - let mut config = Config::new(mesh.base.clone(), mesh.key.clone()); - config.relays = vec!["https://relay.example".parse().unwrap()]; - let restarted = Coordinator::new(config).unwrap(); - assert_eq!(enroll(restarted.challenge().challenge), ErrorCode::InvalidChallenge); -} - -#[tokio::test] -async fn renew_retries_a_transient_refusal() { - kio::clock::host_time(); // real HTTP, and the backoff is armed only after its refusal - use wiremock::matchers::{method, path}; - use wiremock::{Mock, MockServer, ResponseTemplate}; - - let mesh = Mesh::with_ttl(Duration::from_secs(3)).await; - let signer = Arc::new(Ed25519KeyPair::generate().unwrap()); - let device = Client::new(mesh.base.clone(), signer.clone()).unwrap(); - let enrollment = device.enroll(&mesh.key(true), None).await.unwrap(); - let fresh = device.refresh().await.unwrap(); - - // A coordinator that restarts between issuing the challenge and seeing the proof. - let server = MockServer::start().await; - Mock::given(method("POST")) - .and(path("/mesh/challenge")) - .respond_with(ResponseTemplate::new(200).set_body_json(mesh.coordinator.challenge())) - .mount(&server) - .await; - Mock::given(method("POST")) - .and(path("/mesh/refresh")) - .respond_with( - ResponseTemplate::new(400).set_body_json(ErrorReply::new(ErrorCode::InvalidChallenge, "unknown challenge")), - ) - .up_to_n_times(1) - .with_priority(1) - .mount(&server) - .await; - Mock::given(method("POST")) - .and(path("/mesh/refresh")) - .respond_with(ResponseTemplate::new(200).set_body_json(&fresh)) - .mount(&server) - .await; - - let device = Client::new(format!("{}/mesh", server.uri()).parse().unwrap(), signer).unwrap(); - let renewed = device.renew(&enrollment).await.unwrap(); - assert_eq!(renewed, fresh); -} - -#[test] -fn a_key_that_cannot_sign_refuses_to_start() { - let key = Key::generate(Algorithm::ES256, None).unwrap().to_public().unwrap(); - let config = Config::new("https://mesh.example".parse().unwrap(), key); - assert!(Coordinator::new(config).is_err()); -} - -#[test] -fn a_failed_save_spends_nothing() { - let dir = tempfile::tempdir().unwrap(); - let key = Key::generate(Algorithm::HS256, None).unwrap(); - let mut config = Config::new("https://mesh.example".parse().unwrap(), key); - config.state = Some(dir.path().join("mesh.json")); - let coordinator = Coordinator::new(config).unwrap(); - let secret = coordinator.create_key(&NewKey::default()).unwrap().key.unwrap(); - - // A directory where the temporary file goes makes every save fail. - std::fs::create_dir(dir.path().join("mesh.tmp")).unwrap(); - let signer = Ed25519KeyPair::generate().unwrap(); - let base: url::Url = "https://mesh.example".parse().unwrap(); - let enroll = |coordinator: &Coordinator| EnrollRequest { - proof: Proof::sign(&signer, &base, coordinator.challenge().challenge), - key: secret.clone(), - name: None, - }; - let err = coordinator.enroll(&enroll(&coordinator)).unwrap_err(); - assert_eq!(err.error, ErrorCode::ServerError); - assert!(coordinator.devices().is_empty()); - assert!(!coordinator.keys()[0].used); - - std::fs::remove_dir(dir.path().join("mesh.tmp")).unwrap(); - coordinator.enroll(&enroll(&coordinator)).unwrap(); - assert!(coordinator.keys()[0].used); -} - -#[tokio::test] -async fn a_malformed_body_is_an_oauth_error() { - let mesh = Mesh::new().await; - let url = format!("{}/enroll", mesh.base); - let response = reqwest::Client::builder() - .use_preconfigured_tls(crate::crypto::tls().unwrap()) - .build() - .unwrap() - .post(url) - .header("content-type", "application/json") - .body("{}") - .send() - .await - .unwrap(); - assert_eq!(response.status(), reqwest::StatusCode::BAD_REQUEST); - let reply: ErrorReply = response.json().await.unwrap(); - assert_eq!(reply.error, ErrorCode::InvalidRequest); -} diff --git a/rs/moq-auth/src/mesh/coordinator.rs b/rs/moq-auth/src/mesh/coordinator.rs deleted file mode 100644 index cee6b7e5206..00000000000 --- a/rs/moq-auth/src/mesh/coordinator.rs +++ /dev/null @@ -1,783 +0,0 @@ -//! The reference coordinator behind `moq auth serve --mesh-url`. -//! -//! [`Coordinator::router`] answers the device-facing contract; [`Coordinator::admin_router`] -//! is the owner's API, mounted on the auth server's own listener, which has no -//! authentication and stays on loopback. Enrollment keys and devices persist to a JSON -//! state file. Challenges are sealed with a secret held in memory, so a restart only makes -//! a device fetch a new challenge. -//! -//! A challenge is single-use per endpoint: each carries a sequence number, and an endpoint's -//! proof is accepted only over a challenge newer than the last one it was granted with. The -//! replay state is one number per enrolled device, so no caller can spend another's room. - -use std::collections::HashMap; -use std::path::PathBuf; -use std::sync::atomic::{AtomicU64, Ordering}; -use std::sync::{Arc, Mutex}; -use std::time::Duration; - -use axum::extract::{Path, State}; -use axum::http::StatusCode; -use axum::response::{IntoResponse, Response}; -use axum::routing::{delete, get, post}; -use axum::{Json, Router}; -use moq_crypto::sign::PublicKey; -use serde::{Deserialize, Serialize}; -use serde_with::serde_as; - -use super::{CHALLENGE_LEN, Challenge, EnrollRequest, Enrollment, ErrorCode, ErrorReply, Member, Proof}; -use crate::{Claims, Key, Pattern}; - -/// How long a challenge stays valid. -const CHALLENGE_TTL: Duration = Duration::from_secs(60); - -/// A challenge is its expiry (unix seconds), its sequence number, then a truncated HMAC -/// over both. The HMAC makes it unpredictable; the sequence makes it single-use. -const TAG_LEN: usize = CHALLENGE_LEN - 16; - -/// The longest a device name may be: one DNS label. -const MAX_NAME: usize = 63; - -/// What the coordinator issues, and where it keeps its state. -/// -/// `#[non_exhaustive]`, so build it with [`Config::new`] and set the fields. -#[derive(Clone, Debug)] -#[non_exhaustive] -pub struct Config { - /// The public base URL devices reach the contract at, which every proof is bound to. - pub url: url::Url, - /// The key that signs mesh tokens; the relays must verify it. - pub key: Key, - /// The relays an [`Enrollment`] tells a device to dial. - pub relays: Vec, - /// The JSON file enrollment keys and devices persist to; `None` keeps them in memory. - pub state: Option, - /// How long an issued token lasts. - pub token_ttl: Duration, -} - -impl Config { - /// A coordinator at `url` signing with `key`, with hour-long tokens, no relays, and no - /// state file. - pub fn new(url: url::Url, key: Key) -> Self { - Self { - url, - key, - relays: Vec::new(), - state: None, - token_ttl: Duration::from_secs(3600), - } - } -} - -/// The state that survives a restart. -#[derive(Clone, Default, Serialize, Deserialize)] -struct Store { - #[serde(default)] - keys: Vec, - #[serde(default)] - devices: Vec, -} - -/// An enrollment key, holding only the SHA-256 of its secret. -#[serde_as] -#[derive(Clone, Serialize, Deserialize)] -struct StoredKey { - id: String, - /// Hex, so the file stays readable. - hash: String, - #[serde(default, skip_serializing_if = "Vec::is_empty")] - tags: Vec, - #[serde_as(as = "Option")] - #[serde(default, skip_serializing_if = "Option::is_none")] - expires: Option, - reusable: bool, - #[serde(default)] - used: bool, - #[serde(default)] - revoked: bool, -} - -/// An enrolled device. A revoked one stays recorded so a reusable key cannot re-enroll it. -#[serde_as] -#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] -#[non_exhaustive] -pub struct Device { - /// Its endpoint ID. - pub endpoint: PublicKey, - /// Its unique, DNS-safe name. - pub name: String, - /// The tags of the enrollment key it joined with. - #[serde(default, skip_serializing_if = "Vec::is_empty")] - pub tags: Vec, - /// When it first enrolled, unix seconds. - #[serde_as(as = "kio::clock::serde::UnixSeconds")] - pub enrolled: kio::clock::Wall, - /// Whether the owner revoked it; its next refresh fails. - #[serde(default)] - pub revoked: bool, -} - -/// An enrollment key as the owner sees it, without its secret. -#[serde_as] -#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] -#[non_exhaustive] -pub struct KeyInfo { - /// The ID the owner revokes it by. - pub id: String, - /// The tags every device it enrolls carries. - #[serde(default, skip_serializing_if = "Vec::is_empty")] - pub tags: Vec, - /// When it stops enrolling, unix seconds. - #[serde_as(as = "Option")] - #[serde(default, skip_serializing_if = "Option::is_none")] - pub expires: Option, - /// Whether it enrolls more than one device. - pub reusable: bool, - /// Whether a single-use key has enrolled its device. - pub used: bool, - /// Whether the owner revoked it. - pub revoked: bool, - /// The key itself, returned only when it is created. - #[serde(default, skip_serializing_if = "Option::is_none")] - pub key: Option, -} - -impl From<&StoredKey> for KeyInfo { - fn from(key: &StoredKey) -> Self { - Self { - id: key.id.clone(), - tags: key.tags.clone(), - expires: key.expires, - reusable: key.reusable, - used: key.used, - revoked: key.revoked, - key: None, - } - } -} - -/// `POST /mesh/keys` on the admin API: what the new enrollment key allows. -#[serde_as] -#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)] -#[serde(default)] -#[non_exhaustive] -pub struct NewKey { - /// Tags every device it enrolls carries. - pub tags: Vec, - /// How long until it stops enrolling; `None` never expires. Whole seconds on the wire. - #[serde_as(as = "Option>")] - pub expires_in: Option, - /// Whether it enrolls more than one device. - pub reusable: bool, -} - -#[derive(Default)] -struct Inner { - store: Store, - /// The sequence of the last challenge each endpoint was granted with. Only an - /// enrolled device gets an entry, so it grows with the devices and nothing else. - used: HashMap, -} - -/// The mesh coordinator: issues challenges, enrolls devices, and refreshes their tokens. -#[derive(Clone)] -pub struct Coordinator { - config: Arc, - /// Seals challenges, so an outstanding one needs no state. - secret: Arc, - /// The next challenge's sequence number. It restarts with the secret, which voids - /// every earlier challenge. - sequence: Arc, - inner: Arc>, -} - -type Reply = Result; - -impl Coordinator { - /// A coordinator for `config`, loading its state file if one exists. - pub fn new(config: Config) -> crate::Result { - let store = match &config.state { - Some(path) => match std::fs::read_to_string(path) { - Ok(json) => serde_json::from_str(&json).map_err(|err| crate::Error::MeshState(err.to_string()))?, - Err(err) if err.kind() == std::io::ErrorKind::NotFound => Store::default(), - Err(err) => return Err(crate::Error::MeshState(err.to_string())), - }, - None => Store::default(), - }; - // Sign once now, so a key that cannot sign these claims fails at startup rather - // than after an enrollment key is spent. - config - .key - .sign(&claims(PublicKey::from_bytes([0; 32]), kio::clock::wall(), None))?; - Ok(Self { - config: Arc::new(config), - secret: Arc::new(rind::hmac::Key::new(&rind::hmac::HMAC_SHA256, &random(32))), - sequence: Default::default(), - inner: Arc::new(Mutex::new(Inner { - store, - ..Default::default() - })), - }) - } - - /// Issue a fresh challenge. It holds no state until a proof uses it, so no number of - /// requests can crowd out one a device is still signing. - pub fn challenge(&self) -> Challenge { - let now = kio::clock::wall(); - let expires = - now.checked_duration_since(kio::clock::Wall::UNIX_EPOCH) - .unwrap_or_default() - .as_secs() + CHALLENGE_TTL.as_secs(); - let mut challenge = expires.to_be_bytes().to_vec(); - challenge.extend_from_slice(&self.sequence.fetch_add(1, Ordering::Relaxed).to_be_bytes()); - let tag = self.tag(&challenge); - challenge.extend_from_slice(&tag); - Challenge { - challenge, - expires: kio::clock::Wall::UNIX_EPOCH.saturating_add(Duration::from_secs(expires)), - } - } - - /// Enroll with an enrollment key. - pub fn enroll(&self, request: &EnrollRequest) -> Reply { - let mut inner = self.inner.lock().unwrap(); - let sequence = self.check(&inner, &request.proof)?; - - let (id, secret) = request - .key - .split_once('.') - .ok_or_else(|| ErrorReply::new(ErrorCode::InvalidGrant, "unknown enrollment key"))?; - let hash = hex(&sha256(secret.as_bytes())); - let now = kio::clock::wall(); - let key = inner - .store - .keys - .iter() - .find(|key| key.id == id && constant_eq(key.hash.as_bytes(), hash.as_bytes())) - .ok_or_else(|| ErrorReply::new(ErrorCode::InvalidGrant, "unknown enrollment key"))?; - if key.revoked { - return Err(ErrorReply::new( - ErrorCode::InvalidGrant, - "the enrollment key is revoked", - )); - } - if key.expires.is_some_and(|expires| expires <= now) { - return Err(ErrorReply::new( - ErrorCode::InvalidGrant, - "the enrollment key has expired", - )); - } - if !key.reusable && key.used { - return Err(ErrorReply::new( - ErrorCode::InvalidGrant, - "the enrollment key was already used", - )); - } - let tags = key.tags.clone(); - let reusable = key.reusable; - let id = id.to_string(); - - let endpoint = request.proof.endpoint; - let enrollment = self.commit(&mut inner, |store| { - self.admit(store, endpoint, request.name.as_deref(), tags)?; - if !reusable && let Some(key) = store.keys.iter_mut().find(|key| key.id == id) { - key.used = true; - } - self.issue(store, endpoint) - })?; - inner.used.insert(endpoint, sequence); - Ok(enrollment) - } - - /// Refresh an enrolled device's token, refusing a revoked or unknown one. - pub fn refresh(&self, proof: &Proof) -> Reply { - let mut inner = self.inner.lock().unwrap(); - let sequence = self.check(&inner, proof)?; - let enrollment = match inner - .store - .devices - .iter() - .find(|device| device.endpoint == proof.endpoint) - { - Some(device) if !device.revoked => self.issue(&inner.store, proof.endpoint)?, - Some(_) => return Err(ErrorReply::new(ErrorCode::AccessDenied, "the device is revoked")), - None => return Err(ErrorReply::new(ErrorCode::AccessDenied, "the device is not enrolled")), - }; - inner.used.insert(proof.endpoint, sequence); - Ok(enrollment) - } - - /// Create an enrollment key, returning it with its secret, which is never shown again. - pub fn create_key(&self, new: &NewKey) -> crate::Result { - let mut inner = self.inner.lock().unwrap(); - let id = hex(&random(8)); - let secret = encode(&random(32)); - let stored = StoredKey { - id: id.clone(), - hash: hex(&sha256(secret.as_bytes())), - tags: new.tags.clone(), - expires: new.expires_in.map(|ttl| kio::clock::wall().saturating_add(ttl)), - reusable: new.reusable, - used: false, - revoked: false, - }; - let mut info = KeyInfo::from(&stored); - info.key = Some(format!("{id}.{secret}")); - self.commit(&mut inner, |store| { - store.keys.push(stored); - Ok(()) - })?; - Ok(info) - } - - /// Every enrollment key, without secrets. - pub fn keys(&self) -> Vec { - self.inner - .lock() - .unwrap() - .store - .keys - .iter() - .map(KeyInfo::from) - .collect() - } - - /// Revoke an enrollment key so it enrolls nothing new. Devices it enrolled stay. - /// Returns whether the key exists. - pub fn revoke_key(&self, id: &str) -> crate::Result { - let mut inner = self.inner.lock().unwrap(); - let found = self.commit(&mut inner, |store| { - let key = store.keys.iter_mut().find(|key| key.id == id); - Ok(key.map(|key| key.revoked = true).is_some()) - })?; - Ok(found) - } - - /// Every device, revoked ones included. - pub fn devices(&self) -> Vec { - self.inner.lock().unwrap().store.devices.clone() - } - - /// Revoke a device: its next refresh fails, and it cannot enroll again. - /// Returns whether the device exists. - pub fn revoke_device(&self, endpoint: PublicKey) -> crate::Result { - let mut inner = self.inner.lock().unwrap(); - let found = self.commit(&mut inner, |store| { - let device = store.devices.iter_mut().find(|device| device.endpoint == endpoint); - Ok(device.map(|device| device.revoked = true).is_some()) - })?; - Ok(found) - } - - /// The device-facing contract, mounted at [`Config::url`]'s path. - pub fn router(&self) -> Router { - let router = Router::new() - .route("/challenge", post(challenge_handler)) - .route("/enroll", post(enroll_handler)) - .route("/refresh", post(refresh_handler)) - .with_state(self.clone()); - match self.config.url.path().trim_end_matches('/') { - "" => router, - base => Router::new().nest(base, router), - } - } - - /// The owner's API under `/mesh`, with no authentication of its own: mount it only on - /// a listener the owner alone reaches. - pub fn admin_router(&self) -> Router { - Router::new() - .route("/mesh/keys", get(keys_handler).post(create_key_handler)) - .route("/mesh/keys/{id}", delete(revoke_key_handler)) - .route("/mesh/devices", get(devices_handler)) - .route("/mesh/devices/{endpoint}", delete(revoke_device_handler)) - .with_state(self.clone()) - } - - /// Serve the device-facing contract on a TCP listener until the task is dropped. - pub async fn serve(&self, listener: tokio::net::TcpListener) -> std::io::Result<()> { - axum::serve(listener, self.router()).await - } - - /// Verify the proof's challenge and signature, returning the challenge's sequence. - /// - /// The caller records the sequence as the endpoint's last only once the request is - /// granted, so a refused proof changes nothing. - fn check(&self, inner: &Inner, proof: &Proof) -> Reply { - let invalid = |reason: &str| ErrorReply::new(ErrorCode::InvalidChallenge, reason); - let challenge = &proof.challenge; - if challenge.len() != CHALLENGE_LEN - || !constant_eq( - &self.tag(&challenge[..CHALLENGE_LEN - TAG_LEN]), - &challenge[CHALLENGE_LEN - TAG_LEN..], - ) { - return Err(invalid("unknown challenge")); - } - let expires = u64::from_be_bytes(challenge[..8].try_into().expect("8 bytes")); - let expires = kio::clock::Wall::UNIX_EPOCH.saturating_add(Duration::from_secs(expires)); - if expires <= kio::clock::wall() { - return Err(invalid("the challenge expired")); - } - if !proof.verify(&self.config.url) { - return Err(ErrorReply::new( - ErrorCode::InvalidSignature, - "the signature does not verify against the endpoint ID", - )); - } - let sequence = u64::from_be_bytes(challenge[8..16].try_into().expect("8 bytes")); - if inner.used.get(&proof.endpoint).is_some_and(|last| sequence <= *last) { - return Err(invalid("the endpoint already used this challenge or a newer one")); - } - Ok(sequence) - } - - /// The truncated HMAC sealing a challenge's expiry and nonce. - fn tag(&self, body: &[u8]) -> Vec { - rind::hmac::sign(&self.secret, body).as_ref()[..TAG_LEN].to_vec() - } - - /// Apply `change` to a copy of the store and save it, adopting the copy only once it - /// is on disk, so a failed save leaves no credential spent and no device admitted. - fn commit(&self, inner: &mut Inner, change: impl FnOnce(&mut Store) -> Reply) -> Reply { - let mut store = inner.store.clone(); - let out = change(&mut store)?; - self.save(&store)?; - inner.store = store; - Ok(out) - } - - /// Record `endpoint` as enrolled, or update its name and tags if it already is. - fn admit(&self, store: &mut Store, endpoint: PublicKey, name: Option<&str>, tags: Vec) -> Reply<()> { - refuse_revoked(store, endpoint)?; - let devices = &mut store.devices; - let current = devices.iter().position(|device| device.endpoint == endpoint); - let name = match (name, current) { - (None, Some(index)) => devices[index].name.clone(), - (name, _) => unique_name(devices, endpoint, &sanitize(name.unwrap_or_default())), - }; - match current { - Some(index) => { - devices[index].name = name; - devices[index].tags = tags; - } - None => devices.push(Device { - endpoint, - name, - tags, - enrolled: kio::clock::wall(), - revoked: false, - }), - } - Ok(()) - } - - /// Sign a token for an enrolled `endpoint` and list the mesh. - fn issue(&self, store: &Store, endpoint: PublicKey) -> Reply { - let internal = |err: crate::Error| ErrorReply::new(ErrorCode::ServerError, err.to_string()); - let now = kio::clock::wall(); - // `exp` is whole seconds on the wire; truncate so the reply matches the token. - let secs = now - .checked_duration_since(kio::clock::Wall::UNIX_EPOCH) - .unwrap_or_default() - .as_secs(); - let expires = kio::clock::Wall::UNIX_EPOCH.saturating_add(Duration::from_secs(secs) + self.config.token_ttl); - let token = self - .config - .key - .sign(&claims(endpoint, now, Some(expires))) - .map_err(internal)?; - let device = store - .devices - .iter() - .find(|device| device.endpoint == endpoint) - .expect("admitted before it is issued"); - Ok(Enrollment { - token, - expires, - relays: self.config.relays.clone(), - name: device.name.clone(), - members: store - .devices - .iter() - .filter(|device| !device.revoked) - .map(|device| Member { - endpoint: device.endpoint, - name: device.name.clone(), - tags: device.tags.clone(), - }) - .collect(), - }) - } - - /// Write the store atomically: a temporary file beside it, renamed over it. - fn save(&self, store: &Store) -> Reply<()> { - let Some(path) = &self.config.state else { - return Ok(()); - }; - let failed = |err: std::io::Error| { - tracing::error!(path = %path.display(), %err, "failed to save the mesh state"); - ErrorReply::new(ErrorCode::ServerError, "the coordinator could not save its state") - }; - let json = serde_json::to_string_pretty(store).expect("the store serializes"); - let tmp = path.with_extension("tmp"); - crate::fs::write_private(&tmp, &json).map_err(failed)?; - std::fs::rename(&tmp, path).map_err(failed) - } -} - -fn refuse_revoked(store: &Store, endpoint: PublicKey) -> Reply<()> { - if store - .devices - .iter() - .any(|device| device.endpoint == endpoint && device.revoked) - { - return Err(ErrorReply::new(ErrorCode::AccessDenied, "the device is revoked")); - } - Ok(()) -} - -/// The grant every mesh token carries: publish under its own ID, subscribe to the mesh. -fn claims(endpoint: PublicKey, issued: kio::clock::Wall, expires: Option) -> Claims { - Claims::default() - .with_publish([Pattern::subtree(&format!("mesh/{endpoint}")).expect("a z-base-32 segment is a valid path")]) - .with_subscribe([Pattern::subtree("mesh").expect("a valid path")]) - .with_issued(issued) - .with_expires(expires) -} - -/// `name` as one DNS label: lowercase ASCII letters, digits, and inner hyphens, at most -/// 63 bytes, `device` when nothing is left. -fn sanitize(name: &str) -> String { - let mut out = String::with_capacity(name.len().min(MAX_NAME)); - for c in name.chars() { - let c = c.to_ascii_lowercase(); - let c = if c.is_ascii_alphanumeric() { c } else { '-' }; - if c == '-' && (out.is_empty() || out.ends_with('-')) { - continue; - } - out.push(c); - } - out.truncate(MAX_NAME); - let out = out.trim_end_matches('-'); - if out.is_empty() { "device".into() } else { out.into() } -} - -/// `name`, or `name-2`, `name-3`, ... when another endpoint holds it. -fn unique_name(devices: &[Device], endpoint: PublicKey, name: &str) -> String { - let taken = |candidate: &str| { - devices - .iter() - .any(|device| device.endpoint != endpoint && device.name == candidate) - }; - if !taken(name) { - return name.into(); - } - (2..) - .map(|n| { - let suffix = format!("-{n}"); - let base = &name[..name.len().min(MAX_NAME - suffix.len())]; - format!("{}{suffix}", base.trim_end_matches('-')) - }) - .find(|candidate| !taken(candidate)) - .expect("an unbounded search finds a free name") -} - -/// base64url without padding, the encoding of every random secret in the contract. -fn encode(bytes: &[u8]) -> String { - use base64::Engine; - base64::engine::general_purpose::URL_SAFE_NO_PAD.encode(bytes) -} - -fn random(len: usize) -> Vec { - let mut bytes = vec![0u8; len]; - rind::rand::fill(&mut bytes).expect("system randomness"); - bytes -} - -fn sha256(bytes: &[u8]) -> Vec { - rind::digest::digest(&rind::digest::SHA256, bytes).as_ref().to_vec() -} - -fn constant_eq(a: &[u8], b: &[u8]) -> bool { - rind::constant_time::verify(a, b).is_ok() -} - -fn hex(bytes: &[u8]) -> String { - bytes.iter().map(|byte| format!("{byte:02x}")).collect() -} - -impl IntoResponse for ErrorReply { - fn into_response(self) -> Response { - // RFC 6749 §5.2: 400 for every OAuth error except a failed client authentication. - let status = match self.error { - ErrorCode::ServerError => StatusCode::INTERNAL_SERVER_ERROR, - _ => StatusCode::BAD_REQUEST, - }; - (status, Json(self)).into_response() - } -} - -/// A JSON body that refuses with the contract's error shape rather than axum's text. -struct Body(T); - -impl axum::extract::FromRequest for Body { - type Rejection = ErrorReply; - - async fn from_request(request: axum::extract::Request, state: &S) -> Result { - Json::::from_request(request, state) - .await - .map(|Json(body)| Self(body)) - .map_err(|err| ErrorReply::new(ErrorCode::InvalidRequest, err.body_text())) - } -} - -async fn challenge_handler(State(coordinator): State) -> Json { - Json(coordinator.challenge()) -} - -async fn enroll_handler( - State(coordinator): State, - Body(request): Body, -) -> Reply> { - coordinator.enroll(&request).map(Json) -} - -async fn refresh_handler(State(coordinator): State, Body(proof): Body) -> Reply> { - coordinator.refresh(&proof).map(Json) -} - -fn admin_error(err: crate::Error) -> Response { - (StatusCode::INTERNAL_SERVER_ERROR, err.to_string()).into_response() -} - -async fn keys_handler(State(coordinator): State) -> Json> { - Json(coordinator.keys()) -} - -async fn create_key_handler(State(coordinator): State, Body(new): Body) -> Response { - match coordinator.create_key(&new) { - Ok(info) => (StatusCode::CREATED, Json(info)).into_response(), - Err(err) => admin_error(err), - } -} - -async fn revoke_key_handler(State(coordinator): State, Path(id): Path) -> Response { - match coordinator.revoke_key(&id) { - Ok(true) => StatusCode::NO_CONTENT.into_response(), - Ok(false) => (StatusCode::NOT_FOUND, "not found").into_response(), - Err(err) => admin_error(err), - } -} - -async fn devices_handler(State(coordinator): State) -> Json> { - Json(coordinator.devices()) -} - -async fn revoke_device_handler(State(coordinator): State, Path(endpoint): Path) -> Response { - let Ok(endpoint) = endpoint.parse() else { - return (StatusCode::BAD_REQUEST, "invalid endpoint ID").into_response(); - }; - match coordinator.revoke_device(endpoint) { - Ok(true) => StatusCode::NO_CONTENT.into_response(), - Ok(false) => (StatusCode::NOT_FOUND, "not found").into_response(), - Err(err) => admin_error(err), - } -} - -#[cfg(test)] -mod tests { - use super::*; - use crate::{Algorithm, KeyId}; - - #[test] - fn names_are_dns_labels() { - assert_eq!(sanitize("Luke's MacBook Pro"), "luke-s-macbook-pro"); - assert_eq!(sanitize("--robot_01.local--"), "robot-01-local"); - assert_eq!(sanitize("日本"), "device"); - assert_eq!(sanitize(&"a".repeat(100)).len(), MAX_NAME); - } - - #[test] - fn names_stay_unique() { - let device = |byte, name: &str| Device { - endpoint: PublicKey::from_bytes([byte; 32]), - name: name.into(), - tags: Vec::new(), - enrolled: kio::clock::wall(), - revoked: false, - }; - let devices = vec![device(1, "robot"), device(2, "robot-2")]; - assert_eq!(unique_name(&devices, PublicKey::from_bytes([1; 32]), "robot"), "robot"); - assert_eq!( - unique_name(&devices, PublicKey::from_bytes([3; 32]), "robot"), - "robot-3" - ); - let long = "a".repeat(MAX_NAME); - let devices = vec![device(1, &long)]; - let name = unique_name(&devices, PublicKey::from_bytes([3; 32]), &long); - assert_eq!(name.len(), MAX_NAME); - assert!(name.ends_with("-2")); - } - - #[test] - fn a_challenge_is_single_use_per_endpoint() { - let key = Key::generate(Algorithm::HS256, None).unwrap(); - let base: url::Url = "https://mesh.example".parse().unwrap(); - let coordinator = Coordinator::new(Config::new(base.clone(), key)).unwrap(); - let secret = coordinator - .create_key(&NewKey { - reusable: true, - ..Default::default() - }) - .unwrap() - .key - .unwrap(); - let a = rind::signature::Ed25519KeyPair::generate().unwrap(); - let b = rind::signature::Ed25519KeyPair::generate().unwrap(); - let enroll = |signer: &rind::signature::Ed25519KeyPair, challenge: &Challenge| EnrollRequest { - proof: Proof::sign(signer, &base, challenge.challenge.clone()), - key: secret.clone(), - name: None, - }; - let older = coordinator.challenge(); - let newer = coordinator.challenge(); - - // A refused proof records nothing, so an unknown caller holds no state. - assert_eq!( - coordinator.refresh(&enroll(&a, &older).proof).unwrap_err().error, - ErrorCode::AccessDenied - ); - assert!(coordinator.inner.lock().unwrap().used.is_empty()); - - // Once `a` uses the newer challenge, its own older one is spent too. - coordinator.enroll(&enroll(&a, &newer)).unwrap(); - for challenge in [&newer, &older] { - let err = coordinator.enroll(&enroll(&a, challenge)).unwrap_err(); - assert_eq!(err.error, ErrorCode::InvalidChallenge); - } - - // Another endpoint's state is its own: `a` spending challenges costs `b` nothing. - coordinator.enroll(&enroll(&b, &older)).unwrap(); - coordinator.enroll(&enroll(&b, &newer)).unwrap(); - assert_eq!(coordinator.inner.lock().unwrap().used.len(), 2); - } - - #[test] - fn state_survives_a_restart() { - let dir = tempfile::tempdir().unwrap(); - let mut config = Config::new( - "https://mesh.example".parse().unwrap(), - Key::generate(Algorithm::HS256, Some(KeyId::decode("mesh").unwrap())).unwrap(), - ); - config.state = Some(dir.path().join("mesh.json")); - let coordinator = Coordinator::new(config.clone()).unwrap(); - let key = coordinator.create_key(&NewKey::default()).unwrap(); - assert!(coordinator.revoke_key(&key.id).unwrap()); - - let restarted = Coordinator::new(config).unwrap(); - let keys = restarted.keys(); - assert_eq!(keys.len(), 1); - assert!(keys[0].revoked); - assert_eq!(keys[0].key, None); - } -} diff --git a/rs/moq-auth/src/mesh/mod.rs b/rs/moq-auth/src/mesh/mod.rs deleted file mode 100644 index ae832bbb42f..00000000000 --- a/rs/moq-auth/src/mesh/mod.rs +++ /dev/null @@ -1,301 +0,0 @@ -//! Mesh enrollment: how a device with an endpoint key joins a mesh. -//! -//! A device proves it holds its Ed25519 endpoint key by signing a fresh challenge from -//! the coordinator, and presents an enrollment key the mesh owner created, so enrollment -//! needs no human in the loop. The coordinator answers with an -//! [`Enrollment`]: the relays to dial and a JWT granting publish on `mesh//**` and -//! subscribe on `mesh/**`. The device refreshes it before it expires by proving its key -//! again, so revoking a device makes its next refresh fail. -//! -//! These types are the JSON contract. [`Coordinator`] (feature `serve`) answers it behind -//! `moq auth serve`; [`Client`] (feature `client`) drives it from a device. A hosted -//! service implements the same contract with its own accounts. -//! -//! | Request | Body | Reply | -//! |---|---|---| -//! | `POST {base}/challenge` | none | [`Challenge`] | -//! | `POST {base}/enroll` | [`EnrollRequest`] | [`Enrollment`] | -//! | `POST {base}/refresh` | [`Proof`] | [`Enrollment`] | -//! -//! Every refusal is a 4xx with an [`ErrorReply`]. - -use std::fmt; - -use moq_crypto::sign::PublicKey; -use serde::{Deserialize, Serialize}; -use serde_with::serde_as; - -#[cfg(feature = "client")] -mod client; -#[cfg(all(test, feature = "client", feature = "serve"))] -mod contract_tests; -#[cfg(feature = "serve")] -mod coordinator; - -#[cfg(feature = "client")] -pub use client::*; -#[cfg(feature = "serve")] -pub use coordinator::*; - -/// The context string every [`Proof`] signature starts with, so a signature made for -/// enrollment can never be replayed as anything else the endpoint key signs. -const PROOF_CONTEXT: &[u8] = b"moq-mesh-enroll-00"; - -/// The length of a challenge in bytes. -#[cfg_attr( - not(feature = "serve"), - allow(dead_code, reason = "only the coordinator issues challenges") -)] -pub(crate) const CHALLENGE_LEN: usize = 32; - -const SIGNATURE_LEN: usize = 64; - -/// The message a [`Proof`] signs: [`PROOF_CONTEXT`], a zero byte, the coordinator's base -/// URL as the device addressed it, a zero byte, then the raw challenge. -/// -/// The URL binds the proof to one coordinator, so a coordinator the device trusts cannot -/// relay another's challenge and enroll the device's key there. -fn proof_message(base: &url::Url, challenge: &[u8]) -> Vec { - let base = audience(base); - let mut message = Vec::with_capacity(PROOF_CONTEXT.len() + base.len() + challenge.len() + 2); - message.extend_from_slice(PROOF_CONTEXT); - message.push(0); - message.extend_from_slice(base.as_bytes()); - message.push(0); - message.extend_from_slice(challenge); - message -} - -/// The base URL without a trailing slash, the form both sides sign over. -fn audience(base: &url::Url) -> &str { - base.as_str().trim_end_matches('/') -} - -/// A fresh, single-use challenge from `POST {base}/challenge`. -#[serde_as] -#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] -pub struct Challenge { - /// 32 bytes, opaque to the device, base64url without padding. - #[serde_as(as = "serde_with::base64::Base64")] - pub challenge: Vec, - /// When the coordinator stops accepting a proof over it, unix seconds. - #[serde_as(as = "kio::clock::serde::UnixSeconds")] - pub expires: kio::clock::Wall, -} - -/// Proof that the caller holds `endpoint`'s secret key: its signature over a challenge the -/// coordinator issued and has not yet seen used. -/// -/// The signed message is `moq-mesh-enroll-00`, a zero byte, the coordinator's base URL as -/// the device addressed it without a trailing slash, a zero byte, then the raw challenge. -#[serde_as] -#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] -pub struct Proof { - /// The endpoint ID, which is the key that signed. - pub endpoint: PublicKey, - /// The challenge, as issued. - #[serde_as(as = "serde_with::base64::Base64")] - pub challenge: Vec, - /// The Ed25519 signature over the challenge, bound to the coordinator's base URL. - #[serde_as(as = "serde_with::base64::Base64")] - pub signature: Vec, -} - -impl Proof { - /// A proof for `challenge` at the coordinator `base`, signed by `signer`. - pub fn sign(signer: &dyn Signer, base: &url::Url, challenge: Vec) -> Self { - let signature = signer.sign(&proof_message(base, &challenge)).to_vec(); - Self { - endpoint: signer.endpoint(), - challenge, - signature, - } - } - - /// Whether the signature verifies for the coordinator `base`. The challenge's - /// freshness is the coordinator's to check. - pub fn verify(&self, base: &url::Url) -> bool { - use rind::signature::{PublicKey, verify}; - self.signature.len() == SIGNATURE_LEN - && PublicKey::new(&verify::ED25519, self.endpoint.as_bytes()).is_ok_and(|key| { - key.verify(&proof_message(base, &self.challenge), &self.signature) - .is_ok() - }) - } -} - -/// An endpoint's secret key, held by whatever stores it. -/// -/// Implemented for [`rind::signature::Ed25519KeyPair`]; a daemon implements it for -/// its own key type. -pub trait Signer: Send + Sync { - /// The endpoint ID: this key's public half. - fn endpoint(&self) -> PublicKey; - /// The Ed25519 signature over `message`. - fn sign(&self, message: &[u8]) -> [u8; SIGNATURE_LEN]; -} - -impl Signer for rind::signature::Ed25519KeyPair { - fn endpoint(&self) -> PublicKey { - PublicKey::from_bytes( - self.public_key() - .as_ref() - .try_into() - .expect("a 32-byte Ed25519 public key"), - ) - } - - fn sign(&self, message: &[u8]) -> [u8; SIGNATURE_LEN] { - rind::signature::Ed25519KeyPair::sign(self, message) - .as_ref() - .try_into() - .expect("a 64-byte Ed25519 signature") - } -} - -/// `POST {base}/enroll`: join with a pre-authorized enrollment key. -#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] -pub struct EnrollRequest { - /// Proof of the endpoint key. - #[serde(flatten)] - pub proof: Proof, - /// The enrollment key the mesh owner created. - pub key: String, - /// The device's requested name, usually its hostname. The coordinator may change it to - /// keep names unique and DNS-safe; the [`Enrollment`] carries the one it assigned. - #[serde(default, skip_serializing_if = "Option::is_none")] - pub name: Option, -} - -/// What a device needs to join: the relays to dial and the token to present. -#[serde_as] -#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] -pub struct Enrollment { - /// A JWT granting publish on `mesh//**` and subscribe on `mesh/**`, rooted at `/`. - pub token: String, - /// When the token expires, unix seconds. Refresh before then. - #[serde_as(as = "kio::clock::serde::UnixSeconds")] - pub expires: kio::clock::Wall, - /// The relays to dial, presenting the token. - pub relays: Vec, - /// The name the coordinator assigned this device. - pub name: String, - /// Every enrolled member of the mesh, this device included. - pub members: Vec, -} - -/// One enrolled device, as listed in an [`Enrollment`]. -#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] -pub struct Member { - /// Its endpoint ID. - pub endpoint: PublicKey, - /// Its unique, DNS-safe name. - pub name: String, - /// The tags of the enrollment key it joined with. - #[serde(default, skip_serializing_if = "Vec::is_empty")] - pub tags: Vec, -} - -/// Why the coordinator refused, in the shape of an OAuth error response (RFC 6749 §5.2). -#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, thiserror::Error)] -#[error("{error}{}", description.as_deref().map(|d| format!(": {d}")).unwrap_or_default())] -#[non_exhaustive] -pub struct ErrorReply { - /// The machine-readable reason. - pub error: ErrorCode, - /// A human-readable detail. - #[serde(rename = "error_description", default, skip_serializing_if = "Option::is_none")] - pub description: Option, -} - -impl ErrorReply { - /// A refusal for `error` with a human-readable detail. - pub fn new(error: ErrorCode, description: impl Into) -> Self { - Self { - error, - description: Some(description.into()), - } - } -} - -/// The reasons a coordinator refuses. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)] -#[serde(rename_all = "snake_case")] -#[non_exhaustive] -pub enum ErrorCode { - /// The body did not parse. - InvalidRequest, - /// The challenge is unknown, used, or expired; fetch a new one. - InvalidChallenge, - /// The signature does not verify against the endpoint ID. - InvalidSignature, - /// The enrollment key is unknown, used, revoked, or expired. - InvalidGrant, - /// The coordinator is busy; retry shortly. - SlowDown, - /// The device is revoked or not enrolled. - AccessDenied, - /// The coordinator failed on its side; retry later. - ServerError, - /// A reason this version does not know. - #[serde(other)] - Unknown, -} - -impl fmt::Display for ErrorCode { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - let value = serde_json::to_value(self).expect("an enum serializes"); - f.write_str(value.as_str().unwrap_or("unknown")) - } -} - -#[cfg(test)] -mod tests { - use super::*; - - fn signer() -> rind::signature::Ed25519KeyPair { - rind::signature::Ed25519KeyPair::generate().unwrap() - } - - #[test] - fn endpoint_vectors() { - let vectors: serde_json::Value = - serde_json::from_str(include_str!("../../../../drafts/moq-sign-00.json")).unwrap(); - for row in vectors["keys"].as_array().unwrap() { - let text = row["endpoint"].as_str().unwrap(); - let member: Member = serde_json::from_value(serde_json::json!({"endpoint": text, "name": "node"})).unwrap(); - assert_eq!(member.endpoint.to_string(), text, "{}", row["id"]); - assert_eq!(serde_json::to_value(&member).unwrap()["endpoint"], text); - } - for row in vectors["invalid_ids"].as_array().unwrap() { - assert!( - serde_json::from_value::(serde_json::json!({"endpoint": row["text"], "name": "node"})).is_err(), - "{}", - row["id"] - ); - } - } - - #[test] - fn a_proof_is_bound_to_its_coordinator() { - let key = signer(); - let base: url::Url = "https://a.example/mesh".parse().unwrap(); - let proof = Proof::sign(&key, &base, vec![7; CHALLENGE_LEN]); - assert!(proof.verify(&base)); - assert!(proof.verify(&"https://a.example/mesh/".parse().unwrap())); - assert!(!proof.verify(&"https://b.example/mesh".parse().unwrap())); - - let mut forged = proof.clone(); - forged.endpoint = signer().endpoint(); - assert!(!forged.verify(&base)); - } - - #[test] - fn error_replies_read_unknown_codes() { - let reply: ErrorReply = serde_json::from_str(r#"{"error":"access_denied"}"#).unwrap(); - assert_eq!(reply.error, ErrorCode::AccessDenied); - let reply: ErrorReply = serde_json::from_str(r#"{"error":"new_thing","error_description":"x"}"#).unwrap(); - assert_eq!(reply.error, ErrorCode::Unknown); - assert_eq!(ErrorCode::SlowDown.to_string(), "slow_down"); - } -} diff --git a/rs/moq-auth/src/serve.rs b/rs/moq-auth/src/serve.rs index 12c474a6b4c..17a4c819177 100644 --- a/rs/moq-auth/src/serve.rs +++ b/rs/moq-auth/src/serve.rs @@ -675,7 +675,6 @@ fn hash(token: &str) -> u64 { pub struct Server { policy: Arc>>, sessions: Arc>, - mesh: Option, secret: Option>, } @@ -687,7 +686,6 @@ impl Server { Ok(Self { policy: Arc::new(std::sync::RwLock::new(Arc::new(Self::validate(policy)?))), sessions: Default::default(), - mesh: None, secret: None, }) } @@ -737,13 +735,6 @@ impl Server { self } - /// Also answer the mesh owner's API under `/mesh` with `coordinator`, behind the same - /// [`secret`](Self::with_secret) as `POST /`. - pub fn with_mesh(mut self, coordinator: crate::mesh::Coordinator) -> Self { - self.mesh = Some(coordinator); - self - } - /// The cadence the session table ages out by, or `None` when no limit needs a table. fn cadence(policy: &Policy) -> Option { let limits = policy.limits; @@ -784,14 +775,10 @@ impl Server { } } - /// The router: `POST /` reading a [`Request`] and writing a [`Grant`], plus the mesh - /// owner's API when [`with_mesh`](Self::with_mesh) set one, all behind the + /// The router: `POST /` reading a [`Request`] and writing a [`Grant`], behind the /// [`secret`](Self::with_secret) when there is one. pub fn router(&self) -> Router { - let mut router = Router::new().route("/", post(handle)).with_state(self.clone()); - if let Some(mesh) = &self.mesh { - router = router.merge(mesh.admin_router()); - } + let router = Router::new().route("/", post(handle)).with_state(self.clone()); match &self.secret { Some(secret) => router.layer(axum::middleware::from_fn_with_state(secret.clone(), guard)), None => router, @@ -1067,15 +1054,12 @@ mod tests { } #[tokio::test] - async fn the_secret_guards_every_route() { - let key = Key::generate(Algorithm::HS256, None).unwrap(); - let config = crate::mesh::Config::new("https://mesh.example".parse().unwrap(), key); + async fn the_secret_guards_the_route() { let server = Server::new(Policy { public: rules(&["**"], &[]), ..Default::default() }) .unwrap() - .with_mesh(crate::mesh::Coordinator::new(config).unwrap()) .with_secret("hunter2:x"); let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); let addr = listener.local_addr().unwrap(); @@ -1092,25 +1076,14 @@ mod tests { ("relay:hunter2%3Ax@", true), (":hunter2%3Ax@", true), ] { - let status = |path: &'static str| { - let url = format!("http://{userinfo}{addr}{path}"); - let http = http.clone(); - async move { - match path { - "/" => http.post(url).json(&request("/demo")).send().await, - _ => http.get(url).send().await, - } - .unwrap() - .status() - } - }; - for path in ["/", "/mesh/keys"] { - assert_eq!( - status(path).await == StatusCode::UNAUTHORIZED, - !admitted, - "{userinfo}{path}" - ); - } + let status = http + .post(format!("http://{userinfo}{addr}/")) + .json(&request("/demo")) + .send() + .await + .unwrap() + .status(); + assert_eq!(status == StatusCode::UNAUTHORIZED, !admitted, "{userinfo}"); } // The relay's client sends the secret from its URL's userinfo. @@ -1124,7 +1097,8 @@ mod tests { let credentials = base64::engine::general_purpose::STANDARD.encode("relay:hunter2:x"); for scheme in ["basic", "BASIC", "Basic", "Bearer"] { let status = http - .get(format!("http://{addr}/mesh/keys")) + .post(format!("http://{addr}/")) + .json(&request("/demo")) .header(reqwest::header::AUTHORIZATION, format!("{scheme} {credentials}")) .send() .await diff --git a/rs/moq-cli/src/auth.rs b/rs/moq-cli/src/auth.rs index 63f1a03c8d6..c2c47eab3e0 100644 --- a/rs/moq-cli/src/auth.rs +++ b/rs/moq-cli/src/auth.rs @@ -129,7 +129,6 @@ impl Args { println!("{payload:#?}"); } - Command::Mesh(mesh) => mesh.run().await?, Command::Sessions(filter) => filter.get("/sessions").await?, Command::Revalidate(filter) => filter.post("/sessions/revalidate").await?, } @@ -220,9 +219,6 @@ enum Command { /// Answer a relay's auth requests with keys, public rules, an mTLS grant, tiers, and session limits. Serve(Serve), - /// Manage a mesh served by `moq auth serve --mesh-url`: enrollment keys and devices. - Mesh(Mesh), - /// List live sessions on a relay's internal listener. Sessions(SessionFilter), @@ -453,10 +449,6 @@ pub struct Serve { #[usage(flatten)] #[serde(default)] limit: SessionLimits, - - #[usage(flatten)] - #[serde(default)] - mesh: MeshServe, } /// What an anonymous session may do. @@ -519,64 +511,6 @@ struct SessionLimits { remote: Option, } -/// The mesh coordinator served beside the auth server. -#[derive(usage_rs::Args, Clone, Debug, PartialEq, serde::Serialize, serde::Deserialize)] -#[usage(unknown_flags = "error", args_override_self = false)] -#[serde(deny_unknown_fields)] -struct MeshServe { - /// Coordinate a mesh: the public base URL devices enroll at, which every enrollment - /// proof is bound to. Serve it behind TLS; devices refuse plain `http://` beyond loopback. - #[usage(name = "mesh-url", long = "mesh-url", setting = "mesh.url")] - #[serde(default, skip_serializing_if = "Option::is_none")] - url: Option, - - /// Where to answer devices at `--mesh-url`, usually behind a TLS-terminating proxy. - #[usage( - name = "mesh-listen", - long = "mesh-listen", - default = "127.0.0.1:4441", - setting = "mesh.listen" - )] - listen: SocketAddr, - - /// The private key that signs mesh tokens; the relays verify it like any other. - #[usage(name = "mesh-key", long = "mesh-key", value_hint = usage_rs::ValueHint::FilePath, extensions("jwk", "json"), setting = "mesh.key")] - #[serde(default, skip_serializing_if = "Option::is_none")] - key: Option, - - /// A relay URL enrolled devices dial (repeatable). - #[usage(name = "mesh-relay", long = "mesh-relay", setting = "mesh.relay")] - #[serde(default)] - relay: Vec, - - /// The JSON file enrollment keys and devices persist to. Without one, a restart forgets them. - #[usage(name = "mesh-state", long = "mesh-state", value_hint = usage_rs::ValueHint::FilePath, setting = "mesh.state")] - #[serde(default, skip_serializing_if = "Option::is_none")] - state: Option, - - /// How long a mesh token lasts; devices refresh it two thirds of the way through. - #[usage( - name = "mesh-token-ttl", - long = "mesh-token-ttl", - default = "1h", - setting = "mesh.token_ttl" - )] - token_ttl: moq_tokio::cli::Duration, -} - -impl Default for MeshServe { - fn default() -> Self { - Self { - url: None, - listen: SocketAddr::from(([127, 0, 0, 1], 4441)), - key: None, - relay: Vec::new(), - state: None, - token_ttl: std::time::Duration::from_secs(3600).into(), - } - } -} - /// Every setting `moq auth serve` merges from its file, declared for the merge's registry. #[allow( dead_code, @@ -608,8 +542,6 @@ struct ServeSettings { expires: Option, #[usage(flatten)] limit: LimitSettings, - #[usage(flatten)] - mesh: MeshSettings, } #[allow( @@ -653,32 +585,8 @@ struct LimitSettings { remote: Option, } -#[allow( - dead_code, - reason = "declares the settings for the usage registry; nothing reads the fields" -)] -#[derive(usage_rs::Config)] -#[usage(prefix = "mesh")] -struct MeshSettings { - #[usage(cli("--mesh-url"))] - url: Option, - #[usage(cli("--mesh-listen"))] - listen: Option, - #[usage(cli("--mesh-key"))] - key: Option, - #[usage(cli("--mesh-relay"))] - relay: Option>, - #[usage(cli("--mesh-state"))] - state: Option, - #[usage(cli("--mesh-token-ttl"))] - token_ttl: Option, -} - impl Serve { /// These flags over the settings in [`file`](Self::file), read from disk now. - /// - /// A mesh setting without a mesh URL is refused here rather than by the parser, so - /// the URL can come from the file and the rest from the command line. fn merged(&self, cli: &usage_rs::config::CliLayer, env: &usage_rs::config::EnvLayer) -> anyhow::Result { let value = self .file @@ -694,7 +602,7 @@ impl Serve { .as_deref() .zip(value.as_ref()) .map(|(path, value)| moq_tokio::cli::FileSource { path, value }); - let (mut merged, resolved) = moq_tokio::cli::Merge { + let (mut merged, _) = moq_tokio::cli::Merge { registry: ServeSettings::SETTINGS_REGISTRY, cli, env, @@ -703,14 +611,6 @@ impl Serve { .apply(self.clone()) .map_err(|err| anyhow::anyhow!("{err}"))?; merged.file = self.file.clone(); - - if merged.mesh.url.is_none() { - for key in ["mesh.listen", "mesh.key", "mesh.relay", "mesh.state", "mesh.token_ttl"] { - if let Some(origin) = resolved.origin_key(key) { - anyhow::bail!("{} needs --mesh-url (mesh.url)", origin.describe()); - } - } - } Ok(merged) } @@ -823,33 +723,6 @@ impl Serve { Ok(Listen::Tcp(addr)) } - /// The mesh coordinator these settings describe, if `--mesh-url` asks for one. - fn coordinator(&self) -> anyhow::Result> { - let Some(url) = &self.mesh.url else { - return Ok(None); - }; - if self.listen_public { - anyhow::bail!( - "--mesh-url serves the mesh owner's API on --listen, which every relay holding --listen-secret could reach, so it cannot be combined with --listen-public" - ); - } - let path = self - .mesh - .key - .as_ref() - .context("--mesh-url needs --mesh-key to sign tokens")?; - let key = moq_auth::Key::from_file(path).with_context(|| format!("failed to read {}", path.display()))?; - let ttl = std::time::Duration::from(self.mesh.token_ttl); - if ttl.is_zero() { - anyhow::bail!("--mesh-token-ttl must be longer than 0s"); - } - let mut config = moq_auth::mesh::Config::new(url.clone(), key); - config.relays = self.mesh.relay.clone(); - config.state = self.mesh.state.clone(); - config.token_ttl = ttl; - Ok(Some(moq_auth::mesh::Coordinator::new(config)?)) - } - /// Re-read the file and answer with its policy, leaving `server` as it was when /// either is refused. fn apply( @@ -863,14 +736,9 @@ impl Serve { Ok(serve) } - /// The settings only a restart applies: the listeners, their secret, and the mesh. - fn restart_only(&self) -> (&str, bool, Option<&PathBuf>, &MeshServe) { - ( - &self.listen, - self.listen_public, - self.listen_secret.as_ref(), - &self.mesh, - ) + /// The settings only a restart applies: the listener and its secret. + fn restart_only(&self) -> (&str, bool, Option<&PathBuf>) { + (&self.listen, self.listen_public, self.listen_secret.as_ref()) } /// The secret in [`listen_secret`](Self::listen_secret), without its trailing newline. @@ -902,7 +770,7 @@ impl Serve { let reload = |quiet: bool| match self.apply(server, cli, &usage_rs::config::EnvLayer::from_process()) { Ok(serve) if serve.restart_only() != running.restart_only() => tracing::error!( file = %path.display(), - "reloaded the policy; listener and mesh changes take effect on restart" + "reloaded the policy; listener changes take effect on restart" ), Ok(_) if quiet => {} Ok(_) => tracing::info!(file = %path.display(), "reloaded the policy"), @@ -925,25 +793,7 @@ impl Serve { if let Some(secret) = serve.secret()? { server = server.with_secret(secret); } - // The device-facing contract runs beside the auth server; either stopping ends both. - let mut mesh = None; - if let Some(coordinator) = serve.coordinator()? { - let addr = serve.mesh.listen; - let listener = tokio::net::TcpListener::bind(addr) - .await - .with_context(|| format!("failed to bind {addr}"))?; - let addr = listener.local_addr().context("failed to read the bound address")?; - tracing::info!(%addr, url = ?serve.mesh.url, "mesh coordinator listening"); - mesh = Some((coordinator.clone(), listener)); - server = server.with_mesh(coordinator); - } let reload = self.reload(&serve, &server, cli); - let mesh = async move { - match mesh { - Some((coordinator, listener)) => coordinator.serve(listener).await, - None => std::future::pending().await, - } - }; let auth = async { match listen { Listen::Tcp(addr) => { @@ -970,133 +820,11 @@ impl Serve { }; tokio::select! { res = auth => res, - res = mesh => res.context("the mesh coordinator stopped"), res = reload => res, } } } -/// `moq auth mesh`: the mesh owner's commands, sent to the auth server's own listener. -#[derive(usage_rs::Args, Clone, Debug)] -#[usage(unknown_flags = "error", args_override_self = false)] -pub struct Mesh { - /// The auth server's listener, which serves the owner's API alongside `POST /`: - /// `http://` for a TCP `--listen`, or `unix:///path` for a unix socket. - /// With `--listen-secret`, put the secret in the URL as its password: `http://owner:SECRET@127.0.0.1:4440`. - #[usage(long, default = "http://127.0.0.1:4440")] - url: url::Url, - - #[usage(subcommand)] - command: MeshCommand, -} - -#[derive(usage_rs::Subcommands, Clone, Debug)] -enum MeshCommand { - /// Create an enrollment key, printing it once. - CreateKey { - /// A tag every device it enrolls carries (repeatable). - #[usage(long)] - tag: Vec, - - /// How long until it stops enrolling. Never, by default. - #[usage(long)] - expires: Option, - - /// Enroll any number of devices, instead of one. - #[usage(long)] - reusable: bool, - }, - - /// List enrollment keys, without their secrets. - Keys, - - /// Revoke an enrollment key so it enrolls nothing new. Its devices stay. - RevokeKey { - /// The key's ID: the part before the dot. - id: String, - }, - - /// List enrolled devices. - Devices, - - /// Revoke a device: its next refresh fails, and it cannot enroll again. - Revoke { - /// The device's endpoint ID. - endpoint: String, - }, -} - -impl Mesh { - async fn run(self) -> anyhow::Result<()> { - let (client, base) = self.client()?; - let url = |path: &str, segments: &[&str]| { - let mut url = base.clone(); - let prefix = url.path().trim_end_matches('/').to_string(); - url.set_path(&format!("{prefix}{path}")); - if let Ok(mut parts) = url.path_segments_mut() { - parts.extend(segments); - } - url - }; - let request = match &self.command { - MeshCommand::CreateKey { tag, expires, reusable } => { - let mut new = moq_auth::mesh::NewKey::default(); - new.tags = tag.clone(); - new.expires_in = expires.map(std::time::Duration::from); - new.reusable = *reusable; - client.post(url("/mesh/keys", &[])).json(&new) - } - MeshCommand::Keys => client.get(url("/mesh/keys", &[])), - MeshCommand::RevokeKey { id } => client.delete(url("/mesh/keys", &[id])), - MeshCommand::Devices => client.get(url("/mesh/devices", &[])), - MeshCommand::Revoke { endpoint } => client.delete(url("/mesh/devices", &[endpoint])), - }; - let response = request.send().await.with_context(|| { - let mut shown = self.url.clone(); - let _ = shown.set_password(None); - format!("failed to reach {shown}") - })?; - let status = response.status(); - let body = response.text().await.context("failed to read the reply")?; - if status == reqwest::StatusCode::NOT_FOUND { - // The routes exist only with `--mesh-url`, so a miss there is the likelier cause. - anyhow::bail!(match self.command { - MeshCommand::RevokeKey { .. } if !body.is_empty() => "no enrollment key has that ID", - MeshCommand::Revoke { .. } if !body.is_empty() => "no device has that endpoint ID", - _ => "not found; is the server running with --mesh-url?", - }); - } - if !status.is_success() { - anyhow::bail!("{status}: {body}"); - } - match serde_json::from_str::(&body) { - Ok(value) => println!("{}", serde_json::to_string_pretty(&value)?), - Err(_) if body.is_empty() => println!("ok"), - Err(_) => println!("{body}"), - } - Ok(()) - } - - /// The HTTP client for `--url` and the URL its requests target: a `unix://` URL names - /// the socket, and the requests go to the server's root over it. - fn client(&self) -> anyhow::Result<(reqwest::Client, url::Url)> { - let builder = reqwest::Client::builder(); - match self.url.scheme() { - "http" | "https" => Ok((builder.build()?, self.url.clone())), - #[cfg(unix)] - "unix" => { - let path = self - .url - .to_file_path() - .map_err(|()| anyhow::anyhow!("--url {} names no socket path", self.url))?; - let target = url::Url::parse("http://localhost/").expect("a constant URL parses"); - Ok((builder.unix_socket(path).build()?, target)) - } - _ => anyhow::bail!("--url expects http://, https://, or unix://, got {}", self.url), - } - } -} - #[derive(Debug)] enum Listen { Tcp(SocketAddr), @@ -1289,102 +1017,6 @@ mod tests { assert_eq!(merged.key, Some(key)); } - #[test] - fn mesh_flags_need_a_mesh_url_and_key() { - let err = merged(&["moq", "auth", "serve", "--mesh-relay", "https://relay.example"]) - .err() - .unwrap(); - assert!(err.to_string().contains("--mesh-relay needs --mesh-url"), "{err}"); - - let flags = serve(&["moq", "auth", "serve", "--mesh-url", "https://mesh.example"]); - let err = flags.coordinator().err().expect("no signing key"); - assert!(err.to_string().contains("--mesh-key"), "{err}"); - - let dir = tempfile::tempdir().unwrap(); - let key = dir.path().join("mesh.jwk"); - moq_auth::Key::generate(Algorithm::HS256, None) - .unwrap() - .to_file(&key) - .unwrap(); - let flags = serve(&[ - "moq", - "auth", - "serve", - "--mesh-url", - "https://mesh.example", - "--mesh-key", - key.to_str().unwrap(), - "--mesh-relay", - "https://a.example", - "--mesh-relay", - "https://b.example", - ]); - assert_eq!(flags.mesh.relay.len(), 2); - assert!(flags.mesh.listen.ip().is_loopback()); - assert!(flags.coordinator().unwrap().is_some()); - assert!(serve(&["moq", "auth", "serve"]).coordinator().unwrap().is_none()); - - // The owner's API rides --listen, so a public one would hand out enrollment keys. - let flags = serve(&[ - "moq", - "auth", - "serve", - "--listen", - "0.0.0.0:4440", - "--listen-public", - "--mesh-url", - "https://mesh.example", - "--mesh-key", - key.to_str().unwrap(), - ]); - let err = flags.coordinator().err().expect("a public owner API"); - assert!(err.to_string().contains("--listen-public"), "{err}"); - } - - #[cfg(unix)] - #[tokio::test] - async fn mesh_commands_reach_a_unix_listener() { - let dir = tempfile::tempdir().unwrap(); - let key = moq_auth::Key::generate(Algorithm::HS256, None).unwrap(); - let config = moq_auth::mesh::Config::new("https://mesh.example".parse().unwrap(), key); - let coordinator = moq_auth::mesh::Coordinator::new(config).unwrap(); - let server = moq_auth::serve::Server::new(Policy::default()) - .unwrap() - .with_mesh(coordinator.clone()); - let path = dir.path().join("auth.sock"); - let listener = tokio::net::UnixListener::bind(&path).unwrap(); - tokio::spawn(async move { server.serve_unix(listener).await }); - - let url = format!("unix://{}", path.display()); - run(&["moq", "auth", "mesh", "--url", &url, "create-key", "--expires", "1d"]) - .await - .unwrap(); - let keys = coordinator.keys(); - assert_eq!(keys.len(), 1); - assert!(keys[0].expires.is_some()); - } - - #[test] - fn mesh_commands_parse() { - for args in [ - &[ - "moq", - "auth", - "mesh", - "create-key", - "--tag", - "robot", - "--reusable", - "--expires", - "1d", - ][..], - &["moq", "auth", "mesh", "devices"], - &["moq", "auth", "mesh", "--url", "http://127.0.0.1:9000", "revoke", "ab"], - ] { - assert!(matches!(parse(args).unwrap().command, Command::Mesh(_)), "{args:?}"); - } - } - #[test] fn serve_refuses_a_public_bind_without_saying_so() { let err = serve(&["moq", "auth", "serve", "--listen", "0.0.0.0:4440"]) @@ -1547,35 +1179,18 @@ link = "peer" publish = ["alice/**"] link = "none" publish_rate = 1000000 - -[mesh] -relay = ["https://relay.example"] "# ), ) .unwrap(); let path = file.to_str().unwrap(); - let err = merged(&["moq", "auth", "serve", path]).err().unwrap(); - assert!(err.to_string().contains("mesh.relay needs --mesh-url"), "{err}"); - - let (serve, _) = merged(&[ - "moq", - "auth", - "serve", - path, - "--tier", - "from-flag", - "--mesh-url", - "https://mesh.example", - ]) - .unwrap(); + let (serve, _) = merged(&["moq", "auth", "serve", path, "--tier", "from-flag"]).unwrap(); assert!(matches!(serve.listener(), Ok(Listen::Tcp(addr)) if !addr.ip().is_loopback())); assert_eq!( serve.listen_secret.as_deref(), Some(std::path::Path::new("/etc/moq/auth.secret")) ); - assert_eq!(serve.mesh.relay.len(), 1); let policy = serve.policy().unwrap(); assert_eq!(policy.tier.as_deref(), Some("from-flag")); assert_eq!(policy.revalidate, Some(std::time::Duration::from_secs(30))); @@ -1588,17 +1203,7 @@ relay = ["https://relay.example"] assert_eq!(rule.publish_rate, moq_auth::Rate::from_bps(1_000_000).ok()); // A flag replaces the file's list rather than adding to it. - let (serve, _) = merged(&[ - "moq", - "auth", - "serve", - path, - "--mtls-subscribe", - "room/**", - "--mesh-url", - "https://mesh.example", - ]) - .unwrap(); + let (serve, _) = merged(&["moq", "auth", "serve", path, "--mtls-subscribe", "room/**"]).unwrap(); assert_eq!(serve.mtls.subscribe, vec!["room/**".parse::().unwrap()]); for (body, needs) in [ From 82c9e15d27ba86a5c061dd76e4cf0dfcc35153c5 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 14 Nov 2026 10:48:38 -0800 Subject: [PATCH 2/5] quest: move mesh, teleop, and link quality plans to Kixel moq keeps what a large live-media platform would use; mesh work moves to Kixel, which plans and builds it in kixel-ai/kixel.ai. This deletes the mesh epic (relay node, Endpoint, address hints, sparse overlay, status track, certificate credentials), its m2 contract quests, the parked mesh guide, JS enrollment, ICE, and keys quests, the teleop questline, link quality, the certificate challenge, and the stored signing key quest, plus the teleoperation use-case guide and its moq-json example. Co-Authored-By: Claude Opus 5.5 --- doc/.vitepress/config.ts | 1 - doc/bin/demo.md | 3 +- doc/concept/use-case/index.md | 1 - doc/concept/use-case/teleoperation.md | 76 -------------- quest/m1/stream/README.md | 2 +- quest/m2/README.md | 8 +- quest/m2/api-mesh-enrollment.md | 24 ----- quest/m2/mesh-device-bounds.md | 12 --- quest/m2/mesh-grant-shape.md | 43 -------- quest/m3/README.md | 6 +- quest/m3/cache-routing/README.md | 2 +- quest/m3/cluster-routing/README.md | 1 - quest/m3/cluster-routing/federation.md | 4 +- quest/m3/mesh/README.md | 138 ------------------------- quest/m3/mesh/addr.md | 48 --------- quest/m3/mesh/cert-credentials.md | 67 ------------ quest/m3/mesh/endpoint.md | 64 ------------ quest/m3/mesh/node.md | 51 --------- quest/m3/mesh/overlay.md | 106 ------------------- quest/m3/mesh/status.md | 35 ------- quest/m4/cert-challenge.md | 28 ----- quest/m4/crypto-key-load.md | 20 ---- quest/m5/link-quality.md | 38 ------- quest/m5/mesh-guide.md | 21 ---- quest/m5/mesh-ice.md | 22 ---- quest/m5/mesh-js-enroll.md | 26 ----- quest/m5/mesh-keys.md | 23 ----- quest/m5/teleop-arbitration.md | 45 -------- quest/m5/teleop-mavlink.md | 93 ----------------- quest/m5/teleop-proof.md | 40 ------- quest/m5/teleop-robot.md | 104 ------------------- rs/moq-json/Cargo.toml | 5 - rs/moq-json/examples/teleop.rs | 79 -------------- 33 files changed, 13 insertions(+), 1223 deletions(-) delete mode 100644 doc/concept/use-case/teleoperation.md delete mode 100644 quest/m2/api-mesh-enrollment.md delete mode 100644 quest/m2/mesh-device-bounds.md delete mode 100644 quest/m2/mesh-grant-shape.md delete mode 100644 quest/m3/mesh/README.md delete mode 100644 quest/m3/mesh/addr.md delete mode 100644 quest/m3/mesh/cert-credentials.md delete mode 100644 quest/m3/mesh/endpoint.md delete mode 100644 quest/m3/mesh/node.md delete mode 100644 quest/m3/mesh/overlay.md delete mode 100644 quest/m3/mesh/status.md delete mode 100644 quest/m4/cert-challenge.md delete mode 100644 quest/m4/crypto-key-load.md delete mode 100644 quest/m5/link-quality.md delete mode 100644 quest/m5/mesh-guide.md delete mode 100644 quest/m5/mesh-ice.md delete mode 100644 quest/m5/mesh-js-enroll.md delete mode 100644 quest/m5/mesh-keys.md delete mode 100644 quest/m5/teleop-arbitration.md delete mode 100644 quest/m5/teleop-mavlink.md delete mode 100644 quest/m5/teleop-proof.md delete mode 100644 quest/m5/teleop-robot.md delete mode 100644 rs/moq-json/examples/teleop.rs diff --git a/doc/.vitepress/config.ts b/doc/.vitepress/config.ts index d54e36f5013..e89174b3ebe 100644 --- a/doc/.vitepress/config.ts +++ b/doc/.vitepress/config.ts @@ -132,7 +132,6 @@ export default defineConfig({ { text: "vs RTMP/SRT", link: "/concept/use-case/contribution" }, { text: "vs WebRTC", link: "/concept/use-case/conferencing" }, { text: "For AI", link: "/concept/use-case/ai" }, - { text: "For teleoperation", link: "/concept/use-case/teleoperation" }, ], }, ], diff --git a/doc/bin/demo.md b/doc/bin/demo.md index 2d779d90fc9..7162754af98 100644 --- a/doc/bin/demo.md +++ b/doc/bin/demo.md @@ -46,8 +46,7 @@ What it demonstrates: Code: [`rs/moq-boy`](https://github.com/moq-dev/moq/tree/main/rs/moq-boy) (emulator and publisher) and [`demo/boy`](https://github.com/moq-dev/moq/tree/main/demo/boy) -(the `` element and page). The same -shape covers teleoperation: swap the emulator for a robot. +(the `` element and page). ## CARLA diff --git a/doc/concept/use-case/index.md b/doc/concept/use-case/index.md index aacba7eaac8..c27a345c0a1 100644 --- a/doc/concept/use-case/index.md +++ b/doc/concept/use-case/index.md @@ -15,4 +15,3 @@ of scale come from. | [Contribution](/concept/use-case/contribution) | RTMP, SRT | Pull-based delivery, optional on-demand encoding, and instance-based failover. | | [Conferencing](/concept/use-case/conferencing) | WebRTC | Bidirectional on one session, browser control of the pipeline, no SDP or TURN. | | [AI](/concept/use-case/ai) | WebRTC, WebSockets | Adjustable reliability, on-demand inference, media and prompts on one connection. | -| [Teleoperation](/concept/use-case/teleoperation) | VPN plus video and control flows | Fresh media, replaceable state, and ordered commands share one session. | diff --git a/doc/concept/use-case/teleoperation.md b/doc/concept/use-case/teleoperation.md deleted file mode 100644 index 618970adc52..00000000000 --- a/doc/concept/use-case/teleoperation.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -title: MoQ for Teleoperation -description: Carry video, current control state, and ordered commands on one session ---- - -# MoQ for Teleoperation - -A remote operator needs fresh video from a machine and a responsive control path -back to it. MoQ carries both directions over one session to a relay. The machine -connects outbound, publishes its video and telemetry, and subscribes to the -operator's control broadcast. Additional viewers subscribe through the relay -without another upload from the machine. - -A VPN plus separate video and control UDP flows can provide reachability, but the -application still has to coordinate their delivery and congestion. With MoQ, -tracks sent in the same direction share a connection and its congestion budget, -so the machine can rank telemetry above its video. Priority only orders data -queued at one sender: the operator's commands travel the other way, so they never -compete with the machine's video, and priority does not guarantee a deadline or -make a command safe to execute. Keep each kind of data on a track with the -framing it needs. - -## Choose the delivery shape - -| Data | Shape | What falling behind means | -| --- | --- | --- | -| Steering, throttle, current position, sensor state | JSON snapshots | Catch up to the newest state; intermediate values are superseded. | -| Arm, stop, a parameter change, mission or file records | Ordered JSON stream | Consume every record in order, or report an error. | -| Camera and microphone | Media groups | Join at a decodable boundary and choose the playback buffer. | - -Use [`moq-json`](https://docs.rs/moq-json) snapshot mode for replaceable state. It -starts a group with a full value and carries changes as deltas. A reader can skip -to a newer group and reconstruct the current value. Stream mode puts an ordered -log in one group; it reports a broken or rolled stream instead of silently -presenting a gap as a complete log. In Rust that group grows for the track's -life, but the cache keeps only its newest 32 MiB, so a reader that falls further -behind or joins late fails with `Evicted`; `@moq/json` still bounds it at 8,192 -records. Either way, open a new command track per control session rather than -replaying one forever. These are framing choices, not a universal -'reliable' switch on the session. - -Transport delivery is not command execution. An application that needs to know -whether a command ran sends a command ID and an acknowledgment, defines duplicate -handling, and rejects expired control state at the machine. It also decides who -may control the machine. A subscriber being connected does not grant operator -ownership. - -## Run a non-media example - -From the repository root: - -```bash -nix develop --command cargo run -p moq-json --example teleop -``` - -The [example source](https://github.com/moq-dev/moq/blob/main/rs/moq-json/examples/teleop.rs) -creates `control.json` and `commands.json` tracks in one in-process broadcast. -It writes several control states before reading and gets only the latest one; -it reads all three commands in order. It uses fixed media timestamps and needs -no network service or robot. - -```text -Current control: Control { speed: 0, turn: 0 } -Command: Arm -Command: Stop -Command: Disarm -``` - -To send these tracks over a network, publish the broadcast through a -[`moq-tokio` origin](https://docs.rs/moq-tokio/latest/moq_tokio/origin/index.html) -and connect that origin to a [relay](/bin/relay/). Publish the machine's media -and telemetry on its broadcast and the operator's control on another. Hang's -[catalog](/concept/hang) advertises JSON and binary tracks beside audio and video -when an application needs to discover them together. Give each machine and -operator the [token permissions](/bin/relay/auth) for their own publish and -subscribe paths. diff --git a/quest/m1/stream/README.md b/quest/m1/stream/README.md index 82ee813ba3d..f3967bb7d7a 100644 --- a/quest/m1/stream/README.md +++ b/quest/m1/stream/README.md @@ -37,7 +37,7 @@ Decided 2026-10-25 with the maintainer (mesh planning interview): - Connection setup is generic: a listener accepts connections a dialer announces under a path prefix, and the API names peers by path. Endpoint identity, signing, and service names belong to the - [mesh](/quest/m3/mesh/README.md), which builds on this. + mesh (Kixel's, downstream), which builds on this. Amended 2026-11-01 with the maintainer (socket planning interview), 2026-11-03 when sockets became endpoints, and 2026-11-04 when the socket's diff --git a/quest/m2/README.md b/quest/m2/README.md index 86bb7a1ffb7..39dae59ece1 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -38,14 +38,14 @@ Decided 2026-11-12 (maintainer, /quest-audit before the cut): offset goes on the wire version after lite-07 (rejected for each: staying in m2). The padding and JS rung disable quests are parked in m5. - ✅ Joined m2, each a small break the cut would otherwise freeze: the mesh - [grant shape](/quest/m2/mesh-grant-shape.md) and - [key budget](/quest/m2/mesh-device-bounds.md), since `moq_auth::mesh` - publishes for the first time at this cut; + grant shape and key budget, since `moq_auth::mesh` + publishes for the first time at this cut (both dropped 2026-11-14, when + `moq_auth::mesh` left moq for Kixel); [auth key validation](/quest/m2/auth-key-validation.md); the [RTMPS connect](/quest/m2/rtmp-rtmps-tcurl.md) signature; [usage's derived settings](/quest/m2/usage-derive-settings.md); [first-publish versions](/quest/m2/first-publish-versions.md); and three - API fences: [mesh enrollment](/quest/m2/api-mesh-enrollment.md), + API fences: mesh enrollment (dropped with it), [the capture selections](/quest/m2/api-capture-fences.md), and [the acknowledgment hook's trait](/quest/m2/api-ack-hook-trait.md). None is on the routes chain, which stays the critical path. diff --git a/quest/m2/api-mesh-enrollment.md b/quest/m2/api-mesh-enrollment.md deleted file mode 100644 index 3c9722bb737..00000000000 --- a/quest/m2/api-mesh-enrollment.md +++ /dev/null @@ -1,24 +0,0 @@ -# [XS] A mesh enrollment reply can grow - -## Goal - -`moq_auth::mesh::Enrollment` and the other reply and request structs the mesh -module exports are `#[non_exhaustive]` with constructors where a caller builds -one, so a later field (a certificate, a revocation hint) is additive. The cut -publishes the module for the first time, and today `Enrollment` is a plain -struct with a public `token: String`. - -## Plan - -Decided 2026-11-12 (maintainer, /quest-audit before the cut): - -- ✅ Fence the structs now (rejected: publishing them open and breaking - later). -- ✅ The mesh [relay node](/quest/m3/mesh/node.md) takes a credential handle - and the application enrolls however it likes (rejected: the coordinator and - `Enrollment` learning to issue certificates; deferring). So this quest adds - no certificate field; it only keeps one possible. - -Sweep `rs/moq-auth/src/mesh/` for every public struct and enum a caller -matches or builds, and fence each the same way. Public API: breaking for a -caller that builds or exhaustively destructures one. Wire: none. diff --git a/quest/m2/mesh-device-bounds.md b/quest/m2/mesh-device-bounds.md deleted file mode 100644 index 2a9976d684f..00000000000 --- a/quest/m2/mesh-device-bounds.md +++ /dev/null @@ -1,12 +0,0 @@ -# [S] Mesh enrollment is bounded and revocation takes effect - -## Goal - -The mesh coordinator (`rs/moq-auth/src/mesh/coordinator.rs`) stops assuming its device store stays small and its revocations wait for token expiry: - -- A reusable enrollment key carries a use budget set when it is created, so the coordinator holds at most what its keys allow. Today a reusable key enrolls any number of endpoints, and each enroll clones the whole store, rewrites the state file and returns every member under one mutex, so a leaked key can mint thousands. -- A revoked device loses access within a bounded time, not only at its token's `exp` (1h by default): either `decide` checks revocation for mesh tokens, or mesh tokens are short-lived with revalidation on. - -## Plan - -Found by the 2026-11-07 invariant audit (auth-7, auth-8). The budget belongs to the key's owner, not a server-wide constant or rate limit, per REVIEW.md's rule against constant caps. diff --git a/quest/m2/mesh-grant-shape.md b/quest/m2/mesh-grant-shape.md deleted file mode 100644 index 12e234606d1..00000000000 --- a/quest/m2/mesh-grant-shape.md +++ /dev/null @@ -1,43 +0,0 @@ -# [S] Mesh grants scope nodes and services apart - -## Goal - -A mesh member's grant publishes `mesh/node//**` and the two service -patterns `mesh/service/*/.server/` and `mesh/service/*/.client/`, in -place of `mesh//**`. A service named after another member can no longer -put `.server/` under its own grant and answer as that member. The -shape lands before the cut publishes `moq_auth::mesh` for the first time, so -no enrolled device ever holds the old grant from a release. - -## Plan - -Split from the [Endpoint API](/quest/m3/mesh/endpoint.md) on 2026-11-12 -(maintainer, /quest-audit before the cut): ✅ the grant shape lands in m2 on -its own (rejected: landing with the Endpoint after the cut, with the -coordinator issuing both shapes for one release). `rs/moq-auth/src/mesh` -exists on `main` and not on `release`, so there is nothing to migrate and no -dual-grant rollout. - -The shape was decided 2026-11-07 (maintainer, /quest-complete, from the -review of [moq-dev/moq#8214](https://github.com/moq-dev/moq/pull/8214)): -members own `mesh/node//**` and each service lives at -`mesh/service/` (rejected: `mesh/.service/`; keeping -`mesh/` and refusing service names shaped like an endpoint ID). A -service name is one path segment, so `*` in the grant covers it. - -- `moq_auth::mesh`'s coordinator issues the new patterns - (`rs/moq-auth/src/mesh/coordinator.rs`), with its docs - (`rs/moq-auth/src/mesh/mod.rs`) and contract tests. -- `doc/bin/relay/auth.md`'s mesh enrollment section names the new grant. -- moq-crypto's and `@moq/crypto`'s sign examples and tests that use - `mesh//agent` paths move to `mesh/node//agent`. -- A hosted coordinator implementing the JSON contract issues the same grant; - the PR's Impact section says so. Applications built on `main` re-enroll. - -Public API: the grant a mesh coordinator issues changes; no Rust or JS -signature does. Wire: none. - -## Related - -- [Mesh enrollment is bounded](/quest/m2/mesh-device-bounds.md) - the other mesh contract change before the cut -- [Mesh enrollment can grow](/quest/m2/api-mesh-enrollment.md) - fences the enrollment reply diff --git a/quest/m3/README.md b/quest/m3/README.md index 8b91777dfea..f516b9c3f12 100644 --- a/quest/m3/README.md +++ b/quest/m3/README.md @@ -26,14 +26,16 @@ a new necessary API break brings that decision to the maintainer before proceeding. Decided 2026-11-03 (maintainer, Kixel's quest audit): [byte streams](/quest/m1/stream/README.md) -and the [mesh libraries](/quest/m3/mesh/README.md) rank first here, ahead +and the mesh libraries rank first here, ahead of [release execution](/quest/m3/release/README.md). Kixel's daemon builds on them from `main` through a git submodule, so it needs them merged, not released. They stay out of m1 and m2 so the cut does not wait on them; each starts as soon as its prerequisites (sockets, fastest-reader pacing, outbound federation) land. Superseded 2026-11-12 (maintainer, /quest-audit): both moved to [m1](/quest/m1/README.md), since the cut no longer waits on m1. The mesh -libraries came back 2026-11-13, below; byte streams stay in m1. +libraries came back 2026-11-13, below; byte streams stay in m1. Both left +moq for Kixel on 2026-11-14 (maintainer): moq keeps what a large live-media +platform would use. Decided 2026-11-12 (maintainer, /quest-audit before the cut), from a read of every quest against the code and the downstream quest trees: diff --git a/quest/m3/cache-routing/README.md b/quest/m3/cache-routing/README.md index f5eb60e30eb..08e865e68b5 100644 --- a/quest/m3/cache-routing/README.md +++ b/quest/m3/cache-routing/README.md @@ -12,7 +12,7 @@ stays loop-free under stale state, converges once demand and topology stop changing, and sends no per-broadcast message for an idle broadcast. Non-goals: steering clients to the relay that holds a broadcast; decisions -from measured load or RTT ([Link quality](/quest/m5/link-quality.md)); +from measured load or RTT (link quality, planned downstream in Kixel); browsers serving copies they receive; a relay on neither consumer's cheapest route becoming a shared stop; one copy across a multi-hop sparse overlay rather than among direct neighbors; and creating links, since routing diff --git a/quest/m3/cluster-routing/README.md b/quest/m3/cluster-routing/README.md index 0a259809ddb..4c3cf057c5f 100644 --- a/quest/m3/cluster-routing/README.md +++ b/quest/m3/cluster-routing/README.md @@ -160,5 +160,4 @@ setup after the capabilities land. - [P2P](/quest/m4/p2p/README.md) - application-selected peer sessions feed the shared routing model - [Routing guide](/quest/m3/routing-guide.md) - concept page and federation setup; P2P extends it in m4 - [Route wakes](/quest/m4/route-wakes.md) - local work follows affected fronts rather than the whole covered table -- [Link quality](/quest/m5/link-quality.md) - optional measured link cost, distinct from delivery-health failover - [Cache-aware routing](/quest/m3/cache-routing/README.md) - per-broadcast warm routes ranked against these node routes diff --git a/quest/m3/cluster-routing/federation.md b/quest/m3/cluster-routing/federation.md index d9239984dd7..83099481702 100644 --- a/quest/m3/cluster-routing/federation.md +++ b/quest/m3/cluster-routing/federation.md @@ -64,8 +64,8 @@ session integration (landed in [moq-dev/moq#8023](https://github.com/moq-dev/moq [route epic](/quest/m2/cluster-routes/README.md). Federation is relay-to-relay, so browser route-key proofs, the Playwright pin they wait on, and the mixed-version follow-up (mount takeover) gates the -lite-07 cut, not this. Its consumer, the [mesh node](/quest/m3/mesh/node.md), -is a Rust relay too. Acceptance still brings the js/net route layer for its +lite-07 cut, not this. Its consumer, Kixel's mesh relay node (downstream, in +kixel-ai/kixel.ai), is a Rust relay too. Acceptance still brings the js/net route layer for its Rust/JS interop checks. Rejected: session integration alone, which drops the proven contract the wire rule below relies on. diff --git a/quest/m3/mesh/README.md b/quest/m3/mesh/README.md deleted file mode 100644 index 268b1d25a54..00000000000 --- a/quest/m3/mesh/README.md +++ /dev/null @@ -1,138 +0,0 @@ -# Mesh libraries - -## Goal - -A Rust application or a relay node enrolls into a mesh served by a relay -(self-hosted with `moq auth`, or a hosted coordinator implementing the same -contract) and reaches any other node by its Ed25519 endpoint ID, through -relays or directly. Nodes that can dial each other (one LAN, a shared VPC or -Docker network, a public address) talk directly over a sparse, deterministic -overlay that forwards for the rest; everything else, and any partition of -the overlay, goes through relays. Content is signed by default, and each node -publishes a `.status` track describing its links. - -This repository ships the libraries. The mesh binary and its UI live outside -it: Kixel builds a `kixel` daemon on these libraries, and any other binary -can do the same. - -Non-goals: a mesh daemon or CLI here (`moq mesh` is not added to moq-cli), -MagicDNS, agent protocols, a graph UI, internet P2P or NAT traversal in m3 -(remote traffic always crosses a relay; [mesh over ICE](/quest/m5/mesh-ice.md) -is parked in m5), TUN devices or IP routing, encryption (parked in -[mesh keys](/quest/m5/mesh-keys.md)), a member list that a compromised -coordinator cannot extend, and Windows. - -## Plan - -Moved from m1 to m3 2026-11-13 (maintainer, /quest-plan): it waits on m2 -work, and no quest requires one in a later milestone, since a blocker ranks -by its own milestone. ✅ m3 (rejected: m2, where the cut would wait on it; -pulling the m2 blocker into m1). It ranks right after m2, when it can start. - -Decided 2026-11-02 with the maintainer (Kixel's planning interview, which -moved moq.pro's hosted mesh and T3 Code lines to Kixel): **moq for -libraries, Kixel for the binary and UI.** This epic keeps the `Endpoint` API, -address hints, the sparse overlay, and the `.status` track, beside the byte -streams, sockets, federation, and `moq_auth::mesh` contract it builds on. The -daemon, MagicDNS, agents and their MCP server, the graph component, robot -builds and tests, and the user guide moved to Kixel. Why: Kixel is the brand -for non-media work on MoQ, and a mesh node is a moq-relay instance, so any -moq library, media included, can point at one. - -Decided 2026-10-25 with the maintainer (planning interview), still in force: - -- Traffic is app-level [byte streams](/quest/m1/stream/README.md) and named - tracks, never IP packets. -- The coordinator contract is generic and lives here: enrollment issues a - relay URL and a token, and self-hosters run it with `moq auth`. A hosted - service implements the same contract with its own accounts and billing. -- Direct paths only to addresses a node can already dial: `[cluster.lan]` - mDNS, plus [address hints](/quest/m3/mesh/addr.md) published on mesh - tracks. No ICE, STUN, or hole punching; the [P2P](/quest/m4/p2p/README.md) - line is separate. -- An endpoint ID is a persistent Ed25519 device key, iroh-style. It is not - the [routing identity](/quest/m2/cluster-routes/README.md), which each - process declares fresh. -- Addressing is announcement: each node publishes its address and status - tracks under `mesh/node//...`, and reaching an ID is subscribing - there. There is no separate directory. Services are not under a member - subtree: each is one socket endpoint at `mesh/service/` (decided - 2026-11-07, see [Endpoint API](/quest/m3/mesh/endpoint.md)). -- Certificates can stand in for the token (decided 2026-11-07, maintainer, - Kixel's planning): a downstream mesh that must admit and evict devices - with no internet issues long-lived certificates from its own signers, and - [certificate credentials](/quest/m3/mesh/cert-credentials.md) checks them - against a trust store it supplies. The coordinator's token stays the - default for `moq auth`. -- The token is the binding: enrollment proves the device key, then the - coordinator issues a grant to publish `mesh/node//**` and subscribe - `mesh/**`, plus publish on its own [socket](/doc/concept/socket.md) - endpoints, `mesh/service/*/.server/` and `mesh/service/*/.client/`, - so it can accept and dial under its own id (amended 2026-11-01, - 2026-11-03 when sockets became endpoints, 2026-11-06 when endpoints split - into server and client roles, and 2026-11-07 when member and service - paths were scoped apart; the [Endpoint API](/quest/m3/mesh/endpoint.md) - changes the shipped `mesh//**` grant). Relays need no new check. Membership is the coordinator's - token only; a Tailnet-Lock analog was declined. -- Protection is per broadcast, with levels that nest: identity (the token - only), authentication (per-frame Ed25519 signatures, the default), and - later encryption (signed plus moq-crypto's e2ee, waiting on - [mesh keys](/quest/m5/mesh-keys.md)). -- Enrollment is automated, with no human in the loop (decided 2026-10-30): a - device presents an enrollment key the owner created and then refreshes by - proving its key. A hosted coordinator may put its own sign-in in front of - minting that key. -- The API is iroh-shaped (not wire-compatible with iroh). -- It builds on [outbound federation](/quest/m3/cluster-routing/federation.md): - a node's local relay dials its relays as `upstream` links and ranks LAN - peers above them. - -Decided 2026-10-31 with the maintainer (moq.pro's mesh planning interview): - -- mDNS alone is not enough. It fails in Docker bridge and overlay networks, - cloud VPCs, across subnets and VLANs, on isolated Wi-Fi, and for a macOS - launchd agent denied Local Network access, so nodes publish - [address hints](/quest/m3/mesh/addr.md). -- Direct links form a deterministic [sparse overlay](/quest/m3/mesh/overlay.md) - computed locally from the member list, with an optional per-node degree - cap. Every node keeps its relay uplink, which carries traffic across an - overlay partition. -- `.status` carries each link's RTT, and an application embedding `Endpoint` - publishes it too ([status track](/quest/m3/mesh/status.md)). - -This epic owns the end-to-end test: three [mesh relay nodes](/quest/m3/mesh/node.md), -two in network namespaces sharing a LAN segment and a third behind a relay -only, enrolled through `moq auth`; an `Endpoint` on each dials the others' -services, taking the LAN path between the first two and the relay path to -the third, and survives a relay restart. - -Decided 2026-11-12 (maintainer, /quest-audit before the cut): - -- ✅ (Superseded 2026-11-13: this epic is back in m3, above.) This epic and [byte streams](/quest/m1/stream/README.md) moved to m1 - from the head of m3 (rejected: staying in m3). The cut no longer waits on - m1, and a downstream application builds on them from `main` now. -- ✅ The epic no longer lists federation and the stream epic under - `Required` (rejected: keeping them). An epic's `Required` holds back every - child, so certificate credentials and the Endpoint waited on federation and - on JS streams they do not use. The relay node and the Endpoint each list - their own blocker. -- ✅ The relay node takes a credential handle and the application enrolls - however it likes (rejected: the coordinator issuing certificates; - deferring). `moq_auth::mesh`'s token enrollment stays one way to get a - handle. [Mesh enrollment can grow](/quest/m2/api-mesh-enrollment.md) - fences the reply before the cut publishes it. -- ✅ The contract changes land in m2, because the cut publishes - `moq_auth::mesh` for the first time: the - [grant shape](/quest/m2/mesh-grant-shape.md) and the - [key budget](/quest/m2/mesh-device-bounds.md). -- The JS enrollment client and the guide are parked in m5: the application - they were planned for enrolls with certificates and documents itself. - -## Related - -- [Outbound federation](/quest/m3/cluster-routing/federation.md) - the relay node requires it; the other children do not -- [Rust byte streams](/quest/m1/stream/rust.md) - the Endpoint requires it; the other children do not -- [P2P](/quest/m4/p2p/README.md) - internet peer sessions, which the m3 mesh does not use -- [Mesh over ICE](/quest/m5/mesh-ice.md) - deferred direct paths between mesh nodes on different networks -- [End-to-end encryption](/quest/m5/e2ee/README.md) - the AEAD the encryption level will reuse -- [Teleoperation](/quest/m5/teleop-robot.md) - robot track conventions that could ride the mesh diff --git a/quest/m3/mesh/addr.md b/quest/m3/mesh/addr.md deleted file mode 100644 index 23b82172d6d..00000000000 --- a/quest/m3/mesh/addr.md +++ /dev/null @@ -1,48 +0,0 @@ -# [M] Address hints - -## Goal - -Mesh nodes that can reach each other directly but cannot discover each other -over mDNS still connect directly: each node publishes its dialable addresses -(interface addresses, its listen port, its certificate fingerprint), signed by -its endpoint key, on a track under `mesh/node//`, and every node dials peers' -hinted addresses alongside `[cluster.lan]`. Two nodes on a network namespace -with multicast disabled (the stand-in for a Docker bridge or a cloud VPC) find -the direct path and prefer it over the relay. - -## Plan - -Decided 2026-10-31 (maintainer, moq.pro mesh planning interview): mDNS fails -in Docker bridge and overlay networks, cloud VPCs, across subnets and VLANs, -on isolated Wi-Fi, and for macOS launchd agents denied Local Network access. -Hints cover the cases where peers can reach each other. Generic: the -coordinator adds nothing, and self-hosted meshes get it. - -Decided 2026-10-31 (maintainer): a mesh node is a relay, and the rule in -`doc/bin/relay/cluster.md` ("never a URL learned from an announcement") does -not apply when the operator opts in to the mesh and everything is signed. -Dial a hinted address only when the mesh is enabled, the hint verifies against -the endpoint key named by its `mesh/node//` path, and the dial pins the hinted -fingerprint; admit the peer only after it proves its own enrollment, as LAN -peers do. Amend that sentence of the docs with the mesh exception. - -- Settle the track name and schema and record them here; keep it separate - from `.status` so a node can read hints without the status track's payload. -- Publish every non-loopback interface address, public ones included: a host - with a public address is directly dialable from anywhere, with no punching. - A private address that collides with an unrelated host on another network - fails the fingerprint pin, so it costs one failed dial, not a wrong peer. -- Bound and back off failed dials per peer, so a fleet of unreachable hints - does not become a dial storm; record the bound chosen. -- The [sparse overlay](/quest/m3/mesh/overlay.md) decides which reachable - peers to keep; until it lands, dial every hinted peer. - -Tests: namespaces with multicast off take the direct path; a forged hint -(signed by another key) is never dialed; a colliding private address fails the -pin and falls back to the relay; a peer that stops answering is backed off. -Update `doc/bin/relay/cluster.md` and the [guide](/quest/m5/mesh-guide.md) -if it exists. - -## Required - -- [Mesh relay node](/quest/m3/mesh/node.md) - the enrolled relay node that publishes and dials the hints diff --git a/quest/m3/mesh/cert-credentials.md b/quest/m3/mesh/cert-credentials.md deleted file mode 100644 index d2d85838372..00000000000 --- a/quest/m3/mesh/cert-credentials.md +++ /dev/null @@ -1,67 +0,0 @@ -# [M] Certificate credentials - -## Goal - -A client can present an X.509 certificate chain over its Ed25519 key in place -of a JWT: in the `jwt` query, as a moq-transport SETUP token, and on every -transport, browsers included. The credential is the chain plus a proof -signature over the relay's host name, the dialed path, and the current time, -so a captured chain alone admits nothing. `moq-auth` parses and verifies it -against a trust store the embedding application supplies and swaps at -runtime: its anchors, and a check for whether a certificate is revoked. A -relay or node using that store refuses a chain that fails it, and closes -sessions it admitted earlier once their certificate is revoked or the anchors -change, at its next re-check. The application turns a verified chain into a -grant, the way in-process admissions already decide. `moq_auth::mesh::Client` -can present such a credential, and `@moq/auth` builds one from a -non-extractable WebCrypto Ed25519 key. - -## Plan - -Decided 2026-11-07 (maintainer, Kixel's `/quest-plan`): a mesh whose devices -must admit and evict each other with no internet needs credentials that last -longer than a coordinator token and can be checked offline, so Kixel moved its -devices from coordinator-minted JWTs to 30-day certificates from the mesh's -own signers. Presenting the certificate as the token, rather than over mTLS, -gives one path that also crosses relays that accept only their own cluster's -client certificates, WebSocket fallbacks, and browsers, which cannot present -client certificates over WebTransport. The signer log, revocation set, and -their sync stay in Kixel; this quest is the generic format and hook. - -- Verify chains with webpki. Name constraints work only on dNSName (webpki - rejects URI constraints and lacks directoryName), so document scopes - carried as DNS names. -- The trust store is a handle the application updates, not a callback per - connection. Re-check live sessions on the relay's existing `--revalidate` - cadence. -- The time-window proof comes first. Its replay window (a relay that saw a - credential could reuse it at the same host within the skew) closes with the - [challenge proof](/quest/m4/cert-challenge.md). Settle the skew and the - signed encoding before coding, and record them in `doc/bin/relay/auth.md`. -- Browsers: WebCrypto Ed25519 keys sign in current Chrome, Firefox, and - Safari; confirm it before relying on it, and fail loud where it is missing. -- Overlaps [moq-dev/moq#8149](https://github.com/moq-dev/moq/pull/8149)'s - bounded revocation for `moq auth`'s own mesh coordinator, which keeps - JWTs; this quest does not change that coordinator's tokens. - -Tests: a valid chain admits; a revoked or unanchored one, a proof for another -host or path, and a stale proof are refused; a session admitted before its -certificate is revoked closes on the next re-check; swapping the anchors -applies without a restart; a JS client presents a credential from a WebCrypto -key and a Rust relay admits it. - -Public API: new credential types in `moq-auth` and `@moq/auth`, and a trust -store handle on the relay. Wire: a new credential form wherever a JWT is -accepted. - -Added 2026-11-07 (maintainer, /quest-plan from the API sweep of open PRs and remaining quests), before any code: pin the credential encoding, since it is permanent once shipped. - -- A self-describing, versioned encoding (for example a `moqc1.` prefix, or a distinct token type) in a slot named by role. If the query key stays `jwt` for compatibility, the prefix still makes it dispatchable without sniffing, and moq-transport uses its AUTHORIZATION TOKEN type field. -- One proof structure whose signed payload has a `nonce | timestamp` field, so [the challenge](/quest/m4/cert-challenge.md) reuses it instead of adding a second format. -- Revocation is data in the trust-store handle (a revoked SPKI-hash set, or a set of serials qualified by their issuer, which the application replaces), not a per-connection predicate. A bare serial is never enough, since two trusted issuers can reuse one; a test revokes one issuer's serial and still admits the other issuer's certificate with the same serial. -- Chains are 2 to 4 KB in base64, close to common 8 KB URL limits; cache verified chains by fingerprint, and say in the PR how a long chain is carried. -- The shared token corpus planned in [moq-dev/moq#8151](https://github.com/moq-dev/moq/pull/8151) includes this credential form. - -## Related - -- [Mesh relay node](/quest/m3/mesh/node.md) - checks these credentials for its LAN peers, and only uses them, so this lands first diff --git a/quest/m3/mesh/endpoint.md b/quest/m3/mesh/endpoint.md deleted file mode 100644 index 1d9e719c8b0..00000000000 --- a/quest/m3/mesh/endpoint.md +++ /dev/null @@ -1,64 +0,0 @@ -# [M] Endpoint API - -## Goal - -A Rust application builds an `Endpoint` from its secret key and a mesh -enrollment, then `connect(endpoint_id, service)` opens a byte stream to that -endpoint's named service and `accept()` yields incoming ones with the remote -endpoint ID verified. The surface mirrors iroh's (`Endpoint`, `id()`, -`connect`, `accept`, bi streams, `remote_id()`) so an iroh application ports -with small changes, without iroh's wire or discovery. - -## Plan - -Decided 2026-10-25 (maintainer): iroh-shaped, not drop-in compatible; tickets, -`EndpointAddr`, and `Router` are out unless porting proves they are needed. - -- A service is the iroh ALPN analog: a socket endpoint per service name - (below), while each node's address and status tracks stay under its - `mesh/node//`. -- Decided 2026-11-01 (maintainer, socket planning interview), and updated - 2026-11-03 when sockets became endpoints and 2026-11-04 when the segment - became `.socket` with the protocol in the session name: an `Endpoint` wraps - a [socket](/doc/concept/socket.md) endpoint, which since 2026-11-06 is a - server at `.server/`, a client at `.client/`, or both. - `connect` dials with the caller's own endpoint ID, and `accept` reads the - remote ID from the dial, which the socket's answer check proves and the - grant pins (publish `.server/` and `.client/` only). -- Streams are [byte streams](/quest/m1/stream/rust.md) at the protection - level the service declares: identity reads `moq-net` directly, and - authentication, the mesh default, reads through `moq_crypto::sign` against the - `PublicKey` parsed from the peer's endpoint id, which names the endpoint - broadcast it subscribes to. -- The secret key comes from `moq_crypto::sign::SecretKey::load_or_generate`. -- A mesh daemon (Kixel's, outside this repository, decided 2026-11-02) is - built on this API, so a process either embeds an `Endpoint` or talks to a - local [mesh relay node](/quest/m3/mesh/node.md). -- Write a short porting note into `doc/` beside the Endpoint's own page; the - mesh [guide](/quest/m5/mesh-guide.md) is parked. - -Decided 2026-11-07 (maintainer, /quest-plan from the API sweep of open PRs and remaining quests): one socket endpoint per service, with a grant to publish on it under the member's own id, so `moq_net::socket` ships unchanged and needs no service field. - -✅ Decided 2026-11-07 (maintainer, /quest-complete): scope both -namespaces. Members own `mesh/node//**` (was `mesh//**`) and each -service lives at `mesh/service/` (was `mesh/`), so the -grant adds publish `mesh/service/*/.server/` and -`mesh/service/*/.client/` (rejected: `mesh/.service/`; -keeping `mesh/` and refusing service names shaped like an endpoint -ID). Review of [moq-dev/moq#8214](https://github.com/moq-dev/moq/pull/8214) -found that each member publishing `mesh//**` lets a service named after -member A put `.server/B` under A's own grant, so A could answer as B; the -`node` and `service` segments keep the two apart. A service name is still -one path segment, validated when the `Endpoint` is built, so `*` in the -grant covers it (`video/h264` would fall outside). - -The grant change itself is its own quest, in m2, so it lands before the cut -publishes `moq_auth::mesh` (split 2026-11-12, maintainer, /quest-audit): -[mesh grants scope nodes and services apart](/quest/m2/mesh-grant-shape.md). -This quest builds the `Endpoint` on the new grant, and the dual-grant rollout -question it carried is gone, since no release ever issued the old shape. - -## Required - -- [Rust byte streams](/quest/m1/stream/rust.md) - the stream this API returns -- [Mesh grant shape](/quest/m2/mesh-grant-shape.md) - the grant that lets a member publish its service endpoints diff --git a/quest/m3/mesh/node.md b/quest/m3/mesh/node.md deleted file mode 100644 index 3247c82548b..00000000000 --- a/quest/m3/mesh/node.md +++ /dev/null @@ -1,51 +0,0 @@ -# [M] Mesh relay node - -## Goal - -A relay node (the `moq-relay` library, and its binary through a `[mesh]` -config section) joins a mesh: it enrolls with an enrollment key through -`moq_auth::mesh::Client`, signing with its endpoint key, renews before the -token expires, dials the mesh's relays as `upstream` links, and admits a -`[cluster.lan]` peer only after the peer proves its own enrollment. A restart -keeps the endpoint key and enrollment, so a node never re-enrolls on its -own. A mesh daemon outside this repository (Kixel's) embeds this rather than -reimplementing it. - -## Plan - -Decided 2026-11-02 (maintainer, Kixel planning): moq keeps the libraries and -Kixel the binary. This is the library half of the deleted daemon quest; its -CLI verbs, service install, and UI moved to Kixel. Carried over from it: - -- Mesh relays are `upstream` links; LAN peers come from `[cluster.lan]` mDNS - and rank above them, with no extra shared secret to configure. -- Enroll and renew with `moq_auth::mesh::Client`; the contract is in - `doc/bin/relay/auth.md#mesh-enrollment`. Pass the hostname as the default - name. -- Nothing is served unless the embedding application names it. -- A LAN peer proves its enrollment with the coordinator token, or with a - [certificate credential](/quest/m3/mesh/cert-credentials.md) when the - embedding application supplies a trust store (decided 2026-11-07, - maintainer). - -Decided 2026-11-12 (maintainer, /quest-audit before the cut): ✅ the node -takes a credential handle, and the embedding application decides how it -enrolls (rejected: `moq_auth::mesh`'s coordinator and `Enrollment` learning -to issue certificates, which puts one consumer's contract in a public crate; -deferring). The handle yields what the node presents to the mesh's relays -and to LAN peers and says when to renew; `moq_auth::mesh::Client`'s token -enrollment is the implementation this repository ships and the `[mesh]` -config section uses, and a -[certificate credential](/quest/m3/mesh/cert-credentials.md) is another. The -Goal's "enrolls with an enrollment key" describes that default, not the only -path. Keep the handle a small trait or enum in moq-relay's mesh module, private -to the crate until an embedder outside this repository needs to implement it. - -Tests in network namespaces: two relay nodes enroll through `moq auth`, take -the LAN link when they share a segment and the relay path otherwise, refuse -a LAN peer without an enrollment, and survive a relay restart. Update -`doc/bin/relay/cluster.md` inline. - -## Required - -- [Outbound federation](/quest/m3/cluster-routing/federation.md) - upstream links and LAN preference diff --git a/quest/m3/mesh/overlay.md b/quest/m3/mesh/overlay.md deleted file mode 100644 index 02f41c9ed23..00000000000 --- a/quest/m3/mesh/overlay.md +++ /dev/null @@ -1,106 +0,0 @@ -# [L] Sparse overlay - -## Goal - -A mesh relay node keeps direct links to about log N of its directly reachable peers -instead of all of them, chosen deterministically so every node computes the -same overlay from the member list, and forwards for the rest. Any two nodes in -a reachable group reach each other through the overlay in about log N hops, -a departed node's links heal, an optional per-node degree cap bounds a -constrained robot's links, and traffic crosses the relay uplink, which every -relay node keeps, when the overlay partitions. Every node is a core unless -its application marks it an edge: an edge links to the cores it can reach -and its links cost more, so traffic between edges crosses a core while one -is reachable, and edges ring among themselves when none is. An application -embedding `Endpoint` reaches the overlay through a relay node. - -## Plan - -Decided 2026-10-31 (maintainer, moq.pro mesh planning interview): -deterministic and local, not coordinator-assigned, so self-hosted meshes get -it; the coordinator is only the member list (announcements under `mesh/**`) -and the fallback path. - -Decided 2026-11-09 (maintainer, /quest-plan from Kixel's audit of its fleet -plan): ✅ the ring stays the link choice, with no measured cost and no exit -designation (rejected: links chosen by measured proximity; a coordinator -computing the graph), on the working assumption that a peer is either on the -same network or out of reach. Two cases the ring as written does not cover -join this quest: - -- Partial reach. On radios A reaches B and B reaches C while A cannot reach - C, so each node computes its ring over a different set of peers. Test it: - in namespaces with filtered links (a line and a grid), every pair with a - path connects without crossing the relay, a flapping link reroutes without - a loop, and the links and hops are reported. A node with no uplink has no - fallback, so report whether the overlay stays connected whenever the reach - graph is, and record what such a node can know about being cut off. -- ~~Nodes that do not forward.~~ Superseded 2026-11-10 by the roles below: - Kixel dropped its `--leaf` flag, and every node forwards. - -Decided 2026-11-10 (maintainer, /quest-plan on lite-07 routing, asked how a -beefier node should attract traffic once Kixel's non-forwarding leaf is -gone): ✅ **two roles, core and edge, and both link shaping and link -pricing.** In the maintainer's words: "Kinda like how moq.pro wants to have -edges and cores. Edges should prioritize connecting to cores, but if there's -no cores, do a ring of edges? I just want this to work by default, which I -guess means everything is a core unless stated otherwise." Rejected: pricing -alone; shaping alone; ranking links to cheaper nodes by priority, whose -outcome under the feasibility rule depends on which route arrived first. -That reason was written under Babel. Under the path rule -([routes epic](/quest/m2/cluster-routes/README.md) Plan item 27) a -relay takes the cheapest route whose path does not name it, whatever arrived -first. - -- Every node is a core unless the embedding application says it is an edge. - Every node forwards; the role only changes which links it keeps and what - they cost. -- Shaping: cores ring among themselves as below. An edge links to the cores - it can reach, and rings with other edges only when it reaches none, so the - mesh works with no core at all. Settle in the PR what an edge that reaches - a core does for an edge that reaches none (partial reach); recommended: it - accepts that edge's link. -- Pricing: a node declares a cost, 1 for a core and more for an edge, and a - link costs the product of its two ends' costs. A cost added per hop cannot - move traffic onto a core: a direct link is always cheaper than the same - link plus another. With a core at 1 and an edge at 3, edge to edge costs 9 - and edge to core to edge costs 6, whichever route arrived first. This is - local pricing of each peer's declared cost - ([link costs](/doc/bin/relay/cluster.md#link-costs)) and needs no wire - change; confirm that, and the edge's default cost, in the PR. -- Recommended, to confirm in the PR: an edge marks its links to cores - `upstream`, as moq.pro's edges do, so it never carries one core's routes to - another. - -- Ring: order the reachable peers (found through mDNS or - [address hints](/quest/m3/mesh/addr.md)) by a hash of their endpoint ID, and - link to the first peer at or after distance 2^k for each k. Accept incoming - links from peers whose ring picks this node. A target that cannot be dialed - is replaced by the next peer on the ring. -- Degree cap: an optional per-node setting for the maximum direct links, - unset by default; the ring alone decides when it is unset. Over the cap, - drop the longest fingers first. Small meshes come out close to full mesh on - their own. -- Forwarding and route choice are [cluster routing](/quest/m3/cluster-routing/README.md)'s: - overlay links join relay nodes as LAN-cost links, and relay - uplinks stay `upstream` links costed above them. Nothing routes outside the - shared priority/cost rules. -- Membership churn recomputes the ring; hold a change briefly so a flapping - node does not churn every link, and record the hold chosen. - -Tests in network namespaces, on one segment beside the partial-reach layouts -above: each node's direct link count -follows the ring (and never exceeds the cap when set); every pair connects -without crossing the relay; killing a node heals the overlay; splitting the -segment sends cross-half traffic through the relay and healing restores the -direct path. Report the link counts and hop counts the test measures. -Update `doc/bin/relay/cluster.md` and the [guide](/quest/m5/mesh-guide.md) if it exists. - -## Required - -- [Mesh relay node](/quest/m3/mesh/node.md) - the enrolled relay node that dials the overlay - -## Related - -- [Address hints](/quest/m3/mesh/addr.md) - a second source of reachable peers -- [Link quality](/quest/m5/link-quality.md) - measured link cost a fading overlay link could use diff --git a/quest/m3/mesh/status.md b/quest/m3/mesh/status.md deleted file mode 100644 index e0db44aff97..00000000000 --- a/quest/m3/mesh/status.md +++ /dev/null @@ -1,35 +0,0 @@ -# [M] Status track - -## Goal - -Each mesh node, a relay node with the mesh enabled or an application -embedding `Endpoint`, publishes `mesh/node//.status`, signed: its peers, its -links (each with its smoothed and minimum RTT, and for a relay uplink the -relay's identity), its routes, and its services. A subscriber can draw the -whole mesh from these tracks alone. - -## Plan - -Decided 2026-10-25 (maintainer): the data is a track on the node's -member subtree (`mesh/node/` since 2026-11-07), not an -HTTP API. - -Decided 2026-10-31 (maintainer, moq.pro mesh planning interview): per-link -RTTs, and embedded apps publish `.status` so they appear as nodes. RTT comes -from the session's existing connection stats; coalesce updates to a few a -minute. - -Decided 2026-11-02 (maintainer, Kixel planning): the graph component, the -localhost page, and the hosted dashboard moved to Kixel, which draws them -from this track. Keep the schema generic and documented, with no extension -point for a hosting service's own nodes. Settle the schema and record it in -`doc/` with this quest. - -Tests: two nodes in network namespaces publish `.status` naming each other's -link with an RTT, a relay uplink names its relay, and a frame signed by -another key is rejected by a subscriber. - -## Required - -- [Endpoint API](/quest/m3/mesh/endpoint.md) - the application node whose links the track describes -- [Mesh relay node](/quest/m3/mesh/node.md) - the relay node whose links the track describes diff --git a/quest/m4/cert-challenge.md b/quest/m4/cert-challenge.md deleted file mode 100644 index 99bdf521aa4..00000000000 --- a/quest/m4/cert-challenge.md +++ /dev/null @@ -1,28 +0,0 @@ -# [S] Challenge proof for certificate credentials - -## Goal - -A relay can ask a client presenting a [certificate credential](/quest/m3/mesh/cert-credentials.md) -to sign a fresh nonce instead of a timestamp, so a credential replayed within -the time window admits nothing. Clients that support it answer the -challenge; a relay configured to require it refuses time-window proofs. - -## Plan - -Decided 2026-11-07 (maintainer, Kixel's `/quest-plan`: "plan the challenge, -but do the simple thing first"): certificate credentials ship with a -time-window proof, and this closes its replay window later. Settle where the -nonce rides (SETUP parameters on moq-transport and moq-lite, and what a -WebSocket or HTTP admission does) and what one extra round trip costs a -connect, measured, before coding. - -Tests: a replayed time-window proof is refused when the relay requires the -challenge; a client signs the nonce and is admitted; a mismatched nonce is -refused. - -Public API: a challenge option on the relay and the clients. Wire: a nonce -exchanged during setup. - -## Required - -- [Certificate credentials](/quest/m3/mesh/cert-credentials.md) - the credential this hardens diff --git a/quest/m4/crypto-key-load.md b/quest/m4/crypto-key-load.md deleted file mode 100644 index 7237610d6f9..00000000000 --- a/quest/m4/crypto-key-load.md +++ /dev/null @@ -1,20 +0,0 @@ -# [S] @moq/crypto loads a stored signing key - -## Goal - -`@moq/crypto`'s `SecretKey` loads a stored Ed25519 key (raw bytes, and a -non-extractable WebCrypto `CryptoKey` for browsers) and exposes `sign` -publicly, mirroring Rust's `moq_crypto::sign::SecretKey` -(`load_or_generate`, `sign`), so one device key both proves mesh identity -and signs broadcasts. - -## Plan - -Planned 2026-11-03 (maintainer) from Kixel's `@kixel/mesh` -([moq-dev/kixel.ai#16](https://github.com/moq-dev/kixel.ai/pull/16)), which -had to carry its own Ed25519 and z-base-32 code because `SecretKey` cannot -load a stored key and its `sign` is internal. Keep generated keys working, -keep the browser path non-extractable, and test that a key loaded from bytes -signs what Rust verifies. A raw 32-byte seed already loads through -`SecretKey.fromSeed` (`js/crypto/src/sign/key.ts`), but it and `sign` are -marked `@internal`, and no `CryptoKey` path exists. Moved to m4 2026-11-12 (maintainer, /quest-audit): the downstream application it was ranked for made moq-crypto opt-in and plans nothing on it. diff --git a/quest/m5/link-quality.md b/quest/m5/link-quality.md deleted file mode 100644 index 8bc39e2b0f5..00000000000 --- a/quest/m5/link-quality.md +++ /dev/null @@ -1,38 +0,0 @@ -# [M] Link quality - -## Goal - -A link can take its cost from its measured quality instead of configuration, -so a drone mesh steers around a fading radio link without manual tuning, -while a small wobble never flips a route. Static configured cost stays the -default and is what the CDN runs; measured cost is opt-in per link. This is -local policy feeding the one metric on the wire, with no wire change. - -## Plan - -Decided 2026-10-01: the wire carries one additive metric and each node -computes its own link costs. - -Decided 2026-10-08: m2, and Related rather than Required for the -[cluster routing line](/quest/m3/cluster-routing/README.md): static costs are -the default and measured cost is opt-in, so the line ships without it. - -- Derive the cost from what QUIC already measures (smoothed RTT, loss); an - ETX-style estimate (Babel RFC 8966 Appendix A, B.A.T.M.A.N.'s TQ) is the - reference. Pick the formula from the harness, not from intuition. -- Hysteresis: only advertise a cost change past a threshold or after it - holds, so the route layer sees a few changes per real shift. Report how - many ROUTE changes a fading link causes. -- Never mix it into business pricing: an operator's priced topology keeps - RTT out of monetary costs, and that stays true because measured cost is - opt-in on links the operator chooses. -- A lossy-link fixture over the mock transport with mocked time. - -Detecting stalled epoch-bound track delivery is out of scope. - -Public API: a per-link way to ask for measured cost in the peer entry. -Wire: none. - -## Required - -- [Routes and announces](/quest/m2/cluster-routes/README.md) - a cost change costs one ROUTE per origin there, instead of a re-announce per broadcast diff --git a/quest/m5/mesh-guide.md b/quest/m5/mesh-guide.md deleted file mode 100644 index e83d9898445..00000000000 --- a/quest/m5/mesh-guide.md +++ /dev/null @@ -1,21 +0,0 @@ -# [S] Mesh library guide - -## Goal - -doc.moq.dev has a mesh page for developers building on the libraries: -enrolling a node against `moq auth`, the enrollment contract a hosted -coordinator implements, `Endpoint` services and byte streams, LAN discovery -and address hints, the sparse overlay, protection levels, the `.status` -schema, and porting from iroh. It links Kixel as one binary built on them. - -## Plan - -Decided 2026-10-25 (maintainer): a new page, because no feature quest can -carry the whole setup. Each feature quest still updates the docs it makes -stale. Decided 2026-11-02 (maintainer, Kixel planning): installing and -running a mesh daemon is Kixel's documentation, so this page covers only the -libraries. - -## Required - -- [Endpoint API](/quest/m3/mesh/endpoint.md) - the API the page centers on diff --git a/quest/m5/mesh-ice.md b/quest/m5/mesh-ice.md deleted file mode 100644 index 4d219957b3c..00000000000 --- a/quest/m5/mesh-ice.md +++ /dev/null @@ -1,22 +0,0 @@ -# [L] Mesh over ICE - -## Goal - -Mesh nodes on different networks that cannot dial each other's addresses -connect directly through the [P2P](/quest/m4/p2p/README.md) line's native -QUIC-over-ICE pairs, with the relay as rendezvous and fallback, and the -direct link joins the [sparse overlay](/quest/m3/mesh/overlay.md) as an -ordinary link. - -## Plan - -Parked 2026-10-31 (maintainer, moq.pro mesh planning interview): remote mesh -traffic is meant to cross relays, so hole punching waits in m5 until a mesh -user needs it. Reuse the P2P line's ICE, signaling under the mesh's own -prefix, and operator-supplied ICE servers; no TURN, since the relay already -carries the fallback route. - -## Required - -- [Native pairs over ICE and QUIC](/quest/m4/p2p/native-quic.md) - the transport this reuses -- [Sparse overlay](/quest/m3/mesh/overlay.md) - the overlay the direct link joins diff --git a/quest/m5/mesh-js-enroll.md b/quest/m5/mesh-js-enroll.md deleted file mode 100644 index c6a8bb7614a..00000000000 --- a/quest/m5/mesh-js-enroll.md +++ /dev/null @@ -1,26 +0,0 @@ -# [S] JS mesh enrollment client - -## Goal - -`@moq/auth` exports a JS mesh enrollment client matching -`moq_auth::mesh::Client` (`rs/moq-auth/src/mesh/client.rs`): challenge, -enroll with an enrollment key, and refresh by proving the endpoint key, -interoperating with `moq auth serve --mesh-url` and any coordinator that -implements the same contract. - -## Plan - -Planned 2026-11-03 (maintainer) from a downstream application's JS mesh -client, which implemented this contract then. Port its hardening where it is -generic: https except on loopback, no redirects, bounded time and reply -size, and every reply field checked. Test against `moq auth serve` the way -`rs/moq-auth/src/mesh/contract_tests.rs` does. - -Moved to m5 2026-11-12 (maintainer, /quest-audit): that client has since -moved to a certificate enrollment reply, which `moq_auth::mesh::Client`'s -token-only `Enrollment` (`rs/moq-auth/src/mesh/mod.rs`) does not carry, so it -no longer wraps this contract and nothing else asks for a JS client. - -## Required - -- [`@moq/crypto` loads a stored signing key](/quest/m4/crypto-key-load.md) - the endpoint key this proves diff --git a/quest/m5/mesh-keys.md b/quest/m5/mesh-keys.md deleted file mode 100644 index a24ce8ca614..00000000000 --- a/quest/m5/mesh-keys.md +++ /dev/null @@ -1,23 +0,0 @@ -# [L] Mesh keys - -## Goal - -A mesh broadcast can be encrypted for its authorized subscribers, one-to-one -or fanout, and a mesh can authenticate frames with symmetric keys instead of -per-frame signatures. A publisher distributes per-broadcast keys on a -cacheable `.keys` track, wrapped to each subscriber's endpoint key, and rotates -them when a subscriber is removed. - -## Plan - -Parked 2026-10-25 (maintainer): encryption is out of the first mesh; identity -and authentication ship first. Research recorded in the planning PR: SFrame -and IETF secure objects leave key distribution open and give no sender -authenticity; MLS per broadcast is heavy for churning viewers; a sender-keys -style wrapped key track fans out like media. The AEAD stays moq-crypto's e2ee. -Revisit when a mesh user needs confidentiality from relays. - -## Related - -- [Mesh](/quest/m3/mesh/README.md) - the protection levels this completes -- [End-to-end encryption](/quest/m5/e2ee/README.md) - the AEAD and its key contract diff --git a/quest/m5/teleop-arbitration.md b/quest/m5/teleop-arbitration.md deleted file mode 100644 index 4fd076cd8b1..00000000000 --- a/quest/m5/teleop-arbitration.md +++ /dev/null @@ -1,45 +0,0 @@ -# [M] Operator arbitration - -## Goal - -Exactly one controller commands a vehicle at a time, with explicit handoff, and -the boundary between arbitration and authorization is stated rather than -assumed. - -## Plan - -Decided 2026-10-08: parked in m5 with [robot](/quest/m5/teleop-robot.md), -which it builds on. - -`moq-boy` merges every viewer's input, which is right for crowd control and -wrong for a vehicle. Arbitration is the part of a teleoperation stack that -cannot be lifted from it: one controller is active, the others are read-only, -and handoff is explicit rather than last-writer-wins. - -### Arbitration is not authorization - -The robot subscribes an announce prefix that is a sibling of what it publishes, -so on its own nothing stops anyone able to announce under `control//` from -joining the pool. `moq-boy`'s fan-in is safe because a hostile viewer can only -press B; a vehicle is not that. - -Admission belongs to the relay as a path-scoped grant, not to this crate. -Scoped signing keys landed in moq#2416 (`6b86e612b`), so the enforcing -mechanism already exists: `Scope` in `rs/moq-auth/src/claims.rs` and -`Key::with_scope` in `key.rs` bind a key to publish and subscribe path -prefixes. Write the concrete claims down (a robot publishes `robot/` and -subscribes `control//`; an operator publishes `control//` and -subscribes `robot/`) and prove with a relay test that a key scoped -elsewhere cannot announce under `control//`. Documenting the requirement is -not an acceptable outcome now that enforcement ships. Revoking a superseded or -compromised operator's grant mid-session is relay auth work, not something -this crate implements; scoped keys only gate admission. - -Link loss is the other half: a class whose contract is "drop stale samples" -needs a stated behavior when samples stop arriving, and the vehicle's own -failsafe is what actually holds. Say what the crate reports and what it does -not attempt. - -## Required - -- [Robot teleoperation conventions](/quest/m5/teleop-robot.md) diff --git a/quest/m5/teleop-mavlink.md b/quest/m5/teleop-mavlink.md deleted file mode 100644 index 6f653a91bb3..00000000000 --- a/quest/m5/teleop-mavlink.md +++ /dev/null @@ -1,93 +0,0 @@ -# [L] MAVLink bridge - -## Goal - -A `moq-mavlink` gateway replaces the VPN plus two unmanaged UDP flows that -every cellular ArduPilot vehicle runs today. QGroundControl, Mission Planner -and MAVProxy connect unchanged. - -## Plan - -Parked in m5 until a real ArduPilot user or partner asks (2026-09-30 audit). - -Decided 2026-10-08: the lossy class is a snapshot track per key with a -bounded key set, since one snapshot track carries only one latest value. - -### Shape - -A library crate reached through `moq-cli`, the way `moq-srt` and `moq-rtmp` -already are (neither is a binary; both are crates behind subcommands). - -Each side is bidirectional on one session. The vehicle side publishes what the -flight controller emits and subscribes the operator's commands to write back -into it; the operator side publishes commands and subscribes telemetry to serve -UDP 14550, because QGC speaks UDP and TCP only and its forwarding is one-way. -Neither side is a one-way media import or export, so do not let the subcommand -naming imply one leg. - -### Route by msgid, never parse payloads - -Read the header only: the 24-bit msgid in v2, 8-bit in v1. Payloads stay opaque -bytes and route to a delivery class through a config table, with an unknown -msgid falling to a default. Every track declares `Timescale::MILLI` -explicitly, since an undeclared Rust timescale means untimed. - -This is what makes frame-aware routing free of the dialect tax. CRC_EXTRA (the -per-message schema hash) is only needed to validate a frame, never to read its -header, so custom dialects, new messages and version skew all just work. The -alternative, compiling in `ardupilotmega` the way `mavlink-camera-manager` and -`mavlink-server` do, buys per-field logic we do not need and couples us to a -schema that changes without us. - -### One lossy track for every msgid would starve the slow ones - -Latest-wins is per track. A single lossy track carrying every routed msgid as -its own group lets 50 Hz `ATTITUDE` supersede a 1 Hz `HEARTBEAT` or a GPS fix, -because the newest group wins regardless of which message it holds. Putting -them all in one long-lived group has the opposite failure: nothing can be -skipped and head-of-line blocking is back. - -So the lossy class is a plain track carrying the latest opaque frame, with -one self-contained group per update, so a newer value -never waits behind an older one. One detail decides whether it actually -delivers latest-value: - -- **Key on `(sysid, compid, msgid)`, not msgid alone.** A vehicle is several - components, and an autopilot, a gimbal and a camera all emit `HEARTBEAT` - under msgid 0; keying on msgid alone lets them overwrite each other. Decide - explicitly what to do about instances distinguished only inside a payload, - which the msgid-only rule cannot see. -- **One latest-value track per key.** Per-key latest-value means a track per - `(sysid, compid, msgid)`, named from the key and advertised in the catalog - as it appears. That state grows with whatever the flight-controller link - emits, so bound it: cap the number of keys (or derive the set from the - config table) and fail loud past the cap rather than creating tracks - without limit. - -Not `moq_json::snapshot`: a MAVLink frame is not JSON, and its default batches -up to `MAX_DELTA_FRAMES` merge patches into one ordered group, so an earlier -50 Hz attitude delta would head-of-line block a later heartbeat. - -The reliable class is one bounded ordered group on a plain track: commands, -ACKs, mission, parameter and file transfer are stop-and-wait exchanges carried -in one ordered group. Note the scope in -[robot](/quest/m5/teleop-robot.md): that is gap-free for a live reader, not -recoverable after a lag or a reconnect, so these MAVLink services keep relying -on their own retransmission across a link drop. - -The [compression migration](https://github.com/moq-dev/moq/pull/7046) deleted the -moq-flate snapshot and stream helpers. Keep these framing rules on plain -tracks, setting `compression` when useful, rather than depending on those helpers. - -### Deliberately out of scope - -Advertising the stream in-band. No `VIDEO_STREAM_TYPE` value can describe a -MoQ endpoint, so a GCS discovers the link out of band. Check the current -enumeration when this is picked up: it grew a WHEP entry after the original -four, which is the precedent that adding one upstream in `common.xml` is a -small, real contribution and its own future quest. - -## Required - -- [Robot teleoperation conventions](/quest/m5/teleop-robot.md) -- [Operator arbitration](/quest/m5/teleop-arbitration.md) diff --git a/quest/m5/teleop-proof.md b/quest/m5/teleop-proof.md deleted file mode 100644 index 4bdb3d6e483..00000000000 --- a/quest/m5/teleop-proof.md +++ /dev/null @@ -1,40 +0,0 @@ -# [M] SITL proof and browser ground station - -## Goal - -ArduPilot SITL plus a synthetic camera, flown from a browser ground station -through a real relay, with no hardware. Someone with a laptop reproduces it in -five minutes. - -## Plan - -Parked in m5 until a real ArduPilot user or partner asks (2026-09-30 audit). - -The browser client subscribes video and telemetry and publishes control, the -shape Blue Robotics' Cockpit proves is viable. It is the demo and the -end-to-end proof of the primitive, not a bid to out-feature QGroundControl: -real users reach the same link through the `moq-mavlink` gateway. - -SITL is the right proof surface precisely because it removes the camera, which -otherwise dominates the latency budget and would make the demo a measurement of -somebody's webcam. - -The ground station's catalog and delivery-class shapes come from a zod -schema mirroring the documented robot conventions, written once and extending the root -schema through the same seam `js/hang/src/catalog/root.ts` uses (there is no -`@moq/mux`). Decided 2026-10-08: no separate `@moq/robot` package until a -second browser consumer needs it; the schema folds in here. A browser -observer cannot degrade the operator's classes (the publisher's -`Info::max_age` bounds the window), so the schema is for reuse, not safety. - -Standing SITL up in CI is separate work with its own build dependencies. -Reproducibility by hand is the bar here; automate it later if it proves worth -the maintenance. - -## Required - -- [MAVLink bridge](/quest/m5/teleop-mavlink.md) - the gateway the vehicle side runs - -## Related - -- [Robot teleoperation conventions](/quest/m5/teleop-robot.md) - the conventions the zod schema mirrors diff --git a/quest/m5/teleop-robot.md b/quest/m5/teleop-robot.md deleted file mode 100644 index e813c769696..00000000000 --- a/quest/m5/teleop-robot.md +++ /dev/null @@ -1,104 +0,0 @@ -# [S] Robot teleoperation conventions - -## Goal - -The track layout, discovery, and timing every teleoperated machine needs are -documented conventions in `doc/`, beside the delivery shapes the -[teleoperation guide](/doc/concept/use-case/teleoperation.md) already -documents, so an integrator writes an adapter rather than a stack. There is no -`moq-robot` crate, and this quest ships documentation only; -[the MAVLink gateway](/quest/m5/teleop-mavlink.md) implements the conventions -in its own quest. It is the gate for the rest of the questline. - -## Plan - -Decided 2026-11-07 (maintainer, /quest-plan from the API sweep of open PRs and remaining quests): ✅ no `moq-robot` crate (rejected: a generic robot -crate, too niche for a published library with no second consumer). The -shapes below become documented conventions, and `moq-mavlink` ships as a -moq-cli gateway like moq-srt. If a second consumer appears, generalize -an announce-prefix fan-in helper into moq-net instead of adding a robot -crate. - -Decided 2026-10-08: parked in m5. No teleoperation consumer needs it yet; -telemetry beside video needs [data sync](/quest/m5/watch-data-sync.md), -not this primitive. - -Narrowed 2026-11-12 (/quest-audit): the teleoperation guide -(`doc/concept/use-case/teleoperation.md`, with `rs/moq-json/examples/teleop.rs`) -already documents the two delivery shapes (moq-json snapshot for replaceable -state, an ordered moq-json stream for commands), the group cache bound and -`Evicted`, a new command track per control session, command IDs and -acknowledgments, and that a connected subscriber is not an authorized -operator. What remains extends that guide rather than restating it. - -### Direction - -The operator publishes and the robot subscribes. The robot serves its own -broadcast and subscribes an announce prefix for controllers; observers get the -control stream for free, and nothing needs relay support that does not already -exist. - -| Broadcast | Tracks | -|---|---| -| `robot/` | `video`, `telemetry` (lossy), `rpc` (reliable) | -| `control//` | `command` (lossy), `rpc` (reliable) | - -### Contents - -- The broadcast and track layout above, and how a robot learns its operators - by announce prefix, documented against moq-net's origin announce stream - (`origin::Consumer`'s announcements under a prefix). moq-boy no longer reads - viewer broadcasts (its viewers dial a socket endpoint), so there is no - fan-in code to move; a helper lands in moq-net only when a second consumer - needs one. -- The catalog entries. The hang catalog already advertises data tracks in its - `json` and `binary` sections (`rs/hang/src/catalog/{json,binary}.rs`), and - each entry carries `extra` fields. Decide whether the robot's fields ride - there or in a namespaced root section through `moq-mux`'s catalog - extension. No hang schema change. -- Opaque bytes: plain tracks with one group per latest value for the lossy - class, or one bounded ordered group for the append-log, setting the track's - `compression` flag when useful. The lossy class is lossy on the wire because - the publisher's `Info::max_age` evicts an aged-out group with `Error::Old` - (`rs/moq-net/src/model/track.rs`), resetting its stream. -- Per-stage timestamp instrumentation, generalised from moq-boy's `status` - track (`rs/moq-boy/src/status.rs`). Check it against the publisher-reported - stats track ([media stats](/doc/concept/stats.md#media-reports)) before - adding a second stats surface. Capability only: publishing a competitive - benchmark is out of scope, since camera and USB dominate glass-to-glass - latency. - -### Cross-track correlation - -Decided in the 2026-09-30 audit: correlation folds in here rather than being -its own quest. A command, the telemetry sample it produced, and the video frame -showing the result share one timebase, so a recording is usable as training -data and an operator sees what the machine actually saw. - -The bridge is each broadcast's fixed catalog `media.clock`, documented in -`doc/concept/hang.md`. Media and command tracks keep their own timescales; -convert PTS explicitly into the broadcast clock before joining samples. The -robot's video and telemetry share a clock; the operator's command broadcast -supplies its own mapping. Command and telemetry tracks declare -`Timescale::MILLI` explicitly: an undeclared Rust timescale means untimed, so a -track that leaves it out carries no timestamps to join. - -The clock assumption: the two hosts' wall clocks are synchronized by the -deployment, not by the library. State this in the docs, since a join across -unsynchronized hosts looks valid while being wrong. Report whether a mapping is -present, but never infer synchronization from its presence. No per-record -anchors or clock-sync mechanism; see -[#2278](https://github.com/moq-dev/moq/issues/2278). - -### Check against moq-boy - -Check the conventions against `moq-boy`, the only existing consumer, as the -no-arbitration case: it merges viewers' JSON button states, where the gateway -routes MAVLink frames, so if the documented shapes cannot describe its crowd -control they are the wrong shapes. moq-boy changes only where it contradicts a -convention. - -## Related - -- [Unbounded groups](/quest/m3/unbounded-groups/README.md) - changes the group cache bound the reliable class's one log relies on -- [arbitration](/quest/m5/teleop-arbitration.md) - which controller is obeyed diff --git a/rs/moq-json/Cargo.toml b/rs/moq-json/Cargo.toml index 253f71ccea0..f47346a10d4 100644 --- a/rs/moq-json/Cargo.toml +++ b/rs/moq-json/Cargo.toml @@ -60,10 +60,5 @@ harness = false name = "window" harness = false -# Run the documented example's inline test in the normal nextest gate. -[[example]] -name = "teleop" -test = true - [lints] workspace = true diff --git a/rs/moq-json/examples/teleop.rs b/rs/moq-json/examples/teleop.rs deleted file mode 100644 index a04b86145c2..00000000000 --- a/rs/moq-json/examples/teleop.rs +++ /dev/null @@ -1,79 +0,0 @@ -//! Carry replaceable control state and ordered commands on separate MoQ tracks. -//! -//! Run `cargo run -p moq-json --example teleop`. This in-process example teaches the -//! framing; attach the same broadcast to a moq-tokio origin to carry it over a session. -use std::task::Poll; - -use moq_json::{snapshot, stream}; -use moq_net::{Timed, Timescale, Timestamp, broadcast, track}; -use serde::{Deserialize, Serialize}; - -#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] -struct Control { - speed: i32, - turn: i32, -} - -#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] -enum Command { - Arm, - Stop, - Disarm, -} - -fn main() -> Result<(), Box> { - let broadcast = broadcast::Info::new().produce(); - let info = track::Info::default().with_timescale(Timescale::MILLI); - let state = broadcast.create_track("control.json", info.clone())?; - let commands = broadcast.create_track("commands.json", info)?; - let mut state_reader = snapshot::Consumer::::new(state.consume(), Default::default()); - let mut command_reader = stream::Consumer::::new(commands.subscribe(None), Default::default()); - let mut state = snapshot::Producer::new(state, Default::default()); - let mut commands = stream::Producer::new(commands, Default::default()); - - // The reader falls behind. Replaying the old steering values would be harmful. - for (tick, control) in [ - Control { speed: 1, turn: -1 }, - Control { speed: 2, turn: 1 }, - Control { speed: 0, turn: 0 }, - ] - .iter() - .enumerate() - { - state.update(Timed::from(control).at(Timestamp::from_millis(tick as u64)?))?; - } - let _state = state.finish()?; - - // Every discrete command is meaningful, including Stop between Arm and Disarm. - let expected = [Command::Arm, Command::Stop, Command::Disarm]; - for (tick, command) in expected.iter().enumerate() { - commands.append(Timed::from(command).at(Timestamp::from_millis(tick as u64)?))?; - } - let _commands = commands.finish()?; - - let waiter = kio::Waiter::noop(); - // `latest` skips the stale steering values; `next` would replay them in order. - let Poll::Ready(Ok(Some(latest))) = state_reader.poll_latest(&waiter) else { - return Err("finished control track did not yield its latest state".into()); - }; - assert_eq!(latest.value, Control { speed: 0, turn: 0 }); - assert_eq!(latest.at, Some(Timestamp::from_millis(2)?)); - println!("Current control: {:?} at {:?}", latest.value, latest.at); - assert!(matches!(state_reader.poll_latest(&waiter), Poll::Ready(Ok(None)))); - for (tick, expected) in expected.into_iter().enumerate() { - let Poll::Ready(Ok(Some(command))) = command_reader.poll_next(&waiter) else { - return Err("finished command track lost an ordered command".into()); - }; - assert_eq!(command.value, expected); - assert_eq!(command.at, Some(Timestamp::from_millis(tick as u64)?)); - println!("Command: {:?} at {:?}", command.value, command.at); - } - assert!(matches!(command_reader.poll_next(&waiter), Poll::Ready(Ok(None)))); - Ok(()) -} - -#[cfg(test)] -#[test] -fn latest_state_and_every_command() { - main().unwrap(); -} From 3b7e4b4cf1b9ad7740b55ed34fd69f43cac2849d Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 14 Nov 2026 11:55:14 -0800 Subject: [PATCH 3/5] docs: non-media work belongs to Kixel Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 1 + 1 file changed, 1 insertion(+) diff --git a/AGENTS.md b/AGENTS.md index 94079a17a58..a3c59a50e4c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -38,6 +38,7 @@ PRs target `main`, where breaking API changes are allowed. Wire changes stay backwards compatible with *published* drafts/versions; an unpublished `-wip` version changes freely. moq.pro rolls out with relays on the newest and previous release side by side, so those two versions must interoperate as relays. A temporary migration for that is fine when it is clearly marked and easy to delete at the next version migration. Our libraries prefer moq-lite, so moq-transport compatibility only matters against third-party implementations. +Ask before planning or building work that a large live-media platform would not use, such as mesh networking, robotics, AI agents, P2P, or TCP tunnels: it belongs to Kixel (kixel-ai/kixel.ai), which builds on moq rather than waiting on it. Build output may live outside the checkout and gets wiped, so scratch never goes in `target/`. From a09b65aa463f4f14d27dbf26f9e98e1725e48edc Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 14 Nov 2026 12:09:01 -0800 Subject: [PATCH 4/5] quest: only the mesh libraries left for Kixel here Applies Pullfrog's suggestion. Co-Authored-By: Claude Opus 5.5 --- quest/m3/README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/quest/m3/README.md b/quest/m3/README.md index f516b9c3f12..68733b2358f 100644 --- a/quest/m3/README.md +++ b/quest/m3/README.md @@ -33,9 +33,9 @@ released. They stay out of m1 and m2 so the cut does not wait on them; each starts as soon as its prerequisites (sockets, fastest-reader pacing, outbound federation) land. Superseded 2026-11-12 (maintainer, /quest-audit): both moved to [m1](/quest/m1/README.md), since the cut no longer waits on m1. The mesh -libraries came back 2026-11-13, below; byte streams stay in m1. Both left -moq for Kixel on 2026-11-14 (maintainer): moq keeps what a large live-media -platform would use. +libraries came back 2026-11-13, below; byte streams stay in m1. The mesh +libraries left moq for Kixel on 2026-11-14 (maintainer): moq keeps what a +large live-media platform would use. Decided 2026-11-12 (maintainer, /quest-audit before the cut), from a read of every quest against the code and the downstream quest trees: From 082fdb6deb889c0451d7476af95249831cf5964b Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 15 Nov 2026 02:54:23 -0800 Subject: [PATCH 5/5] fix(moq-json): mutate's doc example passes a Timestamp Co-Authored-By: Claude Opus 5.5 --- rs/moq-json/src/snapshot/producer.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rs/moq-json/src/snapshot/producer.rs b/rs/moq-json/src/snapshot/producer.rs index 0b840024fc4..72c1327d824 100644 --- a/rs/moq-json/src/snapshot/producer.rs +++ b/rs/moq-json/src/snapshot/producer.rs @@ -125,7 +125,7 @@ impl Producer { /// # fn example(producer: &mut moq_json::snapshot::Producer) -> moq_json::Result<()> { /// producer.mutate(|catalog| { /// catalog["scte35"] = serde_json::json!({ "id": 1 }); - /// }, None) + /// }, moq_net::Timestamp::from_millis(100).unwrap()) /// # } /// ``` ///