Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion doc/bin/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -431,7 +431,8 @@ moq --cluster-lan --cluster-lan-secret /etc/moq/cluster.key import capture
MoQ process and `moq-relay` on the LAN, with no other configuration. Without
`--cluster-lan-secret`, anyone who can reach the listener joins, so leave it
unset only on networks you trust. `--cluster-connect` is a MoQ side on its own,
so it needs no `--connect`. WAN cluster dials (`--cluster-connect`,
so it needs no `--connect`. `--connect-token` goes only to `--connect`, never to
a cluster peer: give those `--cluster-token`. WAN cluster dials (`--cluster-connect`,
`--cluster-api-url`) bind `--connect-bind`'s address. On port 0, each cluster
session takes its own ephemeral port, so a redial leaves from a fresh one. A
fixed port pins every WAN cluster dial to it, while LAN dials still take
Expand Down
5 changes: 4 additions & 1 deletion doc/bin/relay/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -294,7 +294,10 @@ See [Clustering](/bin/relay/cluster).

## \[connect]

Settings for outbound dials (cluster peers, auth API).
Settings for outbound dials (cluster peers, auth API). Where and with what
credential to dial come from `[cluster]`, so the relay refuses to start with
`connect.url` (`--connect`): use `cluster.connect`; or `connect.tokens`
(`--connect-token`): use `cluster.token` or a peer's `token`.

```toml
[connect]
Expand Down
3 changes: 1 addition & 2 deletions quest/m2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,7 @@ removal, a rename, or a wire, catalog, or format change that is free until
item re-checked for a downstream caller first.
- **Relay and transports.** [CPU](/quest/m2/relay-cpu.md), which the
memory pairing found; [congestion names](/quest/m2/congestion-names.md);
[config names](/quest/m2/relay-config-names.md),
[the two dial keys](/quest/m2/relay-connect-keys.md), and
[config names](/quest/m2/relay-config-names.md) and
[the 0.14 leftovers](/quest/m2/relay-014-leftovers.md);
[transport names](/quest/m2/transport-names.md); and
[stats names](/quest/m2/stats-old-names.md).
Expand Down
1 change: 0 additions & 1 deletion quest/m2/relay-config-names.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,5 +92,4 @@ key renamed, one nix option renamed. Wire: none.

## Related

- [The relay refuses dial keys it never uses](/quest/m2/relay-connect-keys.md) - the other config change at this freeze
- [The switch](/quest/m2/multicast/switch.md) - retires what else reads a hop bound
72 changes: 0 additions & 72 deletions quest/m2/relay-connect-keys.md

This file was deleted.

83 changes: 83 additions & 0 deletions rs/moq-relay/src/cluster.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1647,6 +1647,9 @@ impl Cluster {
/// A fixed bind port is kept instead: static and API peers share one socket
/// on it, while LAN dials still take ephemeral ports.
///
/// The template's [`tokens`](moq_tokio::connect::Config::tokens) are ignored: a
/// peer is handed only `cluster.token` or its own `token`.
///
/// Required when any peer is configured; [`start`](Self::start) returns an
/// error otherwise.
pub fn with_connect(mut self, connect: moq_tokio::connect::Config, quic: moq_tokio::quic::Config) -> Self {
Expand Down Expand Up @@ -2319,6 +2322,9 @@ impl Cluster {
.as_deref()
.cloned()
.context("cluster dial without a dial template (call Cluster::with_connect)")?;
// A peer gets only the credential the cluster settings chose for it, so the
// template's would reach every peer, LAN ones included.
connect.tokens.clear();
if ephemeral {
let mut bind = connect.resolve().bind;
bind.set_port(0);
Expand Down Expand Up @@ -4397,6 +4403,83 @@ mod tests {
}
}

/// Cluster dials carry only the credentials the cluster settings name: a dial
/// template's own tokens reach neither a static peer nor a LAN peer, in SETUP
/// or the URL. lite-07 puts the template's first token in SETUP and the rest on
/// AUTH streams, so an empty SETUP token means the template sent none. The
/// template's other settings (version, TLS) still dial both.
#[tokio::test]
async fn cluster_dials_present_no_template_tokens() {
const TIMEOUT: std::time::Duration = std::time::Duration::from_secs(10);
moq_tokio::crypto::install_default().unwrap();

let mut server = moq_tokio::server::Config::default();
server.listen.bind = Some("127.0.0.1:0".parse().unwrap());
server.listen.tls.generate = vec!["localhost".into()];
server.listen.version = vec!["moq-lite-07-wip".parse().unwrap()];
let server = server.init().expect("server");
let fingerprint = server
.certificates()
.fingerprints()
.into_iter()
.next()
.expect("generated fingerprint");
let mut listener = server.listen().await.expect("listen");
let port = listener.local_addr().expect("quic addr").port();
let (sender, mut seen) = tokio::sync::mpsc::unbounded_channel();
tokio::spawn(async move {
while let Ok(request) = listener.accept().await {
let token = request.token().map(|token| token.value.clone());
sender
.send((request.path().to_string(), request.query().map(str::to_string), token))
.ok();
tokio::spawn(async move {
let Ok(_session) = request.ok().await else { return };
std::future::pending::<()>().await;
});
}
});
let mut next = async || {
kio::clock::timeout(TIMEOUT, seen.recv())
.await
.expect("no dial arrived")
.expect("server stopped")
};

let mut connect = moq_tokio::connect::Config::default();
connect.tls.insecure = Some(true);
connect.version = vec!["moq-lite-07-wip".parse().unwrap()];
connect.tokens = vec!["template-setup".parse().unwrap(), "template-auth".parse().unwrap()];

let config = Config {
connect: vec![Peer::new(format!("moqt://localhost:{port}/static"))],
..Default::default()
};
let cluster = new_cluster(config)
.expect("cluster")
.with_connect(connect, Default::default());
let started = cluster.clone().start().await.expect("start");
let _running = tokio::spawn(started.run());
assert_eq!(next().await, ("/static".to_string(), None, None), "static peer");

cluster.set_lan_credential("lan-proof");
let url: Url = format!("moqt://127.0.0.1:{port}{CLUSTER_PATH}/lan-proof")
.parse()
.expect("url");
let target = DialTarget {
key: PeerKey::canonicalize(&format!("moqt://127.0.0.1:{port}/")),
url: url.clone(),
urls: vec![url],
cost: None,
fingerprint: Some(fingerprint),
lan: true,
upstream: false,
};
let _lan = cluster.dial_lan_target(&target).expect("dial");
let (path, query, token) = next().await;
assert_eq!((query, token), (None, None), "LAN peer at {path}");
}

/// A second close awaits the shutdown the first started, and still finishes it
/// when the first caller gives up, so no caller returns while links are open.
#[tokio::test]
Expand Down
132 changes: 130 additions & 2 deletions rs/moq-relay/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -157,9 +157,35 @@ struct Cli {
/// flattens it has its own spec, so `Config::spec()` would hand back the wrong
/// one and read as though it were theirs.
pub fn spec() -> &'static usage_rs::spec::Spec<'static> {
Cli::spec()
static SPEC: std::sync::OnceLock<usage_rs::spec::Spec<'static>> = std::sync::OnceLock::new();
SPEC.get_or_init(|| {
let base = Cli::spec();
// `[connect]` is the shared client fragment, flattened whole; its `url` and `tokens`
// still parse so [`Config::resolve`] can refuse them, but help no longer offers them.
let flags = base
.root
.flags
.iter()
.map(|meta| usage_rs::spec::FlagMeta {
hide: meta.hide || meta.flag.longs.iter().any(|long| REFUSED_CONNECT.contains(long)),
..*meta
})
.collect::<Vec<_>>();
let root = usage_rs::spec::CommandMeta {
flags: flags.leak(),
..*base.root
};
usage_rs::spec::Spec {
root: Box::leak(Box::new(root)),
..*base
}
})
}

/// The `[connect]` flags a relay refuses: cluster dials take their peers and
/// credentials from `[cluster]`.
const REFUSED_CONNECT: [&str; 2] = ["connect", "connect-token"];

/// Every setting the relay merges, as its flags declare them.
pub(crate) const fn registry() -> usage_rs::config::Registry {
Cli::SETTINGS_REGISTRY
Expand Down Expand Up @@ -219,7 +245,7 @@ impl Config {
let (cli, cli_layer) = match Cli::parse_from_with_settings(&argv) {
Ok(parsed) => parsed,
Err(err) => {
let answer = moq_tokio::cli::answer(Cli::spec(), Cli::command(), &argv, err);
let answer = moq_tokio::cli::answer(spec(), Cli::command(), &argv, err);
if answer.is_question() {
answer.exit();
}
Expand Down Expand Up @@ -322,6 +348,7 @@ impl Config {
pub(crate) fn resolve(mut self) -> anyhow::Result<Resolved> {
let deprecated = self.deprecated();
anyhow::ensure!(deprecated.is_empty(), "{deprecated}");
self.check_connect()?;
// Here rather than at an endpoint, since io_uring workers and lazily built
// dial clients would otherwise check `[quic]` late or never. `[limits.peer.quic]`
// is checked by `Cluster::new`, which embedders call directly.
Expand Down Expand Up @@ -349,6 +376,33 @@ impl Config {
Ok(Resolved(self))
}

/// Refuse the `[connect]` keys a cluster dial has no use for.
///
/// The relay flattens the whole client fragment as the template for cluster dials,
/// but a peer's address and credential come from `[cluster]`: a URL here would be
/// read by nothing, and tokens here would reach every peer, LAN ones included.
fn check_connect(&self) -> anyhow::Result<()> {
let mut refused = Vec::new();
if self.connect.url.is_some() {
refused.push(
"--connect / MOQ_CONNECT / connect.url: name the peers to dial with \
--cluster-connect / MOQ_CLUSTER_CONNECT / cluster.connect",
);
}
if !self.connect.tokens.is_empty() {
refused.push(
"--connect-token / MOQ_CONNECT_TOKEN / connect.tokens: give peers a credential with \
--cluster-token / MOQ_CLUSTER_TOKEN / cluster.token, or a `token` on a cluster.connect peer",
);
}
anyhow::ensure!(
refused.is_empty(),
"moq-relay does not read these settings; remove them and try again:\n {}",
refused.join("\n ")
);
Ok(())
}

/// Refuse worker options that need a worker group, or a build, this relay lacks.
fn check_runtime(&self) -> anyhow::Result<()> {
let workers = self.runtime.workers().is_some();
Expand Down Expand Up @@ -670,6 +724,80 @@ mod tests {
}
}

/// What the refusal says for `[connect]`'s URL and its tokens.
const REFUSED_URL: &str = "--connect / MOQ_CONNECT / connect.url: name the peers to dial with \
--cluster-connect / MOQ_CLUSTER_CONNECT / cluster.connect";
const REFUSED_TOKENS: &str = "--connect-token / MOQ_CONNECT_TOKEN / connect.tokens: give peers a credential \
with --cluster-token / MOQ_CLUSTER_TOKEN / cluster.token";

/// `[connect]`'s URL and tokens stop the relay from a flag or the file, naming
/// what replaces each, while the rest of `[connect]` still resolves.
#[test]
fn connect_url_and_tokens_refuse_to_boot() {
if !test_env::Env::without(&["MOQ_CONNECT", "MOQ_CONNECT_TOKEN"]).isolate() {
return;
}
let refusal = |config: Config| config.resolve().expect_err("must refuse").to_string();

let flags = Config::parse_and_merge(["moq-relay", "--connect", "https://peer.example/"]).unwrap();
let err = refusal(flags);
assert!(err.contains(REFUSED_URL) && !err.contains("connect.tokens"), "{err}");
let flags = Config::parse_and_merge(["moq-relay", "--connect-token", "secret"]).unwrap();
let err = refusal(flags);
assert!(err.contains(REFUSED_TOKENS) && !err.contains("connect.url"), "{err}");

let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("relay.toml");
std::fs::write(&path, "[connect]\nurl = 'https://peer.example/'\ntokens = ['secret']\n").unwrap();
let file = Config::parse_and_merge([std::ffi::OsString::from("moq-relay"), path.into()]).unwrap();
let err = refusal(file);
assert!(err.contains(REFUSED_URL) && err.contains(REFUSED_TOKENS), "{err}");

let rest = Config::parse_and_merge([
"moq-relay",
"--connect-bind",
"[::]:0",
"--connect-timeout",
"5s",
"--connect-version",
"moq-lite-05",
"--connect-tls-root",
"ca.pem",
])
.unwrap();
rest.resolve().expect("the dial template's other keys still resolve");
}

/// The same refusal from the environment, the half a deployment most often uses.
#[test]
fn connect_env_refuses_to_boot() {
if !test_env::Env::default()
.set("MOQ_CONNECT", "https://peer.example/")
.set("MOQ_CONNECT_TOKEN", "secret")
.isolate()
{
return;
}
let config = Config::parse_and_merge(["moq-relay"]).unwrap();
let err = config.resolve().expect_err("must refuse").to_string();
assert!(err.contains(REFUSED_URL) && err.contains(REFUSED_TOKENS), "{err}");
}

/// `--help` no longer offers the two refused `[connect]` flags, and still lists the rest.
#[test]
fn help_omits_the_refused_connect_flags() {
let argv = [std::ffi::OsStr::new("--help")];
let err = Cli::parse_from(&argv).unwrap_err();
let help = moq_tokio::cli::answer(spec(), Cli::command(), &argv, err)
.message()
.to_string();
assert!(help.contains("--connect-bind"), "{help}");
assert!(
!help.contains("--connect <") && !help.contains("--connect-token"),
"{help}"
);
}

/// A released config file is parsed where the tables used to live, so the relay
/// can name `[quic]` instead of failing with `unknown field quic`, which is what
/// `deny_unknown_fields` would say on its own.
Expand Down