Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions doc/bin/relay/cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -278,16 +278,18 @@ decides which relays receive the cluster token.

## Identity

Each relay has a Hop ID: the value it adds to a route's hop list for loop
detection and shortest-path routing. It is random on every start, so a
Each relay has an identity that it uses for loop detection and routing, and
declares in SETUP on moq-lite-05 and later: a Hop ID added to each route's hop
list, or on the opt-in `moq-lite-07-wip` a Node ID naming the route's origin. It is random on every start, so a
restarted relay is a new node to the rest of the mesh, and there is no setting
to pin it.

A relay never dials its own `cluster.node`, so every relay can share one
`cluster.connect` list or API response. Peer URLs that differ only in query,
userinfo, fragment, or slashes name one peer and get one link. A relay
refuses a session that declares its own Hop ID, which means a peer URL
reaches the relay itself under another name, such as an IP for its hostname.
refuses a session that declares its own identity (its Hop ID, or on
`moq-lite-07-wip` its Node ID), which means a peer URL reaches the relay itself
under another name, such as an IP for its hostname.

## Failure detection

Expand Down
12 changes: 11 additions & 1 deletion drafts/draft-lcurley-moq-lite.md
Original file line number Diff line number Diff line change
Expand Up @@ -966,6 +966,8 @@ The following Setup Parameters are defined:
|------|----------------|-------------|
| 0x4 | Cost | Cost (i) |
|------|----------------|-------------|
| 0x5 | Node ID | Node ID (i) |
|------|----------------|-------------|
| 0x6 | Implementation | Tokens (s) |
|------|----------------|-------------|
| 0x7 | Token | Token (b) |
Expand Down Expand Up @@ -1022,6 +1024,14 @@ Both endpoints send it and the two values need not match: the parameter prices t

A declared cost is an assertion, not an instruction: a receiver MAY charge a locally configured value of at least 1 instead, so a peer cannot reprice its neighbours by declaring itself cheap.

### Node ID Parameter {#node-id-parameter}
The Node ID Parameter declares the Node ID the sender originates its own routes under (see [Routing](#routing)).
Its value MUST NOT be 0 or have the most significant bit set; a receiver MUST close the session with a PROTOCOL_VIOLATION if it does.

A receiver that receives its own Node ID has reached itself, such as a relay dialing an address of its own, and SHOULD close the session.
A Node ID in SETUP is declared, not proven, and a receiver MUST NOT authorize on it.
Either endpoint MAY send it, and an endpoint that originates no routes omits it, having no Node ID to declare. A relay MUST NOT forward it.

### Implementation Parameter {#implementation-parameter}
The Implementation Parameter names the sender's software, so a peer can attribute a session to its build.
The Parameter Value is a UTF-8 string of space-separated `name/version` product tokens, outermost first, such as `moq-ffi/0.4.12 moq-net-rs/0.3.10`; it is otherwise opaque to moq-lite.
Expand Down Expand Up @@ -2189,7 +2199,7 @@ The `Message Length` describes the payload size on the wire.
- An advertiser MAY announce a path it answers through a claim under the claim's Instance, so subscriptions resolved through the claim continue, and a claim still serving that Instance when the announcement ends restarts under a new one.
- A receiver mints an Instance, starting with `~`, for each announcement a session of an earlier version delivers.
- Added announce compression: ANNOUNCE_START gains `Path Base` and `Path Keep` to copy the head of a live claim's suffix.
- Split routing into two layers on the Announce Stream. ROUTE_START (0x6), ROUTE_UPDATE (0x7), and ROUTE_END (0x8) advertise how the publisher reaches an origin node, named by a Node ID, with a seqno, an additive metric, and a down-only bit, and loops are prevented by Babel's feasibility condition. Every ANNOUNCE_START names its route's Route ID in place of the Hop ID list, so a link change sends one ROUTE_UPDATE per origin rather than an update per claim. Removed the Hop ID list, ANNOUNCE_OK's Hop ID, and the SETUP Hop parameter (0x5).
- Split routing into two layers on the Announce Stream. ROUTE_START (0x6), ROUTE_UPDATE (0x7), and ROUTE_END (0x8) advertise how the publisher reaches an origin node, named by a Node ID, with a seqno, an additive metric, and a down-only bit, and loops are prevented by Babel's feasibility condition. Every ANNOUNCE_START names its route's Route ID in place of the Hop ID list, so a link change sends one ROUTE_UPDATE per origin rather than an update per claim. Removed the Hop ID list and ANNOUNCE_OK's Hop ID, and replaced the SETUP Hop parameter (0x5) with a [Node ID parameter](#node-id-parameter), so an endpoint that reaches itself detects it at setup.
- Added ROUTE_HOLD (0xB), which holds a route while it recovers; ROUTE_END lets it go. Neither withdraws the origin's claims, and the subscriber echoes every ROUTE_END back.
- Added ROUTE_REQUEST (0xA), sent by the subscriber on the Announce Stream to ask for a route with a newer seqno when every route it has is infeasible.
- Added ANNOUNCE_FLOOR (0x9), closing each batch of a route's claims with the origin Revision at which the stream's claims across the requested prefix are certified plus one, or 0 to certify nothing, so a stale copy never revives a claim whose withdrawal a newer one delivered. A floor certifies only the claims its own request shows.
Expand Down
10 changes: 6 additions & 4 deletions js/net/src/lite/connection.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ import {
hasDatagramTracks,
hasGoaway,
hasHidden,
hasNodeId,
hasPadding,
hasProbeRtt,
hasRouteKey,
Expand Down Expand Up @@ -422,9 +423,10 @@ export class Connection implements Session {
// path and leave routing to the URL. The probe level reflects what this transport
// can actually measure; we never pad, so we never advertise Increase.
// Role stays Both: the publish origin starts empty and fills later, and consume is
// called after this point, so there is nothing to narrow yet. The origin declares
// our session identity so the peer can filter
// reflected announcements (lite-06 removed ANNOUNCE_REQUEST's exclude_hop for it).
// called after this point, so there is nothing to narrow yet. Before lite-07 the hop
// declares our session identity so the peer can filter reflected announcements
// (lite-06 removed ANNOUNCE_REQUEST's exclude_hop for it). From lite-07 the parameter
// is a Node ID, which this per-connection hop is not: we originate no routes, so we omit it.
async #sendSetup(): Promise<void> {
const writer = await Writer.open(this.#quic, { version: this.#version });
try {
Expand All @@ -433,7 +435,7 @@ export class Connection implements Session {
const routeKey = hasRouteKey(this.#version) ? await proveRouteKey(this.#quic, this.#client) : undefined;
await new Setup({
probe,
hop: this.hop,
hop: hasNodeId(this.#version) ? undefined : this.hop,
token: this.#token,
implementation: this.#implementation,
routeKey,
Expand Down
10 changes: 9 additions & 1 deletion js/net/src/lite/setup.test.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { expect, test } from "bun:test";
import { HopSchema } from "../hop.ts";
import { Reader, Writer } from "../stream.ts";
import { encodeVarint, Reader, Writer } from "../stream.ts";
import * as Varint from "../varint.ts";
import { ProbeLevel, Role, Setup } from "./setup.ts";
import { Version } from "./version.ts";
Expand Down Expand Up @@ -149,6 +149,14 @@ test("hop 0 decodes as absent", async () => {
expect(got.hop).toBeUndefined();
});

test("lite-07 Node ID is never 0 or qualified", async () => {
for (const id of [0n, (1n << 63n) | 7n]) {
await expect(decodeParam(PARAM_HOP, encodeVarint(id, Version.DRAFT_07), Version.DRAFT_07)).rejects.toThrow();
}
const qualified = new Setup({ hop: HopSchema.parse((1n << 63n) | 7n) });
await expect(bytes((w) => qualified.encode(w, Version.DRAFT_07), Version.DRAFT_07)).rejects.toThrow();
});

test("SETUP with implementation round-trips", async () => {
const got = await roundTrip(
new Setup({ hop: HopSchema.parse(7n), implementation: "moq-ffi/0.4.12 moq-net-rs/0.3.10" }),
Expand Down
17 changes: 13 additions & 4 deletions js/net/src/lite/setup.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import { type Hop, HopSchema, wireHop } from "../hop.ts";
import { encodeVarint, Reader, type Writer } from "../stream.ts";
import { decodeUtf8 } from "../util/utf8.ts";
import * as Message from "./message.ts";
import { hasAuth, hasImplementation, hasRouteKey, hasSetupStream, type Version } from "./version.ts";
import { hasAuth, hasImplementation, hasNodeId, hasRouteKey, hasSetupStream, type Version } from "./version.ts";

const clientParameters = new WeakSet<Setup>();

Expand All @@ -24,8 +24,10 @@ const PARAM_PROBE = 0x1n;
const PARAM_PATH = 0x2n;
/** Setup Parameter id for the client's intended {@link Role} (client-only). */
const PARAM_ROLE = 0x3n;
/** Setup Parameter id for the endpoint's Hop ID. */
/** Setup Parameter id for the endpoint's Hop ID, or from lite-07 its Node ID. */
const PARAM_HOP = 0x5n;
/** The top bit, which marks a Node ID a relay qualified; an endpoint never declares one. */
const QUALIFIED = 1n << 63n;
/** Setup Parameter id for the endpoint's implementation name and version. */
const PARAM_IMPLEMENTATION = 0x6n;
/** Setup Parameter id for the client's connection credential (client-only, lite-07+). */
Expand Down Expand Up @@ -218,7 +220,9 @@ export class Setup {
* This endpoint's Hop ID. The peer uses it to filter announcements and
* subscriptions whose route flows through this endpoint (lite-06 removed the
* per-stream `exclude_hop` in its favor). `undefined` when the endpoint declares
* no identity; a wire value of 0 decodes the same way.
* no identity; a wire value of 0 decodes the same way. From lite-07 it is the
* endpoint's Node ID instead, never 0 or with the top bit set, so a peer that
* receives its own has reached itself.
*/
hop?: Hop;

Expand Down Expand Up @@ -270,6 +274,7 @@ export class Setup {
params.setVarint(PARAM_ROLE, this.role, version);
}
if (this.hop !== undefined && this.hop !== 0n) {
if (hasNodeId(version) && this.hop >= QUALIFIED) throw new Error("setup Node ID cannot be qualified");
Comment thread
pullfrog[bot] marked this conversation as resolved.
Resolved
params.setVarint(PARAM_HOP, wireHop(this.hop, version), version);
}
if (this.implementation) {
Expand Down Expand Up @@ -302,8 +307,12 @@ export class Setup {
const roleCode = await params.getVarint(PARAM_ROLE, version);
const role = roleCode === undefined ? Role.Both : roleFromCode(roleCode);

// 0 carries no identity (it cannot be excluded), so it decodes as absent.
// Before lite-07, 0 carries no identity (it cannot be excluded), so it decodes as
// absent. A lite-07 Node ID is never 0 or qualified.
const hopRaw = await params.getVarint(PARAM_HOP, version);
if (hopRaw !== undefined && hasNodeId(version) && (hopRaw === 0n || hopRaw >= QUALIFIED)) {
throw new Error("setup Node ID must be non-zero and unqualified");
}
const hop = hopRaw === undefined || hopRaw === 0n ? undefined : HopSchema.parse(hopRaw);

// An older version does not know the token parameter, so it ignores it like any other.
Expand Down
16 changes: 16 additions & 0 deletions js/net/src/lite/version.ts
Original file line number Diff line number Diff line change
Expand Up @@ -672,6 +672,22 @@ export function hasSynced(version: Version): boolean {
}
}

/** Whether SETUP's 0x5 parameter carries a Node ID rather than a Hop ID, from lite-07. */
export function hasNodeId(version: Version): boolean {
// Explicitly list older versions so future versions default to carrying it.
switch (version) {
case Version.DRAFT_01:
case Version.DRAFT_02:
case Version.DRAFT_03:
case Version.DRAFT_04:
case Version.DRAFT_05:
case Version.DRAFT_06:
return false;
default:
return true;
}
}

/** Whether SETUP may carry a route key proof, added in lite-07. */
export function hasRouteKey(version: Version): boolean {
// Explicitly list older versions so future versions default to carrying it.
Expand Down
1 change: 0 additions & 1 deletion quest/m2/cluster-routing/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,6 @@ setup after the capabilities land.

- [Outbound federation](/quest/m2/cluster-routing/federation.md) - an outbound-only local relay exchanges authorized broadcasts with the CDN and survives partition/healing
- [Routes and announces](/quest/m2/cluster-routing/routes/README.md) - implement the proven route layer with Rust/JS parity and tiered-cluster acceptance
- [A relay refuses a lite-07 session with itself](/quest/m2/cluster-routing/lite07-self-dial.md) - the self-dial refusal lite-06 has, on lite-07, whose SETUP carries no Hop ID

## Related

Expand Down
27 changes: 0 additions & 27 deletions quest/m2/cluster-routing/lite07-self-dial.md

This file was deleted.

1 change: 1 addition & 0 deletions rs/moq-net/src/client.rs
Original file line number Diff line number Diff line change
Expand Up @@ -325,6 +325,7 @@ impl Client {
cost: self.cost,
// Filled by `lite::start` from the attached origin handles.
hop: None,
node: None,
// Older versions carry the credential in the URL query instead.
token: self.setup_token.clone().filter(|_| version.has_auth()),
ietf: self.extensions.ietf && version.has_ietf(),
Expand Down
9 changes: 9 additions & 0 deletions rs/moq-net/src/lite/session.rs
Original file line number Diff line number Diff line change
Expand Up @@ -265,6 +265,15 @@ pub fn start(config: Config) -> Result<SessionStart, Error> {
.or_else(|| subscribe.as_ref().map(|origin| origin.hop()))
.map(Hop::from);
}
// Lite-07 declares the Node ID our routes originate under instead, so a peer that is
// this same process recognizes itself. Read without creating the route node, which
// the session's route layer creates when it starts.
if version.has_routes() && our_setup.node.is_none() {
our_setup.node = publish
.as_ref()
.and_then(|origin| origin.node_id())
.or_else(|| subscribe.as_ref().and_then(|origin| origin.consume().node_id()));
}

// Prove our origin's route key, bound to this session, so a peer that scopes us files
// our routes under it. A transport without an exporter cannot bind a proof, so neither
Expand Down
78 changes: 52 additions & 26 deletions rs/moq-net/src/lite/setup.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ const PARAM_PATH: u64 = 0x2;
const PARAM_ROLE: u64 = 0x3;
/// Setup Parameter id for the link cost the dialer assigns to this connection.
const PARAM_COST: u64 = 0x4;
/// Setup Parameter id for the endpoint's Hop ID (before lite-07, which names routes instead).
/// Setup Parameter id for the endpoint's Hop ID before lite-07, and its Node ID from lite-07.
const PARAM_HOP: u64 = 0x5;
/// Setup Parameter id for the sender's implementation, as `name/version` product tokens.
const PARAM_IMPLEMENTATION: u64 = 0x6;
Expand Down Expand Up @@ -198,9 +198,12 @@ pub struct Setup {
/// The peer uses it to serve this endpoint's subscriptions from a route that
/// does not flow through it (the same split horizon the announce filter
/// applies). `None` when the endpoint has no meaningful identity (a
/// leaf that never forwards); a wire value of 0 decodes as `None`. Lite-07 names
/// routes instead, so it never sends one and ignores one received.
/// leaf that never forwards); a wire value of 0 decodes as `None`. Lite-07 declares
/// [`node`](Self::node) instead, so it never sends one.
pub hop: Option<crate::Hop>,
/// This endpoint's own Node ID (lite-07+), the identity it originates routes under.
/// A peer that receives its own Node ID has reached itself. Never 0 or qualified.
pub node: Option<std::num::NonZeroU64>,
/// The connection credential (lite-07+), which the empty-token Auth Stream names.
/// Sent only by the client. The parameter carries no Token Type, so it is always
/// [`Token::OUT_OF_BAND`](crate::setup::Token::OUT_OF_BAND).
Expand All @@ -227,6 +230,7 @@ impl std::fmt::Debug for Setup {
.field("role", &self.role)
.field("cost", &self.cost)
.field("hop", &self.hop)
.field("node", &self.node)
.field("token", &self.token)
.field("ietf", &self.ietf)
.field("implementation", &self.implementation)
Expand Down Expand Up @@ -270,6 +274,15 @@ impl<'a> Message<'a> for Setup {
if let Some(hop) = self.hop.filter(|_| !version.has_routes()) {
params.set_varint(PARAM_HOP, hop.wire_id(version.into()), version)?;
}
if let Some(node) = self.node {
if !version.has_routes() {
return Err(EncodeError::Version);
}
if node.get() & crate::route::QUALIFIED != 0 {
return Err(EncodeError::InvalidState);
}
params.set_varint(PARAM_HOP, node.get(), version)?;
}
if let Some(implementation) = &self.implementation {
params.set_bytes(PARAM_IMPLEMENTATION, implementation.as_bytes().to_vec());
}
Expand Down Expand Up @@ -336,15 +349,20 @@ impl<'a> Message<'a> for ReceivedSetup {
if cost == Some(0) && version.has_routes() {
return Err(DecodeError::InvalidValue);
}
// 0 is legal on the wire but carries no identity (it can't be excluded),
// so it decodes as "not declared" rather than an error. Lite-07 dropped the
// parameter, so it ignores one like any unknown parameter.
let hop = match version.has_routes() {
true => None,
false => params
.get_varint(PARAM_HOP, version)?
.and_then(std::num::NonZeroU64::new)
.map(crate::Hop::from),
// Before lite-07, 0 is legal on the wire but carries no identity (it can't be
// excluded), so it decodes as "not declared" rather than an error. A lite-07
// Node ID is never 0 or qualified.
let declared = params.get_varint(PARAM_HOP, version)?;
let (hop, node) = match version.has_routes() {
true => match declared {
None => (None, None),
Some(id) if id & crate::route::QUALIFIED != 0 => return Err(DecodeError::InvalidValue),
Some(id) => (
None,
Some(std::num::NonZeroU64::new(id).ok_or(DecodeError::InvalidValue)?),
),
},
false => (declared.and_then(std::num::NonZeroU64::new).map(crate::Hop::from), None),
};
let implementation = if version.has_implementation() {
crate::setup::decode_implementation(params.get_bytes(PARAM_IMPLEMENTATION))?
Expand Down Expand Up @@ -382,6 +400,7 @@ impl<'a> Message<'a> for ReceivedSetup {
role,
cost,
hop,
node,
token,
ietf,
implementation,
Expand Down Expand Up @@ -992,28 +1011,35 @@ mod tests {
assert_eq!(Setup::decode_slice(&buf, Version::Lite06).unwrap().0.hop, None);
}

/// Lite-07 names routes instead of hops: it never sends the Hop parameter, and ignores
/// one a peer sends like any unknown parameter.
/// Lite-07 carries the endpoint's Node ID in the parameter that names its Hop ID before.
#[test]
fn lite07_drops_the_hop() {
fn lite07_declares_the_node_instead_of_the_hop() {
let version = Version::Lite07;
let msg = Setup {
hop: Some(crate::Hop::new(42).unwrap()),
node: std::num::NonZeroU64::new(7),
..Default::default()
};
assert_eq!(
msg.encode_bytes(version).unwrap(),
Setup::default().encode_bytes(version).unwrap()
let got = Setup::decode_slice(&msg.encode_bytes(version).unwrap(), version)
.unwrap()
.0;
assert_eq!(got.hop, None);
assert_eq!(got.node, msg.node);
assert!(
msg.encode_bytes(Version::Lite06).is_err(),
"only lite-07 carries a Node ID"
);

let mut params = Parameters::default();
params.set_varint(super::PARAM_HOP, 42, version).unwrap();
let mut data = Vec::new();
let mut writer = Encoder::new(&mut data, version.into());
let prefix = writer.prefix_varint();
params.encode(&mut writer, version).unwrap();
writer.fill(prefix).unwrap();
assert_eq!(Setup::decode_slice(&data, version).unwrap().0.hop, None);
for id in [0, crate::route::QUALIFIED | 7] {
let mut params = Parameters::default();
params.set_varint(super::PARAM_HOP, id, version).unwrap();
let mut data = Vec::new();
let mut writer = Encoder::new(&mut data, version.into());
let prefix = writer.prefix_varint();
params.encode(&mut writer, version).unwrap();
writer.fill(prefix).unwrap();
assert!(Setup::decode_slice(&data, version).is_err(), "Node ID {id:#x}");
}
}

// A declared id of 0 carries no identity (it cannot be excluded), so it
Expand Down
Loading
Loading