From 21dc5b9f29dc2eaa473bd1b7181b57cc5a209095 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 7 Oct 2026 11:47:07 -0700 Subject: [PATCH 1/9] quest: triage Fastly's moq-relay-interop report Five m0 fixes ahead of Seattle (per-draft parameters, NAMESPACE on the SUBSCRIBE_NAMESPACE stream, split horizon for dialed sessions, no empty d14 prefix, subgroups at object 0) and two m1 quests (cold-relay Largest, documenting the deliberate deviations). Co-Authored-By: Claude Opus 5.5 --- quest/README.md | 6 ++-- quest/m0/README.md | 11 +++++++ quest/m0/dial-split-horizon.md | 49 ++++++++++++++++++++++++++++ quest/m0/ietf-d14-root-prefix.md | 28 ++++++++++++++++ quest/m0/ietf-first-object-zero.md | 30 +++++++++++++++++ quest/m0/ietf-namespace-stream.md | 42 ++++++++++++++++++++++++ quest/m0/ietf-params-per-draft.md | 52 ++++++++++++++++++++++++++++++ quest/m1/README.md | 2 ++ quest/m1/ietf-cold-largest.md | 43 ++++++++++++++++++++++++ quest/m1/ietf-deviations-doc.md | 26 +++++++++++++++ 10 files changed, 286 insertions(+), 3 deletions(-) create mode 100644 quest/m0/dial-split-horizon.md create mode 100644 quest/m0/ietf-d14-root-prefix.md create mode 100644 quest/m0/ietf-first-object-zero.md create mode 100644 quest/m0/ietf-namespace-stream.md create mode 100644 quest/m0/ietf-params-per-draft.md create mode 100644 quest/m1/ietf-cold-largest.md create mode 100644 quest/m1/ietf-deviations-doc.md diff --git a/quest/README.md b/quest/README.md index f1ed4436bfe..f11fef2994f 100644 --- a/quest/README.md +++ b/quest/README.md @@ -7,8 +7,8 @@ grouped into milestones ordered by priority. ## Plan -m0 is everything in flight now: relay hardening and the last known IETF -interop fix ahead of Seattle, wildcard routing, broadcast epochs, and audio playout (the jitter target). m1 is the next wave across reliability, features, performance, and +m0 is everything in flight now: relay hardening and the IETF interop +fixes ahead of Seattle, wildcard routing, broadcast epochs, and audio playout (the jitter target). m1 is the next wave across reliability, features, performance, and planning. m2 holds later features, design studies, and experiments. m3 is gated on the outside world (hardware, a partner, a consumer, or a provider's offer) or is speculative work with no named consumer yet. m4 waits on an upstream release. @@ -20,7 +20,7 @@ move the blocked quest to the milestone its priority belongs in. ## Required -- [m0: immediate priorities](/quest/m0/README.md) - everything in flight now: relay hardening and the last known IETF interop fix for Seattle, wildcard routing, broadcast epochs, and audio playout +- [m0: immediate priorities](/quest/m0/README.md) - everything in flight now: relay hardening and the IETF interop fixes for Seattle, wildcard routing, broadcast epochs, and audio playout - [m1: next wave](/quest/m1/README.md) - reliability, capabilities, performance, and the planning that settles their contracts - [m2: later work](/quest/m2/README.md) - deferred features, design studies, and experiments - [m3: deferred](/quest/m3/README.md) - gated on the outside world (hardware, a partner, a consumer, or a provider's offer), or speculative with no named consumer diff --git a/quest/m0/README.md b/quest/m0/README.md index 847b9ef5562..b179ac2e1bc 100644 --- a/quest/m0/README.md +++ b/quest/m0/README.md @@ -26,6 +26,12 @@ serve instead of enumerating broadcasts. Serving the relay's ingested-only view (`origin::Consumer::local()`) to localhost workers belongs to moq.pro's edge, which embeds moq-relay; it moved there on 2026-09-28. +Interop: Fastly's moq-relay-interop report (run of 2026-09-23, build +7ee2b02) was triaged against `main` on 2026-10-07. Its End of Track, SETUP, +UNSUBSCRIBE and error-code items were already fixed; the five fixes below +LOCATION_FILTER come from it and go ahead of Seattle. The cold-relay Largest +and the deviations it flagged are in m1. + Identity: the [broadcast epoch](/quest/m0/broadcast-epoch/README.md) line gates the next release (decided 2026-10-03: #4741 resumes an un-epoched republish into the old broadcast and stalls its @@ -43,6 +49,11 @@ a published `@moq/watch` break. - [Expiry wakes](/quest/m0/expiry-wakes.md) - an appended group wakes only the parked reads it expires, so a 2.5 ms-frame publisher with 2 s of parked serves stays near idle and the Python and Go interop rows pass - [Draft-22 LOCATION_FILTER](/quest/m0/ietf-location-filter-22.md) - moqt-22 LOCATION_FILTER carries its type instead of a Length in Rust and JS, so a draft-22 peer reads our Next Object correctly +- [moq-transport parameters per draft](/quest/m0/ietf-params-per-draft.md) - every control message accepts exactly the parameters its draft allows, so moqx's draft-16 SUBSCRIBE_NAMESPACE with FORWARD stops failing +- [NAMESPACE on the stream](/quest/m0/ietf-namespace-stream.md) - every draft-16+ SUBSCRIBE_NAMESPACE stream carries NAMESPACE for each match, whatever the peer's SETUP options +- [Dialed split horizon](/quest/m0/dial-split-horizon.md) - a session moq-net dials gets a per-connection hop, so routes never echo back to the peer they came from +- [No empty d14 prefix](/quest/m0/ietf-d14-root-prefix.md) - draft-14 never sends SUBSCRIBE_NAMESPACE with an empty prefix +- [Subgroup at object 0](/quest/m0/ietf-first-object-zero.md) - a draft-18 subgroup with FIRST_OBJECT clear that starts at object 0 is read whole - [Request caps](/quest/m0/request-caps.md) - lite message sizes, IETF request IDs, and per-session announces and subscriptions are bounded - [Prefix route fronts](/quest/m0/prefix-route-fronts.md) - a prefix route cannot be made to mint one front per requested path - [qmux credit](/quest/m0/qmux-credit.md) - qmux returns connection credit for dropped and stopped streams and delivers its close frame, on both lines diff --git a/quest/m0/dial-split-horizon.md b/quest/m0/dial-split-horizon.md new file mode 100644 index 00000000000..eaf35097962 --- /dev/null +++ b/quest/m0/dial-split-horizon.md @@ -0,0 +1,49 @@ +# [S] Dialed sessions get split horizon + +## Goal + +A session moq-net dials, whose peer negotiates no identity, never has a +route it learned from that peer offered back to it, and a SUBSCRIBE it sends +never routes back to it. Today such a session is anonymous: moq-relay +dialing moxygen, moqx or moq-rs over moq-transport (d16, or d17+ without +HOP_ID) echoes the peer's namespaces back, and both sides subscribe to each +other while no objects flow. + +## Plan + +Triaged from Fastly's moq-relay-interop report (2026-09-23 run; items 3 and +13's echo) on 2026-10-07. The draft leaves loop prevention to +implementations (d16 §6.2). + +Facts from `main`: + +- Split horizon keys on the peer's identity: `exclude()` is + `peer.identity().or(self.peer_hop).unwrap_or(UNKNOWN)` + (`rs/moq-net/src/ietf/publisher.rs`, around line 445), and `Hop::UNKNOWN` + excludes nothing (`visible_to`, `rs/moq-net/src/model/origin.rs`). +- Accepted sessions already get a random assigned hop + (`rs/moq-net/src/server.rs`, around lines 231, 381 and 417). +- `Client::with_peer_hop` (`rs/moq-net/src/client.rs:128`) exists, but the + relay's cluster dial (`run_remote_session`, `rs/moq-relay/src/cluster.rs`) + never calls it. An `upstream: true` link already hides upstream-learned + routes, but that is not the default. + +Decided 2026-10-07: fix it in moq-net's `Client`, not only the relay's +dial. A session with no negotiated identity gets a random per-connection hop, +the same way `server.rs` does. Rejected: calling `with_peer_hop` from +moq-relay only, which leaves other moq-net users anonymous. Check whether +`with_peer_hop` still has a caller afterwards, and delete it if not. + +The same gap let a d16 moq-dev origin route its edges' SUBSCRIBEs to two of +the edges (the report's T4 fanout-late-join, two runs). Check that this case +is covered too. + +Test: a dialed anonymous ietf session that announces a namespace does not +receive it back, and a SUBSCRIBE that arrives on it is not routed back to it. + +Public API: possibly removes `Client::with_peer_hop`. Wire: none. + +## Related + +- [moq-transport cluster peers](/quest/m2/ietf-cluster-peers.md) - multi-hop loop safety for moq-transport peers through the cluster extension +- [Cluster routing](/quest/m1/cluster-routing/README.md) - the route layer split horizon feeds diff --git a/quest/m0/ietf-d14-root-prefix.md b/quest/m0/ietf-d14-root-prefix.md new file mode 100644 index 00000000000..1bd1c26d945 --- /dev/null +++ b/quest/m0/ietf-d14-root-prefix.md @@ -0,0 +1,28 @@ +# [XS] No empty namespace prefix on draft-14 + +## Goal + +On draft-14, moq-net never sends SUBSCRIBE_NAMESPACE with an empty prefix. +d14 §9.28 makes a prefix with N = 0 a PROTOCOL_VIOLATION, and moxygen and +moqx refuse it today when an unscoped moq-relay cluster link dials them. + +## Plan + +Triaged from Fastly's moq-relay-interop report (2026-09-23 run, item 2) on +2026-10-07. + +Decided 2026-10-07: on d14, skip a SUBSCRIBE_NAMESPACE whose prefix is +empty, and rely on the peer's unsolicited PUBLISH_NAMESPACE, which d14 §6.2 +allows. Keep today's warn-and-continue when the peer refuses a namespace +subscription. Rejected: failing the link on refusal, and requiring a scope +on d14 links at startup. + +Where it lives: `subscribe_prefixes` (`rs/moq-net/src/ietf/subscriber.rs`, +around line 600) returns `interest_prefixes(origin.allowed())`, which is +`[""]` for an unscoped origin. `run_subscribe_namespace` (around line 861) +sends it. Log at debug when the root is skipped. + +Test: a d14 session with an unscoped origin sends no SUBSCRIBE_NAMESPACE, +and a scoped one still does. Check `js/net` for the same path. + +Public API: none. Wire: d14 stops sending a message the draft forbids. diff --git a/quest/m0/ietf-first-object-zero.md b/quest/m0/ietf-first-object-zero.md new file mode 100644 index 00000000000..beb6c674f29 --- /dev/null +++ b/quest/m0/ietf-first-object-zero.md @@ -0,0 +1,30 @@ +# [XS] A subgroup starting at object 0 is whole + +## Goal + +A draft-18+ subgroup stream with FIRST_OBJECT clear whose first Object ID +is 0 is read as a whole group, in `rs/moq-net`, instead of being dropped as +"a group with no head". A clear bit with any other first ID is still dropped +as it is today. + +## Plan + +Triaged from Fastly's moq-relay-interop report (2026-09-23 run, item 7) on +2026-10-07. d18 §2.2 says the original publisher MUST set FIRST_OBJECT when +it opens a subgroup, and a relay MUST keep it set, so the probe that left it +clear was the one out of spec. The first Object ID is absolute whatever +the bit says (§11.4.2), and object IDs in our model start at 0, so ID 0 is +the head either way. + +Decided 2026-10-07: accept it anyway, to be lenient toward publishers that +are out of spec. Rejected: keeping the drop and documenting it. + +Where it lives: `rs/moq-net/src/ietf/subscriber.rs` (around line 2611) +drops the stream before reading any object. Peek the first object's ID +before deciding. `next_object_id` already enforces the ID sequence. Check +whether `js/net` drops these streams at all; it has no matching check today. + +Test: a d18 stream with FIRST_OBJECT clear and first ID 0 delivers its +group; one with first ID 3 is still dropped. + +Public API: none. Wire: none. diff --git a/quest/m0/ietf-namespace-stream.md b/quest/m0/ietf-namespace-stream.md new file mode 100644 index 00000000000..5447e3a547b --- /dev/null +++ b/quest/m0/ietf-namespace-stream.md @@ -0,0 +1,42 @@ +# [S] SUBSCRIBE_NAMESPACE streams carry NAMESPACE + +## Goal + +On draft-16 and later, every SUBSCRIBE_NAMESPACE response stream carries +NAMESPACE and NAMESPACE_DONE for each matching namespace, whatever the +peer's SETUP options, in `rs/moq-net` and `js/net`. Today a peer that did not +send our SOLICIT option (0x40B5A), which is every other implementation, gets +an empty stream and only the unsolicited PUBLISH_NAMESPACE pushes. + +## Plan + +Triaged from Fastly's moq-relay-interop report (2026-09-23 run) on +2026-10-07. d18 §6.2 says the publisher "MUST send a NAMESPACE message to +any subscriber that has sent SUBSCRIBE_NAMESPACE" for a matching prefix, +and §10.18 puts them on the response stream. d16 §9.25 says the same +without the MUST. Both drafts let a publisher send PUBLISH_NAMESPACE to any +subscriber, and neither says anything about duplicates. + +Decided 2026-10-07: always fill the stream on d16+, and keep the unsolicited +pushes to non-SOLICIT peers, so those peers hear each namespace twice. +Rejected: stopping unsolicited pushes on d16+ (a peer that never subscribes +would learn nothing), and filling only on d18+ (two behaviours for one +message). + +Where it lives: `rs/moq-net/src/ietf/publisher.rs` (around line 2180) +picks `origin.empty()`, or only the hidden remainder, when +`declared.solicit` is unset. Its comment explains the old reason: a peer +hearing both would hold two sources for one namespace. A NAMESPACE is +discovery only, not a route, so that no longer applies; rewrite the comment. +Drafts 14 and 15 keep answering with PUBLISH_NAMESPACE requests, unchanged. +`js/net` mirrors the solicit logic (`js/net/src/ietf/connection.ts`, +`publisher.ts`). + +Update `doc/concept/standard.md` (around lines 100-106 and 141-144), which +documents the empty stream. + +Test: a non-SOLICIT d16 and d18 peer's SUBSCRIBE_NAMESPACE receives +NAMESPACE for an existing match and for one announced later, then +NAMESPACE_DONE when it ends. + +Public API: none. Wire: moq-transport replies move closer to the drafts. diff --git a/quest/m0/ietf-params-per-draft.md b/quest/m0/ietf-params-per-draft.md new file mode 100644 index 00000000000..c535bdc3558 --- /dev/null +++ b/quest/m0/ietf-params-per-draft.md @@ -0,0 +1,52 @@ +# [M] moq-transport parameters follow each draft's lists + +## Goal + +Every control message accepts exactly the parameters its draft allows, in +`rs/moq-net` and `js/net`, for each supported draft (14 through 22). A +parameter the draft defines for that message is decoded or skipped; a +parameter defined only for another message is ignored on drafts 14 to 17 and +closes the session with PROTOCOL_VIOLATION on draft-18 and later; an unknown +key still closes it with PROTOCOL_VIOLATION. The case seen in the wild: +moqx's draft-16 SUBSCRIBE_NAMESPACE carries FORWARD (0x10), which we refuse, +and moqx redials in a loop. + +## Plan + +Triaged from Fastly's moq-relay-interop report (run of 2026-09-23, build +7ee2b02) on 2026-10-07. Facts from the draft text: + +- Unknown message parameter: close with PROTOCOL_VIOLATION (d16 §9.2, d18 + §10.2, d22 §9.20). That rule stays. +- A known parameter in a message it is not defined for: d16 §9.2.2 says + "it MUST be ignored"; d18 §10.2.1 and d22 §9.20.1 close the session. +- d16 allows FORWARD on SUBSCRIBE, REQUEST_UPDATE, PUBLISH, PUBLISH_OK and + SUBSCRIBE_NAMESPACE. d18 drops it from SUBSCRIBE_NAMESPACE (moved to + SUBSCRIBE_TRACKS). + +Decided 2026-10-07: audit every message against every draft rather than +patch FORWARD alone, since the report's EXPIRES case (fixed in #4195) and +this one are the same class. + +Where it lives: the `decode_params!` macro (`rs/moq-net/src/ietf/parameters.rs`, +around line 473) returns `InvalidValue` for any key outside a message's list. +Its call sites are in `fetch.rs`, `publish_namespace.rs`, `publish.rs`, +`subscribe_namespace.rs`, `request_stream.rs`, `request.rs` and +`subscribe.rs`. Draft-14 already skips unknown parameters +(`Parameters::skip`). Accepting FORWARD=0 on a d16 SUBSCRIBE_NAMESPACE can +be accept-and-ignore: it only sets FORWARD on the PUBLISH messages it +triggers, and we send none for that subscription (Subscribe Options 0x01). + +Coordinate with [moq-transport request codes](/quest/m2/ietf-request-codes.md), +which accepts the delivery-timeout parameters on REQUEST_UPDATE; whichever +lands second drops the overlap. + +Tests: one codec test per gap the audit finds, plus one each for the d16 +ignore and d18 close rules on a known-but-misplaced parameter. Mirror in +`js/net/src/ietf` in the same PR. + +Public API: none. Wire: none new; decoding moves closer to the drafts. + +## Related + +- [Malformed moq-transport input](/quest/m2/ietf-malformed-close.md) - owns the other session-closing cases diff --git a/quest/m1/README.md b/quest/m1/README.md index 1ea8e567a7e..b773179e9d1 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -39,6 +39,7 @@ blocks. The quests that gated m0 lines moved under them. - [Cluster routing](/quest/m1/cluster-routing/README.md) - any node routes toward a broadcast's origin over CDN and P2P links alike, with per-origin routes and path-less announces - [One route cost](/quest/m1/route-cost.md) - Warm and Cold collapse to one static route cost - [Track tail interop](/quest/m1/track-tail-interop.md) - a Rust publisher ending a track with a group in flight is read to its end by the JS subscriber, and the reverse, in `just test interop` +- [moq-transport deviations](/quest/m1/ietf-deviations-doc.md) - `standard.md` lists the deviations Fastly's interop report flagged (one publisher per path, dropped object properties, OK before an old lite source answers) with their reasons - [Transcode fetch demand](/quest/m1/transcode-fetch-demand.md) - the transcode fetch handler drops a request once nobody wants the group - [Flat questlines](/quest/m1/quest-flat-lines.md) - moq pins the current quest CLI, lands its questline branches on main, and retires them - [moq-bot may push workflow changes](/quest/m1/bot-workflows-permission.md) - condition: the maintainer grants moq-bot's GitHub App the `workflows` permission, so back-merges carrying workflow changes go through @@ -52,6 +53,7 @@ blocks. The quests that gated m0 lines moved under them. - [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - every stream group in a lite subscription arrives or is dropped by name, and lite-07 drops its stream count for it - [#2991](/quest/m1/2991-net-coalesce-dynamic-tracks-and-preserve-sequences-across.md) - one dynamic producer per track name in both languages, with the sequence namespace surviving a replacement - [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - a lite-07 SUBSCRIBE asks for past and live ranges in either order and replaces FETCH; relays fill misses by range, including over moq-transport +- [Cold relay Largest](/quest/m1/ietf-cold-largest.md) - a relay with nothing cached reports its upstream's Largest, so a d14-19 joining FETCH through it gets the current group's head - [Live media time](/quest/m1/subscribe-live-time.md) - re-scoped against `set_live`: a lite-07 SUBSCRIBE_OK carries the publisher's current media time only if a reader still needs it - [Cross-relay bursts re-run](/quest/m1/cross-relay-bursts.md) - condition: the #4349 reporter re-runs their A/B/C comparison against current cdn.moq.pro - [Flapping peer drill](/quest/m1/cross-relay-flap-drill.md) - the cluster burst drill survives a flapping peer link, the regression for 0.15.6 route-flap drops diff --git a/quest/m1/ietf-cold-largest.md b/quest/m1/ietf-cold-largest.md new file mode 100644 index 00000000000..98c782285b7 --- /dev/null +++ b/quest/m1/ietf-cold-largest.md @@ -0,0 +1,43 @@ +# [M] A cold relay reports its upstream's Largest + +## Goal + +On drafts 14 to 19, a relay that has no cached object for a track +reports its upstream's Largest Location in its own SUBSCRIBE_OK. A relative +joining FETCH (start 0) through it then gets the head of the current group, +filled upstream if needed, instead of INVALID_RANGE. + +## Plan + +Triaged from Fastly's moq-relay-interop report (2026-09-23 run, item 14) +on 2026-10-07. The report's DOES_NOT_EXIST was fixed by #4253: the empty +join is INVALID_RANGE now, which the draft requires when no Largest was sent. +What remains is that the Largest is lost: + +- The publisher's SUBSCRIBE_OK carries `live_edge(&cache).largest` + (`rs/moq-net/src/ietf/publisher.rs`, around lines 664 and 717), which is + `None` until a group is cached, so `(Filter::NextObject, None)` becomes + `Joined::Empty`. +- The subscriber only calls `track.start_at(largest.group)` + (`rs/moq-net/src/ietf/subscriber.rs`, around line 2013) with its + upstream's Largest, and does not record the object. + +Decided 2026-10-07: carry the upstream's Largest into the model, so the +publisher can report it, and serve the joining FETCH through the existing +one-group upstream fill. Rejected: leaving INVALID_RANGE, which is compliant +but loses late joiners' first group head. This reports state learned from +the upstream; nothing waits on a peer. + +Scrutinize the model change: keep any new track state crate-private unless a +consumer needs it. + +Test: through a cold relay, a d16 subscriber with a relative joining FETCH +(start 0) receives the current group from object 0, and its SUBSCRIBE_OK +carries the upstream's Largest. + +Public API: none expected. Wire: none new. + +## Related + +- [moq-transport ranges](/quest/m1/subscribe-ranges/ietf.md) - the same upstream fill path, for multi-group ranges +- [Fetch without SUBSCRIBE](/quest/m1/ietf-fetch-only.md) - the other FETCH gap at a relay diff --git a/quest/m1/ietf-deviations-doc.md b/quest/m1/ietf-deviations-doc.md new file mode 100644 index 00000000000..91e1eaf0f00 --- /dev/null +++ b/quest/m1/ietf-deviations-doc.md @@ -0,0 +1,26 @@ +# [XS] Document the relay's moq-transport deviations + +## Goal + +`doc/concept/standard.md` lists, under moq-transport, the deliberate +deviations that Fastly's moq-relay-interop report flagged, each with its +reason and draft section, so the next interop run has something to point at: + +- One publisher per broadcast path: a SUBSCRIBE goes to one route, not to + every matching publisher (d16 §8.4), because a path names one content. +- Unknown object properties are dropped, not forwarded (d18 §2.5), because + the model carries only payload and timestamp. +- A source with no track stream (lite-01 to 04) behind the relay gets + SUBSCRIBE_OK before the source answers, so a missing track ends as + PUBLISH_DONE, not REQUEST_ERROR. + +## Plan + +Triaged on 2026-10-07. Decided then: these are the product's model (see +[A gapped IETF object ID](/quest/m1/ietf-object-gaps.md) and the AGENTS.md +rule that a name always means the same content), so they are documented, +not changed. Check the wording against `doc/concept/moq-lite.md`, which +already explains what moq-lite leaves out, and link to it rather than +repeat it. + +Public API: none. Wire: none. From ee71215ccc74dd117cd870dc63d4b83f9bd8028a Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 7 Oct 2026 11:57:48 -0700 Subject: [PATCH 2/9] quest: correct per-draft parameter rules and keep with_peer_hop Address review: d14/d15 ignore unknown parameters and d17 closes on misplaced ones; keep Client::with_peer_hop as an override of the random dial hop; retarget the standard.md note and the first-object check. Co-Authored-By: Claude Opus 5.5 --- quest/m0/dial-split-horizon.md | 9 ++++++--- quest/m0/ietf-first-object-zero.md | 8 +++++--- quest/m0/ietf-namespace-stream.md | 5 +++-- quest/m0/ietf-params-per-draft.md | 32 ++++++++++++++++++------------ 4 files changed, 33 insertions(+), 21 deletions(-) diff --git a/quest/m0/dial-split-horizon.md b/quest/m0/dial-split-horizon.md index eaf35097962..172fda2ace2 100644 --- a/quest/m0/dial-split-horizon.md +++ b/quest/m0/dial-split-horizon.md @@ -31,8 +31,10 @@ Facts from `main`: Decided 2026-10-07: fix it in moq-net's `Client`, not only the relay's dial. A session with no negotiated identity gets a random per-connection hop, the same way `server.rs` does. Rejected: calling `with_peer_hop` from -moq-relay only, which leaves other moq-net users anonymous. Check whether -`with_peer_hop` still has a caller afterwards, and delete it if not. +moq-relay only, which leaves other moq-net users anonymous. Keep +`with_peer_hop` as an explicit override of the random default, as the +server side does: a random hop changes on every redial, so only a pinned one +lets every session dialing the same relay resolve to one route. The same gap let a d16 moq-dev origin route its edges' SUBSCRIBEs to two of the edges (the report's T4 fanout-late-join, two runs). Check that this case @@ -40,8 +42,9 @@ is covered too. Test: a dialed anonymous ietf session that announces a namespace does not receive it back, and a SUBSCRIBE that arrives on it is not routed back to it. +Check that a redialed anonymous peer's routes do not pile up across hops. -Public API: possibly removes `Client::with_peer_hop`. Wire: none. +Public API: none. Wire: none. ## Related diff --git a/quest/m0/ietf-first-object-zero.md b/quest/m0/ietf-first-object-zero.md index beb6c674f29..701c72f4980 100644 --- a/quest/m0/ietf-first-object-zero.md +++ b/quest/m0/ietf-first-object-zero.md @@ -20,9 +20,11 @@ Decided 2026-10-07: accept it anyway, to be lenient toward publishers that are out of spec. Rejected: keeping the drop and documenting it. Where it lives: `rs/moq-net/src/ietf/subscriber.rs` (around line 2611) -drops the stream before reading any object. Peek the first object's ID -before deciding. `next_object_id` already enforces the ID sequence. Check -whether `js/net` drops these streams at all; it has no matching check today. +drops the stream before reading any object. `next_object_id` already +refuses a first ID other than the group's start (0 for a whole group), so +check whether letting a clear-bit stream through to it is enough before +adding a second peek. Check whether `js/net` drops these streams at all; it +has no matching check today. Test: a d18 stream with FIRST_OBJECT clear and first ID 0 delivers its group; one with first ID 3 is still dropped. diff --git a/quest/m0/ietf-namespace-stream.md b/quest/m0/ietf-namespace-stream.md index 5447e3a547b..d546189cd8d 100644 --- a/quest/m0/ietf-namespace-stream.md +++ b/quest/m0/ietf-namespace-stream.md @@ -32,8 +32,9 @@ Drafts 14 and 15 keep answering with PUBLISH_NAMESPACE requests, unchanged. `js/net` mirrors the solicit logic (`js/net/src/ietf/connection.ts`, `publisher.ts`). -Update `doc/concept/standard.md` (around lines 100-106 and 141-144), which -documents the empty stream. +Update `doc/concept/standard.md` where it describes solicit: a non-SOLICIT +d16+ peer now also gets NAMESPACE on its SUBSCRIBE_NAMESPACE stream, so it +hears each namespace twice. Test: a non-SOLICIT d16 and d18 peer's SUBSCRIBE_NAMESPACE receives NAMESPACE for an existing match and for one announced later, then diff --git a/quest/m0/ietf-params-per-draft.md b/quest/m0/ietf-params-per-draft.md index c535bdc3558..529d1433187 100644 --- a/quest/m0/ietf-params-per-draft.md +++ b/quest/m0/ietf-params-per-draft.md @@ -4,10 +4,10 @@ Every control message accepts exactly the parameters its draft allows, in `rs/moq-net` and `js/net`, for each supported draft (14 through 22). A -parameter the draft defines for that message is decoded or skipped; a -parameter defined only for another message is ignored on drafts 14 to 17 and -closes the session with PROTOCOL_VIOLATION on draft-18 and later; an unknown -key still closes it with PROTOCOL_VIOLATION. The case seen in the wild: +parameter the draft defines for that message is decoded or skipped. Drafts +14 and 15 ignore unknown and misplaced parameters alike; draft-16 closes the +session with PROTOCOL_VIOLATION on an unknown key but ignores a misplaced +one; draft-17 and later close it on both. The case seen in the wild: moqx's draft-16 SUBSCRIBE_NAMESPACE carries FORWARD (0x10), which we refuse, and moqx redials in a loop. @@ -16,10 +16,12 @@ and moqx redials in a loop. Triaged from Fastly's moq-relay-interop report (run of 2026-09-23, build 7ee2b02) on 2026-10-07. Facts from the draft text: -- Unknown message parameter: close with PROTOCOL_VIOLATION (d16 §9.2, d18 - §10.2, d22 §9.20). That rule stays. -- A known parameter in a message it is not defined for: d16 §9.2.2 says - "it MUST be ignored"; d18 §10.2.1 and d22 §9.20.1 close the session. +- Unknown message parameter: d14 and d15 §9.2 say "Receivers ignore + unrecognized parameters"; d16 §9.2, d17 §9.3, d18 §10.2 and d22 §9.20 + close with PROTOCOL_VIOLATION. +- A known parameter in a message it is not defined for: d14, d15 and d16 + (§9.2.2) say "it MUST be ignored"; d17 §9.3.1, d18 §10.2.1 and d22 + §9.20.1 close the session. - d16 allows FORWARD on SUBSCRIBE, REQUEST_UPDATE, PUBLISH, PUBLISH_OK and SUBSCRIBE_NAMESPACE. d18 drops it from SUBSCRIBE_NAMESPACE (moved to SUBSCRIBE_TRACKS). @@ -32,8 +34,11 @@ Where it lives: the `decode_params!` macro (`rs/moq-net/src/ietf/parameters.rs`, around line 473) returns `InvalidValue` for any key outside a message's list. Its call sites are in `fetch.rs`, `publish_namespace.rs`, `publish.rs`, `subscribe_namespace.rs`, `request_stream.rs`, `request.rs` and -`subscribe.rs`. Draft-14 already skips unknown parameters -(`Parameters::skip`). Accepting FORWARD=0 on a d16 SUBSCRIBE_NAMESPACE can +`subscribe.rs`. `Parameters::skip` ignores unknown keys, but only d14 +SUBSCRIBE, FETCH and PUBLISH use it; the legacy d14 to d17 decodes of +SUBSCRIBE_NAMESPACE, PUBLISH_NAMESPACE and the `request*.rs` messages go +through `decode_params!`, so they refuse unknown keys on d14 and d15 today +(`test_param_unknown_rejected` asserts that). Those are gaps. Accepting FORWARD=0 on a d16 SUBSCRIBE_NAMESPACE can be accept-and-ignore: it only sets FORWARD on the PUBLISH messages it triggers, and we send none for that subscription (Subscribe Options 0x01). @@ -41,9 +46,10 @@ Coordinate with [moq-transport request codes](/quest/m2/ietf-request-codes.md), which accepts the delivery-timeout parameters on REQUEST_UPDATE; whichever lands second drops the overlap. -Tests: one codec test per gap the audit finds, plus one each for the d16 -ignore and d18 close rules on a known-but-misplaced parameter. Mirror in -`js/net/src/ietf` in the same PR. +Tests: one codec test per gap the audit finds, plus one per rule above +(unknown and misplaced, on each side of the d16 and d17 boundaries). Mirror +in `js/net/src/ietf` in the same PR. `doc/concept/standard.md` says an +undefined parameter always closes the session; correct it. Public API: none. Wire: none new; decoding moves closer to the drafts. From f053b40ac451e7f5b0c8d33d06630a9a8fc8e6f3 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 7 Oct 2026 12:18:51 -0700 Subject: [PATCH 3/9] quest: reflow ietf-params-per-draft Co-Authored-By: Claude Opus 5.5 --- quest/m0/ietf-params-per-draft.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/quest/m0/ietf-params-per-draft.md b/quest/m0/ietf-params-per-draft.md index 529d1433187..fd81987dca6 100644 --- a/quest/m0/ietf-params-per-draft.md +++ b/quest/m0/ietf-params-per-draft.md @@ -38,9 +38,11 @@ Its call sites are in `fetch.rs`, `publish_namespace.rs`, `publish.rs`, SUBSCRIBE, FETCH and PUBLISH use it; the legacy d14 to d17 decodes of SUBSCRIBE_NAMESPACE, PUBLISH_NAMESPACE and the `request*.rs` messages go through `decode_params!`, so they refuse unknown keys on d14 and d15 today -(`test_param_unknown_rejected` asserts that). Those are gaps. Accepting FORWARD=0 on a d16 SUBSCRIBE_NAMESPACE can -be accept-and-ignore: it only sets FORWARD on the PUBLISH messages it -triggers, and we send none for that subscription (Subscribe Options 0x01). +(`test_param_unknown_rejected` asserts that). Those are gaps. + +Accepting FORWARD=0 on a d16 SUBSCRIBE_NAMESPACE can be accept-and-ignore: +it only sets FORWARD on the PUBLISH messages it triggers, and we send none +for that subscription (Subscribe Options 0x01). Coordinate with [moq-transport request codes](/quest/m2/ietf-request-codes.md), which accepts the delivery-timeout parameters on REQUEST_UPDATE; whichever From bd8e9a3b5c02cccfedad9cf0e33e653fc1f7ce36 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 7 Oct 2026 13:18:04 -0700 Subject: [PATCH 4/9] quest: honor Subscribe Options and report the max Largest Co-Authored-By: Claude Opus 5.5 --- quest/m0/ietf-namespace-stream.md | 29 +++++++++++++++++++++-------- quest/m1/ietf-cold-largest.md | 18 ++++++++++++------ 2 files changed, 33 insertions(+), 14 deletions(-) diff --git a/quest/m0/ietf-namespace-stream.md b/quest/m0/ietf-namespace-stream.md index d546189cd8d..cb7d6fc276e 100644 --- a/quest/m0/ietf-namespace-stream.md +++ b/quest/m0/ietf-namespace-stream.md @@ -2,11 +2,12 @@ ## Goal -On draft-16 and later, every SUBSCRIBE_NAMESPACE response stream carries -NAMESPACE and NAMESPACE_DONE for each matching namespace, whatever the -peer's SETUP options, in `rs/moq-net` and `js/net`. Today a peer that did not -send our SOLICIT option (0x40B5A), which is every other implementation, gets -an empty stream and only the unsolicited PUBLISH_NAMESPACE pushes. +On draft-16 and later, a SUBSCRIBE_NAMESPACE that asks for namespaces +gets NAMESPACE and NAMESPACE_DONE on its response stream for each matching +namespace, whatever the peer's SETUP options, in `rs/moq-net` and `js/net`. +Today a peer that did not send our SOLICIT option (0x40B5A), which is every +other implementation, gets an empty stream and only the unsolicited +PUBLISH_NAMESPACE pushes. ## Plan @@ -17,8 +18,19 @@ and §10.18 puts them on the response stream. d16 §9.25 says the same without the MUST. Both drafts let a publisher send PUBLISH_NAMESPACE to any subscriber, and neither says anything about duplicates. -Decided 2026-10-07: always fill the stream on d16+, and keep the unsolicited -pushes to non-SOLICIT peers, so those peers hear each namespace twice. +On d16 and d17 the message's Subscribe Options choose what the subscriber +wants: PUBLISH (0x00), NAMESPACE (0x01) or both (0x02) (d16 §9.25, d17 +§9.20). d18 moved track requests to SUBSCRIBE_TRACKS, so its +SUBSCRIBE_NAMESPACE always asks for namespaces. Both `rs/moq-net` and +`js/net` decode the field but drop it before dispatch, so the publisher +cannot tell the options apart today. + +Decided 2026-10-07: fill the stream on d16+ whenever NAMESPACE is +requested, and keep the unsolicited pushes to non-SOLICIT peers, so those +peers hear each namespace twice. Carry Subscribe Options through dispatch; +0x00 gets no NAMESPACE. We send no PUBLISH for a namespace subscription, so +decide whether a request for tracks (0x00, 0x02) is refused or answered +without them; refusing is the recommendation, since it fails loud. Rejected: stopping unsolicited pushes on d16+ (a peer that never subscribes would learn nothing), and filling only on d18+ (two behaviours for one message). @@ -38,6 +50,7 @@ hears each namespace twice. Test: a non-SOLICIT d16 and d18 peer's SUBSCRIBE_NAMESPACE receives NAMESPACE for an existing match and for one announced later, then -NAMESPACE_DONE when it ends. +NAMESPACE_DONE when it ends. On d16, options 0x00, 0x01 and 0x02 each get +the chosen behaviour, in both languages. Public API: none. Wire: moq-transport replies move closer to the drafts. diff --git a/quest/m1/ietf-cold-largest.md b/quest/m1/ietf-cold-largest.md index 98c782285b7..27ce72c7a42 100644 --- a/quest/m1/ietf-cold-largest.md +++ b/quest/m1/ietf-cold-largest.md @@ -2,9 +2,10 @@ ## Goal -On drafts 14 to 19, a relay that has no cached object for a track -reports its upstream's Largest Location in its own SUBSCRIBE_OK. A relative -joining FETCH (start 0) through it then gets the head of the current group, +On drafts 14 to 19, a relay's SUBSCRIBE_OK reports the larger of its +upstream's Largest Location and the largest object it has received for the +track, including when it has cached nothing yet. A relative joining FETCH +(start 0) through a cold relay then gets the head of the current group, filled upstream if needed, instead of INVALID_RANGE. ## Plan @@ -12,7 +13,10 @@ filled upstream if needed, instead of INVALID_RANGE. Triaged from Fastly's moq-relay-interop report (2026-09-23 run, item 14) on 2026-10-07. The report's DOES_NOT_EXIST was fixed by #4253: the empty join is INVALID_RANGE now, which the draft requires when no Largest was sent. -What remains is that the Largest is lost: +What remains is that the Largest is lost. d18 §10.2.11 and d19 §10.2.16 +say "A relay MUST set LARGEST_OBJECT to the largest of" the upstream's +LARGEST_OBJECT and the largest object received upstream, whatever the +cache holds; d14 to d17 state no rule, so follow the same one there. - The publisher's SUBSCRIBE_OK carries `live_edge(&cache).largest` (`rs/moq-net/src/ietf/publisher.rs`, around lines 664 and 717), which is @@ -23,7 +27,7 @@ What remains is that the Largest is lost: upstream's Largest, and does not record the object. Decided 2026-10-07: carry the upstream's Largest into the model, so the -publisher can report it, and serve the joining FETCH through the existing +publisher can report the maximum of it and the cached objects, and serve the joining FETCH through the existing one-group upstream fill. Rejected: leaving INVALID_RANGE, which is compliant but loses late joiners' first group head. This reports state learned from the upstream; nothing waits on a peer. @@ -33,7 +37,9 @@ consumer needs it. Test: through a cold relay, a d16 subscriber with a relative joining FETCH (start 0) receives the current group from object 0, and its SUBSCRIBE_OK -carries the upstream's Largest. +carries the upstream's Largest. On d18 and d19, a relay whose cached +objects are behind the upstream's Largest reports the upstream's, and one +whose cache is ahead reports its own. Public API: none expected. Wire: none new. From 4c737fbc2507554771c0bd96b94865a3ab550719 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 7 Oct 2026 14:46:10 -0700 Subject: [PATCH 5/9] quest(m0): End of Track keeps its Location Fastly's rerun shows a relay re-sends END_OF_TRACK at the next group's object 0 instead of the upstream's Location on the last group's stream. Co-Authored-By: Claude Opus 5.5 --- quest/m0/README.md | 8 +++-- quest/m0/ietf-end-of-track-location.md | 41 ++++++++++++++++++++++++++ 2 files changed, 46 insertions(+), 3 deletions(-) create mode 100644 quest/m0/ietf-end-of-track-location.md diff --git a/quest/m0/README.md b/quest/m0/README.md index 2c482f90ed6..dbf49ae114e 100644 --- a/quest/m0/README.md +++ b/quest/m0/README.md @@ -27,9 +27,10 @@ view (`origin::Consumer::local()`) to localhost workers belongs to moq.pro's edge, which embeds moq-relay; it moved there on 2026-09-28. Interop: Fastly's moq-relay-interop report (run of 2026-09-23, build -7ee2b02) was triaged against `main` on 2026-10-07. Its End of Track, SETUP, -UNSUBSCRIBE and error-code items were already fixed; the five fixes below -LOCATION_FILTER come from it and go ahead of Seattle. The cold-relay Largest +7ee2b02) was triaged against `main` on 2026-10-07. Its SETUP, UNSUBSCRIBE +and error-code items were already fixed. End of Track ends cleanly, but a +relay moves the marker's Location (Fastly's rerun on 2026-10-07). The six +fixes below LOCATION_FILTER come from it and go ahead of Seattle. The cold-relay Largest and the deviations it flagged are in m1. Identity: the [broadcast epoch](/quest/m0/broadcast-epoch/README.md) line @@ -53,6 +54,7 @@ a published `@moq/watch` break. - [NAMESPACE on the stream](/quest/m0/ietf-namespace-stream.md) - every draft-16+ SUBSCRIBE_NAMESPACE stream carries NAMESPACE for each match, whatever the peer's SETUP options - [Dialed split horizon](/quest/m0/dial-split-horizon.md) - a session moq-net dials gets a per-connection hop, so routes never echo back to the peer they came from - [No empty d14 prefix](/quest/m0/ietf-d14-root-prefix.md) - draft-14 never sends SUBSCRIBE_NAMESPACE with an empty prefix +- [End of Track Location](/quest/m0/ietf-end-of-track-location.md) - a relay forwards END_OF_TRACK at its upstream's Location, on the last group's stream, instead of re-sending it at the next group's object 0 - [Subgroup at object 0](/quest/m0/ietf-first-object-zero.md) - a draft-18 subgroup with FIRST_OBJECT clear that starts at object 0 is read whole - [Request caps](/quest/m0/request-caps.md) - lite message sizes, IETF request IDs, and per-session announces and subscriptions are bounded - [Prefix route fronts](/quest/m0/prefix-route-fronts.md) - a prefix route cannot be made to mint one front per requested path diff --git a/quest/m0/ietf-end-of-track-location.md b/quest/m0/ietf-end-of-track-location.md new file mode 100644 index 00000000000..71d20518f3b --- /dev/null +++ b/quest/m0/ietf-end-of-track-location.md @@ -0,0 +1,41 @@ +# [S] End of Track keeps its Location + +## Goal + +A relay forwards END_OF_TRACK at the Location its upstream sent, in +`rs/moq-net` and `js/net`. An upstream that ends the track at 5/5 on group +5's stream yields 5/5 on group 5's stream downstream, not 6/0 on a new +stream. A track whose last group stream has already closed still gets the +standalone marker at the next group's object 0, as today. + +## Plan + +Triaged from Fastly's follow-up on #5020 (runs of 2026-10-07 on `0f310a5`, +d18 and d21, cells `status-end-of-track` and `publish-done`). PUBLISH_DONE +was already right; only the marker moved. 6/0 is still true under d18's +"equal to or greater than" wording, but it drops the fact that group 5 +ended after object 4, which a subscriber cannot learn any other way on +d16+ (END_OF_GROUP is gone). + +Root cause, both sides: + +- Ingress: `rs/moq-net/src/ietf/subscriber.rs` (`Ended::Track`, around line + 2688) finishes the group producer, then calls `end_track(group + 1)`. An + egress reader that sees the group finish cannot yet know the track ends + after it. Set the end first. +- Egress: `write_end_of_track` in `rs/moq-net/src/ietf/publisher.rs` always + opens its own stream at `end`/0. When a group stream finishes and the + track's `final_sequence()` is that group + 1, append END_OF_TRACK at + (group, next object ID) before FIN, and skip the standalone marker for + that subscription. + +`js/net` mirrors both (`#runEndOfTrack` in `js/net/src/ietf/publisher.ts`); +check its ingress order too. + +Test: a d18 subscriber stream that ends 5/0..5/4 then END_OF_TRACK at 5/5, +forwarded through the ietf publisher, reaches the downstream as 5/5 on the +group 5 stream, with no extra stream and a PUBLISH_DONE count to match. +A track finished after its last group closed still gets 6/0. + +Public API: none. Wire: none (the marker's Location changes, its format +does not). From 761a2124328471870c98b9c97ab69a5441b7216c Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 7 Oct 2026 15:42:19 -0700 Subject: [PATCH 6/9] quest(m0): End of Group status, narrow d14 prefix, guard capped End of Track Fastly's pcap runs show moxygen never pushes PUBLISH_NAMESPACE on d14 unasked, so the d14 quest documents the gap instead of relying on it. imquic's End of Group status on a marked stream gets its own quest, and the End of Track append skips streams the subscription's end caps. Co-Authored-By: Claude Opus 5.5 --- quest/m0/README.md | 12 ++++---- quest/m0/ietf-d14-root-prefix.md | 21 +++++++++---- quest/m0/ietf-end-of-group-status.md | 25 ++++++++++++++++ quest/m0/ietf-end-of-track-location.md | 41 ++++++++++++++++---------- 4 files changed, 74 insertions(+), 25 deletions(-) create mode 100644 quest/m0/ietf-end-of-group-status.md diff --git a/quest/m0/README.md b/quest/m0/README.md index dbf49ae114e..eaba0ebb218 100644 --- a/quest/m0/README.md +++ b/quest/m0/README.md @@ -28,10 +28,11 @@ edge, which embeds moq-relay; it moved there on 2026-09-28. Interop: Fastly's moq-relay-interop report (run of 2026-09-23, build 7ee2b02) was triaged against `main` on 2026-10-07. Its SETUP, UNSUBSCRIBE -and error-code items were already fixed. End of Track ends cleanly, but a -relay moves the marker's Location (Fastly's rerun on 2026-10-07). The six -fixes below LOCATION_FILTER come from it and go ahead of Seattle. The cold-relay Largest -and the deviations it flagged are in m1. +and error-code items were already fixed. Fastly's reruns that day found +that a relay moves End of Track's Location and refuses imquic's End of +Group status. The seven fixes below LOCATION_FILTER come from them and go +ahead of Seattle. The cold-relay Largest and the deviations it flagged are +in m1. Identity: the [broadcast epoch](/quest/m0/broadcast-epoch/README.md) line gates the next release (decided 2026-10-03: @@ -53,8 +54,9 @@ a published `@moq/watch` break. - [moq-transport parameters per draft](/quest/m0/ietf-params-per-draft.md) - every control message accepts exactly the parameters its draft allows, so moqx's draft-16 SUBSCRIBE_NAMESPACE with FORWARD stops failing - [NAMESPACE on the stream](/quest/m0/ietf-namespace-stream.md) - every draft-16+ SUBSCRIBE_NAMESPACE stream carries NAMESPACE for each match, whatever the peer's SETUP options - [Dialed split horizon](/quest/m0/dial-split-horizon.md) - a session moq-net dials gets a per-connection hop, so routes never echo back to the peer they came from -- [No empty d14 prefix](/quest/m0/ietf-d14-root-prefix.md) - draft-14 never sends SUBSCRIBE_NAMESPACE with an empty prefix +- [No empty d14 prefix](/quest/m0/ietf-d14-root-prefix.md) - draft-14 never sends SUBSCRIBE_NAMESPACE with an empty prefix, and the docs say to scope a d14 link to moxygen - [End of Track Location](/quest/m0/ietf-end-of-track-location.md) - a relay forwards END_OF_TRACK at its upstream's Location, on the last group's stream, instead of re-sending it at the next group's object 0 +- [End of Group status](/quest/m0/ietf-end-of-group-status.md) - an End of Group status on a stream whose header already marks the group's end is accepted, so imquic's last object per group arrives - [Subgroup at object 0](/quest/m0/ietf-first-object-zero.md) - a draft-18 subgroup with FIRST_OBJECT clear that starts at object 0 is read whole - [Request caps](/quest/m0/request-caps.md) - lite message sizes, IETF request IDs, and per-session announces and subscriptions are bounded - [Prefix route fronts](/quest/m0/prefix-route-fronts.md) - a prefix route cannot be made to mint one front per requested path diff --git a/quest/m0/ietf-d14-root-prefix.md b/quest/m0/ietf-d14-root-prefix.md index 1bd1c26d945..3c3fc0866d2 100644 --- a/quest/m0/ietf-d14-root-prefix.md +++ b/quest/m0/ietf-d14-root-prefix.md @@ -3,8 +3,10 @@ ## Goal On draft-14, moq-net never sends SUBSCRIBE_NAMESPACE with an empty prefix. -d14 §9.28 makes a prefix with N = 0 a PROTOCOL_VIOLATION, and moxygen and -moqx refuse it today when an unscoped moq-relay cluster link dials them. +d14 §9.28 makes a prefix with N = 0 a PROTOCOL_VIOLATION, and moxygen +refuses it today when an unscoped moq-relay cluster link dials it. The +relay docs say an unscoped d14 link discovers nothing from moxygen, so +operators scope it. ## Plan @@ -12,15 +14,24 @@ Triaged from Fastly's moq-relay-interop report (2026-09-23 run, item 2) on 2026-10-07. Decided 2026-10-07: on d14, skip a SUBSCRIBE_NAMESPACE whose prefix is -empty, and rely on the peer's unsolicited PUBLISH_NAMESPACE, which d14 §6.2 -allows. Keep today's warn-and-continue when the peer refuses a namespace +empty. Keep today's warn-and-continue when the peer refuses a namespace subscription. Rejected: failing the link on refusal, and requiring a scope on d14 links at startup. +Revised 2026-10-07 after Fastly's pcap runs (#issuecomment-6046932019): +moxygen sends no PUBLISH_NAMESPACE on d14 to a peer that has not +subscribed, and moqx no longer speaks d14. So skipping alone leaves an +unscoped moq-dev downstream of moxygen with no namespaces. Decided to keep +the quest narrow and document the gap, since moxygen on d14 is the only +affected pair and scoped links already work. Rejected: subscribing per +prefix on demand when a local subscriber misses, and routing unknown +SUBSCRIBEs upstream; both need a hook on origin misses. + Where it lives: `subscribe_prefixes` (`rs/moq-net/src/ietf/subscriber.rs`, around line 600) returns `interest_prefixes(origin.allowed())`, which is `[""]` for an unscoped origin. `run_subscribe_namespace` (around line 861) -sends it. Log at debug when the root is skipped. +sends it. Log at debug when the root is skipped. The note belongs in +`doc/bin/relay/cluster.md`. Test: a d14 session with an unscoped origin sends no SUBSCRIBE_NAMESPACE, and a scoped one still does. Check `js/net` for the same path. diff --git a/quest/m0/ietf-end-of-group-status.md b/quest/m0/ietf-end-of-group-status.md new file mode 100644 index 00000000000..898a575eef2 --- /dev/null +++ b/quest/m0/ietf-end-of-group-status.md @@ -0,0 +1,25 @@ +# [XS] End of Group status on a marked stream + +## Goal + +`rs/moq-net` and `js/net` accept an End of Group status object (0x3) on a +subgroup stream whose header sets the END_OF_GROUP bit, and end the group +there, instead of refusing the stream. imquic sends both on d16, d18 and +d21, and moq-dev loses each group's last object today. + +## Plan + +Triaged from Fastly's follow-up on #5020 (#issuecomment-6046932019, runs of +2026-10-07 on `0f310a5`). d16 §10.4.2 says the bit only lets a subscriber +infer the group's end at FIN, and nothing forbids sending the status too. + +Where it lives: `IngestPhase::Status` in `rs/moq-net/src/ietf/subscriber.rs` +(around line 3992) accepts 0x3 only when `!self.has_end` and otherwise +returns `Unsupported`, which refetches the rest of the group upstream and +resets the downstream stream. Accept it either way, as END_OF_TRACK +already is. Check the matching status parse in `js/net/src/ietf/object.ts`. + +Test: a d18 stream with the END_OF_GROUP bit, objects 0..4, then a 0x3 +status at 5 delivers a finished group with five frames. + +Public API: none. Wire: none. diff --git a/quest/m0/ietf-end-of-track-location.md b/quest/m0/ietf-end-of-track-location.md index 71d20518f3b..02ada23445a 100644 --- a/quest/m0/ietf-end-of-track-location.md +++ b/quest/m0/ietf-end-of-track-location.md @@ -12,30 +12,41 @@ standalone marker at the next group's object 0, as today. Triaged from Fastly's follow-up on #5020 (runs of 2026-10-07 on `0f310a5`, d18 and d21, cells `status-end-of-track` and `publish-done`). PUBLISH_DONE -was already right; only the marker moved. 6/0 is still true under d18's -"equal to or greater than" wording, but it drops the fact that group 5 -ended after object 4, which a subscriber cannot learn any other way on -d16+ (END_OF_GROUP is gone). +was already right; only the marker moved. A subscriber still learns that +group 5 ended after object 4 from the subgroup header's END_OF_GROUP bit, +so the reasons are conformance with the upstream's Location and one stream +fewer. Root cause, both sides: - Ingress: `rs/moq-net/src/ietf/subscriber.rs` (`Ended::Track`, around line 2688) finishes the group producer, then calls `end_track(group + 1)`. An egress reader that sees the group finish cannot yet know the track ends - after it. Set the end first. + after it. Set the end first, and abort the group producer if `end_track` + returns `ProtocolViolation`, so it is never left unfinished. - Egress: `write_end_of_track` in `rs/moq-net/src/ietf/publisher.rs` always - opens its own stream at `end`/0. When a group stream finishes and the - track's `final_sequence()` is that group + 1, append END_OF_TRACK at - (group, next object ID) before FIN, and skip the standalone marker for - that subscription. - -`js/net` mirrors both (`#runEndOfTrack` in `js/net/src/ietf/publisher.ts`); -check its ingress order too. + opens its own stream at `end`/0. When a group stream reaches the group's + real end (the slice has no `until`, and the group finished rather than + aborted or expired) and the track's `final_sequence()` is that group + 1, + append END_OF_TRACK at (group, next object ID) before FIN, and skip the + standalone marker for that subscription. A stream capped by the + subscription's end Location (`GroupSlice::until`) keeps the standalone + marker, since appending would deny objects past the cap that do exist. + +The capped stream has a related false claim on `main`: its header still +sets the END_OF_GROUP bit (`has_end` defaults to true). Clear it when +`until` cuts the group short. + +`js/net` mirrors both: `#runEndOfTrack` in `js/net/src/ietf/publisher.ts` +runs after every group, and its ingress closes the group before +`track.finishAt`. Test: a d18 subscriber stream that ends 5/0..5/4 then END_OF_TRACK at 5/5, forwarded through the ietf publisher, reaches the downstream as 5/5 on the group 5 stream, with no extra stream and a PUBLISH_DONE count to match. -A track finished after its last group closed still gets 6/0. +A track finished after its last group closed still gets 6/0. A +subscription whose end Location falls inside the last group gets 6/0 and a +header without END_OF_GROUP. -Public API: none. Wire: none (the marker's Location changes, its format -does not). +Public API: none. Wire: none (the marker's Location and a capped stream's +header bit change, their formats do not). From 6612e96e24d4559bd4d83ef035008711cd325a65 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 7 Oct 2026 16:20:18 -0700 Subject: [PATCH 7/9] quest: keep a Largest high-watermark past eviction, answer 0x02 with namespaces, scope group-status recovery --- quest/m0/ietf-end-of-group-status.md | 7 ++++--- quest/m0/ietf-namespace-stream.md | 10 ++++++---- quest/m1/README.md | 2 +- quest/m1/ietf-cold-largest.md | 14 ++++++++++---- 4 files changed, 21 insertions(+), 12 deletions(-) diff --git a/quest/m0/ietf-end-of-group-status.md b/quest/m0/ietf-end-of-group-status.md index 898a575eef2..d3e79786d1a 100644 --- a/quest/m0/ietf-end-of-group-status.md +++ b/quest/m0/ietf-end-of-group-status.md @@ -15,9 +15,10 @@ infer the group's end at FIN, and nothing forbids sending the status too. Where it lives: `IngestPhase::Status` in `rs/moq-net/src/ietf/subscriber.rs` (around line 3992) accepts 0x3 only when `!self.has_end` and otherwise -returns `Unsupported`, which refetches the rest of the group upstream and -resets the downstream stream. Accept it either way, as END_OF_TRACK -already is. Check the matching status parse in `js/net/src/ietf/object.ts`. +returns `Unsupported`, which aborts the local group producer (`recv_group` +then returns `Ok`, so the session is not stopped). A downstream reader only +gets the rest of the group if the model's fetch path recovers it. Accept the +status either way, as END_OF_TRACK already is. Check the matching status parse in `js/net/src/ietf/object.ts`. Test: a d18 stream with the END_OF_GROUP bit, objects 0..4, then a 0x3 status at 5 delivers a finished group with five frames. diff --git a/quest/m0/ietf-namespace-stream.md b/quest/m0/ietf-namespace-stream.md index cb7d6fc276e..020c9ed928e 100644 --- a/quest/m0/ietf-namespace-stream.md +++ b/quest/m0/ietf-namespace-stream.md @@ -27,10 +27,12 @@ cannot tell the options apart today. Decided 2026-10-07: fill the stream on d16+ whenever NAMESPACE is requested, and keep the unsolicited pushes to non-SOLICIT peers, so those -peers hear each namespace twice. Carry Subscribe Options through dispatch; -0x00 gets no NAMESPACE. We send no PUBLISH for a namespace subscription, so -decide whether a request for tracks (0x00, 0x02) is refused or answered -without them; refusing is the recommendation, since it fails loud. +peers hear each namespace twice. Carry Subscribe Options through dispatch: +0x01 and 0x02 both ask for namespaces, so both get NAMESPACE; 0x00 gets none. +We send no PUBLISH for a namespace subscription, so decide how the track half +is answered. For 0x00 (tracks only), refusing is the recommendation, since it +fails loud. For 0x02, the recommendation is to answer with namespaces only, +since refusing would drop the NAMESPACE it asked for. Rejected: stopping unsolicited pushes on d16+ (a peer that never subscribes would learn nothing), and filling only on d18+ (two behaviours for one message). diff --git a/quest/m1/README.md b/quest/m1/README.md index 52faf1bca8a..77645af0002 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -53,7 +53,7 @@ blocks. The quests that gated m0 lines moved under them. - [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - every stream group in a lite subscription arrives or is dropped by name, and lite-07 drops its stream count for it - [#2991](/quest/m1/2991-net-coalesce-dynamic-tracks-and-preserve-sequences-across.md) - one dynamic producer per track name in both languages, with the sequence namespace surviving a replacement - [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - a lite-07 SUBSCRIBE asks for past and live ranges in either order and replaces FETCH; relays fill misses by range, including over moq-transport -- [Cold relay Largest](/quest/m1/ietf-cold-largest.md) - a relay with nothing cached reports its upstream's Largest, so a d14-19 joining FETCH through it gets the current group's head +- [Cold relay Largest](/quest/m1/ietf-cold-largest.md) - a relay reports the larger of its upstream's Largest and the largest object it received, even with nothing cached, so a d14-19 joining FETCH through it gets the current group's head - [Live media time](/quest/m1/subscribe-live-time.md) - re-scoped against `set_live`: a lite-07 SUBSCRIBE_OK carries the publisher's current media time only if a reader still needs it - [Cross-relay bursts re-run](/quest/m1/cross-relay-bursts.md) - condition: the #4349 reporter re-runs their A/B/C comparison against current cdn.moq.pro - [Flapping peer drill](/quest/m1/cross-relay-flap-drill.md) - the cluster burst drill survives a flapping peer link, the regression for 0.15.6 route-flap drops diff --git a/quest/m1/ietf-cold-largest.md b/quest/m1/ietf-cold-largest.md index 27ce72c7a42..6795b11a926 100644 --- a/quest/m1/ietf-cold-largest.md +++ b/quest/m1/ietf-cold-largest.md @@ -26,9 +26,14 @@ cache holds; d14 to d17 state no rule, so follow the same one there. (`rs/moq-net/src/ietf/subscriber.rs`, around line 2013) with its upstream's Largest, and does not record the object. -Decided 2026-10-07: carry the upstream's Largest into the model, so the -publisher can report the maximum of it and the cached objects, and serve the joining FETCH through the existing -one-group upstream fill. Rejected: leaving INVALID_RANGE, which is compliant +- The model's `largest()` derives from the newest cached group, so it + drops back once that group is evicted, and the upstream's Largest is kept + only as `live_floor` (`rs/moq-net/src/model/track.rs`). + +Decided 2026-10-07: carry the upstream's Largest into the model, and keep a +high-watermark of the largest object received that cache eviction does not +lower, so the publisher reports the maximum of the two. Serve the joining +FETCH through the existing one-group upstream fill. Rejected: leaving INVALID_RANGE, which is compliant but loses late joiners' first group head. This reports state learned from the upstream; nothing waits on a peer. @@ -39,7 +44,8 @@ Test: through a cold relay, a d16 subscriber with a relative joining FETCH (start 0) receives the current group from object 0, and its SUBSCRIBE_OK carries the upstream's Largest. On d18 and d19, a relay whose cached objects are behind the upstream's Largest reports the upstream's, and one -whose cache is ahead reports its own. +whose cache is ahead reports its own, and still reports it after that +group is evicted. Public API: none expected. Wire: none new. From 7440d3a3ee0c14442c58519736d50b45e7b9adbd Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 7 Oct 2026 19:06:44 -0700 Subject: [PATCH 8/9] quest: ground cold-largest's high-watermark in a demoted group; refuse unknown Subscribe Options --- quest/m0/ietf-namespace-stream.md | 7 ++++--- quest/m1/ietf-cold-largest.md | 33 ++++++++++++++++++------------- 2 files changed, 23 insertions(+), 17 deletions(-) diff --git a/quest/m0/ietf-namespace-stream.md b/quest/m0/ietf-namespace-stream.md index 020c9ed928e..93ad66c5789 100644 --- a/quest/m0/ietf-namespace-stream.md +++ b/quest/m0/ietf-namespace-stream.md @@ -32,7 +32,8 @@ peers hear each namespace twice. Carry Subscribe Options through dispatch: We send no PUBLISH for a namespace subscription, so decide how the track half is answered. For 0x00 (tracks only), refusing is the recommendation, since it fails loud. For 0x02, the recommendation is to answer with namespaces only, -since refusing would drop the NAMESPACE it asked for. +since refusing would drop the NAMESPACE it asked for. A value above 0x02 is +malformed and refused. Rejected: stopping unsolicited pushes on d16+ (a peer that never subscribes would learn nothing), and filling only on d18+ (two behaviours for one message). @@ -52,7 +53,7 @@ hears each namespace twice. Test: a non-SOLICIT d16 and d18 peer's SUBSCRIBE_NAMESPACE receives NAMESPACE for an existing match and for one announced later, then -NAMESPACE_DONE when it ends. On d16, options 0x00, 0x01 and 0x02 each get -the chosen behaviour, in both languages. +NAMESPACE_DONE when it ends. On d16, options 0x00, 0x01, 0x02 and 0x03 each +get the chosen behaviour, in both languages. Public API: none. Wire: moq-transport replies move closer to the drafts. diff --git a/quest/m1/ietf-cold-largest.md b/quest/m1/ietf-cold-largest.md index 6795b11a926..700aca43a80 100644 --- a/quest/m1/ietf-cold-largest.md +++ b/quest/m1/ietf-cold-largest.md @@ -22,20 +22,24 @@ cache holds; d14 to d17 state no rule, so follow the same one there. (`rs/moq-net/src/ietf/publisher.rs`, around lines 664 and 717), which is `None` until a group is cached, so `(Filter::NextObject, None)` becomes `Joined::Empty`. -- The subscriber only calls `track.start_at(largest.group)` - (`rs/moq-net/src/ietf/subscriber.rs`, around line 2013) with its - upstream's Largest, and does not record the object. - -- The model's `largest()` derives from the newest cached group, so it - drops back once that group is evicted, and the upstream's Largest is kept - only as `live_floor` (`rs/moq-net/src/model/track.rs`). +- The subscriber passes its upstream's Largest only to + `track.start_at(largest.group)` and its held subscription state + (`rs/moq-net/src/ietf/subscriber.rs`, around line 2013), and does not + record the object. `live_floor` (`model/track.rs`) is a cache-validity + group floor, not the upstream's Largest. +- The latest group is protected from eviction while the track has a + producer, but `live_edge` falls back through the cache when the newest + group has no object yet. An empty newer group demotes the last + object-bearing group, which can then expire, losing its Location. Decided 2026-10-07: carry the upstream's Largest into the model, and keep a -high-watermark of the largest object received that cache eviction does not -lower, so the publisher reports the maximum of the two. Serve the joining -FETCH through the existing one-group upstream fill. Rejected: leaving INVALID_RANGE, which is compliant -but loses late joiners' first group head. This reports state learned from -the upstream; nothing waits on a peer. +high-watermark of the largest object received that expiry does not lower, +so the publisher reports the maximum of the two. Serve the joining FETCH +through the existing one-group upstream fill. Rejected: leaving +INVALID_RANGE, which is compliant but loses late joiners' first group head. +This reports state learned from the upstream; nothing waits on a peer. The +same Largest goes out wherever the draft asks for it (SUBSCRIBE_OK, and on +d18+ an inbound PUBLISH and REQUEST_UPDATE_OK). Scrutinize the model change: keep any new track state crate-private unless a consumer needs it. @@ -44,8 +48,9 @@ Test: through a cold relay, a d16 subscriber with a relative joining FETCH (start 0) receives the current group from object 0, and its SUBSCRIBE_OK carries the upstream's Largest. On d18 and d19, a relay whose cached objects are behind the upstream's Largest reports the upstream's, and one -whose cache is ahead reports its own, and still reports it after that -group is evicted. +whose cache is ahead reports its own. Regression for the high-watermark: +create an empty newer group, expire the older object-bearing group it +demoted, and SUBSCRIBE_OK still reports that group's received Location. Public API: none expected. Wire: none new. From 82cfcbbd5ccdce7393c0fdff4bc2f0ec7df411ba Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 7 Oct 2026 19:11:20 -0700 Subject: [PATCH 9/9] quest(cold-largest): report Largest only in responses the relay sends --- quest/m1/ietf-cold-largest.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/quest/m1/ietf-cold-largest.md b/quest/m1/ietf-cold-largest.md index 700aca43a80..0b84d0bca42 100644 --- a/quest/m1/ietf-cold-largest.md +++ b/quest/m1/ietf-cold-largest.md @@ -38,8 +38,9 @@ so the publisher reports the maximum of the two. Serve the joining FETCH through the existing one-group upstream fill. Rejected: leaving INVALID_RANGE, which is compliant but loses late joiners' first group head. This reports state learned from the upstream; nothing waits on a peer. The -same Largest goes out wherever the draft asks for it (SUBSCRIBE_OK, and on -d18+ an inbound PUBLISH and REQUEST_UPDATE_OK). +same Largest goes out in every response this relay sends that carries one: +SUBSCRIBE_OK and the track-update REQUEST_OK. Inbound PUBLISH is refused +today (`run_publish_stream`), so it is out of scope. Scrutinize the model change: keep any new track state crate-private unless a consumer needs it.