From a443a522b32005c0abb968b5912bb15156e695ef Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 23:27:21 -0700 Subject: [PATCH 01/11] chore(quest): claim quest/m1/start-budget-update.md Quest-Claim: 93b6bff0b2f8c121 From 4b2c41d044d3622672161a8d36324bb152d69e76 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 23:31:45 -0700 Subject: [PATCH 02/11] chore(quest): record START budget ordering blocker Co-Authored-By: GPT-6.1 Sol --- quest/m1/README.md | 2 +- quest/m1/start-budget-update.md | 116 ++++++++++++++++++++------------ 2 files changed, 73 insertions(+), 45 deletions(-) diff --git a/quest/m1/README.md b/quest/m1/README.md index f9ea35b1639..f2db9ef1fdc 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -139,7 +139,7 @@ removed). - [Announcement takeover](/quest/m1/announce-takeover.md) - a new independent route explicitly continues instances served through its predecessor, with shared Rust/JS semantics and lite-07 wire - [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - a lite-07 SUBSCRIBE asks for past and live ranges with absolute or relative ends, in either order, and replaces FETCH; merged ranges never starve a subscriber; relays fill misses by range, including over moq-transport - [Drop set](/quest/m1/drop-set.md) - each cursor judges missing groups against its own budget, so a narrow reader drops promptly and a widening re-offers groups already reported dropped on lite-07 -- [A tightened SUBSCRIBE keeps the start's budget honest](/quest/m1/start-budget-update.md) - a relay copy records the delay its start was resolved under, so a SUBSCRIBE_UPDATE tightening cannot make later readers skip fillable history +- [Bind a resolved start to its subscriber budget](/quest/m1/start-budget-update.md) - decide an unpublished-wire budget binding so updates cannot make later readers skip fillable history; lite-06 cannot prove the applied budget - [Aborted copies don't hold an end open](/quest/m1/settled-abort.md) - a track's declared end settles when a fetched copy finishes a group whose live copy was cut or evicted - [Broadcast archives](/quest/m1/archive/README.md) - ANNOUNCE, TRACKS, and a SUBSCRIBE per track record every track under a prefix to any object_store, a host serves a stored broadcast back, and the catalog drops its archive pointer for a per-rendition `hls` field - [HLS recorder target](/quest/m1/hls-record-target.md) - the moq-cli HLS recorder takes its target duration from the declared `hls` duration diff --git a/quest/m1/start-budget-update.md b/quest/m1/start-budget-update.md index 04ecc8aafae..24cc599c468 100644 --- a/quest/m1/start-budget-update.md +++ b/quest/m1/start-budget-update.md @@ -1,54 +1,82 @@ -# [S] A tightened SUBSCRIBE keeps the start's budget honest +# [M] Bind a resolved start to its subscriber budget ## Goal -A relay copy remembers the `max_delay` that was in effect when its -publisher resolved SUBSCRIBE_START, not only the one sent at SUBSCRIBE. If a -SUBSCRIBE_UPDATE tightens the delay before the start is picked, a later -reader whose delay is looser than the tightened one, but no looser than the -original, still fills the history between the two starts. Released wires are -unchanged. +A relay copy knows the subscriber `max_delay` used to resolve its upstream +start, so a later reader with a looser budget fills the history it can use. +The response proves which budget was applied even when SUBSCRIBE_UPDATE +crosses it in flight. Published wires remain unchanged; this requires an +unpublished wire change rather than the originally planned lite-06 fix. ## Plan -Decided 2026-10-29 (maintainer, `/quest-plan` of moq.pro's announce ramp -loss follow-ups): its own quest in m1, rather than waiting on -[Drop set](/quest/m1/drop-set.md) or -[Delete the old start bookkeeping](/quest/m1/subscribe-ranges/start-bookkeeping.md), -since it fixes released lite-06 behavior. - -Found by review of [#7168](https://github.com/moq-dev/moq/pull/7168): the -lite subscriber sets `requested_max_delay` once, at SUBSCRIBE -(`rs/moq-net/src/lite/subscriber.rs`). `Misses` then gives up every group -below the start for any reader whose budget is no looser than that value. -After a tightening update, the start was resolved under the tighter budget, -so the recorded looser one makes some readers skip history they could have -filled. It loses history only in that window; it does not bring back #7168's -over-fetch. - -Record the budget the start was actually resolved under. SUBSCRIBE_START -carries a group, not which update it was picked under, and updates can cross -it in flight, so the budget current when START arrives is not enough: an -update sent after the publisher picked the start would be recorded in its -place, too loose after a widening or too tight after a tightening. When the -subscriber cannot tell which of its sent budgets the publisher applied, -record the loosest of them. That keeps today's history loss in the -ambiguous window but never brings back the over-fetch. - -Add integration tests in `rs/moq-net/tests/integration`, direct and through -a relay: subscribe, tighten before the start, then a second reader widens -partly, which fails on `main` and passes with the fix; and a tightening and -a widening that each cross an already-picked START, where the recorded -budget must never be tighter than the one the start was picked under. - -Open question for whoever starts this: which lite-06 ordering, if any, tells -the subscriber the publisher applied a tightening before picking the start. -Without one, every tightening sent before START arrives stays ambiguous, the -loosest-budget rule records the SUBSCRIBE budget as today, and the Goal is -out of reach on the released wire. Then propose rewriting this quest to a -wire change for a later draft, or deleting it. +Scope decision pending: the original released-wire Goal is unattainable. +Recommend keeping this quest for an explicit budget binding on lite-07, +coordinated with +[Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) and +[Delete the old start bookkeeping](/quest/m1/subscribe-ranges/start-bookkeeping.md). +The alternative is folding this work into that bookkeeping quest or +abandoning it and accepting the released-wire limitation. No wire design has +been approved by this investigation. + +### Ordering evidence + +Found by review of [moq-dev/moq#7168](https://github.com/moq-dev/moq/pull/7168): +the lite subscriber fixes `requested_max_delay` at SUBSCRIBE and passes it to +`start_resolved`. If the publisher applies a tightening before resolving the +start, a later reader between the two budgets gives up fillable history. + +There is no lite-06 ordering that identifies the applied update: + +- [`SubscribeStart`](https://github.com/moq-dev/moq/blob/0f68c6f9564e4dbfcd5b1f457eb3b2fdb420faa0/rs/moq-net/src/lite/subscribe.rs#L506) + encodes only `group` on lite-06. `largest` is lite-07-only and carries no + budget or request revision either. The + [TypeScript codec](https://github.com/moq-dev/moq/blob/0f68c6f9564e4dbfcd5b1f457eb3b2fdb420faa0/js/net/src/lite/subscribe.ts#L611) + has the same layout. +- [`SubscribeUpdate`](https://github.com/moq-dev/moq/blob/0f68c6f9564e4dbfcd5b1f457eb3b2fdb420faa0/rs/moq-net/src/lite/subscribe.rs#L581) + carries replacement preferences, with no acknowledgment or revision. + [The publisher applies them](https://github.com/moq-dev/moq/blob/0f68c6f9564e4dbfcd5b1f457eb3b2fdb420faa0/rs/moq-net/src/lite/publisher.rs#L3904) + without sending a fresh START. +- The [release-branch draft](https://github.com/moq-dev/moq/blob/8677afaa6bc1420a235a249253dd56d09a378d2b/drafts/draft-lcurley-moq-lite.md#L1135) + calls the same response SUBSCRIBE_OK and gives it only `Group`. + Updates travel subscriber to publisher; the response travels publisher to + subscriber. Ordering within each direction does not order the two directions. + +For example, send SUBSCRIBE with 30 seconds, then UPDATE with 5 seconds, +then receive START at group 100. The publisher may have selected group 100 +under 30 seconds before receiving the update, or selected it under 5 seconds +after applying the update. Both can produce the same received bytes: a start +does not prove that any earlier group was available under either budget. +Successful writes, flushes, transport acknowledgments, and local poll order +do not prove which preferences the publisher used to select it. + +Recording the current 5-second budget can mislabel a start selected under +30 seconds. Recording the loosest possibly applied budget keeps 30 seconds, +so a later 15-second reader still skips the history this quest sought to +recover. The same ambiguity applies when a widening crosses START. Do not +ship either inference as a fix or claim the released-wire Goal is met. + +### Remaining implementation after scope approval + +1. Define a response binding the resolved start to the budget used to select + it on an unpublished wire. Prefer echoing that budget over keeping a + history of sent updates solely to recover it. Coordinate the response + shape and any per-update responses with the two quests above; sending a + fresh START alone does not identify the applied budget. +2. Carry the budget with the selection through pending publisher state, + rather than sampling the current budget when START is serialized. Record + that proven budget in the relay's model. Implement Rust and TypeScript + together, and update the lite draft and user-facing concepts as needed. + Preserve published encodings and do not infer provenance on those wires. +3. Add deterministic direct and relayed integration coverage for a tightening + applied before selection followed by a reader between the budgets, and + for tightenings and widenings crossing an already-selected response. + Verify which budget selected the start, not just when START arrived. +4. Run the affected checks, `just drafts check`, and `just test interop --all`, + including published-wire compatibility, before completing the quest. ## Related - [Drop set](/quest/m1/drop-set.md) - records the budget each drop was judged under, the same idea for drops -- [Delete the old start bookkeeping](/quest/m1/subscribe-ranges/start-bookkeeping.md) - moves where a subscription starts into the model, which should carry this field +- [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - owns the resolved-start response for ranges +- [Delete the old start bookkeeping](/quest/m1/subscribe-ranges/start-bookkeeping.md) - moves start state into the model and considers a response per update From e6f3e4fc8967939d0a6135bf3ea2aa34cd72e8cb Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 29 Oct 2026 09:31:09 -0700 Subject: [PATCH 03/11] feat(lite)!: bind START to its selecting budget Co-Authored-By: GPT-6.1 Sol --- drafts/draft-lcurley-moq-lite.md | 10 ++- js/net/src/lite/publisher.test.ts | 24 ++++++ js/net/src/lite/publisher.ts | 9 +- js/net/src/lite/subscribe.test.ts | 13 ++- js/net/src/lite/subscribe.ts | 17 +++- quest/m1/start-budget-update.md | 17 ++-- rs/moq-net/src/coding/writer.rs | 1 + rs/moq-net/src/fuzz.rs | 1 + rs/moq-net/src/lite/publisher.rs | 73 ++++++++++++++-- rs/moq-net/src/lite/subscribe.rs | 56 ++++++++++++- rs/moq-net/src/lite/subscriber.rs | 27 +++--- rs/moq-net/tests/integration/support/mock.rs | 20 ++++- .../tests/integration/widened_max_age.rs | 83 +++++++++++++++---- 13 files changed, 295 insertions(+), 56 deletions(-) diff --git a/drafts/draft-lcurley-moq-lite.md b/drafts/draft-lcurley-moq-lite.md index 3c1dcce1448..e71085958a4 100644 --- a/drafts/draft-lcurley-moq-lite.md +++ b/drafts/draft-lcurley-moq-lite.md @@ -1426,7 +1426,7 @@ The name a TRACKS_ADD on the stream listed. A SUBSCRIBE_OK message confirms a subscription and resolves its absolute start position. It is the first message the publisher sends on the Subscribe Stream, once the start position is known. -This is the trimmed-down counterpart of MoqTransport's SUBSCRIBE_OK: it retains the name and the role of the publisher's positive response, but carries only the resolved start position and the publisher's largest position (all other per-track properties live in [TRACK_INFO](#track-info)). +This is the trimmed-down counterpart of MoqTransport's SUBSCRIBE_OK: it retains the name and the role of the publisher's positive response, but carries only the resolved start position, its subscriber budget, and the publisher's largest position (all other per-track properties live in [TRACK_INFO](#track-info)). ~~~ SUBSCRIBE_OK Message { @@ -1435,6 +1435,7 @@ SUBSCRIBE_OK Message { Group (i) Largest Group (i) [Largest Frame (i)] + Subscriber Max Age (i) } ~~~ @@ -1464,6 +1465,12 @@ The largest position the publisher knows of for the track when it answers: the l `Largest Group` is the group sequence plus one, and 0 means the track has nothing yet, in which case `Largest Frame` is absent. A subscriber takes it as where the live feed is: a relay holding groups from an earlier subscription serves them from cache only once this says they are current, which a publisher answering promptly lets it do without waiting for a frame. +**Subscriber Max Age**: +The `Subscriber Max Age` from SUBSCRIBE or SUBSCRIBE_UPDATE used when selecting the start group, in milliseconds. +The publisher MUST retain it with a selected group while waiting to send this response; a later update does not change it. +A subscriber MAY give up history below `Group` for readers whose max age is no greater than this value. +A reader with a greater max age can still request that history. + ## SUBSCRIBE_END {#subscribe-end} A SUBSCRIBE_END message is sent by the publisher to signal that no group at or after a given sequence will be produced. @@ -1792,6 +1799,7 @@ The `Message Length` describes the payload size on the wire. - Added `Group Order` to SUBSCRIBE and SUBSCRIBE_UPDATE: descending (0x0, the previous fixed order) or ascending (0x1). - Listed the shared EXPIRED_AUTH_TOKEN (0x18) session code. A request that expires on its own resets with the UNAUTHORIZED stream code. +- Added `Subscriber Max Age` to SUBSCRIBE_OK, binding its start to the budget used at selection. - SUBSCRIBE_OK `Group` names where delivery starts when the subscription resolves, which need not be the first group sent. It is not a new floor. A subscriber does not wait for a group between the requested floor and it, but a publisher still delivers one that arrives later within Subscriber Max Age. - Replaced `Group Start`, `Group End`, `Frame Start`, and `Frame End` in SUBSCRIBE and SUBSCRIBE_UPDATE with a non-empty list of Ranges. Each end is an Anchor: none, an absolute Position (exclusive as an end), a count of Groups back from the latest, or a presentation time. - A subscription's range bounds datagrams like groups, and FETCH never returns a datagram. diff --git a/js/net/src/lite/publisher.test.ts b/js/net/src/lite/publisher.test.ts index 7298b8a31b1..9657bc90a74 100644 --- a/js/net/src/lite/publisher.test.ts +++ b/js/net/src/lite/publisher.test.ts @@ -1222,6 +1222,30 @@ for (const version of [Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07]) { } } +for (const updated of [5_000, 60_000]) { + test(`lite draft-07: a held selection keeps its 30000ms budget across an update to ${updated}`, async () => { + const sub = await servedSubscription({ version: Version.DRAFT_07, startGroup: 2 }); + try { + hooks.startTrack(sub.track, { requested: 2 }); + sub.serve(6); + await flush(); + await new SubscribeUpdate({ priority: 0, startGroup: 2, maxDelay: updated }).encode( + sub.client.writer, + Version.DRAFT_07, + ); + while (sub.track.subscription.peek()?.maxDelay !== updated) await sub.track.subscription.changed(); + hooks.startTrack(sub.track, { start: 3, requested: 2 }); + const response = await decodeSubscribeResponse(sub.client.reader, Version.DRAFT_07); + if (!("start" in response)) throw new Error("expected SUBSCRIBE_START"); + expect(response.start.group).toBe(3); + expect(response.start.maxDelay).toBe(TEST_MAX_DELAY_MS); + expect(await sub.servedSequence()).toBe(6); + } finally { + await sub.close(); + } + }); +} + // A relay's group stream can beat the upstream START on its own stream. Resolving from that // group would give up the backlog the START still promises, so the first group waits for it. for (const version of [Version.DRAFT_06, Version.DRAFT_07]) { diff --git a/js/net/src/lite/publisher.ts b/js/net/src/lite/publisher.ts index 7e2aa8eb7e3..7707d0aa5e0 100644 --- a/js/net/src/lite/publisher.ts +++ b/js/net/src/lite/publisher.ts @@ -1165,7 +1165,7 @@ export class Publisher { let startSequence: number | undefined; // The first group and the edge seen before it was popped, held until the source // resolves where its feed starts. - let first: { group: group.Consumer; largest: track.Position | undefined } | undefined; + let first: { group: group.Consumer; largest: track.Position | undefined; maxDelay: number } | undefined; let endSent = false; // Drop whatever part of `[start, end)` this subscription has not been served or dropped @@ -1301,6 +1301,7 @@ export class Publisher { !startSent && hasLargest(this.version) ? newer(first?.largest, newer(hooks.bufferedLargest(track), hooks.peekLargest(track).largest)) : undefined; + const maxDelay = first?.maxDelay ?? track.subscription.peek()?.maxDelay ?? Number.MAX_SAFE_INTEGER; const recv = first ? ({ kind: "group", group: first.group } as const) : hooks.tryRecvGroup(track, emitRange); @@ -1327,7 +1328,7 @@ export class Publisher { startSent = true; startSequence = bounds.startGroup; raiseFloor(bounds.startGroup); - const start = new SubscribeStart(bounds.startGroup, largest); + const start = new SubscribeStart(bounds.startGroup, largest, maxDelay); if ( !(await controls.response(encodeSubscribeResponse(stream, { start }, this.version))) ) @@ -1386,7 +1387,7 @@ export class Publisher { // A group stream can beat the upstream START on its own stream, and resolving // from the group alone would give up the backlog that START still promises. if (source === "pending") { - first = { group, largest }; + first = { group, largest, maxDelay }; await waitForSubscription(controls, track, drops); continue; } @@ -1410,7 +1411,7 @@ export class Publisher { !(await controls.response( encodeSubscribeResponse( stream, - { start: new SubscribeStart(startSequence, largest) }, + { start: new SubscribeStart(startSequence, largest, maxDelay) }, this.version, ), )) diff --git a/js/net/src/lite/subscribe.test.ts b/js/net/src/lite/subscribe.test.ts index df0620912c8..290676764f7 100644 --- a/js/net/src/lite/subscribe.test.ts +++ b/js/net/src/lite/subscribe.test.ts @@ -227,7 +227,7 @@ test("SubscribeStart round-trips on draft-05", async () => { test("SubscribeStart carries the largest position on draft-07", async () => { // Type, length, group, largest group + 1, largest frame. expect(await encode(Version.DRAFT_07, { start: new SubscribeStart(4, { group: 3, frame: 2 }) })).toEqual( - new Uint8Array([0, 3, 4, 4, 2]), + new Uint8Array([0, 4, 4, 4, 2, 0]), ); for (const largest of [undefined, { group: 3, frame: 2 }]) { const got = await responseRoundtrip(Version.DRAFT_07, { start: new SubscribeStart(4, largest) }); @@ -240,6 +240,17 @@ test("SubscribeStart carries the largest position on draft-07", async () => { ); }); +test("SubscribeStart binds its budget only on the unpublished wire", async () => { + const start = new SubscribeStart(4, { group: 3, frame: 2 }, 5000); + expect(await encode(Version.DRAFT_07, { start })).toEqual(new Uint8Array([0, 5, 4, 4, 2, 0x93, 0x88])); + for (const version of [Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07]) { + const got = await responseRoundtrip(version, { start }); + if (!("start" in got)) throw new Error("expected start"); + expect(got.start.maxDelay).toBe(version === Version.DRAFT_07 ? 5000 : undefined); + if (version !== Version.DRAFT_07) expect(await encode(version, { start })).toEqual(new Uint8Array([0, 1, 4])); + } +}); + test("SubscribeEnd round-trips on draft-05", async () => { // Type, length, exclusive final group. expect(await encode(Version.DRAFT_05, { end: new SubscribeEnd(7) })).toEqual(new Uint8Array([1, 1, 7])); diff --git a/js/net/src/lite/subscribe.ts b/js/net/src/lite/subscribe.ts index 64cd752a96f..e8138c3d627 100644 --- a/js/net/src/lite/subscribe.ts +++ b/js/net/src/lite/subscribe.ts @@ -617,9 +617,13 @@ export class SubscribeStart { */ largest?: Position; - constructor(group: number, largest?: Position) { + /** Subscriber Max Age used to select the start, absent before draft-07. */ + maxDelay?: number; + + constructor(group: number, largest?: Position, maxDelay = 0) { this.group = group; this.largest = largest; + this.maxDelay = maxDelay; } async encode(w: Writer, version: Version): Promise { @@ -633,16 +637,21 @@ export class SubscribeStart { await w.u53(this.largest.group + 1); await w.u53(this.largest.frame); } + if (this.maxDelay === undefined) throw new Error("missing start budget"); + await w.u53(this.maxDelay); }); } static async decode(r: Reader, version: Version): Promise { return Message.decode(r, async (r) => { const group = await r.u53(); - if (!hasLargest(version)) return new SubscribeStart(group); + const start = new SubscribeStart(group); + start.maxDelay = undefined; + if (!hasLargest(version)) return start; const largest = await r.u53(); - if (largest === 0) return new SubscribeStart(group); - return new SubscribeStart(group, { group: largest - 1, frame: await r.u53() }); + if (largest !== 0) start.largest = { group: largest - 1, frame: await r.u53() }; + start.maxDelay = await r.u53(); + return start; }); } } diff --git a/quest/m1/start-budget-update.md b/quest/m1/start-budget-update.md index 24cc599c468..c1573f05353 100644 --- a/quest/m1/start-budget-update.md +++ b/quest/m1/start-budget-update.md @@ -10,14 +10,15 @@ unpublished wire change rather than the originally planned lite-06 fix. ## Plan -Scope decision pending: the original released-wire Goal is unattainable. -Recommend keeping this quest for an explicit budget binding on lite-07, -coordinated with +Decided 2026-10-29 (explicit implementation brief under `/afk`): keep this +quest for the draft's recommended unpublished-wire scope. This supersedes +the earlier investigation-only brief. Echo the selecting Subscriber Max Age +in lite-07 START in Rust and JS, retaining it alongside a pending selected +group. Keep published encodings and their conservative ambiguous-budget +behavior. Do not add per-update STARTs or update revisions; coordinate with [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) and -[Delete the old start bookkeeping](/quest/m1/subscribe-ranges/start-bookkeeping.md). -The alternative is folding this work into that bookkeeping quest or -abandoning it and accepting the released-wire limitation. No wire design has -been approved by this investigation. +[Delete the old start bookkeeping](/quest/m1/subscribe-ranges/start-bookkeeping.md) +without taking over their work. Advance the PR only, never merge or enqueue. ### Ordering evidence @@ -56,7 +57,7 @@ so a later 15-second reader still skips the history this quest sought to recover. The same ambiguity applies when a widening crosses START. Do not ship either inference as a fix or claim the released-wire Goal is met. -### Remaining implementation after scope approval +### Implementation and validation 1. Define a response binding the resolved start to the budget used to select it on an unpublished wire. Prefer echoing that budget over keeping a diff --git a/rs/moq-net/src/coding/writer.rs b/rs/moq-net/src/coding/writer.rs index ef4e06e53cc..b6010fe9e5d 100644 --- a/rs/moq-net/src/coding/writer.rs +++ b/rs/moq-net/src/coding/writer.rs @@ -463,6 +463,7 @@ mod tests { writer .buffer(&lite::SubscribeStart { + max_delay: Some(std::time::Duration::ZERO), group: 3, largest: None, }) diff --git a/rs/moq-net/src/fuzz.rs b/rs/moq-net/src/fuzz.rs index 86018d5e628..1aad5e5c21a 100644 --- a/rs/moq-net/src/fuzz.rs +++ b/rs/moq-net/src/fuzz.rs @@ -829,6 +829,7 @@ impl Default for Messages { }], }, lite_start: lite::SubscribeResponse::Start(lite::SubscribeStart { + max_delay: Some(std::time::Duration::ZERO), group: 1_234, largest: None, }), diff --git a/rs/moq-net/src/lite/publisher.rs b/rs/moq-net/src/lite/publisher.rs index 126754c94b0..b8ae7b84bdd 100644 --- a/rs/moq-net/src/lite/publisher.rs +++ b/rs/moq-net/src/lite/publisher.rs @@ -3828,7 +3828,7 @@ struct TrackRun { // Where SUBSCRIBE_START resolved the feed, once sent. start: Option, // The first servable group, held until the source resolves where its feed starts. - first: Option, + first: Option<(group::Consumer, Duration)>, // Groups given up before the start resolved, which it must not name, with why. skipped: RangeMap, end_sent: bool, @@ -4054,14 +4054,14 @@ impl TrackRun { // The first group waits for the source to resolve where its feed starts; datagrams // keep flowing meanwhile. - if let Some(group) = self.first.take() { + if let Some((group, max_delay)) = self.first.take() { match self.track.poll_start(waiter) { Poll::Ready(source) => { - self.start(group, source, writer)?; + self.start((group, max_delay), source, writer)?; return Poll::Ready(Ok(ControlFlow::Continue(()))); } Poll::Pending => { - self.first = Some(group); + self.first = Some((group, max_delay)); if self.datagrams && let Poll::Ready(Some(datagram)) = self.track.poll_recv_stored_datagram(waiter)? { @@ -4089,6 +4089,12 @@ impl TrackRun { { self.start = Some(start); writer.buffer(&lite::SubscribeResponse::Start(lite::SubscribeStart { + max_delay: Some( + self.track + .subscription() + .max_delay + .unwrap_or(Duration::from_millis(super::MAX_SAFE_DELAY_MS)), + ), group: start, largest: Some(largest), }))?; @@ -4135,7 +4141,15 @@ impl TrackRun { return Poll::Ready(Ok(ControlFlow::Continue(()))); }; match self.emit_range && self.start.is_none() { - true => self.first = Some(group), + true => { + self.first = Some(( + group, + self.track + .subscription() + .max_delay + .unwrap_or(Duration::from_millis(super::MAX_SAFE_DELAY_MS)), + )) + } false => self.serve(group), } } @@ -4319,7 +4333,7 @@ impl TrackRun { /// drop the older one for good. fn start( &mut self, - group: group::Consumer, + (group, max_delay): (group::Consumer, Duration), source: Option, writer: &mut Writer, ) -> Result<(), Error> { @@ -4344,6 +4358,7 @@ impl TrackRun { }; let largest = live.max(self.track.peek_largest()); writer.buffer(&lite::SubscribeResponse::Start(lite::SubscribeStart { + max_delay: Some(max_delay), group: start, largest, }))?; @@ -6142,7 +6157,10 @@ mod serve_group_test { // SUBSCRIBE_START at 0, a DROP for the skipped group 1 with NOT_FOUND (0x33), which // the end gives up on at once, and SUBSCRIBE_END at 3. - assert_eq!(*log.writes.lock().unwrap(), [0, 3, 0, 1, 0, 2, 3, 1, 1, 0x33, 1, 1, 3]); + assert_eq!( + *log.writes.lock().unwrap(), + [0, 4, 0, 1, 0, 0, 2, 3, 1, 1, 0x33, 1, 1, 3] + ); } /// A relay that holds only the first group of a backlog answers with the largest its @@ -7123,6 +7141,40 @@ mod serve_group_test { assert_eq!(relay.opened(), 3, "group 0 is cached and now in range"); } + /// A selection waiting for upstream START keeps the budget used at the group pop, + /// even when a tightening or widening is applied before the response can be written. + #[moq_clock::test] + async fn a_held_start_keeps_its_selected_budget() { + for updated in [5, 60] { + let mut track = track::Producer::new( + Arc::new(broadcast::Info::default()), + "test", + crate::track::Info::timed(), + ); + let mut relay = RelayRun::new(&mut track, 0); + write_group(&mut track, 5, 0); + relay.settle(); + assert!(relay.run.first.is_some(), "selection waits for upstream START"); + relay.run.update(lite::SubscribeUpdate { + wanted_group: None, + order: Order::Descending, + priority: 0, + max_delay: Duration::from_secs(updated), + ranges: vec![lite::Range::OLDEST], + }); + relay.settle(); + track.start_at(0).unwrap(); + relay.settle(); + let writes = relay.session.log.writes.lock().unwrap(); + let (lite::SubscribeResponse::Start(start), _) = + lite::SubscribeResponse::decode_slice(&writes, Version::Lite07).unwrap() + else { + panic!("expected START") + }; + assert_eq!(start.max_delay, Some(Duration::from_secs(30))); + } + } + /// A SUBSCRIBE_UPDATE landing while the first group waits on the source's start keeps /// the floor it raised: resolving the start from the held group must not lower it. #[moq_clock::test] @@ -7276,7 +7328,7 @@ mod serve_group_test { assert_eq!( *log.writes.lock().unwrap(), // The DROP carries OLD (0x34). - [0, 3, 0, 1, 0, 2, 3, 0, 0, 0x34, 1, 1, 2], + [0, 4, 0, 1, 0, 0, 2, 3, 0, 0, 0x34, 1, 1, 2], "the expired group is dropped and the end precedes stream credit" ); @@ -7286,7 +7338,10 @@ mod serve_group_test { *open = true; drop(open); run.await.unwrap(); - assert_eq!(*log.writes.lock().unwrap(), [0, 3, 0, 1, 0, 2, 3, 0, 0, 0x34, 1, 1, 2]); + assert_eq!( + *log.writes.lock().unwrap(), + [0, 4, 0, 1, 0, 0, 2, 3, 0, 0, 0x34, 1, 1, 2] + ); } } diff --git a/rs/moq-net/src/lite/subscribe.rs b/rs/moq-net/src/lite/subscribe.rs index cef996b2e90..22418c852d5 100644 --- a/rs/moq-net/src/lite/subscribe.rs +++ b/rs/moq-net/src/lite/subscribe.rs @@ -504,6 +504,8 @@ impl<'a> Message<'a> for SubscribeOk { /// subscriber that asked for group 5 frame 15 and receives group 6 starts at frame 0. #[derive(Clone, Debug)] pub struct SubscribeStart { + /// The Subscriber Max Age used to select the start, absent before lite-07. + pub max_delay: Option, pub group: u64, /// The publisher's largest (group, frame) when it answered, `None` for a track with /// nothing yet. Lite07+ only; not on the wire before, where it decodes as `None`. @@ -528,7 +530,15 @@ impl<'a> Message<'a> for SubscribeStart { }, false => None, }; - Ok(Self { group, largest }) + let max_delay = version + .has_ranges() + .then(|| r.varint().map(std::time::Duration::from_millis)) + .transpose()?; + Ok(Self { + group, + largest, + max_delay, + }) } fn encode_msg(&self, w: &mut Encoder<'_>, version: Version) -> Result<(), EncodeError> { @@ -542,6 +552,10 @@ impl<'a> Message<'a> for SubscribeStart { w.varint(largest.frame)?; } } + if version.has_ranges() { + let max_delay = self.max_delay.ok_or(EncodeError::Version)?; + w.varint(u64::try_from(max_delay.as_millis()).map_err(|_| EncodeError::BoundsExceeded)?)?; + } Ok(()) } } @@ -839,6 +853,7 @@ mod test { #[test] fn subscribe_start_roundtrips_on_lite05() { let resp = SubscribeResponse::Start(SubscribeStart { + max_delay: Some(std::time::Duration::ZERO), group: 42, largest: None, }); @@ -857,7 +872,11 @@ mod test { #[test] fn subscribe_start_carries_the_largest_position_on_lite07() { for largest in [None, Some(crate::track::Position { group: 3, frame: 2 })] { - let resp = SubscribeResponse::Start(SubscribeStart { group: 4, largest }); + let resp = SubscribeResponse::Start(SubscribeStart { + group: 4, + largest, + max_delay: Some(std::time::Duration::ZERO), + }); let mut buf = Vec::new(); resp.encode( &mut crate::coding::Encoder::new(&mut buf, Version::Lite07.into()), @@ -870,6 +889,7 @@ mod test { } } let resp = SubscribeResponse::Start(SubscribeStart { + max_delay: Some(std::time::Duration::ZERO), group: 4, largest: Some(crate::track::Position { group: 3, frame: 2 }), }); @@ -882,6 +902,38 @@ mod test { assert_eq!(buf, [0, 1, 4], "lite-06 has no largest position"); } + #[test] + fn subscribe_start_binds_the_budget_only_on_lite07() { + let response = SubscribeResponse::Start(SubscribeStart { + group: 4, + largest: Some(crate::track::Position { group: 3, frame: 2 }), + max_delay: Some(std::time::Duration::from_millis(5000)), + }); + for version in [Version::Lite05, Version::Lite06, Version::Lite07] { + let mut bytes = Vec::new(); + response + .encode(&mut Encoder::new(&mut bytes, version.into()), version) + .unwrap(); + assert_eq!( + bytes, + if version.has_ranges() { + vec![0, 5, 4, 4, 2, 0x93, 0x88] + } else { + vec![0, 1, 4] + } + ); + let (SubscribeResponse::Start(start), _) = SubscribeResponse::decode_slice(&bytes, version).unwrap() else { + panic!("expected START") + }; + assert_eq!( + start.max_delay, + version.has_ranges().then(|| std::time::Duration::from_millis(5000)) + ); + } + // A lite-07 peer must prove its budget, including when its track is empty. + assert!(SubscribeResponse::decode_slice(&[0, 2, 4, 0], Version::Lite07).is_err()); + } + #[test] fn subscribe_end_roundtrips_on_lite05() { let resp = SubscribeResponse::End(SubscribeEnd { group: 7 }); diff --git a/rs/moq-net/src/lite/subscriber.rs b/rs/moq-net/src/lite/subscriber.rs index 8199a9ffbf5..a49776185b4 100644 --- a/rs/moq-net/src/lite/subscriber.rs +++ b/rs/moq-net/src/lite/subscriber.rs @@ -2143,7 +2143,7 @@ mod tests { .unwrap(), id: 0, max_delay: Duration::ZERO, - requested_max_delay: Duration::ZERO, + sent_max_delay: Duration::ZERO, start: None, priority: 0, requested: None, @@ -2279,7 +2279,7 @@ mod tests { .unwrap(), id: 0, max_delay: Duration::ZERO, - requested_max_delay: Duration::ZERO, + sent_max_delay: Duration::ZERO, start: Some(floor), priority: 0, requested: Some(floor), @@ -2743,7 +2743,7 @@ mod tests { stream, id: 0, max_delay: Duration::ZERO, - requested_max_delay: Duration::ZERO, + sent_max_delay: Duration::ZERO, start: Some(2), priority: 0, requested: Some(2), @@ -3436,6 +3436,7 @@ mod tests { for version in [Version::Lite05, Version::Lite06, Version::Lite07] { let mut responses = Vec::new(); lite::SubscribeResponse::Start(lite::SubscribeStart { + max_delay: Some(std::time::Duration::ZERO), group: 0, largest: None, }) @@ -3554,6 +3555,7 @@ mod tests { let mut answer = Vec::new(); let mut encoder = crate::coding::Encoder::new(&mut answer, version.into()); lite::SubscribeResponse::Start(lite::SubscribeStart { + max_delay: Some(std::time::Duration::ZERO), group: 0, largest: None, }) @@ -3608,6 +3610,7 @@ mod tests { let largest = track::Position { group: 4, frame: 2 }; let mut responses = Vec::new(); lite::SubscribeResponse::Start(lite::SubscribeStart { + max_delay: Some(std::time::Duration::ZERO), group: 4, largest: Some(largest), }) @@ -3670,6 +3673,7 @@ mod tests { let code = crate::coding::StreamCodes::encode_stream_code(&Version::Lite07, &crate::StreamError::App(5)); for resp in [ lite::SubscribeResponse::Start(lite::SubscribeStart { + max_delay: Some(std::time::Duration::ZERO), group: 0, largest: None, }), @@ -3810,6 +3814,7 @@ mod tests { let mut responses = Vec::new(); if started { lite::SubscribeResponse::Start(lite::SubscribeStart { + max_delay: Some(std::time::Duration::ZERO), group: 0, largest: None, }) @@ -6302,9 +6307,9 @@ struct SubStream { /// start sits elsewhere the request-tracked floor stands instead, and demand /// returning here makes the declaration valid again. requested: Option, - /// The max delay the SUBSCRIBE itself carried, fixed for the stream's life: a lite-06+ - /// SUBSCRIBE_START resolves the start from it, skipping older groups as stale. - requested_max_delay: Duration, + /// The loosest budget sent on this stream. Published STARTs cannot identify which + /// update selected their group, so only this conservative budget can judge history. + sent_max_delay: Duration, /// The groups received for this subscription, shared with its [`TrackEntry`]. tail: kio::Producer, /// The first group the publisher serves (SUBSCRIBE_START), once declared. @@ -6953,6 +6958,7 @@ fn wire_max_delay(max_delay: Option) -> std::time::Duration /// Buffer the current SUBSCRIBE parameters. The caller flushes. fn buffer_update(active: &mut SubStream) -> Result<(), Error> { + active.sent_max_delay = active.sent_max_delay.max(active.max_delay); active.bounded |= bounded(&active.ranges); active.stream.writer.buffer(&lite::SubscribeUpdate { wanted_group: active.wanted, @@ -7065,7 +7071,7 @@ impl Establish { stream, id: self.id, max_delay: wire_max_delay(self.subscription.max_delay), - requested_max_delay: wire_max_delay(self.subscription.max_delay), + sent_max_delay: wire_max_delay(self.subscription.max_delay), start: self.subscription.start(), ranges: self.ranges.clone(), priority: self.subscription.priority, @@ -8011,9 +8017,10 @@ impl ServeLoop { // request-tracked floor stands rather than a guess. if active.start == active.requested { let _ = match serve.subscriber.version.resolves_start() { - true => { - self.serving.start_resolved(start.group, active.requested_max_delay) - } + true => self.serving.start_resolved( + start.group, + start.max_delay.unwrap_or(active.sent_max_delay), + ), false => self.serving.start_at(start.group), }; } diff --git a/rs/moq-net/tests/integration/support/mock.rs b/rs/moq-net/tests/integration/support/mock.rs index 1b96c550eae..dc0e65a81a3 100644 --- a/rs/moq-net/tests/integration/support/mock.rs +++ b/rs/moq-net/tests/integration/support/mock.rs @@ -204,7 +204,7 @@ impl MockSendStream { && size > 0 { if let Some(types) = self.first_byte.take() { - types.lock().unwrap().push(first.expect("non-empty write")); + types.lock().push(first.expect("non-empty write")); } self.line.arrivals.lock().unwrap().push_back(self.conn.arrival()); } @@ -427,7 +427,7 @@ impl Drop for MockChunks { type Resets = Arc>>; /// The first byte of each bidi stream one side has opened. -type StreamTypes = Arc>>; +type StreamTypes = Arc>>; /// One side's knobs. struct Side { @@ -832,7 +832,21 @@ impl MockSession { /// The first byte, the stream type, of every bidi stream this side has written to, /// in the order each was first written. pub fn bidi_types(&self) -> Vec { - self.side.bidi_types.lock().unwrap().clone() + self.side.bidi_types.read().clone() + } + + /// Wait for `count` streams of `kind` to reach the wire. + pub async fn bidi_count(&self, kind: u8, count: usize) { + kio::wait(|waiter| { + self.side.bidi_types.poll(waiter, |types| { + if types.iter().filter(|&&byte| byte == kind).count() >= count { + Poll::Ready(()) + } else { + Poll::Pending + } + }) + }) + .await; } /// Every code this side has reset a stream with, in order. diff --git a/rs/moq-net/tests/integration/widened_max_age.rs b/rs/moq-net/tests/integration/widened_max_age.rs index b2df58aaa21..af468b0fb6d 100644 --- a/rs/moq-net/tests/integration/widened_max_age.rs +++ b/rs/moq-net/tests/integration/widened_max_age.rs @@ -11,7 +11,7 @@ use std::collections::BTreeSet; use std::time::Duration; use moq_net::track::{Info, Range, Subscription}; -use moq_net::{Hop, Timescale, Timestamp, Version, origin, track}; +use moq_net::{Hop, Timescale, Timestamp, Version, kio, origin, track}; use crate::support::harness::{MockConnectOptions, MockPair, connect_mock, peer, spawn}; @@ -45,44 +45,57 @@ struct Chain { source: track::Producer, viewer: origin::Producer, links: Vec, + source_gate: kio::Producer, _hold: Vec>, } impl Chain { /// Without `relay`, the viewer reads the publisher directly. async fn new(version: &str, relay: bool) -> Self { + let chain = Self::empty(version, relay).await; + chain.populate(); + chain + } + + async fn empty(version: &str, relay: bool) -> Self { let version: Version = version.parse().unwrap(); let publisher = origin(1); let relay = relay.then(|| origin(2)); let viewer = origin(3); let mut links = Vec::new(); - match &relay { - Some(relay) => { - links.push(peer(version, &publisher, relay).await); - links.push(peer(version, relay, &viewer).await); - } - None => links.push(peer(version, &publisher, &viewer).await), + let source_gate = kio::Producer::new(true); + let mut options = MockConnectOptions::new(version); + options.server_publish = Some(publisher.consume()); + options.client_subscribe = Some(relay.as_ref().unwrap_or(&viewer).clone()); + options.server_gate = Some(source_gate.consume()); + links.push(connect_mock(options).await); + if let Some(relay) = &relay { + links.push(peer(version, relay, &viewer).await); } let broadcast = publisher.create_broadcast("live").unwrap(); let source = broadcast .create_track("video", Info::default().with_timescale(Timescale::MILLI)) .unwrap(); - for sequence in 0..GROUPS { - let mut group = source.create_group(moq_net::group::Info { sequence }).unwrap(); - group - .write_frame(Timestamp::from_millis(sequence * 1000).unwrap(), &b"x"[..]) - .unwrap(); - group.finish().unwrap(); - } broadcast.announce(Default::default()).unwrap(); Self { source, viewer, links, + source_gate, _hold: vec![Box::new(publisher), Box::new(relay), Box::new(broadcast)], } } + fn populate(&self) { + for sequence in 0..GROUPS { + let mut group = self.source.create_group(moq_net::group::Info { sequence }).unwrap(); + group + .write_frame(Timestamp::from_millis(sequence * 1000).unwrap(), &b"x"[..]) + .unwrap(); + group.finish().unwrap(); + } + } + /// The FETCH streams opened on any lite link, by their stream type 3. fn fetches(&self) -> usize { self.links @@ -140,6 +153,48 @@ async fn a_fresh_reader_fills_nothing_below_its_start() { } } +/// Tighten while no start is selectable, then ask for a budget between the old and +/// selected ones. Hold the publisher's driver so only an actual fill request, rather +/// than its live re-offer, can prove the copy retained the selected budget. +#[moq_clock::test] +async fn a_tightened_start_leaves_history_fillable_by_an_intermediate_reader() { + for relay in [false, true] { + let chain = Chain::empty(VERSION, relay).await; + let mut first = chain.subscribe(WIDE).await; + let mut source = chain.source.clone(); + while source.subscription_changed().await.unwrap().is_none() {} + first + .update( + Subscription::default() + .with_max_delay(NARROW) + .with_ranges([Range::oldest()]), + ) + .unwrap(); + while source.subscription_changed().await.unwrap().and_then(|s| s.max_delay) != Some(NARROW) {} + chain.populate(); + assert_eq!(received(&mut first).await, (6..GROUPS).collect()); + + *chain.source_gate.write().ok().unwrap() = false; + let before = chain.fetches(); + let mut intermediate = chain.subscribe(Duration::from_millis(4500)).await; + // Drive the reader's miss walk too, with the upstream deliberately parked. + let reading = moq_clock::sim::spawn(async move { + let mut got = BTreeSet::new(); + while got.len() < 6 { + got.insert(intermediate.recv_group().await.unwrap().unwrap().sequence); + } + got + }); + let upstream = &chain.links[0].client_transport; + moq_clock::timeout(TIMEOUT, upstream.bidi_count(3, before + 1)) + .await + .expect("the proven tighter start allows fills"); + *chain.source_gate.write().ok().unwrap() = true; + // This reader can also receive the feed's re-offer once the source resumes. + assert_eq!(reading.await.unwrap(), (4..GROUPS).collect(), "relay={relay}"); + } +} + /// What the subscriber reads after widening its own max age, through a relay or not. #[moq_clock::test] async fn a_widened_reader_gets_what_was_skipped() { From 103f556e388d6b0122b49940d3ce2abce678026c Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 29 Oct 2026 09:33:09 -0700 Subject: [PATCH 04/11] test(lite): cover budget updates before selection Co-Authored-By: GPT-6.1 Sol --- js/net/src/lite/publisher.test.ts | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/js/net/src/lite/publisher.test.ts b/js/net/src/lite/publisher.test.ts index 9657bc90a74..e57b446721d 100644 --- a/js/net/src/lite/publisher.test.ts +++ b/js/net/src/lite/publisher.test.ts @@ -1222,6 +1222,26 @@ for (const version of [Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07]) { } } +for (const maxDelay of [5_000, 60_000]) { + test(`lite draft-07: an update before selection uses its ${maxDelay}ms budget`, async () => { + const sub = await servedSubscription({ version: Version.DRAFT_07, startGroup: 2 }); + try { + await new SubscribeUpdate({ priority: 0, startGroup: 2, maxDelay }).encode( + sub.client.writer, + Version.DRAFT_07, + ); + while (sub.track.subscription.peek()?.maxDelay !== maxDelay) await sub.track.subscription.changed(); + sub.serve(6); + const response = await decodeSubscribeResponse(sub.client.reader, Version.DRAFT_07); + if (!("start" in response)) throw new Error("expected SUBSCRIBE_START"); + expect(response.start.maxDelay).toBe(maxDelay); + expect(await sub.servedSequence()).toBe(6); + } finally { + await sub.close(); + } + }); +} + for (const updated of [5_000, 60_000]) { test(`lite draft-07: a held selection keeps its 30000ms budget across an update to ${updated}`, async () => { const sub = await servedSubscription({ version: Version.DRAFT_07, startGroup: 2 }); From bf61d4735af83bef1c3d397577bad1e9fbd9fb27 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 29 Oct 2026 09:33:54 -0700 Subject: [PATCH 05/11] docs(quest): retain the remaining START validation Co-Authored-By: GPT-6.1 Sol --- quest/m1/start-budget-update.md | 97 +++++++++------------------------ 1 file changed, 26 insertions(+), 71 deletions(-) diff --git a/quest/m1/start-budget-update.md b/quest/m1/start-budget-update.md index c1573f05353..af6f8f86ed1 100644 --- a/quest/m1/start-budget-update.md +++ b/quest/m1/start-budget-update.md @@ -2,82 +2,37 @@ ## Goal -A relay copy knows the subscriber `max_delay` used to resolve its upstream +A relay copy knows the subscriber `max_delay` used to select its upstream start, so a later reader with a looser budget fills the history it can use. -The response proves which budget was applied even when SUBSCRIBE_UPDATE -crosses it in flight. Published wires remain unchanged; this requires an -unpublished wire change rather than the originally planned lite-06 fix. +The response proves the selecting budget even when SUBSCRIBE_UPDATE crosses +it in flight. Published wires retain their encodings and conservative +ambiguous-budget behavior. ## Plan -Decided 2026-10-29 (explicit implementation brief under `/afk`): keep this -quest for the draft's recommended unpublished-wire scope. This supersedes -the earlier investigation-only brief. Echo the selecting Subscriber Max Age -in lite-07 START in Rust and JS, retaining it alongside a pending selected -group. Keep published encodings and their conservative ambiguous-budget -behavior. Do not add per-update STARTs or update revisions; coordinate with -[Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) and -[Delete the old start bookkeeping](/quest/m1/subscribe-ranges/start-bookkeeping.md) -without taking over their work. Advance the PR only, never merge or enqueue. - -### Ordering evidence - -Found by review of [moq-dev/moq#7168](https://github.com/moq-dev/moq/pull/7168): -the lite subscriber fixes `requested_max_delay` at SUBSCRIBE and passes it to -`start_resolved`. If the publisher applies a tightening before resolving the -start, a later reader between the two budgets gives up fillable history. - -There is no lite-06 ordering that identifies the applied update: - -- [`SubscribeStart`](https://github.com/moq-dev/moq/blob/0f68c6f9564e4dbfcd5b1f457eb3b2fdb420faa0/rs/moq-net/src/lite/subscribe.rs#L506) - encodes only `group` on lite-06. `largest` is lite-07-only and carries no - budget or request revision either. The - [TypeScript codec](https://github.com/moq-dev/moq/blob/0f68c6f9564e4dbfcd5b1f457eb3b2fdb420faa0/js/net/src/lite/subscribe.ts#L611) - has the same layout. -- [`SubscribeUpdate`](https://github.com/moq-dev/moq/blob/0f68c6f9564e4dbfcd5b1f457eb3b2fdb420faa0/rs/moq-net/src/lite/subscribe.rs#L581) - carries replacement preferences, with no acknowledgment or revision. - [The publisher applies them](https://github.com/moq-dev/moq/blob/0f68c6f9564e4dbfcd5b1f457eb3b2fdb420faa0/rs/moq-net/src/lite/publisher.rs#L3904) - without sending a fresh START. -- The [release-branch draft](https://github.com/moq-dev/moq/blob/8677afaa6bc1420a235a249253dd56d09a378d2b/drafts/draft-lcurley-moq-lite.md#L1135) - calls the same response SUBSCRIBE_OK and gives it only `Group`. - Updates travel subscriber to publisher; the response travels publisher to - subscriber. Ordering within each direction does not order the two directions. - -For example, send SUBSCRIBE with 30 seconds, then UPDATE with 5 seconds, -then receive START at group 100. The publisher may have selected group 100 -under 30 seconds before receiving the update, or selected it under 5 seconds -after applying the update. Both can produce the same received bytes: a start -does not prove that any earlier group was available under either budget. -Successful writes, flushes, transport acknowledgments, and local poll order -do not prove which preferences the publisher used to select it. - -Recording the current 5-second budget can mislabel a start selected under -30 seconds. Recording the loosest possibly applied budget keeps 30 seconds, -so a later 15-second reader still skips the history this quest sought to -recover. The same ambiguity applies when a widening crosses START. Do not -ship either inference as a fix or claim the released-wire Goal is met. - -### Implementation and validation - -1. Define a response binding the resolved start to the budget used to select - it on an unpublished wire. Prefer echoing that budget over keeping a - history of sent updates solely to recover it. Coordinate the response - shape and any per-update responses with the two quests above; sending a - fresh START alone does not identify the applied budget. -2. Carry the budget with the selection through pending publisher state, - rather than sampling the current budget when START is serialized. Record - that proven budget in the relay's model. Implement Rust and TypeScript - together, and update the lite draft and user-facing concepts as needed. - Preserve published encodings and do not infer provenance on those wires. -3. Add deterministic direct and relayed integration coverage for a tightening - applied before selection followed by a reader between the budgets, and - for tightenings and widenings crossing an already-selected response. - Verify which budget selected the start, not just when START arrived. -4. Run the affected checks, `just drafts check`, and `just test interop --all`, - including published-wire compatibility, before completing the quest. +Decided 2026-10-29 (explicit implementation brief under `/afk`): adopt the +draft's recommended unpublished-wire scope, superseding the earlier +investigation-only brief. Lite-06's group-only response cannot prove which +update selected its start, so this cannot recover that history on a +published wire. + +Implemented in [moq-dev/moq#7235](https://github.com/moq-dev/moq/pull/7235): +lite-07 START echoes Subscriber Max Age after its existing fields in Rust +and JS. The publisher captures it with the selected group, retaining both +while waiting for upstream START. Rust uses it in the existing budget-aware +fill ledger. Published responses omit it, and Rust conservatively records +the loosest sent budget when provenance is absent. No update revisions or +per-update STARTs are added; range responses and start bookkeeping retain +their own scope and branches. + +Remaining: complete `just ci`, `just test interop --all`, and published-wire +compatibility validation; resolve relevant failures, then remove this quest +and mark the PR ready. The deterministic crossed-update tests, direct and +relayed intermediate-budget history-fill test, and exact published-wire +fixtures are implemented. Do not merge, enqueue, or watch this PR. ## Related -- [Drop set](/quest/m1/drop-set.md) - records the budget each drop was judged under, the same idea for drops +- [Drop set](/quest/m1/drop-set.md) - records the budget each drop was judged under - [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - owns the resolved-start response for ranges -- [Delete the old start bookkeeping](/quest/m1/subscribe-ranges/start-bookkeeping.md) - moves start state into the model and considers a response per update +- [Delete the old start bookkeeping](/quest/m1/subscribe-ranges/start-bookkeeping.md) - moves start state into the model and considers per-update responses From 015ce027249c024a6cfbe45a0782629042f40a04 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 29 Oct 2026 09:36:23 -0700 Subject: [PATCH 06/11] docs(quest): align the START milestone entry with implementation Co-Authored-By: GPT-6.1 Sol --- quest/m1/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/quest/m1/README.md b/quest/m1/README.md index fd9543046be..e04662cd04c 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -139,7 +139,7 @@ removed). - [Announcement takeover](/quest/m1/announce-takeover.md) - a new independent route explicitly continues instances served through its predecessor, with shared Rust/JS semantics and lite-07 wire - [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - a lite-07 SUBSCRIBE asks for past and live ranges with absolute or relative ends, in either order, and replaces FETCH; merged ranges never starve a subscriber; relays fill misses by range, including over moq-transport - [Drop set](/quest/m1/drop-set.md) - each cursor judges missing groups against its own budget, so a narrow reader drops promptly and a widening re-offers groups already reported dropped on lite-07 -- [Bind a resolved start to its subscriber budget](/quest/m1/start-budget-update.md) - decide an unpublished-wire budget binding so updates cannot make later readers skip fillable history; lite-06 cannot prove the applied budget +- [Bind a resolved start to its subscriber budget](/quest/m1/start-budget-update.md) - bind unpublished-wire START to its selecting budget so later readers can fill history; published encodings stay unchanged - [Broadcast archives](/quest/m1/archive/README.md) - ANNOUNCE, TRACKS, and a SUBSCRIBE per track record every track under a prefix to any object_store, a host serves a stored broadcast back, and the catalog drops its archive pointer for a per-rendition `hls` field - [HLS recorder target](/quest/m1/hls-record-target.md) - the moq-cli HLS recorder takes its target duration from the declared `hls` duration - [Finalize moq-lite-07](/quest/m1/lite07-finalize.md) - when the maintainer cuts it, lite-07 negotiates as `moq-lite-07` and the next release ships it From 5e16b4817ef91f8ecb2203a12d2affc3c91463b7 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 29 Oct 2026 09:37:33 -0700 Subject: [PATCH 07/11] fix(lite): require explicit START budget provenance in JS Co-Authored-By: GPT-6.1 Sol --- js/net/src/lite/connection.test.ts | 2 +- js/net/src/lite/subscribe.test.ts | 12 ++++++---- js/net/src/lite/subscribe.ts | 3 +-- js/net/src/lite/subscriber.test.ts | 15 ++++++++---- js/net/src/lite/tail.test.ts | 34 ++++++++++++++-------------- js/net/src/lite/track_stream.test.ts | 2 +- test/interop/bare-fin.ts | 2 +- 7 files changed, 39 insertions(+), 31 deletions(-) diff --git a/js/net/src/lite/connection.test.ts b/js/net/src/lite/connection.test.ts index 211bde08a52..bf476dfd049 100644 --- a/js/net/src/lite/connection.test.ts +++ b/js/net/src/lite/connection.test.ts @@ -829,7 +829,7 @@ for (const violation of ["a partial group", "a group past the declared end"] as sub = stream; } } - await encodeSubscribeResponse(sub.writer, { start: new SubscribeStart(0) }, version); + await encodeSubscribeResponse(sub.writer, { start: new SubscribeStart(0, undefined, 0) }, version); let group: GroupMessage; if (violation === "a partial group") { diff --git a/js/net/src/lite/subscribe.test.ts b/js/net/src/lite/subscribe.test.ts index 290676764f7..4daad1266f5 100644 --- a/js/net/src/lite/subscribe.test.ts +++ b/js/net/src/lite/subscribe.test.ts @@ -218,7 +218,7 @@ test("a lite-07 list decodes as the span covering it", async () => { }); test("SubscribeStart round-trips on draft-05", async () => { - const got = await responseRoundtrip(Version.DRAFT_05, { start: new SubscribeStart(42) }); + const got = await responseRoundtrip(Version.DRAFT_05, { start: new SubscribeStart(42, undefined, 0) }); expect("start" in got).toBe(true); if (!("start" in got)) throw new Error("expected start"); expect(got.start.group).toBe(42); @@ -226,16 +226,16 @@ test("SubscribeStart round-trips on draft-05", async () => { test("SubscribeStart carries the largest position on draft-07", async () => { // Type, length, group, largest group + 1, largest frame. - expect(await encode(Version.DRAFT_07, { start: new SubscribeStart(4, { group: 3, frame: 2 }) })).toEqual( + expect(await encode(Version.DRAFT_07, { start: new SubscribeStart(4, { group: 3, frame: 2 }, 0) })).toEqual( new Uint8Array([0, 4, 4, 4, 2, 0]), ); for (const largest of [undefined, { group: 3, frame: 2 }]) { - const got = await responseRoundtrip(Version.DRAFT_07, { start: new SubscribeStart(4, largest) }); + const got = await responseRoundtrip(Version.DRAFT_07, { start: new SubscribeStart(4, largest, 0) }); if (!("start" in got)) throw new Error("expected start"); expect([got.start.group, got.start.largest]).toEqual([4, largest]); } // Draft-06 has no largest position on the wire. - expect(await encode(Version.DRAFT_06, { start: new SubscribeStart(4, { group: 3, frame: 2 }) })).toEqual( + expect(await encode(Version.DRAFT_06, { start: new SubscribeStart(4, { group: 3, frame: 2 }, 0) })).toEqual( new Uint8Array([0, 1, 4]), ); }); @@ -251,6 +251,10 @@ test("SubscribeStart binds its budget only on the unpublished wire", async () => } }); +test("draft-07 refuses a START without a selecting budget", async () => { + await expect(encode(Version.DRAFT_07, { start: new SubscribeStart(4) })).rejects.toThrow("missing start budget"); +}); + test("SubscribeEnd round-trips on draft-05", async () => { // Type, length, exclusive final group. expect(await encode(Version.DRAFT_05, { end: new SubscribeEnd(7) })).toEqual(new Uint8Array([1, 1, 7])); diff --git a/js/net/src/lite/subscribe.ts b/js/net/src/lite/subscribe.ts index e8138c3d627..53c2cac81ca 100644 --- a/js/net/src/lite/subscribe.ts +++ b/js/net/src/lite/subscribe.ts @@ -620,7 +620,7 @@ export class SubscribeStart { /** Subscriber Max Age used to select the start, absent before draft-07. */ maxDelay?: number; - constructor(group: number, largest?: Position, maxDelay = 0) { + constructor(group: number, largest?: Position, maxDelay?: number) { this.group = group; this.largest = largest; this.maxDelay = maxDelay; @@ -646,7 +646,6 @@ export class SubscribeStart { return Message.decode(r, async (r) => { const group = await r.u53(); const start = new SubscribeStart(group); - start.maxDelay = undefined; if (!hasLargest(version)) return start; const largest = await r.u53(); if (largest !== 0) start.largest = { group: largest - 1, frame: await r.u53() }; diff --git a/js/net/src/lite/subscriber.test.ts b/js/net/src/lite/subscriber.test.ts index 91bc02e6ead..770b9aa841f 100644 --- a/js/net/src/lite/subscriber.test.ts +++ b/js/net/src/lite/subscriber.test.ts @@ -1019,7 +1019,7 @@ test.each([Version.DRAFT_06, Version.DRAFT_07])( await answerTrackInfo(await opened(0), version, 1000); const request = await opened(1); await request.reading; - await respond(request, version, [{ start: new SubscribeStart(0) }]); + await respond(request, version, [{ start: new SubscribeStart(0, undefined, 0) }]); const received = track.recvDatagram(); await writer.write(new DatagramMessage(0n, 0, 0, new Uint8Array(1198)).encode(version)); await writer.write(new DatagramMessage(0n, 1, 1, new Uint8Array(1197)).encode(version)); @@ -1657,7 +1657,7 @@ test.each([ await subscribe.reading; if (outcome === "answered") { - await respond(subscribe, version, [{ start: new SubscribeStart(0) }]); + await respond(subscribe, version, [{ start: new SubscribeStart(0, undefined, 0) }]); await drainReactions(); await pool.replace(); await drainReactions(); @@ -1666,7 +1666,12 @@ test.each([ await pool.withdraw(); await drainReactions(); expect(track.closed.peek()).toBeUndefined(); - await respond(subscribe, version, [{ start: new SubscribeStart(0) }, { end: new SubscribeEnd(0) }], true); + await respond( + subscribe, + version, + [{ start: new SubscribeStart(0, undefined, 0) }, { end: new SubscribeEnd(0) }], + true, + ); expect(await track.closed).toBeNull(); } else { await pool.replace(); @@ -2240,7 +2245,7 @@ test("a lite-07 publisher FIN ends a covered bounded subscription", async () => new WritableStream({ write: (chunk) => void chunks.push(new Uint8Array(chunk)) }), version, ); - await encodeSubscribeResponse(writer, { start: new SubscribeStart(0) }, version); + await encodeSubscribeResponse(writer, { start: new SubscribeStart(0, undefined, 0) }, version); await encodeSubscribeResponse( writer, { drop: new SubscribeDrop({ start: 0, end: 1, error: StreamCode.NotFound }) }, @@ -2332,7 +2337,7 @@ test.each([ version, ); if (start) { - await encodeSubscribeResponse(writer, { start: new SubscribeStart(0) }, version); + await encodeSubscribeResponse(writer, { start: new SubscribeStart(0, undefined, 0) }, version); await encodeSubscribeResponse( writer, { drop: new SubscribeDrop({ start: 0, end: 0, error: StreamCode.NotFound }) }, diff --git a/js/net/src/lite/tail.test.ts b/js/net/src/lite/tail.test.ts index 1fdcbea68e4..b63aee6b7a3 100644 --- a/js/net/src/lite/tail.test.ts +++ b/js/net/src/lite/tail.test.ts @@ -112,7 +112,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver test("the source start waits for SUBSCRIBE_START where it resolves the start", async () => { const { reader, respond } = await subscribed(version, GRACE, { start: { included: 2 } }); expect(hooks.sourceStart(reader)).toBe(resolvesStart(version) ? "pending" : undefined); - await respond({ start: new SubscribeStart(3) }); + await respond({ start: new SubscribeStart(3, undefined, 0) }); await settle(); expect(hooks.sourceStart(reader)).toBe(3); reader.close(); @@ -120,7 +120,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver test("a group stream that arrives after the subscribe stream's FIN is delivered", async () => { const { subscriber, reader, respond, fin } = await subscribed(version); - await respond({ start: new SubscribeStart(0) }); + await respond({ start: new SubscribeStart(0, undefined, 0) }); const first = groupStream(subscriber, 0); first.write("0.0"); first.finish(); @@ -144,7 +144,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver test("a group read across the subscribe stream's FIN is delivered whole", async () => { const { subscriber, reader, respond, fin } = await subscribed(version); - await respond({ start: new SubscribeStart(0) }); + await respond({ start: new SubscribeStart(0, undefined, 0) }); const group = groupStream(subscriber, 0); group.write("0.0"); @@ -163,7 +163,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver test("a group reset after the subscribe stream's FIN is not presented as complete", async () => { const { subscriber, reader, respond, fin } = await subscribed(version); - await respond({ start: new SubscribeStart(0) }); + await respond({ start: new SubscribeStart(0, undefined, 0) }); const group = groupStream(subscriber, 0); group.write("0.0"); await respond({ end: new SubscribeEnd(1) }); @@ -201,7 +201,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver subscriber = sub.subscriber; const { reader, respond, fin } = sub; const ordered = reader.fork({ maxDelay: GRACE }).ordered(); - await respond({ start: new SubscribeStart(0) }); + await respond({ start: new SubscribeStart(0, undefined, 0) }); const group = groupStream(subscriber, 1); group.write("1.0"); group.finish(); @@ -240,7 +240,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver // after SUBSCRIBE_END. The reader receives it before its end, which then comes at once. test("a lower group that lands after SUBSCRIBE_END reaches the reader", async () => { const { subscriber, reader, respond, fin } = await subscribed(version, Milli(60_000)); - await respond({ start: new SubscribeStart(0) }); + await respond({ start: new SubscribeStart(0, undefined, 0) }); const high = groupStream(subscriber, 1); high.finish(); await high.handled; @@ -268,7 +268,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver test("a subscription ends without waiting once every group is accounted for", async () => { // A max delay far past the test's patience: only the accounting may end it. const { subscriber, reader, respond, fin } = await subscribed(version, Milli(60_000)); - await respond({ start: new SubscribeStart(0) }); + await respond({ start: new SubscribeStart(0, undefined, 0) }); await respond({ drop: new SubscribeDrop({ start: 0, end: 0, error: 0 }) }); const group = groupStream(subscriber, 1); group.write("1.0"); @@ -285,7 +285,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver // bytes read as OLD there. test("a SUBSCRIBE_DROP ends a group still arriving, before the FIN", async () => { const { subscriber, reader, respond } = await subscribed(version, Milli(60_000)); - await respond({ start: new SubscribeStart(0) }); + await respond({ start: new SubscribeStart(0, undefined, 0) }); const arriving = groupStream(subscriber, 0); arriving.write("0.0"); const group = await reader.recvGroup(); @@ -322,7 +322,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver // lite-05 specified an inclusive end, so there the group costs only its own stream. test(`a group at or past SUBSCRIBE_END ${version === Version.DRAFT_05 ? "is dropped" : "aborts the track"}`, async () => { const { subscriber, reader, respond } = await subscribed(version); - await respond({ start: new SubscribeStart(0) }); + await respond({ start: new SubscribeStart(0, undefined, 0) }); await respond({ end: new SubscribeEnd(2) }); expect(await reader.finished()).toEqual({ group: 2, frame: 0 }); @@ -346,7 +346,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver const { subscriber, reader, respond, fin } = await subscribed(version, Milli(60_000), { start: { included: 1 }, }); - await respond({ start: new SubscribeStart(3) }); + await respond({ start: new SubscribeStart(3, undefined, 0) }); const group = groupStream(subscriber, 3); group.finish(); await group.handled; @@ -361,7 +361,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver test("a lowered floor owes the groups it newly asked for", async () => { const maxDelay = Milli(60_000); const { subscriber, respond, fin } = await subscribed(version, maxDelay, { start: { included: 3 } }); - await respond({ start: new SubscribeStart(3) }); + await respond({ start: new SubscribeStart(3, undefined, 0) }); const group = groupStream(subscriber, 3); group.write("3.0"); group.finish(); @@ -400,7 +400,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver // below a received group only costs the early boundary. test(`a SUBSCRIBE_END below a group already received ${version === Version.DRAFT_05 ? "finishes clean" : "aborts the track"}`, async () => { const { subscriber, reader, respond, fin } = await subscribed(version); - await respond({ start: new SubscribeStart(0) }); + await respond({ start: new SubscribeStart(0, undefined, 0) }); const group = groupStream(subscriber, 3); group.finish(); await group.handled; @@ -417,7 +417,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver test("lite-07 settles a skipped group with SUBSCRIBE_DROP", async () => { const { subscriber, reader, respond, fin } = await subscribed(Version.DRAFT_07, Milli(60_000)); - await respond({ start: new SubscribeStart(0) }); + await respond({ start: new SubscribeStart(0, undefined, 0) }); for (const sequence of [0, 2]) { const group = groupStream(subscriber, sequence); group.write(`${sequence}.0`); @@ -436,7 +436,7 @@ test("lite-07 settles a skipped group with SUBSCRIBE_DROP", async () => { test("lite-07 settles a dropped range without grace", async () => { const { reader, respond, fin } = await subscribed(Version.DRAFT_07, Milli(60_000)); - await respond({ start: new SubscribeStart(0) }); + await respond({ start: new SubscribeStart(0, undefined, 0) }); await respond({ end: new SubscribeEnd(3) }); await respond({ drop: new SubscribeDrop({ start: 0, end: 2, error: 0 }) }); await fin(); @@ -449,7 +449,7 @@ test("lite-07 settles a dropped range without grace", async () => { for (const started of [false, true]) { test(`a bare FIN ${started ? "after SUBSCRIBE_START" : "without responses"} aborts the track`, async () => { const { reader, respond, fin } = await subscribed(Version.DRAFT_05); - if (started) await respond({ start: new SubscribeStart(0) }); + if (started) await respond({ start: new SubscribeStart(0, undefined, 0) }); await fin(); expect(await reader.closed).toBeInstanceOf(ProtocolViolation); await expect(reader.recvGroup()).rejects.toThrow(ProtocolViolation); @@ -467,7 +467,7 @@ test("a subscribe stream reset preserves the publisher's failure", async () => { test("lite-07 FIN waits for the skipped and reset final range to settle", async () => { const { subscriber, reader, respond, fin, subscriberFin } = await subscribed(Version.DRAFT_07, Milli(60_000)); - await respond({ start: new SubscribeStart(0) }); + await respond({ start: new SubscribeStart(0, undefined, 0) }); const first = groupStream(subscriber, 0); first.finish(); await first.handled; @@ -492,7 +492,7 @@ test("lite-07 FIN waits for the skipped and reset final range to settle", async test("draft-07's SUBSCRIBE_START says the largest before any group arrives", async () => { const { reader, respond } = await subscribed(Version.DRAFT_07); - await respond({ start: new SubscribeStart(4, { group: 4, frame: 2 }) }); + await respond({ start: new SubscribeStart(4, { group: 4, frame: 2 }, 0) }); expect(await reader.largest()).toEqual({ group: 4, frame: 2 }); reader.close(); }); diff --git a/js/net/src/lite/track_stream.test.ts b/js/net/src/lite/track_stream.test.ts index b934dc0eec4..bdfb22cf807 100644 --- a/js/net/src/lite/track_stream.test.ts +++ b/js/net/src/lite/track_stream.test.ts @@ -61,7 +61,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver const fin = track.reader.done(); expect(await settlesWithin(fin, 50)).toBe(false); - await encodeSubscribeResponse(sub.writer, { start: new SubscribeStart(0) }, version); + await encodeSubscribeResponse(sub.writer, { start: new SubscribeStart(0, undefined, 0) }, version); expect(await fin).toBe(true); reader.close(); diff --git a/test/interop/bare-fin.ts b/test/interop/bare-fin.ts index 2448935fe64..5fb9302d590 100644 --- a/test/interop/bare-fin.ts +++ b/test/interop/bare-fin.ts @@ -52,7 +52,7 @@ if (version !== undefined) { assert(peer); assert.equal(await peer.reader.u53(), StreamId.Subscribe); await Subscribe.decode(peer.reader, version); - if (input.started) await encodeSubscribeResponse(output, { start: new SubscribeStart(0) }, version); + if (input.started) await encodeSubscribeResponse(output, { start: new SubscribeStart(0, undefined, 0) }, version); if (input.clean) await encodeSubscribeResponse(output, { end: new SubscribeEnd(0) }, version); } else { assert.equal(input.version, "moqt-18"); From 34718ececab20b0b1ef4f10cf0426f8015587311 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 29 Oct 2026 09:43:59 -0700 Subject: [PATCH 08/11] docs(quest): record the released compatibility prerequisite Co-Authored-By: GPT-6.1 Sol --- quest/m1/start-budget-update.md | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/quest/m1/start-budget-update.md b/quest/m1/start-budget-update.md index af6f8f86ed1..22367a0ac53 100644 --- a/quest/m1/start-budget-update.md +++ b/quest/m1/start-budget-update.md @@ -25,11 +25,17 @@ the loosest sent budget when provenance is absent. No update revisions or per-update STARTs are added; range responses and start bookkeeping retain their own scope and branches. -Remaining: complete `just ci`, `just test interop --all`, and published-wire -compatibility validation; resolve relevant failures, then remove this quest -and mark the PR ready. The deterministic crossed-update tests, direct and -relayed intermediate-budget history-fill test, and exact published-wire -fixtures are implemented. Do not merge, enqueue, or watch this PR. +The crossed-update Rust/JS tests, direct and relayed intermediate-budget +history-fill test, and exact published-wire fixtures pass. `just drafts check` +and all 173 `just test interop --all` checks pass. + +Remaining: finish the local CI gate and published-release compatibility +validation, then remove this quest and mark ready. `just test wire-compat` +stops before its session cases because main's accepted pins still name +moq-cli 0.14.8 instead of published 0.14.9. The existing +[wire compatibility prerequisite](https://github.com/moq-dev/moq/pull/7061) +owns the blocked remeasurement; this change does not repin releases. +Do not merge, enqueue, or watch this PR. ## Related From 67972ed17057a8c572953a0a0962c3bdca0cbc46 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 29 Oct 2026 10:05:22 -0700 Subject: [PATCH 09/11] docs(quest): checkpoint START validation blockers Co-Authored-By: GPT-6.1 Sol --- quest/m1/start-budget-update.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/quest/m1/start-budget-update.md b/quest/m1/start-budget-update.md index 22367a0ac53..ce78f90f010 100644 --- a/quest/m1/start-budget-update.md +++ b/quest/m1/start-budget-update.md @@ -29,8 +29,13 @@ The crossed-update Rust/JS tests, direct and relayed intermediate-budget history-fill test, and exact published-wire fixtures pass. `just drafts check` and all 173 `just test interop --all` checks pass. -Remaining: finish the local CI gate and published-release compatibility -validation, then remove this quest and mark ready. `just test wire-compat` +Remaining: complete the local CI gate and published-release compatibility +validation, then remove this quest and mark ready. All code, feature, and +static checks passed; the workspace test run passed 8,525 tests, then stopped +on two unchanged moq-uring UDP tests whose worker setup exhausted this user's +8 MiB `RLIMIT_MEMLOCK`. 472 selected tests were not run after fail-fast. +Validate on a host with sufficient io-uring locked memory; do not weaken or +disable those assertions. `just test wire-compat` stops before its session cases because main's accepted pins still name moq-cli 0.14.8 instead of published 0.14.9. The existing [wire compatibility prerequisite](https://github.com/moq-dev/moq/pull/7061) From 220d4047a50aeac97b0525282de0009b08a767f4 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 30 Oct 2026 22:46:56 -0700 Subject: [PATCH 10/11] fix(moq-net): keep history below a resolved start fillable Design change: instead of echoing the selecting Subscriber Max Age in lite-07 START, a relay copy no longer infers missing history from the start's budget. It records only that the publisher resolved the start; every reader judges the history below it by its own budget and asks a fill, which ends empty when upstream holds nothing. The START field is removed from Rust, JS, and the draft. The crossed tightening integration test stays as the regression proof. Co-Authored-By: Claude Opus 5.5 --- quest/m1/README.md | 2 +- quest/m1/start-budget-update.md | 61 +++++++------------ rs/moq-net/src/lite/subscriber.rs | 11 +--- rs/moq-net/src/model/track.rs | 40 ++++++------ rs/moq-net/src/model/track/miss.rs | 26 +++----- .../tests/integration/widened_max_age.rs | 7 ++- 6 files changed, 55 insertions(+), 92 deletions(-) diff --git a/quest/m1/README.md b/quest/m1/README.md index bababeef4e1..001124765da 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -138,7 +138,7 @@ removed). - [Announcement takeover](/quest/m1/announce-takeover.md) - a new independent route explicitly continues instances served through its predecessor, with shared Rust/JS semantics and lite-07 wire - [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - a lite-07 SUBSCRIBE asks for past and live ranges with absolute or relative ends, in either order, and replaces FETCH; merged ranges never starve a subscriber; relays fill misses by range, including over moq-transport - [Drop set](/quest/m1/drop-set.md) - each cursor judges missing groups against its own budget, so a narrow reader drops promptly and a widening re-offers groups already reported dropped on lite-07 -- [Bind a resolved start to its subscriber budget](/quest/m1/start-budget-update.md) - bind unpublished-wire START to its selecting budget so later readers can fill history; published encodings stay unchanged +- [A tightened SUBSCRIBE keeps history below the start fillable](/quest/m1/start-budget-update.md) - a relay copy treats history below a resolved start as fillable instead of inferring it from the start's budget, so an update crossing START cannot hide it - [Broadcast archives](/quest/m1/archive/README.md) - ANNOUNCE, TRACKS, and a SUBSCRIBE per track record every track under a prefix to any object_store, a host serves a stored broadcast back, and the catalog drops its archive pointer for a per-rendition `hls` field - [HLS recorder target](/quest/m1/hls-record-target.md) - the moq-cli HLS recorder takes its target duration from the declared `hls` duration - [Lite-07 padding datagram](/quest/m1/lite07-padding-datagram.md) - lite-07 defines a padding datagram that receivers discard, so browser probes can pad below media diff --git a/quest/m1/start-budget-update.md b/quest/m1/start-budget-update.md index ce78f90f010..7bcc48bb8b3 100644 --- a/quest/m1/start-budget-update.md +++ b/quest/m1/start-budget-update.md @@ -1,49 +1,32 @@ -# [M] Bind a resolved start to its subscriber budget +# [S] A tightened SUBSCRIBE keeps history below the start fillable ## Goal -A relay copy knows the subscriber `max_delay` used to select its upstream -start, so a later reader with a looser budget fills the history it can use. -The response proves the selecting budget even when SUBSCRIBE_UPDATE crosses -it in flight. Published wires retain their encodings and conservative -ambiguous-budget behavior. +A relay copy never infers which history is missing from the budget its +publisher resolved SUBSCRIBE_START under. If a SUBSCRIBE_UPDATE crosses the +start in flight, a later reader whose budget reaches below the start still +fills that history from upstream. No wire changes. ## Plan -Decided 2026-10-29 (explicit implementation brief under `/afk`): adopt the -draft's recommended unpublished-wire scope, superseding the earlier -investigation-only brief. Lite-06's group-only response cannot prove which -update selected its start, so this cannot recover that history on a -published wire. - -Implemented in [moq-dev/moq#7235](https://github.com/moq-dev/moq/pull/7235): -lite-07 START echoes Subscriber Max Age after its existing fields in Rust -and JS. The publisher captures it with the selected group, retaining both -while waiting for upstream START. Rust uses it in the existing budget-aware -fill ledger. Published responses omit it, and Rust conservatively records -the loosest sent budget when provenance is absent. No update revisions or -per-update STARTs are added; range responses and start bookkeeping retain -their own scope and branches. - -The crossed-update Rust/JS tests, direct and relayed intermediate-budget -history-fill test, and exact published-wire fixtures pass. `just drafts check` -and all 173 `just test interop --all` checks pass. - -Remaining: complete the local CI gate and published-release compatibility -validation, then remove this quest and mark ready. All code, feature, and -static checks passed; the workspace test run passed 8,525 tests, then stopped -on two unchanged moq-uring UDP tests whose worker setup exhausted this user's -8 MiB `RLIMIT_MEMLOCK`. 472 selected tests were not run after fail-fast. -Validate on a host with sufficient io-uring locked memory; do not weaken or -disable those assertions. `just test wire-compat` -stops before its session cases because main's accepted pins still name -moq-cli 0.14.8 instead of published 0.14.9. The existing -[wire compatibility prerequisite](https://github.com/moq-dev/moq/pull/7061) -owns the blocked remeasurement; this change does not repin releases. -Do not merge, enqueue, or watch this PR. +Decided 2026-10-31 (maintainer, rethink of +[moq-dev/moq#7235](https://github.com/moq-dev/moq/pull/7235)): drop the +history inference instead of adding a lite-07 START field. The earlier +attempt echoed the selecting Subscriber Max Age in START so the relay could +record which budget skipped the groups below it; that field is removed from +Rust, JS, and the draft. + +The copy now records only that its start was resolved by the publisher. +Every group below it stays fillable: each reader judges a hole by its own +budget against the copy's media time, and a fill that finds nothing ends +empty. Lite-06 copies keep relying on the publisher's re-offer, since their +fills cannot carry a budget. + +Keep the direct and relayed integration test: tighten before the start is +selectable, then an intermediate-budget reader fills the history between +the two starts while the publisher is paused. ## Related - [Drop set](/quest/m1/drop-set.md) - records the budget each drop was judged under -- [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - owns the resolved-start response for ranges -- [Delete the old start bookkeeping](/quest/m1/subscribe-ranges/start-bookkeeping.md) - moves start state into the model and considers per-update responses +- [Delete the old start bookkeeping](/quest/m1/subscribe-ranges/start-bookkeeping.md) - moves where a subscription starts into the model diff --git a/rs/moq-net/src/lite/subscriber.rs b/rs/moq-net/src/lite/subscriber.rs index 8dd35ae830f..8cfdc210b6a 100644 --- a/rs/moq-net/src/lite/subscriber.rs +++ b/rs/moq-net/src/lite/subscriber.rs @@ -2016,7 +2016,6 @@ mod tests { .unwrap(), id: 0, max_delay: Duration::ZERO, - requested_max_delay: Duration::ZERO, start: None, priority: 0, requested: None, @@ -2191,7 +2190,6 @@ mod tests { .unwrap(), id: 0, max_delay: Duration::ZERO, - requested_max_delay: Duration::ZERO, start: Some(floor), priority: 0, requested: Some(floor), @@ -2656,7 +2654,6 @@ mod tests { stream, id: 0, max_delay: Duration::ZERO, - requested_max_delay: Duration::ZERO, start: Some(2), priority: 0, requested: Some(2), @@ -6282,9 +6279,6 @@ struct SubStream { /// start sits elsewhere the request-tracked floor stands instead, and demand /// returning here makes the declaration valid again. requested: Option, - /// The max delay the SUBSCRIBE itself carried, fixed for the stream's life: a lite-06+ - /// SUBSCRIBE_START resolves the start from it, skipping older groups as stale. - requested_max_delay: Duration, /// The groups received for this subscription, shared with its [`TrackEntry`]. tail: kio::Producer, /// The first group the publisher serves (SUBSCRIBE_START), once declared. @@ -7030,7 +7024,6 @@ impl Establish { stream, id: self.id, max_delay: wire_max_delay(self.subscription.max_delay), - requested_max_delay: wire_max_delay(self.subscription.max_delay), start: self.subscription.start(), ranges: self.ranges.clone(), priority: self.subscription.priority, @@ -7954,9 +7947,7 @@ impl ServeLoop { // request-tracked floor stands rather than a guess. if active.start == active.requested { let _ = match (serve.subscriber.version.resolves_start(), served) { - (true, Some(served)) => { - self.serving.start_resolved(served, active.requested_max_delay) - } + (true, Some(served)) => self.serving.start_resolved(served), _ => self.serving.start_at(served), }; } diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index da4102ad639..312557a9cfe 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -435,9 +435,9 @@ pub(crate) struct TrackState { // [`Subscriber::poll_start`]) wait on it; everything else treats the floor as usual. start_pending: bool, - // The budget the publisher resolved `start_sequence` from (a lite-06+ SUBSCRIBE_START), - // skipping every older group as stale under it. Cleared by any other declaration. - start_budget: Option, + // Whether the publisher resolved `start_sequence` (a lite-06+ SUBSCRIBE_START), which + // may sit above history a reader can still fill. Cleared by any other declaration. + start_resolved: bool, // A copy received from a remote publisher, whose declared end waits for groups still // in flight below it (see `tail`), and whose readers' relative and time anchors are @@ -2343,7 +2343,7 @@ impl TrackState { self.latch_settled(); self.start_sequence = start_sequence; self.start_pending = pending; - self.start_budget = None; + self.start_resolved = false; if let Some(start) = start_sequence { self.cache.wakes().arrived(..start); let now = self.cache.pool().now_wall(); @@ -3317,13 +3317,14 @@ impl Producer { Ok(()) } - /// [`Self::start_at`] a start the publisher resolved from `budget`: every older group - /// is stale under it, so a reader whose budget is no looser gives them up unasked. + /// [`Self::start_at`] a start the publisher resolved from its subscriber's budget. An + /// update can cross the response, so that budget is unknown: every older group stays + /// fillable, and a fill that finds none ends empty. #[cfg_attr(not(feature = "async"), allow(dead_code, reason = "async-only"))] - pub(crate) fn start_resolved(&mut self, sequence: u64, budget: Duration) -> Result<()> { + pub(crate) fn start_resolved(&mut self, sequence: u64) -> Result<()> { let mut state = self.modify()?; state.set_start(Some(sequence), false); - state.start_budget = Some(budget); + state.start_resolved = true; Ok(()) } @@ -18098,29 +18099,26 @@ mod test { assert!(dynamic.poll_requested_fill(&waiter).is_pending()); } - /// Below a start its publisher resolved from a budget, a reader no looser than it gives - /// every group up unasked, while a looser one may still reach some, so it asks. + /// Below a start its publisher resolved, a reader asks for the history its budget + /// reaches: an update may have crossed the start, so the copy cannot tell which budget + /// skipped those groups, and a fill that finds none ends empty. #[test] #[cfg(feature = "async")] - fn below_a_start_resolved_from_a_budget_only_a_looser_reader_asks() { + fn below_a_resolved_start_a_reader_asks() { let mut producer = track_producer("test", Info::timed()); let dynamic = producer.requests(); - let mut narrow = producer.subscribe(replay()); - let mut wide = producer.subscribe(Subscription::default().with_max_delay(Duration::from_secs(60))); - producer.start_resolved(2, Duration::from_secs(30)).unwrap(); + let mut reader = producer.subscribe(replay()); + producer.start_resolved(2).unwrap(); write_group(&producer, 2); write_group(&producer, 3); let waiter = kio::Waiter::noop(); - assert_eq!(drain_groups(&mut narrow), [2, 3]); - assert_eq!(drained_drops(&mut narrow), [(0, 2)]); + assert_eq!(drain_groups(&mut reader), [2, 3]); assert!( - dynamic.poll_requested_fill(&waiter).is_pending(), - "the narrow reader asks nothing" + drained_drops(&mut reader).is_empty(), + "nothing below the start is given up unasked" ); - - assert_eq!(drain_groups(&mut wide), [2, 3]); let Poll::Ready(Ok(fill)) = dynamic.poll_requested_fill(&waiter) else { - panic!("the wide reader asks below the start"); + panic!("the reader asks below the start"); }; assert_eq!((fill.start().group, fill.end()), (0, 2)); } diff --git a/rs/moq-net/src/model/track/miss.rs b/rs/moq-net/src/model/track/miss.rs index 6bc2dc064ec..814682ef17e 100644 --- a/rs/moq-net/src/model/track/miss.rs +++ b/rs/moq-net/src/model/track/miss.rs @@ -508,20 +508,12 @@ impl Misses { self.judged.insert(run.clone()); } let start = state.start_sequence.unwrap_or(0); - let (stale_below, unasked) = match state.start_budget { - // Its publisher skipped every older group as stale under a budget no tighter - // than this reader's, so they are stale for it too. - Some(budget) if drift.budget <= budget => (start, false), - // Such a copy cannot tell a group a looser reader's budget reaches, so it leaves - // them to the feed's re-offer. - Some(_) => (0, state.fills_from_start), - // The feed began at the live edge or the floor the copy asked for, so older - // groups are history only a reader naming them asked for. - None => ( - 0, - state.fills_from_start || state.received && named.is_none_or(|named| named >= start), - ), - }; + // Below a start its publisher resolved from a budget, which an update may have changed + // in flight, every older group is history a fill may still find. A feed that began at + // the live edge or the floor the copy asked for leaves older groups as history only a + // reader naming them asked for. + let unasked = state.fills_from_start + || !state.start_resolved && state.received && named.is_none_or(|named| named >= start); if unasked && start > 0 { candidates.remove(0..start); } @@ -585,10 +577,8 @@ impl Misses { // budget is stale throughout. let stale: Vec> = asks .iter() - .filter_map(|run| match state.drifted(run.end - 1, &drift.edge, drift.budget) { - true => Some(run.clone()), - false => Some(run.start..run.end.min(stale_below)).filter(|run| !run.is_empty()), - }) + .filter(|run| state.drifted(run.end - 1, &drift.edge, drift.budget)) + .cloned() .collect(); for run in stale { asks.remove(run.clone()); diff --git a/rs/moq-net/tests/integration/widened_max_age.rs b/rs/moq-net/tests/integration/widened_max_age.rs index af468b0fb6d..d3df5f87ee0 100644 --- a/rs/moq-net/tests/integration/widened_max_age.rs +++ b/rs/moq-net/tests/integration/widened_max_age.rs @@ -129,7 +129,8 @@ async fn received(subscriber: &mut track::Subscriber) -> BTreeSet { /// A fresh reader asks no fill for the history below where its feed starts, through a relay /// or not: below a start resolved from a budget no tighter than its own, every group is -/// stale, and below a live edge start it names no floor under, none was asked for. +/// stale by its media time, and below a live edge start it names no floor under, none was +/// asked for. #[moq_clock::test] async fn a_fresh_reader_fills_nothing_below_its_start() { for (version, want) in [ @@ -155,7 +156,7 @@ async fn a_fresh_reader_fills_nothing_below_its_start() { /// Tighten while no start is selectable, then ask for a budget between the old and /// selected ones. Hold the publisher's driver so only an actual fill request, rather -/// than its live re-offer, can prove the copy retained the selected budget. +/// than its live re-offer, can prove the copy left the history below its start fillable. #[moq_clock::test] async fn a_tightened_start_leaves_history_fillable_by_an_intermediate_reader() { for relay in [false, true] { @@ -188,7 +189,7 @@ async fn a_tightened_start_leaves_history_fillable_by_an_intermediate_reader() { let upstream = &chain.links[0].client_transport; moq_clock::timeout(TIMEOUT, upstream.bidi_count(3, before + 1)) .await - .expect("the proven tighter start allows fills"); + .expect("the history below the start is fillable"); *chain.source_gate.write().ok().unwrap() = true; // This reader can also receive the feed's re-offer once the source resumes. assert_eq!(reading.await.unwrap(), (4..GROUPS).collect(), "relay={relay}"); From 57911aa07c551cd85210edd0d77f93144721ba1d Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 30 Oct 2026 23:44:40 -0700 Subject: [PATCH 11/11] quest: finish the start budget quest Co-Authored-By: Claude Opus 5.5 --- quest/m1/README.md | 1 - quest/m1/start-budget-update.md | 32 -------------------------------- 2 files changed, 33 deletions(-) delete mode 100644 quest/m1/start-budget-update.md diff --git a/quest/m1/README.md b/quest/m1/README.md index 001124765da..5e5d57b9249 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -138,7 +138,6 @@ removed). - [Announcement takeover](/quest/m1/announce-takeover.md) - a new independent route explicitly continues instances served through its predecessor, with shared Rust/JS semantics and lite-07 wire - [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - a lite-07 SUBSCRIBE asks for past and live ranges with absolute or relative ends, in either order, and replaces FETCH; merged ranges never starve a subscriber; relays fill misses by range, including over moq-transport - [Drop set](/quest/m1/drop-set.md) - each cursor judges missing groups against its own budget, so a narrow reader drops promptly and a widening re-offers groups already reported dropped on lite-07 -- [A tightened SUBSCRIBE keeps history below the start fillable](/quest/m1/start-budget-update.md) - a relay copy treats history below a resolved start as fillable instead of inferring it from the start's budget, so an update crossing START cannot hide it - [Broadcast archives](/quest/m1/archive/README.md) - ANNOUNCE, TRACKS, and a SUBSCRIBE per track record every track under a prefix to any object_store, a host serves a stored broadcast back, and the catalog drops its archive pointer for a per-rendition `hls` field - [HLS recorder target](/quest/m1/hls-record-target.md) - the moq-cli HLS recorder takes its target duration from the declared `hls` duration - [Lite-07 padding datagram](/quest/m1/lite07-padding-datagram.md) - lite-07 defines a padding datagram that receivers discard, so browser probes can pad below media diff --git a/quest/m1/start-budget-update.md b/quest/m1/start-budget-update.md deleted file mode 100644 index 7bcc48bb8b3..00000000000 --- a/quest/m1/start-budget-update.md +++ /dev/null @@ -1,32 +0,0 @@ -# [S] A tightened SUBSCRIBE keeps history below the start fillable - -## Goal - -A relay copy never infers which history is missing from the budget its -publisher resolved SUBSCRIBE_START under. If a SUBSCRIBE_UPDATE crosses the -start in flight, a later reader whose budget reaches below the start still -fills that history from upstream. No wire changes. - -## Plan - -Decided 2026-10-31 (maintainer, rethink of -[moq-dev/moq#7235](https://github.com/moq-dev/moq/pull/7235)): drop the -history inference instead of adding a lite-07 START field. The earlier -attempt echoed the selecting Subscriber Max Age in START so the relay could -record which budget skipped the groups below it; that field is removed from -Rust, JS, and the draft. - -The copy now records only that its start was resolved by the publisher. -Every group below it stays fillable: each reader judges a hole by its own -budget against the copy's media time, and a fill that finds nothing ends -empty. Lite-06 copies keep relying on the publisher's re-offer, since their -fills cannot carry a budget. - -Keep the direct and relayed integration test: tighten before the start is -selectable, then an intermediate-budget reader fills the history between -the two starts while the publisher is paused. - -## Related - -- [Drop set](/quest/m1/drop-set.md) - records the budget each drop was judged under -- [Delete the old start bookkeeping](/quest/m1/subscribe-ranges/start-bookkeeping.md) - moves where a subscription starts into the model