diff --git a/doc/concept/moq-lite.md b/doc/concept/moq-lite.md index a99b376019a..0356329b249 100644 --- a/doc/concept/moq-lite.md +++ b/doc/concept/moq-lite.md @@ -140,10 +140,12 @@ its scope can use. ## Subscriptions -A subscriber names a broadcast and track. Delivery starts at the oldest group -it can still use, which at the default budget is the latest one, so every group -must begin at a point a fresh subscriber can decode from (a keyframe, a full -JSON snapshot). Groups can also be fetched by sequence number, which is how the +A subscriber names a broadcast and track. By default delivery starts at the +live edge: the oldest group it can still use, which at the default budget is the +latest one, so every group must begin at a point a fresh subscriber can decode +from (a keyframe, a full JSON snapshot). A resumed subscriber names a floor +instead, where it left off, and can ask for both, in which case the lower start +wins. Groups can also be fetched by sequence number, which is how the [HLS gateway](/bin/hls) and the relay's [HTTP fetch](/bin/relay/http) serve history. @@ -152,7 +154,6 @@ Each subscription carries the knobs that decide behavior under congestion: | Knob | Effect | | --- | --- | | **Priority** (0..255) | Higher-priority tracks get bandwidth first. Audio above video, base layer above enhancement. | -| **Order** | Which group to send first when several are pending. Newest first for live, oldest first for catch-up. | | **Max delay** | How far a non-latest group may fall behind the live edge before it is skipped. Zero means "live edge only", and raising it is also what asks for history. | Max delay is measured on the media timeline, not the wall clock, so a backlog @@ -163,8 +164,7 @@ publisher only ever sees the most tolerant budget across its subscribers. A track is timed or untimed: it declares a timescale or none, and a frame that doesn't match is refused. No receiver fills in an arrival time. An untimed -group never falls past max delay, so a new subscriber starts at its latest -group unless it names a start. Tracks from moq-lite before 05, or from +group never falls past max delay, so its live edge is the latest group. Tracks from moq-lite before 05, or from moq-transport without a `TIMESCALE` (every track on drafts 14-16), arrive untimed. No moq-lite version can declare an untimed track, so on 05 and later its frames go out stamped with their send time. @@ -183,10 +183,10 @@ timestamps and always keeps the newest group. IETF carries the same idea as Put together, a conference might use: -| Track | Priority | Order | Max delay | -| --- | --- | --- | --- | -| audio | 100 | ascending | 500 ms | -| video | 50 | descending | 2 s | +| Track | Priority | Max delay | +| --- | --- | --- | +| audio | 100 | 500 ms | +| video | 50 | 2 s | Under light congestion video drops the tail of a group; under heavy congestion video stops and audio lags by at most 500 ms. No protocol change, just knobs. diff --git a/drafts/draft-lcurley-moq-lite.md b/drafts/draft-lcurley-moq-lite.md index 0ad17479a76..1c551fe4b78 100644 --- a/drafts/draft-lcurley-moq-lite.md +++ b/drafts/draft-lcurley-moq-lite.md @@ -140,7 +140,7 @@ When a new Group is started, the previous Group is closed and may be dropped for The duration before an incomplete group is dropped is determined by the application and the publisher/subscriber's latency target. Every subscription is scoped to a single Track. -A subscription starts at a configurable Group (defaulting to the latest) and continues until a configurable end Group or until either the publisher or subscriber cancels the subscription. +A subscription starts at the live edge, at a floor Group, or at whichever of the two is lower, and continues until a configurable end Group or until either the publisher or subscriber cancels the subscription. Both bounds may be refined to a Frame within their Group, so a subscription can start or stop partway through a Group rather than only on a Group boundary (see [Positions](#positions)). The subscriber and publisher both indicate their delivery preference: @@ -469,7 +469,7 @@ A subscriber opens Subscribe Streams to request a Track. The subscriber MUST start a Subscribe Stream with a SUBSCRIBE message followed by any number of SUBSCRIBE_UPDATE messages. The publisher replies with a SUBSCRIBE_OK message once the start group is resolved, followed by a SUBSCRIBE_END message once the subscription ends. For a live track the publisher MAY withhold SUBSCRIBE_OK until the first matching group resolves the start; if the track has already ended with no matching groups, it sends SUBSCRIBE_END with no preceding SUBSCRIBE_OK. -A requested start past the publisher's largest position is the exception: the publisher MUST send SUBSCRIBE_OK without waiting, with `Group` set to the requested start group, so a subscriber resuming just past what it holds learns where the live feed is from the answer alone (a quiet track may not reach the start for a long time). +A floor past the publisher's largest position without `Live` is the exception: the publisher MUST send SUBSCRIBE_OK without waiting, with `Group` set to the floor's group, so a subscriber resuming just past what it holds learns where the live feed is from the answer alone (a quiet track may not reach the floor for a long time). A rejection is a stream reset: a publisher that cannot serve the subscription (no such track, an ended broadcast, or any other refusal) MUST promptly reset the stream rather than leave it pending, so a subscriber distinguishes "pending" from "refused" by the reset, not by a timeout. A route claims capability rather than inventory, so a subscription for a covered path that names nothing is refused this way too. @@ -1026,9 +1026,11 @@ SUBSCRIBE Message { Track Name (s) Subscriber Priority (8) Subscriber Max Age (i) - Group Start (i) + Live (8) + Floor (8) + [Group Start (i)] + [Frame Start (i)] Group End (i) - Frame Start (i) Frame End (i) } ~~~ @@ -1050,14 +1052,25 @@ A value of `0` means the subscriber wants only the latest group in live delivery This is a delivery-time preference, not a retention rule: the publisher MAY still hold these groups for FETCH or future subscriptions (see `Publisher Max Age` in [TRACK_INFO](#track-info)). See the [Expiration](#expiration) section for more information. +**Live**: +1 to deliver from the live edge, 0 otherwise. +Any other value is a PROTOCOL_VIOLATION. +The live edge is frame 0 of the oldest group that [Expiration](#expiration) has not expired, so a `Subscriber Max Age` of 0 starts at the latest group, and a subscriber that buffers is handed the head of what it can still play. +On a Track without presentation time, which no age can measure, it is frame 0 of the latest group. +`Live` lowers the floor to the live edge whenever that Position sorts below it, so a floor above the latest group never hides the latest group from a subscription that also asks for `Live`. + +**Floor**: +1 when `Group Start` and `Frame Start` follow, 0 when the subscription has no floor. +Any other value is a PROTOCOL_VIOLATION. +A subscription with neither `Live` nor a floor asks for nothing, and the publisher MUST treat it as a PROTOCOL_VIOLATION. + **Group Start**: -The minimum group sequence to deliver: an absolute floor, defaulting to 0 (no floor). +The minimum group sequence to deliver: an absolute floor. A floor is not a request; `Subscriber Max Age` is the only thing that asks for data. -The publisher SHOULD start at the oldest group at or above the floor that [Expiration](#expiration) has not expired, and MUST NOT deliver an older one. -A `Subscriber Max Age` of 0 therefore starts at the latest group, since every older group is already stale. -A subscriber that buffers is then handed the head of what it can still play instead of only the live edge, and is never sent history it would discard on arrival: the same bound decides what to start at and what to expire, so the two cannot disagree. -A floor above the latest group simply waits there: that is a resumed subscription naming where it left off. +The publisher SHOULD start at the oldest group at or above the floor that [Expiration](#expiration) has not expired, and MUST NOT deliver an older one unless `Live` lowers the floor. +The subscriber is then never sent history it would discard on arrival: the same bound decides what to start at and what to expire, so the two cannot disagree. +Without `Live`, a floor above the latest group simply waits there: that is a resumed subscription naming where it left off. Reaching back is best-effort, not a guarantee that the groups still exist; see `Publisher Max Age` in [TRACK_INFO](#track-info). **Group End**: @@ -1067,8 +1080,8 @@ A non-zero value is the absolute group sequence + 1. **Frame Start**: The index of the first frame to deliver within the `Group Start` group (see [Positions](#positions)). -A value of 0 means from the start of that group (default), so the group is delivered whole. -Frames before this index are not delivered, even when delivery begins at that exact group; a start resolved at a later group is always delivered from frame 0. +A value of 0 means from the start of that group, so the group is delivered whole. +Frames before this index are not delivered, even when delivery begins at that exact group; a start resolved at a later group, or lowered by `Live`, is always delivered from frame 0. A subscriber that has received no group MUST send 0, since it cannot number the frames of a group it has not seen. **Frame End**: @@ -1077,7 +1090,7 @@ A value of 0 means the whole group (default). A non-zero value is the absolute frame index + 1, matching `Group End`. MUST be 0 when `Group End` is 0, since an unbounded subscription has no end group to qualify. -`Group Start` and `Group End` are offset by 1 only so 0 can mean "absent"; every other group field in this document is a plain absolute sequence. +`Group End` is offset by 1 only so 0 can mean "absent"; every other group field in this document is a plain absolute sequence. ## SUBSCRIBE_UPDATE A subscriber can modify a subscription with a SUBSCRIBE_UPDATE message. @@ -1089,14 +1102,17 @@ SUBSCRIBE_UPDATE Message { Message Length (i) Subscriber Priority (8) Subscriber Max Age (i) - Group Start (i) + Live (8) + Floor (8) + [Group Start (i)] + [Frame Start (i)] Group End (i) - Frame Start (i) Frame End (i) } ~~~ See [SUBSCRIBE](#subscribe) for information about each field. +Every field replaces the subscription's: an update clears the floor with `Floor` 0 and the live edge with `Live` 0, and clearing both is a PROTOCOL_VIOLATION. Moving `Frame Start` forward within a group that is already being delivered does not retract frames the publisher has already sent; like `Group Start`, it only bounds what the publisher sends from here on. @@ -1187,13 +1203,13 @@ Set to 0x0 to indicate a SUBSCRIBE_OK message. **Group**: The absolute sequence number of the first group that will be delivered. -It MUST be greater than or equal to the requested start group; any groups in between are unavailable. -A subscriber that requested the latest group learns the resolved sequence here. +It MUST be greater than or equal to the floor's group, unless `Live` lowered it; any groups in between are unavailable. +A subscriber that requested `Live` learns the resolved sequence here. There is no matching frame field, because the start frame is never in doubt: a partial group is only delivered when it was asked for, so the subscription starts either exactly where it asked or at the beginning of a later group (see [Positions](#positions)). The subscriber derives the start frame from `Group` and its own request: -- `Group` equals the requested start group: delivery begins at the requested `Frame Start`. +- `Group` equals the floor's group, and `Live` did not lower the floor: delivery begins at the requested `Frame Start`. - `Group` is greater: delivery begins at frame 0. The second case is easy to get wrong, so to be explicit: a subscriber that requested group 5 frame 15 and receives `Group` = 6 starts at **frame 0** of group 6, not frame 15. @@ -1395,6 +1411,7 @@ The `Message Length` describes the payload size on the wire. - A GOAWAY recipient MAY keep opening requests on the current session until it has moved to a replacement session; previously it MUST NOT open new streams. The sender SHOULD keep answering them and MAY reset one with GOING_AWAY, which the recipient SHOULD retry on another route. - Replaced Warm and Cold Route Cost with one static Route Cost in moq-lite-07-wip; lite-06 retains both fields, reading Warm and writing Cold at saturation. - Made TRACK_INFO Publisher Max Age optional, encoded as milliseconds plus one with zero meaning no limit. +- Added `Live` and `Floor` to SUBSCRIBE and SUBSCRIBE_UPDATE. `Live` delivers from the live edge, lowering the floor when the oldest unexpired group sorts below it. `Group Start` and `Frame Start` are a plain absolute floor, present only when `Floor` is 1. A subscription with neither is a PROTOCOL_VIOLATION. lite-06 reads `Group Start` 0 with `Frame Start` 0 as `Live`, and earlier versions read an absent `Group Start` as `Live`. - Added `Largest Group` and `Largest Frame` to SUBSCRIBE_OK: the publisher's largest position when it answers, which a subscriber takes as where the live feed is. A publisher MUST answer at once when the requested start is past it. Earlier versions carry no such position, so a subscriber takes the first frame instead. - Added `Epoch` to ANNOUNCE_START, SUBSCRIBE, TRACK, and FETCH: a UUIDv7 naming the publisher instance, or empty. A path and an Epoch name one Broadcast. A relay MAY move a subscription between routes with the same Epoch, continuing from the first frame the subscriber lacks instead of at a group boundary, and never between routes with different Epochs or none. The newest Epoch wins a path and ends subscriptions to the older one. Replaces the first-hop identity. - An untimed Track's frames and datagrams carry their send time. lite-05 and lite-06 publishers do the same. diff --git a/js/net/src/broadcast.test.ts b/js/net/src/broadcast.test.ts index 02a93b52e19..00e69ad8fdd 100644 --- a/js/net/src/broadcast.test.ts +++ b/js/net/src/broadcast.test.ts @@ -300,6 +300,7 @@ test("a request exposes the aggregate subscription options", async () => { expect(request?.subscription).toEqual({ priority: 7, maxDelay: Milli(250), + live: false, groups: { start: { included: 0 }, end: { excluded: 30 } }, }); expect(request?.priority).toBe(7); diff --git a/js/net/src/ietf/publisher.test.ts b/js/net/src/ietf/publisher.test.ts index 80ae59bae26..aee5edeb985 100644 --- a/js/net/src/ietf/publisher.test.ts +++ b/js/net/src/ietf/publisher.test.ts @@ -1820,6 +1820,76 @@ test("draft-20: an absolute filter trims the range it serves", async () => { } }); +/** + * Relative 1 floors demand at the current group. It is not `live` (that asks a relay for + * the retained window) and not an unfiltered start at group 0. + */ +test("draft-20: relative 1 floors demand at the current group", async () => { + const fx = fixture(); + const track = fx.broadcast.createTrack("video", { timescale: Timescale.MILLI }); + for (let i = 0; i < 3; i++) writeGroup(track, 1); + + const { client } = await runSubscribe( + fx, + new Subscribe({ + requestId: 7n, + trackNamespace: Path.from("test"), + trackName: "video", + subscriberPriority: 0, + filter: { kind: "relative", groups: 1n }, + }), + ); + + try { + expect(track.subscription.peek()).toMatchObject({ + live: false, + groups: { start: { included: 2 } }, + }); + + const served = await nextUni(fx.uni); + if (!served) throw new Error("the current group was never served"); + expect(await readGroup(served)).toEqual({ + sequence: 2, + firstObject: true, + endOfGroup: true, + objects: [{ id: 0, payload: "2.0" }], + }); + } finally { + fx.close(); + client.close(); + } +}); + +/** + * Draft-20 unfiltered stays a floor at group 0, the retained cache, not `live`. + */ +test("draft-20: an unfiltered subscription floors demand at the origin", async () => { + const fx = fixture(); + const track = fx.broadcast.createTrack("video", { timescale: Timescale.MILLI }); + writeGroup(track, 1); + + const { client } = await runSubscribe( + fx, + new Subscribe({ + requestId: 7n, + trackNamespace: Path.from("test"), + trackName: "video", + subscriberPriority: 0, + filter: { kind: "unfiltered" }, + }), + ); + + try { + expect(track.subscription.peek()).toMatchObject({ + live: false, + groups: { start: { included: 0 } }, + }); + } finally { + fx.close(); + client.close(); + } +}); + /** * The draft's own current-group join: a Next Object subscription for the live tail, plus a * StartGroup=1 fill for the head already published. The two must meet exactly, so the head diff --git a/js/net/src/ietf/publisher.ts b/js/net/src/ietf/publisher.ts index 78723708fea..02e0d8d6adc 100644 --- a/js/net/src/ietf/publisher.ts +++ b/js/net/src/ietf/publisher.ts @@ -401,12 +401,17 @@ export class Publisher { // The wire request tells an upstream what we need; the cursor is what actually // trims this subscriber, since the producer fans every cached group out to every - // sink regardless. An absent start joins at the latest group, which is what - // moq-lite means by joining a live track. The range's end is the inclusive last - // group; the model's `endGroup` is exclusive. + // sink regardless. A resolved start is a floor, including relative 1 at the + // current group. That is not `live` (this serving budget would ask a relay for + // the retained window) and not an unfiltered {0, 0}. An absent start, a relative + // filter before any group exists, is still the live edge: there is no group to + // floor at yet. The range's end is the inclusive last group; the model's + // `endGroup` is exclusive. + const live = !range.start; track.update({ priority, maxDelay: Milli(Varint.MAX_U53), + live, groups: { start: range.start ? { included: Number(range.start.group) } : undefined, end: range.end ? { included: Number(range.end.group) } : undefined, @@ -1638,11 +1643,11 @@ function subscribeRange(msg: Subscribe, edge: LiveEdge, version: IetfVersion): S /** The Locations a single Location Filter selects, resolved against the live edge. */ function filterRange(filter: Filter.Filter, edge: LiveEdge): ServeRange { switch (filter.kind) { - // No restriction. moq-lite starts at the beginning of the latest group, which is the - // join point it is built around; a subscription passes objects as they are published, - // so an absent filter is not a request to replay history. + // Every Object, which the draft defines as an open ended absolute {0, 0}: the same + // start an explicit one asks for. A relay folding `live` into a floor sends it to + // reach back over what its readers may still use, and filters locally. case "unfiltered": - return {}; + return { start: { group: 0n, object: 0n } }; // `{Largest.Group, Largest.Object + 1}`. Everything below it, including the already // published head of the current group, is outside the requested range, so the join is // mid-group by construction. The draft pairs this with a fill when the subscriber diff --git a/js/net/src/integration.test.ts b/js/net/src/integration.test.ts index 11949553db6..ffdda64235b 100644 --- a/js/net/src/integration.test.ts +++ b/js/net/src/integration.test.ts @@ -171,8 +171,6 @@ test("integration: lite subscription options and updates reach the publisher", a })(); const remote = wireOf(client).consume(Path.from("test")); - // A floor of 1, not 0: a pre-06 wire folds a vacuous floor of 0 back to absent, since - // its encoding of group 0 means "replay from the beginning" instead. const subscriber = remote.track("video").subscribe({ priority: 3, maxDelay: Milli(250), @@ -182,6 +180,7 @@ test("integration: lite subscription options and updates reach the publisher", a expect(producer.subscription.peek()).toEqual({ priority: 3, maxDelay: Milli(250), + live: false, groups: { start: { included: 1 }, end: { excluded: 9 } }, }); @@ -194,6 +193,7 @@ test("integration: lite subscription options and updates reach the publisher", a expect(await updated).toEqual({ priority: 8, maxDelay: Milli(500), + live: false, groups: { start: { included: 2 }, end: { excluded: 12 } }, }); diff --git a/js/net/src/internal.ts b/js/net/src/internal.ts index 0fa1f7f7378..315aa07a093 100644 --- a/js/net/src/internal.ts +++ b/js/net/src/internal.ts @@ -31,6 +31,12 @@ export function groupBounds(groups: Groups = {}): { start: number; end?: number return { start: bound(groups.start, true) ?? 0, end: bound(groups.end, false) }; } +/** Where a read cursor starts: the floor, lowered by `live` to the live edge `edge`. */ +export function liveStart(live: boolean, floor: number | undefined, edge: number): number { + if (!live) return floor ?? 0; + return floor === undefined ? edge : Math.min(floor, edge); +} + /** * The announce-interest prefix a scope needs on a prefix-shaped wire: its literal head. * The peer echoes every suffix beneath it, and the caller filters what arrives. diff --git a/js/net/src/lite/connection.test.ts b/js/net/src/lite/connection.test.ts index 552ed2b5359..03f4157fda4 100644 --- a/js/net/src/lite/connection.test.ts +++ b/js/net/src/lite/connection.test.ts @@ -280,7 +280,7 @@ async function servedRawSubscription() { track: "video", priority: 0, maxDelay: 10_000, - startGroup: 0, + start: { live: false, startGroup: 0, startFrame: 0 }, }).encode(subscriber.writer, version); // Finish the track only once it is served: a track closed before anyone subscribed is gone. @@ -448,7 +448,7 @@ test("close waits for a request served while withdrawals are in flight", async ( track: "video", priority: 0, maxDelay: 10_000, - startGroup: 0, + start: { live: false, startGroup: 0, startFrame: 0 }, }).encode(subscriber.writer, version); expect("start" in (await decodeSubscribeResponse(subscriber.reader, version))).toBe(true); producer.close(); diff --git a/js/net/src/lite/publisher.test.ts b/js/net/src/lite/publisher.test.ts index e6643f8803b..89cb55ff010 100644 --- a/js/net/src/lite/publisher.test.ts +++ b/js/net/src/lite/publisher.test.ts @@ -563,7 +563,10 @@ async function servedSubscription( broadcast: Path.from("test"), track: "video", priority: 0, - startGroup: options.startGroup, + start: + options.startGroup === undefined + ? undefined + : { live: false, startGroup: options.startGroup, startFrame: 0 }, endGroup: options.endGroup, endFrame: options.endFrame, }); @@ -705,7 +708,10 @@ test("lite draft-06: a queued floor update applies before the next group pop", a await sub.parked; sub.serve(1); - await replayUpdate({ priority: 0, startGroup: 2 }).encode(sub.client.writer, Version.DRAFT_06); + await replayUpdate({ priority: 0, start: { live: false, startGroup: 2, startFrame: 0 } }).encode( + sub.client.writer, + Version.DRAFT_06, + ); sub.serve(2); await flush(); expect(ranges).toHaveBeenCalledTimes(1); @@ -731,7 +737,10 @@ test("lite draft-06: a widening update lowers the serving floor", async () => { sub.serve(10); expect(await sub.servedSequence()).toBe(10); - await replayUpdate({ priority: 0, startGroup: 5 }).encode(sub.client.writer, Version.DRAFT_06); + await replayUpdate({ priority: 0, start: { live: false, startGroup: 5, startFrame: 0 } }).encode( + sub.client.writer, + Version.DRAFT_06, + ); await flush(); sub.serve(5); @@ -743,6 +752,23 @@ test("lite draft-06: a widening update lowers the serving floor", async () => { } }); +// A resume floored above a quiet track's newest group, then joined by a live subscriber: the +// relay's update gains `live`, which drops the cursor to the live edge so the newest group is +// served at once rather than once the floor's group exists. +test("lite draft-06: gaining live serves the newest group below the floor", async () => { + const sub = await servedSubscription({ version: Version.DRAFT_06, startGroup: 4 }); + try { + sub.serve(3); + await flush(); + + await replayUpdate({ priority: 0 }).encode(sub.client.writer, Version.DRAFT_06); + await flush(); + expect(await sub.servedSequence()).toBe(3); + } finally { + await sub.close(); + } +}); + // The queued update raises the floor within the next buffered group. Control-first polling // applies the new frame start before popping the group, so excluded frames never reach the wire. test("lite draft-06: a queued frame floor applies before the next group pop", async () => { @@ -758,7 +784,10 @@ test("lite draft-06: a queued frame floor applies before the next group pop", as await sub.parked; sub.serve(1); - await replayUpdate({ priority: 0, startGroup: 1, startFrame: 2 }).encode(sub.client.writer, Version.DRAFT_06); + await replayUpdate({ priority: 0, start: { live: false, startGroup: 1, startFrame: 2 } }).encode( + sub.client.writer, + Version.DRAFT_06, + ); await flush(); expect(ranges).toHaveBeenCalledTimes(1); expect(lastGroups(ranges)).toEqual({ start: { included: 0 }, end: undefined }); @@ -831,7 +860,6 @@ test("lite draft-06: a popped group keeps its frame bounds across a queued updat test("lite draft-06: a burst of updates coalesces before the next group pop", async () => { const sub = await servedSubscription({ version: Version.DRAFT_06, - startGroup: 0, endGroup: 1, endFrame: 2, frames: ["a", "b", "c"], @@ -875,7 +903,6 @@ test("lite draft-06: a burst of updates coalesces before the next group pop", as test("lite draft-06: a newer update wins before a buffered group backlog drains", async () => { const sub = await servedSubscription({ version: Version.DRAFT_06, - startGroup: 0, gated: true, }); let second!: Promise; @@ -914,7 +941,7 @@ test("lite draft-06: a newer update wins before a buffered group backlog drains" // Scheduling affects streams already in flight, so it cannot wait behind a response write. // The full update remains atomic to observers, while the local range cursor stays serialized. test("lite draft-06: scheduling updates apply while SUBSCRIBE_START is blocked", async () => { - const sub = await servedSubscription({ version: Version.DRAFT_06, startGroup: 0, gated: true }); + const sub = await servedSubscription({ version: Version.DRAFT_06, gated: true }); const ranges = spyOn(hooks, "replaceGroups"); try { sub.serve(0); @@ -927,6 +954,7 @@ test("lite draft-06: scheduling updates apply while SUBSCRIBE_START is blocked", expect(sub.track.subscription.peek()).toEqual({ priority: 9, maxDelay: TEST_MAX_DELAY_MS, + live: true, groups: { start: undefined, end: { excluded: 6 } }, }); expect(ranges).not.toHaveBeenCalled(); @@ -1188,8 +1216,10 @@ async function serve( broadcast: Path.from("test"), track: "video", priority: 0, - startGroup: bounds.startGroup, - startFrame: bounds.startFrame ?? 0, + start: + bounds.startGroup === undefined + ? undefined + : { live: false, startGroup: bounds.startGroup, startFrame: bounds.startFrame ?? 0 }, endGroup: bounds.endGroup, endFrame: bounds.endFrame, }); @@ -1663,7 +1693,7 @@ test.each([0, 1])("lite draft-07 reports the cached largest position when starti broadcast: Path.from("quiet"), track: "video", priority: 0, - startGroup, + start: { live: false, startGroup, startFrame: 0 }, }), server, ); diff --git a/js/net/src/lite/publisher.ts b/js/net/src/lite/publisher.ts index 93f7a48348a..b25e7250c11 100644 --- a/js/net/src/lite/publisher.ts +++ b/js/net/src/lite/publisher.ts @@ -5,7 +5,7 @@ import * as DatagramStream from "../datagram_stream.ts"; import { error, NotFound, ProtocolViolation, reason, StreamCode, StreamError } from "../error.ts"; import type * as group from "../group.ts"; import { Cost, type Hop, type Route, routesEqual } from "../hop.ts"; -import { hiddenBelow, hooks, presented } from "../internal.ts"; +import { hiddenBelow, hooks, liveStart, presented } from "../internal.ts"; import type { Consumer as OriginConsumer } from "../origin.ts"; import type * as Path from "../path.ts"; import { type Reader, type Stream, Writer } from "../stream.ts"; @@ -22,6 +22,7 @@ import { Probe } from "./probe.ts"; import { encodeSubscribeResponse, exclusiveGroupEnd, + type Start, type Subscribe, SubscribeEnd, SubscribeOk, @@ -133,6 +134,10 @@ function supportsTrackStream(version: Version): boolean { * subscriber's read cursor (see {@link frameRange}). */ type FrameBounds = { + /** Whether the subscription wants the live edge, which starts a group at frame 0. */ + live: boolean; + /** An untimed track's latest group when the bounds were set: where `live` starts there. */ + untimedEdge?: number; /** The group {@link startFrame} qualifies, if the subscription named one. */ startGroup?: number; /** First frame to send within {@link startGroup}; every other group starts at 0. */ @@ -156,11 +161,30 @@ type FrameBounds = { */ function frameRange(bounds: FrameBounds, sequence: number): { start: number; end?: number } { return { - start: bounds.startGroup === sequence ? bounds.startFrame : 0, + start: bounds.startGroup === sequence && floorApplies(bounds) ? bounds.startFrame : 0, end: bounds.endGroup === sequence ? bounds.endFrame : undefined, }; } +/** + * Whether the floor's frame applies: only while the floor is where delivery starts. `live` + * starts from a group's first frame, so it takes over once that position sorts at or below + * the floor: always on a timed track, where anything below the live edge is expired anyway, + * and on an untimed one when the floor is not below its latest group. + */ +function floorApplies(bounds: FrameBounds): boolean { + if (!bounds.live) return true; + return ( + bounds.startGroup !== undefined && bounds.untimedEdge !== undefined && bounds.startGroup < bounds.untimedEdge + ); +} + +/** An untimed track's latest group, where `live` starts there; undefined on a timed track. */ +function untimedEdge(info: track.Info | undefined, subscriber: track.Subscriber): number | undefined { + if (!info || info.timescale != null) return undefined; + return subscriber.latest(); +} + /** What serving one group needs beyond the group itself. */ type ServeGroup = { /** The Subscribe ID the GROUP message references. */ @@ -351,8 +375,9 @@ function carriesMaxDelay(version: Version): boolean { * an unbounded budget so nothing is dropped under them (see {@link servingMaxDelay}), which * must not read as a request to replay the whole cache on join. */ -function positionCursor(track: track.Subscriber, version: Version, startGroup: number | undefined) { - if (resolvesStart(version) || startGroup !== undefined) return; +function positionCursor(track: track.Subscriber, version: Version, start: Start) { + // A pre-06 wire decodes either `live` alone (no `Group Start`) or a floor alone. + if (resolvesStart(version) || !start.live) return; const latest = track.latest(); if (latest !== undefined) hooks.replaceGroups(track, { start: { included: latest } }); @@ -622,12 +647,13 @@ export class Publisher { const track = wireOf(front).subscribe(msg.track, { priority: msg.priority, maxDelay: Milli(servingMaxDelay(this.version, msg.maxDelay)), + live: msg.start.live, groups: { - start: msg.startGroup === undefined ? undefined : { included: msg.startGroup }, + start: msg.start.startGroup === undefined ? undefined : { included: msg.start.startGroup }, end: endGroup === undefined ? undefined : { excluded: endGroup }, }, }); - positionCursor(track, this.version, msg.startGroup); + positionCursor(track, this.version, msg.start); hooks.replaceGroups(track, { end: endGroup === undefined ? undefined : { excluded: endGroup } }); // The best-effort datagram loop, started once serving begins. It parks when the @@ -638,6 +664,7 @@ export class Publisher { try { let timescale: Timescale = Timescale.MILLI; + let info: track.Info | undefined; if (supportsTrackStream(this.version)) { // Lite-05+ accepts implicitly: no SUBSCRIBE_OK (the immutable @@ -649,14 +676,14 @@ export class Publisher { // they always agree. Awaiting info() also surfaces a rejected track // (accept never called, track closed) as an error here, which resets the // stream. - const info = await track.info(); + info = await track.info(); timescale = wireTimescale(info); } else { // Older drafts acknowledge with SUBSCRIBE_OK and stream frames verbatim. const ok = new SubscribeOk({ priority: msg.priority, maxDelay: msg.maxDelay, - startGroup: msg.startGroup, + startGroup: msg.start.startGroup, endGroup: msg.endGroup, }); await encodeSubscribeResponse(stream.writer, { ok }, this.version); @@ -679,8 +706,12 @@ export class Publisher { track.update({ priority: update.priority, maxDelay: Milli(servingMaxDelay(this.version, update.maxDelay)), + live: update.start.live, groups: { - start: update.startGroup === undefined ? undefined : { included: update.startGroup }, + start: + update.start.startGroup === undefined + ? undefined + : { included: update.start.startGroup }, end: end === undefined ? undefined : { excluded: end }, }, }); @@ -690,9 +721,12 @@ export class Publisher { sub: msg.id, broadcast: msg.broadcast, timescale, + info, bounds: { - startGroup: msg.startGroup, - startFrame: msg.startFrame, + live: msg.start.live, + untimedEdge: untimedEdge(info, track), + startGroup: msg.start.startGroup, + startFrame: msg.start.startFrame, endGroup: msg.endGroup, endFrame: msg.endFrame, }, @@ -797,9 +831,15 @@ export class Publisher { track: track.Subscriber, stream: Writer, controls: SubscriptionControls, - serving: { sub: bigint; broadcast: Path.Valid; timescale: Timescale; bounds: FrameBounds }, + serving: { + sub: bigint; + broadcast: Path.Valid; + timescale: Timescale; + info: track.Info | undefined; + bounds: FrameBounds; + }, ): Promise { - const { sub, broadcast, timescale, bounds } = serving; + const { sub, broadcast, timescale, info, bounds } = serving; // Lite-05+ resolves the range on the subscribe stream: SUBSCRIBE_START once the // first group is known, SUBSCRIBE_END when the track finishes. const emitRange = supportsTrackStream(this.version); @@ -868,12 +908,25 @@ export class Publisher { case "update": { const update = control.update; console.debug(`subscribe update: broadcast=${broadcast} track=${track.name}`); + // A floor alone moves the read cursor to it. Gaining `live` drops the + // cursor back to the live edge: it only rises on its own, and a finished + // group below the old floor (a quiet catalog) is never served otherwise. + // A cursor that was already live stays where the first served group put + // it. Pre-06 `live` means the latest group, which positionCursor applied. + const edge = untimedEdge(info, track); + const start = update.start.live + ? !bounds.live && resolvesStart(this.version) + ? liveStart(true, update.start.startGroup, edge ?? 0) + : undefined + : update.start.startGroup; hooks.replaceGroups(track, { - start: update.startGroup === undefined ? undefined : { included: update.startGroup }, + start: start === undefined ? undefined : { included: start }, end: update.endGroup === undefined ? undefined : { included: update.endGroup }, }); - bounds.startGroup = update.startGroup; - bounds.startFrame = update.startFrame; + bounds.live = update.start.live; + bounds.untimedEdge = edge; + bounds.startGroup = update.start.startGroup; + bounds.startFrame = update.start.startFrame; bounds.endGroup = update.endGroup; bounds.endFrame = update.endFrame; await yieldToControls(); @@ -895,7 +948,13 @@ export class Publisher { // what it holds) is answered at once with the largest position, on // versions that carry it: a quiet track may not reach that start for a // while, and the subscriber judges what it holds against the answer. - if (emitRange && !startSent && hasLargest(this.version) && bounds.startGroup !== undefined) { + if ( + emitRange && + !startSent && + hasLargest(this.version) && + !bounds.live && + bounds.startGroup !== undefined + ) { const startFrame = bounds.startFrame; if ( largest !== undefined && diff --git a/js/net/src/lite/subscribe.test.ts b/js/net/src/lite/subscribe.test.ts index ea847db0599..03f93d92682 100644 --- a/js/net/src/lite/subscribe.test.ts +++ b/js/net/src/lite/subscribe.test.ts @@ -3,10 +3,12 @@ import * as Path from "../path.ts"; import { Reader, Writer } from "../stream.ts"; import { decodeSubscribeResponse, + EMPTY_START, emptyRange, encodeSubscribeResponse, exclusiveGroupEnd, inclusiveGroupEnd, + type Start, Subscribe, SubscribeDrop, SubscribeEnd, @@ -79,71 +81,112 @@ test("SubscribeOk round-trips priority/groups on draft-04", async () => { expect(got.ok.startGroup).toBe(3); }); -test("Subscribe round-trips every option including startGroup 0", async () => { +const LIVE: Start = { live: true, startFrame: 0 }; +const floored = (startGroup: number, startFrame = 0): Start => ({ live: false, startGroup, startFrame }); +const both = (startGroup: number, startFrame = 0): Start => ({ live: true, startGroup, startFrame }); + +/** What `start` reads back as on `version`, through SUBSCRIBE and SUBSCRIBE_UPDATE. */ +async function roundtrip(version: Version, start: Start): Promise { + const subscribe = new Subscribe({ id: 4n, broadcast: Path.from("test"), track: "video", priority: 7, start }); + const got = await Subscribe.decode( + new Reader(undefined, await encodeMessage(version, subscribe), version), + version, + ); + const update = new SubscribeUpdate({ priority: 8, start }); + const updated = await SubscribeUpdate.decode( + new Reader(undefined, await encodeMessage(version, update), version), + version, + ); + expect(updated.start).toEqual(got.start); + return got.start; +} + +test("Subscribe round-trips every option", async () => { const message = new Subscribe({ id: 4n, broadcast: Path.from("test"), track: "video", priority: 7, maxDelay: 250, - startGroup: 0, + start: floored(3, 2), endGroup: 9, + endFrame: 4, }); - // Lite-06 carries the raw floor, and a floor of 0 with no frame offset is the same - // absence of a constraint as no floor at all, so it canonicalizes to undefined. - const got = await Subscribe.decode( - new Reader(undefined, await encodeMessage(Version.DRAFT_06, message), Version.DRAFT_06), - Version.DRAFT_06, - ); - expect(got.priority).toBe(7); - expect(got.maxDelay).toBe(250); - expect(got.startGroup).toBeUndefined(); - expect(got.endGroup).toBe(9); - - // Group 0 stays named when a Frame Start qualifies it: a subscription can resume - // partway through group 0 (a catalog never leaves it). - message.startFrame = 4; - const resumed = await Subscribe.decode( - new Reader(undefined, await encodeMessage(Version.DRAFT_06, message), Version.DRAFT_06), - Version.DRAFT_06, - ); - expect(resumed.startGroup).toBe(0); - expect(resumed.startFrame).toBe(4); - message.startFrame = 0; - - // A pre-06 wire folds the vacuous floor back to absent: an explicit group 0 there - // would mean "replay from the beginning", which is not what a floor of 0 asks for. - const folded = await Subscribe.decode( - new Reader(undefined, await encodeMessage(Version.DRAFT_05, message), Version.DRAFT_05), - Version.DRAFT_05, - ); - expect(folded.startGroup).toBeUndefined(); - expect(folded.endGroup).toBe(9); + for (const version of [Version.DRAFT_06, Version.DRAFT_07]) { + const got = await Subscribe.decode( + new Reader(undefined, await encodeMessage(version, message), version), + version, + ); + expect(got.priority).toBe(7); + expect(got.maxDelay).toBe(250); + expect(got.start).toEqual(floored(3, 2)); + expect(got.endGroup).toBe(9); + expect(got.endFrame).toBe(4); + } }); -test("SubscribeUpdate round-trips every option including startGroup 0", async () => { - const message = new SubscribeUpdate({ - priority: 8, - maxDelay: 500, - startGroup: 0, - endGroup: 12, +test("lite-07 carries live beside the floor", async () => { + for (const start of [LIVE, floored(0), floored(3, 5), both(0), both(4, 2)]) { + expect(await roundtrip(Version.DRAFT_07, start)).toEqual(start); + } + + // `Live`, then the floor's presence, then the floor itself, ahead of the two end varints; + // the same bytes the Rust codec pins. + const bytes = async (start: Start) => + Array.from( + await encodeMessage( + Version.DRAFT_07, + new Subscribe({ id: 1n, broadcast: Path.from("room"), track: "video", priority: 0, start }), + ), + ); + expect((await bytes(both(4, 2))).slice(-6)).toEqual([1, 1, 4, 2, 0, 0]); + expect((await bytes(LIVE)).slice(-4)).toEqual([1, 0, 0, 0]); +}); + +test("lite-07 refuses neither live nor a floor", async () => { + const neither = new Subscribe({ + id: 1n, + broadcast: Path.from("room"), + track: "video", + priority: 0, + start: { live: false, startFrame: 0 }, }); - const got = await SubscribeUpdate.decode( - new Reader(undefined, await encodeMessage(Version.DRAFT_06, message), Version.DRAFT_06), - Version.DRAFT_06, + await expect(encodeMessage(Version.DRAFT_07, neither)).rejects.toThrow(EMPTY_START); + + // Clear `Live` on an encoded live-only SUBSCRIBE: the absent floor and the two end + // varints follow it. + const bytes = await encodeMessage( + Version.DRAFT_07, + new Subscribe({ id: 1n, broadcast: Path.from("room"), track: "video", priority: 0 }), ); - expect(got.priority).toBe(8); - expect(got.maxDelay).toBe(500); - expect(got.startGroup).toBeUndefined(); - expect(got.endGroup).toBe(12); - - // The same fold as SUBSCRIBE on a pre-06 wire. - const folded = await SubscribeUpdate.decode( - new Reader(undefined, await encodeMessage(Version.DRAFT_05, message), Version.DRAFT_05), - Version.DRAFT_05, + const live = bytes.length - 4; + expect(bytes[live]).toBe(1); + bytes[live] = 0; + await expect(Subscribe.decode(new Reader(undefined, bytes, Version.DRAFT_07), Version.DRAFT_07)).rejects.toThrow( + EMPTY_START, ); - expect(folded.startGroup).toBeUndefined(); - expect(folded.endGroup).toBe(12); +}); + +test("lite-06 folds live into group start zero", async () => { + expect(await roundtrip(Version.DRAFT_06, LIVE)).toEqual(LIVE); + // `live` with a floor goes out as (0, 0), and the receiver filters locally. + expect(await roundtrip(Version.DRAFT_06, both(4))).toEqual(LIVE); + expect(await roundtrip(Version.DRAFT_06, floored(7, 4))).toEqual(floored(7, 4)); + // A catalog resume partway through group 0 stays a floor. + expect(await roundtrip(Version.DRAFT_06, floored(0, 4))).toEqual(floored(0, 4)); + // A floor of (0, 0) reads back as `live`, which lite-06 cannot tell apart. + expect(await roundtrip(Version.DRAFT_06, floored(0))).toEqual(LIVE); +}); + +test("pre-06 folds live into an explicit group zero", async () => { + for (const version of [Version.DRAFT_03, Version.DRAFT_04, Version.DRAFT_05]) { + expect(await roundtrip(version, LIVE)).toEqual(LIVE); + // Replay from the beginning, so the receiver can filter by age and floor. + expect(await roundtrip(version, both(2))).toEqual(floored(0)); + expect(await roundtrip(version, floored(7))).toEqual(floored(7)); + // An explicit group 0 is 1 on the wire, not folded back to absent. + expect(await roundtrip(version, floored(0))).toEqual(floored(0)); + } }); test("SubscribeStart round-trips on draft-05", async () => { @@ -222,9 +265,6 @@ test("frame bounds without their group bounds are rejected before encoding", asy track: "video", priority: 0, }; - await expect(encodeSubscribe(new Subscribe({ ...base, startFrame: 3 }))).rejects.toThrow( - "frame bound without a group bound", - ); await expect(encodeSubscribe(new Subscribe({ ...base, endFrame: 7 }))).rejects.toThrow( "frame bound without a group bound", ); @@ -247,4 +287,7 @@ test("a requested range is empty when its bounds meet anywhere", () => { expect(emptyRange({ startGroup: 6, endGroup: 5 })).toBe(true); expect(emptyRange({ startGroup: 5, endGroup: 6 })).toBe(false); expect(emptyRange({ startGroup: 5 })).toBe(false); + // The live edge may sit below any floor, so only an end of 0 empties a live range. + expect(emptyRange({ live: true, startGroup: 5, endGroup: 5 })).toBe(false); + expect(emptyRange({ live: true, startGroup: 5, endGroup: 0 })).toBe(true); }); diff --git a/js/net/src/lite/subscribe.ts b/js/net/src/lite/subscribe.ts index ec6b4b13d8a..e5cb725dab0 100644 --- a/js/net/src/lite/subscribe.ts +++ b/js/net/src/lite/subscribe.ts @@ -4,101 +4,130 @@ import type { Reader, Writer } from "../stream.ts"; import type { Location } from "../track.ts"; import { decodeEpoch, encodeEpoch } from "./epoch.ts"; import * as Message from "./message.ts"; -import { hasFrameBounds, hasGroupOrder, hasLargest, hasStreamCount, resolvesStart, Version } from "./version.ts"; +import { + hasFrameBounds, + hasGroupOrder, + hasLargest, + hasLive, + hasStreamCount, + resolvesStart, + Version, +} from "./version.ts"; /** - * Encode the `Group Start` field shared by SUBSCRIBE and SUBSCRIBE_UPDATE. + * Where a subscription may start, as SUBSCRIBE and SUBSCRIBE_UPDATE carry it: the live + * edge, a floor (`startGroup` qualified by `startFrame`), or both. * - * Lite-06 writes the raw floor (`undefined` and 0 are the same absence of a constraint), - * while a pre-06 wire encodes the sequence + 1 and gets a vacuous floor folded back to - * absent: an explicit group 0 there means "replay from the beginning", which is not what - * a floor of 0 asks for. + * Lite-07 carries both. Older versions have only `Group Start`, so `live` folds into it: + * lite-06 `Group Start` 0 with `Frame Start` 0 is `live` and any other pair a floor (`(0, N)` + * is a catalog resume), a pre-06 absent `Group Start` is `live`, and lite-01/02 carry none. */ -async function encodeStartGroup(w: Writer, version: Version, startGroup?: number) { - if (resolvesStart(version)) { - await w.u53(startGroup ?? 0); - return; - } - await w.u53(startGroup !== undefined && startGroup > 0 ? startGroup + 1 : 0); +export interface Start { + /** Deliver from the live edge. */ + live: boolean; + /** The floor's group, or undefined for no floor. */ + startGroup?: number; + /** The floor's first frame within `startGroup`; 0 without a floor. */ + startFrame: number; +} + +/** `live` alone, which is what a version without `Group Start` asks for. */ +const LIVE: Start = { live: true, startFrame: 0 }; + +/** The floor a version without `Live` carries: none for `live` alone, (0, 0) beside a floor. */ +function folded(start: Start): { startGroup?: number; startFrame: number } { + if (!start.live) return { startGroup: start.startGroup, startFrame: start.startFrame }; + return start.startGroup === undefined ? { startFrame: 0 } : { startGroup: 0, startFrame: 0 }; } /** - * Decode the `Group Start` field shared by SUBSCRIBE and SUBSCRIBE_UPDATE. + * Encode the start shared by SUBSCRIBE and SUBSCRIBE_UPDATE: everything on lite-07 and pre-06, + * and `Group Start` alone on lite-06, whose `Frame Start` trails the end group. * - * The inverse of {@link encodeStartGroup}. Callers canonicalize with - * {@link canonicalStartGroup} once the frame bounds are known. + * Neither `live` nor a floor asks for nothing, so it has no encoding. A pre-06 `Group Start` + * cannot qualify a frame, so a floor partway through a group is refused rather than widened. */ -async function decodeStartGroup(r: Reader, version: Version): Promise { - const value = await r.u53(); - if (resolvesStart(version)) return value; - return value > 0 ? value - 1 : undefined; +async function encodeStart(w: Writer, version: Version, start: Start) { + if (!start.live && start.startGroup === undefined) throw new Error(EMPTY_START); + if (hasLive(version)) { + await w.bool(start.live); + await w.bool(start.startGroup !== undefined); + if (start.startGroup !== undefined) { + await w.u53(start.startGroup); + await w.u53(start.startFrame); + } + return; + } + const floor = folded(start); + if (resolvesStart(version)) { + await w.u53(floor.startGroup ?? 0); + return; + } + if (floor.startFrame !== 0) throw new Error("frame bounds not supported for this version"); + // The sequence + 1, so an explicit group 0 (replay from the beginning) is 1. + await w.u53(floor.startGroup !== undefined ? floor.startGroup + 1 : 0); } -/** - * Canonicalize a decoded floor: a lite-06 `Group Start` of 0 with no frame offset is the - * same absence of a constraint as no floor at all, so it decodes as undefined. Group 0 - * stays named only when a `Frame Start` actually qualifies it (a subscription can resume - * partway through group 0: a catalog never leaves it). - */ -function canonicalStartGroup(version: Version, startGroup: number | undefined, startFrame: number): number | undefined { - if (resolvesStart(version) && startGroup === 0 && startFrame === 0) return undefined; - return startGroup; +/** Decode what {@link encodeStart} writes; lite-06's trailing `Frame Start` is applied after. */ +async function decodeStart(r: Reader, version: Version): Promise { + if (hasLive(version)) { + const live = await r.bool(); + if (!(await r.bool())) { + if (!live) throw new Error(EMPTY_START); + return LIVE; + } + return { live, startGroup: await r.u53(), startFrame: await r.u53() }; + } + const value = await r.u53(); + if (resolvesStart(version)) return { live: false, startGroup: value, startFrame: 0 }; + return value > 0 ? { live: false, startGroup: value - 1, startFrame: 0 } : LIVE; } /** - * Encode the trailing `Frame Start` / `Frame End` pair shared by SUBSCRIBE and - * SUBSCRIBE_UPDATE. A no-op before lite-06, which has nowhere to put them. + * Encode the trailing frame fields shared by SUBSCRIBE and SUBSCRIBE_UPDATE: lite-06's + * `Frame Start` and lite-06+'s `Frame End`. A no-op before lite-06, which has nowhere to put + * them. */ async function encodeFrameBounds( w: Writer, version: Version, - { - startGroup, - startFrame, - endGroup, - endFrame, - }: { startGroup?: number; startFrame: number; endGroup?: number; endFrame?: number }, + { start, endGroup, endFrame }: { start: Start; endGroup?: number; endFrame?: number }, ) { - if ((startFrame !== 0 && startGroup === undefined) || (endFrame !== undefined && endGroup === undefined)) { - throw new Error("frame bound without a group bound"); - } + if (endFrame !== undefined && endGroup === undefined) throw new Error("frame bound without a group bound"); if (!hasFrameBounds(version)) { // Silently widening to the whole group would deliver frames we excluded. - if (startFrame !== 0 || endFrame !== undefined) { - throw new Error("frame bounds not supported for this version"); - } + if (endFrame !== undefined) throw new Error("frame bounds not supported for this version"); return; } - await w.u53(startFrame); + if (!hasLive(version)) await w.u53(folded(start).startFrame); await w.u53(endFrame !== undefined ? endFrame + 1 : 0); } /** - * Decode the trailing `Frame Start` / `Frame End` pair, defaulting to the whole group. + * Decode the trailing frame fields, defaulting to the whole group, and read a lite-06 start + * back: `(0, 0)` is `live`. * - * A frame bound without the group bound it qualifies is a protocol violation: frames - * are numbered per group, so there is nothing to count from. + * A frame bound without the group bound it qualifies is a protocol violation: frames are + * numbered per group, so there is nothing to count from. */ async function decodeFrameBounds( r: Reader, version: Version, - startGroup?: number, + start: Start, endGroup?: number, -): Promise<{ startFrame: number; endFrame?: number }> { - if (!hasFrameBounds(version)) { - return { startFrame: 0 }; - } - - const startFrame = await r.u53(); - const endFrame = await r.u53(); +): Promise<{ start: Start; endFrame?: number }> { + if (!hasFrameBounds(version)) return { start }; - if ((startFrame !== 0 && startGroup === undefined) || (endFrame !== 0 && endGroup === undefined)) { - throw new Error("frame bound without a group bound"); + if (!hasLive(version)) { + const startFrame = await r.u53(); + start = start.startGroup === 0 && startFrame === 0 ? LIVE : { ...start, startFrame }; } + const endFrame = await r.u53(); + if (endFrame !== 0 && endGroup === undefined) throw new Error("frame bound without a group bound"); - return { startFrame, endFrame: endFrame > 0 ? endFrame - 1 : undefined }; + return { start, endFrame: endFrame > 0 ? endFrame - 1 : undefined }; } /** Step over the retired `Ordered` byte on a version whose layout still has it. */ @@ -124,6 +153,9 @@ export function exclusiveGroupEnd(inclusive?: number): number | undefined { /** The error for a requested range the wire cannot carry; see {@link emptyRange}. */ export const EMPTY_RANGE = "empty subscription range cannot be encoded"; +/** The error for a subscription with neither `live` nor a floor, which asks for nothing. */ +export const EMPTY_START = "a subscription needs live or a floor"; + /** * Inclusive last group a Subscribe message carries, from an exclusive model end. * @@ -141,37 +173,37 @@ export function inclusiveGroupEnd(exclusive?: number): number | undefined { * * The wire has no encoding for one: its bounds are inclusive, so flooring the end would * either hand back the group the caller excluded (0 means unbounded) or invert the - * range once the two bounds meet. An absent start is the live edge, so it only empties - * the range when the end is 0. + * range once the two bounds meet. The live edge may sit below any floor, so with `live` the + * range is empty only when the end is 0. */ -export function emptyRange({ startGroup, endGroup }: { startGroup?: number; endGroup?: number }): boolean { - return endGroup !== undefined && (startGroup ?? 0) >= endGroup; +export function emptyRange({ + live, + startGroup, + endGroup, +}: { + live?: boolean; + startGroup?: number; + endGroup?: number; +}): boolean { + const lowest = (live ?? startGroup === undefined) ? 0 : (startGroup ?? 0); + return endGroup !== undefined && lowest >= endGroup; } export class SubscribeUpdate { priority: number; /** Subscriber max delay in milliseconds; zero skips once a newer group is available. */ maxDelay: number; - startGroup?: number; + /** See {@link Subscribe.start}. */ + start: Start; endGroup?: number; - /** See {@link Subscribe.startFrame}. */ - startFrame: number; /** See {@link Subscribe.endFrame}. */ endFrame?: number; - constructor(props: { - priority: number; - maxDelay?: number; - startGroup?: number; - endGroup?: number; - startFrame?: number; - endFrame?: number; - }) { + constructor(props: { priority: number; maxDelay?: number; start?: Start; endGroup?: number; endFrame?: number }) { this.priority = props.priority; this.maxDelay = props.maxDelay ?? 0; - this.startGroup = props.startGroup; + this.start = props.start ?? LIVE; this.endGroup = props.endGroup; - this.startFrame = props.startFrame ?? 0; this.endFrame = props.endFrame; } @@ -185,7 +217,7 @@ export class SubscribeUpdate { await w.u8(this.priority); await padGroupOrder(w, version); await w.u53(this.maxDelay); - await encodeStartGroup(w, version, this.startGroup); + await encodeStart(w, version, this.start); await w.u53(this.endGroup !== undefined ? this.endGroup + 1 : 0); await encodeFrameBounds(w, version, this); break; @@ -201,17 +233,11 @@ export class SubscribeUpdate { const priority = await r.u8(); await skipGroupOrder(r, version); const maxDelay = await r.u53(); - const startGroup = await decodeStartGroup(r, version); + const start = await decodeStart(r, version); const endGroup = (await r.u53()) || undefined; const end = endGroup !== undefined ? endGroup - 1 : undefined; - const frames = await decodeFrameBounds(r, version, startGroup, end); - return new SubscribeUpdate({ - priority, - maxDelay, - startGroup: canonicalStartGroup(version, startGroup, frames.startFrame), - endGroup: end, - ...frames, - }); + const frames = await decodeFrameBounds(r, version, start, end); + return new SubscribeUpdate({ priority, maxDelay, endGroup: end, ...frames }); } } } @@ -239,15 +265,12 @@ export class Subscribe { /** Subscriber max delay in milliseconds; zero skips once a newer group is available. */ maxDelay: number; - startGroup?: number; - endGroup?: number; - /** - * First frame to deliver within `startGroup`'s group; 0 is the whole group. - * Lite-06+. It qualifies the named group, so it needs `startGroup` to name one - * (defined, including 0: group 0 can host a mid-group resume). + * Where delivery may start; see {@link Start}. A floor's `startFrame` is lite-06+ and + * qualifies its group, group 0 included (a catalog can resume partway through it). */ - startFrame: number; + start: Start; + endGroup?: number; /** * Last frame to deliver (inclusive) within `endGroup`'s group, or undefined for the @@ -262,9 +285,8 @@ export class Subscribe { track: string; priority: number; maxDelay?: number; - startGroup?: number; + start?: Start; endGroup?: number; - startFrame?: number; endFrame?: number; }) { this.id = props.id; @@ -273,9 +295,8 @@ export class Subscribe { this.track = props.track; this.priority = props.priority; this.maxDelay = props.maxDelay ?? 0; - this.startGroup = props.startGroup; + this.start = props.start ?? LIVE; this.endGroup = props.endGroup; - this.startFrame = props.startFrame ?? 0; this.endFrame = props.endFrame; } @@ -293,7 +314,7 @@ export class Subscribe { default: await padGroupOrder(w, version); await w.u53(this.maxDelay); - await encodeStartGroup(w, version, this.startGroup); + await encodeStart(w, version, this.start); await w.u53(this.endGroup !== undefined ? this.endGroup + 1 : 0); await encodeFrameBounds(w, version, this); break; @@ -314,21 +335,11 @@ export class Subscribe { default: { await skipGroupOrder(r, version); const maxDelay = await r.u53(); - const startGroup = await decodeStartGroup(r, version); + const start = await decodeStart(r, version); const endGroup = (await r.u53()) || undefined; const end = endGroup !== undefined ? endGroup - 1 : undefined; - const frames = await decodeFrameBounds(r, version, startGroup, end); - return new Subscribe({ - id, - broadcast, - epoch, - track, - priority, - maxDelay, - startGroup: canonicalStartGroup(version, startGroup, frames.startFrame), - endGroup: end, - ...frames, - }); + const frames = await decodeFrameBounds(r, version, start, end); + return new Subscribe({ id, broadcast, epoch, track, priority, maxDelay, endGroup: end, ...frames }); } } } diff --git a/js/net/src/lite/subscriber.ts b/js/net/src/lite/subscriber.ts index 726c93f0370..0517daa447b 100644 --- a/js/net/src/lite/subscriber.ts +++ b/js/net/src/lite/subscriber.ts @@ -36,9 +36,11 @@ import { decodeSubscribeResponse, decodeSubscribeResponseMaybe, EMPTY_RANGE, + EMPTY_START, emptyRange, exclusiveGroupEnd, inclusiveGroupEnd, + type Start, Subscribe, SubscribeUpdate, } from "./subscribe.ts"; @@ -78,6 +80,18 @@ function supportsTrackStream(version: Version): boolean { } } +/** What `subscription` asks a peer for: the live edge, its floor, or both. */ +function wireStart(subscription: track.Subscription): Start { + const startGroup = subscription.groups?.start === undefined ? undefined : groupBounds(subscription.groups).start; + return { live: subscription.live ?? startGroup === undefined, startGroup, startFrame: 0 }; +} + +/** The floor below which the peer serves nothing, or undefined for the live edge, which reaches below any floor. */ +function servedFloor(subscription: track.Subscription): number | undefined { + const start = wireStart(subscription); + return start.live ? undefined : start.startGroup; +} + interface SubscribeEntry { // The write side: incoming GROUP streams are routed here. The application reads // the matching track.Subscriber it got from broadcast.Consumer.subscribe. @@ -555,8 +569,13 @@ export class Subscriber { async #runSubscribe(broadcast: Path.Valid, epoch: Epoch.Valid | undefined, request: track.Request) { const id = this.#subscribeNext++; const subscription = request.subscription; + const start = wireStart(subscription); + if (!start.live && start.startGroup === undefined) { + request.reject(new Error(EMPTY_START)); + return; + } const initialBounds = groupBounds(subscription.groups); - if (emptyRange({ startGroup: initialBounds.start, endGroup: initialBounds.end })) { + if (emptyRange({ ...start, endGroup: initialBounds.end })) { request.reject(new Error(EMPTY_RANGE)); return; } @@ -575,7 +594,7 @@ export class Subscriber { track: request.name, priority: subscription.priority ?? 0, maxDelay: subscription.maxDelay, - startGroup: subscription.groups?.start === undefined ? undefined : bounds.start, + start, endGroup: inclusiveGroupEnd(bounds.end), }); @@ -728,7 +747,7 @@ export class Subscriber { return maxDelay > 0 ? maxDelay : TAIL_GRACE_MS; }, }), - requested: msg.startGroup, + requested: msg.start.live ? undefined : msg.start.startGroup, held: state.track, }; this.#subscribes.set(id, entry); @@ -1051,9 +1070,9 @@ export class Subscriber { // Owed from the floor the demand last asked for, which an update can move either // way, or where SUBSCRIBE_START resolved a live-edge one. The groups the SUBSCRIBE // asked for below its SUBSCRIBE_START were accounted for when it arrived. - const groups = track.subscription.peek()?.groups; - const bounds = groupBounds(groups ?? {}); - const start = groups?.start === undefined ? entry.start : bounds.start; + const current = track.subscription.peek(); + const bounds = groupBounds(current?.groups ?? {}); + const start = (current && servedFloor(current)) ?? entry.start; const end = bounds.end === undefined ? entry.end : Math.min(entry.end, bounds.end); return tail.covers(start, end); }; @@ -1084,8 +1103,9 @@ export class Subscriber { let lastSent: track.Subscription = { priority: msg.priority, maxDelay: Time.Milli(msg.maxDelay), + live: msg.start.live, groups: { - start: msg.startGroup === undefined ? undefined : { included: msg.startGroup }, + start: msg.start.startGroup === undefined ? undefined : { included: msg.start.startGroup }, end: msg.endGroup === undefined ? undefined : { excluded: exclusiveGroupEnd(msg.endGroup) ?? 0 }, }, }; @@ -1101,13 +1121,16 @@ export class Subscriber { // Demand collapsing to nothing is refused the same way an initial empty // request is: the error closes the track, so every local subscriber sees it. + const start = wireStart(current); + if (!start.live && start.startGroup === undefined) throw new Error(EMPTY_START); const bounds = groupBounds(current.groups); - if (emptyRange({ startGroup: bounds.start, endGroup: bounds.end })) throw new Error(EMPTY_RANGE); + if (emptyRange({ ...start, endGroup: bounds.end })) throw new Error(EMPTY_RANGE); // A lowered floor owes groups nobody asked for until now. - if (current.groups?.start !== undefined) { - const floor = lastSent.groups?.start === undefined ? entry.start : groupBounds(lastSent.groups).start; - entry.tail.demand(bounds.start, floor ?? Number.POSITIVE_INFINITY); + const floor = servedFloor(current); + if (floor !== undefined) { + const last = servedFloor(lastSent) ?? entry.start; + entry.tail.demand(floor, last ?? Number.POSITIVE_INFINITY); } // Round-trip the other Subscribe parameters so the publisher doesn't @@ -1115,7 +1138,7 @@ export class Subscriber { const update = new SubscribeUpdate({ priority: current.priority ?? 0, maxDelay: current.maxDelay, - startGroup: current.groups?.start === undefined ? undefined : bounds.start, + start, endGroup: inclusiveGroupEnd(bounds.end), }); await update.encode(stream.writer, this.version); @@ -1130,6 +1153,8 @@ export class Subscriber { return ( (a.priority ?? 0) === (b.priority ?? 0) && (a.maxDelay ?? 0) === (b.maxDelay ?? 0) && + wireStart(a).live === wireStart(b).live && + (a.groups?.start === undefined) === (b.groups?.start === undefined) && ag.start === bg.start && ag.end === bg.end ); diff --git a/js/net/src/lite/version.ts b/js/net/src/lite/version.ts index 55ae9639479..94a4cfd6cd9 100644 --- a/js/net/src/lite/version.ts +++ b/js/net/src/lite/version.ts @@ -213,6 +213,25 @@ export function resolvesStart(version: Version): boolean { } } +/** + * Whether SUBSCRIBE and SUBSCRIBE_UPDATE carry a `Live` flag beside an optional floor. + * Added in lite-07; older versions fold `live` into `Group Start`. + */ +export function hasLive(version: Version): boolean { + // Explicitly list older versions so future versions keep the lite-07+ behavior. + switch (version) { + case Version.DRAFT_01: + case Version.DRAFT_02: + case Version.DRAFT_03: + case Version.DRAFT_04: + case Version.DRAFT_05: + case Version.DRAFT_06: + return false; + default: + return true; + } +} + /** Whether ANNOUNCE_REQUEST carries the hidden opt-in. Added in lite-07. */ export function hasHidden(version: Version): boolean { // Explicitly list older versions so future versions keep the lite-07+ behavior. diff --git a/js/net/src/track.test.ts b/js/net/src/track.test.ts index faeec7e24e1..22124c613e9 100644 --- a/js/net/src/track.test.ts +++ b/js/net/src/track.test.ts @@ -285,6 +285,7 @@ test("subscriber options and updates are forwarded to the producer's aggregate", expect(producer.subscription.peek()).toEqual({ priority: 0, maxDelay: Milli(0), + live: false, groups: { start: { included: 0 } }, }); @@ -294,6 +295,7 @@ test("subscriber options and updates are forwarded to the producer's aggregate", expect(await next).toEqual({ priority: 7, maxDelay: Milli(250), + live: false, groups: { start: { included: 2 }, end: { excluded: 9 } }, }); }); @@ -379,6 +381,7 @@ test("multiple subscriber options aggregate like Rust", async () => { expect(producer.subscription.peek()).toEqual({ priority: 7, maxDelay: Milli(250), + live: false, groups: { start: { included: 5 } }, }); @@ -387,6 +390,7 @@ test("multiple subscriber options aggregate like Rust", async () => { expect(await narrowed).toEqual({ priority: 2, maxDelay: Milli(100), + live: false, groups: { start: { included: 10 }, end: { excluded: 20 } }, }); @@ -395,6 +399,58 @@ test("multiple subscriber options aggregate like Rust", async () => { expect(await none).toBeUndefined(); }); +test("a live-only subscriber keeps another's floor and adds the live edge", () => { + const producer = new TrackProducer("test"); + // A resume names where it left off, which is a floor alone. + const resumed = producer.subscribe({ groups: { start: { included: 4 } } }); + expect(producer.subscription.peek()?.live).toBe(false); + + // Clearing the floor starved the resume; keeping only the floor starved the live one. + const live = producer.subscribe(); + expect(producer.subscription.peek()).toMatchObject({ live: true, groups: { start: { included: 4 } } }); + + // The floor and `live` together, as one subscriber. + const both = producer.subscribe({ live: true, groups: { start: { included: 2 } } }); + expect(producer.subscription.peek()).toMatchObject({ live: true, groups: { start: { included: 2 } } }); + for (const sub of [resumed, live, both]) sub.close(); +}); + +test("a live subscriber starts below a floor above a quiet track's live edge", async () => { + const producer = new TrackProducer("test"); + producer.accept({ timescale: Timescale.MILLI }); + for (let sequence = 0; sequence < 4; sequence++) { + const group = new GroupProducer(sequence); + group.writeFrame({ payload: new Uint8Array([sequence]), timestamp: Timestamp.fromMillis(sequence * 100) }); + group.close(); + producer.writeGroup(group); + } + + const resumed = producer.subscribe({ groups: { start: { included: 4 } } }); + const live = producer.subscribe(); + expect((await live.recvGroup())?.sequence).toBe(3); + + const group = new GroupProducer(4); + group.close(); + producer.writeGroup(group); + expect((await live.recvGroup())?.sequence).toBe(4); + expect((await resumed.recvGroup())?.sequence).toBe(4); +}); + +test("an untimed live subscriber starts at the lower of its floor and the latest group", async () => { + const producer = new TrackProducer("test"); + producer.accept({}); + for (let sequence = 0; sequence < 5; sequence++) { + const group = new GroupProducer(sequence); + group.close(); + producer.writeGroup(group); + } + + const below = producer.subscribe({ live: true, groups: { start: { included: 2 } } }); + expect((await below.recvGroup())?.sequence).toBe(2); + const above = producer.subscribe({ live: true, groups: { start: { included: 7 } } }); + expect((await above.recvGroup())?.sequence).toBe(4); +}); + test("the producer aggregate is clamped without changing subscriber options", async () => { const producer = new TrackProducer("test"); const track = producer.subscribe({ maxDelay: Milli(10_000) }); diff --git a/js/net/src/track.ts b/js/net/src/track.ts index 93f51298678..64963fdd61b 100644 --- a/js/net/src/track.ts +++ b/js/net/src/track.ts @@ -10,6 +10,7 @@ import { type Frame, type Consumer as GroupConsumer, Producer as GroupProducer } import { groupBounds, hooks, + liveStart, type Recv, type TrackRequestOptions, type TrackSequence, @@ -143,13 +144,24 @@ export interface Subscription { */ maxDelay?: Milli; /** - * The lowest group the publisher may deliver (a floor), or omit for none. + * Deliver from the live edge: the oldest group {@link maxDelay} has not expired, or the + * latest group on an untimed track. + * + * Lowers the floor (`groups.start`) whenever the live edge sits below it, so a floor above + * the live edge never hides the latest group from a subscriber that also wants it. + * Defaults to `true` without a floor and `false` with one, so a floor alone is a resume. + * Neither `live` nor a floor asks for nothing, which no wire carries. + * Aggregated across subscribers as an OR. + */ + live?: boolean; + /** + * The lowest group the publisher may deliver (a floor) as `start`, or omit it for none, + * and the end of the requested range. * * A floor, not a request: only {@link maxDelay} asks for data, and the floor bounds how - * far back it may reach. Omitting it and a floor of 0 differ only on an untimed track, - * where nothing is ever stale: omitted starts at the latest group, and an explicit floor - * replays from there. A floor above the live edge simply waits there (a resumed - * subscription naming where it left off). + * far back it may reach. A floor above the live edge simply waits there (a resumed + * subscription naming where it left off), unless {@link live} lowers it. Aggregated across + * subscribers as the lowest floor, where none is neutral. */ groups?: Groups; } @@ -162,6 +174,7 @@ function subscriptionDefaults(subscription: Subscription = {}): Subscription { return { priority: priorityByte(subscription.priority ?? 0), maxDelay: wireMillis("maxDelay", subscription.maxDelay ?? Milli.zero), + live: subscription.live ?? subscription.groups?.start === undefined, groups: { start: subscription.groups?.start === undefined ? undefined : { included: bounds.start }, end: bounds.end === undefined ? undefined : { excluded: bounds.end }, @@ -182,16 +195,23 @@ function combineSubscriptions(states: Iterable): Subscription | unde combined.priority = Math.max(combined.priority ?? 0, subscription.priority ?? 0); combined.maxDelay = Milli(Math.max(combined.maxDelay ?? Milli.zero, subscription.maxDelay ?? Milli.zero)); + // Anyone wanting the live edge gets it, without lowering anyone's floor. + combined.live = combined.live || subscription.live; - // A floor only restricts, so a subscriber without one clears the aggregate: - // its budget may reach below any floor the others set. + // The lowest floor wins. A live-only subscriber has no floor, the neutral value, so it + // never clears another's: that starved the floored subscriber's resume. const a = groupBounds(combined.groups ?? {}); const b = groupBounds(subscription.groups ?? {}); - combined.groups = { - start: - combined.groups?.start === undefined || subscription.groups?.start === undefined + const start = + combined.groups?.start === undefined + ? subscription.groups?.start === undefined ? undefined - : { included: Math.min(a.start, b.start) }, + : b.start + : subscription.groups?.start === undefined + ? a.start + : Math.min(a.start, b.start); + combined.groups = { + start: start === undefined ? undefined : { included: start }, end: a.end === undefined || b.end === undefined ? undefined : { excluded: Math.max(a.end, b.end) }, }; } @@ -1180,26 +1200,31 @@ export class Subscriber { private constructor(name: string, state: TrackState) { this.name = name; this.#state = state; - // The cursor's floor is the group the subscription named, or 0. A floor is the - // only thing a start contributes; {@link Subscription.maxDelay} is what asks for - // data, and delivery skips everything above the floor that the budget convicts. + // The cursor starts at the group the subscription named (its floor), or 0, and + // `live` lowers it to the live edge. A floor is the only thing a start contributes; + // {@link Subscription.maxDelay} is what asks for data, and delivery skips everything + // above the start that the budget convicts, so on a timed track `live` starts at 0. // Nothing on an untimed track is ever stale, so there the budget would replay the - // whole cache: an unfloored subscriber starts at the latest group instead. - const groups = state.update.peek()?.groups ?? {}; + // whole cache: `live` starts at the latest group instead. + const subscription = state.update.peek(); + const groups = subscription?.groups ?? {}; const bounds = groupBounds(groups); - const start = groups.start === undefined ? this.#untimedStart(bounds.end) : undefined; - this.#cursor.set({ start: start ?? bounds.start }); - // Timedness is unknown until accept(), so an unfloored subscriber made before it - // settles its start then, against the groups that existed at that moment. - if (groups.start === undefined && !state.info.peek()) state.settle = () => this.#settleStart(); - } - - // Raise the floor of an unfloored subscriber made before accept(), once the track is - // known to be untimed. - #settleStart(): void { + const floor = groups.start === undefined ? undefined : bounds.start; + const live = subscription?.live ?? floor === undefined; + const edge = live ? this.#untimedStart(bounds.end) : undefined; + this.#cursor.set({ start: liveStart(live, floor, edge ?? 0) }); + // Timedness is unknown until accept(), so a live subscriber made before it settles + // its start then, against the groups that existed at that moment. + if (live && !state.info.peek()) state.settle = () => this.#settleStart(floor); + } + + // Raise the start of a live subscriber made before accept(), once the track is known to + // be untimed. + #settleStart(floor: number | undefined): void { this.#state.settle = undefined; - const start = this.#untimedStart(this.#cursor.peek().end); - if (start === undefined) return; + const edge = this.#untimedStart(this.#cursor.peek().end); + if (edge === undefined) return; + const start = liveStart(true, floor, edge); this.#cursor.update((cursor) => ({ ...cursor, start: Math.max(cursor.start, start) })); } @@ -1431,15 +1456,19 @@ export class Subscriber { #tryRecvGroup(): Recv { const groups = this.#state.groups.peek(); const { start, end } = this.#cursor.peek(); - while (groups.length > 0 && groups[0].sequence < start) groups.shift()?.close(); + // Groups below the floor stay buffered rather than dropped: a serving cursor lowers + // its floor again when the subscription gains `live` (see replaceGroups), and the + // cache window still evicts them. + let first = 0; + while (first < groups.length && groups[first].sequence < start) first++; const drift = this.#drift(); for (;;) { // The buffer is sequence-sorted, so an in-range group that arrives behind a // beyond-cap one sorts in front of it and is never blocked by it. - const group = groups[0]; + const group = groups[first]; if (!group || (end !== undefined && group.sequence >= end)) break; - groups.shift(); + groups.splice(first, 1); if (this.#isStale(group, drift)) { group.close(); continue; @@ -1447,7 +1476,7 @@ export class Subscriber { return { kind: "group", group: this.#guard(group) }; } - const group = groups[0]; + const group = groups[first]; const closed = this.#state.closed.peek(); if (closed instanceof Error) return { kind: "error", error: closed }; if (closed === undefined && !this.#complete()) return { kind: "idle" }; diff --git a/quest/m1/README.md b/quest/m1/README.md index 75fa63b5c3d..00ae9797b56 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -50,7 +50,6 @@ were done or not worth their cost. - [Binding audio delay](/quest/m1/binding-surface.md) - moq-ffi and every wrapper configure and observe audio playout delay - [Bump web-transport-iroh for the capsule close](/quest/m1/iroh-capsule-bump.md) - condition: moq-dev/web-transport#419 ships in a release, then moq's iroh HTTP/3 client reports a peer's close capsule - [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - every stream group in a lite subscription arrives or is dropped by name, and lite-07 drops its stream count for it -- [lite-07 Live flag](/quest/m1/lite-live.md) - a separate `Live` field on lite-07 SUBSCRIBE, so merged floors never starve a subscriber; late lower groups build on it - [FETCH_OK properties](/quest/m1/fetch-ok-properties.md) - our FETCH_OK carries the track properties SUBSCRIBE_OK does, as draft 16+ requires - [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - a lite-07 SUBSCRIBE asks for past and live ranges in either order and replaces FETCH; relays fill misses by range, including over moq-transport - [Cross-relay bursts re-run](/quest/m1/cross-relay-bursts.md) - condition: the #4349 reporter re-runs their A/B/C comparison against current cdn.moq.pro diff --git a/quest/m1/cache-max-age.md b/quest/m1/cache-max-age.md index 7c8f22eaf3c..4c1ce12d26a 100644 --- a/quest/m1/cache-max-age.md +++ b/quest/m1/cache-max-age.md @@ -123,7 +123,6 @@ encoding change; Max Age semantics in the lite draft change. ## Related - [Held group wakes](/quest/m1/held-group-wakes.md) - the `Wakes` entry this quest's wall-clock deadline rides on -- [lite-07 Live flag](/quest/m1/lite-live.md) - its untimed `Live` start (the latest group) follows this rule instead: replay what is not stale - [JS track handover](/quest/m1/js-group-handover.md) - mirrors the failover rule in JS - [Cache expiry growth](/quest/m1/cache-expiry-growth.md) - relay memory past the expiry window, in the same cache - [Generated @moq/net](/quest/m1/rs2ts/README.md) - retires js/net's hand-written model diff --git a/quest/m1/lite-late-lower-group.md b/quest/m1/lite-late-lower-group.md index a646bad88ce..4ce48377243 100644 --- a/quest/m1/lite-late-lower-group.md +++ b/quest/m1/lite-late-lower-group.md @@ -24,8 +24,7 @@ Decided: - An explicit floor delivers late lower groups within `max_delay`, matching moq-transport, draft-ietf-moq-transport ("subscriptions without a filter pass all Objects"), and moq-mux's floor handling (#3258). -- A live-only subscription (no floor, per - [lite-07 Live flag](/quest/m1/lite-live.md)) joins where the publisher +- A live-only subscription (`live` without a floor) joins where the publisher starts: the first served group becomes the floor, so a group created below it is dropped by design. This is what moq-mux's `container::Consumer` and the IETF publisher already assume. Fix the `Subscription` docs in moq-net (which say `None` equals a floor of @@ -34,10 +33,9 @@ Decided: (`floor.is_none()`). With an explicit floor, merged with `live` or not, `SUBSCRIBE_START` still reports the first served group, but nothing below it is suppressed; each subscriber's cursor filters locally. -- Aggregation follows [lite-07 Live flag](/quest/m1/lite-live.md): letting - an explicit floor survive mixing with no floor starved a floorless subscriber - until the floor's group existed (#5000's review), so the floor and `Live` - become separate fields and merge as min and OR. +- Aggregation already keeps the floor and `live` apart, merging them as min + and OR (lite-07 carries `Live`; older wires fold it into the floor), so this + PR drops its own floor-combining change. - `start_floor_suppresses_late_lower_arrivals` stays as the live-only case: it subscribes with no floor, receives group 7, raises the start to 7 as `SUBSCRIBE_START` does, and group 5 is still suppressed. Add its explicit-floor @@ -54,10 +52,6 @@ relay, with the in-process and moq-transport controls. Add a mixed case through a relay: a live-only subscriber already receiving group 1, then one with a floor of group 0, and a fresh group 0 reaches only the second. -## Required - -- [lite-07 Live flag](/quest/m1/lite-live.md) - floors and `Live` are separate fields, so merging them never starves a subscriber - ## Closes - [#4595](https://github.com/moq-dev/moq/issues/4595) - moq-lite: a group created below the first served group is silently dropped diff --git a/quest/m1/lite-live.md b/quest/m1/lite-live.md deleted file mode 100644 index 710fc3e7731..00000000000 --- a/quest/m1/lite-live.md +++ /dev/null @@ -1,92 +0,0 @@ -# [M] lite-07 Live flag - -## Goal - -Subscribers with different group and frame floors that share one upstream -subscription, in-process or through a relay, each receive everything their own -floor and `Subscriber Max Age` allow. No subscriber's floor starves another: -a resumed subscriber whose floor sits above the live edge never hides the -latest group from a subscriber that wants it. - -Today a floor and "no floor" are one field. On lite-06 `Group Start` 0 means no -floor, and the model keeps a separate `None` from pre-06's "absent means the -latest group". Merging an explicit floor with `None` either drops the late -group the floor asked for (main) or starves the floorless subscriber until the -floor's group exists, which on a quiet catalog track is never (#5000, the -failure #4940 fixed from the other side). - -## Plan - -Decided (maintainer, 2026-10-07): - -- lite-07 SUBSCRIBE and SUBSCRIBE_UPDATE carry a separate `Live` boolean. - `Group Start` and `Frame Start` become a plain absolute floor, with no - "0 means none" sentinel. -- A live-only subscriber has no floor, the neutral value for the minimum, so - lite-07 makes the floor optional. A subscription with neither a floor nor - `Live` is refused. The draft names how an absent floor is encoded (not a - value-plus-one sentinel) and how SUBSCRIBE_UPDATE clears a floor or `Live`. -- `Live: true` lowers the floor to frame 0 of the oldest group that - `Subscriber Max Age` has not expired, when that position sorts below the - floor. That is the start lite-06 already resolves for `Group Start` 0, so a - buffering subscriber keeps its backlog, and a max age of 0 still means the - latest group. On an untimed track the wall clock alone judges staleness - ([One max_age meaning](/quest/m1/cache-max-age.md)), so `Live` starts at - the oldest group that isn't stale there too; until that lands it means the - latest group (as `model/subscription.rs` documents today). Comparing - positions, not groups, means a merged floor of (3, 5) with group 3 as the - latest still delivers frames 0-4. -- [Model ranges](/quest/m1/subscribe-ranges/model.md) replaces the floor - with ranges after this lands: the floor becomes the lowest range start, and - `Live` stays its own flag. -- The model mirrors the wire: `Subscription { floor: Option<(group, frame)>, - live }` in Rust and `@moq/net`. -- Merging across subscribers (`Subscription::poll_combined`, JS - `combineSubscriptions`): `live` is the OR, and the floor is the minimum - `(group, frame)`. Each subscriber's own cursor still filters what it sees. -- Older wires map at the codec only, with no change to a published version: - lite-06 `Group Start` 0 with `Frame Start` 0 is `live`, any other pair a - floor (`(0, N)` is a catalog resume); pre-06 absent is `live`. - moq-transport has no current-group filter, so `live` is the bridge - `ietf::subscriber::subscribe_join` already sends: `Largest Object` plus the - current group's prefix, from a `Relative(1)` fill on draft-20+ or a relative - joining FETCH on older drafts. `Next Group Start` (draft-20 `Relative(0)`) - keeps its draft meaning, a floor at the group after Largest Object; a bare - `Largest Object` is a floor at the next object; draft-20 `Relative(1)` is - `live`; `Absolute Start` is a floor. -- When merged subscriptions need both `live` and a floor that an older - upstream wire cannot express in one SUBSCRIBE, a moq-transport upstream is - subscribed from `Absolute Start` at group 0, and the relay filters locally - by age and floor. The relay can't learn the oldest unexpired group - upstream: the ordinary join (`subscribe_join`) fetches only the latest - group's prefix. Decided 2026-10-08: no TRACK_STATUS step to find the - largest group first, since it covers only an unbuffered `live` and adds a - round trip. For an empty track the `live` mapping alone covers both, since - moq-transport starts it at {0, 0}. moq-lite has no - TRACK_STATUS, so a lite-06 or lite-03 to 05 upstream is subscribed at floor - 0 and the relay filters locally. On lite-06 that is `Group Start` 0, exactly - the `live` mapping. On lite-03 to 05 the floor must go out as an explicit - group 0 (replay from the beginning): the codec folds a floor of 0 to absent - today (`encode_start_group`), which those versions read as the latest - group, so a merged `{floor: 2, live}` with groups 2-4 retained would lose - 2-3. lite-01 and 02 carry no `Group Start` at all, so a merged floor through - them gets what they send from the latest group; that is a limitation of - those published versions, not something the codec can fix. -- Docs stay inline: the lite draft (field, semantics, lite-07 changelog), - `doc/concept`, and the Rust and JS API docs. - -Test with mock time: the starvation case (a floor-4 subscriber and a `live` -subscriber on a quiet track whose newest group is 3) in-process and through a -relay on lite-06 and lite-07-wip, a buffering `live` subscriber (max age > 0) -merged with a floor above the live edge, a frame floor merged with `live`, a -merged `{floor: 2, live}` through a lite-03 to 05 upstream with groups 2-4 -retained that delivers all three, a buffered `live` merged with a floor-5 -subscriber over a moq-transport upstream with fresh groups 2-4 that still -delivers 2-4, an untimed `live` merged with a floor, and the codec mapping -of each older wire. - -## Related - -- [One max_age meaning](/quest/m1/cache-max-age.md) - decides where an untimed `Live` starts -- [Late lower group](/quest/m1/lite-late-lower-group.md) - requires this; #5000 rebases onto it and drops its own floor-combining change -- [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - names undelivered groups once publishers report them diff --git a/quest/m1/lite07-finalize.md b/quest/m1/lite07-finalize.md index f96f99fa694..177efbfe829 100644 --- a/quest/m1/lite07-finalize.md +++ b/quest/m1/lite07-finalize.md @@ -72,7 +72,6 @@ is published; older versions are unchanged. ## Required -- [lite-07 Live flag](/quest/m1/lite-live.md) - SUBSCRIBE carries `Live` apart from its floor - [Restart](/quest/m0/broadcast-epoch/restart.md) - lite-07 carries the `Restart` announce - [In-band auth](/quest/m1/auth/README.md) - lite-07 carries the Auth Stream and UNAUTHORIZED (0x3B) - [Routes and announces](/quest/m1/cluster-routing/routes.md) - lite-07 carries the route layer: ROUTE per origin node and path-less ANNOUNCE, with the hop list gone diff --git a/quest/m1/pipeline-requests/subscribe.md b/quest/m1/pipeline-requests/subscribe.md index fadb81be329..a04abc7e045 100644 --- a/quest/m1/pipeline-requests/subscribe.md +++ b/quest/m1/pipeline-requests/subscribe.md @@ -94,6 +94,5 @@ blocks the Related quests below; whichever lands second rebases. ## Related - [Pipelined first FETCH](/quest/m1/pipeline-requests/fetch.md) - the same change for fetch-only readers -- [lite-07 Live flag](/quest/m1/lite-live.md) - reshapes the same SUBSCRIBE fields - [One max_age meaning](/quest/m1/cache-max-age.md) - the staleness rule that makes the wire clamp redundant - [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - replaces the SUBSCRIBE floor with ranges diff --git a/quest/m1/subscribe-ranges/README.md b/quest/m1/subscribe-ranges/README.md index 4f0a972d8d7..512ef7f1b7d 100644 --- a/quest/m1/subscribe-ranges/README.md +++ b/quest/m1/subscribe-ranges/README.md @@ -40,8 +40,7 @@ relay bounds a sparse FETCH only if the model can request ranges: group's existing prefix but never wait for its future objects. Downstream, an IETF FETCH is served from the model's ranges, capped at the Largest Object. -- The `Live` flag from [lite-07 Live flag](/quest/m1/lite-live.md) - stays a separate field beside the range list (maintainer, 10-07): a range +- The lite-07 `Live` flag stays a separate field beside the range list (maintainer, 10-07): a range starting above the live edge must not hide the latest group from a subscriber merged with it. - Ranges are frame-precise (`Position`), not whole groups. The IETF joining @@ -56,7 +55,6 @@ every sequence delivered or dropped. ## Required -- [lite-07 Live flag](/quest/m1/lite-live.md) - the `Live` field the range list sits beside - [Model ranges](/quest/m1/subscribe-ranges/model.md) - `Subscription` carries ranges and order, and `Dynamic` fills misses by range - [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - publishers name every group they won't deliver - [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - the lite-07 wire, Rust publisher and subscriber, and the draft diff --git a/quest/m1/subscribe-ranges/lite.md b/quest/m1/subscribe-ranges/lite.md index 102dd8709f7..e7e4a908742 100644 --- a/quest/m1/subscribe-ranges/lite.md +++ b/quest/m1/subscribe-ranges/lite.md @@ -20,6 +20,5 @@ FETCH, answered through the model's ranges. Run `just drafts check` and ## Required -- [lite-07 Live flag](/quest/m1/lite-live.md) - reshapes the same SUBSCRIBE first - [Model ranges](/quest/m1/subscribe-ranges/model.md) - the model this serves - [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - lite-07's DROP for holes diff --git a/quest/m1/subscribe-ranges/model.md b/quest/m1/subscribe-ranges/model.md index 37e8ae8c851..6dca138d267 100644 --- a/quest/m1/subscribe-ranges/model.md +++ b/quest/m1/subscribe-ranges/model.md @@ -25,6 +25,3 @@ Guidance: - `max_delay` caps every range. - Benchmark range count and span as separate axes (AGENTS.md fan-out rule). -## Required - -- [lite-07 Live flag](/quest/m1/lite-live.md) - reshapes the same `Subscription` first diff --git a/rs/moq-c/src/api.rs b/rs/moq-c/src/api.rs index 8960fd09afb..77593f3d15b 100644 --- a/rs/moq-c/src/api.rs +++ b/rs/moq-c/src/api.rs @@ -588,7 +588,9 @@ impl From<&moq_subscription> for moq_net::track::Subscription { .with_priority(subscription.priority) .with_max_delay(std::time::Duration::from_micros(subscription.max_delay_us)); if subscription.group_start_present { - out = out.with_start(moq_net::track::Position::group(subscription.group_start)); + out = out + .with_live(false) + .with_floor(moq_net::track::Position::group(subscription.group_start)); } if subscription.group_end_present { out = out.with_end(moq_net::track::Position::group(subscription.group_end)); diff --git a/rs/moq-c/src/consume.rs b/rs/moq-c/src/consume.rs index aed2e64588e..7c2118b16c2 100644 --- a/rs/moq-c/src/consume.rs +++ b/rs/moq-c/src/consume.rs @@ -638,7 +638,14 @@ impl Consume { fn apply_raw_subscription(track: &mut moq_net::track::Ordered, subscription: Option) { let subscription = subscription.unwrap_or_default(); - let start = subscription.start.map(|start| start.group).or_else(|| track.latest()); + // The live edge is the latest group, lowering any floor below it. + let floor = subscription.floor.map(|floor| floor.group); + let start = match subscription.live { + true => track + .latest() + .map(|latest| floor.map_or(latest, |floor| floor.min(latest))), + false => floor, + }; track.set_groups(( start.map_or(Bound::Unbounded, Bound::Included), subscription diff --git a/rs/moq-ffi/src/consumer.rs b/rs/moq-ffi/src/consumer.rs index 2061b1fb143..d4f5816ae3a 100644 --- a/rs/moq-ffi/src/consumer.rs +++ b/rs/moq-ffi/src/consumer.rs @@ -86,7 +86,8 @@ impl From for moq_net::track::Subscription { moq_net::track::Subscription::default() .with_priority(s.priority) .with_max_delay(std::time::Duration::from_micros(s.max_delay_us)) - .with_start(s.group_start.map(moq_net::track::Position::group)) + .with_live(s.group_start.is_none()) + .with_floor(s.group_start.map(moq_net::track::Position::group)) .with_end(s.group_end.map(moq_net::track::Position::group)) } } diff --git a/rs/moq-mux/src/container/consumer.rs b/rs/moq-mux/src/container/consumer.rs index 7fc0e95eb08..39be4e3dece 100644 --- a/rs/moq-mux/src/container/consumer.rs +++ b/rs/moq-mux/src/container/consumer.rs @@ -32,9 +32,10 @@ pub(crate) enum Event { /// skipped. A missing sequence gets the same tolerance: there is no way to tell a stream /// that lost the delivery race from one the cache evicted. /// -/// Delivery starts at the [`Subscription::start`](moq_net::track::Subscription::start) -/// floor when one is named, waiting for that group under the budget; without one it -/// starts wherever the publisher does, adopted from the first served groups. From there +/// Delivery starts at the [`Subscription::floor`](moq_net::track::Subscription::floor) +/// when one is named without [`live`](moq_net::track::Subscription::live), waiting for +/// that group under the budget; at the live edge it starts wherever the publisher does, +/// adopted from the first served groups. From there /// the same budget is the one rule that catches the consumer up to the live edge, and /// history the publisher no longer serves expires like any other gap, since its reach /// sits a full budget behind the newest content. @@ -121,13 +122,16 @@ impl Consumer { /// publisher preserves the same replay window. pub fn new(track: moq_net::track::Subscriber, format: F) -> Self { let subscription = track.subscription(); - // Delivery starts at the subscription floor; without one it starts wherever - // the publisher does, adopted from the first arrivals (see poll_read). Either - // way the age budget is the one rule that catches the cursor up to the live - // edge from there. The budget is clamped to the track's retention window, - // since history the publisher no longer keeps can't be waited for and would - // otherwise stall the catch-up by the excess. - let start = subscription.start.map(|position| position.group); + // Delivery starts at the subscription floor; at the live edge, which may sit + // below any floor, it starts wherever the publisher does, adopted from the first + // arrivals (see poll_read). Either way the age budget is the one rule that + // catches the cursor up to the live edge from there. The budget is clamped to + // the track's retention window, since history the publisher no longer keeps + // can't be waited for and would otherwise stall the catch-up by the excess. + let start = subscription + .floor + .filter(|_| !subscription.live) + .map(|position| position.group); let max_delay = subscription .max_delay .min(track.info().max_age.unwrap_or(std::time::Duration::MAX)); @@ -1905,7 +1909,8 @@ mod tests { let mut track = track_producer("test", hang::container::track_info(hang::catalog::PRIORITY.video)); let consumer_track = track.subscribe( moq_net::track::Subscription::default() - .with_start(moq_net::track::Position::group(0)) + .with_live(false) + .with_floor(moq_net::track::Position::group(0)) .with_max_delay(Duration::from_millis(500)), ); let mut consumer = Consumer::new(consumer_track, Container::Legacy(crate::container::Kind::Data)); diff --git a/rs/moq-mux/src/container/ts/export_test.rs b/rs/moq-mux/src/container/ts/export_test.rs index 52fb87c816d..5c80ee963eb 100644 --- a/rs/moq-mux/src/container/ts/export_test.rs +++ b/rs/moq-mux/src/container/ts/export_test.rs @@ -2233,7 +2233,8 @@ async fn read_si_groups(consumer: &moq_net::broadcast::Consumer, name: &str) -> .unwrap() .subscribe( moq_net::track::Subscription::default() - .with_start(moq_net::track::Position::group(0)) + .with_live(false) + .with_floor(moq_net::track::Position::group(0)) .with_max_delay(Duration::from_secs(5)), ) .await diff --git a/rs/moq-net/benches/group.rs b/rs/moq-net/benches/group.rs index a6461a17598..bf0e7608b65 100644 --- a/rs/moq-net/benches/group.rs +++ b/rs/moq-net/benches/group.rs @@ -214,7 +214,8 @@ fn filled_track(n: usize, payload: &Bytes) -> TrackCtx { fn replay() -> track::Subscription { track::Subscription::default() .with_max_delay(Duration::MAX) - .with_start(track::Position::group(0)) + .with_live(false) + .with_floor(track::Position::group(0)) } /// Drain N cached groups from a track, in arrival order and in sequence order. diff --git a/rs/moq-net/src/fuzz.rs b/rs/moq-net/src/fuzz.rs index 0ec56ed3683..e3e639c7440 100644 --- a/rs/moq-net/src/fuzz.rs +++ b/rs/moq-net/src/fuzz.rs @@ -345,9 +345,8 @@ impl LiteSample { track: "video".into(), priority: 2, max_delay: std::time::Duration::from_secs(10), - start_group: None, + start: lite::Start::LIVE, end_group: None, - start_frame: 0, end_frame: None, } .encode(w, version) @@ -529,17 +528,15 @@ impl Default for Messages { track: "video".into(), priority: 3, max_delay: Duration::from_millis(500), - start_group: Some(1_000), + start: lite::Start::floored(crate::track::Position::group(1_000)), end_group: None, - start_frame: 0, end_frame: None, }, lite_update: lite::SubscribeUpdate { priority: 4, max_delay: Duration::from_millis(500), - start_group: Some(1_000), + start: lite::Start::floored(crate::track::Position::group(1_000)), end_group: Some(2_000), - start_frame: 0, end_frame: None, }, lite_start: lite::SubscribeResponse::Start(lite::SubscribeStart { diff --git a/rs/moq-net/src/ietf/publisher.rs b/rs/moq-net/src/ietf/publisher.rs index db8ea33d87b..ce169a08aee 100644 --- a/rs/moq-net/src/ietf/publisher.rs +++ b/rs/moq-net/src/ietf/publisher.rs @@ -709,10 +709,16 @@ where // cover the group with no gap and no overlap. let edge = live_edge(&cache); let range = subscribe_range(&msg, edge, self.version); - subscription.start = range.start.map(|start| track::Position { + // A resolved start is a floor, including `Relative(1)` at frame 0 of the + // current group. That is not `live`: `live` with this serving budget asks a + // relay for the retained window. It is not an unfiltered {0, 0} either. + // No start at all (a relative filter before any group exists) is still the + // live edge, because there is no group to floor at yet. + subscription.floor = range.start.map(|start| track::Position { group: start.group, frame: start.object, }); + subscription.live = subscription.floor.is_none(); subscription.end = range.end.and_then(|end| match end.object { Some(object) => track::Position::after(end.group, object), None => track::Position::after_group(end.group), @@ -4739,6 +4745,88 @@ mod serve_tests { } } + /// `Relative(1)` floors demand at the current group. It is not `live` (that asks a + /// relay for the retained window) and not an unfiltered start at {0, 0}. + #[moq_net_sim::test] + async fn relative_one_floors_demand_at_the_current_group() { + let h = serve(Version::Draft20); + for (sequence, payload) in [(0u64, &b"old-0"[..]), (1, &b"old-1"[..]), (2, &b"now-2"[..])] { + let mut group = h.track.create_group(group::Info { sequence }).unwrap(); + group.write_frame(timestamp(), payload).unwrap(); + group.finish().unwrap(); + } + + let stream = Stream::open(&mut h.session.clone(), Version::Draft20).await.unwrap(); + let mut serve = std::pin::pin!( + h.publisher + .clone() + .run_subscribe_stream(stream, subscribe(Filter::Relative(1), None)) + ); + // The bytes can land before the origin's query subscription catches the + // narrowed demand. Wait for both, or a stale `live` still looks like the bug. + serve_until( + serve.as_mut(), + "Relative(1) floored demand at the current group", + || { + occurrences(&h.log, b"now-2") == 1 + && h.track.subscription().is_some_and(|subscription| { + !subscription.live && subscription.floor == Some(track::Position::group(2)) + }) + }, + ) + .await; + + let subscription = h.track.subscription().expect("the subscribe registered demand"); + assert!(!subscription.live, "Relative(1) is not the live flag: {subscription:?}"); + assert_eq!( + subscription.floor, + Some(track::Position::group(2)), + "the floor is the current group, not the retained window" + ); + assert_eq!(occurrences(&h.log, b"old-0"), 0); + assert_eq!(occurrences(&h.log, b"old-1"), 0); + + h.track.finish().unwrap(); + serve.await.unwrap(); + } + + /// Draft-20 Unfiltered stays a floor at {0, 0}, the whole retained cache, not `live`. + #[moq_net_sim::test] + async fn an_unfiltered_subscription_floors_demand_at_the_origin() { + let h = serve(Version::Draft20); + for (sequence, payload) in [(0u64, &b"old-0"[..]), (1, &b"old-1"[..]), (2, &b"now-2"[..])] { + let mut group = h.track.create_group(group::Info { sequence }).unwrap(); + group.write_frame(timestamp(), payload).unwrap(); + group.finish().unwrap(); + } + + let stream = Stream::open(&mut h.session.clone(), Version::Draft20).await.unwrap(); + let mut serve = std::pin::pin!( + h.publisher + .clone() + .run_subscribe_stream(stream, subscribe(Filter::Unfiltered, None)) + ); + serve_until(serve.as_mut(), "Unfiltered floored demand at the origin", || { + occurrences(&h.log, b"old-0") == 1 + && occurrences(&h.log, b"old-1") == 1 + && occurrences(&h.log, b"now-2") == 1 + && h.track.subscription().is_some_and(|subscription| { + !subscription.live && subscription.floor == Some(track::Position::group(0)) + }) + }) + .await; + + let subscription = h.track.subscription().expect("the subscribe registered demand"); + assert!( + !subscription.live, + "Unfiltered is a floor at the origin, not live: {subscription:?}" + ); + assert_eq!(subscription.floor, Some(track::Position::group(0))); + + h.track.finish().unwrap(); + serve.await.unwrap(); + } + /// A Next Object subscription never receives the already-published head of the /// current group: everything below the snapshot is outside the requested range. #[moq_net_sim::test] @@ -8066,10 +8154,13 @@ fn subscribe_range(msg: &ietf::Subscribe<'_>, edge: LiveEdge, version: Version) /// The Locations a single Location Filter selects, resolved against the live edge. fn filter_range(filter: Filter, edge: LiveEdge) -> ServeRange { match filter { - // No restriction. moq-lite starts at the beginning of the latest group, which is - // the join point it is built around; a subscription passes objects as they are - // published, so an absent filter is not a request to replay history. - Filter::Unfiltered => ServeRange::default(), + // Every Object, which the draft defines as an open ended absolute {0, 0}: the same + // start an explicit one asks for. A relay folding `live` into a floor sends it to + // reach back over what its readers may still use, and filters locally. + Filter::Unfiltered => ServeRange { + start: Some(Location { group: 0, object: 0 }), + end: None, + }, // `{Largest.Group, Largest.Object + 1}`. Everything below it, including the // already-published head of the current group, is outside the requested range, // so the join is mid-group by construction. The draft pairs this with a fill @@ -8247,12 +8338,18 @@ mod range_tests { assert_eq!(subscribe_range(&msg, EDGE, Version::Draft19), ServeRange::default()); } - /// An absent filter is "no restriction on what is forwarded", not a request for - /// history, so it joins at the live edge. + /// An absent filter is every Object, an open ended absolute {0, 0}, so it starts at the + /// origin like the explicit spelling the draft defines it as. #[test] - fn an_unfiltered_subscription_stays_live() { + fn an_unfiltered_subscription_starts_at_the_origin() { let msg = subscribe(Filter::Unfiltered); - assert_eq!(subscribe_range(&msg, EDGE, Version::Draft20), ServeRange::default()); + assert_eq!( + subscribe_range(&msg, EDGE, Version::Draft20), + ServeRange { + start: Some(Location { group: 0, object: 0 }), + end: None, + } + ); } /// Next Object starts one past the Largest Object, mid-group. Everything below it, diff --git a/rs/moq-net/src/ietf/subscriber.rs b/rs/moq-net/src/ietf/subscriber.rs index 5ed8e6dd67c..bfeafc36f35 100644 --- a/rs/moq-net/src/ietf/subscriber.rs +++ b/rs/moq-net/src/ietf/subscriber.rs @@ -1842,14 +1842,12 @@ where group_fetches: &mut TaskSet, ) -> Option { let subscription = target.subscription(); - let start = subscription.as_ref().and_then(|s| s.start); + // moq-transport has no `live` field, so it folds into the floor: `live` with a + // floor subscribes from group 0, and each reader's cursor filters by age and floor. + let start = subscription.as_ref().and_then(|s| s.folded_floor()); // A live join delivers nothing below the group SUBSCRIBE_OK names as Largest. let live = start.is_none(); - let join = match subscribe_join( - subscription.as_ref().and_then(|s| s.start), - subscription.as_ref().and_then(|s| s.end), - self.version, - ) { + let join = match subscribe_join(start, subscription.as_ref().and_then(|s| s.end), self.version) { Ok(join) => join, Err(err) => { target.fail(err); @@ -1886,10 +1884,7 @@ where state.subscribes.insert( request_id, TrackState { - resume: subscription - .as_ref() - .and_then(|s| s.start) - .filter(|start| start.frame != 0), + resume: start.filter(|start| start.frame != 0), // A resumed copy is the session's to abort from here on. producer: target.producer().cloned(), ..TrackState::pending(track_name.to_owned(), broadcast_path.to_owned(), fill.clone(), joining) @@ -9250,7 +9245,7 @@ mod joining_fetch_tests { let track = consumer.track("video").unwrap(); let subscription = match start { None => track.subscribe(None), - Some(start) => track.subscribe(track::Subscription::default().with_start(start)), + Some(start) => track.subscribe(track::Subscription::default().with_live(false).with_floor(start)), }; let request = dynamic.requested_track().await.expect("no track requested"); diff --git a/rs/moq-net/src/lite/publisher.rs b/rs/moq-net/src/lite/publisher.rs index d777c681766..6ba34bb1090 100644 --- a/rs/moq-net/src/lite/publisher.rs +++ b/rs/moq-net/src/lite/publisher.rs @@ -103,8 +103,9 @@ fn serving_max_delay(version: Version, requested: Duration) -> Duration { /// /// `latest` is the newest group when the SUBSCRIBE arrived, not once the subscription /// resolved: anything written in between is newer, so it is delivered. -fn position_cursor(track: &mut track::Subscriber, version: Version, start_group: Option, latest: Option) { - if version.resolves_start() || start_group.is_some() { +fn position_cursor(track: &mut track::Subscriber, version: Version, start: lite::Start, latest: Option) { + // A pre-06 wire decodes either `live` alone (no `Group Start`) or a floor alone. + if version.resolves_start() || !start.live { return; } @@ -1525,7 +1526,7 @@ impl Request for SubscribeServe { }; let bounds = Bounds::from(msg.as_ref()); - position_cursor(&mut track, shared.version, bounds.start_group, latest); + position_cursor(&mut track, shared.version, bounds.start, latest); let mut run = TrackRun::new(sub, track, bounds, track_priority_tx); if let Some(update) = update { run.update(update); @@ -1771,20 +1772,20 @@ mod test { ); let mut legacy = served(Version::Lite01); - position_cursor(&mut legacy, Version::Lite01, None, producer.latest()); + position_cursor(&mut legacy, Version::Lite01, lite::Start::LIVE, producer.latest()); assert_eq!(drain(&mut legacy), vec![2]); // Lite03-05 declare a budget, but their drafts define it as a staleness tolerance // and an absent start as the latest group, so they are pinned all the same. let mut tolerant = producer.subscribe(track::Subscription::default().with_max_delay(std::time::Duration::from_secs(5))); - position_cursor(&mut tolerant, Version::Lite05, None, producer.latest()); + position_cursor(&mut tolerant, Version::Lite05, lite::Start::LIVE, producer.latest()); assert_eq!(drain(&mut tolerant), vec![2]); // On lite-06 the declared budget is what resolves the start, so it stands. let mut declared = producer.subscribe(track::Subscription::default().with_max_delay(std::time::Duration::from_secs(5))); - position_cursor(&mut declared, Version::Lite06, None, producer.latest()); + position_cursor(&mut declared, Version::Lite06, lite::Start::LIVE, producer.latest()); assert_eq!(drain(&mut declared), vec![0, 1, 2]); } @@ -2374,30 +2375,25 @@ fn position_group(group: &mut group::Consumer, start: Option<(u64, u64)>, end: O /// A subscription's requested delivery range, exactly as it arrived on the wire. /// -/// The frame bounds qualify the start and end group, so they only mean anything paired -/// with one; [`Self::start_frame`] / [`Self::end_frame`] hand back that pairing. +/// The end frame qualifies the end group, so it only means anything paired with one; +/// [`Self::end_frame`] hands back that pairing. struct Bounds { - start_group: Option, - start_frame: u64, + start: lite::Start, end_group: Option, end_frame: Option, } impl Bounds { - /// The requested range as the model's half-open pair of [`track::Position`]s. + /// The requested range as the model's [`Subscription`]: its start, and a half-open + /// end. /// - /// The wire carries the two halves of each bound separately and both ends - /// inclusive; the model carries whole positions with an exclusive end. The - /// [`Subscription`] builders own that conversion, so this goes through them - /// rather than repeating it. + /// The wire carries the two halves of the end separately and inclusive; the model + /// carries a whole position with an exclusive end. The [`Subscription`] builders own + /// that conversion, so this goes through them rather than repeating it. fn positions(&self) -> crate::track::Subscription { - let mut sub = crate::track::Subscription::default(); - if let Some(group) = self.start_group { - sub = sub.with_start(track::Position { - group, - frame: self.start_frame, - }); - } + let mut sub = crate::track::Subscription::default() + .with_live(self.start.live) + .with_floor(self.start.floor); if let Some(group) = self.end_group { sub = sub.with_end(match self.end_frame { Some(frame) => track::Position::after(group, frame), @@ -2409,10 +2405,18 @@ impl Bounds { /// The group to apply a frame offset to and the offset itself, or `None` when /// delivery starts on a group boundary. - fn start_frame(&self) -> Option<(u64, u64)> { - self.start_group - .map(|group| (group, self.start_frame)) - .filter(|(_, frame)| *frame != 0) + /// + /// The floor's frame applies only while the floor is where delivery starts. `live` + /// starts from a group's first frame, so it takes over once that position sorts at or + /// below the floor: always on a timed track, where anything below the live edge is + /// expired anyway, and on an untimed one when the floor is not below `untimed_live`, + /// its latest group. + fn start_frame(&self, untimed_live: Option) -> Option<(u64, u64)> { + let floor = self.start.floor.filter(|floor| floor.frame != 0)?; + if self.start.live && untimed_live.is_none_or(|live| floor.group >= live) { + return None; + } + Some((floor.group, floor.frame)) } /// The group to cap and the last frame to serve within it (inclusive). @@ -2424,8 +2428,7 @@ impl Bounds { impl From<&lite::Subscribe<'_>> for Bounds { fn from(msg: &lite::Subscribe<'_>) -> Self { Self { - start_group: msg.start_group, - start_frame: msg.start_frame, + start: msg.start, end_group: msg.end_group, end_frame: msg.end_frame, } @@ -2435,8 +2438,7 @@ impl From<&lite::Subscribe<'_>> for Bounds { impl From<&lite::SubscribeUpdate> for Bounds { fn from(msg: &lite::SubscribeUpdate) -> Self { Self { - start_group: msg.start_group, - start_frame: msg.start_frame, + start: msg.start, end_group: msg.end_group, end_frame: msg.end_frame, } @@ -2553,6 +2555,15 @@ impl Subscription { } } +/// Where [`track::Subscription::live`] starts on an untimed track: its latest group, or +/// `None` on a timed track (or an untimed one with nothing yet). +fn untimed_live(track: &track::Subscriber) -> Option { + match track.info().timescale { + Some(_) => None, + None => track.latest(), + } +} + /// A subscription's run loop: one subscriber cursor serving groups, datagrams, /// and SUBSCRIBE_UPDATE messages, with an in-flight group machine per group. struct TrackRun { @@ -2600,7 +2611,7 @@ impl TrackRun { let datagrams = ctx.version.has_datagrams() && ctx.session.max_datagram_size() > 0; Self { - start_frame: bounds.start_frame(), + start_frame: bounds.start_frame(untimed_live(&track)), end_frame: bounds.end_frame(), ctx, track, @@ -2630,30 +2641,32 @@ impl TrackRun { // Feed the full update into the model subscriber so the producer's // aggregate reflects it (and a relay re-forwards it upstream). // Read first: `update` replaces these preferences. - let floored = self.track.subscription().start.is_some_and(|start| start.group > 0); + let was_live = self.track.subscription().live; let bounds = Bounds::from(&upd); let _ = self.track.update(crate::track::Subscription { priority: upd.priority, max_delay: serving_max_delay(self.ctx.version, upd.max_delay), ..bounds.positions() }); - // An explicit start moves the read cursor. Lite-06+ encodes an absent - // start as no floor, so a subscription that had one above group 0 has to - // drop back to 0: the cursor only rises on its own, and a finished group - // below the old floor (a quiet catalog) is never served otherwise. A - // cursor that was already at 0 stays where the first served group put it. - // No fresh SUBSCRIBE_START follows: the subscriber clears its permanent-miss - // floor on the same update, so both sides have to change together. - // Pre-06 an absent start means the latest group, which `position_cursor` - // already applied. - match upd.start_group { - Some(start_group) => self.track.start_at(start_group), - None if floored && self.ctx.version.resolves_start() => self.track.start_at(0), - None => {} + // A floor alone moves the read cursor to it. Gaining `live` drops the cursor + // back to the live edge: the cursor only rises on its own, and a finished group + // below the old floor (a quiet catalog) is never served otherwise. A cursor that + // was already live stays where the first served group put it. No fresh + // SUBSCRIBE_START follows: the subscriber clears its permanent-miss floor on the + // same update, so both sides have to change together. Pre-06 `live` means the + // latest group, which `position_cursor` already applied. + let untimed_live = untimed_live(&self.track); + match (upd.start.live, upd.start.floor) { + (false, Some(floor)) => self.track.start_at(floor.group), + (true, floor) if !was_live && self.ctx.version.resolves_start() => { + let live = untimed_live.unwrap_or(0); + self.track.start_at(floor.map_or(live, |floor| floor.group.min(live))); + } + _ => {} } self.track .end_at(upd.end_group.map_or(Bound::Unbounded, Bound::Included)); - self.start_frame = bounds.start_frame(); + self.start_frame = bounds.start_frame(untimed_live); self.end_frame = bounds.end_frame(); } @@ -2713,7 +2726,9 @@ impl TrackRun { if self.emit_range && !self.start_sent && self.ctx.version.has_largest() - && let Some(start) = self.track.subscription().start + && let subscription = self.track.subscription() + && !subscription.live + && let Some(start) = subscription.floor && let Poll::Ready(Some(largest)) = self.track.poll_live(waiter) && start > largest { @@ -3167,8 +3182,7 @@ mod serve_group_test { #[test] fn bounds_convert_to_positions() { let whole = Bounds { - start_group: None, - start_frame: 0, + start: lite::Start::LIVE, end_group: Some(5), end_frame: None, }; @@ -3181,24 +3195,41 @@ mod serve_group_test { assert_eq!(capped.positions().end, Some(track::Position { group: 5, frame: 3 })); let started = Bounds { - start_group: Some(5), - start_frame: 3, + start: lite::Start::floored(crate::track::Position { group: 5, frame: 3 }), end_group: None, end_frame: None, }; let positions = started.positions(); - assert_eq!(positions.start, Some(track::Position { group: 5, frame: 3 })); + assert_eq!( + (positions.live, positions.floor), + (false, Some(track::Position { group: 5, frame: 3 })) + ); assert_eq!(positions.end, None); + assert_eq!(started.start_frame(None), Some((5, 3))); + + // `live` starts from a group's first frame, so it takes over the floor's frame + // whenever it sorts at or below the floor: always on a timed track (`None`), and on + // an untimed one unless the floor is below its latest group. + let live = Bounds { + start: lite::Start { + live: true, + floor: Some(track::Position { group: 5, frame: 3 }), + }, + ..started + }; + assert_eq!(live.start_frame(None), None); + assert_eq!(live.start_frame(Some(5)), None); + assert_eq!(live.start_frame(Some(4)), None); + assert_eq!(live.start_frame(Some(6)), Some((5, 3))); // A frame bound the peer sent without its group has nothing to count from, so it // cannot reach the model at all. let orphan = Bounds { - start_group: None, - start_frame: 3, + start: lite::Start::LIVE, end_group: None, end_frame: Some(7), }; - assert_eq!((orphan.positions().start, orphan.positions().end), (None, None)); + assert_eq!((orphan.positions().floor, orphan.positions().end), (None, None)); } /// A group whose head the publisher no longer holds is skipped, not served short: @@ -3633,8 +3664,7 @@ mod serve_group_test { opens: Default::default(), }; let bounds = Bounds { - start_group: Some(0), - start_frame: 0, + start: lite::Start::floored(crate::track::Position::group(0)), end_group: None, end_frame: None, }; @@ -3682,7 +3712,8 @@ mod serve_group_test { fn new(track: &mut track::Producer, start_group: u64) -> Self { track.request_start(Some(0)).unwrap(); let subscription = track::Subscription::default() - .with_start(track::Position::group(start_group)) + .with_live(false) + .with_floor(track::Position::group(start_group)) .with_max_delay(Duration::from_secs(30)); let subscriber = track.subscribe(subscription); @@ -3703,8 +3734,7 @@ mod serve_group_test { opens: opens.clone(), }; let bounds = Bounds { - start_group: Some(start_group), - start_frame: 0, + start: lite::Start::floored(crate::track::Position::group(start_group)), end_group: None, end_frame: None, }; @@ -3753,9 +3783,8 @@ mod serve_group_test { relay.run.update(lite::SubscribeUpdate { priority: 0, max_delay: Duration::from_secs(30), - start_group: None, + start: lite::Start::LIVE, end_group: None, - start_frame: 0, end_frame: None, }); relay.settle(); @@ -3779,9 +3808,8 @@ mod serve_group_test { relay.run.update(lite::SubscribeUpdate { priority: 0, max_delay: Duration::from_secs(30), - start_group: None, + start: lite::Start::LIVE, end_group: None, - start_frame: 0, end_frame: None, }); relay.settle(); @@ -3802,9 +3830,8 @@ mod serve_group_test { let update = lite::SubscribeUpdate { priority: 0, max_delay: Duration::ZERO, - start_group: Some(7), + start: lite::Start::floored(crate::track::Position::group(7)), end_group: None, - start_frame: 0, end_frame: None, }; relay.run.update(update); @@ -4376,9 +4403,8 @@ mod tests { track: "video".into(), priority: 0, max_delay: Duration::ZERO, - start_group: None, + start: lite::Start::LIVE, end_group: None, - start_frame: 0, end_frame: None, }; leave_request::>(version, &subscribe, wait, close).await; @@ -4510,9 +4536,8 @@ mod tests { track: "video".into(), priority: 1, max_delay: Duration::ZERO, - start_group: None, + start: lite::Start::LIVE, end_group: None, - start_frame: 0, end_frame: None, } .encode(&mut crate::coding::Encoder::new(&mut script, version.into()), version) @@ -4533,9 +4558,8 @@ mod tests { let update = lite::SubscribeUpdate { priority: 7, max_delay: Duration::ZERO, - start_group: None, + start: lite::Start::LIVE, end_group: None, - start_frame: 0, end_frame: None, }; session.push(&update.encode_bytes(version).unwrap()); @@ -4577,9 +4601,8 @@ mod tests { track: "video".into(), priority: 0, max_delay: Duration::ZERO, - start_group: None, + start: lite::Start::LIVE, end_group: None, - start_frame: 0, end_frame: None, } .encode(&mut crate::coding::Encoder::new(script, VERSION.into()), VERSION) diff --git a/rs/moq-net/src/lite/subscribe.rs b/rs/moq-net/src/lite/subscribe.rs index 2dc96c40470..9d7da85885d 100644 --- a/rs/moq-net/src/lite/subscribe.rs +++ b/rs/moq-net/src/lite/subscribe.rs @@ -20,21 +20,141 @@ pub struct Subscribe<'a> { pub track: Cow<'a, str>, pub priority: u8, pub max_delay: std::time::Duration, - /// The minimum group to deliver (a floor). On lite-06 the wire carries the raw - /// sequence and `None` is interchangeable with `Some(0)`: a floor of 0 constrains - /// nothing, and the start resolves from `max_delay`. Pre-06 wires encode the - /// sequence + 1 and an absent start means the latest group. - pub start_group: Option, + /// Where delivery may start; see [`Start`]. + pub start: Start, pub end_group: Option, - /// First frame to deliver within `start_group`'s group; 0 is the whole group. - /// Lite06+ only. It qualifies the named group, so it needs `start_group` to name one - /// (`Some`, including `Some(0)`: group 0 can host a mid-group resume). - pub start_frame: u64, /// Last frame to deliver (inclusive) within `end_group`'s group, or `None` for the /// whole group. Lite06+ only, and meaningless without an explicit `end_group`. pub end_frame: Option, } +/// Where a subscription may start: the live edge, a floor, or both, as SUBSCRIBE and +/// SUBSCRIBE_UPDATE carry them. See [`crate::track::Subscription::live`]. +/// +/// Lite-07 carries both fields. Older versions have only `Group Start`, so the codec folds +/// `live` into it (see [`crate::track::Subscription::folded_floor`]) and reads it back: +/// lite-06 `Group Start` 0 with `Frame Start` 0 is `live`, and any other pair a floor +/// (`(0, N)` is a catalog resume); a pre-06 absent `Group Start` is `live`, and lite-01/02 +/// carry none at all. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] +pub struct Start { + /// Deliver from the live edge. + pub live: bool, + /// The lowest position to deliver, or `None` for no floor. + pub floor: Option, +} + +impl Start { + /// Live alone, which is what a wire without a `Group Start` asks for. + pub const LIVE: Self = Self { + live: true, + floor: None, + }; + + /// The subscription's start. Neither `live` nor a floor asks for nothing. + pub(crate) fn of(subscription: &crate::track::Subscription) -> Self { + Self { + live: subscription.live, + floor: subscription.floor, + } + } + + /// The floor a version without `Live` carries; see [`Self`]. + fn folded(self) -> Option { + crate::track::Subscription::default() + .with_live(self.live) + .with_floor(self.floor) + .folded_floor() + } + + /// Decode the start of a SUBSCRIBE or SUBSCRIBE_UPDATE: everything on lite-07 and + /// pre-06, and the `Group Start` half on lite-06, whose `Frame Start` trails the end + /// group and is applied by [`Self::decode_frame`]. + fn decode(r: &mut Decoder<'_>, version: Version) -> Result { + if version.has_live() { + let live = r.bool()?; + let floor = match r.bool()? { + true => Some(crate::track::Position { + group: r.varint()?, + frame: r.varint()?, + }), + false => None, + }; + if !live && floor.is_none() { + return Err(DecodeError::InvalidSubscribeLocation); + } + return Ok(Self { live, floor }); + } + if version.resolves_start() { + return Ok(Self::floored(crate::track::Position::group(r.varint()?))); + } + Ok(match r.varint_opt()? { + Some(group) => Self::floored(crate::track::Position::group(group)), + None => Self::LIVE, + }) + } + + /// Apply lite-06's trailing `Frame Start`, then read the pair back: `(0, 0)` is `live`. + fn decode_frame(self, r: &mut Decoder<'_>, version: Version) -> Result { + if version.has_live() || !version.has_frame_bounds() { + return Ok(self); + } + let frame = r.varint()?; + Ok(match self.floor { + Some(floor) if floor.group == 0 && frame == 0 => Self::LIVE, + Some(floor) => Self::floored(crate::track::Position { + group: floor.group, + frame, + }), + None => self, + }) + } + + pub(crate) fn floored(floor: crate::track::Position) -> Self { + Self { + live: false, + floor: Some(floor), + } + } + + /// Encode what [`Self::decode`] reads. + /// + /// Neither `live` nor a floor has no encoding, since it asks for nothing. A pre-06 + /// `Group Start` cannot qualify a frame, so a floor partway through a group is refused + /// rather than widened to the whole group. + fn encode(self, w: &mut Encoder<'_>, version: Version) -> Result<(), EncodeError> { + if !self.live && self.floor.is_none() { + return Err(EncodeError::InvalidState); + } + if version.has_live() { + w.bool(self.live); + w.bool(self.floor.is_some()); + if let Some(floor) = self.floor { + w.varint(floor.group)?; + w.varint(floor.frame)?; + } + return Ok(()); + } + let folded = self.folded(); + if version.resolves_start() { + return w.varint(folded.map_or(0, |floor| floor.group)); + } + if folded.is_some_and(|floor| floor.frame != 0) { + return Err(EncodeError::Version); + } + // The sequence + 1, so an explicit group 0 (replay from the beginning) is 1. + w.varint_opt(folded.map(|floor| floor.group)) + } + + /// Encode lite-06's trailing `Frame Start`; see [`Self::decode_frame`]. + fn encode_frame(self, w: &mut Encoder<'_>, version: Version) -> Result<(), EncodeError> { + if version.has_live() || !version.has_frame_bounds() { + return Ok(()); + } + w.varint(self.folded().map_or(0, |floor| floor.frame)) + } +} + impl Version { /// Whether this version's SUBSCRIBE carries the subscriber's max delay preference. /// @@ -54,19 +174,19 @@ impl Message for Subscribe<'_> { let track = Cow::Owned(r.string()?); let priority = r.u8()?; - let (max_delay, start_group, end_group) = match version { - Version::Lite01 | Version::Lite02 => (std::time::Duration::ZERO, None, None), + let (max_delay, start, end_group) = match version { + Version::Lite01 | Version::Lite02 => (std::time::Duration::ZERO, Start::LIVE, None), _ => { skip_group_order(r, version)?; let max_delay = std::time::Duration::from_millis(r.varint()?); - let start_group = decode_start_group(r, version)?; + let start = Start::decode(r, version)?; let end_group = r.varint_opt()?; - (max_delay, start_group, end_group) + (max_delay, start, end_group) } }; - let (start_frame, end_frame) = decode_frame_bounds(r, version, start_group, end_group)?; - let start_group = canonical_start_group(version, start_group, start_frame); + let start = start.decode_frame(r, version)?; + let end_frame = decode_end_frame(r, version, end_group)?; Ok(Self { id, @@ -75,9 +195,8 @@ impl Message for Subscribe<'_> { track, priority, max_delay, - start_group, + start, end_group, - start_frame, end_frame, }) } @@ -94,21 +213,13 @@ impl Message for Subscribe<'_> { _ => { pad_group_order(w, version)?; w.varint(u64::try_from(self.max_delay.as_millis()).map_err(|_| EncodeError::BoundsExceeded)?)?; - encode_start_group(w, version, self.start_group)?; + self.start.encode(w, version)?; w.varint_opt(self.end_group)?; } } - encode_frame_bounds( - w, - version, - self.start_group, - self.start_frame, - self.end_group, - self.end_frame, - )?; - - Ok(()) + self.start.encode_frame(w, version)?; + encode_end_frame(w, version, self.end_group, self.end_frame) } } @@ -131,92 +242,43 @@ pub(super) fn pad_group_order(w: &mut Encoder<'_>, version: Version) -> Result<( Ok(()) } -/// Decode the `Group Start` field shared by SUBSCRIBE and SUBSCRIBE_UPDATE. +/// Decode the trailing `Frame End` shared by SUBSCRIBE and SUBSCRIBE_UPDATE. /// -/// Lite-06 carries the raw floor, so every value names a concrete group and 0 decodes as -/// `Some(0)`. Pre-06 wires encode the sequence + 1, with 0 meaning the latest group -/// (`None`). Callers canonicalize with [`canonical_start_group`] once the frame bounds -/// are known. -fn decode_start_group(r: &mut Decoder<'_>, version: Version) -> Result, DecodeError> { - if version.resolves_start() { - return Ok(Some(r.varint()?)); - } - r.varint_opt() -} - -/// Canonicalize a decoded floor: a lite-06 `Group Start` of 0 with no frame offset is the -/// same absence of a constraint as no floor at all, so it decodes as `None`. Group 0 stays -/// named only when a `Frame Start` actually qualifies it (a subscription can resume -/// partway through group 0, e.g. a catalog that never leaves it). -fn canonical_start_group(version: Version, start_group: Option, start_frame: u64) -> Option { - match (start_group, start_frame) { - (Some(0), 0) if version.resolves_start() => None, - (start_group, _) => start_group, - } -} - -/// Encode the `Group Start` field shared by SUBSCRIBE and SUBSCRIBE_UPDATE. -/// -/// The inverse of [`decode_start_group`]: lite-06 writes the raw floor (`None` and -/// `Some(0)` are the same absence of a constraint), while a pre-06 wire gets `Some(0)` -/// folded back to absent. On those wires an explicit group 0 means "replay from the -/// beginning", which is not what a vacuous floor asks for. -fn encode_start_group(w: &mut Encoder<'_>, version: Version, start_group: Option) -> Result<(), EncodeError> { - if version.resolves_start() { - return w.varint(start_group.unwrap_or(0)); - } - w.varint_opt(start_group.filter(|&group| group > 0)) -} - -/// Decode the trailing `Frame Start` / `Frame End` pair shared by SUBSCRIBE, -/// SUBSCRIBE_UPDATE, and FETCH. -/// -/// Older versions carry no such fields, so they decode as the whole group. A frame bound +/// Older versions carry no such field, so they decode as the whole group. A frame bound /// without the group bound it qualifies is a protocol violation: frames are numbered per /// group, so there is nothing to count from. -fn decode_frame_bounds( - r: &mut Decoder<'_>, - version: Version, - start_group: Option, - end_group: Option, -) -> Result<(u64, Option), DecodeError> { +fn decode_end_frame(r: &mut Decoder<'_>, version: Version, end_group: Option) -> Result, DecodeError> { if !version.has_frame_bounds() { - return Ok((0, None)); + return Ok(None); } - let start_frame = r.varint()?; let end_frame = r.varint_opt()?; - - if (start_frame != 0 && start_group.is_none()) || (end_frame.is_some() && end_group.is_none()) { + if end_frame.is_some() && end_group.is_none() { return Err(DecodeError::InvalidSubscribeLocation); } - - Ok((start_frame, end_frame)) + Ok(end_frame) } -/// Encode the trailing `Frame Start` / `Frame End` pair, a no-op before lite-06. -fn encode_frame_bounds( +/// Encode the trailing `Frame End`, a no-op before lite-06. +fn encode_end_frame( w: &mut Encoder<'_>, version: Version, - start_group: Option, - start_frame: u64, end_group: Option, end_frame: Option, ) -> Result<(), EncodeError> { - if (start_frame != 0 && start_group.is_none()) || (end_frame.is_some() && end_group.is_none()) { + if end_frame.is_some() && end_group.is_none() { return Err(EncodeError::InvalidState); } if !version.has_frame_bounds() { - // Nothing carries the bounds, so silently widening to the whole group would + // Nothing carries the bound, so silently widening to the whole group would // deliver frames the caller excluded. Refuse instead. - if start_frame != 0 || end_frame.is_some() { + if end_frame.is_some() { return Err(EncodeError::Version); } return Ok(()); } - w.varint(start_frame)?; w.varint_opt(end_frame) } @@ -376,16 +438,16 @@ impl Message for SubscribeEnd { /// Sent by the subscriber to update subscription parameters. /// -/// Lite03+ only. +/// Lite03+ only. Every field replaces the subscription's, so an update clears a floor or +/// `live` by leaving it out. #[allow(dead_code)] #[derive(Clone, Debug)] pub struct SubscribeUpdate { pub priority: u8, pub max_delay: std::time::Duration, - pub start_group: Option, + /// See [`Subscribe::start`]. + pub start: Start, pub end_group: Option, - /// See [`Subscribe::start_frame`]. - pub start_frame: u64, /// See [`Subscribe::end_frame`]. pub end_frame: Option, } @@ -402,21 +464,16 @@ impl Message for SubscribeUpdate { let priority = r.u8()?; skip_group_order(r, version)?; let max_delay = std::time::Duration::from_millis(r.varint()?); - let start_group = decode_start_group(r, version)?; - let end_group = match r.varint()? { - 0 => None, - group => Some(group - 1), - }; - - let (start_frame, end_frame) = decode_frame_bounds(r, version, start_group, end_group)?; - let start_group = canonical_start_group(version, start_group, start_frame); + let start = Start::decode(r, version)?; + let end_group = r.varint_opt()?; + let start = start.decode_frame(r, version)?; + let end_frame = decode_end_frame(r, version, end_group)?; Ok(Self { priority, max_delay, - start_group, + start, end_group, - start_frame, end_frame, }) } @@ -432,21 +489,10 @@ impl Message for SubscribeUpdate { w.u8(self.priority); pad_group_order(w, version)?; w.varint(u64::try_from(self.max_delay.as_millis()).map_err(|_| EncodeError::BoundsExceeded)?)?; - - encode_start_group(w, version, self.start_group)?; - + self.start.encode(w, version)?; w.varint_opt(self.end_group)?; - - encode_frame_bounds( - w, - version, - self.start_group, - self.start_frame, - self.end_group, - self.end_frame, - )?; - - Ok(()) + self.start.encode_frame(w, version)?; + encode_end_frame(w, version, self.end_group, self.end_frame) } } @@ -710,6 +756,8 @@ mod test { assert_eq!(buf[0], 1); } + use crate::track::Position; + fn subscribe_sample() -> Subscribe<'static> { Subscribe { epoch: None, @@ -718,40 +766,227 @@ mod test { track: Cow::Borrowed("video"), priority: 3, max_delay: std::time::Duration::from_millis(250), - start_group: Some(7), + start: Start::floored(Position { group: 7, frame: 4 }), end_group: Some(9), - start_frame: 4, end_frame: Some(2), } } + fn encode(msg: &Subscribe<'_>, version: Version) -> Result, EncodeError> { + let mut buf = Vec::new(); + msg.encode_msg(&mut Encoder::new(&mut buf, version.into()), version)?; + Ok(buf) + } + + fn decode(buf: &[u8], version: Version) -> Result, DecodeError> { + crate::coding::decode_buf(&mut &buf[..], version, Subscribe::decode_msg) + } + + /// What `start` reads back as on `version`, through SUBSCRIBE and SUBSCRIBE_UPDATE. + fn roundtrip(start: Start, version: Version) -> Start { + let msg = Subscribe { + start, + end_group: None, + end_frame: None, + ..subscribe_sample() + }; + let got = decode(&encode(&msg, version).unwrap(), version).unwrap().start; + + let update = SubscribeUpdate { + priority: 0, + max_delay: std::time::Duration::ZERO, + start, + end_group: None, + end_frame: None, + }; + let mut buf = Vec::new(); + update + .encode_msg(&mut Encoder::new(&mut buf, version.into()), version) + .unwrap(); + let updated = crate::coding::decode_buf(&mut buf.as_slice(), version, SubscribeUpdate::decode_msg) + .unwrap() + .start; + assert_eq!(got, updated, "{version:?}: SUBSCRIBE and SUBSCRIBE_UPDATE agree"); + got + } + + fn both(floor: Position) -> Start { + Start { + live: true, + floor: Some(floor), + } + } + #[test] fn subscribe_frame_bounds_roundtrip() { let msg = subscribe_sample(); - let mut buf = Vec::new(); - msg.encode_msg(&mut Encoder::new(&mut buf, Version::Lite06.into()), Version::Lite06) - .unwrap(); - let got = crate::coding::decode_buf(&mut buf.as_slice(), Version::Lite06, Subscribe::decode_msg).unwrap(); - assert_eq!((got.start_group, got.start_frame), (Some(7), 4)); - assert_eq!((got.end_group, got.end_frame), (Some(9), Some(2))); + for version in [Version::Lite06, Version::Lite07] { + let got = decode(&encode(&msg, version).unwrap(), version).unwrap(); + assert_eq!(got.start, msg.start, "{version:?}"); + assert_eq!((got.end_group, got.end_frame), (Some(9), Some(2)), "{version:?}"); + } + } + + /// Lite-07 carries `Live` and an optional floor separately, so every combination that + /// asks for something survives the round trip, including a floor of (0, 0). + #[test] + fn lite07_carries_live_beside_the_floor() { + for start in [ + Start::LIVE, + Start::floored(Position::group(0)), + Start::floored(Position { group: 3, frame: 5 }), + both(Position::group(0)), + both(Position { group: 4, frame: 2 }), + ] { + assert_eq!(roundtrip(start, Version::Lite07), start); + } + + // `Live`, then the floor's presence, then the floor itself. + let msg = Subscribe { + start: both(Position { group: 4, frame: 2 }), + end_group: None, + end_frame: None, + ..subscribe_sample() + }; + let buf = encode(&msg, Version::Lite07).unwrap(); + let without_floor = encode( + &Subscribe { + start: Start::LIVE, + ..msg.clone() + }, + Version::Lite07, + ) + .unwrap(); + assert_eq!(buf.len(), without_floor.len() + 2, "the floor's two varints are gone"); + assert!(buf.ends_with(&[1, 1, 4, 2, 0, 0]), "{buf:?}"); + assert!(without_floor.ends_with(&[1, 0, 0, 0]), "{without_floor:?}"); + } + + /// Neither `Live` nor a floor asks for nothing, so it has no encoding and decoding one is + /// a protocol violation. + #[test] + fn lite07_refuses_neither_live_nor_a_floor() { + let neither = Subscribe { + start: Start::default(), + ..subscribe_sample() + }; + assert!(matches!( + encode(&neither, Version::Lite07), + Err(EncodeError::InvalidState) + )); + + let mut buf = encode( + &Subscribe { + start: Start::LIVE, + end_group: None, + end_frame: None, + ..subscribe_sample() + }, + Version::Lite07, + ) + .unwrap(); + // Clear `Live`, which sits just before the absent floor and the two end varints. + let live_at = buf.len() - 4; + assert_eq!(buf[live_at], 1); + buf[live_at] = 0; + assert!(matches!( + decode(&buf, Version::Lite07), + Err(DecodeError::InvalidSubscribeLocation) + )); + // Any other byte is not a boolean. + buf[live_at] = 2; + assert!(matches!(decode(&buf, Version::Lite07), Err(DecodeError::InvalidValue))); + } + + /// Lite-06 has no `Live`: `Group Start` 0 with `Frame Start` 0 is `live`, and any other + /// pair a floor, so `live` with a floor goes out as (0, 0) and the receiver filters. + #[test] + fn lite06_folds_live_into_group_start_zero() { + let version = Version::Lite06; + assert_eq!(roundtrip(Start::LIVE, version), Start::LIVE); + assert_eq!(roundtrip(both(Position::group(4)), version), Start::LIVE); + let floor = Start::floored(Position { group: 7, frame: 4 }); + assert_eq!(roundtrip(floor, version), floor); + // A catalog resume partway through group 0 stays a floor. + let catalog = Start::floored(Position { group: 0, frame: 4 }); + assert_eq!(roundtrip(catalog, version), catalog); + // A floor of (0, 0) reads back as `live`, which lite-06 cannot tell apart. + assert_eq!(roundtrip(Start::floored(Position::group(0)), version), Start::LIVE); + + // Byte-identical to what lite-06 always sent for "no floor". + let live = Subscribe { + start: Start::LIVE, + ..subscribe_sample() + }; + let folded = Subscribe { + start: both(Position::group(4)), + ..subscribe_sample() + }; + assert_eq!(encode(&live, version).unwrap(), encode(&folded, version).unwrap()); + } + + /// Pre-06 wires encode `Group Start` as the sequence + 1, with 0 meaning the latest + /// group: absent is `live`, and `live` with a floor goes out as an explicit group 0, + /// replay from the beginning, so the receiver can filter locally. + #[test] + fn pre06_folds_live_into_an_explicit_group_zero() { + for version in [Version::Lite03, Version::Lite04, Version::Lite05] { + assert_eq!(roundtrip(Start::LIVE, version), Start::LIVE, "{version:?}"); + assert_eq!( + roundtrip(both(Position::group(2)), version), + Start::floored(Position::group(0)), + "{version:?}" + ); + let floor = Start::floored(Position::group(7)); + assert_eq!(roundtrip(floor, version), floor, "{version:?}"); + // An explicit group 0 is 1 on the wire, not folded back to absent. + assert_eq!( + roundtrip(Start::floored(Position::group(0)), version), + Start::floored(Position::group(0)), + "{version:?}" + ); + } + + let msg = Subscribe { + start: Start::floored(Position::group(7)), + end_frame: None, + ..subscribe_sample() + }; + let lite05 = encode(&msg, Version::Lite05).unwrap(); + let lite06 = encode(&msg, Version::Lite06).unwrap(); + assert_ne!(lite05, lite06, "7 + 1 on lite-05, 7 on lite-06"); + } + + /// Lite-01 and 02 carry no `Group Start`, so every subscription decodes as `live`. + #[test] + fn lite01_carries_no_start() { + for version in [Version::Lite01, Version::Lite02] { + let msg = Subscribe { + start: Start::floored(Position::group(7)), + end_group: None, + end_frame: None, + ..subscribe_sample() + }; + assert_eq!( + decode(&encode(&msg, version).unwrap(), version).unwrap().start, + Start::LIVE + ); + } } /// The whole-group defaults are what a version without the fields decodes to, so /// lite-05 stays byte-identical. Compared without a floor, since `Group Start` itself - /// encodes differently across the two (see `group_start_is_absolute_on_lite06`). + /// encodes differently across the two. #[test] fn subscribe_drops_the_retired_ordered_byte_on_lite06() { - let mut msg = subscribe_sample(); - msg.start_group = None; - msg.start_frame = 0; - msg.end_frame = None; + let msg = Subscribe { + start: Start::LIVE, + end_frame: None, + ..subscribe_sample() + }; - let mut lite05 = Vec::new(); - msg.encode_msg(&mut Encoder::new(&mut lite05, Version::Lite05.into()), Version::Lite05) - .unwrap(); - let mut lite06 = Vec::new(); - msg.encode_msg(&mut Encoder::new(&mut lite06, Version::Lite06.into()), Version::Lite06) - .unwrap(); + let lite05 = encode(&msg, Version::Lite05).unwrap(); + let lite06 = encode(&msg, Version::Lite06).unwrap(); // The two layouts diverge in exactly one place: the retired byte lite-05 still // reserves. A deployed peer's field offsets depend on it being there and zero. @@ -768,103 +1003,34 @@ mod test { assert_eq!(&lite06[..spliced.len()], &spliced[..]); assert_eq!(&lite06[spliced.len()..], &[0, 0]); - let got = crate::coding::decode_buf(&mut lite05.as_slice(), Version::Lite05, Subscribe::decode_msg).unwrap(); - assert_eq!((got.start_frame, got.end_frame), (0, None)); - } - - /// Lite06 carries the raw floor; pre-06 wires encode the sequence + 1 with 0 meaning - /// the latest group. A vacuous floor folds to absent on the old wire, where an - /// explicit group 0 would mean "replay from the beginning" instead. - #[test] - fn group_start_is_absolute_on_lite06() { - let mut msg = subscribe_sample(); - msg.start_frame = 0; - msg.end_frame = None; - - let mut lite05 = Vec::new(); - msg.encode_msg(&mut Encoder::new(&mut lite05, Version::Lite05.into()), Version::Lite05) - .unwrap(); - let mut lite06 = Vec::new(); - msg.encode_msg(&mut Encoder::new(&mut lite06, Version::Lite06.into()), Version::Lite06) - .unwrap(); - - let on05 = crate::coding::decode_buf(&mut lite05.as_slice(), Version::Lite05, Subscribe::decode_msg).unwrap(); - let on06 = crate::coding::decode_buf(&mut lite06.as_slice(), Version::Lite06, Subscribe::decode_msg).unwrap(); - assert_eq!(on05.start_group, Some(7)); - assert_eq!(on06.start_group, Some(7)); - // The raw byte differs: 7 on the wire, not 7 + 1. - assert_ne!(lite05, lite06); - - // No floor and a floor of 0 are the same absence of a constraint on lite-06: - // byte-identical on the wire, and canonicalized to absent on decode. - msg.start_group = None; - let mut absent = Vec::new(); - msg.encode_msg(&mut Encoder::new(&mut absent, Version::Lite06.into()), Version::Lite06) - .unwrap(); - msg.start_group = Some(0); - let mut zero = Vec::new(); - msg.encode_msg(&mut Encoder::new(&mut zero, Version::Lite06.into()), Version::Lite06) - .unwrap(); - assert_eq!(absent, zero); - let got = crate::coding::decode_buf(&mut zero.as_slice(), Version::Lite06, Subscribe::decode_msg).unwrap(); - assert_eq!(got.start_group, None); - - // On the pre-06 wire the vacuous floor folds to absent (the latest group). - let mut folded = Vec::new(); - msg.encode_msg(&mut Encoder::new(&mut folded, Version::Lite05.into()), Version::Lite05) - .unwrap(); - let got = crate::coding::decode_buf(&mut folded.as_slice(), Version::Lite05, Subscribe::decode_msg).unwrap(); - assert_eq!(got.start_group, None); - } - - /// A subscription can resume partway through group 0 (a catalog never leaves it), so - /// a `Frame Start` qualifying the zero floor must survive the round trip. - #[test] - fn frame_start_may_qualify_group_zero_on_lite06() { - let mut msg = subscribe_sample(); - msg.start_group = Some(0); - msg.start_frame = 4; - - let mut buf = Vec::new(); - msg.encode_msg(&mut Encoder::new(&mut buf, Version::Lite06.into()), Version::Lite06) - .unwrap(); - let got = crate::coding::decode_buf(&mut buf.as_slice(), Version::Lite06, Subscribe::decode_msg).unwrap(); - assert_eq!((got.start_group, got.start_frame), (Some(0), 4)); + let got = decode(&lite05, Version::Lite05).unwrap(); + assert_eq!((got.start, got.end_frame), (Start::LIVE, None)); } /// Silently widening to the whole group would deliver frames the caller excluded. #[test] fn subscribe_frame_bounds_rejected_before_lite06() { - let mut buf = Vec::new(); - assert!( - subscribe_sample() - .encode_msg(&mut Encoder::new(&mut buf, Version::Lite05.into()), Version::Lite05) - .is_err() - ); + assert!(matches!( + encode(&subscribe_sample(), Version::Lite05), + Err(EncodeError::Version) + )); + let mid_group = Subscribe { + end_frame: None, + ..subscribe_sample() + }; + assert!(matches!(encode(&mid_group, Version::Lite05), Err(EncodeError::Version))); } /// Frames are numbered per group, so a frame bound without its group bound has /// nothing to count from. #[test] fn subscribe_frame_bound_without_group_bound_is_invalid() { - let mut msg = subscribe_sample(); - msg.start_group = None; - msg.start_frame = 4; - msg.end_group = None; - msg.end_frame = None; - - let mut buf = Vec::new(); - assert!(matches!( - msg.encode_msg(&mut Encoder::new(&mut buf, Version::Lite06.into()), Version::Lite06), - Err(EncodeError::InvalidState) - )); - - msg.start_frame = 0; - msg.end_frame = Some(7); - assert!(matches!( - msg.encode_msg(&mut Encoder::new(&mut buf, Version::Lite06.into()), Version::Lite06), - Err(EncodeError::InvalidState) - )); + let msg = Subscribe { + end_group: None, + end_frame: Some(7), + ..subscribe_sample() + }; + assert!(matches!(encode(&msg, Version::Lite06), Err(EncodeError::InvalidState))); } #[test] diff --git a/rs/moq-net/src/lite/subscriber.rs b/rs/moq-net/src/lite/subscriber.rs index f7c8ed0470d..c7a3b70b4fb 100644 --- a/rs/moq-net/src/lite/subscriber.rs +++ b/rs/moq-net/src/lite/subscriber.rs @@ -1538,7 +1538,7 @@ mod tests { stream, id: 0, max_delay: Duration::ZERO, - start: Some(Position::group(2)), + wire: lite::Start::floored(Position::group(2)), priority: 0, requested: Some(Position::group(2)), tail: Default::default(), @@ -1557,15 +1557,15 @@ mod tests { Some(2..10), "the groups below START are accounted for on arrival" ); - sub.start = Some(Position::group(6)); + sub.wire = lite::Start::floored(Position::group(6)); assert_eq!(sub.owed(None), Some(6..10), "a raised floor owes nothing below it"); - sub.start = Some(Position::group(1)); + sub.wire = lite::Start::floored(Position::group(1)); assert_eq!( sub.owed(None), Some(1..10), "a lowered floor owes what it newly asked for" ); - sub.start = None; + sub.wire = lite::Start::LIVE; assert_eq!( sub.owed(Some(8)), Some(4..8), @@ -2172,7 +2172,8 @@ mod tests { /// bound in the same group. fn mid_group_demand() -> Subscription { Subscription::default() - .with_start(Position { group: 5, frame: 3 }) + .with_live(false) + .with_floor(Position { group: 5, frame: 3 }) .with_end(Position::after(5, 7)) } @@ -2207,8 +2208,8 @@ mod tests { ); let msg = crate::coding::decode_buf(&mut wire, Version::Lite05, lite::Subscribe::decode).unwrap(); // The group bounds survive; only the frame offsets are widened away. - assert_eq!((msg.start_group, msg.end_group), (Some(5), Some(5))); - assert_eq!((msg.start_frame, msg.end_frame), (0, None)); + assert_eq!(msg.start, lite::Start::floored(Position::group(5))); + assert_eq!((msg.end_group, msg.end_frame), (Some(5), None)); } /// A lite-07 SUBSCRIBE keeps a mid-group start: its answer carries the largest position, @@ -2239,7 +2240,11 @@ mod tests { lite::ControlType::Subscribe ); let msg = crate::coding::decode_buf(&mut wire, version, lite::Subscribe::decode).unwrap(); - assert_eq!((msg.start_frame, msg.end_frame), (start_frame, Some(7)), "{version:?}"); + assert_eq!( + (msg.start.floor.map(|floor| floor.frame), msg.end_frame), + (Some(start_frame), Some(7)), + "{version:?}" + ); } } @@ -2252,20 +2257,20 @@ mod tests { #[test] fn wire_bounds_convert_the_exclusive_end() { // The whole of group 5 is the head of group 6. - let bounds = WireBounds::new(None, Some(Position::group(6))); + let bounds = WireBounds::new(lite::Start::LIVE, Some(Position::group(6))); assert_eq!((bounds.end_group, bounds.end_frame), (Some(5), None)); // Group 5 through frame 2 is the head of frame 3. - let bounds = WireBounds::new(None, Some(Position { group: 5, frame: 3 })); + let bounds = WireBounds::new(lite::Start::LIVE, Some(Position { group: 5, frame: 3 })); assert_eq!((bounds.end_group, bounds.end_frame), (Some(5), Some(2))); // Unbounded stays unbounded. - let bounds = WireBounds::new(None, None); + let bounds = WireBounds::new(lite::Start::LIVE, None); assert_eq!((bounds.end_group, bounds.end_frame), (None, None)); // Starts are inclusive on both sides, so they pass straight through. - let bounds = WireBounds::new(Some(Position { group: 5, frame: 3 }), None); - assert_eq!((bounds.start_group, bounds.start_frame), (Some(5), 3)); + let start = lite::Start::floored(Position { group: 5, frame: 3 }); + assert_eq!(WireBounds::new(start, None).start, start); } /// The builders produce exactly what the wire conversion expects, so an inclusive @@ -2273,16 +2278,18 @@ mod tests { #[test] fn wire_bounds_match_the_builders() { let whole = Subscription::default().with_end(Position::after_group(5)); - let bounds = WireBounds::new(whole.start, whole.end); + let bounds = WireBounds::new(lite::Start::of(&whole), whole.end); assert_eq!((bounds.end_group, bounds.end_frame), (Some(5), None)); let capped = Subscription::default().with_end(Position::after(5, 2)); - let bounds = WireBounds::new(capped.start, capped.end); + let bounds = WireBounds::new(lite::Start::of(&capped), capped.end); assert_eq!((bounds.end_group, bounds.end_frame), (Some(5), Some(2))); - let started = Subscription::default().with_start(Position { group: 5, frame: 3 }); - let bounds = WireBounds::new(started.start, started.end); - assert_eq!((bounds.start_group, bounds.start_frame), (Some(5), 3)); + let started = Subscription::default() + .with_live(false) + .with_floor(Position { group: 5, frame: 3 }); + let bounds = WireBounds::new(lite::Start::of(&started), started.end); + assert_eq!(bounds.start, lite::Start::floored(Position { group: 5, frame: 3 })); } /// A subscription that asks for nothing opens nothing. @@ -2347,7 +2354,8 @@ mod tests { let mut sub = Sub::None; let empty = Subscription::default() - .with_start(Position::group(5)) + .with_live(false) + .with_floor(Position::group(5)) .with_end(Position::group(5)); h.serve .handle_subscription(&mut h.producer, &mut sub, Some(empty), true, Some(Timescale::default())) @@ -2379,7 +2387,8 @@ mod tests { let established = h.wire().len(); let empty = Subscription::default() - .with_start(Position::group(5)) + .with_live(false) + .with_floor(Position::group(5)) .with_end(Position::group(5)); h.serve .handle_subscription(&mut h.producer, &mut sub, Some(empty), true, Some(Timescale::default())) @@ -2401,14 +2410,15 @@ mod tests { let h = Harness::new(Version::Lite05); let mut subscription = Subscription::default() - .with_start(Position { + .with_live(false) + .with_floor(Position { group: u64::MAX, frame: 1, }) .with_end(Position::after(u64::MAX, 5)); h.serve.widen_frame_bounds(&mut subscription); - assert_eq!(subscription.start, Some(Position::group(u64::MAX))); + assert_eq!(subscription.floor, Some(Position::group(u64::MAX))); // Past the last group there is no position to round up to, and unbounded is the // wider request. assert_eq!(subscription.end, None); @@ -2449,7 +2459,7 @@ mod tests { let wire = h.wire(); let mut wire = &wire[established..]; let msg = crate::coding::decode_buf(&mut wire, Version::Lite05, lite::SubscribeUpdate::decode).unwrap(); - assert_eq!((msg.start_group, msg.start_frame), (Some(5), 0)); + assert_eq!(msg.start, lite::Start::floored(Position::group(5))); assert_eq!((msg.end_group, msg.end_frame), (Some(5), None)); } @@ -2464,8 +2474,14 @@ mod tests { let mut h = Harness::new(Version::Lite05); let mut sub = Sub::None; - let demand = |group: u64| Some(Subscription::default().with_start(Position::group(group))); - let applies = |sub: &Sub| matches!(sub, Sub::Active(active) if active.start == active.requested); + let demand = |group: u64| { + Some( + Subscription::default() + .with_live(false) + .with_floor(Position::group(group)), + ) + }; + let applies = |sub: &Sub| matches!(sub, Sub::Active(active) if active.start() == active.requested); // Establish from group 3; the peer's START is considered in flight. h.serve @@ -2481,7 +2497,8 @@ mod tests { &mut sub, Some( Subscription::default() - .with_start(Position::group(3)) + .with_live(false) + .with_floor(Position::group(3)) .with_end(Position::group(9)), ), true, @@ -2516,7 +2533,13 @@ mod tests { let mut h = Harness::new(Version::Lite05); let mut sub = Sub::None; - let demand = |group: u64| Some(Subscription::default().with_start(Position::group(group))); + let demand = |group: u64| { + Some( + Subscription::default() + .with_live(false) + .with_floor(Position::group(group)), + ) + }; // Establish from group 5: the floor tracks the request until START lands. h.serve @@ -3379,27 +3402,29 @@ mod tests { } } -/// The four wire fields a subscription's half-open range encodes to. +/// The wire fields a subscription's start and half-open end encode to. /// -/// The inverse of the publisher's `Bounds::positions`: the model carries whole -/// positions with an exclusive end, while the wire splits each bound into a group and a -/// frame and states both ends inclusive. The range must be non-empty, since an inclusive -/// end has nothing to say below the first position it excludes. +/// The inverse of the publisher's `Bounds::positions`: the model carries a whole end +/// position, exclusive, while the wire splits it into a group and a frame and states it +/// inclusive. The range must be non-empty, since an inclusive end has nothing to say below +/// the first position it excludes. struct WireBounds { - start_group: Option, - start_frame: u64, + start: lite::Start, end_group: Option, end_frame: Option, } impl WireBounds { - fn new(start: Option, end: Option) -> Self { + fn new(start: lite::Start, end: Option) -> Self { // An empty range has no wire encoding: flooring its end below asks for the - // position it excludes, or inverts the range. `handle_subscription` drops such a - // subscription instead. An absent start is the live edge, so it only makes the - // range empty when the end sits at the very first position. + // position it excludes, or inverts the range. `begin_subscription` drops such a + // subscription instead. debug_assert!( - end.is_none_or(|end| end > start.unwrap_or_default()), + !Subscription::default() + .with_live(start.live) + .with_floor(start.floor) + .with_end(end) + .is_empty(), "an empty range cannot be encoded; it should have been dropped as no demand" ); @@ -3412,14 +3437,19 @@ impl WireBounds { }; Self { - start_group: start.map(|start| start.group), - start_frame: start.map_or(0, |start| start.frame), + start, end_group, end_frame, } } } +/// The floor below which the peer serves nothing for `subscription`, or `None` for the +/// live edge, which its SUBSCRIBE_START resolves. `live` reaches below any floor. +fn served_floor(subscription: &Subscription) -> Option { + subscription.floor.filter(|_| !subscription.live) +} + /// The at-most-one live upstream subscription: its control stream plus the params /// echoed in every SUBSCRIBE_UPDATE. struct SubStream { @@ -3428,7 +3458,7 @@ struct SubStream { /// Original SUBSCRIBE params, echoed in every SUBSCRIBE_UPDATE; refreshed as the /// downstream aggregate changes. max_delay: Duration, - start: Option, + wire: lite::Start, priority: u8, /// The start the SUBSCRIBE itself carried, fixed for the stream's life. A /// SUBSCRIBE_START describes this demand and no fresh one follows an update, @@ -3445,6 +3475,11 @@ struct SubStream { } impl SubStream { + /// The floor the peer serves from, following `wire`; see [`served_floor`]. + fn start(&self) -> Option { + self.wire.floor.filter(|_| !self.wire.live) + } + /// The groups the publisher still owes once it has ended the subscription. /// /// `None` when nothing says which: drafts before SUBSCRIBE_END only have the FIN. @@ -3456,7 +3491,7 @@ impl SubStream { let end = self.end.as_ref()?.group; let end = requested_end.map_or(end, |requested| requested.min(end)); let start = match self.served { - Some(served) => self.start.map_or(served, |start| start.group), + Some(served) => self.start().map_or(served, |start| start.group), None => end, }; Some(start..end) @@ -3465,7 +3500,7 @@ impl SubStream { enum Sub { None, - Active(SubStream), + Active(Box>), } /// Every advertisement the peer currently has live on one announce stream. @@ -3722,7 +3757,7 @@ impl TrackServe { // subscriber holds, so ask from the head of its group instead: the first frame then // arrives at once and says where the live feed is. if !self.subscriber.version.has_largest() { - subscription.start = subscription.start.map(|start| Position::group(start.group)); + subscription.floor = subscription.floor.map(|floor| Position::group(floor.group)); } if self.subscriber.version.has_frame_bounds() { return; @@ -3731,7 +3766,7 @@ impl TrackServe { // Round both bounds outward to the enclosing group, so the peer sends at least // what was asked for and never less. Rounding the end down instead would be able // to empty a non-empty range, which has no wire encoding at all. - let start = subscription.start.map(|start| Position::group(start.group)); + let floor = subscription.floor.map(|floor| Position::group(floor.group)); let end = subscription.end.and_then(|end| match end.frame { 0 => Some(end), // Past the last group there is no position to round up to, and unbounded is @@ -3739,17 +3774,28 @@ impl TrackServe { _ => Position::after_group(end.group), }); - if (start, end) != (subscription.start, subscription.end) { + if (floor, end) != (subscription.floor, subscription.end) { tracing::debug!( track = %self.name, version = ?self.subscriber.version, "widening frame bounds to whole groups for an older peer" ); } - subscription.start = start; + subscription.floor = floor; subscription.end = end; } + /// Fold `live` into the floor for a peer whose SUBSCRIBE has no `Live` field: `live` + /// with a floor asks from the very first position, and the copy's readers filter + /// locally by age and floor. + fn fold_live(&self, subscription: &mut Subscription) { + if self.subscriber.version.has_live() { + return; + } + subscription.floor = subscription.folded_floor(); + subscription.live = subscription.floor.is_none(); + } + /// Apply a subscription-demand change: hand back an [`Establish`] to open the /// upstream SUBSCRIBE on the first subscriber, buffer a SUBSCRIBE_UPDATE while /// live (the caller flushes), or cancel outright when the last one leaves. @@ -3761,24 +3807,27 @@ impl TrackServe { supports_update: bool, timescale: Option, ) -> Result, Error> { - // An empty half-open range asks for nothing, and the wire cannot say that: its - // bounds are inclusive, so the nearest encoding either hands back the position - // the caller excluded or inverts the range outright once the two bounds meet. No - // demand at all is the faithful translation. An absent start is the live edge, wherever that lands, so the only end that is - // certainly empty is the very first position, which is what `Position::default()` - // stands in for. - let pref = pref.filter(|sub| sub.end.is_none_or(|end| end > sub.start.unwrap_or_default())); + // A subscription that asks for nothing (neither `live` nor a floor, or an empty + // half-open range) has no wire encoding: an empty range's bounds are inclusive, so + // the nearest encoding either hands back the position the caller excluded or + // inverts the range outright once the two bounds meet. No demand at all is the + // faithful translation. + let pref = pref.filter(|sub| !sub.is_empty()); match pref { Some(mut subscription) => { + self.fold_live(&mut subscription); self.widen_frame_bounds(&mut subscription); match sub { Sub::None => { // An idle copy is asked from the head of the newest group it cached when - // the subscriber has no start of its own: a quiet track (a catalog) sends - // that group again, which says the cache is current. - if subscription.start.is_none() { - subscription.start = producer.idle_newest().map(Position::group); + // the subscriber wants only the live edge: a quiet track (a catalog) + // sends that group again, which says the cache is current. + if subscription.live + && subscription.floor.is_none() + && let Some(newest) = producer.idle_newest() + { + subscription = subscription.with_live(false).with_floor(Position::group(newest)); } // Open an upstream SUBSCRIBE for the first subscriber. Ok(Begin::Establish(self.prepare_establish( @@ -3790,18 +3839,19 @@ impl TrackServe { Sub::Active(active) => { // Downstream preferences changed: forward them upstream as a // SUBSCRIBE_UPDATE (Lite03+ only; older peers can't carry one). - let start_moved = active.start != subscription.start; + let start = served_floor(&subscription); + let start_moved = active.start() != start; active.priority = subscription.priority; active.max_delay = subscription.max_delay; if let Ok(mut tail) = active.tail.write() { tail.set_grace(tail::grace(subscription.max_delay)); // A lowered floor owes groups nobody asked for until now. - if let Some(start) = subscription.start { - let floor = active.start.map(|start| start.group).or(active.served); + if let Some(start) = start { + let floor = active.start().map(|start| start.group).or(active.served); tail.demand(start.group..floor.unwrap_or(u64::MAX), self.subscriber.runtime.now()); } } - active.start = subscription.start; + active.wire = lite::Start::of(&subscription); if supports_update { // The floor follows the requested start, in both directions: // moving below a declared SUBSCRIBE_START reopens those groups @@ -3812,7 +3862,7 @@ impl TrackServe { // start returns to the demand that produced it (see // `SubStream::requested`). if start_moved { - let _ = producer.start_at(active.start.map(|start| start.group)); + let _ = producer.start_at(active.start().map(|start| start.group)); } buffer_update(active, subscription.end)?; } @@ -3863,7 +3913,7 @@ impl TrackServe { // previous subscription's declaration must not outlive its demand, and // live-edge demand (None) starts with no floor at all. Lite-06+ only resolves // the start with its SUBSCRIBE_START, so until then the floor is just a request. - let floor = subscription.start.map(|start| start.group); + let floor = served_floor(&subscription).map(|start| start.group); let _ = match self.subscriber.version.resolves_start() { true => producer.request_start(floor), false => producer.start_at(floor), @@ -3906,7 +3956,7 @@ impl TrackServe { let id = est.id; match kio::wait(move |waiter| est.poll(waiter)).await { Ok(active) => { - *sub = Sub::Active(active); + *sub = Sub::Active(Box::new(active)); Ok(()) } Err(err) => { @@ -3932,7 +3982,7 @@ impl TrackServe { let mut est = Box::new(est); let id = est.id; match kio::wait(move |waiter| est.poll(waiter)).await { - Ok(active) => *sub = Sub::Active(active), + Ok(active) => *sub = Sub::Active(Box::new(active)), Err(err) => { self.subscriber.remove_subscribe(id); return Err(err); @@ -3967,13 +4017,12 @@ fn buffer_update( active: &mut SubStream, end: Option, ) -> Result<(), Error> { - let bounds = WireBounds::new(active.start, end); + let bounds = WireBounds::new(active.wire, end); active.stream.writer.buffer(&lite::SubscribeUpdate { priority: active.priority, max_delay: active.max_delay, - start_group: bounds.start_group, + start: bounds.start, end_group: bounds.end_group, - start_frame: bounds.start_frame, end_frame: bounds.end_frame, }) } @@ -4017,7 +4066,7 @@ impl Establish { &mut cx ))?; - let bounds = WireBounds::new(self.subscription.start, self.subscription.end); + let bounds = WireBounds::new(lite::Start::of(&self.subscription), self.subscription.end); let msg = lite::Subscribe { epoch: self.serve.epoch.clone(), id: self.id, @@ -4025,9 +4074,8 @@ impl Establish { track: self.serve.name.as_str().into(), priority: self.subscription.priority, max_delay: self.subscription.max_delay, - start_group: bounds.start_group, + start: bounds.start, end_group: bounds.end_group, - start_frame: bounds.start_frame, end_frame: bounds.end_frame, }; stream.writer.buffer(&lite::ControlType::Subscribe)?; @@ -4069,9 +4117,9 @@ impl Establish { stream, id: self.id, max_delay: self.subscription.max_delay, - start: self.subscription.start, + wire: lite::Start::of(&self.subscription), priority: self.subscription.priority, - requested: self.subscription.start, + requested: served_floor(&self.subscription), tail: self.tail.clone(), served: None, end: None, @@ -4405,7 +4453,7 @@ impl ServeLoop { let id = est.id; self.mode = ServeMode::Select; match res { - Ok(active) => self.sub = Sub::Active(active), + Ok(active) => self.sub = Sub::Active(Box::new(active)), Err(err) => { // Opening the upstream failed (usually the session dying): hand // the track back for another route to resume. @@ -4573,7 +4621,7 @@ impl ServeLoop { // that moved the start makes it stale, and one that // moved back restores it); elsewhere the // request-tracked floor stands rather than a guess. - if active.start == active.requested { + if active.start() == active.requested { let _ = self.serving.start_at(start.group); } active.served = Some(start.group); diff --git a/rs/moq-net/src/lite/version.rs b/rs/moq-net/src/lite/version.rs index fe1c15b04ac..05cbbff905e 100644 --- a/rs/moq-net/src/lite/version.rs +++ b/rs/moq-net/src/lite/version.rs @@ -216,6 +216,17 @@ impl Version { } } + /// Whether SUBSCRIBE and SUBSCRIBE_UPDATE carry a `Live` flag beside an optional + /// floor. Added in lite-07; older versions fold `live` into `Group Start`. + #[allow(clippy::match_like_matches_macro)] + pub(crate) fn has_live(self) -> bool { + // Match form so future versions default forward (AGENTS.md convention). + match self { + Self::Lite01 | Self::Lite02 | Self::Lite03 | Self::Lite04 | Self::Lite05 | Self::Lite06 => false, + _ => true, + } + } + /// Whether ANNOUNCE_START and ANNOUNCE_UPDATE may copy a path head or hop-chain tail /// from a live announcement on the same stream. Added in lite-07. #[allow(clippy::match_like_matches_macro)] diff --git a/rs/moq-net/src/model/origin.rs b/rs/moq-net/src/model/origin.rs index be84d9ad28e..dcada369a72 100644 --- a/rs/moq-net/src/model/origin.rs +++ b/rs/moq-net/src/model/origin.rs @@ -2736,14 +2736,33 @@ async fn serve_front(task: FrontTask) -> Vec { { return Poll::Ready(Step::Info(name.clone(), asked.source, result)); } - // The query's subscription goes with the last reader. - if io.held.is_some() + // The query's subscription goes with the last reader. While readers remain + // it tracks their aggregate: the query opened it with the demand at that + // moment, and a later narrowing has to reach the source. A stale `live` + // there keeps the aggregate asking for the retained window. + let held_demand = if io.held.is_some() && let Some(accepted) = &mut io.accepted { - while let Poll::Ready(Ok(_)) = accepted.poll_subscription_changed(waiter) {} - if accepted.subscription().is_none() { - io.held = None; + let mut changed = false; + while let Poll::Ready(Ok(_)) = accepted.poll_subscription_changed(waiter) { + changed = true; + } + match accepted.subscription() { + None => Some(None), + Some(demand) if changed => Some(Some(demand)), + Some(_) => None, + } + } else { + None + }; + match held_demand { + Some(None) => io.held = None, + Some(Some(demand)) => { + if let Some(held) = &mut io.held { + let _ = held.update(demand); + } } + None => {} } // Settled once the copy closes: a group still open below a declared end is // owed until then, and a session closing first means it never came. diff --git a/rs/moq-net/src/model/resume.rs b/rs/moq-net/src/model/resume.rs index 32fd3cffc85..5a1b424da40 100644 --- a/rs/moq-net/src/model/resume.rs +++ b/rs/moq-net/src/model/resume.rs @@ -255,14 +255,29 @@ enum Sub { Ready(track::Subscriber), } +/// `reader`'s preferences for a copy resuming past `floor`, the newest group already +/// handed out, or `None` for a first copy. +/// +/// A resume asks only for what follows: the reader already has the live edge it wanted, +/// up to `floor`, so `live` would only ask the new route for groups below it again. A +/// reader whose own floor sits above `floor` keeps `live`, or nothing would ask for the +/// groups in between. +fn resumed(reader: &Subscription, floor: Option) -> Subscription { + let Some(floor) = floor else { + return reader.clone(); + }; + Subscription { + live: reader.live && reader.floor.is_some_and(|own| own > floor), + floor: max_some(reader.floor, Some(floor)), + ..reader.clone() + } +} + impl Copy { /// What this copy is asked for: the reader's own preferences, from no earlier than /// where it was asked to start, and through no later than where it was replaced. fn wire(&self, reader: &Subscription) -> Subscription { - let mut wire = Subscription { - start: max_some(reader.start, self.floor), - ..reader.clone() - }; + let mut wire = resumed(reader, self.floor); if let Some(until) = self.until { let end = until.and_then(Position::after_group).unwrap_or_default(); wire.end = Some(wire.end.map_or(end, |requested| requested.min(end))); @@ -458,10 +473,7 @@ impl Reader { let floor = self.newest.map(Position::group); // The wire request carries the floor from the first message: some sessions read // the demand only once. - let wire = Subscription { - start: max_some(self.mirrored.start, floor), - ..self.mirrored.clone() - }; + let wire = resumed(&self.mirrored, floor); self.copies.push(Copy { generation, sub: Sub::Pending(track.subscribe(wire)), diff --git a/rs/moq-net/src/model/subscription.rs b/rs/moq-net/src/model/subscription.rs index c3ed10301b8..99e50ad7143 100644 --- a/rs/moq-net/src/model/subscription.rs +++ b/rs/moq-net/src/model/subscription.rs @@ -51,25 +51,36 @@ pub struct Subscription { /// reclamation of idle content is the cache's own policy, not this budget's. /// /// An untimed track (pre-Lite05 moq-lite, or moq-transport without TIMESCALE) has no - /// media time, so none of its groups is ever stale, and a new subscriber with no - /// [`Self::start`] begins at its latest group instead of replaying the cache. + /// media time, so none of its groups is ever stale, and [`Self::live`] begins at its + /// latest group instead of replaying the cache. pub max_delay: Duration, + /// Deliver from the live edge: the first frame of the oldest group [`Self::max_delay`] + /// has not expired, or of the latest group on an untimed track. + /// + /// Lowers [`Self::floor`] to that position whenever it sorts below the floor, so a + /// floor above the live edge (a resumed subscription naming where it left off) never + /// hides the latest group from a subscriber that also wants it. On a timed track a + /// floor adds nothing to `live`: anything below the live edge is expired anyway. + /// + /// A subscription with neither `live` nor a floor asks for nothing, like an empty + /// range: it holds no demand, and no wire carries it. Defaults to `true`. + /// + /// Aggregated across every live subscriber as an OR. + pub live: bool, /// The lowest [`Position`] the publisher may deliver, or `None` for no floor. /// /// A floor, not a request: only [`Self::max_delay`] asks for data, and the floor bounds - /// how far back it may reach. `None` and a floor of group 0 mean the same thing on a - /// timed track, since nothing sits below group 0. On an untimed track, which no budget - /// can measure, `None` starts at the latest group and a floor is honored as given. - /// Delivery starts at the oldest group at or above the floor that the budget still - /// considers fresh, so a floor above the live edge simply waits there (a resumed - /// subscription naming where it left off). + /// how far back it may reach. Delivery starts at the oldest group at or above the floor + /// that the budget still considers fresh, so a floor above the live edge simply waits + /// there (a resumed subscription naming where it left off). [`Self::live`] lowers it. /// - /// Aggregated across every live subscriber (the loosest floor wins, and any subscriber - /// without one clears it), so it says what the publisher sends, not what any one - /// subscriber sees. [`crate::track::Subscriber::set_groups`] is the local read cursor; - /// setting one does not imply the other. See [Local cursor vs wire + /// Aggregated across every live subscriber as the minimum, where `None` is neutral, so + /// it says what the publisher sends, not what any one subscriber sees. Each + /// subscriber's own cursor still filters what it reads. + /// [`crate::track::Subscriber::set_groups`] is the local read cursor; setting one does + /// not imply the other. See [Local cursor vs wire /// preference](crate::track::Subscriber#local-cursor-vs-wire-preference). - pub start: Option, + pub floor: Option, /// First [`Position`] the publisher should *not* deliver, or `None` for no end. /// /// Exclusive, like the end of a [`std::ops::Range`], which is what lets one field @@ -93,7 +104,8 @@ impl Default for Subscription { Self { priority: 0, max_delay: Duration::ZERO, - start: None, + live: true, + floor: None, end: None, } } @@ -113,13 +125,21 @@ impl Subscription { self } - /// Floor delivery at `start`, or leave it unfloored when `None`. Returns `self` for + /// Deliver from the live edge or not, returning `self` for chaining. See [`Self::live`]. + pub fn with_live(mut self, live: bool) -> Self { + self.live = live; + self + } + + /// Floor delivery at `floor`, or leave it unfloored when `None`. Returns `self` for /// chaining. /// /// A floor bounds how far back [`Self::max_delay`] may reach; it does not request data - /// on its own. [`Position::group`] is the whole-group form. - pub fn with_start(mut self, start: impl Into>) -> Self { - self.start = start.into(); + /// on its own. [`Position::group`] is the whole-group form. This leaves + /// [`Self::live`] alone, so a resume that wants only what follows its floor also + /// clears it. + pub fn with_floor(mut self, floor: impl Into>) -> Self { + self.floor = floor.into(); self } @@ -134,20 +154,45 @@ impl Subscription { self } - /// Request the whole groups in `groups`, replacing both [`Self::start`] and - /// [`Self::end`]. Returns `self` for chaining. + /// Request the whole groups in `groups`, replacing [`Self::live`], [`Self::floor`], + /// and [`Self::end`]. Returns `self` for chaining. /// /// Any range of group sequences works: `2..=5`, `2..6`, `..6`, `2..`, or `..` to - /// clear both bounds. An inclusive end past the last group is unbounded, as + /// clear both bounds. A bounded start is a floor alone, and an unbounded one is + /// [`Self::live`]. An inclusive end past the last group is unbounded, as /// [`Position::after_group`] spells it. pub fn with_groups(mut self, groups: impl RangeBounds) -> Self { let unbounded_start = matches!(groups.start_bound(), Bound::Unbounded); let (start, end) = sequence_bounds(groups); - self.start = (!unbounded_start).then(|| Position::group(start)); + self.live = unbounded_start; + self.floor = (!unbounded_start).then(|| Position::group(start)); self.end = end.map(Position::group); self } + /// The floor a wire without a `live` field carries for this subscription: `None` for + /// `live` alone, and the very first position for `live` with a floor, which the + /// receiver filters locally by age and floor. + /// + /// A subscription that asks for nothing ([`Self::is_empty`]) has no such floor; + /// callers drop it first. + pub(crate) fn folded_floor(&self) -> Option { + match self.live { + true => self.floor.map(|_| Position::default()), + false => self.floor, + } + } + + /// Whether this subscription asks for nothing: neither [`Self::live`] nor a floor, or + /// an end at or below the lowest position it could start from. + pub(crate) fn is_empty(&self) -> bool { + if !self.live && self.floor.is_none() { + return true; + } + self.end + .is_some_and(|end| end <= self.folded_floor().unwrap_or_default()) + } + // Fold this subscription into the running aggregate: Ready with the merged // result when it demands more than `combined`, Pending when it's a subset // (so callers can skip a redundant broadcast of the same aggregate). @@ -160,10 +205,13 @@ impl Subscription { priority: self.priority.max(combined.priority), // Sequence-first prioritization is enabled only when every subscriber wants it. max_delay: self.max_delay.max(combined.max_delay), + // Anyone wanting the live edge gets it, without lowering anyone's floor. + live: self.live || combined.live, // Bounds fold as whole positions. Two subscribers starting in the same group // are separated only by their frame, so folding group and frame independently - // would invent a bound neither asked for. - start: min_floored(self.start, combined.start), + // would invent a bound neither asked for. A live-only subscriber has no floor, + // the neutral value, so it never clears another's. + floor: min_some(self.floor, combined.floor), end: max_unbounded(self.end, combined.end), }; @@ -316,13 +364,12 @@ pub(super) fn max_some(a: Option, b: Option) -> Option { } } -/// The lower of two optional floors, `None` absorbing (no floor). The mirror of -/// [`max_unbounded`]: both bounds only ever *restrict*, so a subscriber without one keeps -/// the aggregate unrestricted. -pub(super) fn min_floored(a: Option, b: Option) -> Option { +/// The lower of two optional bounds, `None` neutral. +pub(super) fn min_some(a: Option, b: Option) -> Option { match (a, b) { (Some(a), Some(b)) => Some(a.min(b)), - (None, _) | (_, None) => None, + (Some(a), None) | (None, Some(a)) => Some(a), + (None, None) => None, } } @@ -348,24 +395,29 @@ mod tests { combined } + /// A floor alone, without the live edge. + fn floored(floor: Position) -> Subscription { + Subscription::default().with_live(false).with_floor(floor) + } + /// The exclusive representation runs out at both extremes, and `Option` says so /// rather than saturating into a bound that contradicts the request. /// A group range spells both positions at once, in whichever form the caller has. #[test] fn group_ranges_build_whole_group_positions() { let sub = Subscription::default().with_groups(2..=5); - assert_eq!(sub.start, Some(Position::group(2))); + assert_eq!((sub.live, sub.floor), (false, Some(Position::group(2)))); assert_eq!(sub.end, Some(Position::group(6))); let sub = Subscription::default().with_groups(2..6); assert_eq!(sub.end, Some(Position::group(6))); let sub = Subscription::default().with_groups(..6); - assert_eq!(sub.start, None); + assert_eq!((sub.live, sub.floor), (true, None)); assert_eq!(sub.end, Some(Position::group(6))); let sub = Subscription::default().with_groups(2..); - assert_eq!(sub.start, Some(Position::group(2))); + assert_eq!(sub.floor, Some(Position::group(2))); assert_eq!(sub.end, None); // Through the last group is unbounded, as `after_group` spells it. @@ -373,7 +425,7 @@ mod tests { assert_eq!(sub.end, None); let sub = Subscription::default().with_groups(2..=5).with_groups(..); - assert_eq!((sub.start, sub.end), (None, None)); + assert_eq!((sub.live, sub.floor, sub.end), (true, None, None)); } #[test] @@ -407,18 +459,52 @@ mod tests { } #[test] - fn combined_group_start_keeps_the_loosest_floor() { + fn combined_floor_keeps_the_loosest_floor() { // A floor only restricts, so the lowest one wins across floored subscribers. - let catchup = Subscription::default().with_start(Position::group(10)); - let older_catchup = Subscription::default().with_start(Position::group(5)); + let catchup = floored(Position::group(10)); + let older_catchup = floored(Position::group(5)); let combined = combine(&[catchup.clone(), older_catchup]).unwrap(); - assert_eq!(combined.start, Some(Position::group(5))); + assert_eq!((combined.live, combined.floor), (false, Some(Position::group(5)))); + + // A live-only subscriber has no floor, the neutral value, so it keeps the others' + // floor and adds the live edge rather than clearing it. Clearing it starved the + // floored subscriber's resume; keeping only the floor starved the live one. + let live = Subscription::default(); + let combined = combine(&[catchup.clone(), live.clone()]).unwrap(); + assert_eq!((combined.live, combined.floor), (true, Some(Position::group(10)))); + let combined = combine(&[live, catchup]).unwrap(); + assert_eq!((combined.live, combined.floor), (true, Some(Position::group(10)))); + } + + /// A wire without a `live` field folds it into the floor: `live` alone is no floor, + /// and `live` with a floor reaches back to the very first position, which the receiver + /// filters locally. + #[test] + fn older_wires_fold_live_into_the_floor() { + let live = Subscription::default(); + assert_eq!(live.folded_floor(), None); + assert_eq!(floored(Position::group(4)).folded_floor(), Some(Position::group(4))); + assert_eq!( + live.clone().with_floor(Position { group: 3, frame: 5 }).folded_floor(), + Some(Position::default()) + ); + } - // A subscriber with no floor at all clears the aggregate: its budget may reach - // below any floor the others set. - let unfloored = Subscription::default(); - let combined = combine(&[catchup, unfloored]).unwrap(); - assert_eq!(combined.start, None); + /// Neither `live` nor a floor asks for nothing, like an end at the first position. + #[test] + fn a_subscription_without_live_or_a_floor_is_empty() { + assert!(!Subscription::default().is_empty()); + assert!(Subscription::default().with_live(false).is_empty()); + assert!(!floored(Position::group(0)).is_empty()); + assert!(Subscription::default().with_end(Position::group(0)).is_empty()); + assert!(floored(Position::group(5)).with_end(Position::group(5)).is_empty()); + // The live edge may sit below a floor, so only the first position empties it. + assert!( + !Subscription::default() + .with_floor(Position::group(5)) + .with_end(Position::group(5)) + .is_empty() + ); } #[test] @@ -433,19 +519,19 @@ mod tests { } #[test] - fn combined_start_folds_the_whole_position() { - let early_frame = Subscription::default().with_start(Position { group: 5, frame: 2 }); - let late_frame = Subscription::default().with_start(Position { group: 5, frame: 9 }); + fn combined_floor_folds_the_whole_position() { + let early_frame = floored(Position { group: 5, frame: 2 }); + let late_frame = floored(Position { group: 5, frame: 9 }); // Same group: the earlier frame wins. let combined = combine(&[late_frame.clone(), early_frame.clone()]).unwrap(); - assert_eq!(combined.start, Some(Position { group: 5, frame: 2 })); + assert_eq!(combined.floor, Some(Position { group: 5, frame: 2 })); // An earlier group wins outright, carrying its own frame rather than the // smallest frame across the two. - let earlier_group = Subscription::default().with_start(Position { group: 4, frame: 7 }); + let earlier_group = floored(Position { group: 4, frame: 7 }); let combined = combine(&[early_frame, earlier_group]).unwrap(); - assert_eq!(combined.start, Some(Position { group: 4, frame: 7 })); + assert_eq!(combined.floor, Some(Position { group: 4, frame: 7 })); } #[test] diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index c6c0716d643..6ea3fe6e995 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -704,7 +704,7 @@ impl TrackState { }) } - /// Where a new reader with no explicit start begins on an untimed track: its newest + /// Where [`Subscription::live`] begins on an untimed track: its newest /// servable group below the exclusive `cap`. /// /// A drift budget resolves a timed track's start, but nothing is ever stale on an @@ -2057,11 +2057,12 @@ impl Producer { /// The info is fixed at creation, so there's nothing to wait for (no /// SUBSCRIBE_OK round trip). Pass `None` for [`Subscription::default`]. /// - /// The read cursor starts at the group the subscription named (its floor), or 0. - /// [`Subscription::max_delay`] is what asks for data: delivery skips everything above - /// the floor that the budget convicts, so the default budget of zero delivers only - /// the latest group and a larger one reaches back over what it can still use. An - /// untimed track has nothing to convict, so it starts at its latest group. + /// The read cursor starts at the group the subscription named (its floor), lowered to + /// the live edge by [`Subscription::live`]. [`Subscription::max_delay`] is what asks + /// for data: delivery skips everything above the start that the budget convicts, so + /// the default budget of zero delivers only the latest group and a larger one reaches + /// back over what it can still use. An untimed track has nothing to convict, so `live` + /// starts at its latest group. pub fn subscribe(&self, subscription: impl Into>) -> Subscriber { let preferences = subscription.into().unwrap_or_default(); @@ -2462,16 +2463,28 @@ fn snapshot_subscription(subs: &kio::Shared, bound: Option u64 { - subscription.start.map(|start| start.group).unwrap_or(0) +/// +/// On a timed track `live` therefore starts at 0, leaving the budget to trim. Nothing on an +/// untimed track is ever stale, so `live` starts at its latest servable group below the +/// requested end; `untimed_live` is that group, or `None` on a timed track. +fn first_sequence(subscription: &Subscription, untimed_live: Option) -> u64 { + let floor = subscription.floor.map(|floor| floor.group); + match subscription.live { + true => { + let live = untimed_live.unwrap_or(0); + floor.map_or(live, |floor| floor.min(live)) + } + false => floor.unwrap_or(0), + } } /// Clamp a drift budget to the publisher's retention window: nobody can wait for a late @@ -2798,11 +2811,12 @@ impl Consumer { /// [`Subscriber`] once the track info is available, or the track's abort error (or /// [`Error::Dropped`]) if it is already closed. /// - /// The read cursor starts at the group the subscription named (its floor), or 0. - /// [`Subscription::max_delay`] is what asks for data: delivery skips everything above - /// the floor that the budget convicts, so the default budget of zero delivers only - /// the latest group and a larger one reaches back over what it can still use. An - /// untimed track has nothing to convict, so it starts at its latest group. + /// The read cursor starts at the group the subscription named (its floor), lowered to + /// the live edge by [`Subscription::live`]. [`Subscription::max_delay`] is what asks + /// for data: delivery skips everything above the start that the budget convicts, so + /// the default budget of zero delivers only the latest group and a larger one reaches + /// back over what it can still use. An untimed track has nothing to convict, so `live` + /// starts at its latest group. pub fn subscribe(&self, subscription: impl Into>) -> kio::Pending { let subscription = kio::Producer::new(subscription.into().unwrap_or_default()); @@ -3453,10 +3467,10 @@ impl kio::Task for Fetching { /// /// - [`Self::set_groups`] limits **this subscriber's reads**, filtering exactly what /// this handle returns without changing the publisher's demand. -/// - [`Subscription::start`] / [`Subscription::end`], set via [`Self::update`], -/// are a **request to the publisher**. They're aggregated across every live subscriber -/// (earliest start, widest end), so they say what the publisher should send, not what -/// this subscriber sees. +/// - [`Subscription::live`], [`Subscription::floor`], and [`Subscription::end`], set via +/// [`Self::update`], are a **request to the publisher**. They're aggregated across every +/// live subscriber (anyone's `live`, the lowest floor, the widest end), so they say what +/// the publisher should send, not what this subscriber sees. /// /// They stay separate because their scopes differ: a subscriber can't filter by the /// aggregate, since another subscriber can widen it, and the publisher can't honor a @@ -3658,17 +3672,10 @@ struct Cursor { impl Cursor { fn new(state: kio::Consumer, subscription: kio::Producer) -> Self { - // An explicit start says how far back to reach, so only an unfloored subscription - // jumps to an untimed track's latest group. let min_sequence = { let preferences = subscription.read(); - match preferences.start { - Some(_) => floor_of(&preferences), - None => { - let cap = preferences.end.and_then(|end| Cap::from(end.group_end()).exclusive()); - state.read().untimed_start(cap).unwrap_or(0) - } - } + let cap = preferences.end.and_then(|end| Cap::from(end.group_end()).exclusive()); + first_sequence(&preferences, state.read().untimed_start(cap)) }; Self { state, @@ -3846,11 +3853,16 @@ impl Cursor { self.datagram_index = found_index + 1; let sequence = datagram.sequence; - let mid_group = self - .subscription - .read() - .start - .is_some_and(|start| start.group == sequence && start.frame > 0); + // A floor's frame only applies while the floor is where delivery starts. `live` + // reaches below it on a timed track; on an untimed one it may not, and the + // floor group's single-frame datagram is then delivered, best-effort anyway. + let mid_group = { + let subscription = self.subscription.read(); + !subscription.live + && subscription + .floor + .is_some_and(|floor| floor.group == sequence && floor.frame > 0) + }; if sequence >= self.min_sequence && super::subscription::before_end(sequence, self.end_sequence) && !mid_group @@ -4008,7 +4020,7 @@ impl Subscriber { /// skipped rather than handed over, so a single poll walks off a whole backlog. The /// default is [`Duration::ZERO`], which takes the live /// edge and writes the rest off; raise it to read history. [`Self::set_groups`] and - /// [`Subscription::start`] are filters, not exemptions: backfill needs a budget that + /// [`Subscription::floor`] are filters, not exemptions: backfill needs a budget that /// covers it. [`Consumer::fetch_group`] is the way to ask for one old group outright. /// The budget remains attached to a returned group: if it stalls while newer data /// advances, its pending frame read ends with [`Error::Old`]. @@ -4210,7 +4222,7 @@ impl Subscriber { /// /// A local filter, not a request: it doesn't tell the publisher anything, so the /// skipped groups are still delivered and simply not returned. To ask the publisher - /// to start there instead, set [`Subscription::start`] via [`Self::update`]. + /// to start there instead, set [`Subscription::floor`] via [`Self::update`]. /// See [Local cursor vs wire preference](Self#local-cursor-vs-wire-preference). pub(crate) fn start_at(&mut self, sequence: u64) { match &mut self.inner { @@ -5061,7 +5073,8 @@ mod test { #[test] fn the_group_range_bounds_datagrams() { let mut producer = track_producer("test", None); - let mut subscriber = producer.subscribe(Subscription::default().with_start(Position::group(5))); + let mut subscriber = + producer.subscribe(Subscription::default().with_live(false).with_floor(Position::group(5))); subscriber.set_groups(..7); for sequence in [4, 5, 7, 6] { producer @@ -5079,8 +5092,12 @@ mod test { #[test] fn a_mid_group_start_drops_the_start_groups_datagram() { let mut producer = track_producer("test", None); - let mut mid = producer.subscribe(Subscription::default().with_start(Position { group: 5, frame: 1 })); - let mut whole = producer.subscribe(Subscription::default().with_start(Position::group(5))); + let mut mid = producer.subscribe( + Subscription::default() + .with_live(false) + .with_floor(Position { group: 5, frame: 1 }), + ); + let mut whole = producer.subscribe(Subscription::default().with_live(false).with_floor(Position::group(5))); for sequence in [5, 6] { producer .insert_datagram(sequence, Timestamp::ZERO, bytes::Bytes::from_static(b"x")) @@ -6029,20 +6046,22 @@ mod test { // The budget is the only thing that asks for data; a named start only bounds how // far back it may reach. Naming group 1 at real time still delivers the live edge // alone, since the zero budget calls everything older stale. - let named = Subscription::default().with_start(Position::group(1)); + let named = Subscription::default().with_live(false).with_floor(Position::group(1)); let mut subscriber = producer.subscribe(named); assert_eq!(drain(&mut subscriber), vec![4]); // A budget reaching further back than the floor is cut off at it. let floored = Subscription::default() - .with_start(Position::group(3)) + .with_live(false) + .with_floor(Position::group(3)) .with_max_delay(Duration::from_secs(10)); let mut subscriber = producer.subscribe(floored); assert_eq!(drain(&mut subscriber), vec![3, 4]); // A floor below what the budget admits changes nothing. let slack = Subscription::default() - .with_start(Position::group(1)) + .with_live(false) + .with_floor(Position::group(1)) .with_max_delay(Duration::from_secs(2)); let mut subscriber = producer.subscribe(slack); assert_eq!(drain(&mut subscriber), vec![2, 3, 4]); @@ -6058,7 +6077,8 @@ mod test { // A resumed subscription names where it left off, which may not exist yet. The // cursor sits at the floor rather than sliding back to what is cached. let resumed = Subscription::default() - .with_start(Position::group(7)) + .with_live(false) + .with_floor(Position::group(7)) .with_max_delay(Duration::from_secs(10)); let mut subscriber = producer.subscribe(resumed); assert_eq!(drain(&mut subscriber), Vec::::new()); @@ -6679,13 +6699,15 @@ mod test { // Asking to start at the beginning is a filter, not a request for reliability. // Backfill needs a budget that covers it; without one the live edge still wins. - let mut subscriber = producer.subscribe(Subscription::default().with_start(Position::group(0))); + let mut subscriber = + producer.subscribe(Subscription::default().with_live(false).with_floor(Position::group(0))); subscriber.start_at(0); assert_eq!(drain(&mut subscriber), vec![3]); let mut patient = producer.subscribe( Subscription::default() - .with_start(Position::group(0)) + .with_live(false) + .with_floor(Position::group(0)) .with_max_delay(Duration::from_secs(10)), ); patient.start_at(0); @@ -6823,10 +6845,47 @@ mod test { append_untimed(&mut producer); } - let mut subscriber = producer.subscribe(Subscription::default().with_start(Position::group(2))); + let mut subscriber = + producer.subscribe(Subscription::default().with_live(false).with_floor(Position::group(2))); assert_eq!(drain(&mut subscriber), vec![2, 3, 4]); } + /// `live` with a floor starts at whichever is lower: the floor when it sits below the + /// latest group, and the latest group (the untimed live edge) when the floor is above it. + #[test] + fn untimed_live_with_a_floor_starts_at_the_lower_of_the_two() { + let mut producer = untimed_producer(); + for _ in 0..5 { + append_untimed(&mut producer); + } + + let mut below = producer.subscribe(Subscription::default().with_floor(Position::group(2))); + assert_eq!(drain(&mut below), vec![2, 3, 4]); + let mut above = producer.subscribe(Subscription::default().with_floor(Position::group(7))); + assert_eq!(drain(&mut above), vec![4], "the live edge sits below the floor"); + } + + /// A floor above a quiet track's newest group waits there, while a `live` subscriber + /// gets that group at once. The aggregate keeps both: the floor and `live`. + #[test] + fn a_floor_above_the_live_edge_does_not_hide_it_from_live() { + let mut producer = track_producer("test", None); + for millis in 0..4 { + append_at(&mut producer, millis * 100); + } + + let mut resumed = producer.subscribe(Subscription::default().with_live(false).with_floor(Position::group(4))); + let mut live = producer.subscribe(None); + assert_eq!(drain(&mut live), vec![3]); + assert_eq!(drain(&mut resumed), Vec::::new()); + let combined = producer.subscription().unwrap(); + assert_eq!((combined.live, combined.floor), (true, Some(Position::group(4)))); + + append_at(&mut producer, 400); + assert_eq!(drain(&mut live), vec![4]); + assert_eq!(drain(&mut resumed), vec![4]); + } + /// An unfloored untimed subscription with an end starts at the latest group below it, /// as the lite publisher serves a SUBSCRIBE with an end and no start. #[test] diff --git a/rs/moq-net/tests/announce_to_serve.rs b/rs/moq-net/tests/announce_to_serve.rs index 02a2ad23c42..cfe746ae2f3 100644 --- a/rs/moq-net/tests/announce_to_serve.rs +++ b/rs/moq-net/tests/announce_to_serve.rs @@ -70,7 +70,9 @@ fn outcome(result: &Result) -> String { /// group's frames and then how the subscription ended. Read continuously, as a /// real subscriber would, so the subscription is driven the whole time. async fn read(broadcast: &moq_net::broadcast::Consumer, name: &str) -> mpsc::UnboundedReceiver { - let subscription = moq_net::track::Subscription::default().with_start(moq_net::track::Position::group(0)); + let subscription = moq_net::track::Subscription::default() + .with_live(false) + .with_floor(moq_net::track::Position::group(0)); let mut sub = broadcast .track(name) .unwrap() diff --git a/rs/moq-net/tests/finished_broadcast_mock.rs b/rs/moq-net/tests/finished_broadcast_mock.rs index cf0b33ad0af..760a6a83adc 100644 --- a/rs/moq-net/tests/finished_broadcast_mock.rs +++ b/rs/moq-net/tests/finished_broadcast_mock.rs @@ -53,7 +53,9 @@ async fn round(finish_broadcast: bool) -> (Vec>, Option) .expect("broadcast resolves"); let reader = moq_net_sim::spawn(async move { - let subscription = moq_net::track::Subscription::default().with_start(moq_net::track::Position::group(0)); + let subscription = moq_net::track::Subscription::default() + .with_live(false) + .with_floor(moq_net::track::Position::group(0)); let mut sub = remote .track("video") .unwrap() diff --git a/rs/moq-net/tests/head_fetch.rs b/rs/moq-net/tests/head_fetch.rs index 2323814fa4b..681ff5c691d 100644 --- a/rs/moq-net/tests/head_fetch.rs +++ b/rs/moq-net/tests/head_fetch.rs @@ -92,7 +92,7 @@ async fn round(version: &str) { let mut peer_sub = peer_remote .track("catalog.json") .unwrap() - .subscribe(Subscription::default().with_start(start)) + .subscribe(Subscription::default().with_live(false).with_floor(start)) .await .expect("peer subscribe"); let mut peer_group = moq_net_sim::timeout(TIMEOUT, peer_sub.recv_group()) diff --git a/rs/moq-net/tests/history_groups.rs b/rs/moq-net/tests/history_groups.rs index 0cf508dd2c7..8b2ae2a0935 100644 --- a/rs/moq-net/tests/history_groups.rs +++ b/rs/moq-net/tests/history_groups.rs @@ -79,7 +79,8 @@ async fn round(version: &str, relay: bool, newest_first: bool) -> Vec<(u64, usiz let reader = moq_net_sim::spawn(async move { let subscription = Subscription::default() .with_max_delay(FOREVER) - .with_start(Position::group(0)); + .with_live(false) + .with_floor(Position::group(0)); let mut sub = remote .track("history") .unwrap() @@ -113,9 +114,8 @@ async fn round(version: &str, relay: bool, newest_first: bool) -> Vec<(u64, usiz .await .expect("no subscriber appeared") .unwrap(); - // Pre-06 wires can't name group 0 (it reads as the latest group), so the publisher - // must place its cursor before the groups exist. Simulated time advances only once - // every task is idle, so this settles the subscription first. + // Place the publisher's cursor before the groups exist. Simulated time advances only + // once every task is idle, so this settles the subscription first. moq_net_sim::sleep(Duration::from_millis(10)).await; // The hop the publisher's group streams cross first: into the relay, when there is one. @@ -220,7 +220,8 @@ async fn a_late_subscriber_receives_the_relays_cached_history() { let remote = consumer.request_broadcast("bcast", None).await.unwrap(); let subscription = Subscription::default() .with_max_delay(FOREVER) - .with_start(Position::group(0)); + .with_live(false) + .with_floor(Position::group(0)); let first_subscription = subscription.clone(); let first_reader = moq_net_sim::spawn(async move { let mut sub = remote diff --git a/rs/moq-net/tests/live_floor.rs b/rs/moq-net/tests/live_floor.rs new file mode 100644 index 00000000000..489b69ae7e1 --- /dev/null +++ b/rs/moq-net/tests/live_floor.rs @@ -0,0 +1,317 @@ +//! Subscribers with different floors share one upstream subscription, and each still +//! receives what its own floor and `max_delay` allow. +//! +//! A floor and `live` are separate: the aggregate is the lowest floor plus `live` when +//! anyone wants it. A resumed subscriber whose floor sits above the live edge must never +//! hide the latest group from a subscriber that wants it, and a wire without a `live` +//! field asks for enough that the relay can filter locally. + +mod support; + +use std::time::Duration; + +use moq_net::{Hop, Timestamp, Version, broadcast, group, origin, track}; +use support::harness::{MockConnectOptions, MockPair, connect_mock}; + +const TIMEOUT: Duration = Duration::from_secs(5); + +/// Long enough to say a group is not coming. +const QUIET: Duration = Duration::from_millis(500); + +/// A budget no group outlives. +const FOREVER: Duration = Duration::from_millis((1 << 53) - 1); + +fn produce_origin(hop: u64) -> origin::Producer { + let (producer, driver) = origin::Producer::new(origin::Config::new(Hop::new(hop).unwrap())); + support::harness::spawn(driver); + producer +} + +async fn link(version: Version, from: &origin::Producer, to: &origin::Producer) -> MockPair { + let mut options = MockConnectOptions::new(version); + options.server_publish = Some(from.consume()); + options.client_subscribe = Some(to.clone()); + connect_mock(options).await +} + +async fn request(origin: &origin::Producer) -> broadcast::Consumer { + let consumer = origin.consume(); + moq_net_sim::timeout(TIMEOUT, consumer.routed("live")) + .await + .expect("announce timeout") + .expect("routed"); + moq_net_sim::timeout(TIMEOUT, consumer.request_broadcast("live", None)) + .await + .expect("resolve timeout") + .expect("broadcast resolves") +} + +/// Announce `live/track`, retained for good. +fn publish(origin: &origin::Producer) -> (broadcast::Producer, track::Producer) { + let broadcast = origin.create_broadcast("live").unwrap(); + let track = broadcast + .create_track("track", track::Info::default().with_max_age(FOREVER)) + .unwrap(); + broadcast.announce(Default::default()).unwrap(); + (broadcast, track) +} + +/// Write a finished group of `frames` frames, stamped `millis`. +fn write_group(track: &mut track::Producer, sequence: u64, millis: u64, frames: usize) { + let mut group = track.create_group(group::Info { sequence }).unwrap(); + for _ in 0..frames { + group + .write_frame(Timestamp::from_millis(millis).unwrap(), &b"x"[..]) + .unwrap(); + } + group.finish().unwrap(); +} + +/// A floor alone, as a resumed subscriber names where it left off. +fn floored(floor: track::Position) -> track::Subscription { + track::Subscription::default().with_live(false).with_floor(floor) +} + +/// The next group and the index of its first frame, or `None` if nothing arrives. +async fn next(sub: &mut track::Subscriber, wait: Duration) -> Option<(u64, u64)> { + let group = moq_net_sim::timeout(wait, sub.recv_group()).await.ok()?; + let group = group.expect("recv_group")?; + Some((group.sequence, group.index())) +} + +/// The sequences of the next `count` groups, sorted. +async fn groups(sub: &mut track::Subscriber, count: usize) -> Vec { + let mut got = Vec::new(); + while got.len() < count { + match next(sub, TIMEOUT).await { + Some((sequence, _)) => got.push(sequence), + None => break, + } + } + got.sort(); + got +} + +/// Run `scenario` for each version and report every failure together. +async fn each_version>>( + versions: &[&str], + scenario: impl Fn(Version) -> F, +) { + let mut failures = Vec::new(); + for version in versions { + let version: Version = version.parse().unwrap(); + match moq_net_sim::timeout(Duration::from_secs(60), scenario(version)).await { + Ok(Ok(())) => {} + Ok(Err(err)) => failures.push(format!("{version}: {err}")), + Err(_) => failures.push(format!("{version}: scenario timed out")), + } + } + assert!(failures.is_empty(), "\n{}", failures.join("\n")); +} + +/// Wait until the publisher's aggregate satisfies `done`. +async fn aggregate(track: &track::Producer, done: impl Fn(&track::Subscription) -> bool) -> Result<(), String> { + let mut track = track.clone(); + moq_net_sim::timeout(TIMEOUT, async { + while !track.subscription().is_some_and(|sub| done(&sub)) { + track.subscription_changed().await.unwrap(); + } + }) + .await + .map_err(|_| format!("the publisher never saw the demand: {:?}", track.subscription())) +} + +/// A quiet track's newest group is 3. A resumed subscriber floors at 4, and a live one +/// joins after it through the same relay. The live one gets group 3 at once, the resumed +/// one does not, and both get group 4 once it exists. `downstream` reads one more hop +/// away instead of in-process at the relay. +async fn starve(version: Version, downstream: bool) -> Result<(), String> { + let publisher = produce_origin(1); + let relay = produce_origin(2); + let (_broadcast, mut track) = publish(&publisher); + for sequence in 0..4 { + write_group(&mut track, sequence, sequence * 100, 1); + } + + let _upstream = link(version, &publisher, &relay).await; + let subscriber = produce_origin(3); + let _downstream = link(version, &relay, &subscriber).await; + let remote = request(if downstream { &subscriber } else { &relay }).await; + + let mut resumed = moq_net_sim::timeout( + TIMEOUT, + remote + .track("track") + .unwrap() + .subscribe(floored(track::Position::group(4))), + ) + .await + .map_err(|_| "resume never resolved")? + .map_err(|err| format!("resume: {err}"))?; + aggregate(&track, |sub| sub.floor == Some(track::Position::group(4))).await?; + + let mut live = moq_net_sim::timeout(TIMEOUT, remote.track("track").unwrap().subscribe(None)) + .await + .map_err(|_| "live never resolved")? + .map_err(|err| format!("live: {err}"))?; + match next(&mut live, TIMEOUT).await { + Some((3, 0)) => {} + other => return Err(format!("the live subscriber got {other:?}, not group 3")), + } + if let Some(got) = next(&mut resumed, QUIET).await { + return Err(format!("the resume got {got:?} below its floor")); + } + + write_group(&mut track, 4, 400, 1); + for (who, sub) in [("live", &mut live), ("resumed", &mut resumed)] { + match next(sub, TIMEOUT).await { + Some((4, 0)) => {} + other => return Err(format!("the {who} subscriber got {other:?}, not group 4")), + } + } + Ok(()) +} + +#[moq_net_sim::test] +async fn a_floor_above_the_live_edge_never_starves_a_live_subscriber() { + each_version(&["moq-lite-06", "moq-lite-07-wip"], |version| starve(version, false)).await; +} + +#[moq_net_sim::test] +async fn a_floor_above_the_live_edge_never_starves_a_live_subscriber_downstream() { + each_version(&["moq-lite-06", "moq-lite-07-wip"], |version| starve(version, true)).await; +} + +/// A buffering live subscriber (max age > 0) shares an upstream subscription with a floor +/// above the live edge, and still receives the backlog its budget allows. +#[moq_net_sim::test] +async fn a_buffered_live_subscriber_keeps_its_backlog_beside_a_floor() { + each_version( + &["moq-lite-06", "moq-lite-07-wip", "moq-transport-22"], + |version| async move { + let publisher = produce_origin(1); + let relay = produce_origin(2); + let (_broadcast, mut track) = publish(&publisher); + for sequence in 2..5 { + write_group(&mut track, sequence, sequence * 100, 1); + } + let _upstream = link(version, &publisher, &relay).await; + + let remote = request(&relay).await; + let handle = remote.track("track").unwrap(); + // Registered together, so the first upstream request already merges them. + let resumed = handle.subscribe(floored(track::Position::group(5))); + let live = handle.subscribe(track::Subscription::default().with_max_delay(Duration::from_secs(10))); + let _resumed = resumed.await.map_err(|err| format!("resume: {err}"))?; + let mut live = live.await.map_err(|err| format!("live: {err}"))?; + + match groups(&mut live, 3).await.as_slice() { + [2, 3, 4] => Ok(()), + got => Err(format!("the buffered live subscriber got {got:?}, not 2..=4")), + } + }, + ) + .await; +} + +/// A floor partway through the latest group merges with `live`, which starts the group at +/// its first frame: the live subscriber still gets frames 0-4. +#[moq_net_sim::test] +async fn a_frame_floor_merged_with_live_keeps_the_head_of_the_group() { + each_version(&["moq-lite-06", "moq-lite-07-wip"], |version| async move { + let publisher = produce_origin(1); + let relay = produce_origin(2); + let (_broadcast, mut track) = publish(&publisher); + write_group(&mut track, 3, 300, 8); + let _upstream = link(version, &publisher, &relay).await; + + let remote = request(&relay).await; + let handle = remote.track("track").unwrap(); + let resumed = handle.subscribe(floored(track::Position { group: 3, frame: 5 })); + let live = handle.subscribe(None); + let _resumed = resumed.await.map_err(|err| format!("resume: {err}"))?; + let mut live = live.await.map_err(|err| format!("live: {err}"))?; + + let group = moq_net_sim::timeout(TIMEOUT, live.recv_group()) + .await + .map_err(|_| "no group arrived")? + .map_err(|err| format!("recv: {err}"))? + .ok_or("the track ended")?; + if (group.sequence, group.index()) != (3, 0) { + return Err(format!("got group {} from frame {}", group.sequence, group.index())); + } + let mut group = group; + let mut frames = 0; + while let Ok(Ok(Some(_))) = moq_net_sim::timeout(TIMEOUT, group.read_frame()).await { + frames += 1; + } + match frames { + 8 => Ok(()), + frames => Err(format!("got {frames} of group 3's 8 frames")), + } + }) + .await; +} + +/// `live` merged with a floor of 2 through a peer whose SUBSCRIBE has no `Live` field: the +/// relay asks from group 0 and filters locally, so the floor still gets groups 2-4. +#[moq_net_sim::test] +async fn live_merged_with_a_floor_reaches_back_through_an_older_lite_peer() { + each_version(&["moq-lite-03", "moq-lite-04", "moq-lite-05"], |version| async move { + let publisher = produce_origin(1); + let relay = produce_origin(2); + let (_broadcast, mut track) = publish(&publisher); + for sequence in 2..5 { + write_group(&mut track, sequence, sequence * 100, 1); + } + let _upstream = link(version, &publisher, &relay).await; + + let remote = request(&relay).await; + let handle = remote.track("track").unwrap(); + let floor = handle.subscribe(floored(track::Position::group(2)).with_max_delay(FOREVER)); + let live = handle.subscribe(None); + let mut floor = floor.await.map_err(|err| format!("floor: {err}"))?; + let _live = live.await.map_err(|err| format!("live: {err}"))?; + + match groups(&mut floor, 3).await.as_slice() { + [2, 3, 4] => Ok(()), + got => Err(format!("the floored subscriber got {got:?}, not 2..=4")), + } + }) + .await; +} + +/// A buffering `live` merged with a floor above the live edge, over a moq-transport +/// upstream: the relay subscribes from group 0 and still delivers the fresh groups 2-4. +/// +/// Before draft-20 that start is a joining FETCH from group 0, which spans several groups +/// and is refused (one group per FETCH), so only the live edge arrives there. +#[moq_net_sim::test] +async fn live_merged_with_a_floor_reaches_back_through_moq_transport() { + each_version(&["moq-transport-16", "moq-transport-22"], |version| async move { + let publisher = produce_origin(1); + let relay = produce_origin(2); + let (_broadcast, mut track) = publish(&publisher); + for sequence in 2..5 { + write_group(&mut track, sequence, sequence * 100, 1); + } + let _upstream = link(version, &publisher, &relay).await; + + let remote = request(&relay).await; + let handle = remote.track("track").unwrap(); + let resumed = handle.subscribe(floored(track::Position::group(5))); + let live = handle.subscribe(track::Subscription::default().with_max_delay(Duration::from_secs(10))); + let _resumed = resumed.await.map_err(|err| format!("resume: {err}"))?; + let mut live = live.await.map_err(|err| format!("live: {err}"))?; + + let expected: &[u64] = match version.to_string().as_str() { + "moq-transport-16" => &[4], + _ => &[2, 3, 4], + }; + match groups(&mut live, expected.len()).await { + got if got == expected => Ok(()), + got => Err(format!("the buffered live subscriber got {got:?}, not {expected:?}")), + } + }) + .await; +} diff --git a/rs/moq-net/tests/quiet_catalog.rs b/rs/moq-net/tests/quiet_catalog.rs index 17eb3d8a799..936903d9daa 100644 --- a/rs/moq-net/tests/quiet_catalog.rs +++ b/rs/moq-net/tests/quiet_catalog.rs @@ -157,7 +157,11 @@ async fn a_quiet_catalog_reaches_a_fresh_reader_when_a_peer_resumes_past_it() { let _sub = resume_remote .track("catalog.json") .unwrap() - .subscribe(track::Subscription::default().with_start(track::Position::group(1))) + .subscribe( + track::Subscription::default() + .with_live(false) + .with_floor(track::Position::group(1)), + ) .await .expect("resume subscribe"); moq_net_sim::sleep(Duration::from_secs(60)).await; @@ -167,7 +171,7 @@ async fn a_quiet_catalog_reaches_a_fresh_reader_when_a_peer_resumes_past_it() { // rather than opening its own floorless subscription. let resumed = Some(track::Position::group(1)); moq_net_sim::timeout(TIMEOUT, async { - while track.subscription().map(|sub| sub.start) != Some(resumed) { + while track.subscription().map(|sub| sub.floor) != Some(resumed) { track.subscription_changed().await.unwrap(); } }) diff --git a/rs/moq-net/tests/settled_end_survives_abort.rs b/rs/moq-net/tests/settled_end_survives_abort.rs index e737c31fd25..5859ced2218 100644 --- a/rs/moq-net/tests/settled_end_survives_abort.rs +++ b/rs/moq-net/tests/settled_end_survives_abort.rs @@ -25,7 +25,8 @@ async fn round(abort: bool, ordered: bool) -> (Vec, Option) { // From the first group with a replay window: a late reader is owed the whole track, // not the live edge. let subscription = moq_net::track::Subscription::default() - .with_start(moq_net::track::Position::group(0)) + .with_live(false) + .with_floor(moq_net::track::Position::group(0)) .with_max_delay(std::time::Duration::from_secs(30)); let mut consumer = track.subscribe(subscription); diff --git a/rs/moq-net/tests/subscription_end_integrity.rs b/rs/moq-net/tests/subscription_end_integrity.rs index 401c75a6d9c..7b05b208f31 100644 --- a/rs/moq-net/tests/subscription_end_integrity.rs +++ b/rs/moq-net/tests/subscription_end_integrity.rs @@ -76,7 +76,9 @@ async fn round(drop_session: bool) -> (Vec>, Option) { let (go_tx, go_rx) = futures::channel::oneshot::channel::<()>(); let reader = moq_net_sim::spawn(async move { - let subscription = moq_net::track::Subscription::default().with_start(moq_net::track::Position::group(0)); + let subscription = moq_net::track::Subscription::default() + .with_live(false) + .with_floor(moq_net::track::Position::group(0)); let mut sub = remote .track("video") .unwrap() @@ -218,7 +220,9 @@ async fn killed(version: &str, local: bool) -> (Option, Option Outcome { } = connect(version).await; let reader = moq_net_sim::spawn(async move { - let subscription = moq_net::track::Subscription::default().with_start(moq_net::track::Position::group(0)); + let subscription = moq_net::track::Subscription::default() + .with_live(false) + .with_floor(moq_net::track::Position::group(0)); let mut sub = remote .track("video") .unwrap() @@ -259,7 +261,9 @@ async fn ietf_leaving_cancels_a_blocked_end_of_track() { _keep, } = connect(version).await; - let subscription = moq_net::track::Subscription::default().with_start(moq_net::track::Position::group(0)); + let subscription = moq_net::track::Subscription::default() + .with_live(false) + .with_floor(moq_net::track::Position::group(0)); let mut sub = remote .track("video") .unwrap() @@ -317,7 +321,9 @@ async fn a_lost_datagram_never_delays_the_end() { } = connect(version).await; let reader = moq_net_sim::spawn(async move { - let subscription = moq_net::track::Subscription::default().with_start(moq_net::track::Position::group(0)); + let subscription = moq_net::track::Subscription::default() + .with_live(false) + .with_floor(moq_net::track::Position::group(0)); let mut sub = remote .track("video") .unwrap() diff --git a/rs/moq-net/tests/untimed.rs b/rs/moq-net/tests/untimed.rs index 926ad201105..96b4fc5be36 100644 --- a/rs/moq-net/tests/untimed.rs +++ b/rs/moq-net/tests/untimed.rs @@ -182,7 +182,8 @@ async fn an_explicit_start_holds_on_an_untimed_track() { let consumer = chain.nodes[1].consume().request_broadcast("room", None).await.unwrap(); let subscription = track::Subscription::default() - .with_start(track::Position::group(2)) + .with_live(false) + .with_floor(track::Position::group(2)) .with_max_delay(Duration::from_secs(30)); let mut subscriber = moq_net_sim::timeout(TIMEOUT, consumer.track("data").unwrap().subscribe(subscription)) .await diff --git a/rs/moq-stats/src/produce.rs b/rs/moq-stats/src/produce.rs index dab7e73d18b..6613e74410f 100644 --- a/rs/moq-stats/src/produce.rs +++ b/rs/moq-stats/src/produce.rs @@ -1209,10 +1209,11 @@ mod tests { family.reclaim(&mut requested); assert!(family.tracks.is_empty()); - let subscribing = consumer - .track(name) - .unwrap() - .subscribe(track::Subscription::default().with_start(track::Position::group(floor))); + let subscribing = consumer.track(name).unwrap().subscribe( + track::Subscription::default() + .with_live(false) + .with_floor(track::Position::group(floor)), + ); family.adopt_pair( &broadcast, &mut requested, diff --git a/rs/moq-tokio/tests/broadcast.rs b/rs/moq-tokio/tests/broadcast.rs index d78b806f1b3..948fe4bb814 100644 --- a/rs/moq-tokio/tests/broadcast.rs +++ b/rs/moq-tokio/tests/broadcast.rs @@ -1106,7 +1106,8 @@ async fn broadcast_route_migration() { // Resolve and subscribe: the cheaper route (A) serves the track. let subscription = moq_net::track::Subscription::default() - .with_start(moq_net::track::Position::group(1)) + .with_live(false) + .with_floor(moq_net::track::Position::group(1)) .with_max_delay(Duration::from_secs(10)); let broadcast = tokio::time::timeout(TIMEOUT, sub_consumer.request_broadcast("test", None)) .await diff --git a/rs/moq-video/src/decode/consumer.rs b/rs/moq-video/src/decode/consumer.rs index 7704dd365b7..5ed6902cf27 100644 --- a/rs/moq-video/src/decode/consumer.rs +++ b/rs/moq-video/src/decode/consumer.rs @@ -368,14 +368,14 @@ mod tests { ); // The skip is the local read cursor and nothing else. Asking for it - // through `Subscription::start` would look equivalent and is not: the + // through `Subscription::floor` would look equivalent and is not: the // floor is aggregated across every live subscriber and tells the // publisher what to send, so naming a cached sequence there rewinds the // track for everyone reading it. A rendition switched away from and back // to is the case that bites, because its cached sequence is stale by // then and the publisher resends the broadcast from it. assert_eq!( - published.subscription().and_then(|sub| sub.start), + published.subscription().and_then(|sub| sub.floor), None, "the publisher was asked to rewind the track", );