diff --git a/drafts/draft-lcurley-moq-lite.md b/drafts/draft-lcurley-moq-lite.md index 55bbf3b7164..58ce3bd9c6b 100644 --- a/drafts/draft-lcurley-moq-lite.md +++ b/drafts/draft-lcurley-moq-lite.md @@ -162,7 +162,7 @@ The duration before an incomplete group is dropped is determined by the applicat Every subscription is scoped to a single Track. A subscription starts at a configurable Group (defaulting to the latest) and continues until a configurable end Group or until either the publisher or subscriber cancels the subscription. -Both bounds may be refined to a Frame within their Group, so a subscription can start or stop partway through a Group rather than only on a Group boundary (see [Positions](#positions)). +A subscription reads whole Groups; a FETCH reads one Group, or a slice of one (see [Positions](#positions)). The subscriber and publisher both indicate their delivery preference: - `Priority` indicates if Track A should be transmitted instead of Track B. @@ -194,7 +194,7 @@ The contents are opaque to the moq-lite layer. Frames within a Group are numbered from 0 in the order they were produced. This index is not transmitted per frame; it is implied by position within the Group, anchored by the [GROUP](#group) message's `Frame Start`. -A Group Stream normally starts at frame 0, but MAY start later when the publisher only holds (or was only asked for) part of the Group. +A Group Stream normally starts at frame 0. Each frame of a timed Track carries a presentation timestamp expressed in the Track's `Timescale` (see [TRACK_INFO](#track-info)), used by the moq-lite layer for [expiration](#expiration) decisions. A Track without a `Timescale` is untimed: its frames carry no timestamp. @@ -207,22 +207,16 @@ A relay serves a Track's timestamps on as it received them, including ones it di A Position is a (Group Sequence, Frame Index) pair identifying one frame within a Track. Positions order lexicographically: by group first, then by frame within the group. -SUBSCRIBE and FETCH bound their delivery by Position rather than by Group alone. -A SUBSCRIBE Range's Absolute anchors name Positions (see [SUBSCRIBE](#subscribe)), and FETCH carries a `Frame Start` and `Frame End` within its group; a frame index of 0 at the start, or an absent end frame, is the whole group. - -The bounds are chosen so that two subscriptions abut exactly. -A Range starts at its start Position and stops just before its end Position, so a subscriber that has received frames 0 through `N-1` of group `G` and wants the remainder starts a Range at (`G`, `N`). -Ending the first subscription's Range at (`G`, `N`) covers exactly the complement with no gap and no overlap: the two requests carry the *same* numbers on the wire. +SUBSCRIBE reads whole Groups: a Range's Absolute Anchors name Groups (see [SUBSCRIBE](#subscribe)), so a subscription never starts or stops partway through one. +FETCH reads one Group, or a slice of one: its `Frame Start` and `Frame End` bound the returned frames, and a frame index of 0 at the start, or an absent end frame, is the whole Group. +A subscriber that has received frames 0 through `N-1` of group `G` and wants the rest FETCHes `G` from frame `N` and subscribes from Group `G+1`, so the two requests cover the Track with no gap and no overlap. This is what lets a subscriber move a Track to a different publisher partway through a Group instead of waiting for the next Group to start, which matters for Tracks whose current Group may stay open indefinitely. A Group can therefore be assembled from more than one publisher: each contributes a disjoint run of frames, and the subscriber concatenates them in index order. -A partial Group is only ever delivered to a subscriber that asked for one. -A publisher that cannot serve a Group from the frame the subscription names MUST skip that Group entirely and resolve the subscription to a later one, rather than deliver the part it holds. -The resolved start is therefore always either exactly the requested Position or the beginning of a later Group, never some third Position the subscriber did not choose. - +A partial Group is only ever delivered to a subscriber that asked for one with FETCH, or as the [continuation](#yield) of a paused Group. +A publisher that holds only part of a Group a subscription covers names it in a [SUBSCRIBE_DROP](#subscribe-drop) rather than deliver the part it holds. A Group is the unit of decodability: its earlier Frames may carry a keyframe or compression state the later Frames need. -A subscriber MAY start a Range at any frame index, whether or not it retains the earlier Frames. # Flow This section outlines the flow of messages within a moq-lite session. @@ -889,7 +883,6 @@ A Group MAY contain zero FRAME messages, potentially indicating a gap in the tra A frame MAY contain an empty payload, potentially indicating a gap in the group. The GROUP message's `Frame Start` gives the index of the first FRAME on the stream, so a stream carrying only part of a Group is self-describing. -It is redundant with the subscription's own start Position in the steady state, and carried anyway because a SUBSCRIBE_UPDATE that moves the bound races the Group Streams already in flight: the two travel on different streams with no ordering between them, so the receiver would otherwise have to guess which bound a given Group Stream was opened under. A publisher MUST NOT send two Group Streams for the same Group Sequence on one subscription, except to re-offer a Group whose stream it reset (see [SUBSCRIBE_DROP](#subscribe-drop)), or to [continue](#yield) one. The re-offer can arrive before that reset: a subscriber holds a Group Stream for a sequence whose earlier stream is still arriving unread until that stream ends, then discards it if the earlier stream completed the Group, or reads it if the earlier stream was reset. @@ -1634,7 +1627,6 @@ Range { Anchor { Anchor Type (i) [Group Sequence (i)] - [Frame Index (i)] [Groups Back (i)] [Timestamp (i)] } @@ -1685,7 +1677,7 @@ Each Range implicitly takes the next Range ID on the stream, starting at 0, whic Which fields follow, and what the Anchor names: - None (0x0): no fields. As a `Start`, the oldest Group (no floor); as an `End`, the live edge (unbounded). -- Absolute (0x1): `Group Sequence` and `Frame Index`, a Position. A `Start` delivers from it and an `End` stops just before it. +- Absolute (0x1): `Group Sequence`. A `Start` delivers from that Group and an `End` stops just before it. - Relative (0x2): `Groups Back`, the Group that many before the latest, 0 being the latest. Included as an `End`. - Time (0x3): `Timestamp`, the Group containing that presentation time in the Track's `Timescale` (see [TRACK_INFO](#track-info)). Included as an `End`. On an untimed Track it is refused with TIMESTAMP_MISMATCH. @@ -1704,9 +1696,7 @@ A subscriber that buffers is then handed the head of what it can still play inst A floor above the latest group simply waits there: that is a resumed subscription naming where it left off. Reaching back is best-effort, not a guarantee that the groups still exist; see `Publisher Max Age` in [TRACK_INFO](#track-info). -An Absolute `Start` past frame 0 delivers its group from that frame; a start resolved at a later group is always delivered from frame 0. -A subscriber MAY name any start frame, as described in [Positions](#positions). -An Absolute `End` past frame 0 delivers its group up to that frame. +A Range covers whole Groups; a subscriber reads a slice of a Group with FETCH (see [Positions](#positions)). ## SUBSCRIBE_UPDATE {#subscribe-update} A subscriber can modify a subscription with a SUBSCRIBE_UPDATE message. @@ -1737,7 +1727,6 @@ SUBSCRIBE_UPDATE Message { See [SUBSCRIBE](#subscribe) for information about each field. An `Update` no larger than the previous one only confuses the subscriber's own accounting of stale drops; the publisher echoes the latest value it read. -Moving a `Start` forward within a group that is already being delivered does not retract frames the publisher has already sent; it only bounds what the publisher sends from here on. **Remove Count** and **Range ID**: The live Ranges to retire. @@ -1979,7 +1968,6 @@ A subscriber takes it as where the live feed is: a relay holding groups from an One per Range of the SUBSCRIBE, in Range ID order: the Group where the Range's delivery starts, plus one, so a subscriber learns where a Relative or Time Anchor landed. 0 means the Range has no Group to deliver, including one an update retired before the answer. The Range's Groups from this one are owed as described in [SUBSCRIBE_DROP](#subscribe-drop); an earlier one is not, though it MAY still arrive. -A Range whose `Range Start` is its requested start Group begins at the requested frame, and otherwise at frame 0 (see [Positions](#positions)). A subscriber MUST treat a `Range Start Count` other than the SUBSCRIBE's `Range Count` as a PROTOCOL_VIOLATION. A Range an update adds has its start reported by [SUBSCRIBE_STARTED](#subscribe-started) instead, even one added before SUBSCRIBE_OK. @@ -2328,8 +2316,8 @@ A subscriber MUST handle gaps, potentially caused by congestion. **Frame Start**: The index of the first FRAME message on this stream within the group (see [Positions](#positions)). A plain absolute index; 0 means the stream carries the group from its beginning, which is the common case. -A non-zero value means the leading frames are not on this stream, either because the subscription started partway into this group or because the publisher only holds part of it. -The subscriber MUST NOT assume it will receive the missing frames on another stream; they are a gap unless a separate subscription or FETCH covers them, or the stream [continues](#yield) a paused Group. +A non-zero value means the leading frames are not on this stream. +The subscriber MUST NOT assume it will receive the missing frames on another stream; they are a gap unless a FETCH covers them, or the stream [continues](#yield) a paused Group. ## FRAME @@ -2452,10 +2440,10 @@ An application-specific payload. ### Subscriptions {#changelog-07-subscribe} -- Replaced `Group Start`, `Group End`, `Frame Start`, and `Frame End` in SUBSCRIBE and SUBSCRIBE_UPDATE with a non-empty list of Ranges. Each end is an Anchor: none, an absolute Position (exclusive as an end), a count of Groups back from the latest, or a presentation time. +- Replaced `Group Start`, `Group End`, `Frame Start`, and `Frame End` in SUBSCRIBE and SUBSCRIBE_UPDATE with a non-empty list of Ranges of whole Groups. Each end is an Anchor: none, an absolute Group (exclusive as an end), a count of Groups back from the latest, or a presentation time. - Added `Group Order` to SUBSCRIBE and SUBSCRIBE_UPDATE: descending (0x0, the previous fixed order) or ascending (0x1). - Added an optional `Subscriber Min Age` to SUBSCRIBE and SUBSCRIBE_UPDATE, the subscriber's target latency, encoded as milliseconds plus one with zero meaning not stated. A relay forwards the smallest stated value. -- SUBSCRIBE may start at any frame, including when the subscriber no longer holds the prefix. +- FETCH reads one Group or a slice of one, and SUBSCRIBE reads whole Groups: a read starting partway through a Group FETCHes it. A publisher holding only part of a Group a subscription covers drops it. - A SUBSCRIBE_UPDATE that lowers a `Start` covers the groups below the old start that the publisher still holds, not only those that arrive later. A group already sent is not sent again. - Replaced SUBSCRIBE_OK `Group` with a `Range Start` per Range: the Group where that Range's delivery starts, including at a Relative or Time Anchor, from which its Groups are owed. - Each Range takes an implicit Range ID, and SUBSCRIBE_UPDATE edits the live Ranges instead of restating them: `Remove Count` Range IDs to retire, then the Ranges to add. The live Ranges must stay non-empty and fit in one SUBSCRIBE. A Range resolves its Anchors once, however many updates leave it live. diff --git a/js/net/src/fill.ts b/js/net/src/fill.ts index 4c3ab1c11b3..96529e9419a 100644 --- a/js/net/src/fill.ts +++ b/js/net/src/fill.ts @@ -384,8 +384,11 @@ export class FillState { /** The still-wanted runs of a fill, in ascending order. @internal */ export let wantedRuns: (fill: Fill) => Run[]; -/** Wait for a change to the groups or readers of a fill. @internal */ -export let fillChanged: (fill: Fill) => Promise; +/** + * Wait for a change to the groups or readers of a fill, which a lost {@link race} releases. + * @internal + */ +export let fillChanged: (fill: Fill) => GetPromise; /** * The budget in milliseconds an OLD answer was judged under, and whether that is the publisher's * max age, which binds every reader, rather than the request's own budget. @@ -514,7 +517,7 @@ export class Fill { static { makeFill = (state, pending) => new Fill(state, pending); wantedRuns = (fill) => fill.#pending.owed(); - fillChanged = (fill) => Promise.resolve(fill.#pending.revision.changed()); + fillChanged = (fill) => Signal.race(fill.#pending.revision); rejectGroups = (fill, start, end, error, judged) => { if (toStreamCode(error) !== StreamCode.Old) { fill.refuseGroups(start, end, error); diff --git a/js/net/src/ietf/subscriber.ts b/js/net/src/ietf/subscriber.ts index 6dcea7094a5..2944b802ffc 100644 --- a/js/net/src/ietf/subscriber.ts +++ b/js/net/src/ietf/subscriber.ts @@ -1205,7 +1205,7 @@ export class Subscriber { while (!receiving) { if (!wantedRuns(fill).some((run) => run.start < until && run.end > start)) throw new StreamError(StreamCode.Cancel); - await untilAborted(fillChanged(fill), active); + await untilAborted(Promise.resolve(fillChanged(fill)), active); } await untilAborted(receiving.unwanted(), active); throw new StreamError(StreamCode.Cancel); diff --git a/js/net/src/internal.ts b/js/net/src/internal.ts index 20a5044b06a..1ba26da295b 100644 --- a/js/net/src/internal.ts +++ b/js/net/src/internal.ts @@ -661,6 +661,8 @@ export const hooks: { * by `track.ts`. */ arrivingGroup: (producer: Producer, sequence: bigint) => Arriving | undefined; + /** The cached groups of `[start, end)` still arriving, by sequence, costing what the cache holds. Assigned by `track.ts`. */ + arrivingIn: (producer: Producer, start: bigint, end: bigint) => bigint[]; /** The producer's largest reported location and its subsequent changes. */ producerLargest: (producer: Producer) => Getter; @@ -1229,6 +1231,9 @@ export const hooks: { arrivingGroup: () => { throw new Error("track.ts not loaded"); }, + arrivingIn: () => { + throw new Error("track.ts not loaded"); + }, succeedGroup: () => { throw new Error("group.ts not loaded"); }, diff --git a/js/net/src/lite/copy_fill.test.ts b/js/net/src/lite/copy_fill.test.ts index d6a8ce7a9d2..af801a03d4e 100644 --- a/js/net/src/lite/copy_fill.test.ts +++ b/js/net/src/lite/copy_fill.test.ts @@ -1,7 +1,8 @@ // A received copy asks its publisher for the groups its readers miss, over lite FETCH, as the Rust -// lite subscriber's `FillServeRun` does, or on lite-07 for a reader with no budget, by run on a -// SUBSCRIBE of its own, as its `Recovery` does: below where the copy's feed started, by the copy's -// start rule, and inside the copy, once it evicted what it received. +// lite subscriber's `FillServeRun` does, or on lite-07 for more than one group, through its +// subscription's ranges, as its `RangeFills` does, and on a SUBSCRIBE of its own for what that +// subscription already delivered, as its `Recovery` does: below where the copy's feed started, by +// the copy's start rule, and inside the copy, once it evicted what it received. import { expect, test } from "bun:test"; import { Duration, Elapsed, Manual, sleep, timeout } from "@moq/clock"; import { DEFAULT_EXPIRY } from "../cache.ts"; @@ -127,10 +128,11 @@ test("copy fill: lite-07 fills below a start resolved from the budget, as far as expect((await narrow.recvGroup())?.sequence).toBe(3n); const before = peer.fetches(); - // The copy cannot tell which budget picked its start, so it fills what this one reaches. + // The copy cannot tell which budget picked its start, so it fills what this one reaches, + // through its subscription's ranges rather than a stream of its own. const wide = peer.remote.track("video").subscribe(WIDE); expect(sorted(await take(wide, 5))).toEqual(ALL); - expect(peer.fetches()).toBeGreaterThan(before); + expect(peer.fetches()).toBe(before); narrow.close(); wide.close(); } finally { @@ -370,10 +372,10 @@ test("copy fill: a filled group nobody holds any more stops downloading", async } }); -// Lite-07: a filled group whose stream the publisher yielded, then gave up, ends with the fill's -// drop naming it with the group's own error, as one on a subscription does, rather than waiting -// paused on a continuation that never comes. -test("copy fill: a yielded group the fill's drop names ends", async () => { +// Lite-07: a filled group whose stream the publisher yielded, then gave up, ends with the drop the +// copy's subscription, which the fill rides, names it with: the group's own error, rather than +// waiting paused on a continuation that never comes. +test("copy fill: a yielded group its subscription's drop names ends", async () => { const clock = new Manual(); const installed = clock.install(); const pair = createMockTransportPair(Lite.ALPN_07_WIP); @@ -384,8 +386,9 @@ test("copy fill: a yielded group the fill's drop names ends", async () => { ]); const publication = origin.route(Path.from("room")); const track = publication.broadcast.createTrack("video", { timescale: Timescale.MILLI, maxAge: null }); - write(track.createGroup(1n), 1n, "live"); - // The archive answers group 0 with one frame and keeps it open, so its stream is still open. + write(track.createGroup(2n), 2n, "live"); + // The archive answers group 0 with one frame and keeps it open, so its stream is still open, and + // group 1 whole. const requests = track.requests(); const fills: Fill[] = []; const archived = (async () => { @@ -394,32 +397,36 @@ test("copy fill: a yielded group the fill's drop names ends", async () => { fills.push(fill); const group = fill.accept(0n); group.writeFrame({ payload: enc.encode("old 0"), timestamp: Timestamp.fromMillis(0) }); + write(fill.accept(1n), 1n, "old"); return group; })(); const remote = wireOf(client).consume(Path.from("room")); try { const reader = remote.track("video").subscribe({ ranges: [{ start: { relative: 0n } }] }); const live = await reader.ordered().nextGroup(); - expect(live?.sequence).toBe(1n); - expect(dec.decode((await live?.readFrame())?.payload)).toBe("live 1"); + expect(live?.sequence).toBe(2n); + expect(dec.decode((await live?.readFrame())?.payload)).toBe("live 2"); expect(await live?.readFrame()).toBeUndefined(); live?.close(); + // Two groups, so the fill rides the subscription's ranges. const history = remote .track("video") - .subscribe({ ...WIDE, ranges: [{ start: { absolute: 0n }, end: { absolute: 1n } }] }); + .subscribe({ ...WIDE, ranges: [{ start: { absolute: 0n }, end: { absolute: 2n } }] }); const group = await history.recvGroup(); expect(group?.sequence).toBe(0n); expect(dec.decode((await group?.readFrame())?.payload)).toBe("old 0"); const source = await archived; // The session resets the group's stream to make room, and the group is gone before a new - // stream could continue it, so the publisher drops it on the fill's subscription. - // The live stream has finished. The archive may be served on both the fill and - // the copy's expanded feed, so reset every stream still serving it. + // stream could continue it, so the publisher drops it on the subscription the fill rides. + const one = await history.recvGroup(); + expect(one?.sequence).toBe(1n); + while (await one?.readFrame()); + // The live stream has finished, so the archived group's is the only one still open. await clock.advance(Duration.ZERO); const open = lines(pair.server).streams.list(); - expect(open.length).toBeGreaterThan(0); - for (const stream of open) stream.preempt(); + expect(open.length).toBe(1); + open[0]?.preempt(); source.abort(new StreamError(StreamCode(70), { message: "gone" })); // Still paused without it, which the deadline, fired once every pending task settled, diff --git a/js/net/src/lite/drop_record.test.ts b/js/net/src/lite/drop_record.test.ts index cdd307ef151..f549e785abd 100644 --- a/js/net/src/lite/drop_record.test.ts +++ b/js/net/src/lite/drop_record.test.ts @@ -613,7 +613,8 @@ for (const stale of [false, true]) { // Groups 1 and 2 are judged stale for the relay's subscription, under a 1.5s budget, a run the // relay's copy shares with all its readers, then the relay `lost` that run: it aged out past the // pool's idle window, or the pool shed it under pressure. The readers in `readers` subscribe next: -// a narrow one never takes groups 1 and 2, and a wide one is filled them through the relay from +// a narrow one is judged stale for groups 1 and 2 while the viewer still holds the run, and takes +// them once every hop forgot it, and a wide one is filled them through the relay from // the publisher, which still holds them: fetched once the viewer's copy forgot the run too, and // re-offered over both subscriptions by the widening update while it still holds it stale. async function lostRun(lost: "forgot" | "shed", readers: ("wide" | "narrow")[]): Promise { @@ -645,30 +646,52 @@ async function lostRun(lost: "forgot" | "shed", readers: ("wide" | "narrow")[]): if (lost === "shed") relay.pool.resize(); const before = chained.fetches(); const heard = wire.heard.length; + let received: bigint[] = []; for (const name of readers) { if (name === "wide") { const wide = chained.remote.track("video").subscribe(WIDE); const filled = await drain(clock, wide); expect(filled.filter((row) => row[0] !== "group")).toEqual([]); - expect(groupsOf(filled)).toEqual(expect.arrayContaining([1n, 2n])); + received = groupsOf(filled) as bigint[]; + expect(received).toEqual(expect.arrayContaining([1n, 2n])); if (lost === "forgot") expect(chained.fetches().up).toBeGreaterThan(before.up); wide.close(); } else { - // Judged stale by its own budget, or filled and then judged. + // Judged stale by its own budget while the viewer still holds the run. Once every hop forgot + // it, the relay recovers it with no max delay, as the Rust `Recovery` does, so the narrow + // reader takes the stale history: no hop still holds a newer group to judge it by. const narrow = chained.remote.track("video").subscribe({ delay: { max: Duration.millis(1500) }, ranges: [{ start: { absolute: 1n } }], }); - const judged = await drain(clock, narrow); - expect(groupsOf(judged)).not.toContain(1n); - expect(groupsOf(judged)).not.toContain(2n); + const judged = groupsOf(await drain(clock, narrow)); + if (lost === "shed") { + expect(judged).not.toContain(1n); + expect(judged).not.toContain(2n); + } else expect(judged).toEqual([1n, 2n, 3n, 4n, 5n]); narrow.close(); } } - // Nothing a narrow budget gave up is named dropped on the wire, where it would settle every - // reader, wide ones too. - if (readers.includes("wide")) expect(wire.heard.slice(heard)).toEqual([]); + // Nothing a narrow budget gave up is named dropped for good on the wire, where it would settle + // every reader, wide ones too: a stale drop is reopened by the next update. The update asking + // for a recovered run may re-offer a group its recovery already delivered, whose duplicate + // stream the relay resets, which settles no reader. + if (readers.includes("wide")) { + const final = wire.heard + .slice(heard) + .filter((row) => row.kind !== "drop" || row.error !== StreamCode.Old) + .filter( + (row) => + !( + row.kind === "drop" && + row.error !== StreamCode.NotFound && + row.end === row.start + 1n && + received.includes(row.start) + ), + ); + expect(final).toEqual([]); + } expect(chained.ended()).toBe(false); reader.close(); diff --git a/js/net/src/lite/fills.test.ts b/js/net/src/lite/fills.test.ts new file mode 100644 index 00000000000..1d701a48e68 --- /dev/null +++ b/js/net/src/lite/fills.test.ts @@ -0,0 +1,538 @@ +// The rules a lite-07 copy's `RangeFills` settles its fills by. Mirrors the tests in the Rust +// `lite/subscriber/fills.rs`. +import { expect, spyOn, test } from "bun:test"; +import { yieldNow } from "@moq/clock"; +import { Signal } from "@moq/signals"; +import { Producer as BroadcastProducer } from "../broadcast.ts"; +import { StreamCode, StreamError } from "../error.ts"; +import { hooks, type Run } from "../internal.ts"; +import { Tail } from "../tail.ts"; +import type { Fill, Producer } from "../track.ts"; +import { dropped, passing, RangeFills, type RecoveryHandle, type Refused } from "./fills.ts"; + +const enc = new TextEncoder(); + +// A copy whose first group is 4, read from group 0 with no budget, and the fill for 0 to 4 its +// reader asks of it. +async function missing(): Promise<{ broadcast: BroadcastProducer; track: Producer; fill: Fill; close(): void }> { + const broadcast = new BroadcastProducer(); + const track = broadcast.createTrack("video", { maxAge: null }); + const group = track.createGroup(4n); + group.writeFrame({ payload: enc.encode("live 4") }); + group.finish(); + const requests = track.requests(); + const reader = track.subscribe({ ranges: [{ start: { absolute: 0n } }], delay: { max: null }, order: "ascending" }); + // Reading is what asks. + void reader + .ordered() + .nextGroup() + .catch(() => undefined); + const fill = await requests.requestedFill(); + if (!fill) throw new Error("no fill"); + expect([fill.start, fill.end]).toEqual([0n, 4n]); + return { + broadcast, + track, + fill, + close() { + fill.close(); + reader.close(); + requests.close(); + broadcast.close(); + }, + }; +} + +// Recoveries a test answers by hand. +function recoveries() { + const opened: { + groups: Run; + priority: number; + finish(refused: Refused[]): void; + refuse(refused: Refused): void; + cancelled: boolean; + reading: boolean; + readingChanged: Signal; + }[] = []; + const recover = (groups: Run, priority: number): RecoveryHandle => { + let finish!: (refused: Refused[]) => void; + const done = new Promise((resolve) => { + finish = resolve; + }); + const soFar: Refused[] = []; + const changed = new Signal(0); + const refuse = (refused: Refused) => { + soFar.push(refused); + changed.update((revision) => revision + 1); + }; + const entry = { + groups: { ...groups }, + priority, + finish, + refuse, + cancelled: false, + reading: false, + readingChanged: new Signal(0), + }; + opened.push(entry); + return { + done, + takeRefused: () => soFar.splice(0), + changed, + readingChanged: entry.readingChanged, + reading: () => entry.reading, + cancel: () => { + entry.cancelled = true; + finish([]); + }, + }; + }; + return { opened, recover }; +} + +test("range fills: a fill of more than one group joins the subscription's ranges", async () => { + const peer = await missing(); + const { opened, recover } = recoveries(); + const fills = new RangeFills(peer.track, recover); + fills.push(peer.fill); + expect(fills.poll(new Tail(), true)).toBe(true); + expect(fills.ask.groups).toEqual([{ start: 0n, end: 4n }]); + // Asked through the subscription alone, and nothing changed since. + expect(opened).toEqual([]); + expect(fills.poll(new Tail(), true)).toBe(false); + + fills.close(); + expect(fills.ask.groups).toEqual([]); + peer.close(); +}); + +test("range fills: a stale drop of asked groups is recovered once, on a subscription of its own", async () => { + const peer = await missing(); + const { opened, recover } = recoveries(); + const fills = new RangeFills(peer.track, recover); + const tail = new Tail(); + fills.push(peer.fill); + fills.poll(tail, true); + + fills.refuse(1n, 3n, new StreamError(StreamCode.Old)); + fills.poll(tail, true); + expect(opened.map((entry) => entry.groups)).toEqual([{ start: 1n, end: 3n }]); + // With the fill's priority. The run already asked stays asked whole while the fill owes any of + // it, so the subscription does not change under the recovery. + expect(opened[0]?.priority).toBe(peer.fill.priority); + expect(fills.ask.groups).toEqual([{ start: 0n, end: 4n }]); + + // The recovery delivered nothing, and the next stale drop of the same groups stands. + opened[0]?.finish([]); + await yieldNow(); + fills.poll(tail, true); + fills.refuse(1n, 3n, new StreamError(StreamCode.Old)); + fills.poll(tail, true); + expect(opened.length).toBe(1); + expect(peer.fill.wanted()).toBe(0n); + peer.fill.accept(0n).finish(); + fills.poll(tail, true); + // Refused for this fill, which moves on past them. + expect(peer.fill.wanted()).toBe(3n); + + fills.close(); + peer.close(); +}); + +// The subscription holds its FIN while the fills are not empty, so a recovery ending after the +// ask emptied still reports the change. +test("range fills: the last recovery finishing reports the fills empty", async () => { + const peer = await missing(); + const { opened, recover } = recoveries(); + const fills = new RangeFills(peer.track, recover); + const tail = new Tail(); + fills.push(peer.fill); + fills.poll(tail, true); + fills.refuse(1n, 3n, new StreamError(StreamCode.Old)); + fills.poll(tail, true); + const recovery = opened[0]; + if (!recovery) throw new Error("no recovery"); + + // Every group arrives while a stream of the recovery is still read, which keeps it. + recovery.reading = true; + for (let sequence = peer.fill.wanted(); sequence !== undefined; sequence = peer.fill.wanted()) + peer.fill.accept(sequence).finish(); + expect(fills.poll(tail, true)).toBe(true); + expect(fills.ask.groups).toEqual([]); + expect(fills.isEmpty()).toBe(false); + + // The ask stays empty as the recovery ends. + recovery.reading = false; + recovery.finish([]); + await yieldNow(); + expect(fills.poll(tail, true)).toBe(true); + expect(fills.isEmpty()).toBe(true); + expect(fills.poll(tail, true)).toBe(false); + + fills.close(); + peer.close(); +}); + +// A recovery that ends before the fill reaches its groups still answers them once it does, with +// the publisher's own verdict: a group it never had is recorded, so no reader asks for it again. +test("range fills: a recovery's refusals stand until the fill reaches them", async () => { + const peer = await missing(); + const { opened, recover } = recoveries(); + const fills = new RangeFills(peer.track, recover); + const tail = new Tail(); + fills.push(peer.fill); + fills.poll(tail, true); + tail.account(1n, 3n); + fills.refuse(1n, 3n, new StreamError(StreamCode.Old)); + fills.poll(tail, true); + opened[0]?.finish([{ start: 1n, end: 3n, refusal: dropped(new StreamError(StreamCode.NotFound)) }]); + await yieldNow(); + fills.poll(tail, true); + // The fill still waits on group 0. + expect(peer.fill.wanted()).toBe(0n); + + peer.fill.accept(0n).finish(); + fills.poll(tail, true); + expect(peer.fill.wanted()).toBe(3n); + // Recorded as lost: a later reader is told so rather than waiting on them. + const later = peer.track.subscribe({ ranges: [{ start: { absolute: 1n } }], delay: { max: null } }); + const next = hooks.tryRecvGroup(later, true); + expect(next.kind === "drop" ? [next.start, next.end] : next.kind).toEqual([1n, 3n]); + + later.close(); + fills.close(); + peer.close(); +}); + +// The serve loop waits on the fills once per tail change, so a wait that loses keeps nothing +// registered: a stalled fill under a busy live edge would otherwise grow without bound. +test("range fills: a lost wait releases what it listened to", async () => { + const peer = await missing(); + const { recover } = recoveries(); + const fills = new RangeFills(peer.track, recover); + const tail = new Tail(); + fills.push(peer.fill); + fills.poll(tail, true); + fills.refuse(1n, 3n, new StreamError(StreamCode.Old)); + fills.poll(tail, true); + + // Count the change listeners every signal still holds. + const original = Signal.prototype.changed; + const awaited = original as (this: Signal) => Promise; + const listened = original as (this: Signal, fn: (value: unknown) => void) => () => void; + let live = 0; + const changed = spyOn(Signal.prototype, "changed").mockImplementation(function ( + this: Signal, + fn?: (value: unknown) => void, + ) { + live++; + if (!fn) return awaited.call(this).finally(() => live--); + let held = true; + const release = () => { + if (held) live--; + held = false; + }; + const dispose = listened.call(this, (value: unknown) => { + release(); + fn(value); + }); + return () => { + release(); + dispose(); + }; + } as typeof original); + try { + // Woken by the tail each time, as live groups arriving wake the serve loop. + for (let sequence = 10n; sequence < 110n; sequence++) { + const wait = fills.changed(tail); + tail.account(sequence, sequence + 1n); + await wait; + } + expect(live).toBe(0); + } finally { + changed.mockRestore(); + } + + fills.close(); + peer.close(); +}); + +test("range fills: a range start past the asked groups refuses only the fills owing them now", async () => { + const peer = await missing(); + const { opened, recover } = recoveries(); + const fills = new RangeFills(peer.track, recover); + const tail = new Tail(); + fills.push(peer.fill); + fills.poll(tail, true); + + // The range for 0 to 4 started at 2. + fills.skip({ start: 0n, end: 4n }, 0n, 2n); + fills.poll(tail, true); + expect(peer.fill.wanted()).toBe(2n); + expect(opened).toEqual([]); + + // Nothing recorded: the track still holds no verdict on groups 0 and 1, so a later reader asks + // for them again. + const requests = peer.track.requests(); + const later = peer.track.subscribe({ + ranges: [{ start: { absolute: 0n } }], + delay: { max: null }, + order: "ascending", + }); + void later + .ordered() + .nextGroup() + .catch(() => undefined); + const again = await requests.requestedFill(); + expect(again?.wanted()).toBe(0n); + + again?.close(); + later.close(); + requests.close(); + fills.close(); + peer.close(); +}); + +// A recovery opened on the first poll that holds its fill asks with that fill's priority, not the +// empty ask's lowest. +test("range fills: a recovery the first poll opens asks with the fill's priority", async () => { + const peer = await missing(); + const { opened, recover } = recoveries(); + const fills = new RangeFills(peer.track, recover); + const tail = new Tail(); + // The subscription accounted for groups 1 and 2 before the fill arrived. + tail.account(1n, 3n); + fills.push(peer.fill); + fills.poll(tail, true); + expect(opened.map((entry) => entry.groups)).toEqual([{ start: 1n, end: 3n }]); + expect(opened[0]?.priority).toBe(peer.fill.priority); + + fills.close(); + peer.close(); +}); + +// A refusal the source did not name lost answers only the fills owing it then, so a later fill +// for the same groups asks again rather than being refused by it. +test("range fills: a passing refusal ends with the fills it answered", async () => { + const peer = await missing(); + const { opened, recover } = recoveries(); + const fills = new RangeFills(peer.track, recover); + const tail = new Tail(); + fills.push(peer.fill); + fills.poll(tail, true); + fills.refuse(1n, 3n, new StreamError(StreamCode.Old)); + fills.poll(tail, true); + opened[0]?.finish([{ start: 1n, end: 3n, refusal: passing(new StreamError(StreamCode.Cancel)) }]); + await yieldNow(); + peer.fill.accept(0n).finish(); + fills.poll(tail, true); + expect(peer.fill.wanted()).toBe(3n); + peer.fill.close(); + fills.poll(tail, true); + + const requests = peer.track.requests(); + const later = peer.track.subscribe({ + ranges: [{ start: { absolute: 1n } }], + delay: { max: null }, + order: "ascending", + }); + void later + .ordered() + .nextGroup() + .catch(() => undefined); + const again = await requests.requestedFill(); + if (!again) throw new Error("no fill"); + fills.push(again); + fills.poll(tail, true); + expect(again.wanted()).toBe(1n); + + again.close(); + later.close(); + requests.close(); + fills.close(); + peer.close(); +}); + +// A recovery still out answers the fills owing what it already refused, rather than holding them +// until its other streams finish. +test("range fills: a recovery's refusal answers before it is over", async () => { + const peer = await missing(); + const { opened, recover } = recoveries(); + const fills = new RangeFills(peer.track, recover); + const tail = new Tail(); + fills.push(peer.fill); + fills.poll(tail, true); + fills.refuse(1n, 3n, new StreamError(StreamCode.Old)); + fills.poll(tail, true); + peer.fill.accept(0n).finish(); + fills.poll(tail, true); + expect(peer.fill.wanted()).toBe(1n); + + // Group 1 is refused while group 2's stream is still out. + const woken = fills.changed(tail); + opened[0]?.refuse({ start: 1n, end: 2n, refusal: dropped(new StreamError(StreamCode.NotFound)) }); + await woken; + fills.poll(tail, true); + expect(peer.fill.wanted()).toBe(2n); + + fills.close(); + peer.close(); +}); + +// A recovery still out hands each refusal over once: after the fills owing it end, a later fill for +// the same groups asks again rather than reading it back while the recovery reads another stream. +test("range fills: a recovery's refusal is taken once", async () => { + const peer = await missing(); + const { opened, recover } = recoveries(); + const fills = new RangeFills(peer.track, recover); + const tail = new Tail(); + fills.push(peer.fill); + fills.poll(tail, true); + fills.refuse(1n, 3n, new StreamError(StreamCode.Old)); + fills.poll(tail, true); + const recovery = opened[0]; + if (!recovery) throw new Error("no recovery"); + // Still reading another of its streams, so it stays out with no fill owing it. + recovery.reading = true; + peer.fill.accept(0n).finish(); + recovery.refuse({ start: 1n, end: 2n, refusal: passing(new StreamError(StreamCode.Cancel)) }); + fills.poll(tail, true); + expect(peer.fill.wanted()).toBe(2n); + peer.fill.close(); + fills.poll(tail, true); + expect(fills.recovering).toBe(true); + + const requests = peer.track.requests(); + const later = peer.track.subscribe({ + ranges: [{ start: { absolute: 1n } }], + delay: { max: null }, + order: "ascending", + }); + void later + .ordered() + .nextGroup() + .catch(() => undefined); + const again = await requests.requestedFill(); + if (!again) throw new Error("no fill"); + fills.push(again); + fills.poll(tail, true); + expect(again.wanted()).toBe(1n); + // Asked again, rather than left to the recovery's other streams. + expect(fills.ask.groups.some((run) => run.start <= 1n && 1n < run.end)).toBe(true); + + again.close(); + later.close(); + requests.close(); + fills.close(); + peer.close(); +}); + +// A recovery ending leaves a later one's coverage of groups it released intact, so those groups +// stay off the copy's subscription while the later one asks for them. +test("range fills: an older recovery ending keeps a newer one's coverage", async () => { + const peer = await missing(); + const { opened, recover } = recoveries(); + const fills = new RangeFills(peer.track, recover); + const tail = new Tail(); + fills.push(peer.fill); + fills.poll(tail, true); + fills.refuse(1n, 3n, new StreamError(StreamCode.Old)); + fills.poll(tail, true); + const older = opened[0]; + if (!older) throw new Error("no recovery"); + // It refuses group 1 and keeps reading group 2, past the fill. + older.reading = true; + peer.fill.accept(0n).finish(); + older.refuse({ start: 1n, end: 2n, refusal: passing(new StreamError(StreamCode.Cancel)) }); + fills.poll(tail, true); + peer.fill.close(); + fills.poll(tail, true); + + // A later fill finds group 1 accounted for on the subscription, so a newer recovery asks for it. + tail.account(1n, 2n); + const requests = peer.track.requests(); + const later = peer.track.subscribe({ + ranges: [{ start: { absolute: 1n } }], + delay: { max: null }, + order: "ascending", + }); + void later + .ordered() + .nextGroup() + .catch(() => undefined); + const again = await requests.requestedFill(); + if (!again) throw new Error("no fill"); + fills.push(again); + fills.poll(tail, true); + expect(opened.map((entry) => entry.groups)).toEqual([ + { start: 1n, end: 3n }, + { start: 1n, end: 2n }, + ]); + + // The older one ends first. + older.reading = false; + older.finish([]); + await yieldNow(); + fills.poll(tail, true); + expect(fills.ask.groups.some((run) => run.start <= 1n && 1n < run.end)).toBe(false); + + again.close(); + later.close(); + requests.close(); + fills.close(); + peer.close(); +}); + +// The subscription repeating a stale drop, under a later update, of groups a recovery still asks +// for refuses nothing: the fill waits on the recovery's answer. +test("range fills: a stale drop repeated during its recovery waits on it", async () => { + const peer = await missing(); + const { recover } = recoveries(); + const fills = new RangeFills(peer.track, recover); + const tail = new Tail(); + fills.push(peer.fill); + fills.poll(tail, true); + fills.refuse(1n, 3n, new StreamError(StreamCode.Old)); + fills.poll(tail, true); + fills.refuse(1n, 3n, new StreamError(StreamCode.Old)); + fills.poll(tail, true); + peer.fill.accept(0n).finish(); + fills.poll(tail, true); + expect(peer.fill.wanted()).toBe(1n); + + fills.close(); + peer.close(); +}); + +// A recovery kept only for a stream still being read wakes the fills once that stream ends, so +// they cancel it rather than sleep on it. +test("range fills: a recovery's stream ending wakes the fills", async () => { + const peer = await missing(); + const { opened, recover } = recoveries(); + const fills = new RangeFills(peer.track, recover); + const tail = new Tail(); + fills.push(peer.fill); + fills.poll(tail, true); + fills.refuse(1n, 3n, new StreamError(StreamCode.Old)); + fills.poll(tail, true); + const recovery = opened[0]; + if (!recovery) throw new Error("no recovery"); + recovery.reading = true; + peer.fill.close(); + fills.poll(tail, true); + expect(recovery.cancelled).toBe(false); + + const wait = fills.changed(tail).then(() => true); + recovery.reading = false; + recovery.readingChanged.update((revision) => revision + 1); + const idle = (async () => { + for (let i = 0; i < 10; i++) await yieldNow(); + return false; + })(); + expect(await Promise.race([wait, idle])).toBe(true); + fills.poll(tail, true); + expect(recovery.cancelled).toBe(true); + + fills.close(); + peer.close(); +}); diff --git a/js/net/src/lite/fills.ts b/js/net/src/lite/fills.ts new file mode 100644 index 00000000000..fe05f6d1a43 --- /dev/null +++ b/js/net/src/lite/fills.ts @@ -0,0 +1,443 @@ +// Fills a lite-07 copy asks of its upstream through a subscription rather than FETCH. Mirrors the +// Rust `lite/subscriber/fills.rs`. + +import { type GetPromise, type Getter, race, Signal } from "@moq/signals"; +import { NotFound, StreamCode, toStreamCode } from "../error.ts"; +import { fillChanged, wantedRuns } from "../fill.ts"; +import { hooks, insertRun, type Run, removeRun, runGaps } from "../internal.ts"; +import type { Tail } from "../tail.ts"; +import type * as track from "../track.ts"; +import { max, min } from "../util/sequence.ts"; + +/** The lowest priority, which any fill's outranks. */ +const NO_PRIORITY = -128; + +/** + * Why the subscription that asked for groups will not deliver them. + * + * `lost` when the source no longer has them, which the track records so no reader asks again: + * only a genuine miss its publisher named. Anything else answers only the fills owing them now, + * since another request may still get them: a stale drop judges a budget, and a range starting + * past them is the subscriber's own choice. + */ +export interface Refusal { + error: Error; + lost: boolean; +} + +/** Groups a subscription refused, and why. */ +export interface Refused extends Run { + refusal: Refusal; +} + +/** The publisher gave the groups up with `error`: lost only when it never had them. */ +export function dropped(error: Error): Refusal { + return { error, lost: toStreamCode(error) === StreamCode.NotFound }; +} + +/** Not delivered on this subscription, which says nothing of the source. */ +export function passing(error: Error): Refusal { + return { error, lost: false }; +} + +/** `runs` with `[start, end)` refused by `refusal` instead, so a repeated refusal costs nothing. */ +export function refuseRun(runs: readonly Refused[], start: bigint, end: bigint, refusal: Refusal): Refused[] { + if (start >= end) return [...runs]; + return [...removeRun(runs, start, end), { start, end, refusal }].sort(byStart); +} + +/** + * A subscription asking again, with no max delay, for groups another one dropped as stale or + * already delivered, which a fill still owes: what the subscriber opens for a {@link RangeFills}. + */ +export interface RecoveryHandle { + /** Resolves once it is over, with the groups it refused. */ + done: Promise; + /** + * The groups it refused since the last call, each handed over once, which answer the fills + * owing them before it is over. Its {@link done} resolves with only those never taken. + */ + takeRefused(): Refused[]; + /** Bumped each time it refuses more. */ + readonly changed: Getter; + /** Bumped as its streams arrive and finish, which {@link reading} reports. */ + readonly readingChanged: Getter; + /** Whether a stream of it is still being read, which cancelling would cut. */ + reading(): boolean; + /** Stop asking: what is left is refused. */ + cancel(): void; +} + +class Recovery { + readonly groups: Run; + readonly handle: RecoveryHandle; + refused?: Refused[]; + // The groups it refused and handed over, which it no longer covers, so a later request for + // them asks again rather than waiting on its other streams. + readonly released: Run[] = []; + + // `ended` is called once it is over, so a wait need not hold a reaction on `done`. + constructor(groups: Run, handle: RecoveryHandle, ended: () => void) { + this.groups = groups; + this.handle = handle; + void handle.done.then((refused) => { + this.refused = refused; + ended(); + }); + } +} + +/** What a lite-07 copy's held fills ask of its upstream subscription, beside the demand. */ +export interface FillAsk { + /** The groups to ask for, empty when no fill is held. */ + groups: Run[]; + /** The most urgent priority a held fill asks with, which the subscription takes if higher. */ + priority: number; +} + +/** + * Fills a lite-07 upstream serves through the copy's subscription rather than FETCH: each held + * fill's next owed run joins the subscription's ranges until its groups arrive or the publisher + * names them given up, and the fill answers itself from the cache. + * + * A fill is judged against the subscription that asked for it alone. The track's arrivals still + * account for a group an earlier subscription cut, which this one may still deliver, so only what + * this subscription says it will not deliver (a SUBSCRIBE_DROP, or a range start past the range) + * refuses a fill. A range start speaks only for its range: once the range retires, a later one + * asking for the same groups may start lower. A group it drops as stale is officially dropped, + * and one it already accounted for (delivered before and since evicted) it never sends again, so + * a recovery asks for either on its own subscription with no max delay, once. Nothing newer + * judges it stale there, so once every hop forgot a run, a narrow reader takes its stale groups. + * + * A group stays asked until its stream finishes, since narrowing the subscription under a stream + * still arriving would reset it. + */ +export class RangeFills { + readonly #copy: track.Producer; + readonly #recover: (groups: Run, priority: number) => RecoveryHandle; + #held: track.Fill[] = []; + /** What the subscription asks for the fills now. */ + ask: FillAsk = { groups: [], priority: NO_PRIORITY }; + // Bumped on every input `poll` reads that no other signal reports. + readonly #bumped = new Signal(0); + // Asked groups no fill owes any more whose stream is still arriving, and how each ends. + #reading = new Map>(); + // Asked groups no fill owes any more, already searched for streams still arriving. + #draining: Run[] = []; + // Groups the subscription said it will not deliver, and why. + #refused: Refused[] = []; + // Groups below where a fill's range starts, with the asked run that range carries, which stand + // only while that range is live. + #skipped: (Run & { asked: Run })[] = []; + // Groups dropped as stale, which a recovery has yet to ask for. + #toRecover: Run[] = []; + // Groups a recovery asks for now, which the subscription leaves out meanwhile. + #recovering: Run[] = []; + // Groups recovered once already, which a second stale drop refuses. + #recovered: Run[] = []; + #recoveries = new Set(); + #closed = false; + + constructor(copy: track.Producer, recover: (groups: Run, priority: number) => RecoveryHandle) { + this.#copy = copy; + this.#recover = recover; + } + + /** Whether no fill is held, asked for, or recovering. */ + isEmpty(): boolean { + return this.#held.length === 0 && this.ask.groups.length === 0 && this.#recoveries.size === 0; + } + + /** Whether a recovery is still out. */ + get recovering(): boolean { + return this.#recoveries.size > 0; + } + + push(fill: track.Fill): void { + this.#held.push(fill); + this.#bump(); + } + + /** + * The subscription will not deliver `[start, end)`: a fill owing one is refused with `error`, + * or recovered once when it was dropped as stale. + */ + refuse(start: bigint, end: bigint, error: Error): void { + // Only what a fill asked for: a fill asks for nothing the subscription dropped before it. + for (const run of clip(this.ask.groups, start, end)) { + if (toStreamCode(error) === StreamCode.Old) { + const done = clip(this.#recovered, run.start, run.end); + for (const fresh of runGaps(done, run.start, run.end)) + insertRun(this.#toRecover, fresh.start, fresh.end); + // Recovered once already: this drop stands for the fills owing it now, except where a + // recovery still out asks for it, whose answer it waits on. + for (const again of done) + for (const stands of runGaps(this.#recovering, again.start, again.end)) + this.#setRefused(stands.start, stands.end, dropped(error)); + continue; + } + this.#setRefused(run.start, run.end, dropped(error)); + } + this.#bump(); + } + + /** + * The subscription's range for the asked run `asked` starts past `[start, end)`, which it will + * not deliver while that range is live. + */ + skip(asked: Run, start: bigint, end: bigint): void { + if (start >= end) return; + this.#skipped = [...removeRun(this.#skipped, start, end), { start, end, asked: { ...asked } }].sort(byStart); + this.#bump(); + } + + /** + * Settle each held fill against what the subscription refused, open a recovery for each run it + * will not deliver, and recompute what to ask. `tail` is the subscription's, and `read` whether + * anyone still reads the track. Returns whether the ask changed, or whether it {@link isEmpty}: + * a subscription holding its FIN for the fills waits on either. + */ + poll(tail: Tail, read: boolean): boolean { + const empty = this.isEmpty(); + // The groups the subscription already accounted for, which it never sends again, are + // recovered on their own, once. + const asked: Run[] = []; + for (const fill of this.#held) { + const run = nextRun(fill); + if (run) insertRun(asked, run.start, run.end); + } + for (const run of asked) { + for (const gone of tail.accounted(run.start, run.end)) { + for (const gap of runGaps(this.#recovering, gone.start, gone.end)) { + const done = clip(this.#recovered, gap.start, gap.end); + for (const again of done) { + // Recovered once for these fills; a later request may recover it again. + if (!this.#refusal(again.start)) + this.#setRefused(again.start, again.end, passing(new NotFound(`group ${again.start}`))); + } + for (const fresh of runGaps(done, gap.start, gap.end)) { + if (!this.#refusal(fresh.start)) insertRun(this.#toRecover, fresh.start, fresh.end); + } + } + } + } + // Each stale run goes out on its own subscription with no max delay, once, with the priority + // of the fills held now: the last ask may predate them. + let priority = this.ask.priority; + for (const fill of this.#held) priority = Math.max(priority, fill.priority); + for (const run of this.#toRecover.splice(0)) { + insertRun(this.#recovered, run.start, run.end); + insertRun(this.#recovering, run.start, run.end); + this.#recoveries.add(new Recovery(run, this.#recover(run, priority), () => this.#bump())); + } + // What a finished recovery did not deliver stands refused. + for (const done of this.#recoveries) { + if (!done.refused) continue; + this.#recoveries.delete(done); + for (const refused of done.refused) this.#setRefused(refused.start, refused.end, refused.refusal); + } + // What a recovery still out refused since the last poll answers the fills owing it now, + // rather than waiting on its other streams. Taken once, so once those fills end, a later + // request for the same groups asks again rather than reading it back. + for (const recovery of this.#recoveries) { + for (const refused of recovery.handle.takeRefused()) { + this.#setRefused(refused.start, refused.end, refused.refusal); + insertRun(recovery.released, refused.start, refused.end); + } + } + // Rebuilt from those still out, so one ending leaves a later one's coverage of the same + // groups intact. + this.#cover(); + + const ask: FillAsk = { groups: [], priority: NO_PRIORITY }; + // Every run a held fill still owes, recovered or not. + const owed: Run[] = []; + const held: track.Fill[] = []; + // Not asked of the subscription: a run being recovered waits on its own, and a group already + // refused is answered once its fill reaches it. + const waiting = [...this.#recovering]; + for (const refused of this.#refused) insertRun(waiting, refused.start, refused.end); + for (const fill of this.#held) { + for (;;) { + const run = nextRun(fill); + if (!run) { + fill.close(); + break; + } + // The run extends from the group owed first, which the subscription may have refused + // already. Refusing the run answers it, so the loop moves on. + const first = fill.order === "ascending" ? run.start : run.end - 1n; + const refused = this.#refusal(first); + if (refused) { + answer(refused.refusal, fill, max(run.start, refused.start), min(run.end, refused.end)); + continue; + } + insertRun(owed, run.start, run.end); + for (const gap of runGaps(waiting, run.start, run.end)) insertRun(ask.groups, gap.start, gap.end); + ask.priority = Math.max(ask.priority, fill.priority); + held.push(fill); + break; + } + } + this.#held = held; + + // A refusal answers only the fills owing it now: trimmed to the groups a held fill still + // covers, so a later request for the rest asks again, and a loss it named stays recorded on + // the track alone. + const covered: Run[] = []; + for (const fill of held) if (fill.start < fill.end) insertRun(covered, fill.start, fill.end); + this.#refused = this.#refused.flatMap((run) => + clip(covered, run.start, run.end).map((part) => ({ ...part, refusal: run.refusal })), + ); + + // A recovery no held fill owes any group of is cancelled, once none of its streams is still + // being read or nobody reads the track, and a group is recovered once for the fills owing it + // now: a later request for it may recover it again. + for (const recovery of this.#recoveries) { + const { start, end } = recovery.groups; + if (clip(owed, start, end).length > 0 || (read && recovery.handle.reading())) continue; + recovery.handle.cancel(); + this.#recoveries.delete(recovery); + } + this.#cover(); + this.#recovered = this.#recovered.flatMap(({ start, end }) => clip(owed, start, end)); + + // A run stays asked whole while a fill still owes any of it, so the subscription changes once + // as a run is asked and once as it is done, and the streams of its answered groups keep + // arriving. One no fill owes stays while any of its streams is still arriving, each group + // searched for once. + for (const [sequence, closed] of this.#reading) if (closed.peek() !== undefined) this.#reading.delete(sequence); + const groups = ask.groups.map((run) => ({ ...run })); + const draining: Run[] = []; + for (const run of this.ask.groups) { + if (clip(ask.groups, run.start, run.end).length > 0) { + insertRun(groups, run.start, run.end); + continue; + } + // Only the groups the cache holds, so a wide run costs what it holds. + for (const gap of runGaps(this.#draining, run.start, run.end)) { + for (const sequence of hooks.arrivingIn(this.#copy, gap.start, gap.end)) { + // Only watched until it ends, never taken over, so it is not held. + const arriving = hooks.arrivingGroup(this.#copy, sequence); + arriving?.release(); + if (arriving !== undefined) this.#reading.set(sequence, arriving.settled); + } + } + if ([...this.#reading.keys()].some((sequence) => run.start <= sequence && sequence < run.end)) { + insertRun(groups, run.start, run.end); + insertRun(draining, run.start, run.end); + } + } + for (const sequence of this.#reading.keys()) + if (!draining.some((run) => run.start <= sequence && sequence < run.end)) this.#reading.delete(sequence); + this.#draining = draining; + // What finishes a stream still asks with what its fills asked. + if (this.#held.length === 0 && groups.length > 0) ask.priority = this.ask.priority; + ask.groups = groups; + if (ask.priority === this.ask.priority && sameRuns(ask.groups, this.ask.groups)) + return this.isEmpty() !== empty; + this.#retireSkipped(ask.groups); + this.ask = ask; + return true; + } + + /** Resolves once any input {@link poll} reads may have changed. */ + changed(tail: Tail): Promise { + const wakes: unknown[] = [ + Signal.race( + this.#bumped, + tail.changed, + ...[...this.#recoveries].flatMap((recovery) => [ + recovery.handle.changed, + recovery.handle.readingChanged, + ]), + ), + ]; + for (const fill of this.#held) wakes.push(fillChanged(fill)); + for (const closed of this.#reading.values()) wakes.push(closed); + return race(wakes); + } + + /** The copy closed: every held fill names what is left dropped, and every recovery stops. */ + close(): void { + if (this.#closed) return; + this.#closed = true; + for (const fill of this.#held.splice(0)) fill.close(); + for (const recovery of this.#recoveries) recovery.handle.cancel(); + this.#recoveries.clear(); + this.ask = { groups: [], priority: NO_PRIORITY }; + this.#bump(); + } + + /** Whether {@link close} was called. */ + get closed(): boolean { + return this.#closed; + } + + // Why the subscription or a recovery will not deliver the run of groups from `first`, if it + // said so. + #refusal(first: bigint): Refused | undefined { + const refused = this.#refused.find((run) => run.start <= first && first < run.end); + if (refused) return refused; + // The subscriber's own range starting past them says nothing of the source. + const skipped = this.#skipped.find((run) => run.start <= first && first < run.end); + if (!skipped) return undefined; + return { start: skipped.start, end: skipped.end, refusal: passing(new NotFound(`group ${skipped.start}`)) }; + } + + // What the recoveries still out cover: each its groups less what it refused and handed over. + #cover(): void { + this.#recovering = []; + for (const { groups, released } of this.#recoveries) + for (const gap of runGaps(released, groups.start, groups.end)) + insertRun(this.#recovering, gap.start, gap.end); + } + + #setRefused(start: bigint, end: bigint, refusal: Refusal): void { + this.#refused = refuseRun(this.#refused, start, end, refusal); + } + + // Asking for `asked` from now on retires the range of each run not asked whole any more, and + // what its start skipped with it. + #retireSkipped(asked: readonly Run[]): void { + this.#skipped = this.#skipped.filter((skipped) => + asked.some((run) => run.start === skipped.asked.start && run.end === skipped.asked.end), + ); + } + + #bump(): void { + this.#bumped.update((revision) => revision + 1); + } +} + +// The run `fill` owes first in its order, extending from the group it owes first. +function nextRun(fill: track.Fill): Run | undefined { + const owed = wantedRuns(fill); + return fill.order === "ascending" ? owed[0] : owed.at(-1); +} + +// Answer `[start, end)` of `fill`: a lost group is named dropped, as a FETCH's miss is, and +// anything else is refused for now, recorded nowhere. +function answer(refusal: Refusal, fill: track.Fill, start: bigint, end: bigint): void { + if (refusal.lost) fill.dropGroups(start, end); + else fill.refuseGroups(start, end, refusal.error); +} + +// The parts of `runs` inside `[start, end)`. +function clip(runs: readonly Run[], start: bigint, end: bigint): Run[] { + const parts: Run[] = []; + for (const run of runs) { + const from = max(run.start, start); + const to = min(run.end, end); + if (from < to) parts.push({ start: from, end: to }); + } + return parts; +} + +function sameRuns(a: readonly Run[], b: readonly Run[]): boolean { + return a.length === b.length && a.every((run, i) => run.start === b[i].start && run.end === b[i].end); +} + +function byStart(a: Run, b: Run): number { + return a.start < b.start ? -1 : a.start > b.start ? 1 : 0; +} diff --git a/js/net/src/lite/publisher.ts b/js/net/src/lite/publisher.ts index a96ffabfa39..452e557c944 100644 --- a/js/net/src/lite/publisher.ts +++ b/js/net/src/lite/publisher.ts @@ -67,10 +67,9 @@ import { Group as GroupMessage } from "./group.ts"; import { Probe } from "./probe.ts"; import { encodeSubscribeResponse, - entriesOf, hasWireTime, LiveRanges, - type RangeEntry, + rangesOf, requestedRanges, type Subscribe, SubscribeDrop, @@ -82,6 +81,7 @@ import { SubscribeStarted, SubscribeUpdate, spanOf, + type WireRange, } from "./subscribe.ts"; import { TrackInfo as TrackInfoMessage, type Track as TrackMessage } from "./track.ts"; import { encodeTracksEvent, type Tracks as TracksMessage } from "./tracks.ts"; @@ -301,7 +301,7 @@ type SubscriptionControlOptions = { /** The SUBSCRIBE's ranges, which each lite-07 update edits. */ live: LiveRanges; /** Apply an update, with every live range by Range ID on lite-07. */ - apply: (update: SubscribeUpdate, live: { id: bigint; entry: RangeEntry }[]) => void; + apply: (update: SubscribeUpdate, live: { id: bigint; range: WireRange }[]) => void; }; /** @@ -437,20 +437,15 @@ class SubscriptionControls { // nor looks a time start up again in newer holdings. A start reaching back to the oldest group // held stays as it was, since an absolute start would turn it into a floor the subscriber never // named. -function pin(entry: RangeEntry, resolved: Resolved | undefined): RangeEntry { - if (!resolved) return entry; - const range = { ...entry.range }; - let { from, to } = entry; - if (resolved.floored && (range.start?.relative !== undefined || range.start?.time !== undefined)) { +function pin(asked: WireRange, resolved: Resolved | undefined): WireRange { + if (!resolved) return asked; + const range = { ...asked }; + if (resolved.floored && (range.start?.relative !== undefined || range.start?.time !== undefined)) range.start = { absolute: resolved.start }; - from = [resolved.start, 0]; - } // An end resolves once it settles; until then it is reported open. - if ((range.end?.relative !== undefined || range.end?.time !== undefined) && resolved.end !== undefined) { + if ((range.end?.relative !== undefined || range.end?.time !== undefined) && resolved.end !== undefined) range.end = { absolute: resolved.end }; - to = [resolved.end, 0]; - } - return { range, from, to }; + return range; } // The later of two locations, either of which may be unknown. @@ -1251,7 +1246,7 @@ export class Publisher { } // The SUBSCRIBE's ranges, and the Range ID of each range the model holds, in its order. - const asked = entriesOf(msg); + const asked = rangesOf(msg); // Where each Range ID sits among the model's ranges, as of the last update applied. let positions = new Map(asked.map((_, index) => [BigInt(index), index])); controls = new SubscriptionControls({ @@ -1265,11 +1260,11 @@ export class Publisher { if (wireOf(front).localDatagrams) legacy?.setPriority(update.priority); if (hasRanges(this.version)) { const resolved = hooks.servingResolved(track); - const entries = live.map(({ id, entry }) => { + const ranges = live.map(({ id, range }) => { const index = positions.get(id); - return index === undefined ? entry : pin(entry, resolved[index]); + return index === undefined ? range : pin(range, resolved[index]); }); - Object.assign(update, spanOf(this.version, entries)); + Object.assign(update, spanOf(this.version, ranges)); positions = new Map(live.map(({ id }, index) => [id, index])); } // Every update restates the wanted group. Past our largest it is a join, which diff --git a/js/net/src/lite/subscribe.test.ts b/js/net/src/lite/subscribe.test.ts index 57cb6e1a841..678dc05a089 100644 --- a/js/net/src/lite/subscribe.test.ts +++ b/js/net/src/lite/subscribe.test.ts @@ -188,8 +188,11 @@ test("lite-07 carries the span as its only range, exclusive at the end", async ( new Reader(undefined, await encodeMessage(Version.DRAFT_07, msg), Version.DRAFT_07), Version.DRAFT_07, ); - const got = await decode(message); - expect([got.startGroup, got.startFrame, got.endGroup, got.endFrame]).toEqual([3n, 2, 9n, 4]); + // A range covers whole groups: a slice of one is a FETCH. + await expect(encodeMessage(Version.DRAFT_07, message)).rejects.toThrow("frame bounds not supported"); + await expect(encodeMessage(Version.DRAFT_07, new Subscribe({ ...message, startFrame: 0 }))).rejects.toThrow( + "frame bounds not supported", + ); const whole = await decode(new Subscribe({ ...message, startFrame: 0, endFrame: undefined })); expect([whole.startGroup, whole.startFrame, whole.endGroup, whole.endFrame]).toEqual([3n, 0, 9n, undefined]); const open = await decode( @@ -208,20 +211,19 @@ test("a lite-07 list decodes as the span covering it", async () => { new Reader(undefined, new Uint8Array([body.length, ...body]), Version.DRAFT_07), Version.DRAFT_07, ); - return { ...got, ...spanOf(Version.DRAFT_07, got.entries ?? []) }; + return { ...got, ...spanOf(Version.DRAFT_07, got.ranges ?? []) }; }; - // Groups 2 through 4 and 7 through frame 1 of group 9. - const sparse = await update([2, 1, 2, 0, 1, 5, 0, 1, 7, 0, 1, 9, 2]); - expect([sparse.startGroup, sparse.startFrame, sparse.endGroup, sparse.endFrame]).toEqual([2n, 0, 9n, 1]); - // The ranges stay whole groups, so an end past frame 0 includes its group. + // Groups 2 through 4 and 7 through 9: an absolute anchor is its type and group alone. + const sparse = await update([2, 1, 2, 1, 5, 1, 7, 1, 10]); + expect([sparse.startGroup, sparse.startFrame, sparse.endGroup, sparse.endFrame]).toEqual([2n, 0, 9n, undefined]); expect(sparse.ranges).toEqual([ { start: { absolute: 2n }, end: { absolute: 5n } }, { start: { absolute: 7n }, end: { absolute: 10n } }, ]); // The span covering a relative start has no floor; the ranges keep it for the serving reader. - const relative = await update([2, 2, 3, 1, 5, 0, 1, 7, 0, 1, 9, 0]); + const relative = await update([2, 2, 3, 1, 5, 1, 7, 1, 9]); expect([relative.startGroup, relative.endGroup, relative.endFrame]).toEqual([undefined, 8n, undefined]); expect(relative.ranges).toEqual([ { start: { relative: 3n }, end: { absolute: 5n } }, @@ -236,14 +238,14 @@ test("a lite-07 list decodes as the span covering it", async () => { ]); expect(() => requestedRanges(timed, undefined)).toThrow(TimestampMismatch); - // No start reaches back to group 0; one empty partway through group 5 covers no group. - const oldest = await update([2, 0, 1, 3, 0, 1, 5, 2, 1, 5, 2]); + // No start reaches back to group 0; one ending before group 5 starts covers no group. + const oldest = await update([2, 0, 1, 3, 1, 5, 1, 4]); expect(oldest.ranges).toEqual([{ end: { absolute: 3n } }, { start: { absolute: 5n }, end: { absolute: 5n } }]); // An update may add no range, retiring some or none. expect((await update([0])).ranges).toEqual([]); await expect(update([1, 4, 0, 0])).rejects.toThrow(ProtocolViolation); - await expect(update([1, 0, 1, 0, 0])).rejects.toThrow(ProtocolViolation); + await expect(update([1, 0, 1, 0])).rejects.toThrow(ProtocolViolation); }); test("lite-07 carries every range, relative and time anchors included, and older wires the span alone", async () => { @@ -556,13 +558,13 @@ test("lite-07 updates edit only the changed ranges", async () => { const live = new LiveRanges(version, [a, b]); const step = async (ranges: WireRange[], remove: bigint[], add: WireRange[]) => { const edit = sent.edit(ranges); - expect(edit).toEqual({ remove, add }); + expect(edit).toEqual({ remove, add, fillIds: [] }); const update = new SubscribeUpdate({ priority: 0, remove: edit.remove, ranges: edit.add }); const got = await SubscribeUpdate.decode( new Reader(undefined, await encodeMessage(version, update), version), version, ); - const resolved = live.apply(got).map(({ id, entry }) => [id, entry.range]); + const resolved = live.apply(got).map(({ id, range }) => [id, range]); expect(resolved).toEqual(sent.entries()); }; await step([b, a], [], []); diff --git a/js/net/src/lite/subscribe.ts b/js/net/src/lite/subscribe.ts index 0cc7bcb918e..439b1097f0e 100644 --- a/js/net/src/lite/subscribe.ts +++ b/js/net/src/lite/subscribe.ts @@ -184,12 +184,11 @@ export function hasWireTime(ranges: readonly WireRange[] | undefined): boolean { return ranges?.some((range) => range.start?.time !== undefined || range.end?.time !== undefined) ?? false; } -/** Encode one lite-07 range end, `undefined` for none: an absolute end names a group's first frame. */ +/** Encode one lite-07 range end, `undefined` for none: an absolute end stops before its group. */ async function encodeAnchor(w: Writer, anchor: WireAnchor | undefined) { if (anchor?.absolute !== undefined) { await w.u53(1); await w.u62(anchor.absolute); - await w.u53(0); } else if (anchor?.relative !== undefined) { await w.u53(2); await w.u62(anchor.relative); @@ -223,27 +222,15 @@ async function encodeSpan(w: Writer, version: Version, span: Span, ranges?: read return; } - const { startGroup, startFrame, endGroup, endFrame } = span; - if ((startFrame !== 0 && startGroup === undefined) || (endFrame !== undefined && endGroup === undefined)) { - throw new Error("frame bound without a group bound"); - } + // A lite-07 range covers whole groups: a slice of one is a FETCH. + if (span.startFrame !== 0 || span.endFrame !== undefined) + throw new Error("frame bounds not supported for this version"); await w.u53(1); - // Each end is a type, then its fields: 0 is none (the oldest group as a start, the live - // edge as an end) and 1 an absolute position, which an end stops just before. - if (startGroup === undefined) { - await w.u53(0); - } else { - await w.u53(1); - await w.u62(startGroup); - await w.u53(startFrame); - } - if (endGroup === undefined) { - await w.u53(0); - } else { - await w.u53(1); - await w.u62(endFrame === undefined ? endGroup + 1n : endGroup); - await w.u53(endFrame === undefined ? 0 : endFrame + 1); - } + const range: WireRange = {}; + if (span.startGroup !== undefined) range.start = { absolute: span.startGroup }; + if (span.endGroup !== undefined) range.end = { absolute: span.endGroup + 1n }; + await encodeAnchor(w, range.start); + await encodeAnchor(w, range.end); } /** @@ -273,93 +260,70 @@ export function requestedRanges(msg: Span & { ranges?: WireRange[] }, timescale: return [range]; } -/** A position as a comparable pair, group then frame. */ -export type Point = [group: bigint, frame: number]; - -function before(a: Point, b: Point): boolean { - return a[0] < b[0] || (a[0] === b[0] && a[1] < b[1]); -} - -/** - * Decode one lite-07 range end: its absolute position, if any, and the wire anchor, `undefined` - * for none. An absolute end past frame 0 includes its group. - */ -async function decodeAnchor(r: Reader, end: boolean): Promise<{ point?: Point; anchor?: WireAnchor }> { +/** Decode one lite-07 range end, `undefined` for none. */ +async function decodeAnchor(r: Reader): Promise { const kind = await r.u53(); switch (kind) { case 0: - return {}; - case 1: { - const point: Point = [await r.u62(), await r.u53()]; - return { point, anchor: { absolute: end && point[1] > 0 ? point[0] + 1n : point[0] } }; - } + return undefined; + case 1: + return { absolute: await r.u62() }; case 2: - return { anchor: { relative: await r.u62() } }; + return { relative: await r.u62() }; case 3: - return { anchor: { time: await r.u53() } }; + return { time: await r.u53() }; default: throw new ProtocolViolation(`unknown range anchor: ${kind}`); } } -/** - * A decoded lite-07 range, with the positions its absolute anchors name, which a - * {@link WireRange} rounds to whole groups. - */ -export interface RangeEntry { - range: WireRange; - from?: Point; - to?: Point; -} - /** Decode one lite-07 range: its start, then its end. */ -async function decodeEntry(r: Reader): Promise { - const from = await decodeAnchor(r, false); - const to = await decodeAnchor(r, true); +async function decodeRange(r: Reader): Promise { + const start = await decodeAnchor(r); + const end = await decodeAnchor(r); const range: WireRange = {}; - if (from.anchor) range.start = from.anchor; - if (to.anchor) range.end = to.anchor; - // Empty partway through a group, which whole groups can't say: keep its place, - // covering no group, as the Rust publisher does. - if (from.point && to.point && !before(from.point, to.point)) range.end = { absolute: from.point[0] }; - return { range, from: from.point, to: to.point }; + if (start) range.start = start; + if (end) range.end = end; + // Ending at or before its start: keep its place, covering no group, as the Rust publisher does. + if (start?.absolute !== undefined && end?.absolute !== undefined && end.absolute < start.absolute) + range.end = { absolute: start.absolute }; + return range; } /** Decode a lite-07 `Range Count` and its ranges, which may be none. */ -async function decodeEntries(r: Reader): Promise { +async function decodeRanges(r: Reader): Promise { // Not preallocated: each range costs at least two bytes of the bounded message. - const entries: RangeEntry[] = []; - for (let count = await r.u53(); count > 0; count--) entries.push(await decodeEntry(r)); - return entries; + const ranges: WireRange[] = []; + for (let count = await r.u53(); count > 0; count--) ranges.push(await decodeRange(r)); + return ranges; } /** - * Lite-07 `entries` as their ranges and the one span covering every one of them, which the - * subscriber narrows again. + * Lite-07 `ranges` and the one span covering every one of them, which the subscriber narrows + * again. A range covers whole groups, so the span bounds no frame. */ -export function spanOf(version: Version, entries: readonly RangeEntry[]): Span & { ranges: WireRange[] } { - let start: Point | undefined; - let end: Point | undefined; +export function spanOf(version: Version, ranges: readonly WireRange[]): Span & { ranges: WireRange[] } { + let start: bigint | undefined; + let end: bigint | undefined; let floor = true; let live = false; - for (const { from, to } of entries) { + for (const range of ranges) { + const from = range.start?.absolute; + const to = range.end?.absolute; if (from === undefined) floor = false; - else if (start === undefined || before(from, start)) start = from; + else if (start === undefined || from < start) start = from; if (to === undefined) live = true; - else if (end === undefined || before(end, to)) end = to; + else if (end === undefined || end < to) end = to; } if (!floor) start = undefined; if (live) end = undefined; - // An exclusive end at the very first position names nothing. - if (end !== undefined && end[0] === 0n && end[1] === 0) throw new ProtocolViolation(EMPTY_RANGE); - - const startFrame = start?.[1] ?? 0; + // An exclusive end at the very first group names nothing. + if (end === 0n) throw new ProtocolViolation(EMPTY_RANGE); return { - ranges: entries.map((entry) => entry.range), - startGroup: canonicalStartGroup(version, start?.[0], startFrame), - startFrame, - endGroup: end === undefined ? undefined : end[1] === 0 ? end[0] - 1n : end[0], - endFrame: end === undefined || end[1] === 0 ? undefined : end[1] - 1, + ranges: [...ranges], + startGroup: canonicalStartGroup(version, start, 0), + startFrame: 0, + endGroup: end === undefined ? undefined : end - 1n, }; } @@ -367,10 +331,7 @@ export function spanOf(version: Version, entries: readonly RangeEntry[]): Span & * The inverse of {@link encodeSpan}. A lite-07 list decodes as its ranges and as the one span * covering every range in it, which the subscriber narrows again. */ -async function decodeSpan( - r: Reader, - version: Version, -): Promise { +async function decodeSpan(r: Reader, version: Version): Promise { if (!hasRanges(version)) { const startGroup = await decodeStartGroup(r, version); const endGroup = (await r.u62()) || undefined; @@ -379,45 +340,36 @@ async function decodeSpan( return { startGroup: canonicalStartGroup(version, startGroup, frames.startFrame), endGroup: end, ...frames }; } - const entries = await decodeEntries(r); - if (entries.length === 0) throw new ProtocolViolation("a subscription asks for no range"); - return { ...spanOf(version, entries), entries }; + const ranges = await decodeRanges(r); + if (ranges.length === 0) throw new ProtocolViolation("a subscription asks for no range"); + return spanOf(version, ranges); } /** - * The ranges `msg` carries as lite-07 entries: its decoded list, or else the one span it - * encodes as its only range. + * The ranges `msg` carries on lite-07: its list, or else the one span it encodes as its only + * range, in whole groups. */ -export function entriesOf(msg: Span & { ranges?: WireRange[]; entries?: RangeEntry[] }): RangeEntry[] { - if (msg.entries) return msg.entries; - if (msg.ranges) return msg.ranges.map((range) => ({ range })); +export function rangesOf(msg: Span & { ranges?: WireRange[] }): WireRange[] { + if (msg.ranges) return msg.ranges; const range: WireRange = {}; - const entry: RangeEntry = { range }; - if (msg.startGroup !== undefined) { - range.start = { absolute: msg.startGroup }; - entry.from = [msg.startGroup, msg.startFrame]; - } - if (msg.endGroup !== undefined) { - entry.to = msg.endFrame === undefined ? [msg.endGroup + 1n, 0] : [msg.endGroup, msg.endFrame + 1]; - range.end = { absolute: msg.endGroup + 1n }; - } - return [entry]; + if (msg.startGroup !== undefined) range.start = { absolute: msg.startGroup }; + if (msg.endGroup !== undefined) range.end = { absolute: msg.endGroup + 1n }; + return [range]; } /** The bytes a lite-07 range takes on the wire, whose varints count leading ones. */ -function entrySize({ range, from, to }: RangeEntry): number { +function rangeSize(range: WireRange): number { const size = (value: number | bigint) => { const lo = split(value); return lengthLeadingOnes(parts.hi, lo); }; - const anchor = (anchor: WireAnchor | undefined, point: Point | undefined): number => { - if (point) return size(1) + size(point[0]) + size(point[1]); - if (anchor?.absolute !== undefined) return size(1) + size(anchor.absolute) + size(0); + const anchor = (anchor: WireAnchor | undefined): number => { + if (anchor?.absolute !== undefined) return size(1) + size(anchor.absolute); if (anchor?.relative !== undefined) return size(2) + size(anchor.relative); if (anchor?.time !== undefined) return size(3) + size(anchor.time); return size(0); }; - return anchor(range.start, from) + anchor(range.end, to); + return anchor(range.start) + anchor(range.end); } /** A range's anchors as a key, so equal ranges compare equal. */ @@ -443,39 +395,38 @@ export class LiveRanges { readonly #version: Version; #next = 0n; // In Range ID order, since each id is the next. - readonly #live = new Map(); + readonly #live = new Map(); // The bytes the live ranges take on the wire, held to what one SUBSCRIBE carries. #size = 0; /** How many ranges the SUBSCRIBE carried, whose starts SUBSCRIBE_START reports. */ readonly initial: bigint; /** The ranges a SUBSCRIBE carried. */ - constructor(version: Version, ranges: readonly (RangeEntry | WireRange)[]) { + constructor(version: Version, ranges: readonly WireRange[]) { this.#version = version; this.initial = BigInt(ranges.length); this.#add(ranges); } - #add(ranges: readonly (RangeEntry | WireRange)[]) { + #add(ranges: readonly WireRange[]) { for (const range of ranges) { - const entry = "range" in range ? (range as RangeEntry) : { range: range as WireRange }; // Only lite-07 numbers ranges on the wire, and only it is held to one SUBSCRIBE. - if (hasRanges(this.#version)) this.#size += entrySize(entry); - this.#live.set(this.#next++, entry); + if (hasRanges(this.#version)) this.#size += rangeSize(range); + this.#live.set(this.#next++, range); } } #remove(id: bigint): boolean { - const entry = this.#live.get(id); - if (!entry) return false; - if (hasRanges(this.#version)) this.#size -= entrySize(entry); + const range = this.#live.get(id); + if (!range) return false; + if (hasRanges(this.#version)) this.#size -= rangeSize(range); this.#live.delete(id); return true; } /** The live range with Range ID `id`. */ get(id: bigint): WireRange | undefined { - return this.#live.get(id)?.range; + return this.#live.get(id); } /** Whether `id` was ever assigned. */ @@ -490,12 +441,12 @@ export class LiveRanges { /** Every live range, in Range ID order. */ ranges(): WireRange[] { - return [...this.#live.values()].map((entry) => entry.range); + return [...this.#live.values()]; } /** Every live range with its Range ID, in order. */ entries(): [bigint, WireRange][] { - return [...this.#live].map(([id, entry]) => [id, entry.range]); + return [...this.#live]; } /** Whether the publisher reports where each range starts, which lite-07 does by Range ID. */ @@ -508,18 +459,18 @@ export class LiveRanges { * {@link ProtocolViolation} for an edit it cannot apply. Before lite-07 an update restates * its ranges, so nothing is numbered. */ - apply(update: SubscribeUpdate): { id: bigint; entry: RangeEntry }[] { + apply(update: SubscribeUpdate): { id: bigint; range: WireRange }[] { if (!hasRanges(this.#version)) return []; for (const id of update.remove ?? []) { // An id never assigned, or already retired (including earlier in this list). if (!this.#remove(id)) throw new ProtocolViolation(`unknown range id: ${id}`); } - this.#add(update.entries ?? update.ranges ?? []); + this.#add(update.ranges ?? []); // Asking for nothing is refused like a SUBSCRIBE that does, and a set no SUBSCRIBE // could carry like one too large to send. if (this.#live.size === 0) throw new ProtocolViolation("a subscription asks for no range"); if (this.#size > Message.MAX_MESSAGE_SIZE) throw new ProtocolViolation("too many ranges"); - return [...this.#live].map(([id, entry]) => ({ id, entry })); + return [...this.#live].map(([id, range]) => ({ id, range })); } /** @@ -527,8 +478,16 @@ export class LiveRanges { * to retire and the ranges to add. A range already live keeps its Range ID, so an update that * changes one range sends only that one, and the publisher keeps where the rest resolved. * Before lite-07 replaced ranges take fresh Range IDs. + * + * The last `fills` of `ranges` ask for fills, and `isFill` names the live ones that do: a fill + * asking for what a demand range does still takes a range of its own, so each Range ID stays + * the demand's or a fill's. `fillIds` are the Range IDs the added fill ranges took. */ - edit(ranges: readonly WireRange[]): { remove: bigint[]; add: WireRange[] } { + edit( + ranges: readonly WireRange[], + fills = 0, + isFill: (id: bigint) => boolean = () => false, + ): { remove: bigint[]; add: WireRange[]; fillIds: bigint[] } { if (ranges.length === 0) throw new Error(EMPTY_RANGE); if (!hasRanges(this.#version)) { const same = @@ -538,30 +497,39 @@ export class LiveRanges { this.#live.clear(); this.#add(ranges); } - return { remove: [], add: [...ranges] }; + return { remove: [], add: [...ranges], fillIds: [] }; } + const demand = ranges.length - fills; + const key = (range: WireRange, fill: boolean) => `${fill ? "f" : "d"}${rangeKey(range)}`; // Each live range keeps at most one wanted copy, so duplicates edit like any other. const wanted = new Map(); - for (const range of ranges) wanted.set(rangeKey(range), (wanted.get(rangeKey(range)) ?? 0) + 1); + ranges.forEach((range, index) => { + const k = key(range, index >= demand); + wanted.set(k, (wanted.get(k) ?? 0) + 1); + }); const remove: bigint[] = []; - for (const [id, entry] of this.#live) { - const key = rangeKey(entry.range); - const count = wanted.get(key) ?? 0; - if (count > 0) wanted.set(key, count - 1); + for (const [id, range] of this.#live) { + const k = key(range, isFill(id)); + const count = wanted.get(k) ?? 0; + if (count > 0) wanted.set(k, count - 1); else remove.push(id); } // In the order asked for, so the publisher numbers them the way this side does. - const add = ranges.filter((range) => { - const key = rangeKey(range); - const count = wanted.get(key) ?? 0; - if (count === 0) return false; - wanted.set(key, count - 1); - return true; + const add: WireRange[] = []; + const fillIds: bigint[] = []; + const first = this.#next; + ranges.forEach((range, index) => { + const k = key(range, index >= demand); + const count = wanted.get(k) ?? 0; + if (count === 0) return; + wanted.set(k, count - 1); + add.push(range); + if (index >= demand) fillIds.push(first + BigInt(add.length - 1)); }); for (const id of remove) this.#remove(id); this.#add(add); if (this.#size > Message.MAX_MESSAGE_SIZE) throw new Error("too many ranges for one SUBSCRIBE"); - return { remove, add }; + return { remove, add, fillIds }; } } @@ -663,8 +631,6 @@ export class SubscribeUpdate { update: bigint; /** Lite-07: the Range IDs of the live ranges to retire before adding {@link ranges}. */ remove?: bigint[]; - /** Lite-07: the ranges a received update adds, with the positions they name; see {@link RangeEntry}. */ - entries?: RangeEntry[]; constructor(props: { priority: number; @@ -679,7 +645,6 @@ export class SubscribeUpdate { ranges?: WireRange[]; update?: bigint; remove?: bigint[]; - entries?: RangeEntry[]; }) { if (props.wantedGroup !== undefined && props.wantedGroup < 0n) throw new RangeError("wanted group must be non-negative"); @@ -695,7 +660,6 @@ export class SubscribeUpdate { this.ranges = props.ranges; this.update = props.update ?? 0n; this.remove = props.remove; - this.entries = props.entries; } async #encode(w: Writer, version: Version) { @@ -745,13 +709,12 @@ export class SubscribeUpdate { const order = await decodeOrder(r, version); const maxDelay = await r.u53(); const minDelay = await decodeMinDelay(r, version); - let span: Span & { ranges?: WireRange[]; remove?: bigint[]; entries?: RangeEntry[] }; + let span: Span & { ranges?: WireRange[]; remove?: bigint[] }; if (hasRanges(version)) { // Not preallocated, like the ranges: each id costs a byte of the bounded message. const remove: bigint[] = []; for (let count = await r.u53(); count > 0; count--) remove.push(await r.u62()); - const entries = await decodeEntries(r); - span = { startFrame: 0, remove, entries, ranges: entries.map((entry) => entry.range) }; + span = { startFrame: 0, remove, ranges: await decodeRanges(r) }; } else { span = await decodeSpan(r, version); } @@ -828,8 +791,6 @@ export class Subscribe { * the span alone; decoding a lite-07 list fills them alongside the span covering them all. */ ranges?: WireRange[]; - /** Lite-07: the decoded ranges with the positions they name; see {@link RangeEntry}. */ - entries?: RangeEntry[]; /** * The number of the terms in effect: the SUBSCRIBE is 0 and every SUBSCRIBE_UPDATE carries a @@ -854,7 +815,6 @@ export class Subscribe { endFrame?: number; ranges?: WireRange[]; update?: bigint; - entries?: RangeEntry[]; }) { this.id = props.id; this.broadcast = props.broadcast; @@ -871,7 +831,6 @@ export class Subscribe { this.endFrame = props.endFrame; this.ranges = props.ranges; this.update = props.update ?? 0n; - this.entries = props.entries; } async #encode(w: Writer, version: Version) { diff --git a/js/net/src/lite/subscriber.ts b/js/net/src/lite/subscriber.ts index 54c314bf2d2..7bede1ea93c 100644 --- a/js/net/src/lite/subscriber.ts +++ b/js/net/src/lite/subscriber.ts @@ -22,7 +22,7 @@ import { toStreamCode, unauthorized, } from "../error.ts"; -import { fillChanged, type Judged, rejectGroups, wantedRuns } from "../fill.ts"; +import { fillChanged, type Judged, rejectGroups } from "../fill.ts"; import type { Producer as FrameProducer } from "../frame.ts"; import * as netGroup from "../group.ts"; import { Cost, type Hop, MAX_HOPS, routesEqual, UNKNOWN_HOP, WireRoute } from "../hop.ts"; @@ -39,6 +39,7 @@ import { minDelayOf, type Run, removeRun, + runGaps, scopeCaptures, scopeHead, scopeOverlaps, @@ -55,7 +56,7 @@ import type * as track from "../track.ts"; import { untilAborted } from "../util/abort.ts"; import { closeUnwanted } from "../util/announce.ts"; import { Deadline } from "../util/deadline.ts"; -import { isUnrequested, unrequested, unwanted } from "../util/demand.ts"; +import { isAbandoned, isUnrequested, unrequested, unwanted } from "../util/demand.ts"; import { LIMITS, type Limits, SlotMap, Slots } from "../util/limits.ts"; import { max, min, UNBOUNDED } from "../util/sequence.ts"; import { Budget } from "../util/yield.ts"; @@ -74,6 +75,7 @@ import { import { Datagram as DatagramMessage, Datagrams, DatagramsInfo, DatagramsUpdate } from "./datagram.ts"; import { closeViolation } from "./error.ts"; import { Fetch as FetchMessage } from "./fetch.ts"; +import { dropped, passing, RangeFills, type RecoveryHandle, type Refusal, type Refused, refuseRun } from "./fills.ts"; import { type Carrier, type Claim, Floors } from "./floors.ts"; import { type Group as GroupMessage, type Ingest, readFrames } from "./group.ts"; import { LinkRtt } from "./link_rtt.ts"; @@ -235,8 +237,17 @@ interface SubscribeEntry { sent: number; // The ranges the SUBSCRIBE asked for, as `wireRanges` gives them. asked: track.Range[]; - // A fill's own subscription, whose groups are cancelled once nobody wants them, as a FETCH's. + // A recovery's own subscription, whose groups are cancelled once nobody wants them, as a FETCH's. filling?: boolean; + // Lite-07: the copy's fills of more than one group, which ask through this subscription. + fills?: RangeFills; + // The Range IDs of the fills' live ranges; the rest are the demand's, which alone say where the + // live feed starts. + fillIds: Set; + // The groups the fills' ranges ask for, as last sent. + fillGroups: Run[]; + // Bumped as what the fills ask changes, for the update loop. + fillsChanged: Signal; // Until the SUBSCRIBE's first response (lite-05+); see `Unanswered`. unanswered?: Unanswered; // Lite-07: the groups a YIELDED reset paused, once one did or a stream waits on one. @@ -372,6 +383,17 @@ function judged(ranges: readonly track.Range[]): Run[] { return runs; } +/** The groups a fill's range asks for: absolute at both ends, as `RangeFills` asks them. */ +function fillRun(range: WireRange): Run | undefined { + const start = range.start?.absolute; + const end = range.end?.absolute; + return start === undefined || end === undefined ? undefined : { start, end }; +} + +function sameRuns(a: readonly Run[], b: readonly Run[]): boolean { + return a.length === b.length && a.every((run, i) => run.start === b[i].start && run.end === b[i].end); +} + /** Subscribe to the peer and own its receive-side protocol tasks. * @internal */ @@ -2122,6 +2144,9 @@ export class Subscriber { update: 0n, sent: 0, stream: { kind: "setup" }, + fillIds: new Set(), + fillGroups: [], + fillsChanged: new Signal(0), }; if (hasDatagrams(this.version) && !hasDatagramTracks(this.version) && state.consumer && state.legacyKey) { const info = { priority: producer.priority, timescale: framing.scale }; @@ -2473,28 +2498,54 @@ export class Subscriber { } } - // Serve the received copy's misses as its request handler, until the copy closes: each fill - // FETCHes its groups upstream one at a time, newest first for a descending reader, as the - // Rust `FillServeRun` does. + // Serve the received copy's misses as its request handler, until the copy closes. A lite-07 miss + // of more than one group joins the copy's subscription (see `RangeFills`); any other FETCHes its + // groups upstream one at a time, newest first for a descending reader, as the Rust + // `FillServeRun` does. async #runFills(broadcast: Path.Valid, consumed: Consumed, entry: SubscribeEntry): Promise { const producer = entry.track; const requests = producer.requests(); const jobs = new Set>(); + const fills = hasRanges(this.version) + ? new RangeFills(producer, (groups, priority) => + this.#recover(broadcast, consumed, entry, groups, priority), + ) + : undefined; + entry.fills = fills; + const driving = fills ? this.#driveFills(entry, fills) : undefined; try { // Accepted before its fills run, so this only reads what TRACK_INFO said. const maxAge = (await producer.info().catch(() => undefined))?.maxAge; const framing = { compress: entry.compress, timescale: entry.timescale, maxAge }; for (let fill = await requests.requestedFill(); fill; fill = await requests.requestedFill()) { + // One group is what FETCH carries; more go through the subscription. + if (fills && fill.end - fill.start > 1n) { + fills.push(fill); + continue; + } const job = this.#serveFill(broadcast, consumed, producer, framing, fill); jobs.add(job); void job.then(() => jobs.delete(job)); } } finally { requests.close(); - await Promise.all(jobs); + fills?.close(); + await Promise.all([driving, ...jobs]); } } + // Settle `fills` as their inputs change, until the copy closes, asking what they owe through + // `entry`'s subscription. Mirrors the Rust serve loop's `RangeFills::poll`. + async #driveFills(entry: SubscribeEntry, fills: RangeFills): Promise { + const demand = entry.track.demand(); + while (!fills.closed) { + // A stream still arriving on a recovery is kept only for someone reading it. + if (fills.poll(entry.tail, !isAbandoned(demand))) entry.fillsChanged.update((revision) => revision + 1); + await fills.changed(entry.tail); + } + entry.fillsChanged.update((revision) => revision + 1); + } + // Serve `fill` until nobody waits on it, then close it, naming what is left dropped. async #serveFill( broadcast: Path.Valid, @@ -2513,21 +2564,6 @@ export class Subscriber { await race([fillChanged(fill), reoffers.changed()]); continue; } - // Lite-07 asks a fill with no budget for the whole owed run around it at once, up to a - // re-offered group, so the publisher answers each run it gave up with one verdict - // however many groups it spans. A budget is judged against the whole track, which a - // FETCH does and a bounded range does not, so a bounded fill FETCHes a group at a time. - if (hasRanges(this.version) && fill.maxDelay === null) { - let { start, end } = wantedRuns(fill).find( - (run) => run.start <= sequence && sequence < run.end, - ) ?? { start: sequence, end: sequence + 1n }; - for (const run of reoffered) { - if (run.end <= sequence) start = max(start, run.end); - else end = min(end, run.start); - } - await this.#fillRun(broadcast, consumed, producer, framing, fill, { start, end }); - continue; - } // Every fetch answers its group, so the same one coming back is a bug: closing the // fill names it dropped rather than fetching it forever. if (sequence === last) break; @@ -2539,28 +2575,28 @@ export class Subscriber { } } - // Ask `run` of `fill`, which has no budget, on a lite-07 SUBSCRIBE of its own bounded to the - // run, until the fill owes none of it, as the Rust `Recovery` asks: its groups land in the copy - // as a subscription's do, and the publisher names what it gives up by run. Whatever the - // subscription leaves owed when it ends is refused, so the next run is asked, not this one. - async #fillRun( + // Ask `groups` of `copy` again on a lite-07 SUBSCRIBE of their own with no max delay, as the Rust + // `Recovery` does: `copy`'s subscription dropped them as stale, or delivered them before and they + // were since evicted, so it never sends them again. Its groups land in the copy as a + // subscription's do. It FINs once every group is accounted for and read, and is over at the + // publisher's FIN in reply, refusing whatever it did not deliver. + #recover( broadcast: Path.Valid, consumed: Consumed, - producer: track.Producer, - framing: Framing, - fill: track.Fill, - run: Run, - ): Promise { - const owes = () => wantedRuns(fill).some((owed) => owed.start < run.end && run.start < owed.end); + copy: SubscribeEntry, + groups: Run, + priority: number, + ): RecoveryHandle { + const producer = copy.track; const id = this.#subscribeNext++; - const ranges: track.Range[] = [{ start: { absolute: run.start }, end: { absolute: run.end } }]; + const ranges: track.Range[] = [{ start: { absolute: groups.start }, end: { absolute: groups.end } }]; const msg = this.#withRanges( this.#subscribeMessage( id, broadcast, consumed, producer.name, - { priority: fill.priority, order: fill.order, delay: { max: null } }, + { priority, order: "ascending", delay: { max: null } }, ranges, ), ranges, @@ -2568,8 +2604,8 @@ export class Subscriber { ); const entry: SubscribeEntry = { track: producer, - compress: framing.compress, - timescale: framing.timescale, + compress: copy.compress, + timescale: copy.timescale, tail: new Tail({ grace: () => hooks.tailGrace(producer), bill: hooks.billTrack(producer) }), requested: msg.startGroup, floor: msg.startGroup, @@ -2578,117 +2614,164 @@ export class Subscriber { starts: new Map(), update: 0n, sent: 0, - // Its responses are read by the fill itself, which owns this stream's end. + // Its responses are read here, which owns this stream's end. stream: { kind: "setup" }, filling: true, + fillIds: new Set(), + fillGroups: [], + fillsChanged: new Signal(0), }; const left = new AbortController(); const signal = consumed.valid ? AbortSignal.any([consumed.valid, left.signal]) : left.signal; - // Setup stops once nobody wants the run, as a FETCH's does: a stalled exchange (no stream - // credit, or a blocked write) never sends a subscription nobody asked for anymore. const leaving = new Promise((resolve) => left.signal.addEventListener("abort", () => resolve(), { once: true }), ); - void (async () => { - while (!left.signal.aborted && owes()) await race([fillChanged(fill), leaving]); - left.abort(); - })(); - let failed: Error = new StreamError(StreamCode.Cancel, { message: "fill subscription ended" }); - let draining = Promise.resolve(); + let refused: Refused[] = []; + // Those the fills have yet to take: each refusal is handed over once. + let fresh: Refused[] = []; + const changed = new Signal(0); + // What the publisher gives up of the groups, accounted for in the tail. The latest verdict on a + // group replaces any earlier one, so a peer repeating its drops grows nothing. + const giveUp = (start: bigint, end: bigint, refusal: Refusal) => { + entry.tail.account(start, end); + refused = refuseRun(refused, max(start, groups.start), min(end, groups.end), refusal); + fresh = refuseRun(fresh, max(start, groups.start), min(end, groups.end), refusal); + changed.update((revision) => revision + 1); + }; this.#subscribes.set(id, entry); - try { - while ( - (await this.#auth?.ready(broadcast, "subscribe", { - signal: consumed.valid, - until: () => fillChanged(fill), - })) === false - ) { - if (!owes()) return; - } - if (this.#denied(broadcast)) throw unauthorized(broadcast); - const stream = await this.#exchange( - { sendOrder: sendOrder({ priority: fill.priority }), version: this.version }, - async (stream) => { - await stream.writer.u53(StreamId.Subscribe); - await msg.encode(stream.writer, this.version); - return stream; - }, - signal, - new Deadline(), - ); - let ended = false; - const responses = this.#fillResponses(stream, entry, fill, run).then( - () => { - ended = true; - }, - // A reset says why the rest is refused. - (err: unknown) => { - ended = true; - failed = error(err); - }, - ); + const done = (async () => { + let failed: Error = new StreamError(StreamCode.Cancel, { message: "recovery ended" }); + let stream: Stream | undefined; try { - while (owes() && !ended) await race([fillChanged(fill), responses]); - } catch (err) { - stream.abort(error(err)); - throw err; - } - if (ended) { - stream.close(); - } else { - // Done asking. Its streams are read to their end, or until nobody wants their group, - // as a FETCH's is, and only then FINned, which ends whatever the publisher still serves. - draining = (async () => { - await race([entry.tail.settle(() => true, hooks.trackReleased(producer)), responses]); - stream.writer.close(); + while ((await this.#auth?.ready(broadcast, "subscribe", { signal, until: () => leaving })) === false) { + if (left.signal.aborted) return fresh; + } + if (this.#denied(broadcast)) throw unauthorized(broadcast); + const opened = await this.#exchange( + { sendOrder: sendOrder({ priority }), version: this.version }, + async (stream) => { + await stream.writer.u53(StreamId.Subscribe); + await msg.encode(stream.writer, this.version); + return stream; + }, + signal, + new Deadline(), + ); + stream = opened; + let fin = false; + const responses = this.#recoveryResponses(opened, entry, groups, giveUp, () => fin); + // FIN once every group is accounted for and read, as a subscriber does once its tail + // settles, and the publisher FINs in reply. + let ended = false; + const over = responses.then( + () => { + ended = true; + }, + () => { + ended = true; + }, + ); + while (!ended && !left.signal.aborted) { + if (entry.tail.covers(groups.start, groups.end) && !entry.tail.reading) { + fin = true; + opened.writer.close(); + break; + } + await race([Signal.race(entry.tail.changed), over, leaving]); + } + // After our FIN, a cancel still stops the wait for the publisher's. + if (!ended) await race([over, leaving]); + // Cancelled: nothing more is read, and our own reset ends the reads above. + if (!ended) { + opened.abort(failed); + await over; + } else { await responses; - stream.close(); - })(); - } - } catch (err) { - failed = error(err); - } finally { - left.abort(); - if (owes()) fill.refuseGroups(run.start, run.end, failed); - void draining.then(() => { + opened.close(); + } + } catch (err) { + closeViolation(this.#quic, err); + failed = error(err); + // The publisher reset its side, or reading failed: give back our side too. + stream?.abort(failed); + } finally { + left.abort(); this.#subscribes.delete(id); this.#endPaused(entry); entry.tail.close(); - }); - } + } + // What nothing accounted for is refused, as nothing more arrives on it. + let unanswered: Run[] = [{ ...groups }]; + for (const run of [...entry.tail.accounted(groups.start, groups.end), ...refused]) + unanswered = removeRun(unanswered, run.start, run.end); + for (const gap of unanswered) fresh.push({ ...gap, refusal: passing(failed) }); + // Only what was not handed over while it was out. + return fresh; + })(); + const takeRefused = () => { + const taken = fresh; + fresh = []; + return taken; + }; + return { + done, + takeRefused, + changed, + readingChanged: entry.tail.changed, + reading: () => entry.tail.reading, + cancel: () => left.abort(), + }; } - // Read a fill subscription's responses to its FIN: answer the fill with what the publisher - // names given up, judged as the Rust `RangeFills::refuse` does. - async #fillResponses(stream: Stream, entry: SubscribeEntry, fill: track.Fill, run: Run): Promise { + // Read a recovery's responses to the publisher's FIN, giving up what it names it will not + // deliver. `fin` says whether our FIN went out, which the publisher's answers. + async #recoveryResponses( + stream: Stream, + entry: SubscribeEntry, + groups: Run, + giveUp: (start: bigint, end: bigint, refusal: Refusal) => void, + fin: () => boolean, + ): Promise { + let started = false; + let finished = false; for (;;) { const resp = await decodeSubscribeResponseMaybe(stream.reader, this.version); - if (!resp) return; + if (!resp) { + // The publisher FINs in reply to ours, or once it finished the track. + if (fin() || finished) return; + throw new ProtocolViolation("recovery subscribe ended without SUBSCRIBE_FINISH"); + } if ("start" in resp) { - // The range starts past what it skips, which it will not deliver: refused for this fill - // alone, since another request may still get them. - const start = resp.start.starts[0] ?? run.end; - if (run.start < start) - fill.refuseGroups(run.start, min(start, run.end), new NotFound(`group ${run.start}`)); + // One start, for its one range. + if (started || resp.start.starts.length !== 1) + throw new ProtocolViolation("unexpected recovery SUBSCRIBE_OK"); + started = true; + // Below its start the publisher, asked with no max delay, lacks them. + const from = resp.start.starts[0] ?? groups.end; + giveUp( + groups.start, + max(groups.start, min(from, groups.end)), + passing(new NotFound(`group ${groups.start}`)), + ); } else if ("drop" in resp) { const { start, end } = resp.drop; - entry.tail.account(start, end); const cause = resp.drop.cause(this.version); // A paused group the drop names ends there: no continuation follows. this.#dropPaused(entry, start, end, cause); - // Only a group its publisher never had is lost for every reader; anything else, - // OLD included, is refused for now, as the Rust `Refusal::answer` does. - if (toStreamCode(cause) === StreamCode.NotFound) fill.dropGroups(start, end); - else fill.refuseGroups(start, end, cause); + giveUp(start, end, dropped(cause)); } else if ("skip" in resp) { // Never produced: final, as the subscription feeding the copy records it. const { start, end } = resp.skip; entry.tail.account(start, end); hooks.skipTrackGroups(entry.track, start, end); } else if ("finish" in resp) { - // Nothing is delivered at or past the track's end. - const end = max(resp.finish.group, run.start); - if (end < run.end) fill.refuseGroups(end, run.end, new NotFound(`group ${end}`)); + if (finished) throw new ProtocolViolation("duplicate SUBSCRIBE_FINISH"); + finished = true; + // Nothing at or past the track's end will be produced. + const end = max(resp.finish.group, groups.start); + giveUp(end, groups.end, dropped(new NotFound(`group ${end}`))); + } else { + throw new ProtocolViolation("unexpected recovery subscribe response"); } } } @@ -2929,8 +3012,16 @@ export class Subscriber { ) throw new ProtocolViolation(`unexpected SUBSCRIBE_STARTED: ${id}`); // One retired since is owed nothing. - if (entry.live.get(id)) { + const range = entry.live.get(id); + if (range) { entry.starts.set(id, start); + // A fill's range starting past where it asked skips groups it will not deliver. + const asked = entry.fillIds.has(id) ? fillRun(range) : undefined; + if (asked) { + const skipped = max(asked.start, min(start ?? asked.end, asked.end)); + entry.tail.account(asked.start, skipped); + entry.fills?.skip(asked, asked.start, skipped); + } // The tail judges what is owed again. entry.tail.touch(); this.#declare(entry); @@ -2972,6 +3063,7 @@ export class Subscriber { entry.stale = stale ? owed.reduce((runs, run) => insertDrop(runs, run.start, run.end, hole), entry.stale ?? []) : removeRun(entry.stale ?? [], start, end); + entry.fills?.refuse(start, end, error); } else if ("skip" in resp) { // Never produced, so nothing is waited for and no reader ends. A skip is final and // replaces any drop of the same groups, stale or final, so no update reopens them; a @@ -2980,6 +3072,8 @@ export class Subscriber { this.#reoffered(entry.track, start, end); entry.tail.account(start, end); entry.stale = removeRun(entry.stale ?? [], start, end); + // A fill owing them is answered: the source never had them. + entry.fills?.refuse(start, end, new NotFound(`group ${start}`)); // A group that arrived was produced after all, so the skip contradicts it: it keeps // the group, and the session goes on, since the draft requires no more. const [cached] = hooks.skipTrackGroups(entry.track, start, end); @@ -2998,7 +3092,10 @@ export class Subscriber { for (;;) { const sent = entry.sent; await this.#settleTail(entry); - if (entry.sent === sent) break; + if (entry.sent !== sent) continue; + // Fills still ask through this subscription, which a FIN would leave no way to update. + if (!entry.fills || entry.fills.isEmpty() || hooks.trackReleased(entry.track).peek() !== undefined) break; + await race([Signal.race(entry.fillsChanged), hooks.trackReleased(entry.track)]); } this.#fin(stream, entry); } @@ -3092,6 +3189,9 @@ export class Subscriber { // largest, and every later update restates it. Joins asking for the same group // coalesce into that one update. let sentWanted: bigint | undefined; + // What the fills asked with the last update, and the priority it carried. + let sentFills: Run[] = []; + let sentPriority = msg.priority; const demand = track.demand(); const edge = hooks.producerLargest(track); for (;;) { @@ -3106,9 +3206,23 @@ export class Subscriber { ? wanted : undefined; const current = hooks.requested(track).peek(); - if (current === undefined || (refresh === undefined && this.#sameSubscription(current, lastSent))) { + // Lite-07 asks the held fills' groups through the same subscription, under the demand's + // terms but the more urgent priority. + const fills = entry.fills?.ask; + const fillGroups = fills?.groups ?? []; + const priority = Math.max(current?.priority ?? 0, fills?.priority ?? -128); + if ( + current === undefined || + (refresh === undefined && + priority === sentPriority && + sameRuns(fillGroups, sentFills) && + this.#sameSubscription(current, lastSent)) + ) { // Each wake releases the quiet signals too, including when termination wins. - const next = await race([Signal.race(hooks.requested(track), demand.wanted, edge), stopped]); + const next = await race([ + Signal.race(hooks.requested(track), demand.wanted, edge, entry.fillsChanged), + stopped, + ]); if (next === null) return; continue; } @@ -3129,9 +3243,26 @@ export class Subscriber { // Lite-07 edits the live ranges: a range already live keeps its Range ID, and where it // started. The retired ones owe nothing more. let edit: { remove: bigint[]; add: WireRange[] } | undefined; + const fillRanges: track.Range[] = fillGroups.map((run) => ({ + start: { absolute: run.start }, + end: { absolute: run.end }, + })); if (hasRanges(this.version)) { - edit = entry.live.edit(toWireRanges(ranges, entry.scale)); + // The demand's ranges, then the fills'. + const edited = entry.live.edit( + [...toWireRanges(ranges, entry.scale), ...toWireRanges(fillRanges, undefined)], + fillRanges.length, + (id) => entry.fillIds.has(id), + ); + edit = edited; for (const id of entry.starts.keys()) if (!entry.live.get(id)) entry.starts.delete(id); + for (const id of entry.fillIds) if (!entry.live.get(id)) entry.fillIds.delete(id); + for (const id of edited.fillIds) entry.fillIds.add(id); + // The groups a fill newly asks for wait a fresh grace too. + for (const run of fillGroups) + for (const gap of runGaps(entry.fillGroups, run.start, run.end)) + entry.tail.demand(gap.start, gap.end); + entry.fillGroups = fillGroups; entry.floor = span.start; // Pending while a range the update adds awaits SUBSCRIBE_STARTED. this.#declare(entry); @@ -3140,7 +3271,7 @@ export class Subscriber { // interpret SUBSCRIBE_UPDATE as a reset of ordered/delay/etc. const update = new SubscribeUpdate({ wantedGroup: refresh ?? sentWanted, - priority: current.priority ?? 0, + priority, order: current.order, maxDelay: maxDelayOf(current), minDelay: minDelayOf(current), @@ -3156,7 +3287,7 @@ export class Subscriber { entry.update = update.update; entry.maxDelay = update.maxDelay; entry.tail.updated(); - const held = hooks.supersedeStale(entry.track, judged(ranges)); + const held = hooks.supersedeStale(entry.track, judged([...ranges, ...fillRanges])); // A lite-07 publisher re-offers every group it reported stale on any update, and drops // again under the new number what is still stale, so the tail owes every stale run again. // A group that arrived since its stale drop won over it: the publisher served it, so it @@ -3192,6 +3323,8 @@ export class Subscriber { // A tail settling past SUBSCRIBE_FINISH re-judges what the update now asks for. entry.tail.touch(); lastSent = { ...current }; + sentFills = fillGroups; + sentPriority = priority; console.debug(`subscribe update: id=${id} broadcast=${broadcast} track=${track.name}`); } } @@ -3204,6 +3337,8 @@ export class Subscriber { let pending = entry.start === undefined; for (const id of entry.live.ids()) { + // Only the demand's ranges say where the live feed starts. + if (entry.fillIds.has(id)) continue; if (!entry.starts.has(id)) { pending = true; break; diff --git a/js/net/src/tail.ts b/js/net/src/tail.ts index 02dd3e31650..528ba918548 100644 --- a/js/net/src/tail.ts +++ b/js/net/src/tail.ts @@ -1,5 +1,5 @@ import { type Duration, now, sleep } from "@moq/clock"; -import { type GetPromise, Signal } from "@moq/signals"; +import { type GetPromise, type Getter, Signal } from "@moq/signals"; import { READER_RUN_OVERHEAD, type Reader } from "./cache.ts"; import { insertRun, type Run, runGaps } from "./internal.ts"; import { GRACE, Ledger } from "./ledger.ts"; @@ -194,6 +194,21 @@ export class Tail { this.#ledger.demand(start, end, now()); } + /** The parts of `[start, end)` accounted for: a stream arrived, a drop named them, or the grace gave up. */ + accounted(start: bigint, end: bigint): Run[] { + return runGaps(this.#ledger.gaps(start, end), start, end); + } + + /** Whether a group stream is still being read. */ + get reading(): boolean { + return this.#active > 0; + } + + /** Bumped on every change {@link settle} watches. */ + get changed(): Getter { + return this.#changed; + } + /** Whether every sequence in `[start, end)` is accounted for. */ covers(start: bigint, end: bigint): boolean { return this.#ledger.covers(start, end); diff --git a/js/net/src/track.ts b/js/net/src/track.ts index ef807912a17..83e7e9071db 100644 --- a/js/net/src/track.ts +++ b/js/net/src/track.ts @@ -1360,6 +1360,10 @@ export class Producer { if (!group || group.closed.peek() !== undefined) return undefined; return { settled: group.closed, release: hooks.holdGroup(group) }; }; + hooks.arrivingIn = (producer, start, end) => + cachedIn(producer, start, end).filter( + (sequence) => producer.#cached.get(sequence)?.group.closed.peek() === undefined, + ); sourceStart = (producer) => producer.#wire?.served && producer.#feedStart(); fillFloor = (producer, named) => producer.#fillFloor(named); startsOf = (producer) => producer.#bounds; diff --git a/quest/m1/live-history-demand.md b/quest/m1/live-history-demand.md index af4b64cf749..1ef4cbd9010 100644 --- a/quest/m1/live-history-demand.md +++ b/quest/m1/live-history-demand.md @@ -15,12 +15,8 @@ Decided 2026-11-13 (maintainer, /quest-run follow-up from Unverified. A scratch test during #8643 gave a demand of `[{}]` for that pair, which looks like an open live range with no floor. Start from a test that pins the demand the pair should produce, and compare with Rust's equivalent. -[Lite-07 FETCH is single-group only](/quest/m2/subscribe-ranges/lite07-fetch.md) -rewrites how JS keeps fill and demand ranges apart (`LiveRanges.edit`), so -this starts after it lands. +[moq-dev/moq#8616](https://github.com/moq-dev/moq/pull/8616) (lite-07 FETCH +single-group only) rewrote how JS keeps fill and demand ranges apart +(`LiveRanges.edit`), so check against that code. Public API: none expected. Wire: none. - -## Required - -- [Lite-07 FETCH is single-group only](/quest/m2/subscribe-ranges/lite07-fetch.md) - rewrites the JS fill and demand ranges this checks diff --git a/quest/m2/lite07-finalize.md b/quest/m2/lite07-finalize.md index 25560fed44f..069aeb32d36 100644 --- a/quest/m2/lite07-finalize.md +++ b/quest/m2/lite07-finalize.md @@ -92,7 +92,7 @@ Decided 2026-10-27 (maintainer): of [Subscribe ranges](/quest/m2/subscribe-range only the children that change the lite-07 wire gate the cut: Lite-07 ranges ([moq-dev/moq#7872](https://github.com/moq-dev/moq/pull/7872)), JS ranges ([moq-dev/moq#7696](https://github.com/moq-dev/moq/pull/7696)), and -[Lite-07 FETCH is single-group only](/quest/m2/subscribe-ranges/lite07-fetch.md). The +Lite-07 FETCH is single-group only ([moq-dev/moq#8616](https://github.com/moq-dev/moq/pull/8616)). The fill path, fetch as a range, and the start bookkeeping change no wire. Decided 2026-10-25 (maintainer): addressing groups by timestamp gates the @@ -126,7 +126,6 @@ transcript freezes at the cut and a flaw in it can only be fixed on the wire. ## Required - [Routes and announces](/quest/m2/cluster-routes/README.md) - lite-07 carries the route layer: ROUTE per origin node and path-less ANNOUNCE, with the native hop list gone (only the temporary lite-06 compatibility chain remains) -- [Lite-07 FETCH is single-group only](/quest/m2/subscribe-ranges/lite07-fetch.md) - lite-07 FETCH reads one group or a slice of one, and SUBSCRIBE ranges of whole groups - [Lite-07 by start](/quest/m2/time-addressing/lite07.md) - lite-07 carries group starts, previous-start links, GROUP_END, and time bounds instead of sequence numbers - [@moq/net by start](/quest/m2/time-addressing/js.md) - JS speaks the same lite-07 wire - [A group a subscriber gives up on is always reported](/quest/m2/group-drops.md) - may add SUBSCRIBE_DROP for a stopped group; leaves this list if it takes the no-wire path diff --git a/quest/m2/subscribe-ranges/js-fills.md b/quest/m2/subscribe-ranges/js-fills.md index b74bd590bb4..05e49f54cef 100644 --- a/quest/m2/subscribe-ranges/js-fills.md +++ b/quest/m2/subscribe-ranges/js-fills.md @@ -68,10 +68,6 @@ which also redefine the stale-drop judgment it relies on. Not chosen: splitting the `Fill.time` lookup off to start now. Decided 2026-11-05 (maintainer, /quest-audit): ✅ this also waits for -[Lite-07 FETCH is single-group only](/quest/m2/subscribe-ranges/lite07-fetch.md), +Lite-07 FETCH is single-group only ([moq-dev/moq#8616](https://github.com/moq-dev/moq/pull/8616)), which moves JS multi-group fills in the same `#runFills` onto the subscription. - -## Required - -- [Lite-07 FETCH is single-group only](/quest/m2/subscribe-ranges/lite07-fetch.md) - moves JS multi-group fills onto the subscription in the same `#runFills` diff --git a/quest/m2/subscribe-ranges/lite07-fetch.md b/quest/m2/subscribe-ranges/lite07-fetch.md deleted file mode 100644 index 3f5e1e749ce..00000000000 --- a/quest/m2/subscribe-ranges/lite07-fetch.md +++ /dev/null @@ -1,71 +0,0 @@ -# [L] Lite-07 FETCH is single-group only - -## Goal - -On lite-07, FETCH reads one group or a slice within one group, and SUBSCRIBE -reads ranges of whole groups and never slices within a group. Rust and JS read -any multi-group range through SUBSCRIBE and use FETCH only for a single group -or a partial one, and the draft says so. Published versions (lite-03..06) keep -FETCH as it is. - -## Plan - -Decided 2026-11-03 (maintainer, on -[moq-dev/moq#7675](https://github.com/moq-dev/moq/pull/7675)), replacing the -2026-10-25 plan to remove FETCH from lite-07: "we can keep FETCH then but for a -single group (or slice within a single group). SUBSCRIBE is for multiple groups -and does not support slicing within a group." Route recovery -(`resume::Recover`), which fills from frame N, keeps its FETCH -(`group::Fetch::frame_start`). - -Decided 2026-11-07 (maintainer, /quest-unblock): ✅ lite-07 keeps -single-group FETCH (rejected: FETCH of several ranges delivered in order; -dropping FETCH from lite-07). Head-of-line blocking between groups is not a -feature: under backpressure the head is evicted and cancels every group -queued behind it, and out-of-order arrival helps even a 30 s jitter buffer, -while a relay should serve newer cached groups rather than wait on old ones -from cold storage. Dropping FETCH would move frame and byte offsets into every -SUBSCRIBE range operation instead. Since lite-07 names a group by its start -time, a reader that knows only a time reads it through a SUBSCRIBE range -bounded by times; FETCH stays for readers that know the group's start, such -as route recovery, flights, and byte holes. - -Decided 2026-10-25 (maintainer, on -[moq-dev/moq#6807](https://github.com/moq-dev/moq/pull/6807)) and still -standing: Rust and JS change together, since both request and serve wire FETCH -on lite-07 (`track.Consumer.fetchGroup` in JS). - -- Lite-07 SUBSCRIBE and SUBSCRIBE_UPDATE anchors stop carrying a frame: an - Absolute anchor names a group, so a Range covers whole groups. This is a wire - change to `Range`, the Positions section, and the changelog bullets that say - SUBSCRIBE may start at any frame. A read starting partway through a group goes - through FETCH instead. -- Rust and JS send FETCH on lite-07 only for one group or a slice of one - group; a fill of more than one group goes through the subscription, as the - Rust copy does since - [moq-dev/moq#7872](https://github.com/moq-dev/moq/pull/7872). -- Decided 2026-11-04 (maintainer, on - [moq-dev/moq#7872](https://github.com/moq-dev/moq/pull/7872)): the JS port - of those subscription fills is this quest's, not #7872's. JS copies started - filling only in [moq-dev/moq#7845](https://github.com/moq-dev/moq/pull/7845), - after #7872 was scoped as the Rust copy's fill, and they FETCH each missing - group one at a time, which breaks no wire rule. Port all of #7872's Rust fill - rules to the JS subscriber (`Subscriber.#runFills`), with tests: - - Subscription ranges: a lite-07 miss of more than one whole group joins the - subscription's ranges beside the demand's, told apart by Range ID, or rides - a fills-only SUBSCRIBE when no demand is open. - - Recovery: a group the subscription drops as stale, or delivered before and - since evicted, is asked once on a SUBSCRIBE of its own with no max delay. - - Skips: a range start past what a fill asked refuses only the fills owing - those groups now, while that range is live, and records nothing. -- `drafts/draft-lcurley-moq-lite.md` states the split in its Fetch and - Subscribe sections. Run `just drafts check`. -- Run `just test interop --all`. - -Decided 2026-11-05 (maintainer, /quest-audit): ✅ this waits for -Drop set ([moq-dev/moq#8476](https://github.com/moq-dev/moq/pull/8476)), since both rewrite the JS fill code -(`js/net/src/lite/subscriber.ts` `#runFills`) and the same lite-07 draft -sections. - -Public API: none expected. Wire: lite-07 (unpublished) SUBSCRIBE anchors lose -their frame. diff --git a/quest/m2/subscribe-ranges/recovery-duplicate.md b/quest/m2/subscribe-ranges/recovery-duplicate.md new file mode 100644 index 00000000000..fdb56dbea9e --- /dev/null +++ b/quest/m2/subscribe-ranges/recovery-duplicate.md @@ -0,0 +1,30 @@ +# [S] A recovered run is served once + +## Goal + +A lite-07 relay's copy never receives the same group on both a recovery and +its demand subscription. Today, in Rust and JS, a recovery's stale groups can +race a SUBSCRIBE_UPDATE that reopens the same stale runs on the copy's demand +subscription. The publisher then serves the group on both, and the relay +resets the duplicate stream (code 0) for a group it already holds. No reader +is affected, but each occurrence costs a stream and puts a reset on the wire. + +## Plan + +Found by [moq-dev/moq#8616](https://github.com/moq-dev/moq/pull/8616), which +ported the Rust `Recovery` (`rs/moq-net/src/lite/subscriber/fills.rs`) to JS +(`js/net/src/lite/fills.ts`, `#recover` in `js/net/src/lite/subscriber.ts`). + +Settled on #8616 (maintainer, 2026-11-12), which a fix keeps: + +- ✅ A recovery asks with no max delay, in Rust and JS (rejected: asking with + the most tolerant budget among the fills owing the run). A narrow reader + therefore takes a run's stale history once every hop forgot it. +- ✅ A narrow reader may take stale groups of a recovered run (rejected: + judging a recovery against the whole track, which needs a wire signal or a + change to how a ranged subscription resolves its start). + +Reproduce the race in a two-hop test first, in both languages, then fix it at +the root: either the recovery's run stays out of what the update reopens, or +the copy settles which subscription delivers a group before both send it. +Keep Rust and JS in parity. Wire: none expected. diff --git a/quest/m2/time-addressing/js.md b/quest/m2/time-addressing/js.md index 03945ec437f..e6d7b8b6d1f 100644 --- a/quest/m2/time-addressing/js.md +++ b/quest/m2/time-addressing/js.md @@ -25,7 +25,7 @@ copies fill. JS ranges 2026-10-27 decision required first, merged. Decided 2026-11-05 (maintainer, /quest-audit): ✅ this waits for -[Lite-07 FETCH is single-group only](/quest/m2/subscribe-ranges/lite07-fetch.md), +Lite-07 FETCH is single-group only ([moq-dev/moq#8616](https://github.com/moq-dev/moq/pull/8616)), which moves JS multi-group fills (`js/net/src/lite/subscriber.ts` `#runFills`) onto the subscription this rekeys. @@ -34,4 +34,3 @@ Public API: breaking, mirroring Rust's. Wire: as the lite-07 quest. ## Required - [Lite-07 by start](/quest/m2/time-addressing/lite07.md) - the wire this speaks -- [Lite-07 FETCH is single-group only](/quest/m2/subscribe-ranges/lite07-fetch.md) - moves the JS fills this rekeys onto the subscription diff --git a/quest/m2/time-addressing/lite07.md b/quest/m2/time-addressing/lite07.md index 3abd721ea5d..4ba853d5654 100644 --- a/quest/m2/time-addressing/lite07.md +++ b/quest/m2/time-addressing/lite07.md @@ -49,7 +49,7 @@ lite-07 changelog) and `doc/concept/moq-lite.md`, and run `just drafts check` and `just test interop --all`. Decided 2026-11-05 (maintainer, /quest-audit): ✅ this waits for -[Lite-07 FETCH is single-group only](/quest/m2/subscribe-ranges/lite07-fetch.md), +Lite-07 FETCH is single-group only ([moq-dev/moq#8616](https://github.com/moq-dev/moq/pull/8616)), which rewrites the same lite-07 draft sections (`Range`, Positions, SUBSCRIBE anchors). @@ -61,4 +61,3 @@ SUBSCRIBE_DROP change. ## Required - [Rust groups by start](/quest/m2/time-addressing/model.md) - the model this wire carries -- [Lite-07 FETCH is single-group only](/quest/m2/subscribe-ranges/lite07-fetch.md) - rewrites the same lite-07 draft sections first diff --git a/quest/m2/unbounded-groups/byte-holes.md b/quest/m2/unbounded-groups/byte-holes.md index a6ee9ba579a..aa8a641b840 100644 --- a/quest/m2/unbounded-groups/byte-holes.md +++ b/quest/m2/unbounded-groups/byte-holes.md @@ -23,7 +23,8 @@ by flights, so this waits on Flights and stays in m2 with its epic. Decided 2026-11-05 (maintainer, /quest-audit): ✅ the byte offset rides FETCH's frame start plus the group header, not a SUBSCRIBE range end. SUBSCRIBE ranges are whole groups since 2026-11-03 -([Lite-07 FETCH is single-group only](/quest/m2/subscribe-ranges/lite07-fetch.md), +(Lite-07 FETCH is single-group only, +[moq-dev/moq#8616](https://github.com/moq-dev/moq/pull/8616) and [moq-dev/moq#7675](https://github.com/moq-dev/moq/pull/7675)), and flights ride lite FETCH since [moq-dev/moq#7944](https://github.com/moq-dev/moq/pull/7944). This waits for that quest. @@ -68,7 +69,3 @@ Public API: `Position` gains a byte field, frames gain a chunked read, and `MAX_CACHE_BYTES` stops capping a frame. Wire: a byte offset on the current lite wip's FETCH frame start and group header. - -## Required - -- [Lite-07 FETCH is single-group only](/quest/m2/subscribe-ranges/lite07-fetch.md) - SUBSCRIBE ranges become whole groups, so the offset goes on FETCH diff --git a/quest/m2/unbounded-groups/js-byte-holes.md b/quest/m2/unbounded-groups/js-byte-holes.md index 407ccf2d433..e1433da9c9e 100644 --- a/quest/m2/unbounded-groups/js-byte-holes.md +++ b/quest/m2/unbounded-groups/js-byte-holes.md @@ -37,7 +37,8 @@ moq-transport. What is left here is the wire: Decided 2026-11-05 (maintainer, /quest-audit): ✅ the byte offset rides FETCH's frame start plus the group header, not a SUBSCRIBE range end. SUBSCRIBE ranges are whole groups since 2026-11-03 -([Lite-07 FETCH is single-group only](/quest/m2/subscribe-ranges/lite07-fetch.md), +(Lite-07 FETCH is single-group only, +[moq-dev/moq#8616](https://github.com/moq-dev/moq/pull/8616) and [moq-dev/moq#7675](https://github.com/moq-dev/moq/pull/7675)), and flights ride lite FETCH since [moq-dev/moq#7944](https://github.com/moq-dev/moq/pull/7944). This waits for that quest. @@ -48,4 +49,3 @@ Byte holes defines it. ## Required - [Byte holes](/quest/m2/unbounded-groups/byte-holes.md) - the Rust contract and the lite wire this mirrors -- [Lite-07 FETCH is single-group only](/quest/m2/subscribe-ranges/lite07-fetch.md) - SUBSCRIBE ranges become whole groups, so the offset goes on FETCH diff --git a/rs/moq-net/benches/ranges.rs b/rs/moq-net/benches/ranges.rs index 3bdf6f57144..f4a3e52f994 100644 --- a/rs/moq-net/benches/ranges.rs +++ b/rs/moq-net/benches/ranges.rs @@ -19,8 +19,6 @@ //! - `ranges_time_gaps`: time ranges each ending between two groups of a sparsely numbered //! history, so each end waits on the missing sequences of a gap of its own, beside one time //! end past the live edge whose cap moves twice per group, as in `ranges_time_cap`. -//! - `ranges_frames`: a lite-07 publisher's frame bounds, each range starting and ending -//! partway through its own group, looked up for one group as a served group does. //! //! Run with `cargo bench -p moq-net --features fuzz --bench ranges`. @@ -299,29 +297,6 @@ impl TimeGaps { } } -/// A lite-07 publisher's frame bounds; see `ranges_frames`. -pub(crate) struct Frames { - bounds: moq_net::fuzz::FrameBounds, - ranges: u64, - next: u64, -} - -impl Frames { - pub(crate) fn new(ranges: u64) -> Self { - Self { - bounds: moq_net::fuzz::FrameBounds::new(ranges), - ranges, - next: 0, - } - } - - pub(crate) fn step(&mut self) { - // Each group in turn, a range's or the one between, so a lookup can't stop early. - black_box(self.bounds.serve(self.next % (self.ranges * 2))); - self.next += 1; - } -} - fn sweep(c: &mut Criterion, name: &str, new: impl Fn(u64) -> S, step: impl Fn(&mut S)) { let mut group = c.benchmark_group(name); group.throughput(Throughput::Elements(1)); @@ -354,17 +329,12 @@ fn bench_time_gaps(c: &mut Criterion) { sweep(c, "ranges_time_gaps", TimeGaps::new, TimeGaps::step); } -fn bench_frames(c: &mut Criterion) { - sweep(c, "ranges_frames", Frames::new, Frames::step); -} - criterion_group!( benches, bench_time_end, bench_time_cap, bench_fills_out, bench_sparse, - bench_time_gaps, - bench_frames + bench_time_gaps ); criterion_main!(benches); diff --git a/rs/moq-net/src/fuzz.rs b/rs/moq-net/src/fuzz.rs index c4c58e9722a..e8b0532d015 100644 --- a/rs/moq-net/src/fuzz.rs +++ b/rs/moq-net/src/fuzz.rs @@ -229,22 +229,6 @@ impl CanonicalStorage { } } -/// A lite-07 subscription's frame bounds over many ranges, for the range-count bench. -pub struct FrameBounds(crate::lite::FramesBench); - -impl FrameBounds { - /// `ranges` ranges, each starting and ending partway through its own group. - pub fn new(ranges: u64) -> Self { - Self(crate::lite::FramesBench::new(ranges)) - } - - /// The first and last frame served of `group`, as a served group looks - /// them up. - pub fn serve(&self, group: u64) -> (u64, Option) { - self.0.serve(group) - } -} - /// Every target, keyed by the name of its `fuzz_targets/.rs` shim. pub const TARGETS: &[(&str, Target)] = &[ ("lite", lite_wire), diff --git a/rs/moq-net/src/lite/mod.rs b/rs/moq-net/src/lite/mod.rs index fec8f1f696e..ccc08dc1b63 100644 --- a/rs/moq-net/src/lite/mod.rs +++ b/rs/moq-net/src/lite/mod.rs @@ -72,9 +72,6 @@ use subscriber::*; // The fuzz harness drives a group stream's ingest directly. #[cfg(all(feature = "async", any(test, feature = "fuzz")))] pub(crate) use subscriber::FrameIngest; -// The range-count bench looks up a subscription's frame bounds directly. -#[cfg(all(feature = "async", any(test, feature = "fuzz")))] -pub(crate) use publisher::FramesBench; pub use track::*; pub use tracks::*; pub use version::Version; diff --git a/rs/moq-net/src/lite/publisher.rs b/rs/moq-net/src/lite/publisher.rs index 64f8abe516d..9b8f28b0000 100644 --- a/rs/moq-net/src/lite/publisher.rs +++ b/rs/moq-net/src/lite/publisher.rs @@ -7,7 +7,7 @@ use crate::{SessionError, datagram, frame, group, origin, track}; use rangemap::{RangeMap, RangeSet}; use std::{ borrow::Cow, - collections::{BTreeMap, HashMap}, + collections::HashMap, ops::{Bound, ControlFlow, Range}, sync::{ Arc, Mutex, @@ -4803,75 +4803,31 @@ fn position_group(group: group::Consumer, frames: &Frames) -> Option, -} - -/// What the ranges naming one group partway serve of it, merged. -#[derive(Clone, Copy, Debug, PartialEq, Eq)] -struct Part { - /// The lowest first frame any of them serves. - first: u64, - /// The highest last frame, inclusive, or `None` once one runs to the group's end. - last: Option, + /// The group the range starts partway through, and the first frame it serves of it. + first: Option<(u64, u64)>, + /// The group the range ends partway through, and the last frame it serves of it, inclusive. + last: Option<(u64, u64)>, } impl Frames { - /// Add a range serving `group` from frame `first` through `last`, inclusive, or to the - /// end of the group for `None`. - fn add(&mut self, group: u64, first: u64, last: Option) { - self.parts - .entry(group) - .and_modify(|part| { - part.first = part.first.min(first); - part.last = part.last.zip(last).map(|(a, b)| a.max(b)); - }) - .or_insert(Part { first, last }); - } - - /// The first frame to serve of `group`: 0 unless every range naming it starts partway. + /// The first frame to serve of `group`: 0 unless the range starts partway through it. fn start(&self, group: u64) -> u64 { - self.parts.get(&group).map_or(0, |part| part.first) + match self.first { + Some((at, first)) if at == group => first, + _ => 0, + } } - /// The last frame to serve of `group`, inclusive, or `None` for all of it: a range - /// naming it with no end bound there runs to its end. + /// The last frame to serve of `group`, inclusive, or `None` for all of it. fn end(&self, group: u64) -> Option { - self.parts.get(&group)?.last - } -} - -/// The frame bounds of `ranges` ranges, each starting and ending partway through its own -/// group, for the range-count bench. -#[cfg(all(feature = "async", any(test, feature = "fuzz")))] -pub(crate) struct FramesBench(Frames); - -#[cfg(all(feature = "async", any(test, feature = "fuzz")))] -impl FramesBench { - pub(crate) fn new(ranges: u64) -> Self { - let at = |group, frame| Some(lite::Anchor::Absolute(track::Position { group, frame })); - let ranges: Vec<_> = (0..ranges) - .map(|i| lite::Range { - start: at(i * 2, 1), - end: at(i * 2, 3), - }) - .collect(); - Self(Bounds::new(&ranges, None).expect("group anchors").frames) - } - - /// The frames served of `group`, as a served group looks them up. - pub(crate) fn serve(&self, group: u64) -> (u64, Option) { - (self.0.start(group), self.0.end(group)) + self.last.filter(|(at, _)| *at == group).map(|(_, last)| last) } } @@ -4910,6 +4866,7 @@ impl Bounds { ranges: Vec::with_capacity(ranges.len()), frames: Frames::default(), }; + // Only lite-06 bounds frames, and it carries one range. for range in ranges { // Empty partway through a group, which whole groups can't say: keep its place, // covering no group and narrowing no frame of one. @@ -4923,36 +4880,23 @@ impl Bounds { None => Range::oldest().start, Some(start) => anchor(start)?, }; - let first = match range.start { - Some(lite::Anchor::Absolute(start)) if start.frame != 0 => Some((start.group, start.frame)), - _ => None, - }; - let mut last = None; + if let Some(lite::Anchor::Absolute(start)) = range.start + && start.frame != 0 + { + bounds.frames.first = Some((start.group, start.frame)); + } let end = match range.end { None => None, // A mid-group end includes its group, up to the frame before it. Some(lite::Anchor::Absolute(end)) => Some(Anchor::Absolute(match end.frame.checked_sub(1) { Some(frame) => { - last = Some((end.group, frame)); + bounds.frames.last = Some((end.group, frame)); end.group.checked_add(1).ok_or(Error::ProtocolViolation)? } None => end.group, })), Some(end) => Some(anchor(end)?), }; - match (first, last) { - (Some((group, first)), Some((end, last))) if group == end => { - bounds.frames.add(group, first, Some(last)); - } - _ => { - if let Some((group, first)) = first { - bounds.frames.add(group, first, None); - } - if let Some((group, last)) = last { - bounds.frames.add(group, 0, Some(last)); - } - } - } bounds.ranges.push(Range { start, end }); } Ok(bounds) @@ -7025,45 +6969,10 @@ mod serve_group_test { ); } - /// Ranges naming the same group serve the span covering all of their frames in it. + /// A range inside one group keeps both of its bounds there. #[test] - fn frames_widen_across_ranges() { - let mut frames = Frames::default(); - for (group, first, last) in [ - (3, 5, None), - (3, 2, None), - (4, 1, Some(6)), - (7, 0, Some(1)), - (7, 0, Some(3)), - (8, 0, Some(2)), - (8, 1, None), - ] { - frames.add(group, first, last); - } - assert_eq!((frames.start(3), frames.start(4), frames.start(5)), (2, 1, 0)); - assert_eq!( - (frames.end(3), frames.end(4), frames.end(7), frames.end(6)), - (None, Some(6), Some(3), None) - ); - // One range running to the end of a group serves all of it past the others' ends. - assert_eq!((frames.start(8), frames.end(8)), (0, None)); - - // Disjoint ranges meeting partway through group 5 serve all of it between them. + fn frames_bound_one_group_at_both_ends() { let group = |group, frame| Some(lite::Anchor::Absolute(track::Position { group, frame })); - let ranges = [ - lite::Range { - start: group(4, 0), - end: group(5, 2), - }, - lite::Range { - start: group(5, 2), - end: group(6, 0), - }, - ]; - let abutting = Bounds::new(&ranges, None).unwrap(); - assert_eq!((abutting.frames.start(5), abutting.frames.end(5)), (0, None)); - - // A range inside one group keeps both of its bounds there. let inside = Bounds::new( &[lite::Range { start: group(5, 2), @@ -7073,6 +6982,7 @@ mod serve_group_test { ) .unwrap(); assert_eq!((inside.frames.start(5), inside.frames.end(5)), (2, Some(3))); + assert_eq!((inside.frames.start(4), inside.frames.end(6)), (0, None)); } /// A group whose head the publisher no longer holds is skipped, not served short: @@ -7087,10 +6997,9 @@ mod serve_group_test { let mut group = track.create_group(group::Info { sequence: 3 }).unwrap(); group.start_at(5).unwrap(); group.write_frame(Timestamp::ZERO, b"tail".to_vec()).unwrap(); - let starting = |frame| { - let mut frames = Frames::default(); - frames.add(3, frame, None); - frames + let starting = |frame| Frames { + first: Some((3, frame)), + last: None, }; // Asked for whole groups, so the missing frames 0..5 make this unservable. @@ -7123,10 +7032,9 @@ mod serve_group_test { group.write_frame(Timestamp::ZERO, vec![i]).unwrap(); } let group = group.finish().unwrap(); - let ending = |group, frame| { - let mut frames = Frames::default(); - frames.add(group, 0, Some(frame)); - frames + let ending = |group, frame| Frames { + first: None, + last: Some((group, frame)), }; let mut consumer = position_group(group.consume(), &ending(7, 1)).unwrap(); diff --git a/rs/moq-net/src/lite/subscribe.rs b/rs/moq-net/src/lite/subscribe.rs index 4ba90ab9ef6..5795dbce870 100644 --- a/rs/moq-net/src/lite/subscribe.rs +++ b/rs/moq-net/src/lite/subscribe.rs @@ -61,7 +61,8 @@ impl Subscribe<'_> { /// One end of a [`Range`], as the wire carries it. #[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] pub enum Anchor { - /// A position: a start delivers from it, and an end stops just before it. + /// A position: a start delivers from it, and an end stops just before it. Lite-07 names + /// only a group, so its frame is 0 there; only lite-06 bounds a range partway through one. Absolute(crate::track::Position), /// The group this many before the latest, included as an end. Lite07+ only, except a /// start of 0 (see [`Range`]). @@ -335,7 +336,7 @@ fn range_size(range: &Range, form: Form) -> Option { let size = |value| varint::size(value, form).ok(); let anchor = |anchor: Option| match anchor { None => size(0), - Some(Anchor::Absolute(position)) => Some(size(1)? + size(position.group)? + size(position.frame)?), + Some(Anchor::Absolute(position)) => Some(size(1)? + size(position.group)?), Some(Anchor::Relative(back)) => Some(size(2)? + size(back)?), Some(Anchor::Time(pts)) => Some(size(3)? + size(pts)?), }; @@ -347,24 +348,22 @@ fn range_size(range: &Range, form: Form) -> Option { fn decode_anchor(r: &mut Decoder<'_>) -> Result, DecodeError> { Ok(match r.varint()? { 0 => None, - 1 => Some(Anchor::Absolute(crate::track::Position { - group: r.varint()?, - frame: r.varint()?, - })), + 1 => Some(Anchor::Absolute(crate::track::Position::group(r.varint()?))), 2 => Some(Anchor::Relative(r.varint()?)), 3 => Some(Anchor::Time(r.varint()?)), _ => return Err(DecodeError::InvalidValue), }) } -/// Encode the inverse of [`decode_anchor`]. +/// Encode the inverse of [`decode_anchor`]. A range covers whole groups, so a position +/// partway through one is refused: FETCH reads a slice of a group. fn encode_anchor(w: &mut Encoder<'_>, anchor: Option) -> Result<(), EncodeError> { match anchor { None => w.varint(0), + Some(Anchor::Absolute(position)) if position.frame != 0 => Err(EncodeError::Version), Some(Anchor::Absolute(position)) => { w.varint(1)?; - w.varint(position.group)?; - w.varint(position.frame) + w.varint(position.group) } Some(Anchor::Relative(back)) => { w.varint(2)?; @@ -1542,12 +1541,24 @@ mod test { priority: 3, max_delay: std::time::Duration::from_millis(250), min_delay: None, + // Groups 7 through 9. + ranges: vec![Range { + start: absolute(7, 0), + end: absolute(10, 0), + }], + update: 0, + } + } + + /// The sample bounded partway through its groups, as only lite-06 carries. + fn framed_sample() -> Subscribe<'static> { + Subscribe { // Group 7 from frame 4, through frame 2 of group 9. ranges: vec![Range { start: absolute(7, 4), end: absolute(9, 3), }], - update: 0, + ..subscribe_sample() } } @@ -1580,7 +1591,7 @@ mod test { /// exclusive position just past it here. #[test] fn subscribe_frame_bounds_roundtrip() { - let msg = subscribe_sample(); + let msg = framed_sample(); assert_eq!(roundtrip(&msg, Version::Lite06).ranges, msg.ranges); } @@ -1671,7 +1682,7 @@ mod test { /// a `Frame Start` qualifying the zero floor must survive the round trip. #[test] fn frame_start_may_qualify_group_zero_on_lite06() { - let mut msg = subscribe_sample(); + let mut msg = framed_sample(); msg.ranges[0].start = absolute(0, 4); assert_eq!(roundtrip(&msg, Version::Lite06).ranges, msg.ranges); } @@ -1679,7 +1690,7 @@ mod test { /// Silently widening to the whole group would deliver frames the caller excluded. #[test] fn subscribe_frame_bounds_rejected_before_lite06() { - assert!(encode(&subscribe_sample(), Version::Lite05).is_err()); + assert!(encode(&framed_sample(), Version::Lite05).is_err()); } /// Frames are numbered per group, so a lite-06 `Frame End` without the `Group End` it @@ -1711,8 +1722,8 @@ mod test { msg.ranges = vec![ Range::OLDEST, Range { - start: absolute(7, 4), - end: absolute(9, 3), + start: absolute(7, 0), + end: absolute(9, 0), }, Range { start: Some(Anchor::Relative(2)), @@ -1753,6 +1764,32 @@ mod test { assert_eq!(resolved, msg.ranges); } + /// A lite-07 range covers whole groups: an absolute anchor is its type and group alone, + /// and one partway through a group is refused, since FETCH reads a slice of a group. + #[test] + fn lite07_ranges_cover_whole_groups() { + let mut msg = framed_sample(); + assert!(matches!(encode(&msg, Version::Lite07), Err(EncodeError::Version))); + msg.ranges = vec![Range { + start: absolute(7, 4), + end: None, + }]; + assert!(matches!(encode(&msg, Version::Lite07), Err(EncodeError::Version))); + + msg.ranges = vec![Range { + start: absolute(7, 0), + end: absolute(9, 0), + }]; + let whole = encode(&msg, Version::Lite07).unwrap(); + msg.ranges = vec![Range { + start: Some(Anchor::Relative(7)), + end: Some(Anchor::Relative(9)), + }]; + // The same bytes but the two types: no frame follows either group. + let relative = encode(&msg, Version::Lite07).unwrap(); + assert_eq!(whole.len(), relative.len()); + } + /// An update sends only the ranges that changed, and the publisher resolves the same list, /// with the same Range IDs, as the subscriber holds. #[test] diff --git a/rs/moq-net/src/lite/subscriber/fills.rs b/rs/moq-net/src/lite/subscriber/fills.rs index 8ef0e7757e8..5dd4f73b382 100644 --- a/rs/moq-net/src/lite/subscriber/fills.rs +++ b/rs/moq-net/src/lite/subscriber/fills.rs @@ -93,7 +93,8 @@ impl FillAsk { /// range retires, a later one asking for the same groups may start lower. A group it drops as stale is officially dropped, /// and one it already accounted for (delivered before and since evicted here) it never /// sends again, so a [`Recovery`] asks for either on its own subscription with no max delay, -/// once. +/// once. Nothing newer judges it stale there, so once every hop forgot a run, a narrow reader +/// takes its stale groups. /// /// A group stays asked until its stream finishes, since narrowing the subscription under a /// stream still arriving would reset it. @@ -147,10 +148,13 @@ impl RangeFills { for fresh in self.recovered.gaps(&groups).collect::>() { self.recover.insert(fresh); } - // Recovered once already: this drop stands for the fills owing it now. + // Recovered once already: this drop stands for the fills owing it now, except where + // a recovery still out asks for it, whose answer it waits on. for again in self.recovered.overlapping(&groups).cloned().collect::>() { let again = again.start.max(groups.start)..again.end.min(groups.end); - self.refused.insert(again, Refusal::dropped(err.clone())); + for stands in self.recovering.gaps(&again).collect::>() { + self.refused.insert(stands, Refusal::dropped(err.clone())); + } } continue; } @@ -172,6 +176,54 @@ impl RangeFills { self.skipped = Default::default(); } + /// What the recoveries still out cover: each its groups less what it refused and handed + /// over. + fn cover(&mut self) { + let mut still = RangeSet::new(); + for recovery in &self.recoveries { + for gap in recovery.released.gaps(&recovery.groups) { + still.insert(gap); + } + } + self.recovering = still; + } + + /// Take the refusals a recovery made since the last poll, once: each answers the fills + /// owing its groups now, and [`Self::forget_answered`] ends it with them, so a later request + /// for the same groups asks again rather than reading it back. + /// The groups taken leave the recovery's coverage in `released`. + fn take_refused(&mut self, fresh: &mut RangeMap, released: &mut RangeSet) { + for (groups, refusal) in std::mem::take(fresh) { + released.insert(groups.clone()); + self.recovering.remove(groups.clone()); + self.refused.insert(groups, refusal); + } + } + + /// A refusal answers only the fills owing it now: once no held fill covers its groups, a + /// later request asks again, and a loss it named stays recorded on the track alone. + fn forget_answered(&mut self) { + // Trimmed to the groups a held fill still covers. + let mut covered = RangeSet::new(); + for fill in &self.held { + let groups = fill.start().group..fill.end(); + if !groups.is_empty() { + covered.insert(groups); + } + } + let span = self + .refused + .iter() + .map(|(groups, _)| groups.clone()) + .reduce(|a, b| a.start.min(b.start)..a.end.max(b.end)); + let Some(span) = span else { + return; + }; + for gap in covered.gaps(&span).collect::>() { + self.refused.remove(gap); + } + } + /// Whether a [`Recovery`] is still out, whose groups the subscription's tail does not wait /// for. pub fn recovering(&self) -> bool { @@ -230,19 +282,25 @@ impl RangeFills { } } } - // Each stale run goes out on its own subscription with no max delay, once. + // Each stale run goes out on its own subscription with no max delay, once, with the + // priority of the fills held now: the last ask may predate them. + let priority = self + .held + .iter() + .map(|fill| fill.priority()) + .fold(self.ask.priority, i8::max); for run in std::mem::take(&mut self.recover).iter() { self.recovered.insert(run.clone()); self.recovering.insert(run.clone()); self.recoveries - .push(Recovery::new(serve, serving, run.clone(), &self.ask)); + .push(Recovery::new(serve, serving, run.clone(), priority)); } // What a finished recovery did not deliver stands refused. let mut done = Vec::new(); let mut violation = None; self.recoveries.retain_mut(|recovery| match recovery.poll(waiter) { Poll::Ready(Ok(refused)) => { - done.push((recovery.groups.clone(), refused)); + done.push(refused); false } Poll::Ready(Err(err)) => { @@ -254,17 +312,32 @@ impl RangeFills { if let Some(err) = violation { return Err(err); } - for (groups, refused) in done { - self.recovering.remove(groups); + for refused in done { for (groups, refusal) in refused.iter() { self.refused.insert(groups.clone(), refusal.clone()); } } + // What a recovery still out refused since the last poll answers the fills owing it now, + // rather than waiting on its other streams. + let mut recoveries = std::mem::take(&mut self.recoveries); + for recovery in &mut recoveries { + self.take_refused(&mut recovery.fresh, &mut recovery.released); + } + self.recoveries = recoveries; + // Rebuilt from those still out, so one ending leaves a later one's coverage of the same + // groups intact. + self.cover(); let mut ask = FillAsk::default(); // Every run a held fill still owes, recovered or not. let mut owed = RangeSet::new(); let mut held = std::mem::take(&mut self.held); + // Not asked of the subscription: a run being recovered waits on its own, and a group + // already refused is answered once its fill reaches it. + let mut waiting = self.recovering.clone(); + for (groups, _) in self.refused.iter() { + waiting.insert(groups.clone()); + } let this = &*self; held.retain_mut(|fill| { if fill.poll_withdrawn(waiter).is_ready() { @@ -288,8 +361,7 @@ impl RangeFills { continue; } owed.insert(run.clone()); - // A run being recovered waits on its own subscription. - for gap in this.recovering.gaps(&run) { + for gap in waiting.gaps(&run) { ask.groups.insert(gap); } ask.add(fill); @@ -298,13 +370,14 @@ impl RangeFills { }); self.held = held; + self.forget_answered(); + // A recovery no held fill owes any group of is cancelled, once none of its streams is // still being read or nobody reads the track, and a group is recovered once for the // fills owing it now: a later request for it may recover it again. self.recoveries .retain(|recovery| owed.overlaps(&recovery.groups) || (read && recovery.reading())); - let still: RangeSet = self.recoveries.iter().map(|recovery| recovery.groups.clone()).collect(); - self.recovering = still; + self.cover(); let recovered: Vec<_> = self.recovered.iter().cloned().collect(); self.recovered = RangeSet::new(); for run in recovered { @@ -414,11 +487,18 @@ impl Refusal { /// publisher gives it up, then FINs and waits for the publisher's FIN in reply. struct Recovery { serve: TrackServe, + /// The copy its groups land in. + serving: track::Producer, id: u64, groups: Range, state: RecoveryState, /// The groups this subscription said it will not deliver, and why. refused: RangeMap, + /// Those [`RangeFills`] has yet to take: each refusal is handed over once. + fresh: RangeMap, + /// The groups it refused and handed over, which it no longer covers, so a later request + /// for them asks again rather than waiting on its other streams. + released: RangeSet, } enum RecoveryState { @@ -434,10 +514,10 @@ enum RecoveryState { } impl Recovery { - fn new(serve: &TrackServe, serving: &track::Producer, groups: Range, ask: &FillAsk) -> Self { + fn new(serve: &TrackServe, serving: &track::Producer, groups: Range, priority: i8) -> Self { let subscription = Subscription::default() .with_ranges([track::Range::groups(groups.clone())]) - .with_priority(ask.priority) + .with_priority(priority) .with_delay(Delay::default().with_max(None)) .with_order(Order::Ascending); let ranges = vec![lite::Range { @@ -452,10 +532,13 @@ impl Recovery { let establish = serve.register(subscription, ranges, Some(copy)); Self { serve: serve.clone(), + serving: serving.clone(), id: establish.id, groups, state: RecoveryState::Establish(Box::new(establish)), refused: Default::default(), + fresh: Default::default(), + released: Default::default(), } } @@ -483,6 +566,7 @@ impl Recovery { } for gap in owed.iter() { self.refused.insert(gap.clone(), Refusal::passing(err.clone())); + self.fresh.insert(gap.clone(), Refusal::passing(err.clone())); } self.serve.subscriber.remove_subscribe(self.id); match state { @@ -546,7 +630,8 @@ impl Recovery { } let given = given.start.max(groups.start)..given.end.min(groups.end); if !given.is_empty() { - self.refused.insert(given, refusal); + self.refused.insert(given.clone(), refusal.clone()); + self.fresh.insert(given, refusal); } }; match res { @@ -565,6 +650,14 @@ impl Recovery { Ok(Some(lite::SubscribeResponse::Drop(dropped))) => { given_up(dropped.groups.clone(), Refusal::dropped(dropped.error(version))); } + // Never produced: final, recorded on the copy as the subscription feeding it + // records it, which answers every fill owing the groups. + Ok(Some(lite::SubscribeResponse::Skip(skipped))) => { + if let Ok(mut tail) = active.tail.write() { + tail.account(skipped.groups.clone(), now); + } + self.serving.skip_received(skipped.groups); + } // Nothing at or past the track's end will be produced. Ok(Some(lite::SubscribeResponse::Finish(finish))) if active.end.is_none() => { given_up( @@ -601,7 +694,8 @@ impl Recovery { } } } - RecoveryState::Done => return Poll::Ready(Ok(std::mem::take(&mut self.refused))), + // Only what was not handed over while it was out. + RecoveryState::Done => return Poll::Ready(Ok(std::mem::take(&mut self.fresh))), } } } @@ -638,6 +732,19 @@ mod tests { assert_eq!(fills.refused.get(&3), Some(&Refusal::dropped(StreamError::Old))); assert!(!fills.refused.get(&3).unwrap().lost, "a stale drop is never a loss"); + // A stale drop repeated while a recovery still asks for the groups waits on it. + fills.recovering.insert(3..5); + fills.refuse(3..5, StreamError::Old); + assert_eq!( + fills.refused.get(&3), + Some(&Refusal::dropped(StreamError::Old)), + "kept from before" + ); + fills.refused = Default::default(); + fills.refuse(3..5, StreamError::Old); + assert_eq!(fills.refused.get(&3), None, "a recovery still asks for it"); + fills.recovering = Default::default(); + fills.refuse(7..9, StreamError::NotFound); assert_eq!(fills.refused.get(&7), None, "no fill asked for it"); fills.refuse(5..6, StreamError::NotFound); @@ -645,6 +752,47 @@ mod tests { assert!(fills.refused.get(&5).unwrap().lost, "the publisher never had it"); } + /// A refusal stands while a held fill covers its groups, and ends with them, so a later + /// request asks again and a loss stays recorded on the track alone. + #[test] + fn a_refusal_ends_with_its_fills() { + let (_producer, _requests, _reader, fill) = missed(); + let mut fills = RangeFills::default(); + fills.held.push(fill); + fills.refused.insert(1..3, Refusal::passing(StreamError::Cancel)); + fills.refused.insert(3..6, Refusal::dropped(StreamError::NotFound)); + fills.forget_answered(); + assert!(fills.refused.get(&1).is_some(), "the held fill still owes it"); + assert!(fills.refused.get(&3).is_some(), "the held fill still owes it"); + assert_eq!(fills.refused.get(&4), None, "trimmed to the groups a held fill covers"); + + fills.held.clear(); + fills.forget_answered(); + assert!(fills.refused.is_empty(), "no fill owes them any more"); + } + + /// A recovery still out hands each refusal over once: after the fills owing it end, a + /// later poll does not read it back for a later request. + #[test] + fn a_recovery_refusal_is_taken_once() { + let mut fills = RangeFills::default(); + fills.recovering.insert(1..4); + let mut fresh = RangeMap::new(); + let mut released = RangeSet::new(); + fresh.insert(1..3, Refusal::passing(StreamError::Cancel)); + fills.take_refused(&mut fresh, &mut released); + assert!(fills.refused.get(&1).is_some(), "handed over"); + // No longer covered, so a later request asks again rather than waiting on the rest. + assert!(!fills.recovering.contains(&1), "released"); + assert!(fills.recovering.contains(&3), "still recovering"); + assert!(released.contains(&2)); + + // No fill owes it any more, and the recovery is still out. + fills.forget_answered(); + fills.take_refused(&mut fresh, &mut released); + assert!(fills.refused.is_empty(), "not read back"); + } + /// A track whose cache evicted groups 1..4, with a reader of them whose fill is the next /// request. fn missed() -> (track::Producer, track::Requests, track::Ordered, track::Fill) { diff --git a/rs/moq-net/tests/integration/range_fill.rs b/rs/moq-net/tests/integration/range_fill.rs index 63a405f74f7..bef0116c821 100644 --- a/rs/moq-net/tests/integration/range_fill.rs +++ b/rs/moq-net/tests/integration/range_fill.rs @@ -186,6 +186,44 @@ async fn a_recovery_outlives_the_track_finishing() { .expect("timed out"); } +/// A recovery's publisher may skip a sequence inside it, which the copy records as the live +/// subscription's skip is: the reader gets the groups around it, and the session goes on. +#[kio::test] +async fn a_recovery_takes_a_skip_inside_it() { + kio::clock::timeout(TIMEOUT + cache::DEFAULT_EXPIRY * 3, async { + let fixture = Fixture::new(2).await; + fixture.track.skip_groups(2..3); + for sequence in 3..6 { + let mut group = fixture.track.create_group(moq_net::group::Info { sequence }).unwrap(); + group + .write_frame(Timestamp::from_millis(sequence * 1000).unwrap(), b"x".as_slice()) + .unwrap(); + group.finish().unwrap(); + } + let mut first = fixture.subscribe([Range::oldest()]).await; + assert_eq!(receive(&mut first, 5).await, [0, 1, 3, 4, 5].into()); + + // Superseded, the copy expires them, skip and all, while the reader keeps the + // subscription live. + kio::clock::sleep(cache::DEFAULT_EXPIRY * 2 + Duration::from_secs(2)).await; + let mut second = fixture.subscribe([Range::groups(1..5)]).await; + assert_eq!(receive(&mut second, 3).await, [1, 3, 4].into()); + + // The live subscription goes on. + let mut group = fixture + .track + .create_group(moq_net::group::Info { sequence: 6 }) + .unwrap(); + group + .write_frame(Timestamp::from_millis(6000).unwrap(), b"x".as_slice()) + .unwrap(); + group.finish().unwrap(); + assert_eq!(receive(&mut first, 1).await, [6].into()); + }) + .await + .expect("timed out"); +} + /// One group, or a slice of one, is what lite-07 FETCH carries, so a miss of one group still /// fetches it. #[kio::test]