Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions dart/moq/lib/src/aliases.dart
Original file line number Diff line number Diff line change
Expand Up @@ -170,9 +170,12 @@ extension type Timestamp._(MoqTimestamp _timestamp) implements MoqTimestamp {

/// How far behind the live edge a subscriber or decoder plays: where it starts, and how late it may fall.
extension type Delay._(MoqDelay _delay) implements MoqDelay {
/// `min` is the target latency and where a cold join starts, and `max` is how far a
/// non-latest group may fall behind before it is skipped. Both default to zero, moq-net's
/// default. An explicit null `min` leaves it unstated, which a decoder plays as zero, and an
/// `min` is the target latency, where a cold join starts, and how long a decoder waits for a
/// missing group, and `max` is how far a non-latest group may fall behind before it is skipped.
/// Both default to zero, moq-net's default. An explicit null `min` leaves it unstated, which a
/// decoder starts as zero, waiting a missing group for `max` instead (an audio decoder floors it
/// at zero and waits only its playout estimate, which can be larger; a catalog warmup or an audio
/// decoder's playout delay still states a `min` to the publisher), and an
/// explicit null `max` sets no subscriber limit. A `min` past `max` starts at `max`.
Delay({Duration? min = Duration.zero, Duration? max = Duration.zero})
: this._(MoqDelay(min: min, max: max));
Expand Down
4 changes: 2 additions & 2 deletions doc/concept/audio-jitter.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,8 @@ Natively that holds only as far as the caller lets it. The container consumer
reads groups in sequence order and applies its age budget inside `read()`, so a
group skipped for age is never observed, and a frame queued behind a stalled
group is observed when the stall clears. A missing group, one that has delivered
no frame, is waited for only the playout delay plus any lookahead, since past that
it would play late: a reordered backlog's holes waited out to the whole budget
no frame, is waited for only the playout delay, the subscription's `delay.min`,
since past that it would play late: a reordered backlog's holes waited out to the whole budget
would delay the frames behind them, which the estimator would measure as
jitter. The browser player waits less still, only until the group past the hole
trails its clock by the stretch threshold, past which the ring would skip the
Expand Down
12 changes: 8 additions & 4 deletions go/wrapper/records.go
Original file line number Diff line number Diff line change
Expand Up @@ -76,11 +76,15 @@ type DatagramInfo = ffi.MoqDatagramInfo
// starts at the newest group and skips a superseded group at once.
type Delay struct {
// Min is the target latency: the playout or jitter-buffer delay held behind the live edge,
// and where a cold join starts. A Min past Max starts at Max, since the publisher serves
// nothing older. moq-lite-07 sends it upstream.
// where a cold join starts, and how long a decoder waits for a missing group. A Min past Max
// starts and waits at Max, since the publisher serves nothing older. moq-lite-07 sends it
// upstream.
Min time.Duration
// MinUnstated leaves Min unstated, ignoring it. A decoder plays it as zero, while the
// subscription keeps it unstated, so it never pulls the aggregate across subscribers to zero.
// MinUnstated leaves Min unstated, ignoring it. A decoder starts it as zero and waits a
// missing group for Max (an audio decoder floors it at zero and waits only its playout
// estimate, which can be larger), while the subscription keeps it unstated, so it never
// pulls the aggregate across subscribers to zero, unless a catalog warmup or an audio
// decoder's playout delay states one.
MinUnstated bool
// Max is how far a non-latest group may fall behind the live edge before it is skipped. Zero
// skips immediately; a larger value tolerates that much reordering. It caps every range,
Expand Down
1 change: 0 additions & 1 deletion js/hang/api/container.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ export interface ConsumerProps {
format: Format;
playhead?: Playhead;
predecessor?: Consumer;
wait?: Signals.GetterInit<Net.Clock.Duration | undefined>;
warmup?: Net.Clock.Duration;
}

Expand Down
7 changes: 2 additions & 5 deletions js/hang/src/container/consumer.backlog.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,7 @@ async function play(events: Event[], backlog: (group: number) => boolean): Promi
const track = new Track.Producer("audio");
const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(2000) } }), {
format: new Format("audio"),
delay: { max: Duration.millis(2000) },
wait: Duration.millis(100),
delay: { min: Duration.millis(100), max: Duration.millis(2000) },
});
const oracle = new Jitter();
const base = now().since(Instant.ZERO).asMillis();
Expand Down Expand Up @@ -149,8 +148,7 @@ test("the join's boundary is the publisher's answer, not a group that raced it",
});
const consumer = new Consumer(raced, {
format: new Format("audio"),
delay: { max: Duration.millis(2000) },
wait: Duration.millis(100),
delay: { min: Duration.millis(100), max: Duration.millis(2000) },
});
const write = (sequence: number, media: number) => {
const group = track.createGroup(BigInt(sequence));
Expand Down Expand Up @@ -209,7 +207,6 @@ test("the join's boundary is the publisher's answer, not the largest by the firs
const consumer = new Consumer(raced, {
format: new Format("audio"),
delay: { max: Duration.millis(2000) },
wait: Duration.millis(100),
});
const write = (sequence: number, media: number) => {
const group = track.createGroup(BigInt(sequence));
Expand Down
21 changes: 9 additions & 12 deletions js/hang/src/container/consumer.holes.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ test.each([0, 100])("a sequence hole before a paused data marker expires at %sms
const track = new Track.Producer("data");
const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(30_000) } }), {
format: new Format("data"),
delay: { max: Duration.millis(budget) },
delay: { min: null, max: Duration.millis(budget) },
});
const results: Read[] = [];
const reading = (async () => {
Expand Down Expand Up @@ -66,7 +66,7 @@ test("a filled hole cancels its timer and the next hole gets a fresh budget", as
const track = new Track.Producer("data");
const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(30_000) } }), {
format: new Format("data"),
delay: { max: Duration.millis(100) },
delay: { min: null, max: Duration.millis(100) },
});
const results: Read[] = [];
const reading = (async () => {
Expand Down Expand Up @@ -124,7 +124,7 @@ test("the holes a stalled source left behind share one budget", async () => {
const track = new Track.Producer("data");
const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(30_000) } }), {
format: new Format("data"),
delay: { max: Duration.millis(100) },
delay: { min: null, max: Duration.millis(100) },
});
const results: Read[] = [];
const reading = (async () => {
Expand Down Expand Up @@ -168,7 +168,7 @@ test("a hole is late from the earliest arrival past it", async () => {
const track = new Track.Producer("data");
const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(30_000) } }), {
format: new Format("data"),
delay: { max: Duration.millis(100) },
delay: { min: null, max: Duration.millis(100) },
});
const results: Read[] = [];
const reading = (async () => {
Expand Down Expand Up @@ -275,7 +275,7 @@ test.each([true, false])("the head after a declared end is waited for (arrives:
const track = new Track.Producer("audio");
const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(30_000) } }), {
format: new Format("audio"),
delay: { max: Duration.millis(300) },
delay: { min: null, max: Duration.millis(300) },
});
const results: Read[] = [];
const reading = (async () => {
Expand Down Expand Up @@ -355,7 +355,7 @@ test("a newest-first flush past a hole delivers the groups landing behind its ne
const track = new Track.Producer("audio");
const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(30_000) } }), {
format: new Format("audio"),
delay: { max: Duration.millis(80) },
delay: { min: null, max: Duration.millis(80) },
});
const results: Read[] = [];
const reading = (async () => {
Expand Down Expand Up @@ -408,8 +408,7 @@ test("a hole is waited for the wait while an arrived group plays up to the max d
const track = new Track.Producer("audio");
const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(30_000) } }), {
format: new Format("audio"),
delay: { max: Duration.millis(1000) },
wait: Duration.millis(100),
delay: { min: Duration.millis(100), max: Duration.millis(1000) },
});
const results: Read[] = [];
const reading = (async () => {
Expand Down Expand Up @@ -497,8 +496,7 @@ test.each([
const track = new Track.Producer("audio");
const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(30_000) } }), {
format: new Format("audio"),
delay: { max: Duration.millis(2_000) },
wait: Duration.millis(500),
delay: { min: Duration.millis(500), max: Duration.millis(2_000) },
playhead,
});
const results: Read[] = [];
Expand Down Expand Up @@ -549,8 +547,7 @@ test("a hole ends once the playout clock jumps past the group past it", async ()
const track = new Track.Producer("audio");
const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(30_000) } }), {
format: new Format("audio"),
delay: { max: Duration.millis(2_000) },
wait: Duration.millis(500),
delay: { min: Duration.millis(500), max: Duration.millis(2_000) },
playhead: { now: () => position, anchor },
});
const results: Read[] = [];
Expand Down
4 changes: 2 additions & 2 deletions js/hang/src/container/consumer.nonsequential.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ test("Consumer waits on a non-sequential hang group because it has no per-frame
const track = new Track.Producer("test");
const consumer = new Consumer(track.subscribe(), {
format: new LegacyFormat("data"),
delay: { max: Duration.millis(5000) },
delay: { min: null, max: Duration.millis(5000) },
});

// Group A at a large sequence, completed so the cursor advances past it.
Expand Down Expand Up @@ -149,7 +149,7 @@ test("Consumer reads one-packet Opus groups at sparse sequences back to back", a
});
const consumer = new Consumer(track.subscribe(), {
format: new LegacyFormat(config),
delay: { max: Duration.millis(5000) },
delay: { min: null, max: Duration.millis(5000) },
});
// One 20 ms CELT packet per group, each read before the next lands. The group starting at 40 ms
// never arrives.
Expand Down
2 changes: 1 addition & 1 deletion js/hang/src/container/consumer.outoforder.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ test("the head of the window arriving after the live edge is delivered first", a
subscriber.start = async () => 10n;
const consumer = new Consumer(subscriber, {
format: new LegacyFormat("data"),
delay: { max: Duration.millis(5000) },
delay: { min: null, max: Duration.millis(5000) },
});

// The live edge lands first, and delivery waits for the floor rather than starting there.
Expand Down
2 changes: 1 addition & 1 deletion js/hang/src/container/consumer.skips.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,7 @@ test("a blocking group is kept until its successor starts a budget behind the ne
const track = new Track.Producer("tone");
const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(30_000) } }), {
format: new Format("data"),
delay: { max: Duration.millis(150) },
delay: { min: null, max: Duration.millis(150) },
});
const write = (sequence: bigint, timestamp: number) => {
const group = track.createGroup(sequence);
Expand Down
2 changes: 1 addition & 1 deletion js/hang/src/container/consumer.startup.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ function setup(maxDelay = 100) {
const track = new Track.Producer("video");
const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(30_000) } }), {
format: new Format("video"),
delay: { max: Duration.millis(maxDelay) },
delay: { min: null, max: Duration.millis(maxDelay) },
});
const results: Read[] = [];
const warn = spyOn(console, "warn").mockImplementation(() => {});
Expand Down
6 changes: 3 additions & 3 deletions js/hang/src/container/consumer.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1781,7 +1781,7 @@ test("Consumer waits on a PTS gap instead of skipping to a later buffered group
const track = new Track.Producer("test");
const consumer = new Consumer(replay(track), {
format: timedLegacy,
delay: { max: Duration.millis(10_000) },
delay: { min: null, max: Duration.millis(10_000) },
});

// Group A (seq 1_000_000): content ends at 33_333µs.
Expand Down Expand Up @@ -1873,7 +1873,7 @@ test("Consumer plays the head once a waited-out gap exceeds the budget (Legacy)"
const track = new Track.Producer("test");
const consumer = new Consumer(replay(track), {
format: timedLegacy,
delay: { max: Duration.millis(100) },
delay: { min: null, max: Duration.millis(100) },
});

// A (seq 1000): one frame, ends at 3000 ticks (33_333µs).
Expand Down Expand Up @@ -1908,7 +1908,7 @@ test("Consumer waits on a gap within the budget, then plays the head once it exc
const track = new Track.Producer("test");
const consumer = new Consumer(replay(track), {
format: timedLegacy,
delay: { max: Duration.millis(100) },
delay: { min: null, max: Duration.millis(100) },
});

// A (seq 1000): one frame, ends at 3000 ticks (33_333µs).
Expand Down
36 changes: 14 additions & 22 deletions js/hang/src/container/consumer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,13 @@ export interface ConsumerProps {
*
* The start is the group covering the live edge less `min`, plus any {@link ConsumerProps.warmup}, waiting
* for it when the newest group arrives first; an unstated (`null`) `min` reads as zero. A `min`
* past `max` starts at `max`, since the publisher serves nothing older. `max` is measured as
* the span from the oldest buffered frame to the newest, so it bounds how long a late group is
* waited for, and a missing one unless {@link ConsumerProps.wait} is shorter. It is the local
* past `max` starts at `max`, since the publisher serves nothing older. A missing group is waited
* for `min`, the reader's own buffer, since past it the group would play late, or for `max` when
* `min` is `null`, never longer than `max`. The wait expires this much monotonic time after the
* first group past it arrived, even when that group is a marker on a paused track, or sooner once
* where that group begins falls `max` behind the newest content. `max` is measured as
* the span from the oldest buffered frame to the newest, so it bounds how long a late group that
* did arrive is waited for, letting a player catch up on late media its holes skipped past. It is the local
* half of the subscription's `delay.max`; both measure the same budget, one on the wire and one
* as frames are read. It bounds a join's lateness too: a backlog that landed before the first
* read starts at the first group whose first frame is within it, plus any warmup, of the
Expand All @@ -31,16 +35,6 @@ export interface ConsumerProps {
*/
// Read-only: a Getter (e.g. another component's output) is accepted directly.
delay?: GetterInit<Moq.Track.Delay>;
/**
* How long a missing group is waited for before the consumer skips past it, at most `delay.max`
* (default: `delay.max`). It expires this much monotonic time after the first group past it
* arrived, even when that group is a marker on a paused track, or sooner once where that group
* begins falls `delay.max` behind the newest content. Groups that did arrive still play until they age past
* `delay.max`, so a player can wait on holes for its playout delay alone while still receiving
* the late media it catches up on.
*/
// Read-only: a Getter (e.g. another component's output) is accepted directly.
wait?: GetterInit<Duration | undefined>;
/**
* Cap the bytes of decoded frames held ahead of the reader, counting the whole wire payload they
* share and each frame's and group's own overhead (default: one group at the transport's size
Expand Down Expand Up @@ -84,7 +78,7 @@ export interface ConsumerProps {
* The playout clock this reader feeds, such as the player's `Sync` (default: none). A missing
* group whose successor starts at or before the oldest media the clock still plays could only
* arrive too late to play, so it is skipped then rather than waited for: a reader resuming behind
* the live edge moves on once the clock passes the hole, not a whole {@link ConsumerProps.wait} later.
* the live edge moves on once the clock passes the hole, not a whole `delay.min` later.
*/
playhead?: Playhead;
}
Expand Down Expand Up @@ -205,7 +199,6 @@ export class Consumer {
#track: Moq.Track.Subscriber;
#format: Format;
#delay: Getter<Moq.Track.Delay>;
#wait: Getter<Duration | undefined>;
#playhead?: Playhead;
#maxBuffer: number;
// How much further than the min delay the start reaches, and the age skip tolerates while warming.
Expand Down Expand Up @@ -296,7 +289,6 @@ export class Consumer {
this.#track = track;
this.#format = props.format;
this.#delay = getter(props.delay ?? {});
this.#wait = getter(props.wait);
this.#playhead = props.playhead;
this.#maxBuffer = props.maxBuffer ?? DEFAULT_MAX_BUFFER;
this.#warmup = (props.warmup ?? Duration.ZERO).min(DELAY_MAX);
Expand Down Expand Up @@ -941,7 +933,7 @@ export class Consumer {
this.#track.isSkipped(this.#active, head.consumer.start) ||
ptsContiguous(this.#presentedEnd, head.frames.at(0)?.timestamp.asMicros());
const empty = head.empty && head.consumer.done;
const wait = this.#holeWait(this.#delay.peek(), this.#wait.peek());
const wait = this.#holeWait(this.#delay.peek());
const bound = this.#bound();
const edge = this.#liveEdge();
// Measured against the max delay, not the wait: a reader starved of CPU takes in a burst of
Expand Down Expand Up @@ -1073,8 +1065,7 @@ export class Consumer {
// A clock that jumps moves when it reaches the group past a hole.
if (this.#playhead) effect.get(this.#playhead.anchor);
// The start waits out its reach, and a hole the wait, which an unlimited one never does.
const budget =
starting === undefined ? this.#holeWait(delay, effect.get(this.#wait)) : this.#reach(delay);
const budget = starting === undefined ? this.#holeWait(delay) : this.#reach(delay);
let left = Number.isFinite(budget)
? Duration.micros(budget).saturatingSub(now().since(since))
: undefined;
Expand Down Expand Up @@ -1114,10 +1105,11 @@ export class Consumer {
return Math.max(0, bound - playhead);
}

// How long a missing group is waited for, in microseconds: the wait, at most the max delay.
#holeWait(delay: Moq.Track.Delay, wait: Duration | undefined): number {
// How long a missing group is waited for, in microseconds: the min delay, the reader's own buffer,
// or the max delay when the min is unstated, at most the max delay. Mirrors Rust's `hole_wait`.
#holeWait(delay: Moq.Track.Delay): number {
const max = maxMicros(delay);
return wait === undefined ? max : Math.min(wait.asMicros(), max);
return delay.min === null ? max : Math.min(delay.min?.asMicros() ?? 0, max);
}

// How far behind the live edge the start reaches, in microseconds: the min delay plus the warmup.
Expand Down
6 changes: 4 additions & 2 deletions js/net/src/track.ts
Original file line number Diff line number Diff line change
Expand Up @@ -308,8 +308,10 @@ export interface Range {
export interface Delay {
/**
* The target latency, the playout or jitter-buffer delay held behind the live edge. Omitted
* means `0`; `null` means not stated, which only a peer whose wire cannot carry it (moq-lite
* before 07, moq-transport) sends. A media consumer reads an unstated minimum as `0`.
* means `0`; `null` means not stated, which a peer whose wire cannot carry it (moq-lite
* before 07, moq-transport) sends. A media consumer also waits this long for a missing group,
* since past it the group would play late, and starts an unstated minimum as `0` but waits a
* missing group for {@link Delay.max} instead.
*
* A live join, `{ start: { relative: 0n } }` with no end, starts this far behind the live edge:
* at the newest group that began at or before the live edge minus this, so a player's buffer
Expand Down
5 changes: 2 additions & 3 deletions js/watch/src/audio/decoder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -370,8 +370,8 @@ export class Decoder {
// estimate rising one bucket at a time. The container
// consumer drops aged media at the same max delay, since it observes each frame before applying
// it, and carves out the warmup the subscription adds once it has warmed, as Rust's does. It waits
// on a missing group only for the delay plus the lookahead: past that the group would play late,
// and a reordered backlog's holes waited out to the max would read as jitter to the estimator.
// on a missing group only for the delay, its min: past that the group would play late, and a
// reordered backlog's holes waited out to the max would read as jitter to the estimator.
Comment thread
kixelated marked this conversation as resolved.
Resolved
// See doc/concept/audio-jitter.md.
#runSubscribeMaxDelay(effect: Effect): void {
const budget = this.#budget(effect);
Expand Down Expand Up @@ -951,7 +951,6 @@ export class Decoder {
const consumer = new Container.Consumer(sub, {
format,
delay: this.#delay,
wait: this.sync.out.maxDelay,
warmup: Duration.millis(config.warmup ?? 0).min(DELAY_MAX),
predecessor: continues ? outgoing.consumer : undefined,
playhead: playable(this.sync, this.#output),
Expand Down
Loading