Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 85 additions & 9 deletions doc/concept/audio-jitter.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,19 +107,39 @@ observe(arrival, media):
delay = max(0, (arrival - ref.arrival) - (media - ref.media))

if first observation:
origin = arrival; interval = 0; peak = delay; return
origin = arrival; interval = 0; peak = delay; note(arrival, delay); return

index = floor((arrival - origin) / RESAMPLE)
resumed = ended; ended = false
if index <= interval:
peak = max(peak, delay); return
peak = max(peak, delay); note(arrival, delay); return

fold(bucket of peak)
close(bucket of peak)
if resumed: # a declared silence folds nothing
origin = arrival; index = 0
else:
fold(nothing) for each elapsed empty interval, at most IDLE_MAX
interval = index; peak = delay
settle(); origin = arrival; index = 0
else: # held until this interval closes
pending = index - interval - 1
interval = index; peak = delay; due = +inf; late = -inf
note(arrival, delay)

note(arrival, delay): # due and late start at +inf and -inf
if delay > RESAMPLE: # stalled: due in an earlier interval
due = min(due, arrival - delay); late = max(late, arrival)

close(bucket): # pending starts at 0 and is spent here
if late > -inf and a stall is open
and due < stall.late and due - stall.due < WINDOW:
stall.bucket = max(stall.bucket, bucket); stall.late = late
stall.span += pending + 1; pending = 0
return # still behind the same stall
settle()
if late > -inf: open a stall with bucket, late, due, span = pending
else: fold(nothing) min(pending, IDLE_MAX) times; fold(bucket)
pending = 0

settle(): # a stall is one observation
if a stall is open:
fold(stall.bucket); fold(nothing) min(stall.span, IDLE_MAX) times; close it

end(): # a declared end precedes the next observation
if not first observation: ended = true
Expand Down Expand Up @@ -222,6 +242,50 @@ about **135 s**. The forget factor decays **once per resampled observation, not
once per arrival**. Applying a per-observation factor per arrival is the mistake
the #3517 branch makes, and the resulting number reads plausible either way.

### One observation per stall

An outage spans intervals, and the max per interval alone would count it once
per interval. A link that stops for 1.5 s and then delivers its backlog would
fold four or five peaks of a second or more, far past the 5% the quantile
needs, and a one-off would hold the target, and video with it, a second behind
live for 10 to 25 s after the link recovered. Those intervals are one event for
the same reason a queue's hundred frames are.

A frame whose delay exceeds `RESAMPLE` was due in an earlier interval than the
one it arrived in, so it is **stalled**, and `arrival - delay` is when it was
due. An interval holding a stalled frame due before the open stall's latest
stalled frame arrived is still behind that stall: it joins it instead of
folding. The empty intervals before a stalled interval belong to its stall as
well, since a link that stops outright delivers nothing until its backlog
arrives, so empty intervals fold only once the interval after them closes and
shows whether it is stalled: a recovery that delivers the live edge before its
backlog is still a stall. The stall ends at the
first interval that does not continue it, and only then folds, once, its
largest delay, then one idle fold for each other interval it spanned. So the
estimate keeps its wall-clock time constant, and the stall's single observation
does not take the weight the idle folds return to the prior, which an
observation after them would reclaim up to `FRESH_MAX`.

The test is on due time, not on adjacency, so a publisher that flushes a second
or more of media at once still counts each flush: the first frame of the next
flush was due only after the previous one arrived. Flushes back to back and
newest first do chain, since each is due before the previous one finished
arriving, and locally they look like outages one after another. So a stall
covers at most `WINDOW` of due time, the span the delay itself is measured
over, and longer lateness folds once per window as the recurring stall it is.
Recurring outages still
raise the target, a few of them within a half-life, which is the evidence a
recurring stall deserves. A lateness under one interval never stalls, so
ordinary jitter, shorter flushes, reordering, and a step in the path's delay fold one
observation per interval.

The open stall is four numbers, so a publisher that chains stalls on purpose
costs nothing. While due times advance, one fold per window of due time bounds
how long the estimate goes without evidence. Only a publisher repeating an old
timestamp holds them still, since a frame more than `WINDOW` behind the newest
is not measured, and that holds its own stall open: it freezes its own
broadcast's estimate, at no cost beyond the four numbers.

### The histogram

An empirical distribution of `BUCKETS` buckets `BUCKET` wide, holding total
Expand Down Expand Up @@ -410,7 +474,10 @@ Restarting would drop the target to the cold-start 100 ms at the switch, then
raise it as the lateness is relearned, and the playout follows both moves. Only the
reference window restarts, so the new track's frames are measured against its
own fastest. Its opening backlog would read as seconds late against the old
track's arrivals.
track's arrivals. What the outgoing track measured but has not folded, its
interval in progress and any open stall, folds into the continued histogram as
if that interval closed at the switch: a switch most often follows an outage,
and would otherwise drop the outage's one observation.

## Rise, fall, startup, and the ceiling

Expand All @@ -427,7 +494,11 @@ can hear, so the target should rise readily and fall grudgingly, and it does:
hundreds of punctual frames around it.
- **Falling** is driven by the forget factor alone, over the 20 s half-life
above. Old mass in the tail has to decay out before the quantile can move
down, and nothing can make it decay faster.
down, and nothing can make it decay faster. What keeps a one-off from holding
the target is that it puts little mass there: an outage is
[one observation](#one-observation-per-stall), however many intervals its
backlog spans, while its stalled frames fall within `WINDOW` of due time;
a longer one is an observation per window.
- **A single bad observation does not move the target on its own.** Once the
prior is spent it adds `1 - FORGET` weight, 1.7%, well under the 5% the 95th
percentile needs. It takes a few intervals of genuine lateness before the
Expand Down Expand Up @@ -478,6 +549,9 @@ the same time constant as everything else, which is what an evidence-free
interval should do, and it raises the target rather than lowering it, so the
error is on the safe side.

Idle intervals fold when the interval after them closes, before it, unless
that interval is [stalled](#one-observation-per-stall), whose stall owns them.

`IDLE_MAX` caps how many intervals are folded at once. It exists only to bound
the work: after 1024 folds the retained weight is about 2e-8, so the cap changes
no output an `f64` can represent.
Expand Down Expand Up @@ -603,6 +677,8 @@ index. Tolerance is 1e-6 ms, which the `f64` rules above make generous.
| `tunein` | A stale frame then the live edge 14.5 s later leaves the target near the frame duration. This is the #3517 regression |
| `spike` | A recurring stall raises the target, and it decays once the stalls stop |
| `straggler` | One late interval after the prior is spent leaves the target where it was |
| `batches` | Back-to-back 1 s flushes delivered newest first chain their stalls, which still fold once per window of due time and raise the target |
| `outage` | A one-off 1.5 s outage whose backlog arrives over several intervals is one stall and leaves the target where it was, while three more 3 s apart raise it |
| `fixed` | An explicit delay ignores the arrivals entirely |

`reference.ts` beside them generated the numbers. It is **not normative and not
Expand Down
1 change: 1 addition & 0 deletions doc/concept/audio-jitter/batches.json

Large diffs are not rendered by default.

53 changes: 53 additions & 0 deletions doc/concept/audio-jitter/corpus.ts
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,23 @@ function gap(trace: Trace, at: number, length: number): Trace {
return { arrival, media };
}

/**
* Hold a FIFO queue to one frame per `service` ms from `from` for `length` ms, as a link
* collapsing behind a full queue does, then drain it at 2 ms per frame.
*/
function outage(trace: Trace, from: number, length: number, service: number): Trace {
const arrival = [...trace.arrival];
let free = Number.NEGATIVE_INFINITY;
for (let i = 0; i < arrival.length; i++) {
const start = Math.max(arrival[i], free);
if (start >= from && start < from + length) free = start + service;
else if (start > arrival[i]) free = start + 2;
else free = start;
arrival[i] = free;
}
return { arrival, media: trace.media };
}

function paced(frames: number, frame: number, base: number): Trace {
return flushed(frames, frame, 1, base);
}
Expand Down Expand Up @@ -217,6 +234,42 @@ const traces: Omit<Case, "target">[] = [
return { arrival, media: trace.media };
})(),
},
{
name: "outage",
about: "Even audio with a 1.5 s outage 10 s in: a queue serving one frame per 300 ms, then draining the backlog, so its delays climb past a second across four intervals. The frames stuck behind it are one stall, which folds once, so the one-off leaves the target where it was. So does a 1.5 s blackout 20 s in, recovering with the live edge before its backlog, whose empty intervals belong to the stall that ends them. Three more outages 3 s apart from 30 s still raise it, and it decays once they stop.",
frame: FRAME,
...(() => {
let trace = paced(3000, FRAME, 50);
trace = outage(trace, 10_050, 1500, 300);
trace = outage(trace, 20_050, 1500, 1500);
// The blackout recovers with the live edge first, then the backlog behind it.
const arrival = [...trace.arrival];
const media = [...trace.media];
const [edge] = media.splice(1075, 1);
arrival.splice(1075, 1);
media.splice(1000, 0, edge);
arrival.splice(1000, 0, arrival[1000]);
trace = { arrival, media };
for (const at of [30_000, 33_000, 36_000]) trace = outage(trace, at + 50, 1500, 300);
return trace;
})(),
},
{
name: "batches",
about: "A publisher flushing 1 s of media at once, every second, newest frame first. Each flush is due before the previous one finished arriving, so the flushes chain, but a stall covers at most a window of due time, so they still fold and the target rises to cover the flush span.",
frame: FRAME,
...(() => {
const arrival: number[] = [];
const media: number[] = [];
for (let flush = 0; flush < 40; flush++) {
for (let k = 0; k < 50; k++) {
arrival.push(50 + (flush + 1) * 1000 + k * 2);
media.push(flush * 1000 + (49 - k) * FRAME);
}
}
return { arrival, media };
})(),
},
{
name: "straggler",
about: "Even audio with a single 300 ms stall 8 s in, once the prior is spent. One interval of evidence is not enough to move the target, so the stall costs that one refill instead of raising the target and then dropping it, which skips buffered audio.",
Expand Down
Loading