Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions js/watch/src/audio/decoder.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -326,6 +326,25 @@ it("decodes an audio packet inside a group after a gap", async () => {
}
});

// A subscription's head arrives newest-first. Chrome's AudioDecoder stamps every output from its
// first input, so decoding the older frames after it shifts the rest of the track.
for (const cmaf of [false, true]) {
it(`decodes a ${cmaf ? "CMAF" : "legacy"} subscription head in timestamp order`, async () => {
const playback = await play(Milli(100), true, cmaf);
try {
for (const ms of [140, 120, 100, 80, 60, 40, 20, 0, 160]) {
frameTimestamp = (ms - 20) * 1000;
playback.push(frame());
await microtasks();
}
expect(playback.inserted()).toEqual([140_000, 160_000]);
} finally {
playback.close();
await microtasks();
}
});
}

describe("Decoder across a broadcast disable", () => {
it("releases the graph when its broadcast is disabled and rebuilds on return", async () => {
const playback = await play(Milli(100));
Expand Down
20 changes: 20 additions & 0 deletions js/watch/src/audio/decoder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -476,12 +476,15 @@ export class Decoder {
};
decoder.configure(decoderConfig);

// The newest frame sent to the decoder since it was last reset.
let newest: Micro | undefined;
for (;;) {
const next = await nextMedia(consumer);
if (!next) break;
if (this.#onRead(next)) {
decoder.reset();
decoder.configure(decoderConfig);
newest = undefined;
}
if (next.kind === "gap") this.#arrivals.write(gap(next));
if (next.kind !== "frame") continue;
Expand All @@ -499,6 +502,13 @@ export class Decoder {
bytesReceived: (stats?.bytesReceived ?? 0) + frame.payload.byteLength,
}));

// Decode in timestamp order. A subscription's head groups arrive after the live edge served
// alongside them, and the ring, anchored at the first decoded frame, cannot place audio
// before it. Chrome's AudioDecoder would also stamp every later output from its first
// input, shifting the rest of the track by the reorder and overflowing a buffered ring.
if (newest !== undefined && frame.timestamp < newest) continue;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Retain playable history when the ring is reused

When a replacement subscription reuses the existing ring, such as after a rendition swap or temporary absence, its playhead can still precede the older head frames that arrive after the live edge. The ring is explicitly preserved across these replacements and can place such frames by timestamp (decoder.ts:400-404), but this check permanently discards every frame older than the first replacement frame, producing an avoidable audio hole. The comment's anchoring premise applies only to a new ring; the initial subscription head needs to be reordered before decoding, or filtered against the reused ring's playhead rather than the decoder's newest timestamp. The duplicated CMAF check has the same problem.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Real, but narrow, and decoding those frames doesn't fill the hole on Chrome either: its AudioDecoder stamps every output from its first input, so before this change the older head frames landed after the newest one, in the wrong slots, and shifted everything after them. Skipping them trades that misplacement for a gap.

On a rendition swap the predecessor was live through that span, so its audio already covers the slots below the replacement's first frame. The hole appears only on a return from an absence, where the predecessor stopped short of the new head. Even then it is bounded by the head's span, and only non-Chrome decoders (which keep input timestamps) placed those frames correctly before.

Reordering the head before decoding would hold the live edge until the older groups arrive, which defeats delivering it first. Resetting the decoder on every backward frame to make Chrome restamp would cost an Opus state reset per head frame. Neither seems worth it for this case, so I'm keeping the skip.

(Written by Claude Opus 5.5)

newest = frame.timestamp;

const ring = await this.#ready(effect);
if (!ring) break;

Expand Down Expand Up @@ -570,6 +580,8 @@ export class Decoder {
};
decoder.configure(decoderConfig);

// The newest frame sent to the decoder since it was last reset.
let newest: Micro | undefined;
for (;;) {
const next = await nextMedia(consumer);
if (!next) break;
Expand All @@ -578,6 +590,7 @@ export class Decoder {
if (this.#onRead(next)) {
decoder.reset();
decoder.configure(decoderConfig);
newest = undefined;
}
if (next.kind === "gap") this.#arrivals.write(gap(next));
if (next.kind !== "frame") continue;
Expand All @@ -594,6 +607,13 @@ export class Decoder {
bytesReceived: (stats?.bytesReceived ?? 0) + frame.payload.byteLength,
}));

// Decode in timestamp order. A subscription's head groups arrive after the live edge served
// alongside them, and the ring, anchored at the first decoded frame, cannot place audio
// before it. Chrome's AudioDecoder would also stamp every later output from its first
// input, shifting the rest of the track by the reorder and overflowing a buffered ring.
if (newest !== undefined && frame.timestamp < newest) continue;
newest = frame.timestamp;

const ring = await this.#ready(effect);
if (!ring) break;

Expand Down
Loading