Skip to content
2 changes: 1 addition & 1 deletion doc/bin/hls.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ What that means for you:
- **Every rendition shares the grid.** A group spans from its start timestamp to the next group's and belongs to the first grid line it contains, or, containing none, the line it starts after. Segment N therefore starts at the group containing N times the segment length, every rendition numbers its segments alike, and each cuts its boundaries at its own keyframes. A grid line no group starts a segment at, because a GOP ran past it or the content paused, is listed as an `EXT-X-GAP` so the numbering never skips: on a 2 s grid, a GOP from 0 to 7 s is segment 0, segments 1 and 2 are gaps, and the next GOP starts segment 3. Timestamps that go back start a discontinuity on that rendition alone; `EXT-X-PROGRAM-DATE-TIME`, written after every discontinuity when the catalog has a clock, lets players resync.
- **Include media starts on selection.** The HTTP multivariant playlist reads each root `include` entry's catalog and lists its audio and video beside the broadcast's own. An include shares the broadcast's timeline: its renditions are segmented on the broadcast's join interval and clock, whatever its own catalog declares, so publish it on the same timestamps. One not yet routed is listed once its broadcast is announced. Reading its catalog starts no encoding: only a request to one of its rendition URLs subscribes to that rendition's media. Demand lasts through active responses and, after the last one finishes, for three target durations or the playlist window plus one target duration, whichever is longer (20 s by default); a later request starts it again. The broadcast's own renditions stay subscribed. A segment is servable for as long as the relay's [cache](/bin/relay/config#cache) retains it. Keep `--window` (the playlist length, default 16 s) within that retention and within the media's max age (30 s by default), after which a superseded segment's groups are stale.
- **Live joins cost the window.** The exporter subscribes 30 s behind the live edge and lists at most 256 segments. One joining partway through a segment lists nothing before the first segment it sees start.
- **Segments are timed by the grid.** A segment is listed once a later one starts or the track ends. A GOP that starts off a line is placed as soon as one of its frames passes the next line, without waiting for the next keyframe, so the segment before it lists then. A keyframe may overlap the previous GOP's last frames (a TS content join does), so when that overlap straddles a line, an edge that saw the earlier GOP lists it in the line's segment, while an edge that joined at the overlapping keyframe lists that segment without it. This one segment differs across edges, for that broadcast only (accepted, 2026-11-15). It starts at its first group's timestamp, or at its line when that group starts after it (as one resuming from a pause does), or a gap at its line, and lasts until the next segment starts, so a segment's bytes never change with a catalog estimate. A GOP longer than the segment (an overrun) is listed at its first line, so its duration understates its media and the gaps after it overstate theirs, while the playlist timeline still meets the media again at the next segment. That is this exporter's policy, not the draft's: no listing is accurate, since an `EXTINF` above `EXT-X-TARGETDURATION` is forbidden (RFC 8216 section 4.3.3.1) and the target can't change within a playlist (section 6.2.1).
- **Segments are timed by the grid.** A segment is listed once a later one starts or the track ends. A GOP that starts off a line is placed as soon as one of its frames passes the next line, without waiting for the next keyframe, so the segment before it lists then. It starts at its first group's timestamp, or at its line when that group starts after it (as one resuming from a pause does), or a gap at its line, and lasts until the next segment starts, so a segment's bytes never change with a catalog estimate. A GOP longer than the segment (an overrun) is listed at its first line, so its duration understates its media and the gaps after it overstate theirs, while the playlist timeline still meets the media again at the next segment. That is this exporter's policy, not the draft's: no listing is accurate, since an `EXTINF` above `EXT-X-TARGETDURATION` is forbidden (RFC 8216 section 4.3.3.1) and the target can't change within a playlist (section 6.2.1).
- **The target duration is twice the segment.** Every media playlist keeps one `EXT-X-TARGETDURATION` while its rendition stays in the catalog, even across a reconfigure: twice the segment length, rounded up to whole seconds (twice the interval for a GOP of a second or more, 2 s for 20 ms audio groups), since the GOP containing a grid line starts at most one interval before it, and a gap ends under two segment lengths after its line. Only a final GOP longer than that is listed past it, against RFC 8216's rule: hls.js plays it, but Apple's `mediastreamvalidator` flags it, so keep keyframe intervals within the declared interval.
- **A reconfigure keeps the playlist.** A rendition whose decoder config changes (its resolution, say) keeps its media playlist: the segments already listed stay, under the init they were muxed for, and the new config's segments follow behind an `EXT-X-DISCONTINUITY` and a new `EXT-X-MAP`, as RFC 8216 requires. The old config's segments end at the keyframe declaring the new size, even when that group reaches the export before the catalog update. A resize that opens a segment lists that segment under the new init. One partway through a segment cuts the old config's last segment short there, and the new config lists from the next segment, so the media between them is never listed. The DASH manifest lists only the new config's segments.
- **Disabled renditions are hidden.** A rendition the catalog marks `enabled: false` leaves the multivariant playlist and DASH manifest until it re-enables. Its media playlist keeps serving, so a player already on it is not ended with `EXT-X-ENDLIST`.
Expand Down
10 changes: 5 additions & 5 deletions drafts/draft-lcurley-moq-hang.md
Original file line number Diff line number Diff line change
Expand Up @@ -907,7 +907,7 @@ Every container shares the same group rules:

Each group MUST start with a keyframe, except a group that contains only a discontinuity marker.
If the codec does not support delta frames (e.g. audio), a group MAY consist of multiple keyframes.
Otherwise, a group MUST consist of a single keyframe followed by zero or more delta frames.
Otherwise, a group MUST consist of a keyframe followed by zero or more frames that decode from the frames before them in the group.

A group with no decodable frames is a walk-now discontinuity: one empty codec payload and no media.
An empty payload is data on a data track, so in every container a data track's marker is one object with no bytes, which is no frame; it ends at the furthest point the consumer has read.
Expand All @@ -916,9 +916,9 @@ A consumer MUST NOT submit the marker to a decoder.
Empty groups (zero objects) are permitted and mean nothing.
After a discontinuity the timeline continues forward.
A publisher that stops producing and may resume on the same track (e.g. an encoder idle for lack of demand) SHOULD publish a discontinuity marker when it stops, so the group before the pause does not reach across the gap and read as live.
The first timestamp of a group MUST NOT be lower than the first timestamp of any group with a lower start, and no frame of a group may be lower than the first timestamp of the group before it; a group that breaks either is malformed.
A group MAY start at the same timestamp as the previous group.
Frames MAY present below the previous group's content (B-frames, or a keyframe overlapping the previous group's last frame).
Every frame of a group MUST present after every frame of any group with a lower start; a group that breaks this is malformed.
Frames within a group MAY present out of order (B-frames, open-GOP leading pictures), so a consumer bounds a group that isn't whole by the lowest timestamp it knows in a higher group.
A publisher splicing in new content keeps this by continuing the current group at a keyframe that doesn't present after the group's newest frame, and by dropping a later frame that presents at or before the previous group's newest frame, such as a leading picture that references the old content.
A publisher whose source restarts below the previous group's start begins a new broadcast instead.
A hole between delivered groups is a playhead event unless the publisher declared the starts between them skipped or the boundary is contiguous within 1 ms.
A consumer re-applies startup delay and skip at a playhead event; it does not reset codec state.
Expand Down Expand Up @@ -1084,7 +1084,7 @@ This document has no IANA actions.
- A `ttml` payload is one complete TTML document in the `media` time base, clipped to its frame: its `body` begin matches the frame timestamp, and its `body` end bounds what it presents.
- Specified MSF text metadata preservation and refusal of unrepresentable text packaging during gateway conversion.
- Added optional `bitrate`, `jitter`, and `delay` fields to `json` and `binary` track entries, `delay` measured only from payloads stamped with their capture time on the broadcast clock.
- Replaced the live-edge floor with monotonic group starts: a group may overlap the previous group's content but not start before it.
- Replaced the live-edge floor with groups that never overlap: every frame of a group presents after every frame of a lower group, while frames within a group may reorder. A spliced keyframe that doesn't present after its group's newest frame continues that group.
- A data track marks a discontinuity with a group of one object with no bytes, since its empty payload is data. An object with no bytes on an audio or video track, or sharing its group, is malformed.
- Timed CMAF samples from the frame timestamp, which is the fragment's earliest presentation time; `tfdt` is only relative within the fragment, and places the samples of an untimed frame. A timed CMAF track counts in its `mdhd` ticks, and a fragment's runs lie back to back in its `mdat`, each `data_offset` counting from its `moof`.
- A `loc` frame's codec payload uses the `legacy` framing.
Expand Down
22 changes: 3 additions & 19 deletions js/hang/src/container/consumer.rewind.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ test("Consumer accepts a resuming keyframe before the estimated group end", asyn
});

for (const start of [16_000, 0]) {
test(`Producer and Consumer accept the next group starting at ${start}us`, async () => {
test(`Producer continues the open group at a keyframe at ${start}us`, async () => {
const { track, consumer } = setup();
const producer = new Producer(track, new Format("video"));
try {
Expand All @@ -80,31 +80,15 @@ for (const start of [16_000, 0]) {
expect((await frame(consumer))?.group).toBe(0n);
await frame(consumer);
producer.encode(new Uint8Array([1]), Duration.micros(start), true);
// A keyframe that doesn't start past the open group continues it.
expect((await frame(consumer))?.group).toBe(BigInt(start));
// A keyframe that doesn't start past the newest frame continues the open group.
expect((await frame(consumer))?.group).toBe(0n);
} finally {
consumer.close();
producer.finish();
}
});
}

test("Consumer accepts leading pictures below the previous group's content", async () => {
const { track, consumer } = setup();
try {
group(track, [100_000, 180_000]);
await frame(consumer);
await frame(consumer);
group(track, [200_000, 150_000]);
expect((await frame(consumer))?.frame?.timestamp.asMicros()).toBe(200_000);
expect((await frame(consumer))?.frame?.timestamp.asMicros()).toBe(150_000);
} finally {
consumer.close();
track.finish();
track.close();
}
});

test("Consumer rejects a later frame below the previous group start", async () => {
const { track, consumer } = setup();
try {
Expand Down
41 changes: 41 additions & 0 deletions js/hang/src/container/consumer.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,47 @@ test("Legacy Producer accepts open-GOP leading pictures above the previous group
producer.finish();
});

test("Legacy Producer continues a group at a keyframe overlapping it, and drops a leading picture before it", async () => {
const track = micro("test");
const subscriber = track.subscribe({ delay: { max: Duration.millis(30_000) } });
const producer = new LegacyProducer(track, new LegacyFormat("video"));
for (const [timestamp, keyframe] of [
[100_000, true],
[166_000, false],
[133_000, false],
// A splice dipping below the newest frame continues the group.
[150_000, true],
[183_000, false],
// Past the newest frame: a new group, whose leading picture at 183_000 presents with the last.
[200_000, true],
[183_000, false],
[190_000, false],
] as const) {
producer.encode(new Uint8Array([1]), Duration.micros(timestamp), keyframe);
}
producer.finish();
const groups = (await readGroups(subscriber, 200_000n)).map(([, frames]) =>
frames.filter(([, size]) => size > 0).map(([timestamp]) => timestamp),
);
expect(groups).toEqual([
[100_000, 166_000, 133_000, 150_000, 183_000],
[200_000, 190_000],
]);
});

test("Legacy Producer drops a keyframe overlapping a closed group", () => {
const track = micro("test");
const producer = new LegacyProducer(track, new LegacyFormat("video"));
producer.encode(new Uint8Array([1]), Duration.micros(100_000), true);
producer.encode(new Uint8Array([1]), Duration.micros(166_000), false);
producer.discontinuity(Duration.micros(200_000));
const largest = track.largest()?.group;
producer.encode(new Uint8Array([1]), Duration.micros(150_000), true);
producer.encode(new Uint8Array([1]), Duration.micros(183_000), false);
expect(track.largest()?.group).toBe(largest);
producer.finish();
});

test("Legacy Producer writes a duration marker at the next keyframe", async () => {
const track = micro("test");
const subscriber = track.subscribe({ delay: { max: Duration.millis(30_000) } });
Expand Down
42 changes: 33 additions & 9 deletions js/hang/src/container/legacy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -147,9 +147,15 @@ export class Producer {
#end?: number;
// Furthest timestamp in finished groups, used only for discontinuity markers.
#liveEdge?: number;
// The current group start and the previous group start, which bounds its frames.
// The current group start and the previous group start: a frame below it is a restart.
#start?: number;
#floor?: number;
// The newest frame of every group before the current one: every frame of the current group
// presents after it.
#ceiling?: number;
// A keyframe was dropped with no group open, so the frames decoding from it go too, until a
// keyframe opens a group.
#skipping = false;
// Gap between consecutive timestamps, used to close the last group when no successor exists.
#interval?: number;
// A discontinuity's marker is the newest group, so another one would say nothing new.
Expand All @@ -163,31 +169,49 @@ export class Producer {
this.#track = track;
}

/** Encode and append a frame; a keyframe starts a new group. Throws if the first frame is not a keyframe, a group start goes backwards, or a frame is below the previous group start. */
/**
* Encode and append a frame; a keyframe starts a new group. Every frame of a group presents after
* every frame of the groups before it, so a keyframe that doesn't start after the newest frame
* continues the open group (a splice), and a later frame presenting at or before the groups before
* its own (a leading picture after a splice) is dropped, as is a keyframe at or before the newest
* frame with no group open, with the frames after it until a keyframe opens a group. Throws if the
* first frame is not a keyframe, a group start goes backwards, or a frame is below the previous
* group start.
*/
encode(data: Uint8Array | Source, pts: Duration, keyframe: boolean) {
const timestamp = micros(pts);
const floor = keyframe ? this.#start : this.#floor;
if (floor !== undefined && timestamp < floor) {
throw new Error("frame timestamp is below the previous group start");
}
// A keyframe opens a group starting at its timestamp, which has to start after every group the
// track holds. One that doesn't, such as parameter sets and the picture they precede written
// as two keyframes at one timestamp, continues the open group instead. With none open it is
// A keyframe opens a group starting at its timestamp, which has to start after every frame
// written and every group the track holds. One that doesn't, such as parameter sets and the
// picture they precede written as two keyframes at one timestamp, or a splice dipping below
// the old content's last frames, continues the open group instead. With none open it is
// refused, before anything changes.
const opens = keyframe && startsPast(this.#track, timestamp);
const behind = this.#newest !== undefined && timestamp <= this.#newest;
const opens = keyframe && !behind && startsPast(this.#track, timestamp);
// Dropped: a keyframe with no group to continue, and a frame presenting with the groups
// before its own, such as a leading picture after a splice.
if (keyframe && behind && !this.#group) {
this.#skipping = true;
return;
}
if (!keyframe && !this.#group && this.#skipping) return;
if (keyframe && !opens && !this.#group) throw new Error("a keyframe must start after the previous group");
if (!opens && this.#group && this.#ceiling !== undefined && timestamp <= this.#ceiling) return;
this.#marked = false;
if (opens) {
const rewound = this.#previous !== undefined && timestamp < this.#previous;
this.#close(rewound ? undefined : timestamp);
if (rewound) this.#interval = undefined;
this.#close(timestamp);
// A track's first group says nothing came before it, so a reader can tell it is the first.
if (this.#track.largest() === undefined) {
const start = keyOf(this.#track, timestamp);
if (start > 0n) this.#track.skipGroups(0n, start);
}
this.#group = this.#track.appendGroup(Time.Timestamp.fromMicros(timestamp));
this.#floor = this.#start;
this.#ceiling = this.#newest;
this.#skipping = false;
this.#start = timestamp;
} else if (!this.#group) {
throw new Error("must start with a keyframe");
Expand Down
Loading