Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 8 additions & 5 deletions doc/concept/hang.md
Original file line number Diff line number Diff line change
Expand Up @@ -214,11 +214,14 @@ independent, so an audio group can end at any frame; the first-party encoders
put one packet in each unless told otherwise.

The `description` field carries out-of-band codec setup (an `avcC` box for
H.264). When it is absent, the parameter sets ride inline before each keyframe,
which is what `avc3`/`hev1` tracks do. Decoders should handle both. CMAF is the
exception: its samples are always length-prefixed, so an `avc3`/`hev1` CMAF
track keeps the configuration record as its `description` for the NAL length
size, even when the parameter sets ride in the samples.
H.264) and decides the framing. With one, H.264 and H.265 frames are
length-prefixed by its NAL length size; without one, they are Annex-B and carry
their parameter sets before each keyframe. The codec string only says where the
parameter sets live: `avc1`/`hvc1` keep them in the `description`, while
`avc3`/`hev1` may repeat them in band, with or without one. Decoders should
handle every combination. CMAF samples are always length-prefixed, so a CMAF
track always carries the configuration record, even when the parameter sets
ride in the samples.

## Your own format

Expand Down
8 changes: 5 additions & 3 deletions js/publish/src/video/encoder.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,7 @@ for (const stage of ["support", "flush"] as const) {
if (stage === "flush") expect(FakeVideoEncoder.instances[0]?.state).toBe("closed");
enabled.set(true);
await settle();
expect(encoder.out.catalog.peek()?.codec).toStartWith("avc1");
expect(encoder.out.catalog.peek()?.codec).toStartWith("avc3");
// The abandoned browser promise still has not resolved.
expect(held.mock.calls.length).toBeGreaterThan(1);
} finally {
Expand Down Expand Up @@ -949,10 +949,12 @@ test("the catalog advertises the codec string a probe encode reports", async ()
encoder.config.set({ codec: "avc1" });
await settle();
expect(encoder.out.resolved.peek()?.codec).toBe("avc1.640028");
expect(encoder.out.catalog.peek()?.codec).toBe("avc1.64001F");
// Annex B carries its parameter sets in band, so the catalog says avc3 rather than the
// avc1 the encoder reports.
expect(encoder.out.catalog.peek()?.codec).toBe("avc3.64001F");

// The configured strings were never advertised, not even briefly.
expect([...new Set(advertised)]).toEqual(["vp09.00.31.08", "avc1.64001F"]);
expect([...new Set(advertised)]).toEqual(["vp09.00.31.08", "avc3.64001F"]);
} finally {
unsubscribe();
encoder.close();
Expand Down
15 changes: 13 additions & 2 deletions js/publish/src/video/encoder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -973,7 +973,8 @@ type Probe = {
// The config the browser accepted and encoded a probe frame with. Its bitrate is the ceiling,
// before any bandwidth grant.
config: VideoEncoderConfig & { framerate: number; bitrate: number };
// The codec string the encoder reported for it, which the catalog advertises.
// The codec string the encoder reported for it, as the catalog advertises it: `avc3`/`hev1` for
// Annex B.
reported: string;
};

Expand Down Expand Up @@ -1102,7 +1103,17 @@ async function probeEncoder(
if (requestsAnnexB(config.codec) && (described || !(first && startsWithStartCode(first)))) {
return { refused: "its encoder ignored the requested Annex B format" };
}
return { reported };
return { reported: requestsAnnexB(config.codec) ? inBand(reported) : reported };
}

// The codec string for Annex B output, which carries its parameter sets in band: `avc3`/`hev1`
// rather than the `avc1`/`hvc1` an encoder reports, which would say they live in a description the
// catalog doesn't carry. Only the catalog changes: Safari refuses `avc3` as an encoder string, and the
// watcher opens an `avc3` rendition as `avc1` where it must.
function inBand(codec: string): string {
if (codec.startsWith("avc1.")) return `avc3.${codec.slice(5)}`;
if (codec.startsWith("hvc1.")) return `hev1.${codec.slice(5)}`;
return codec;
}

// The codecs the probe configures for Annex B output.
Expand Down
81 changes: 0 additions & 81 deletions quest/m1/video-framing.md

This file was deleted.

8 changes: 2 additions & 6 deletions quest/m2/video-format-names.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ Decided 2026-11-17 (maintainer, `/quest-import` of
`avc3`/`hev1` say where parameter sets live, not how NAL units are framed:
inside a frame, NAL units are still split by start codes or length
prefixes. Naming the Annex B input after them taught the conflation that
became the native decoder bug in
[video-framing](/quest/m1/video-framing.md). The names follow ffmpeg's raw
became the native decoder bug fixed in
[moq-dev/moq#9066](https://github.com/moq-dev/moq/pull/9066). The names follow ffmpeg's raw
`h264` / `hevc` formats; `h264` already aliases `avc3`.

Where the names live: `rs/moq-mux/src/import/format.rs` (the enum, its doc
Expand All @@ -30,7 +30,3 @@ No alias for the old names.
The catalog's codec strings are not import formats and keep `avc3`/`hev1`.

Public API: breaking, as above. Wire: none.

## Required

- [Video framing](/quest/m1/video-framing.md) - settles the framing contract the new names and docs describe
7 changes: 5 additions & 2 deletions rs/hang/src/catalog/video/h264.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,11 @@ use crate::Error;
#[serde_with::serde_as]
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq)]
pub struct H264 {
/// If true, SPS/PPS are inline in the bitstream (avc3).
/// If false, they must be in the description (avc1).
/// If true (avc3), SPS/PPS may also ride in the bitstream, beside any description.
/// If false (avc1), they live only in the description.
///
/// Says nothing about the NAL framing, which the description decides: with one the
/// NAL units are length-prefixed, without one they are Annex-B.
#[serde(default)]
pub inline: bool,
/// The H.264 profile (e.g., 0x42 for Baseline, 0x4D for Main, 0x64 for High)
Expand Down
9 changes: 5 additions & 4 deletions rs/hang/src/catalog/video/h265.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,13 @@ use crate::Error;
/// H.265/HEVC codec mimetype.
///
/// This struct contains the profile, tier, level, and constraint information
/// needed to identify a specific H.265 variant. The `in_band` flag determines
/// whether parameter sets are included in-band (hev1) or out-of-band (hvc1).
/// needed to identify a specific H.265 variant. The `in_band` flag says whether
/// parameter sets may ride in band (hev1) or live only in the description (hvc1);
/// the description, not the flag, decides the NAL framing.
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq)]
pub struct H265 {
/// If true (hev1), then the SPS/PPS/etc are in the same NAL unit as the IDR.
/// If false (hvc1), then the SPS/PPS/etc are in the description.
/// If true (hev1), the VPS/SPS/PPS may also ride in the bitstream ahead of a keyframe.
/// If false (hvc1), they live only in the description.
pub in_band: bool,

/// Profile space (0 for main profile space, 1-3 for other spaces)
Expand Down
3 changes: 2 additions & 1 deletion rs/moq-cli/src/subscribe.rs
Original file line number Diff line number Diff line change
Expand Up @@ -501,7 +501,8 @@ impl Subscribe {
/// Write the selected TS media to a writer after resolving its route.
pub(super) async fn write_ts(self, mut stdout: impl tokio::io::AsyncWrite + Unpin) -> anyhow::Result<()> {
// TS emits PAT/PMT then a continuous PES stream (re-emitting PAT/PMT at
// keyframes for tune-in). Avc3/Hev1 sources pass through as Annex-B; AAC
// keyframes for tune-in). Annex-B sources (no description) pass through;
// length-prefixed ones are rewritten to Annex-B; AAC
// is re-framed as ADTS. `fragment_duration` does not apply to TS. The catalog stream
// selects the `mpegts` catalog extension so undecoded elementary streams
// (SCTE-35, teletext, DVB AC-3, ...) are re-emitted verbatim on their PIDs.
Expand Down
7 changes: 4 additions & 3 deletions rs/moq-ffi/src/test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10052,10 +10052,11 @@ async fn video_subscribe_configuration_boundary() {
let consumer = broadcast.consume().unwrap();
for (codec, description, supported) in [
("avc1.42001e", None, true),
("avc1.42001e", Some(vec![]), false),
// An empty description is no description: Annex-B framing.
("avc1.42001e", Some(vec![]), true),
("avc1.42001e", Some(vec![1, 0x42, 0, 0x1e]), false),
// hvc1 requires hvcC before any backend is opened, even on HEVC-capable hosts.
("hvc1.1.6.L93.B0", None, false),
// A malformed hvcC is refused before any backend is opened, even on HEVC-capable hosts.
("hvc1.1.6.L93.B0", Some(vec![1, 2, 3]), false),
("vp09.02.10.10", None, false),
("avc1.invalid", None, false),
("unknown", None, false),
Expand Down
Loading