Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 70 additions & 7 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 8 additions & 1 deletion doc/bin/gstreamer.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,10 @@ and stereo use a family 0 head, and three to eight channels need
`channel-mapping-family`, `stream-count`, `coupled-count`, and
`channel-mapping`. Caps that omit that mapping, or that contradict the header,
are refused. `opusenc` supplies both, so a 5.1 encode plays as six channels.
Opus buffers are read the way `opusenc` and `moqsrc` stamp them: at the time
their first kept sample plays, with `GstAudioClippingMeta` clipping the
pre-skip. Each is published the pre-skip, less its own clipping, later, since
a frame timestamp counts the pre-skip.

A `text/x-raw` pad is how captions get in: ffmpeg cannot mux a subtitle track
into fragmented MP4, so `moq import fmp4` can't carry one, while a demuxer that
Expand Down Expand Up @@ -134,7 +138,10 @@ every returning rendition; one it omits gets EOS instead of keeping its old pad.
Within each run, the first media frame fixes one timestamp reference and segment
base for every pad. Audio and video keep their relative broadcast timing, even
when their tracks use different timescales or arrive at different times. Frames
before that reference keep an earlier PTS for downstream to clip. A new broadcast
before that reference keep an earlier PTS for downstream to clip. Opus packets
covering the OpusHead pre-skip carry `GstAudioClippingMeta` for it, and each
Opus packet that keeps samples is stamped where the first one plays, so `opusdec`
drops the priming samples and the audio lines up with video. A new broadcast
chooses a fresh reference on the same pads; playback does not require a wall clock.
A rendition that leaves the catalog
gets EOS once its track ends. A broadcast that ends or loses its publisher holds
Expand Down
3 changes: 3 additions & 0 deletions drafts/draft-lcurley-moq-hang.md
Original file line number Diff line number Diff line change
Expand Up @@ -417,6 +417,8 @@ type AudioDecoderConfigExtensions = {
A consumer MUST reject `ambisonic` with fewer than 4 channels or a count that is not a square, and MUST treat an unrecognized `layout` as absent.
`layout` is not part of the track's identity.

An Opus frame timestamp counts the samples the OpusHead pre-skip trims: a consumer presents each decoded sample the pre-skip, at 48 kHz, earlier than its frame timestamp places it, so the first sample kept plays at the first decoded frame's timestamp.

### PCM {#audio-pcm}

Hang defines the `"pcm"` audio codec for uncompressed samples.
Expand Down Expand Up @@ -1206,6 +1208,7 @@ This document has no IANA actions.
- Added an optional `epoch` beside every rendition and data track `broadcast` reference, and `replayEpoch` beside the archive's `replay` ({{field-epoch}}). A consumer reads each referenced path from the first broadcast it resolves for a catalog instance; a present `epoch` fails a rendition proven to resolve another instance, and is best effort over a route without an Epoch.
- Derived HLS and DASH exports hand disabled references to enabled renditions at the first record numbered past the advertised ones, keeping the record `sequence` as the HLS media sequence and starting a DASH period.
- Added optional `stats` and `echo` catalog sections, publisher/viewer snapshots keyed by rendition ID, and the `.echo` feedback broadcast convention.
- An Opus frame timestamp counts the OpusHead pre-skip: a consumer presents each decoded sample the pre-skip earlier than its frame timestamp places it.

- Fixed a track's identity for its name ({{identity}}): codec, description, container, display aspect, sample rate, and channel count never change, while every other field, including size, bitrate, and framerate, updates in place. A change of identity takes a new track name, which a rendition may name with `track` while keeping its ID, or a new Epoch.
- Specified MSF text metadata preservation and refusal of unrepresentable text packaging during gateway conversion.
Expand Down
2 changes: 1 addition & 1 deletion quest/m1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,7 @@ audit's receive-memory, priority, and lease bugs rank ahead of that reshape.
- [More tests under load](/quest/m1/test-flakes-2/README.md) - the second round of load-only failures, one quest per flake, fixed at the cause
- [Broadcast lifecycle](/quest/m1/broadcast-lifecycle/README.md) - broadcasts are online or offline with no finished state in moq-net, finished-broadcast retention is a fixed moq-mux policy, and each linger timer has one meaning
- [JS interop tone](/quest/m1/interop-js-tone.md) - the `js ->` cells pass the audio check on every draft again, after the 2026-10-23 regression
- [fMP4 edit lists](/quest/m1/fmp4-edit-lists.md) - the fMP4 importer applies edit lists to frame timestamps, so the interop AAC tone passes the phase check
- [A smaller, safer API than release](/quest/m1/api-audit/README.md) - every surface added since release has a consumer or is gone, overlapping operations are collapsed, names mirror across Rust, JS, and the bindings, and every break says how to migrate
- [moq-quic's API improves on quinn's](/quest/m1/quic-api/README.md) - moq-quic and both QUIC drivers ship one mirrored API: `send::`/`recv::` namespaces, one error per direction, consuming terminal operations, plain configs, no quinn baggage
- [JavaScript FETCH](/quest/m1/js-fetch.md) - browser publishers answer IETF FETCH through the JS ranges request surface
Expand All @@ -143,7 +144,6 @@ audit's receive-memory, priority, and lease bugs rank ahead of that reshape.
- [Producers drop the phantom catalog type](/quest/m1/catalog-phantom.md) - moq-video, moq-audio and moq-mux producers lose an `E: CatalogExt` parameter that only `PhantomData` holds
- [JSON encoder options without a downcast](/quest/m1/json-encoder-options.md) - moq-mux's `json_snapshot` takes its `deltas` switch through a typed path any custom catalog entry can use, not an `Any` downcast
- [CI housekeeping](/quest/m1/ci-housekeeping.md) - validate queue nextest at 24 threads and remove the actionlint queue ignore after its release
- [Interop pre-skip](/quest/m1/interop-preskip.md) - the interop tone check anchors phase to timestamps, so a misapplied pre-skip fails
- [Main is no slower than release](/quest/m1/release-perf/README.md) - every benchmark where main is slower than release is fixed or accepted with a reason, checked by a final paired run before the cut
- [Subscribe update camera check](/quest/m1/subscribe-update-churn.md) - a person checks a physical-camera publish for SUBSCRIBE_UPDATE chatter; bbb and the fake camera measured quiet
- [Unprefixed C++ free functions](/quest/m1/cpp-free-function-prefix.md) - generated free functions drop `moq_`, and `moq.hpp` deletes its five pass-through wrappers before C++ first publishes
Expand Down
20 changes: 16 additions & 4 deletions quest/m2/fmp4-edit-lists.md → quest/m1/fmp4-edit-lists.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,35 @@

An fMP4 source with an edit list (AAC priming, a non-zero `media_time`, or an
empty edit) publishes frames at their presentation time, so both players and
every export see the timeline the source meant.
every export see the timeline the source meant. The interop AAC rendition
then passes `tone.py --phase` like Opus does.

## Plan

Today `edts` is forwarded in the catalog init but read by no Rust or JS code,
and `rs/moq-mux/src/container/fmp4/export.rs` strips it on the way out. Both
players present `tfdt` plus the composition offset. ffmpeg's fragmented
output avoids `elst` by default, so this mostly affects GPAC, Bento4, and
Shaka sources (2026-10-07 audit).
output does write `elst`: with the interop's `-movflags cmaf`, its AAC track
carries `media_time` 1024 at `tfdt` 0, so every subscriber plays the AAC tone
1024 samples late (`-880us` at 440 Hz once `--phase` checks it), and its Opus
track carries `media_time` equal to the `dOps` pre-skip.

Decided (2026-10-07): the importer applies the edit list to each frame's
timestamp and drops `edts` from the catalog init, so no consumer needs edit
list code. This builds on decoders taking time from the frame timestamp
rather than `tfdt`. Decide in the PR whether priming samples before the edit
are dropped or published with negative-offset handling, and refuse edit
are dropped or published with negative-offset handling (Apple's 2112-sample
AAC priming is not a whole number of 1024-sample frames), and refuse edit
lists with more than one non-empty entry rather than guess.

Opus frame timestamps count the OpusHead pre-skip (`draft-lcurley-moq-hang`),
so for Opus the importer applies only the part of the edit beyond the
pre-skip, or the pre-skip is trimmed twice.

Once AAC lines up, drop the `opus` condition from `phase_checked` in
`test/interop/interop.sh` and the matching sentence in
`test/interop/README.md`.

Public API: none. Wire: none, but a behavior change: frame timestamps now
include the edit and the init loses `edts`, so a consumer that offsets AAC
priming on its own must stop, or it shifts twice. Note it in the changelog.
Expand Down
32 changes: 0 additions & 32 deletions quest/m1/interop-preskip.md

This file was deleted.

1 change: 0 additions & 1 deletion quest/m2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,6 @@ lite-07; bitrate enforcement is not part of this release.
- [Drop announce="never"](/quest/m2/publish-announce-never.md) - `@moq/publish` offers only announce modes a watcher can reach
- [moqsrc A/V sync](/quest/m2/moqsrc-av-sync.md) - a nightly test measures A/V skew from `moqsrc` through clock-synced sinks
- [MP4 export](/quest/m2/mp4-export.md) - `moq export mp4 --output` records crash-safe fragments, then finishes a regular MP4 with moov at the end
- [fMP4 edit lists](/quest/m2/fmp4-edit-lists.md) - the fMP4 importer applies edit lists to frame timestamps
- [Importers fill the VR fields](/quest/m2/mux-vr-fields.md) - MP4, Matroska, and Opus imports set the catalog's projection, stereo, and ambisonic layout
- [Linux decoded frames](/quest/m2/obs-decode-linux.md) - present supported native decoded surfaces with visible CPU fallback
- [Windows decoded frames](/quest/m2/obs-decode-windows.md) - present decoded D3D11 surfaces in OBS without CPU readback
Expand Down
1 change: 1 addition & 0 deletions rs/moq-gst/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ crate-type = ["cdylib", "rlib"]
anyhow = { workspace = true, features = ["backtrace"] }
bytes = { workspace = true }
gst = { package = "gstreamer", version = "0.25" }
gst-audio = { package = "gstreamer-audio", version = "0.25" }
hang = { workspace = true }
moq-clock = { workspace = true }
moq-mux = { workspace = true }
Expand Down
2 changes: 1 addition & 1 deletion rs/moq-gst/src/sink/imp.rs
Original file line number Diff line number Diff line change
Expand Up @@ -715,7 +715,6 @@ impl MoqSink {
// Map and copy outside the lock so the per-pad lock covers only the producer write. Avoiding the
// copy for an oversized buffer needs a reliable, media-aware size heuristic; moq-net remains the
// authority and rejects FrameTooLarge before reserving its own group slot.
let pts = buffer.pts();
// Only subtitles use this: a cue needs an explicit end, unlike a media frame.
let duration = buffer.duration();
let current_running_time = self.obj().current_running_time();
Expand Down Expand Up @@ -756,6 +755,7 @@ impl MoqSink {
return Ok(gst::FlowSuccess::Ok);
}
let outcome = lifecycle.media.place_wall(&buffer, sample).and_then(|()| {
let pts = lifecycle.media.timeline_pts(&buffer)?;
lifecycle
.media
.push_buffer(data, pts, duration, current_running_time, moq_net::time::Instant::now())
Expand Down
Loading
Loading