Skip to content
Merged
2 changes: 2 additions & 0 deletions quest/m1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,8 @@ bench all require its crate.
- [Audio warmup](/quest/m1/audio-warmup.md) - a viewer joining an Opus rendition mid-stream never hears the unconverged first 80 ms
- [#3021](/quest/m1/3021-moq-gst-anchor-generated-media-timelines-to-wall-clock.md) - moq-gst picks the broadcast wall epoch; a restarted source is a new epoch, not a forward re-anchor
- [TS stitch catalog bound](/quest/m1/ts-follow-catalog-bound.md) - `--linger` bounds a mid-stream `--stitch` until the replacement's catalog arrives, so it ends loudly instead of stalling
- [moqsink first catalog](/quest/m1/gst-first-catalog.md) - a restarted `moqsink` lists every requested pad in its first catalog, so `moqsrc` resumes each on its held pad
- [moqsrc A/V alignment](/quest/m1/gst-src-av-sync.md) - every `moqsrc` pad of a run shares one timestamp reference and segment base
- [TS passthrough export](/quest/m1/ts-passthrough-export.md) - `export ts --passthrough` writes the `m2ts` track back byte-identical (less late drops) on a fixed delay
- [FLV and MKV export delay](/quest/m1/export-delay.md) - FLV and MKV interleave through the shared jitter buffer on a fixed delay, breaking the CLI once
- [MKV lacing](/quest/m1/mkv-lacing.md) - laced MKV blocks import as one timed frame each, refused without DefaultDuration
Expand Down
84 changes: 84 additions & 0 deletions quest/m1/gst-first-catalog.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# [S] moqsink's first catalog lists every requested pad

## Goal

When a `moqsink` run starts, its first catalog lists every pad requested so
far, not only the pads whose caps arrived first. A `moqsrc` following a
restarted `moqsink` then resumes each rendition on its held pad, instead of
ending a late-listed one with EOS and adding a new, unlinked pad for it. A
rendition the new run really drops still ends with EOS.

Non-goals: other publishers. The CLI importers reserve their track set up
front, and `<moq-publish>`'s default `source` mode waits for its enabled
tracks to settle before announcing. `@moq/publish` with `announce="always"`,
or its `Broadcast` class used directly, announces before its renditions
resolve and has the same gap, which is planned separately.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Add the promised browser first-catalog quest

Fresh evidence in this final tree: a repo-wide rg across quest/ finds no browser first-catalog follow-up beyond the two statements in this file. Broadcast still defaults announce to true (js/publish/src/broadcast.ts:91-96) and serves its current catalog before announcing (:238-264), so a direct API user or announce="always" publisher can still send a partial first snapshot; on restart, moqsrc then EOSes the held rendition and later exposes an unlinked replacement pad. Add and link the separate JS quest promised here so this production path is not left unowned. (Written by GPT-5.6 Sol)

AGENTS.md reference: AGENTS.md:L48-L48

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, there is no quest yet for the browser case (announce="always", or Broadcast used directly). The maintainer chose to plan it separately rather than widen this moqsink quest, and planning a new quest belongs to a maintainer-run /quest-plan interview. I'm reporting it back as a follow-up for that, not adding it here.

(Written by Claude Opus 5.5)


## Plan

Decided 2026-10-10, planning the follow-ups of moqsrc's restart work
([#5181](https://github.com/moq-dev/moq/pull/5181), superseded by
[#5191](https://github.com/moq-dev/moq/pull/5191)):

- The fix lives in `moqsink`, and `moqsrc`'s rule stays: a rendition the new
run's first catalog does not list ends with EOS. Holding unlisted
renditions in `moqsrc` instead would let one that never returns stall a
downstream muxer.
- A browser publisher that announces before its tracks settle stays out of
scope (decided in #5189's review), so this quest stays a `moqsink` change.
- Each pad reserves its catalog slot (`moq_mux::catalog::Producer::reserve`)
when it is requested in `request_new_pad` (`rs/moq-gst/src/sink/imp.rs`),
instead of when its caps arrive (`catalog.reserve()` in
`rs/moq-gst/src/sink/pad.rs`), hands it to its importer once the caps
resolve the rendition, and drops it if the pad is released first or its
Comment on lines +31 to +33

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep the catalog gate closed across CAPS renegotiation

When an H.264 pad receives changed CAPS before its first SPS while a sibling is already catalog-ready, this move-only handoff leaves a gate gap: renegotiation is supported (rs/moq-gst/src/sink/pad.rs:1158-1178), but Pad::build finalizes the old importer before constructing its replacement (rs/moq-gst/src/sink/pad.rs:312-313), and dropping that importer's last Reserved synchronously flushes the sibling-only initial snapshot (rs/moq-mux/src/catalog/producer.rs:651-685). The replacement is then only a later catalog update, recreating the unlinked-pad restart bug; require acquiring or retaining a reservation across finalize/rebuild and cover this renegotiation. (Written by GPT-5.6 Sol)

AGENTS.md reference: AGENTS.md:L16-L18

Useful? React with 👍 / 👎.

caps are opaque, which the catalog never lists. The catalog withholds its
Comment on lines +32 to +34

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Drop the gate when caps fail to build a producer

When a requested pad receives structurally accepted but unusable caps, such as AAC without codec_data, or hits a duplicate track-name error, Pad::observe_caps returns CapsOutcome::Failed and the lifecycle becomes terminal (rs/moq-gst/src/sink/pad.rs:283-289, rs/moq-gst/src/sink/imp.rs:782-787). That path neither hands the new pre-caps reservation to an importer nor matches the release, EOS-without-caps, or opaque-caps cases listed here, so it can keep the initial catalog gate closed forever while valid sibling pads stream. Require caps/build failures to drop the reservation and cover that failure with the catalog regression. (Written by GPT-5.6 Sol)

AGENTS.md reference: AGENTS.md:L17-L18

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. CapsOutcome::Failed leaves the lifecycle terminal, so its reservation would never resolve. In e863e76 the Plan says a pad whose caps fail (AAC without codec_data, a duplicate track name) drops its reservation, and the test covers it.

(Written by Claude Opus 5.5)

first snapshot until every reservation resolves, so that snapshot lists
every pad. The catalog producer exists only during a run (`READY` to
`PAUSED`), so each run reserves for the pads that exist when it starts
(`start_session`), and a pad requested mid-run reserves at once.
- A pad admitted while a run starts must not miss the gate. `request_new_pad`
counts the admission under the control lock before `add_pad`, so
`start_session` can run in between and miss the pad in `sink_pads`, and a
sibling can publish the first snapshot before the pad confirms. So key the
slot by pad, not by a count: each pad's lifecycle holds at most one
reservation, and filling it is idempotent. The control state tracks
admitted-but-unconfirmed pads instead of only `control.admissions`;
`start_session` fills the slot of every pad in `sink_pads` or that set, a
mid-run request fills it when it is admitted, and `cancel_admission` or
release drops it. A pad already visible in `sink_pads`, or whose CAPS
arrived through a reentrant `child_added` handler, is then never reserved
twice.

Decided 2026-10-11 by the maintainer, in #5189:

- A pad requested but never fed holds back the run's first catalog, so its
reservation drops when the pad is released, reaches EOS without caps, or

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Release unresolved capped reservations on EOS

When a video pad has accepted caps but has not yet resolved its catalog config, such as H.264 byte-stream caps followed by EOS before an SPS, the reservation is already owned by the importer, so the stated EOS without caps release does not cover it. handle_event(Eos) currently only marks that pad ended, while maybe_finish_locked finalizes producers only after every pad ends (rs/moq-gst/src/sink/imp.rs:823-833,933-977); if an audio sibling keeps streaming, this reservation can never resolve and the first catalog stays withheld. Include this terminal path and its regression in the quest. (Written by GPT-5.6 Sol)

AGENTS.md reference: AGENTS.md:L18-L18

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Once caps arrive the importer owns the reservation, so "EOS without caps" misses a pad that ends before its config resolves, such as H.264 byte-stream ending before its SPS. In the new head the Plan drops the reservation on every terminal path before the rendition resolves, including EOS after caps, and the test covers that case while a sibling streams.

(Written by Claude Opus 5.5)

gets opaque caps. Every other terminal path before the rendition resolves
drops it too: caps that fail (`CapsOutcome::Failed`, such as AAC without
`codec_data` or a duplicate track name), and EOS after caps but before the
importer resolved a config (H.264 byte-stream that ends before its SPS),
whose importer owns the reservation by then. Today `handle_event(Eos)`
only marks the pad ended, and producers finalize once every pad ends, so a
streaming sibling would keep the gate shut. Dropping reservations once
another pad's first frame is ready was rejected: it brings back the race
this quest removes. The initial gate only holds the first snapshot, so
later catalogs never wait.
- m1, ranked after #3021: only a restart of a `moqsink` whose pads negotiate
at different times is affected, so it does not gate the release under
[Broadcast epochs](/quest/m0/broadcast-epoch/README.md).
- Docs stay inline: the implementing PR updates the `moqsink` and `moqsrc`
paragraphs of `doc/bin/gstreamer.md`, including that every linked pad must
negotiate before the broadcast lists anything. No separate docs quest.

Tests: a `moqsink` whose audio pad gets caps after its video pad's first
buffer publishes one first catalog listing both; a pad that is released,
ends without caps, fails its caps, or ends after caps but before its config
lets the first catalog publish while a sibling streams; a pad requested while
the run starts is listed even when a sibling resolves before that pad
confirms, and reserved once even when it is already in `sink_pads`; and a
`moqsrc` following that `moqsink` across a restart keeps both pads without
EOS.

## Related

- [#3115](/quest/m2/3115-moqsink-the-publication-has-no-generation-so-a-flush.md) - a new generation re-reserves for its pads too
60 changes: 60 additions & 0 deletions quest/m1/gst-src-av-sync.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# [M] moqsrc keeps audio and video aligned

## Goal

All of a `moqsrc` run's pads share one timestamp reference and one segment
base, so audio and video keep the relative timing the broadcast gives them,
at the first start and after every switch. Today each pump starts its PTS at
its own first frame (`reference_ts` in `rs/moq-gst/src/source/imp.rs`) and
bases the segment it pushes with its first buffer on the running time that
buffer arrived, so two tracks line up only as well as their first frames'
arrivals happen to match their timestamps. A video track joining at a
keyframe and an audio track joining at its latest group start apart.

Non-goals: aligning to wall time across hosts, which
[#3021](/quest/m1/3021-moq-gst-anchor-generated-media-timelines-to-wall-clock.md)
settles on the sink side; jitter buffering.

## Plan

Decided 2026-10-10, planning the follow-ups of moqsrc's restart work
([#5181](https://github.com/moq-dev/moq/pull/5181), superseded by
[#5191](https://github.com/moq-dev/moq/pull/5191)): the tracks of one
broadcast share one timeline, which the catalog's one `clock` maps to wall
time, so equal times on different tracks present together. They do not
share one scale: CMAF keeps each track's `mdhd` timescale, and LOC can carry
a per-frame one (`moq_mux::container::Frame::timestamp`). `moqsink` already
publishes aligned tracks (`two_pads_keep_av_aligned_through_real_segments`);
only `moqsrc` loses the alignment.

Decided 2026-10-11 by the maintainer, in #5189:

- Per run, the first frame any pump reads fixes the run's reference
timestamp and segment base, and every pad of the run maps through both.
Mapping through the catalog's wall `clock` was rejected: the clock is
optional, assumes synchronized hosts, and only means something for
`moqsink` once #3021 lands.
- Compare times converted to one scale, such as nanoseconds:
`moq_net::Timestamp::checked_sub` refuses mismatched scales, and
`relative_pts` turns that refusal into zero.
- A frame before the run's reference must land before the segment start,
where the segment clips it. Today `relative_pts` clamps it to zero instead.
For example, use the converted time as the PTS and the reference as the
segment start.
- With one base, a pump whose first frame arrives late in wall time but
early in media time reaches a clock-synced sink late. That is synced
playback working as intended, not something to compensate for here.
- m1, ranked after #3021.
- Docs stay inline: the implementing PR updates the `moqsrc` paragraph of
`doc/bin/gstreamer.md`. No separate docs quest.

Test: an audio and a video frame with the same broadcast time reach
`moqsrc`'s two pads at the same running time, at the first start and after a
restart, whichever pump reads first and whichever track's first frame is
earlier. A frame before the run's reference reaches its pad with a PTS before
the segment start, for downstream to clip. One case is a CMAF broadcast whose
tracks use different timescales.

## Related

- [#3021](/quest/m1/3021-moq-gst-anchor-generated-media-timelines-to-wall-clock.md) - the wall epoch on the sink side
Loading