From 266c0c69ec8d853725b644cd20c3f9f173d2a8ad Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 12 Oct 2026 20:29:04 -0700 Subject: [PATCH 1/4] quest(m1): plan rendition failure isolation and record an empty-group largest Add quest/m1/rendition-failure.md: when one @moq/publish rendition's encoder fails, only that rendition ends, loudly, while the others and the broadcast keep running. Follows the Codex P1 declined on #5068. Record in quest/m1/largest.md that a started group with no frames leaves largest() at the previous real frame. Co-Authored-By: Claude Opus 5.5 --- quest/m1/README.md | 1 + quest/m1/largest.md | 8 +++- quest/m1/rendition-failure.md | 81 +++++++++++++++++++++++++++++++++++ 3 files changed, 88 insertions(+), 2 deletions(-) create mode 100644 quest/m1/rendition-failure.md diff --git a/quest/m1/README.md b/quest/m1/README.md index 589159cfc9c..a39beb22077 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -122,6 +122,7 @@ folded into Dropped sources. - [mp4-atom dOps mapping](/quest/m1/mp4-atom-dops-mapping.md) - a released mp4-atom reads and writes any `dOps` channel mapping family and table - [CMAF surround Opus](/quest/m1/cmaf-opus-surround.md) - fMP4 import and export carry an Opus channel mapping table - [GPU CI](/quest/m1/gpu-ci.md) - NVIDIA tests run nightly on a self-hosted GPU runner, and `just rs nvidia` runs them locally instead of skipping +- [A failed rendition fails alone](/quest/m1/rendition-failure.md) - when one `@moq/publish` rendition's encoder fails, only that rendition ends, loudly, while the others and the broadcast keep running under the same epoch - [Rendition preference](/quest/m1/rendition-preference.md) - automatic selection by ``, `Video::ranked`, and WHEP keeps the highest `preference` that decodes, so a compatibility transcode is only picked when nothing preferred decodes - [Simulcast rung disable](/quest/m1/simulcast-rung-disable.md) - JS, Rust/FFI, and OBS publishers stop encoding the top renditions their grant cannot fund, advertise `enabled: false`, and re-enable with hysteresis - [Discover media headroom](/quest/m1/quic-probe.md) - test useful-media pacing before adding redundant probe traffic diff --git a/quest/m1/largest.md b/quest/m1/largest.md index 3fda9f97ae3..9605b2f9ee5 100644 --- a/quest/m1/largest.md +++ b/quest/m1/largest.md @@ -7,7 +7,7 @@ arrives and follows it live. `largest()` replaces `latest()` on the track producer, consumer and subscriber in Rust and `@moq/net`, returning a `Location` (group and frame). It is seeded by the publisher's reported largest (moq-transport SUBSCRIBE_OK, lite-07 SUBSCRIBE_START) and advances as -groups and frames arrive or a newer value is reported. It never lives on +frames arrive or a newer value is reported. It never lives on `Info`, which is per-track and immutable. ## Plan @@ -20,6 +20,9 @@ Decided with the maintainer (2026-10-13), after moq-dev/moq#5369 proposed current location at once if known; each later call waits for a newer one, skipping values in between; it ends (`None`/`undefined`) when the track ends. +- **An empty group is not reported.** A group that has started but has no + frames yet leaves `largest()` at the previous real frame; a `Location` + names a frame, and the empty group has none yet. - **One name everywhere.** Rename on the producer, consumer and subscriber, and update every caller (moq-json, moq-flate, moq-mux, moq-relay, moq-hls, moq-c, and the rest) in the same PR. JS's buffered-only `largest()` takes @@ -34,5 +37,6 @@ Decided with the maintainer (2026-10-13), after moq-dev/moq#5369 proposed locally: a reported largest is a hint about the edge, not a cached group. - Bindings follow the cross-package table if any expose `latest()`. - Cover on mock time in both languages: known before the first group from - SUBSCRIBE_OK and SUBSCRIBE_START, advancing per frame, coalescing, and + SUBSCRIBE_OK and SUBSCRIBE_START, advancing per frame, holding through a + started group with no frames, coalescing, and ending with the track. diff --git a/quest/m1/rendition-failure.md b/quest/m1/rendition-failure.md new file mode 100644 index 00000000000..7c1215d2453 --- /dev/null +++ b/quest/m1/rendition-failure.md @@ -0,0 +1,81 @@ +# [M] A failed rendition fails alone + +## Goal + +When one rendition's encoder fails mid-stream in `@moq/publish`, only that +rendition goes down, and loudly: its track ends with the encoder's real +error, viewers stop selecting it, and the application can see which +rendition failed and why. The other renditions, the catalog, and the network +broadcast keep running under the same epoch. A failure never starts a retry +loop. + +## Plan + +Raised as a declined Codex P1 on moq-dev/moq#5068, which made each track live +as long as its network broadcast. + +Current behaviour: + +- A video encoder error aborts its track (`producer.close(err)` in + `js/publish/src/video/encoder.ts`) and nothing else reacts. The catalog + keeps advertising the rendition as enabled, so viewers keep selecting a + track that has ended. +- A fatal audio encoder error closes the current track and every later one + handed to that rendition, for the encoder's whole life. Its catalog entry + stays too. +- `Broadcast` (`js/publish/src/broadcast.ts`) notices the closed track only + when the rendition set next changes, for any rendition. It then bumps + `#restart`, which publishes a new network broadcast under a new epoch, so + every healthy rendition and the catalog restart with it. With a dead audio + encoder this repeats on every later rendition change, because the new + broadcast's audio track closes at once. + +Direction: + +- Separate failing from recovering. Failing is local and immediate: end the + track with the error, take the rendition out of selection, and expose the + failure to the application (for example an error signal on the encoder or + the `Rendition`). +- Only a change of inputs (codec, config, capture, or an explicit + application action) may re-arm a failed rendition, never the close itself, + so a deterministic failure stays down instead of looping. No timed retry. +- One rendition failing never republishes the network broadcast. Remove the + `#restart` path for closed rendition tracks, or narrow it to whatever the + questions below settle. +- Watch for the interaction with [simulcast rung disable](/quest/m1/simulcast-rung-disable.md), + which also drives `enabled`: a recovering grant must not re-enable a + failed rendition. +- Tests on mock time with a fake encoder: a failing video rendition leaves a + sibling rendition and the catalog writing under the same epoch; viewers + stop being offered the failed rendition; a dead audio encoder does not + republish the broadcast when another rendition registers; any recovery + never reuses a sequence. + +Open questions for the maintainer: + +1. How the catalog shows a failed rendition. (a) `enabled: false` with its + last config, which `@moq/watch` and native consumers already skip + (recommended, since it leaves room to recover); (b) remove its entry, + which fits a permanent failure. +2. How a failed rendition recovers inside the same broadcast. This revisits + #5068's decision 7, which moved every track to a new network broadcast. + (a) It doesn't: it stays down until the broadcast is replaced for another + reason; (b) the replacement track continues the name's sequences, writing + its first group explicitly after the closed track's last one + (recommended: the same rendition continuing its numbering is the same + content, and only that rendition's viewers resubscribe); (c) the + replacement publishes under a new track name, which needs + [catalog rendition IDs](/quest/m1/catalog-track-id.md) to keep the + catalog ID stable. +3. What re-arms a failed rendition. (a) Only a changed input or an explicit + application call (recommended); (b) also one automatic retry with a fresh + `VideoEncoder`, since a GPU reset can kill one instance; audio never + retries. +4. Scope. (a) JS `@moq/publish` only, where the library owns the rendition + tracks (recommended); (b) also the native multi-encoder publishers (OBS + multitrack, moq-ffi), after checking how they handle one encoder failing. + +## Related + +- [Simulcast rung disable](/quest/m1/simulcast-rung-disable.md) - the other driver of a rendition's `enabled` state +- [Catalog rendition IDs](/quest/m1/catalog-track-id.md) - lets a rendition move to a new track name without changing its catalog ID From bc67a2c2bbf755056ba0a7710bf2143fdbad01bd Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 12 Oct 2026 20:34:03 -0700 Subject: [PATCH 2/4] quest(m1): record the maintainer's rendition failure decisions Co-Authored-By: Claude Opus 5.5 --- quest/m1/rendition-failure.md | 67 +++++++++++++++-------------------- 1 file changed, 28 insertions(+), 39 deletions(-) diff --git a/quest/m1/rendition-failure.md b/quest/m1/rendition-failure.md index 7c1215d2453..eaee9ddc31f 100644 --- a/quest/m1/rendition-failure.md +++ b/quest/m1/rendition-failure.md @@ -30,50 +30,39 @@ Current behaviour: encoder this repeats on every later rendition change, because the new broadcast's audio track closes at once. -Direction: +Decided with the maintainer (2026-10-13): + +- **Scope: JS `@moq/publish` only**, where the library owns the rendition + tracks. Native publishers' apps own each track directly. +- **Fail locally and at once.** End the track with the encoder's error, + advertise the rendition as `enabled: false` with its last config (which + `@moq/watch` and native consumers already skip), and expose the failure to + the application, for example as an error signal on the encoder or the + `Rendition`. Failing never republishes the network broadcast, so remove or + narrow the `#restart` path that does today. +- **Only a changed input or an explicit application call re-arms a failed + rendition.** No automatic or timed retry, so a deterministic failure stays + down instead of looping. +- **Do not resume numbering unless it's a continuation (same encoder).** A + recovered rendition publishes under a new track name or a new broadcast + epoch; when unsure, default to one of those, because that's safer. This + keeps #5068's decision 7 for any replacement encoder. A new track name + leaves the siblings running but needs + [catalog rendition IDs](/quest/m1/catalog-track-id.md) to keep the catalog + ID stable; a new epoch restarts the siblings, so reserve it for recovery, + never for the failure itself. +- **Milestone: m1**, as reliability of a shipped feature. + +Also: -- Separate failing from recovering. Failing is local and immediate: end the - track with the error, take the rendition out of selection, and expose the - failure to the application (for example an error signal on the encoder or - the `Rendition`). -- Only a change of inputs (codec, config, capture, or an explicit - application action) may re-arm a failed rendition, never the close itself, - so a deterministic failure stays down instead of looping. No timed retry. -- One rendition failing never republishes the network broadcast. Remove the - `#restart` path for closed rendition tracks, or narrow it to whatever the - questions below settle. - Watch for the interaction with [simulcast rung disable](/quest/m1/simulcast-rung-disable.md), which also drives `enabled`: a recovering grant must not re-enable a failed rendition. - Tests on mock time with a fake encoder: a failing video rendition leaves a - sibling rendition and the catalog writing under the same epoch; viewers - stop being offered the failed rendition; a dead audio encoder does not - republish the broadcast when another rendition registers; any recovery - never reuses a sequence. - -Open questions for the maintainer: - -1. How the catalog shows a failed rendition. (a) `enabled: false` with its - last config, which `@moq/watch` and native consumers already skip - (recommended, since it leaves room to recover); (b) remove its entry, - which fits a permanent failure. -2. How a failed rendition recovers inside the same broadcast. This revisits - #5068's decision 7, which moved every track to a new network broadcast. - (a) It doesn't: it stays down until the broadcast is replaced for another - reason; (b) the replacement track continues the name's sequences, writing - its first group explicitly after the closed track's last one - (recommended: the same rendition continuing its numbering is the same - content, and only that rendition's viewers resubscribe); (c) the - replacement publishes under a new track name, which needs - [catalog rendition IDs](/quest/m1/catalog-track-id.md) to keep the - catalog ID stable. -3. What re-arms a failed rendition. (a) Only a changed input or an explicit - application call (recommended); (b) also one automatic retry with a fresh - `VideoEncoder`, since a GPU reset can kill one instance; audio never - retries. -4. Scope. (a) JS `@moq/publish` only, where the library owns the rendition - tracks (recommended); (b) also the native multi-encoder publishers (OBS - multitrack, moq-ffi), after checking how they handle one encoder failing. + sibling rendition and the catalog writing under the same epoch; the + failed rendition is advertised `enabled: false`; a dead audio encoder + does not republish the broadcast when another rendition registers; a + re-armed rendition never reuses a closed track's sequences. ## Related From 167e7cba2bc6a0ceb2cb3053a6406f7a9dca6682 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 12 Oct 2026 20:34:59 -0700 Subject: [PATCH 3/4] quest(m1): note that WHEP and HLS still serve disabled renditions Co-Authored-By: Claude Opus 5.5 --- quest/m1/rendition-failure.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/quest/m1/rendition-failure.md b/quest/m1/rendition-failure.md index eaee9ddc31f..01ec9816410 100644 --- a/quest/m1/rendition-failure.md +++ b/quest/m1/rendition-failure.md @@ -4,7 +4,7 @@ When one rendition's encoder fails mid-stream in `@moq/publish`, only that rendition goes down, and loudly: its track ends with the encoder's real -error, viewers stop selecting it, and the application can see which +error, the catalog marks it disabled, and the application can see which rendition failed and why. The other renditions, the catalog, and the network broadcast keep running under the same epoch. A failure never starts a retry loop. @@ -36,7 +36,8 @@ Decided with the maintainer (2026-10-13): tracks. Native publishers' apps own each track directly. - **Fail locally and at once.** End the track with the encoder's error, advertise the rendition as `enabled: false` with its last config (which - `@moq/watch` and native consumers already skip), and expose the failure to + `@moq/watch` already skips; WHEP and HLS egress still serve a disabled + rendition, a consumer gap outside this quest), and expose the failure to the application, for example as an error signal on the encoder or the `Rendition`. Failing never republishes the network broadcast, so remove or narrow the `#restart` path that does today. From 02eccf169bb5f0c5da3a7ff776c3c75cea50181f Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 12 Oct 2026 20:58:02 -0700 Subject: [PATCH 4/4] quest(m1): say the broadcast reacts to a failed rendition only later Co-Authored-By: Claude Opus 5.5 --- quest/m1/rendition-failure.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/quest/m1/rendition-failure.md b/quest/m1/rendition-failure.md index 01ec9816410..984ba341cc0 100644 --- a/quest/m1/rendition-failure.md +++ b/quest/m1/rendition-failure.md @@ -17,7 +17,7 @@ as long as its network broadcast. Current behaviour: - A video encoder error aborts its track (`producer.close(err)` in - `js/publish/src/video/encoder.ts`) and nothing else reacts. The catalog + `js/publish/src/video/encoder.ts`) and nothing reacts right away. The catalog keeps advertising the rendition as enabled, so viewers keep selecting a track that has ended. - A fatal audio encoder error closes the current track and every later one