diff --git a/quest/m1/README.md b/quest/m1/README.md index 724679ff297..9a136439473 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -54,6 +54,7 @@ moves what remains of them once their claims merge. - [Tests run on a mocked clock](/quest/m1/mock-clock-tests/README.md) - about 300 tests stop proving nothing happened by waiting, sleeping to settle, or asserting on elapsed time - [QMux reads the app-limited flag on big-endian Linux](/quest/m1/qmux-app-limited-endian.md) - the send-rate filter's app-limited mask follows the target's byte order - [A cancelled dispatch lane stops its phase promptly](/quest/m1/dispatch-cancel-race.md) - a TERM landing just before a lane forks its phase's command still stops that command, instead of waiting out the phase +- [Lifecycles are explicit state](/quest/m1/explicit-state/README.md) - every lifecycle the 2026-11-07 audit found implicit is one enum or union, fixing the bugs in the 11 lifecycles it redesigns, with review and agent rules against new ones ## Related diff --git a/quest/m1/explicit-state/README.md b/quest/m1/explicit-state/README.md new file mode 100644 index 00000000000..2850e41ae87 --- /dev/null +++ b/quest/m1/explicit-state/README.md @@ -0,0 +1,86 @@ +# Lifecycles are explicit state + +## Goal + +Every lifecycle that an audit on 2026-11-07 found encoded implicitly is one +explicit enum or union, in Rust, JS, and the bindings. Each variant carries +its own data, so the combinations that caused bugs can't be represented. The +bugs the audit found in the 11 lifecycles that can reach an invalid +combination, listed in each child, are fixed by these redesigns, and review +and agent rules keep new instances out. + +An implicit state machine here is a lifecycle encoded as several flags, +`Option`s, atomics, or locks whose combinations mean phases, where some path +can reach a combination no phase allows. The audit found four recurring +shapes: + +1. **A terminal state stored in two or more places**: a flag beside the + enum, an error beside a live handle, or a second "closed" source. +2. **"Done" marked before the work that can fail**, leaving an object + half-finished but marked finished. +3. **One-shot handles stored as `Option`s and taken before validation**, so + "answered" and "failed to answer" look the same. +4. **Per-phase data kept outside an enum that already exists**, guarded by + `.expect()` or by the order of calls. + +A value split across locks or atomics is the same bug as shape 1. + +## Plan + +Planned 2026-11-07 with the maintainer (/quest-plan). + +Decided 2026-11-07 (maintainer): + +- ✅ Fix the root cause and redesign freely. In the maintainer's words: "you + have permission to make any API changes and try to address the root + cause" and "you have full permission to redesign anything to avoid these + classes of bug". Not chosen: guarding the one path each bug takes. +- ✅ Fold each bug into the redesign of its shape. Not chosen: small bug + quests beside a separate redesign epic, or widening each bug quest to its + own module only. The user-visible fixes therefore land with their + redesigns. +- ✅ Milestone m1. A child whose design breaks a published surface leaves + this epic and becomes its own m2 quest, since an epic joins the milestone + its outcome needs; its PR marks its title `!` and says how to migrate. Not + chosen: m3 after the cut, or small fixes in m1 with the redesigns in m3. + This replaces the earlier split from the same session (user-visible fixes + in m1, internal ones in m3). +- ✅ Guard against new instances in both REVIEW.md and AGENTS.md (see + [Review and agent rule](/quest/m1/explicit-state/rule.md)). Not chosen: + REVIEW.md only, or no rule change. +- ✅ `quic::driver::Endpoint::rebind` becomes crate-private, since only + tests call it (see [Transport close](/quest/m1/explicit-state/transport-close.md)). + Not chosen: refusing to rebind after close. + +Every child reproduces each folded bug in a test before fixing it. A bug +that doesn't reproduce is dropped from the child, and its PR says why. A +child also sweeps the instances listed in it that have no known bug, but +needs no test for those beyond what the redesign changes. + +A change to moq-net's per-track or per-group state is on the relay's hot +path: benchmark it (`moq-bench`, comparing quiet cells within one run). + +Out of scope: the lite `TrackRun` start shortcut that never sends its +pending drops (`rs/moq-net/src/lite/publisher.rs`, the "start past +everything" branch). It belongs to +[Delete the old start bookkeeping](/quest/m2/subscribe-ranges/start-bookkeeping.md), +which collapses that state and waits on [Drop set](/quest/m1/drop-set.md). + +## Required + +- [Review and agent rule](/quest/m1/explicit-state/rule.md) - REVIEW.md and AGENTS.md ask for one enum per lifecycle, terminal transitions after fallible work, and one lock per value +- [moq-boy pause state](/quest/m1/explicit-state/boy-pause.md) - a new viewer always wakes a paused emulator +- [Transport close](/quest/m1/explicit-state/transport-close.md) - connection status ends with its error, and every Rust QUIC backend agrees when a connection is closed +- [Video codec drains](/quest/m1/explicit-state/video-drain.md) - a hardware codec whose drain fails is ready for input again or terminally failed, never left stuck +- [Watch playback phases](/quest/m1/explicit-state/watch-playback.md) - watch counts a stall only while a rendition plays +- [Publish capture phases](/quest/m1/explicit-state/publish-capture.md) - dismissing the screen-share picker turns sharing off +- [js/qmux close](/quest/m1/explicit-state/qmux-close.md) - a graceful remote close ends qmux streams cleanly +- [FFI handle state](/quest/m1/explicit-state/ffi-state.md) - FFI handles hold one state each: video follow under one lock, requests answered once, drivers Cancelled after shutdown +- [A failed finish is final](/quest/m1/explicit-state/finish-final.md) - producers and importers are never half-finished, and refuse input after finish +- [Track and group end-of-life](/quest/m1/explicit-state/track-end.md) - moq-net's track and group outcomes are one enum each, in Rust and JS +- [moq-net phase data](/quest/m1/explicit-state/net-phases.md) - moq-net's session and subscription state machines keep each phase's data in its variant +- [moq-mux and moq-gst lifecycles](/quest/m1/explicit-state/mux-gst.md) - export sources, TS and fMP4 export tracks, and gst pads are explicit state + +## Related + +- [Delete the old start bookkeeping](/quest/m2/subscribe-ranges/start-bookkeeping.md) - owns the lite `TrackRun` start state and its forgotten-drops bug diff --git a/quest/m1/explicit-state/boy-pause.md b/quest/m1/explicit-state/boy-pause.md new file mode 100644 index 00000000000..dddc2fdb132 --- /dev/null +++ b/quest/m1/explicit-state/boy-pause.md @@ -0,0 +1,34 @@ +# [XS] moq-boy pause state + +## Goal + +A viewer who subscribes to a paused moq-boy session always wakes the +emulator. No interleaving of the pause monitors and the emulator thread +leaves it asleep while a track is wanted. Pause state is one value derived +from which tracks are wanted, written only under the lock the emulator +waits on. + +## Plan + +The shared decisions are in the [epic](/quest/m1/explicit-state/README.md). + +The bug (`rs/moq-boy/src/main.rs`, `Session`): + +- `video_active`, `audio_active`, and `paused` are separate atomics beside a + `(Mutex<()>, Condvar)`. +- `run_track_monitor` and `run_pause_monitor` store `paused = false` and call + `notify_all()` without taking the mutex, while `wait_for_resume` checks + `paused` under the mutex and then waits. +- A notify that lands between that check and `cvar.wait` is lost, so the + emulator sleeps with `paused == false` until the next subscription change. +- Separately, `run_pause_monitor`'s `tokio::join!` of both "unwanted" waits + doesn't re-check a side once it resolves, so it can store `paused = true` + while video is wanted again. + +Recommended shape: the condvar's own mutex holds the demand +(`{ video: bool, audio: bool }`), and paused means neither is wanted. Each +monitor changes the demand under the lock and then notifies. That also lets +the two monitors collapse into one. + +[moq-dev/moq#8115](https://github.com/moq-dev/moq/pull/8115) (socket roles) +also edits this file. Expect a merge conflict, not an overlap. diff --git a/quest/m1/explicit-state/ffi-state.md b/quest/m1/explicit-state/ffi-state.md new file mode 100644 index 00000000000..05f9282b283 --- /dev/null +++ b/quest/m1/explicit-state/ffi-state.md @@ -0,0 +1,65 @@ +# [L] FFI handle state + +## Goal + +Each moq-ffi and moq-c handle holds its lifecycle in one value under one lock: + +- A video producer's track, its bandwidth reservation, and its follower + change together. A finish or abort can't leave a reservation behind. +- A one-shot request handle (track, broadcast, datagram, socket, and server + requests) is pending, answered, or failed. A failed answer leaves the + request pending, or reports why it failed. +- A background driver that never ran reports `Cancelled`, never `Ok`. + Synchronous calls after `moq_ffi_shutdown` fail the same way the + asynchronous ones do. + +The language wrappers stop mirroring Rust state in their own flags. + +## Plan + +The shared decisions are in the [epic](/quest/m1/explicit-state/README.md). + +Bugs to reproduce and fix: + +- **Video follow across locks** (`rs/moq-ffi/src/video.rs`): + - `inner`, `reservation`, and `follow` are three mutexes. + - `write()` releases `inner` and only then calls `follow_track`, so a + `finish()` on another thread can run in between. + - `follow_track` then stores a new reservation for a finished track. + - `abort()` returns early once `inner` is `None` and never clears it. + - Recommended: `Mutex }, Closed }>`. + - The same bug exists in `rs/moq-c/src/video.rs` (`follow_grant` spreads + `reservation`, `follow`, and `followed` across three lock acquisitions). + It also exists in the encoder `apply` of both crates: the encoder is taken + before a fallible `discontinuity()`, so a failure leaves the catalog + saying enabled while frames are dropped. +- **`Background` after shutdown** (`rs/moq-ffi/src/ffi.rs`; `rs/moq-c/src/ffi.rs` + maps the same case to `Panic`): + - `Option>>` encodes five states. + - `check()` reads "dropped without a result" as `Ok`, while `finished()` + reads it as `Cancel`. + - Recommended: `enum Completion { Detached, Running, Done(Result), Dropped }`. + +Also sweep: + +- The `Mutex>` handles: `producer.rs`, `origin.rs`, + `datagram.rs`, `socket.rs`, `server.rs`, and `media.rs`'s `Requested` + target. Each is taken before validation or before an await, and they return + inconsistent errors (`Closed` vs `AlreadyResponded`). +- `Task` (`ffi.rs`): a cancel flag, an `Option` state, and the cleanup. +- `MoqServer` (`server.rs`): its lifecycle is spread over four places. +- `MoqTrackProducer.keep: None`, which means both "finished" and "has no + family". +- moq-c's global id counter, whose wrap lets a stale route id name a newer + broadcast. Use generation-tagged ids or clear ids on close. + +Then sweep the hand-written wrappers: + +- the Python `Client`, which allows nested `async with`; +- Kotlin flows, which cancel only on `CancellationException`; +- the OBS output's mirrored `connected`/`live`; +- the Dart stream. + +[moq-dev/moq#8060](https://github.com/moq-dev/moq/pull/8060) (moq-c parity) +edits `rs/moq-c/src/video.rs` and `ffi.rs`, so expect merge conflicts. diff --git a/quest/m1/explicit-state/finish-final.md b/quest/m1/explicit-state/finish-final.md new file mode 100644 index 00000000000..46daf52f24d --- /dev/null +++ b/quest/m1/explicit-state/finish-final.md @@ -0,0 +1,51 @@ +# [M] A failed finish is final + +## Goal + +No media producer or importer can be left half-finished: + +- A finish that fails leaves the object terminally failed and reports the + cause. It never leaves the object marked finished with its tracks unended, + and never leaves it usable with part of its pipeline torn down. +- Every container importer refuses input after finish in the same way. +- Where the type system can express it, finishing consumes the object, as + the moq-rtmp and moq-srt publishers already do. + +## Plan + +The shared decisions are in the [epic](/quest/m1/explicit-state/README.md). + +Bugs to reproduce and fix: + +- **TS import** (`rs/moq-mux/src/container/ts/import.rs`): `finish` sets + `finished = true` before the fallible `ensure!` checks, the reservation + release, and `finish_stream` for each PID. `Drop` then never retries, so an + input whose sections or private PES have no media timestamp leaves its + streams unfinished. `decode` also never checks `finished`. +- **Audio encode** (`rs/moq-audio/src/encode/producer.rs`): `end()` takes the + resampler before its fallible steps and clears `encoder` only at the end. + After a failure, `write` still passes its `encoder.is_none()` guard and + sends PCM at the input rate straight to the codec, and retrying `finish()` + republishes the terminal packets. +- **JSON window** (`rs/moq-json/src/window/producer.rs`): `finish` sets + `finished` before `track.finish()` can fail, and a retry then returns + `Ok`. After `abort`, `ensure_open` reports `Closed` instead of the cause. + The snapshot producer's design, `aborted: Option` plus openness read + from the track, is the one to match. + +Instances to sweep: + +- Importer lifecycles (`mkv/import.rs`, `fmp4/import.rs`, `ts/import.rs`): + MKV refuses `decode` after finish, but fMP4 and TS don't. MKV also overloads + `initial_reservation == None` with three meanings. Recommended: a shared + `enum Input { Open, Failed(Error), Finished }`. +- `rs/moq-mux/src/container/producer.rs`: `close()` resets the group's + fields only after its fallible steps. +- `rs/moq-transcode/src/rung.rs`: `pending_cut` is taken before the cut that + can fail. +- The moq-json `Track.group` invariant: clearing it must also reset the + encoder. + +The importer and producer `finish` signatures are public API. A move to +`finish(self)` is breaking, so if this child takes it, it leaves the epic and +becomes its own m2 quest under the epic's rule. diff --git a/quest/m1/explicit-state/mux-gst.md b/quest/m1/explicit-state/mux-gst.md new file mode 100644 index 00000000000..f4f0b63c0de --- /dev/null +++ b/quest/m1/explicit-state/mux-gst.md @@ -0,0 +1,57 @@ +# [M] moq-mux and moq-gst lifecycles + +## Goal + +The lifecycles of moq-mux and moq-gst export and ingest are explicit state: +export sources, TS and fMP4 export tracks and programs, the HLS segment +window, the container consumer's startup, and the GStreamer sink and source +pads. Each phase's data lives in its variant, and no `.expect()` or flag +ordering stands in for a phase. + +## Plan + +The shared decisions are in the [epic](/quest/m1/explicit-state/README.md). + +No confirmed bug is folded into this child. Reproduce these two first: + +- **fMP4 export** (`rs/moq-mux/src/container/fmp4/export.rs`): a track that + ends before it is ever described, while the catalog stays live, may keep + `init_ready()` false forever. +- **TS export track** (`rs/moq-mux/src/container/ts/export.rs`): + - Withdrawing a track keeps its parked frames, so it can become both + `withdrawn` and `listed`. + - `finished` carries five meanings, and the "gone" predicate ANDs eight + conditions. + +Instances to redesign: + +- **TS export `Program`**: `catalog`, `catalog_ended`, `cataloged`, + `retired`, `left`, `psi`, and others. `resume` hand-resets six fields. +- **`ExportSource`** (`container/source.rs`): `media`, `priority_control`, + `instance`, `live`, and `after` belong inside the `SourceState` variants. +- **Container `Consumer`** (`container/consumer.rs`): the startup phase and + the `track`/`track_error` reader state. The JS twin is + `js/hang/src/container/consumer.ts`, whose `#checkMaxDelay` moves `#active` + without `#recordPresented`. Change both together. +- **HLS** (`rs/moq-hls`): + - The segment window keeps `ended`, `cut`, and `failed`, and `push` after + `end()` isn't refused. + - Retiring a `Claim` takes two locks. + - The importer's `TrackState` generation fields. +- **Catalog publish gate** (`catalog/producer.rs`): `Reservations` writes its + predicate twice, in two shapes. +- **TS `Follower.refused`** belongs inside `Serving`. The TS `Passthrough` + start and clock, and the `GroupBuffer` content kind, also become explicit. +- **moq-gst sink pads** (`rs/moq-gst/src/sink/request_pad.rs`, `pad.rs`, + `imp.rs`): + - The `ended` bool means something different from `Status::Ended`. + - `releasing` is cleared after removal. + - The element's `drained`/`lingering`/`eos_delivered` and the pad's + `reserved`/`track`/`catalog`/`caps`/`failed` encode phases. +- **moq-gst source** `Entry`'s `reader`/`draining`. +- **Others**: + - The RTMP `ServerSession`: the connected app name belongs inside + `SessionState::Connected`. + - The moq-webrtc ICE `Agent`: its failure isn't sticky. + - The p2p channel handout. + - The jitter `Track`. diff --git a/quest/m1/explicit-state/net-phases.md b/quest/m1/explicit-state/net-phases.md new file mode 100644 index 00000000000..8732175d782 --- /dev/null +++ b/quest/m1/explicit-state/net-phases.md @@ -0,0 +1,78 @@ +# [L] moq-net phase data + +## Goal + +moq-net's session, subscription, and auth state machines, in Rust and in +js/net, keep each phase's data inside that phase's variant. No `.expect()` +or call ordering stands in for a phase. A placeholder such as +`RequestId(0)` never stands in for "not assigned yet". + +## Plan + +The shared decisions are in the [epic](/quest/m1/explicit-state/README.md). + +No reachable bug is folded into this child. Check each item below while +redesigning it, and reproduce any inconsistent state you find. + +Rust: + +- **IETF `SubscribeOnce`/`Established`** (`rs/moq-net/src/ietf/subscriber.rs`): + - `OnceStep` exists, but `stream`, `messages`, `fill`, `established`, + `held`, `handoff`, and `target` sit outside it. About 20 + `.expect("established")` calls stand in for the step. + - `failed`, `ended`, `finished`, `fetch_done`, and `OnceStep::Settle` + duplicate one another. + - `request_id: RequestId(0)` is a placeholder for an id that isn't + assigned yet. +- **IETF publisher** (`rs/moq-net/src/ietf/publisher.rs`): + - `RunSubscription.served` caches the track's drained state. + - An update that reopens `TrackServe` (`draining = false`, a new `tail`) + is never polled again. Reproduce this first; it is the audit's plausible + finding. + - `TrackServe.draining`/`end` and `DatagramForward` should become explicit + phases. +- **Lite subscriber** (`rs/moq-net/src/lite/subscriber.rs`): + - `ServeLoop.finishing` and `parked` belong inside `SubStream`. + - `SubStream.served` plus `started` are really `Option>`. +- **`auth::Slot`** (`rs/moq-net/src/auth.rs`): `streamed`, `refused`, + `withdrawn`, `grant`, `answered`, `retry`, and `ended` are seven fields + coordinated by hand. +- **`Following`** (`rs/moq-net/src/model/follow.rs`): eight fields, of which + `last`, `moved`, and `absorbed` are always reset together. +- **Smaller ones**: + - `ietf/session.rs` `Dispatch` holds two `Option`s, exactly one of which is + set. + - `model/origin.rs` `TrackIo` keeps `request` and `accepted` side by side. + - The driver `ending` field. + - The lite `PeerSetupState`. + - The listing `Queue`. + - `fills.rs` `Recovery`. + +JS (`js/net`): + +- **Auth** (`auth_session.ts`): + - `AuthSession`/`Presented` derive counters by hand at six sites. + - `IssuedGrant` sets `#done` asynchronously, so two updates in one tick + both pass its check. +- **`Reload`** (`connection/reload.ts`): `status` is written at seven sites, + each re-deriving it from `established`, `#draining`, and `#dialing`. +- **Lite `SubscribeEntry`** (`lite/subscriber.ts`): eleven optional fields. +- **`Lister`** (`listing.ts`): it still writes into the listing after + `close()`. +- **`RequestStream`** (`ietf/request_stream.ts`): `refuse()` leaves the stream + open if writing the refusal throws. +- **Signals `Effect`** (`js/signals/src/index.ts`): `get()` during the stale + window pushes into the next run's unwatch list. +- **Origin `Request` and `OriginState`** (`origin.ts`): the closed state is + encoded by about 40 `?.` guards. +- **`Join`** (`ietf/join.ts`): a second fetch stream after the track ended + leaks the first producer. +- **Connections** (`lite/connection.ts`, `ietf/connection.ts`): lite's + `abort()` has no guard. + +Keep the lite `TrackRun` start state out of this child. It belongs to +[Delete the old start bookkeeping](/quest/m2/subscribe-ranges/start-bookkeeping.md). + +## Related + +- [Delete the old start bookkeeping](/quest/m2/subscribe-ranges/start-bookkeeping.md) - owns lite `TrackRun`'s start state diff --git a/quest/m1/explicit-state/publish-capture.md b/quest/m1/explicit-state/publish-capture.md new file mode 100644 index 00000000000..bdb07356c98 --- /dev/null +++ b/quest/m1/explicit-state/publish-capture.md @@ -0,0 +1,50 @@ +# [M] Publish capture phases + +## Goal + +In `@moq/publish` and `@moq/room`, each capture source and encoder slot is in +one explicit phase, and what is advertised follows from that phase. A screen +share is off, prompting, live, or ended, and the room's `screenEnabled` +follows it: dismissing the browser's picker, a capture error, or the user +stopping the share turns sharing off and says why. Nothing re-prompts without +a new user gesture. An encoder whose capture didn't reopen isn't advertised +as enabled. + +## Plan + +The shared decisions are in the [epic](/quest/m1/explicit-state/README.md). + +Bugs to reproduce and fix: + +- **Screen-share dismiss**: + - `js/publish/src/source/screen.ts` swallows a rejected `getDisplayMedia` + (`pending.catch(() => undefined)`). A dismissed picker then looks the same + as one still prompting: enabled, no source, `#stopped` false, and no + error. + - `js/room/src/local.ts` resets `screenEnabled` only on `!live && wasLive`, + and `wasLive` was never true, so the toggle stays on with nothing shared. + - Because `#stopped` stays false, a later change to `share.video` or + `share.audio` re-runs the effect and prompts again outside a user + gesture. + - Recommended: `{ kind: "off" } | { kind: "prompting" } | { kind: "live", + media } | { kind: "ended", reason: "stopped" | "dismissed" | "error" }`. +- **Audio `#muted` latch** (`js/publish/src/audio/encoder.ts`, reproduce + first): + - With `capture="release"`, mute, unmute, and then a failed reopen leave + `#muted` set, so the last config stays listed as enabled. + - The video encoder's `#paused` gives the correct `enabled: false` in the + same situation. + +Also sweep: + +- `js/publish/src/source/retry.ts`: `#failures = LIMIT + 1` is used as a + terminal sentinel. +- The publication entry lifecycle (`js/publish/src/publication.ts` and + `broadcast.ts`), which is resolved by an ordered if-chain. +- The video encoder's `#runCodec`, which keeps `#codec` after an error. +- The preview `Transcode.#active`. +- The `` and `` enable latches (`#enabled`, + `#connected`, `#invalid`, `#resume`). Derive enabled as connected with no + invalid attributes. +- `js/moq-boy/src/game.ts`: a command sent while disconnected replays on + reconnect. diff --git a/quest/m1/explicit-state/qmux-close.md b/quest/m1/explicit-state/qmux-close.md new file mode 100644 index 00000000000..ff7158236cc --- /dev/null +++ b/quest/m1/explicit-state/qmux-close.md @@ -0,0 +1,41 @@ +# [S] js/qmux close + +## Goal + +A js/qmux session's close is one phase: open, closed cleanly, or aborted with +a cause. A peer's graceful APPLICATION_CLOSE ends the incoming stream and +datagram readables cleanly, the same as a local `close()` does. An abnormal +close, such as CONNECTION_CLOSE, a socket error, or an idle timeout, errors +them with its cause. Once the session is closed, nothing can change its +outcome. + +## Plan + +The shared decisions are in the [epic](/quest/m1/explicit-state/README.md). + +The bug (`js/qmux/src/session.ts`): + +- On a received APPLICATION_CLOSE, the session sets + `#closeReason ??= new Error(...)` and then calls the graceful `#close`. +- `#teardown` closes the incoming queues and datagrams with `#closeReason`. + Its own comment relies on a graceful close leaving `#closeReason` unset. +- So a peer's graceful close errors every incoming readable. + +`#closeReason ??=` is also written at several sites before `#abort`'s +`if (this.#closed) return`, so it can change after the session closed. And +`#sendClose` followed by the terminal call is a two-step protocol repeated +at about eight sites. + +Recommended shape: `#state: { kind: "open" } | { kind: "closed", clean: +true, info } | { kind: "closed", clean: false, cause }`, with the close frame +carried in the transition. + +Also sweep `js/qmux/src/recv.ts` and `js/qmux/src/credit.ts`: + +- In `recv.ts`, `#fin`, `#error`, and `#terminal` encode the stream's end, + and `error()` never marks it terminal. +- In `credit.ts`, `#closed` and `#closeReason` are always set together. + +[moq-dev/moq#8136](https://github.com/moq-dev/moq/pull/8136) and +[moq-dev/moq#8088](https://github.com/moq-dev/moq/pull/8088) also edit +`session.ts`. Expect merge conflicts. diff --git a/quest/m1/explicit-state/rule.md b/quest/m1/explicit-state/rule.md new file mode 100644 index 00000000000..2385d618dd9 --- /dev/null +++ b/quest/m1/explicit-state/rule.md @@ -0,0 +1,31 @@ +# [S] Review and agent rule + +## Goal + +Reviews and authoring agents catch the four implicit-state shapes the +[epic](/quest/m1/explicit-state/README.md) lists before they land. REVIEW.md +has a check, and AGENTS.md has one line, saying: + +- a lifecycle is one enum or union whose variants carry their data; +- a terminal transition happens after the work that can fail, or records + the failure; +- one value is never guarded by two locks or split across atomics. + +## Plan + +Decided 2026-11-07 (maintainer): ✅ both REVIEW.md and AGENTS.md. Not +chosen: REVIEW.md only, or no rule change. This is the prompt that the +repository's rule on changing agent instructions requires. + +Follow `quest context`'s Prompting section: keep the AGENTS.md line short, +since it loads into every session, and put the detail and examples in +REVIEW.md. Point at the patterns already in the tree as the shape to copy: + +- `PresentState` in `rs/moq-net/src/lite/session.rs` +- `Join.State` in `js/net/src/ietf/join.ts` +- `CompletionState` in `rs/moq-gst/src/sink/session.rs` +- the consuming `finish(self)` in `rs/moq-rtmp/src/server.rs` + +Check the official guidance the Prompting section links before editing. + +This child lands first so the other children's reviews apply the check. diff --git a/quest/m1/explicit-state/track-end.md b/quest/m1/explicit-state/track-end.md new file mode 100644 index 00000000000..092a290e4df --- /dev/null +++ b/quest/m1/explicit-state/track-end.md @@ -0,0 +1,66 @@ +# [XL] Track and group end-of-life + +## Goal + +In moq-net and js/net, each track and group has one outcome value: open, +ended, sealed, aborted, or dropped. Every predicate that currently re-derives +an outcome from separate flags reads that value instead, and in JS one +`closed` handle stays stable for a consumer's whole life. The Rust and JS +models change together, as the cross-package sync rule requires. + +## Plan + +The shared decisions are in the [epic](/quest/m1/explicit-state/README.md). +This is moq-net's hot path, so benchmark it as the epic says. + +Bug to reproduce and fix: + +- **`recoverGroup` replaces the stable handle** (`js/net/src/group.ts`). + - The bug: `hooks.recoverGroup` resets `group.#closed = undefined`. A + caller that read `closed` before the recovery keeps the old handle, + latched to the error, while a new read builds a handle that can settle + `null`. That breaks the `CombinedClosed` doc's promise of one stable + public handle. + - Decide in the PR whether a non-final error that a recovery continues + past may ever reach a caller. If the current behavior turns out to be + intended, keep it and document it in one place. + +Instances to redesign: + +- **Rust `TrackState`** (`rs/moq-net/src/model/track.rs`): + - Eight fields hold its end-of-life: `end`, `end_error`, `sealed`, + `closed`, `abort`, `resume`, `settled`, and `tail`. + - `end` and `abort` can both be set, and which outcome wins depends on + whether `settled` latched first. + - `sealed || abort.is_some()` is repeated at five or more sites. + - The start and anchor flag triples (`start_sequence`, `start_pending`, + `start_resolved`; `received`, `anchored`, `cached_anchors`) belong to + the same redesign. +- **Rust group** (`rs/moq-net/src/model/group.rs`): + - `GroupState` keeps `fin` and `abort` side by side. + - `Alive` copies the abort into three atomics (`cut`, `failed`, + `aborted`), and readers can see the steps in between. One atomic phase + should replace them. +- **JS track** (`js/net/src/track.ts`): + - `TrackState` keeps `end` as a Signal and `endError` as a plain field + that has to be set first. + - The `Producer` wire-mode flags (`#subscribed`, `#served`, + `#tailPending`, `#starts`). + - `Subscriber` has two closed sources, `#closed` and `#state.closed`, and + its guards check them inconsistently. +- **JS group** (`js/net/src/group.ts`): `#terminal` and `#verdict` are synced + at seven sites, and one of them assigns without `??=`. +- **Resume** (`js/net/src/resume.ts` `Reader`, `rs/moq-net/src/model/resume.rs` + `Copy`): + - Reproduce whether a retained copy can refuse after a clean end + (`#learnEnd` checks only `#closed`, not `#ended`). If it can, `closed` + reports clean while reads fail. + +[Group errors are cached by kind](/quest/m1/group-error-cache.md) and +[Drop set](/quest/m1/drop-set.md) edit the same track state. Coordinate with +whichever lands first rather than wait. + +## Related + +- [Group errors are cached by kind](/quest/m1/group-error-cache.md) - changes how group outcomes are recorded +- [Drop set](/quest/m1/drop-set.md) - rewrites the track's dropped-run state diff --git a/quest/m1/explicit-state/transport-close.md b/quest/m1/explicit-state/transport-close.md new file mode 100644 index 00000000000..20fbd8c0a7d --- /dev/null +++ b/quest/m1/explicit-state/transport-close.md @@ -0,0 +1,76 @@ +# [L] Transport close + +## Goal + +The Rust connection and QUIC transport state each live in one phase value: + +- Once moq-tokio's reconnecting connection gives up, its status stream yields + the terminal error next, never a `Disconnected` first, even on a handle + that never connected. The bindings that expose status report the same + sequence. +- The moq-tokio, moq-uring, and qmux backends agree on when `closed()` + resolves for a local close. +- A closed or driverless endpoint can't be revived. + +## Plan + +The shared decisions are in the [epic](/quest/m1/explicit-state/README.md). +Decided 2026-11-07 (maintainer): ✅ `quic::driver::Endpoint::rebind` becomes +crate-private, since only tests call it. Not chosen: refusing to rebind after +close. This is breaking, so if the rest of this child stays additive, split +the `rebind` change into its own m2 quest. + +Bugs to reproduce and fix: + +- **Trailing `Disconnected`** (`rs/moq-tokio/src/connection.rs`): + - `Shared::end` calls `state.offline()`, which writes + `status = Some(Disconnected)` in the same revision that sets `ended = true`. + - `poll_status` matches only `status`, and kio's `Consumer::poll` runs the + closure before it reports a closed channel (`rs/kio/src/consumer.rs`). So + the caller gets `Ok(Disconnected)` first and only then `Err(terminal)`. + - That contradicts `poll_status`'s doc and `end`'s own comment. + - Recommended: `enum Phase { Connecting { failure }, Live { session, + version, transport, migrating: Option }, Offline { failure }, + Closed(Option) }`, with `Status` and `Observation` derived from it. +- **`rebind` after close** (`rs/moq-tokio/src/quic/driver/endpoint.rs`): + - `rebind` checks neither `driver_lost` nor the pending close, and + `slot.replace(socket)` refills a released slot. + - The result is a bound port that nothing reads. + - Recommended: `enum Lifecycle { Running, Closing { code, reason }, + Released, DriverLost }` on the endpoint state, checked by every entry + point. + +Instances to sweep: + +- `rs/moq-quic/src/connection/mux/mod.rs`, QMux `Connection`: + - `params_sent`, `peer`, `error`, `close`, `close_sent`, and a reused + `conn_state`, with `error.is_some()` checked at about 11 sites. + - `conn_state` starts as `Established` while handshaking. + - Recommended: `Phase { Handshake, Established(Params), Closing { error, + frame }, Closed(error) }`. +- `rs/moq-uring/src/quic/driver/connection.rs` and `rs/qmux/src/session.rs`: + - `closed`, `local_close`, `dead`, and `close_staged` beside the connection. + - moq-tokio resolves `poll_closed` on a local close at once, while the + other two wait for a staged latch. Settle on one behavior. +- `rs/moq-quic/src/connection/streams/recv.rs` and `lease.rs`: `stopped` plus + `evicted`. Also `send.rs`: the peer's STOP lives only in `stop_reason`, and + `fin_pending` and `released` sit beside `SendState`. +- `rs/moq-quic/src/connection/paths.rs`: `challenge` plus + `challenge_pending` plus `validated`. +- `rs/moq-quic/src/connection/mod.rs`: `close` and `draining_reported` beside + `State`. +- `rs/moq-tokio/src/quic/web/session.rs`: WebTransport and raw sessions share + one struct of `Option`s. `rs/moq-uring/src/quic/web.rs` already groups + most of it. +- `rs/moq-relay/src/cluster.rs` (`DialersPhase`) and + `rs/moq-relay/src/shutdown.rs` (`Serving`, whose `closed()` underflows if + it is called twice). +- `rs/qmux/src/driver.rs` (`Timer.closing` duplicates `Phase::Closing`) and + `rs/qmux/src/session.rs` (`SendStream` keeps `fin` and `reset`, and folds + STOP into `fin`). + +Keep the BBR congestion fields that mirror the draft spec as they are unless +a phase enum makes them clearer: they port a published algorithm. + +[moq-dev/moq#8139](https://github.com/moq-dev/moq/pull/8139) edits +`connection.rs`'s tests only. diff --git a/quest/m1/explicit-state/video-drain.md b/quest/m1/explicit-state/video-drain.md new file mode 100644 index 00000000000..929146ceaf3 --- /dev/null +++ b/quest/m1/explicit-state/video-drain.md @@ -0,0 +1,60 @@ +# [M] Video codec drains + +## Goal + +When a moq-video decoder's or encoder's flush or drain fails, for example a +drain that times out, the call returns with the codec either ready for input +again or terminally failed. It is never left in end-of-stream or stopped +while the front end keeps feeding it. A decoder opens its keyframe gate only +once a keyframe has actually decoded. This holds for MediaCodec and V4L2 +decode and encode, and for every other backend with the same drain shape. + +## Plan + +The shared decisions are in the [epic](/quest/m1/explicit-state/README.md). + +Real MediaCodec and V4L2 devices aren't in CI, so reproducing these may need +a seam the backends drive the codec through. The `vicodec`/`visl` virtual +V4L2 drivers are worth checking for the V4L2 half. + +The front end retries a failed read on purpose +(`rs/moq-video/src/decode/consumer.rs`: "Left idle, so the next read asks it +again"). That retry turns each hole below into a stuck codec: + +- **MediaCodec decode** (`decode/backend/mediacodec.rs`): + - `flush` returns at `drain_tail()?` before it reaches + `if self.fed { codec.flush() }`, so a drain that never sees EOS leaves the + codec in end-of-stream. + - `fed: bool` can't tell "fed" from "EOS already signalled". The next + `flush` signals EOS again instead of flushing, so the decoder stays stuck. + - Recommended: `enum Input { Idle, Fed, EndSignalled }`, flushing the codec + whenever it is `EndSignalled`. +- **V4L2 decode** (`decode/backend/v4l2.rs`): `V4L2_DEC_CMD_STOP` followed by + a failed `drain_sequence` skips `V4L2_DEC_CMD_START`, so the decoder stays + stopped and nothing records it. +- **V4L2 encode** (`encode/backend/v4l2.rs`): + - It has the same STOP/START hole. + - `start()` can leave the raw queue streaming with the coded queue off. + After that, `encode` never retries and `drain_tail` returns empty. +- **MediaCodec encode** (`encode/backend/mediacodec.rs`): `ended` is set + before the drain loop, and `encode` never checks it. +- **Keyframe gate** (`decode/decoder.rs`): + - `got_keyframe = true` is set before `backend.decode(...)?`. + - A keyframe that fails to decode still opens the gate, and the deltas that + follow reach a backend with no reference. + +Every caller ends the stream on an encoder error today, so the encoder holes +are latent. Fix them with the same state anyway rather than leave the twins +different. + +Also sweep: + +- `encode/backend/mediafoundation.rs`: five bools. +- `decode/backend/videotoolbox.rs`: three `Option`s kept in lockstep, and a + rebuild drops the old session without draining it. +- `encode/backend/v4l2.rs` and `rs/moq-v4l/src/io/*/stream.rs`: `active` plus + `held`, where a half-start leaves buffers queued. +- `vaapi.rs` and `vpx.rs`: the small flag pairs. + +[moq-dev/moq#7916](https://github.com/moq-dev/moq/pull/7916) also edits +`decode/decoder.rs`, so expect a merge conflict. diff --git a/quest/m1/explicit-state/watch-playback.md b/quest/m1/explicit-state/watch-playback.md new file mode 100644 index 00000000000..222e3eaa205 --- /dev/null +++ b/quest/m1/explicit-state/watch-playback.md @@ -0,0 +1,59 @@ +# [M] Watch playback phases + +## Goal + +`` counts audio and video stalls, stalled time, and the echo +feedback that grades a viewer as stalled only for a rendition that was +playing and then ran dry. None of these count: + +- a retired, failed, or removed audio rendition; +- a switch to `delay="instant"`; +- a video poster waiting for the clock; +- a disabled decoder. + +Each decoder's playback and rendition-switching phase is one value. + +## Plan + +The shared decisions are in the [epic](/quest/m1/explicit-state/README.md). + +Bugs to reproduce and fix: + +- **Audio** (`js/watch/src/audio/decoder.ts`): + - `#retire()` clears the consumer but keeps `#playing` (the rendition key) + and `#played`. + - `#runStalls` counts whenever the ring reports stalled and + `#played && #playing`. So switching to instant mode, a decode failure, or + the rendition leaving the catalog each count a stall that keeps growing + for as long as the state lasts. + - `#out.timestamp` and `#out.stalled` also outlive the ring that fed them. + - Recommended: `{ kind: "stopped" } | { kind: "running", key, consumer } | + { kind: "failed", failure }`, counting only in `running`. +- **Video** (`js/watch/src/video/decoder.ts`): + - The min-delay `DecoderTrack` posts a poster frame before the clock + reaches it, and `#runActive` sets `#played` for any frame, poster + included. + - With a delay over the 500 ms buffering timer, every cold start or resume + counts as a stall, which contradicts the comment that the wait for the + first picture is no stall. + - `#runBuffering` also returns early when the decoder is disabled and + leaves `stalled` set, so the buffering spinner can stay on with + `visible="never"`. + - Recommended: separate poster and playing phases, counting only the move + from playing to stalled. + +Also sweep the video decoder's rendition switching (`#active`, `#pending`, a +closure-local `pending`, `#selected`, `#failed`), which keeps two copies of +"pending". + +Check whether turning on `lookahead` mid-stream re-enters cold startup +anchored at the stream's first picture (`js/watch/src/sync.ts`, `started` is +only kept in buffered mode). If it does, fix it with the sync clock's phase. + +[moq-dev/moq#8058](https://github.com/moq-dev/moq/pull/8058) (audio +time-stretch) rewrites much of the audio decoder. This child doesn't wait on +it, so expect a merge conflict. + +## Related + +- [Zero steady-state audio underruns](/quest/m1/audio-jitter-target/watch.md) - edits the same audio decoder and reads its stall counter diff --git a/quest/m2/subscribe-ranges/start-bookkeeping.md b/quest/m2/subscribe-ranges/start-bookkeeping.md index 4701a4cd0e5..6591c529870 100644 --- a/quest/m2/subscribe-ranges/start-bookkeeping.md +++ b/quest/m2/subscribe-ranges/start-bookkeeping.md @@ -47,6 +47,17 @@ three wire generations of "start". `active.start == active.requested` gate (`lite/subscriber.rs`, `apply_response`). +- The start state must make it impossible to resolve without settling what + was given up before the start. Today there are two paths. `resolve()` + drains `skipped`, while the shortcut that answers a start past the live + edge at once (the "start past everything" branch in `TrackRun`'s poll) only + sets `self.start`. So any drop collected while unresolved is never sent, + and the subscriber waits out its grace for those groups. Reproduce this + first. Found by the 2026-11-07 audit of implicit state machines (see + [Lifecycles are explicit state](/quest/m1/explicit-state/README.md)). + Recommended: `enum Start { Legacy, Unresolved { skipped, first }, + Resolved { start, served } }`, with `skipped` moved out at the transition. + Public API: none expected. Wire: lite-07 (unpublished) may send a START per update; published versions are unchanged.