From 3df9735e9941fc64bce94cc932a5059870040c99 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 17 Oct 2026 22:14:25 -0700 Subject: [PATCH 1/4] quest(m1): plan a non-Clone group producer Split the Clone removal out of public ingest into its own quest, a follow-up to #5725. Co-Authored-By: Claude Opus 5.5 --- quest/m1/public-model/README.md | 1 + quest/m1/public-model/group-writer.md | 110 ++++++++++++++++++++++++++ quest/m1/public-model/ingest.md | 11 +++ 3 files changed, 122 insertions(+) create mode 100644 quest/m1/public-model/group-writer.md diff --git a/quest/m1/public-model/README.md b/quest/m1/public-model/README.md index 3f5981bacde..fb4613e002b 100644 --- a/quest/m1/public-model/README.md +++ b/quest/m1/public-model/README.md @@ -101,6 +101,7 @@ Work owned by this README, after the children land: ## Required +- [One group writer](/quest/m1/public-model/group-writer.md) - `group::Producer` is not `Clone`; the model and sessions hold a crate-private handle that cannot write or finish - [Public ingest](/quest/m1/public-model/ingest.md) - a custom subscriber writes received groups, frames, starts, and ends into a track through public calls - [Public serving](/quest/m1/public-model/serve.md) - a custom publisher reads a track's start, cursor, and group expiry for its wire through public calls - [Public routes and answering](/quest/m1/public-model/answer.md) - a custom relay ingests routes, answers broadcast and track requests on demand, and advertises without echoing a peer's routes back diff --git a/quest/m1/public-model/group-writer.md b/quest/m1/public-model/group-writer.md new file mode 100644 index 00000000000..8bd30d75140 --- /dev/null +++ b/quest/m1/public-model/group-writer.md @@ -0,0 +1,110 @@ +# [M] One group writer + +## Goal + +`moq_net::group::Producer` is not `Clone`. Each group has exactly one +writer, so every frame write, `finish`, and abort on it is ordered by +ownership and `&mut self` rather than by lock ordering. The track cache, the +reader handoff, and the built-in sessions hold a crate-private handle that +cannot write or finish. No reader-visible change except the teardown below, +and no wire change. + +## Plan + +Follow-up to #5725 (live largest), chosen by the maintainer there: its +decision 10 ordered frame writes against `finish` with the group lock because +a frame write on one clone could race `finish` on another. Base the work on +`main` after #5725 lands. + +This is the "drop public `Clone`" half of the owned frame writer decision in +[Public ingest](/quest/m1/public-model/ingest.md) (settled 2026-10-16, +#5717), split out because it stands alone. Ingest keeps the public owned +chunked writer (`create_frame(self)`); this quest only makes it possible. + +What clones a group producer today (compile probe on #5725's head, deleting +`impl Clone`): + +- Outside moq-net, tests only: moq-ffi `MoqGroupProducer::used`, + `rs/moq-net/benches/track.rs` (2), and `rs/moq-net/tests/loom.rs`. +- moq-net unit tests: `group.rs` (6), `track.rs` (2), the IETF publisher + (1) and subscriber (1). +- 13 library sites in moq-net, in four groups: + - Track cache slots (`Slot::group` and the fetched `head` in `track.rs`, + from `insert_group`): cache accounting, eviction, demotion, `bound`, + reads, and abort. + - Reader handoff: `poll_recv_group` and `poll_next_in_range` return a + clone that the caller turns into a consumer. + - Abort through a shared reference, because `abort(self)` consumes: + `Slot::abort`, track abort, the IETF subscriber's fill, and + `group::Producer::frame_abort`. + - Wire ingest: `create_frame_owned` (`frame::ProducerOwned` holds a clone + across polls), the IETF ingest's `writing` clone, the observers for + `demand` (`poll_unused`, `abort_unused`) and `closed` (`poll_closed`), + and the lite subscriber's `reveal_group`. + +Shape: + +- One public writer, `group::Producer`, with no `Clone`. +- A crate-private handle minted from the writer that reads group state, + hands out consumers, waits on `poll_closed` and `poll_unused`, aborts + through `&self` (`abort`, `abort_unused`), and backs the cache slot's + accounting and `reveal_group`. It has no write or finish. The 13 sites + above move onto it. It stays crate-private until a third-party session + needs to abort or watch a group it does not own; the ingest decision + already says that need gets a non-writing handle, never `Clone`. +- `frame::ProducerOwned` takes the writer by value and hands it back when + the frame finishes, instead of cloning it. The wire drivers keep the + handle for `poll_closed` and demand while a frame is open. Making that + owned writer public stays with ingest. +- `answer.md`'s public `poll_unused` and `abort_unused` stay on the writer; + the handle carries the same calls for the sessions. +- Once one writer orders writes against `finish`, simplify the group-lock + ordering #5725 added for that race if nothing else needs it, and replace + the loom model `a_frame_racing_finish_never_reopens_the_largest` (it + races two handles, which no longer compiles) with a `compile_fail` + doctest that `let _: group::Producer = producer.clone();` is refused. + Keep any loom coverage that still races a writer against the handle's + abort. + +Behavior change to verify: the "dropped without `finish()` or `abort()`" +teardown (`group::Alive`'s `Drop`, which warns and releases the cached +frames) runs today only when the last clone drops, and the cache slot holds +one, so a writer that drops an unfinished group leaves its readers waiting +until the slot goes. After the split it runs when the writer drops. Split +`Alive` so the handle shares the atomics (`aborted`, `cut`, `checked`, +`access`) but not the drop guard. Then check, each with a mock-time test: + +- A reader of an unfinished group sees `Dropped` when its writer drops, and + the cache releases the frames. +- A route that dies mid-group still resumes on its replacement from the + committed frame (`resume_frame`, `resume_position`), which reads the count + that outlives the released cache. +- The sessions' `crate::recv::Group` guard and moq-ffi's + `MoqGroupProducer` still end every group they drop with a terminal. + +If any of those reads worse than today, bring it to the maintainer before +landing. + +Open questions, the maintainer's call (do not settle them in the +implementation PR without asking): + +- The crate-private handle's name. +- Whether the public `Error::FrameOpen` goes. Ingest kept it as a backstop + for crate-internal duplicates that could write, which this shape removes. + [Dropped sources](/quest/m1/dropped-sources.md) lists it among the + variants that need a real arm; update it there if it goes. +- `finish(&self)` today: whether it becomes `finish(&mut self)` like every + other write. +- `Deref`: a downstream `producer.clone()` now silently + copies the `Copy` `Info` and fails later. Ingest left "say so in the + changelog, or drop the `Deref`" open. + +Landing: breaking public API (no `Clone`), so `rs/moq-net/MIGRATE.md` gets +an entry under `## Unreleased` naming the `Info` pitfall, plus moq-ffi's if +its exported surface changes (it should not). JS has no equivalent: a JS +`Group.Producer` is a single object and needs no change. Docs: none beyond +MIGRATE, since `doc/` never documents cloning a group producer. + +## Required + +- [Live largest location](/quest/m1/largest.md) - #5725, in the merge queue; this quest builds on its group-lock ordering and loom model diff --git a/quest/m1/public-model/ingest.md b/quest/m1/public-model/ingest.md index fdc1cbfe0a0..5e42048cb3b 100644 --- a/quest/m1/public-model/ingest.md +++ b/quest/m1/public-model/ingest.md @@ -76,6 +76,13 @@ Rust items (`rs/moq-net/src/model/`), as they stood at b4eab0db1: crate-internal duplicates. `track::Producer` keeps `Clone`: several writers creating groups is legitimate, and `claim_sequence` already refuses a duplicate sequence. + - Dropping `Clone` and the crate-private non-writing handle moved to + [One group writer](/quest/m1/public-model/group-writer.md), which + settles the handle and the fate of `FrameOpen`. This item keeps the + public owned chunked writer. Its callers that hold a + `&mut group::Producer` (hang's `Frame::write_to`, moq-mux's LOC and + fMP4 encoders) cannot call a consuming `create_frame(self)`; weigh + that before landing it. - Compile probe at dd95d5804: 26 `group::Producer` clone sites, 16 model-internal and 7 in the sessions (kept by the crate-private duplicate), and 3 outside the crate (2 in @@ -189,6 +196,10 @@ Tests: each public call gets a mock-time test in its language, and the built-in subscribers move onto the public calls, so their existing suites cover the behavior. +## Required + +- [One group writer](/quest/m1/public-model/group-writer.md) - the non-Clone writer the owned chunked writer builds on + ## Related - [Public model after ranges, handover, and the JS tail](/quest/m1/public-model/deferred.md) - the fetch ingest, `mirror`, and `endTrack` items split out of this quest From 11d42f85fbf2cda6191f3e652e13407e05636a1e Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 17 Oct 2026 22:17:32 -0700 Subject: [PATCH 2/4] quest(m1): record the group writer decisions The maintainer settled the handle name, FrameOpen, finish, and Deref on #5892. Co-Authored-By: Claude Opus 5.5 --- quest/m1/public-model/group-writer.md | 34 +++++++++++++-------------- quest/m1/public-model/ingest.md | 3 ++- 2 files changed, 19 insertions(+), 18 deletions(-) diff --git a/quest/m1/public-model/group-writer.md b/quest/m1/public-model/group-writer.md index 8bd30d75140..34ff943f67b 100644 --- a/quest/m1/public-model/group-writer.md +++ b/quest/m1/public-model/group-writer.md @@ -45,7 +45,7 @@ What clones a group producer today (compile probe on #5725's head, deleting Shape: - One public writer, `group::Producer`, with no `Clone`. -- A crate-private handle minted from the writer that reads group state, +- A crate-private `group::Handle` minted from the writer that reads group state, hands out consumers, waits on `poll_closed` and `poll_unused`, aborts through `&self` (`abort`, `abort_unused`), and backs the cache slot's accounting and `reveal_group`. It has no write or finish. The 13 sites @@ -85,22 +85,22 @@ until the slot goes. After the split it runs when the writer drops. Split If any of those reads worse than today, bring it to the maintainer before landing. -Open questions, the maintainer's call (do not settle them in the -implementation PR without asking): - -- The crate-private handle's name. -- Whether the public `Error::FrameOpen` goes. Ingest kept it as a backstop - for crate-internal duplicates that could write, which this shape removes. - [Dropped sources](/quest/m1/dropped-sources.md) lists it among the - variants that need a real arm; update it there if it goes. -- `finish(&self)` today: whether it becomes `finish(&mut self)` like every - other write. -- `Deref`: a downstream `producer.clone()` now silently - copies the `Copy` `Info` and fails later. Ingest left "say so in the - changelog, or drop the `Deref`" open. - -Landing: breaking public API (no `Clone`), so `rs/moq-net/MIGRATE.md` gets -an entry under `## Unreleased` naming the `Info` pitfall, plus moq-ffi's if +Decided by the maintainer on 2026-10-18 (#5892): + +- m1, under the public-model epic, since #5717 already put the Clone + removal there. The public owned chunked writer stays in ingest, which + requires this quest. +- The handle is `group::Handle`. +- Delete the public `Error::FrameOpen`: one writer makes it unreachable. + The in-flight-frame check becomes a `debug_assert`. Update any remaining + references to it. +- `finish(&mut self)`, like every other write. +- Keep `Deref`. A downstream `producer.clone()` now silently + copies the `Copy` `Info` and fails later, so MIGRATE.md names that. + +Landing: breaking public API (no `Clone`, no `FrameOpen`, `finish` takes +`&mut self`), so `rs/moq-net/MIGRATE.md` gets an entry under +`## Unreleased` naming the `Info` pitfall, plus moq-ffi's if its exported surface changes (it should not). JS has no equivalent: a JS `Group.Producer` is a single object and needs no change. Docs: none beyond MIGRATE, since `doc/` never documents cloning a group producer. diff --git a/quest/m1/public-model/ingest.md b/quest/m1/public-model/ingest.md index 5e42048cb3b..b1cd3792c19 100644 --- a/quest/m1/public-model/ingest.md +++ b/quest/m1/public-model/ingest.md @@ -78,7 +78,8 @@ Rust items (`rs/moq-net/src/model/`), as they stood at b4eab0db1: refuses a duplicate sequence. - Dropping `Clone` and the crate-private non-writing handle moved to [One group writer](/quest/m1/public-model/group-writer.md), which - settles the handle and the fate of `FrameOpen`. This item keeps the + names the handle `group::Handle` and deletes `FrameOpen` (decided + 2026-10-18, #5892), so the backstop above goes. This item keeps the public owned chunked writer. Its callers that hold a `&mut group::Producer` (hang's `Frame::write_to`, moq-mux's LOC and fMP4 encoders) cannot call a consuming `create_frame(self)`; weigh From f7a02c62a2821817be6e06334a30c63840535a91 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 17 Oct 2026 22:19:05 -0700 Subject: [PATCH 3/4] quest(m1): a dropped group writer marks the group dead A surviving handle must not keep serving a released group or block its replacement. Spell out the MIGRATE.md replacements for former clones. Co-Authored-By: Claude Opus 5.5 --- quest/m1/public-model/group-writer.md | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/quest/m1/public-model/group-writer.md b/quest/m1/public-model/group-writer.md index 34ff943f67b..b3f175a8076 100644 --- a/quest/m1/public-model/group-writer.md +++ b/quest/m1/public-model/group-writer.md @@ -72,10 +72,19 @@ frames) runs today only when the last clone drops, and the cache slot holds one, so a writer that drops an unfinished group leaves its readers waiting until the slot goes. After the split it runs when the writer drops. Split `Alive` so the handle shares the atomics (`aborted`, `cut`, `checked`, -`access`) but not the drop guard. Then check, each with a mock-time test: +`access`) but not the drop guard. The guard must now mark the group dead, +not only release its frames: today it leaves `aborted` false and +`GroupState::abort` unset, which was harmless because no handle outlived +it. A surviving `Handle` would then keep serving the released group +(`Slot::is_live`), and `covering_group` would refuse a same-sequence +replacement as `Duplicate`. So a writer dropped unfinished commits an abort +(`Dropped`) through the same path as `abort`. Then check, each with a +mock-time test: - A reader of an unfinished group sees `Dropped` when its writer drops, and the cache releases the frames. +- After that drop, a new reader does not get the dead group, and the + publisher can recreate the same sequence. - A route that dies mid-group still resumes on its replacement from the committed frame (`resume_frame`, `resume_position`), which reads the count that outlives the released cache. @@ -100,7 +109,11 @@ Decided by the maintainer on 2026-10-18 (#5892): Landing: breaking public API (no `Clone`, no `FrameOpen`, `finish` takes `&mut self`), so `rs/moq-net/MIGRATE.md` gets an entry under -`## Unreleased` naming the `Info` pitfall, plus moq-ffi's if +`## Unreleased` that says how to migrate: keep the one writer, and watch +readers or closure through `demand()` (its `closed()`), read through +`consume()`, and move a writer that several places used into one owner +(pass it by value, or guard it with a lock). It also names the `Info` +pitfall, `FrameOpen`'s removal, and `finish(&mut self)`. moq-ffi's gets one if its exported surface changes (it should not). JS has no equivalent: a JS `Group.Producer` is a single object and needs no change. Docs: none beyond MIGRATE, since `doc/` never documents cloning a group producer. From fcee2aa686d4f2bb7738f24b8bf3fb775ad369a3 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 17 Oct 2026 22:29:56 -0700 Subject: [PATCH 4/4] quest(m1): unblock the group writer now that #5725 landed Co-Authored-By: Claude Opus 5.5 --- quest/m1/public-model/group-writer.md | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/quest/m1/public-model/group-writer.md b/quest/m1/public-model/group-writer.md index b3f175a8076..578e67f3f2e 100644 --- a/quest/m1/public-model/group-writer.md +++ b/quest/m1/public-model/group-writer.md @@ -13,8 +13,7 @@ and no wire change. Follow-up to #5725 (live largest), chosen by the maintainer there: its decision 10 ordered frame writes against `finish` with the group lock because -a frame write on one clone could race `finish` on another. Base the work on -`main` after #5725 lands. +a frame write on one clone could race `finish` on another. This is the "drop public `Clone`" half of the owned frame writer decision in [Public ingest](/quest/m1/public-model/ingest.md) (settled 2026-10-16, @@ -117,7 +116,3 @@ pitfall, `FrameOpen`'s removal, and `finish(&mut self)`. moq-ffi's gets one if its exported surface changes (it should not). JS has no equivalent: a JS `Group.Producer` is a single object and needs no change. Docs: none beyond MIGRATE, since `doc/` never documents cloning a group producer. - -## Required - -- [Live largest location](/quest/m1/largest.md) - #5725, in the merge queue; this quest builds on its group-lock ordering and loom model