From d3d946b6631add096b9d8131bf48549bfd91c3a7 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 13 Oct 2026 13:34:03 -0700 Subject: [PATCH 1/5] quest(m1): plan a linear parked-group drain Co-Authored-By: Claude Opus 5.5 --- quest/m1/README.md | 1 + quest/m1/ffi-panic-supervision.md | 4 +++- quest/m1/parked-drain.md | 36 +++++++++++++++++++++++++++++++ 3 files changed, 40 insertions(+), 1 deletion(-) create mode 100644 quest/m1/parked-drain.md diff --git a/quest/m1/README.md b/quest/m1/README.md index 01a985a77c7..d26ea9d5288 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -86,6 +86,7 @@ above them. - [Slow source fails](/quest/m1/slow-source.md) - an export whose source runs further off real time than the output may follow fails promptly naming the rate, instead of dropping frames for hours - [Same-instance TS resume](/quest/m1/ts-same-instance-resume.md) - prove valid continuation and fix the unexpected final exporter failure without weakening identity - [Wire compat media](/quest/m1/wire-compat-media.md) - wire-compat always runs same-version media cells through the compat reader, so planned breaks can't hide its regressions +- [Parked drain](/quest/m1/parked-drain.md) - draining parked groups after a cap rise is linear in the backlog, not quadratic - [Binding panic supervision](/quest/m1/ffi-panic-supervision.md) - a panic in any moq-c or moq-ffi background task reaches the host as a panic, never a lost callback or Cancel - [More tests under load](/quest/m1/test-flakes-2/README.md) - the second round of load-only failures, one quest per flake, fixed at the cause - [Plan: a public model](/quest/m1/model-public-surface.md) - every crate-private model item is made public or unnecessary, so a custom publisher or subscriber builds from the public model alone diff --git a/quest/m1/ffi-panic-supervision.md b/quest/m1/ffi-panic-supervision.md index aa062ba1e80..e858965d903 100644 --- a/quest/m1/ffi-panic-supervision.md +++ b/quest/m1/ffi-panic-supervision.md @@ -22,7 +22,9 @@ each task's `JoinHandle` the way #5447 did for moq-c sessions, reusing the `Cancel` its dropped channel produces today. Add a regression per binding that injects a panic into one background task and asserts the host sees `MOQ_ERROR_PANIC` (moq-c) or the binding's panic error (moq-ffi), -while another object keeps working. Also add the regression #5447 left out: +while another object keeps working. A binding's origin task needs its own owner for +the driver, so its panic can be reported; wrapping the helper that aborts the +process (`moq_tokio::origin::spawn`) is not enough. Also add the regression #5447 left out: a moq-tokio accept loop that panics ends its listener. Public API: none expected, beyond the panic error reaching more callbacks. diff --git a/quest/m1/parked-drain.md b/quest/m1/parked-drain.md new file mode 100644 index 00000000000..88d66422814 --- /dev/null +++ b/quest/m1/parked-drain.md @@ -0,0 +1,36 @@ +# [S] Draining parked groups is linear in the backlog + +## Goal + +Once a subscriber's raised cap releases its parked groups, each delivery +costs O(1) amortized, so draining P parked groups is linear in P. The +`origin/resume_overlap` benchmark shows no slope over the parked count. + +## Plan + +Found on [#5446](https://github.com/moq-dev/moq/pull/5446): 1024 parked groups +drain at about a tenth of the rate of 64, on `main` as well. +`track::Subscriber::poll_recv_group` (`rs/moq-net/src/model/track.rs`) runs +`parked.retain(..)` over every parked group on every poll, re-registering an +abort waiter on each, so a drain is quadratic. + +Decided 2026-10-13 (maintainer): m1, not m0. Groups park only when a +subscriber caps its range below arriving groups and later raises the cap; +live players and default relay forwarding set no end cap, so production's +parked set is empty today. + +Recommended: validate lazily. Check only the lowest parked group for an +abort when it is about to be re-offered, and wake the subscriber from the +track (one registration) when a cached group is evicted or expires, rather +than registering on every parked group per poll. Keep the existing +guarantees: a raised `start_at` drops overtaken entries, eviction bounds the +parked set to what the cache retains, a finished track with only aborted +parked groups ends, and an abort between check and registration still wakes +the reader. Extend the benchmark with a parked-count sweep and assert the +fix flattens it. + +Public API: none. Wire: none. + +## Required + +- [Resume duplicates](/quest/m0/broadcast-epoch/resume-duplicates.md) - reshapes `track::Subscriber`'s delivery record in the same code From 74fba35727dead617e5fa45451106e78c1415a63 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 13 Oct 2026 13:38:43 -0700 Subject: [PATCH 2/5] quest(m1): prune aborted parked groups every poll; accept O(P log P) Address the Codex review: popping the aborted low end only on re-offer would hang a finished track whose cap never rises, and BTreeMap removal is O(log P), so the goal claims near-linear rather than O(1). Co-Authored-By: Claude Opus 5.5 --- quest/m1/README.md | 2 +- quest/m1/parked-drain.md | 21 ++++++++++++--------- 2 files changed, 13 insertions(+), 10 deletions(-) diff --git a/quest/m1/README.md b/quest/m1/README.md index d26ea9d5288..908f1bd087a 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -86,7 +86,7 @@ above them. - [Slow source fails](/quest/m1/slow-source.md) - an export whose source runs further off real time than the output may follow fails promptly naming the rate, instead of dropping frames for hours - [Same-instance TS resume](/quest/m1/ts-same-instance-resume.md) - prove valid continuation and fix the unexpected final exporter failure without weakening identity - [Wire compat media](/quest/m1/wire-compat-media.md) - wire-compat always runs same-version media cells through the compat reader, so planned breaks can't hide its regressions -- [Parked drain](/quest/m1/parked-drain.md) - draining parked groups after a cap rise is linear in the backlog, not quadratic +- [Parked drain](/quest/m1/parked-drain.md) - draining parked groups after a cap rise is near-linear in the backlog, not quadratic - [Binding panic supervision](/quest/m1/ffi-panic-supervision.md) - a panic in any moq-c or moq-ffi background task reaches the host as a panic, never a lost callback or Cancel - [More tests under load](/quest/m1/test-flakes-2/README.md) - the second round of load-only failures, one quest per flake, fixed at the cause - [Plan: a public model](/quest/m1/model-public-surface.md) - every crate-private model item is made public or unnecessary, so a custom publisher or subscriber builds from the public model alone diff --git a/quest/m1/parked-drain.md b/quest/m1/parked-drain.md index 88d66422814..85d1a3996d7 100644 --- a/quest/m1/parked-drain.md +++ b/quest/m1/parked-drain.md @@ -1,10 +1,11 @@ -# [S] Draining parked groups is linear in the backlog +# [S] Draining parked groups is near-linear in the backlog ## Goal Once a subscriber's raised cap releases its parked groups, each delivery -costs O(1) amortized, so draining P parked groups is linear in P. The -`origin/resume_overlap` benchmark shows no slope over the parked count. +costs at most O(log P), so draining P parked groups is O(P log P), not +quadratic. The `origin/resume_overlap` benchmark's per-group drain time no +longer grows linearly with the parked count. ## Plan @@ -19,15 +20,17 @@ subscriber caps its range below arriving groups and later raises the cap; live players and default relay forwarding set no end cap, so production's parked set is empty today. -Recommended: validate lazily. Check only the lowest parked group for an -abort when it is about to be re-offered, and wake the subscriber from the -track (one registration) when a cached group is evicted or expires, rather -than registering on every parked group per poll. Keep the existing +Recommended: validate lazily. On every poll, cap raised or not, pop aborted +groups off the low end of `parked` until the lowest is live, and wake the +subscriber from the track (one registration) when a cached group is evicted +or expires, rather than registering on every parked group per poll. Each pop +is paid for by its insertion, and the `BTreeMap`'s O(log P) removal is +accepted rather than swapping the structure. Keep the existing guarantees: a raised `start_at` drops overtaken entries, eviction bounds the parked set to what the cache retains, a finished track with only aborted parked groups ends, and an abort between check and registration still wakes -the reader. Extend the benchmark with a parked-count sweep and assert the -fix flattens it. +the reader. Extend the benchmark with a parked-count sweep that shows the fix +flattens it. Public API: none. Wire: none. From d924070ba17496c5f128e4ca64ce15e167431ece Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 13 Oct 2026 13:44:31 -0700 Subject: [PATCH 3/5] quest(m1): remove exactly the aborted parked entries; sweep subscribers Address the Codex review of 74fba35: eviction and expiry do not follow sequence order, so a low-end pop leaves aborted entries behind, and a track-wide wake adds a subscriber axis the benchmark must sweep. Co-Authored-By: Claude Opus 5.5 --- quest/m1/parked-drain.md | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-) diff --git a/quest/m1/parked-drain.md b/quest/m1/parked-drain.md index 85d1a3996d7..690955b65d1 100644 --- a/quest/m1/parked-drain.md +++ b/quest/m1/parked-drain.md @@ -20,17 +20,21 @@ subscriber caps its range below arriving groups and later raises the cap; live players and default relay forwarding set no end cap, so production's parked set is empty today. -Recommended: validate lazily. On every poll, cap raised or not, pop aborted -groups off the low end of `parked` until the lowest is live, and wake the -subscriber from the track (one registration) when a cached group is evicted -or expires, rather than registering on every parked group per poll. Each pop -is paid for by its insertion, and the `BTreeMap`'s O(log P) removal is -accepted rather than swapping the structure. Keep the existing -guarantees: a raised `start_at` drops overtaken entries, eviction bounds the -parked set to what the cache retains, a finished track with only aborted -parked groups ends, and an abort between check and registration still wakes -the reader. Extend the benchmark with a parked-count sweep that shows the fix -flattens it. +Recommended: validate lazily. Wake the subscriber from the track (one +registration) when a cached group aborts, rather than registering on every +parked group per poll, and on that wake remove exactly the parked entries +that aborted, cap raised or not, for example by walking a per-track log of +aborted sequences from the subscriber's own position. Aborted entries are not +a low-end prefix: eviction rotates recently read groups to the back and +expiry follows last access. Each removal is paid for by its insertion, and +the `BTreeMap`'s O(log P) removal is accepted rather than swapping the +structure. Keep the existing guarantees: a raised `start_at` drops overtaken +entries, eviction bounds the parked set to what the cache retains, a +finished track with only aborted parked groups ends, and an abort between +check and registration still wakes the reader. A track-wide wake fans out to +every capped subscriber, so extend the benchmark to sweep parked count +against capped subscribers per track: the parked axis flattens, and a +per-abort cost that grows with subscribers shows up as a slope. Public API: none. Wire: none. From 411667a1d5f7de9225244e927b98fc07d3a7a2c8 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 13 Oct 2026 13:49:17 -0700 Subject: [PATCH 4/5] quest(m1): state abort-cleanup requirements, not a log design Address the Codex review of d924070: the abort-log example invited an unbounded log and keyed removal by sequence, which a live group and a fetched head share. Keep the requirements and leave the mechanism to the implementer. Co-Authored-By: Claude Opus 5.5 --- quest/m1/parked-drain.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/quest/m1/parked-drain.md b/quest/m1/parked-drain.md index 690955b65d1..8bcd15f4dd9 100644 --- a/quest/m1/parked-drain.md +++ b/quest/m1/parked-drain.md @@ -23,10 +23,11 @@ parked set is empty today. Recommended: validate lazily. Wake the subscriber from the track (one registration) when a cached group aborts, rather than registering on every parked group per poll, and on that wake remove exactly the parked entries -that aborted, cap raised or not, for example by walking a per-track log of -aborted sequences from the subscriber's own position. Aborted entries are not -a low-end prefix: eviction rotates recently read groups to the back and -expiry follows last access. Each removal is paid for by its insertion, and +that aborted, cap raised or not. Aborted entries are not a low-end prefix: +eviction rotates recently read groups to the back and expiry follows last +access. A sequence can name both a live group and a fetched head, so confirm +the parked consumer itself aborted before dropping it, and keep any abort +bookkeeping bounded by what the cache retains, not by lifetime churn. Each removal is paid for by its insertion, and the `BTreeMap`'s O(log P) removal is accepted rather than swapping the structure. Keep the existing guarantees: a raised `start_at` drops overtaken entries, eviction bounds the parked set to what the cache retains, a From dcba6e62a194b70e7043eee46327a689b1292421 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 13 Oct 2026 13:53:13 -0700 Subject: [PATCH 5/5] quest(m0): move the parked-group drain to m0 A same-epoch relay handover caps the replaced copy, so groups park in production and the new route's unbounded demand drains them quadratically. Co-Authored-By: Claude Opus 5.5 --- quest/m0/README.md | 1 + quest/{m1 => m0}/parked-drain.md | 9 +++++---- quest/m1/README.md | 1 - 3 files changed, 6 insertions(+), 5 deletions(-) rename quest/{m1 => m0}/parked-drain.md (84%) diff --git a/quest/m0/README.md b/quest/m0/README.md index fa0d6b06f9b..07a90340a36 100644 --- a/quest/m0/README.md +++ b/quest/m0/README.md @@ -86,3 +86,4 @@ have fixes on `main`: macOS tests (#5335), drill-sensitivity (#5338), loom - [web-transport releases the qmux fixes](/quest/m0/qmux-credit-upstream.md) - waiting on moq-dev/web-transport#412 and #413 to merge and ship, which qmux credit bumps to - [qmux credit](/quest/m0/qmux-credit.md) - qmux returns connection credit for dropped and stopped streams and delivers its close frame, on both lines - [Broadcast epochs](/quest/m0/broadcast-epoch/README.md) - every first-party publisher that can restart mints a fresh route epoch, the newest wins a path, and only routes with the same epoch resume a subscription +- [Parked drain](/quest/m0/parked-drain.md) - draining parked groups after a cap rise, as a same-epoch relay handover causes, is near-linear in the backlog, not quadratic diff --git a/quest/m1/parked-drain.md b/quest/m0/parked-drain.md similarity index 84% rename from quest/m1/parked-drain.md rename to quest/m0/parked-drain.md index 8bcd15f4dd9..8ce8682a52b 100644 --- a/quest/m1/parked-drain.md +++ b/quest/m0/parked-drain.md @@ -15,10 +15,11 @@ drain at about a tenth of the rate of 64, on `main` as well. `parked.retain(..)` over every parked group on every poll, re-registering an abort waiter on each, so a drain is quadratic. -Decided 2026-10-13 (maintainer): m1, not m0. Groups park only when a -subscriber caps its range below arriving groups and later raises the cap; -live players and default relay forwarding set no end cap, so production's -parked set is empty today. +Decided 2026-10-13 (maintainer): m0. Groups park in production during a +same-epoch relay handover: `Copy::wire` (`rs/moq-net/src/model/resume.rs`) +caps the replaced copy's range, the lite publisher treats that end group as +a serving cap, and the new route's unbounded demand then drains the parked +set quadratically. Live players setting no cap of their own doesn't help. Recommended: validate lazily. Wake the subscriber from the track (one registration) when a cached group aborts, rather than registering on every diff --git a/quest/m1/README.md b/quest/m1/README.md index 908f1bd087a..01a985a77c7 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -86,7 +86,6 @@ above them. - [Slow source fails](/quest/m1/slow-source.md) - an export whose source runs further off real time than the output may follow fails promptly naming the rate, instead of dropping frames for hours - [Same-instance TS resume](/quest/m1/ts-same-instance-resume.md) - prove valid continuation and fix the unexpected final exporter failure without weakening identity - [Wire compat media](/quest/m1/wire-compat-media.md) - wire-compat always runs same-version media cells through the compat reader, so planned breaks can't hide its regressions -- [Parked drain](/quest/m1/parked-drain.md) - draining parked groups after a cap rise is near-linear in the backlog, not quadratic - [Binding panic supervision](/quest/m1/ffi-panic-supervision.md) - a panic in any moq-c or moq-ffi background task reaches the host as a panic, never a lost callback or Cancel - [More tests under load](/quest/m1/test-flakes-2/README.md) - the second round of load-only failures, one quest per flake, fixed at the cause - [Plan: a public model](/quest/m1/model-public-surface.md) - every crate-private model item is made public or unnecessary, so a custom publisher or subscriber builds from the public model alone