diff --git a/quest/m1/README.md b/quest/m1/README.md index 17b74156dcc..0bb28bffd77 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -48,7 +48,7 @@ session outcomes, and embedder refusals inside session outcomes. - [Interop close logs](/quest/m1/interop-close-log.md) - logging fix landed; delete once the required Interop run is green - [One max_age meaning](/quest/m1/cache-max-age.md) - a superseded group goes stale on wall clock since its successor arrived or on media time, whichever is first, in Rust and js/net; fixes the untimed failover stall - [FETCH max-delay](/quest/m1/fetch-max-delay.md) - historical readers obey publisher max-age and an optional tighter reader budget, including while holding a group -- [Upstream position regression](/quest/m1/largest-regression.md) - a relay copy that sees upstream's largest group go backwards on moq-transport or epochless lite-07 fails loud instead of serving the old instance's cache +- [Claim answer epochs](/quest/m1/claim-answer-epochs.md) - a claim answers each output by publishing it at its exact path under its own epoch, so a restarted output never reaches a returning reader as the old one's cache - [Untimed decisions](/quest/m1/untimed-decisions.md) - the maintainer decides whether moq-archive keeps refusing untimed tracks, and whether a malformed FETCH object ends its track - [FFI shape](/quest/m1/ffi-shape/README.md) - the bindings mirror Rust's layers: net at the root, then media, json, flate, audio, and video namespaces built from the handle below - [Publishing never invents a timestamp](/quest/m1/publish-timestamp.md) - no Rust or binding publish API fills in a timestamp; an untimed payload goes out untimed diff --git a/quest/m1/announce-takeover.md b/quest/m1/announce-takeover.md index 3f1b732b311..490c6e911ac 100644 --- a/quest/m1/announce-takeover.md +++ b/quest/m1/announce-takeover.md @@ -17,7 +17,8 @@ IETF counterpart remains separate. Ordinary Start/Restart is a supported fallback, including the accepted cold-start reset before this lands. - A worker advertises `transcode/` without an epoch. Demand for a path starts - one job, which announces `transcode/path` with its own freshly minted epoch. + one job, which announces `transcode/path` with its own freshly minted epoch + through `Request::prepare` then `publish`, the claim-to-route operation. Lazy catalog references and initial requests need no anticipated epoch. Identical encoder settings do not make two workers the same instance. - TAKEOVER is START plus a predecessor Announce ID: it carries the child @@ -81,12 +82,13 @@ event, not only route-table checks. Measure route/readers fanout on both axes. Run `just check`, `just test interop --all`, and `just drafts check`; wire regressions and benchmarks into existing CI lanes. -Public API: a claim-to-independent-route operation and a `Takeover` event, -mirrored across languages and exposed bindings. Wire: lite-07-wip gains +Public API: a `Takeover` event, mirrored across languages and exposed +bindings, on top of `Request::prepare` and `publish`. Wire: lite-07-wip gains ANNOUNCE_TAKEOVER; published versions retain their framing. ## Required +- [Claim answer epochs](/quest/m1/claim-answer-epochs.md) - `Request::prepare` then `publish` answers a claim at the exact path under a fresh epoch, the operation TAKEOVER continues - [Routes and announces](/quest/m1/cluster-routing/routes.md) - shared routing layer and per-stream identity ## Related diff --git a/quest/m1/claim-answer-epochs.md b/quest/m1/claim-answer-epochs.md new file mode 100644 index 00000000000..4a28eb1d08b --- /dev/null +++ b/quest/m1/claim-answer-epochs.md @@ -0,0 +1,102 @@ +# [L] A claim answers each output as its own epoch + +## Goal + +A worker behind a prefix claim that restarts a path's output does so under a +new epoch, so every hop sees the change and drops its copy: a returning reader +never gets the old output's groups. The public claim API cannot serve two +outputs at one path under the same instance. Replaces the relay-side +regression check proposed in #5057. + +## Plan + +Facts as of main 90500867d: + +- A claim (`origin::Producer::dynamic`, JS `Producer.dynamic`, moq-ffi, moq-c + and the binding wrappers) answers with any broadcast via `Request::accept`. + Every answer resolves under the claim route's instance (its epoch, or the + entry's generation), shared by every path and every answer for the claim's + lifetime. Once an answer closes, the handler is asked again and the new + broadcast, restarting at group 0, resolves under the same instance; nothing + on any wire changes, so a lingering downstream copy splices old onto new. +- `Dynamic::update` to a new epoch, or a renewal, restarts the whole prefix, + resetting every sibling path's readers. Too coarse for a pool. +- Tracks already avoid this: a broadcast keeps each track name's sequence + across replacement producers. Claims have no equivalent. +- In repo, the production claim handlers are moq-net sessions forwarding a + peer's claim; they relay upstream numbering and keep working as is. + +Decided 2026-10-12: a request is answered by publishing, not by handing over +content. The answer has two steps, so the broadcast holds its tracks (or a +`broadcast::Dynamic`) before anyone resolves through it, and the caller names +the epoch. Roughly: + + impl Request { + /// Start answering the path as a new publisher instance under `epoch`, unannounced until published. + pub fn prepare(self, epoch: Epoch) -> Result; + } + + impl Prepared { + /// Announce the filled broadcast at exactly the requested path and resolve every waiting requester. + pub fn publish(self) -> Result; + } + +`Prepared` (name open) gives access to the broadcast to fill and is the only +way to answer, so an announced empty answer can't be expressed. Dropping it +unpublished rejects the request, as dropping a `Request` does. The broadcast +is announced with the claim's route metadata and that epoch, and requesters +resolve through that route, never the claim's instance. Closing it ends the +route; the next request prepares under a new epoch (`Epoch::mint()` per run, +or a replica set's shared one). Session forwarding keeps a crate-private +accept. Rejected: a single `publish` that announces an empty broadcast, +renewing the prefix on each restart (resets every sibling), and carrying +sequences over to the next answer (relabels a different job as a +continuation). + +Decided 2026-10-12: public claims carry no epoch. A claim is a capability, +as announce takeover already treats it, so `Producer::dynamic` (JS and the +bindings alike) cannot take one; prefer a shape where the route has none over +a runtime refusal. A request pinned to an epoch resolves only through an +exact-path route with that epoch. Sessions keep epochs on the peer routes +they land internally. Migrate the in-repo callers and tests that claim with +an epoch. + +Required: validity and route insertion are atomic. A request the claim +released (an `update` to another instance) or abandoned (the claim dropped) +before `publish` is refused, never announced, as `accept` drops it today. +Regression tests cover both, and a pinned request. + +Look out for: + +- A downstream relay that resolved the path through its copy of the claim + before the exact announcement arrived holds a front under the claim's + instance. Check it can't be rejoined once another route won the path and + then ended; if it can, retire it per path, not by renewing the prefix. The + cold-start reset it implies is accepted until announce takeover lands. +- One exact announcement per live output: benchmark outputs against downstream + cursors. +- No producer-side refusal of a backwards sequence: a fresh broadcast has no + memory of the last run, and backfill below the newest group is legitimate. + +Verification: mocked time, reusing #5057's worker harness +(`tests/largest_regression.rs` on its branch, with Dryvnt credited as +co-author). A worker restarts `pool/p` within the linger, behind one relay +and behind an edge relay of ours, on lite-05, 06, 07 and moq-transport 14, +16, 22: the returning reader gets the new output from group 0, never the old +group, and sibling paths keep their readers. On the wires without an epoch +that rests on our relays retiring a copy on End then Start, as +`tests/restart.rs` shows; arbitrary older caching relays are out of scope, as +in [IETF epochs](/quest/m1/ietf-epochs.md), since base IETF object identity +survives a withdrawal. A same-epoch lagging replica still resumes, checked on +lite-07 only, the one wire here that carries the epoch. JS mirrors the tests. +`just test interop --all`. + +Public API: breaking. A claim request is answered through `prepare` and +`publish`, and a public claim takes no epoch, in Rust, JS, moq-ffi, moq-c and +every binding wrapper, plus the claim docs. Wire: none. + +## Related + +- [Announcement takeover](/quest/m1/announce-takeover.md) - its claim-to-route operation is `Request::prepare` then `publish`; TAKEOVER then removes the cold-start reset +- [IETF epochs](/quest/m1/ietf-epochs.md) - carries the answer epoch over moq-transport, so same-epoch resume and replacement can be told apart there +- [#5057](https://github.com/moq-dev/moq/pull/5057) - the relay-side check this replaces diff --git a/quest/m1/largest-regression.md b/quest/m1/largest-regression.md deleted file mode 100644 index f645ecce759..00000000000 --- a/quest/m1/largest-regression.md +++ /dev/null @@ -1,65 +0,0 @@ -# [S] A relay copy fails loud when upstream's position goes backwards - -## Goal - -A relay copy that subscribes upstream again and hears that upstream's largest -group is below the newest group it has cached treats it as a reused name: the -copy ends with an error, its source closes so the front ends, and the readers -it was serving re-request. A new request arriving as the front ends gets a -fresh front without an error. Today the copy keeps its stale live floor (`set_live` in -`rs/moq-net/src/model/track.rs`), so a viewer returning within the linger to a -publisher that restarted at group 0 gets the old instance's cached group and -then nothing until the new sequence passes it. - -Covers lite-07 and moq-transport, whose answers carry the largest position. -lite-05 and 06 stay uncovered on purpose: their only hint is the first -group's sequence, and they keep waiting on the floor. - -## Plan - -Found while scoping idle fronts (2026-10-07): -under a prefix claim the relay never sees the worker close a broadcast, so a -restart at the same path reaches a lingering copy unannounced. A publisher -that restarts its group sequence under the same name is buggy, and this makes -that bug visible instead of a silent stall. A change of route can't cause -it: every route change, including a per-path winner change under a prefix -pool, reaches downstream as a `Restart` of the route, which unsets the cached -copy of every broadcast nested under it -(maintainer, 2026-10-08, in Restart). -Only a restart behind an unchanged route reaches a copy unannounced. - -Decided 2026-10-08: shrunk to the two answer-carrying wires and moved to m1. -The heuristic is unproven and risks a false failover, so it builds on the -front and route shapes Restart and idle fronts leave rather than ahead of -them. - -Where it lands: since #4914, a rejoining copy goes idle (`set_idle`) and the -route's answer reaches `set_live`, which holds readers in `Feed::Answered` -until the cache shows the answered position. An answer below what the cache -already shows goes straight to `Feed::Live` and keeps the old floor; that is -the case to end instead, when it qualifies below. - -A lower largest group alone doesn't prove a restart: a same-epoch resume onto -a replica that has fallen behind reports one legitimately, and must keep -waiting as today. The copy ends only when the answer comes from the route it -was last served from without an epoch, where no cross-route resume happens, -or names a different epoch than the copy holds. Check for any other -legitimate case before relying on that split. - -Whatever ends a front, it leaves the origin's front table before its broadcast -closes or anything fails, and `request()` treats a front whose broadcast closed -as gone. So a reader re-requesting after seeing the front's end mints a fresh -front. The copy's error must reach readers through that end, not an `Abort` -ahead of it, or a re-request can still join the old front. A request that -joined before the front decided to end is in flight, like a subscription. - -Verification: mocked time, a publisher restarting a path at group 0 within -the linger. The returning reader gets an error and then the new instance from -group 0, never the old group. A same-epoch standby behind the cached copy -keeps the copy and resumes it. - -Public API: none. Wire: none. - -## Related - -- [Announcement takeover](/quest/m1/announce-takeover.md) - proves continuity when a lazy job gains an independent announcement; a replacement never takes over its predecessor's content