diff --git a/quest/m1/2318-js-net-remaining-capability-gaps-vs-rs-moq-net-setup-role.md b/quest/m1/2318-js-net-remaining-capability-gaps-vs-rs-moq-net-setup-role.md index 3599c61ad2f..e6fe70c7454 100644 --- a/quest/m1/2318-js-net-remaining-capability-gaps-vs-rs-moq-net-setup-role.md +++ b/quest/m1/2318-js-net-remaining-capability-gaps-vs-rs-moq-net-setup-role.md @@ -1,4 +1,4 @@ -# [S] js/net: remaining capability gaps vs rs/moq-net (SETUP role, finish_at and final sequence, range controls, typed errors) +# [M] js/net: remaining capability gaps vs rs/moq-net (SETUP role, finish_at and final sequence, range controls, typed errors) ## Goal @@ -8,8 +8,10 @@ within the issue's stated scope and boundaries. ## Plan Rescoped during the 2026-08 grooming: SETUP role, frame naming, typed errors, -and start/end cursors landed on dev. The one surviving gap: track end -semantics (finishAt / finished(), still collapsed into close(abort?)). +and start/end cursors landed on dev. Two gaps survive: track end semantics +(finishAt / finished(), still collapsed into close(abort?)), and the +producer-side `announce(prefix)` a Rust publisher has and a browser one does +not. ### Issue context @@ -20,6 +22,7 @@ Where js/net is a strict subset of the Rust model. The subscription-options mode - \[ ] **Range/cursor controls**: `start_at`, `end_at`, `get_group` (wait for a live sequence), sync cache peek, and `latest()` have no JS equivalents. Some are relay-only and fine to omit, but `startAt`/`endAt` pair with the missing subscription range fields and a JS player implementing catch-up/DVR hits this wall. - \[ ] **Typed errors**: Rust has a `#[non_exhaustive]` enum with stable wire codes; JS throws bare `Error` with prose (only `CacheFull` is a class), so consumers string-match to distinguish `NotFound` from `Unauthorized`, and wire reset codes are not surfaced. At minimum add a stable `code` property before locking the API. - \[ ] **Delete the dead `SubscribeOptions` export** (`js/net/src/track.ts`): introduced by #2167, superseded by `Subscription` in #2170, identical field-for-field, referenced nowhere. Two exported names for one concept invites drift. +- \[ ] **Producer-side `announce(prefix)`**: Rust publishers advertise a prefix route and serve whatever is requested beneath it; JS only consumes prefix routes, so a browser publisher still announces exact paths and cannot advertise a catalog it would materialize on demand. - \[ ] **Frame field naming**: JS uses `payload` for `Datagram` but `data` for both Frame types, and `@moq/hang`'s container Frame is shaped differently from both `@moq/net`'s and Rust's. Pick `payload` consistently (JS `keyframe`/`duration` are genuine WebCodecs needs and fine as additive fields). Related: #2155, #2216, #2073. diff --git a/quest/m1/2756-origin-announce-only-entries-advertise-on-demand-content.md b/quest/m1/2756-origin-announce-only-entries-advertise-on-demand-content.md deleted file mode 100644 index 249da4a6db6..00000000000 --- a/quest/m1/2756-origin-announce-only-entries-advertise-on-demand-content.md +++ /dev/null @@ -1,46 +0,0 @@ -# [M] origin: announce-only entries, advertise on-demand content without instantiating broadcasts - -## Goal - -Implement and verify the behavior tracked in [#2756](https://github.com/moq-dev/moq/issues/2756) -within the issue's stated scope and boundaries. - -## Plan - -Use the public issue's scope, implementation notes, and acceptance criteria -below as the starting plan. Reconcile paths and assumptions with the current -tree before implementation. - -### Issue context - -#### Problem - -Announcing a path requires instantiating a broadcast. `origin.publish` / `create_broadcast` put a live `broadcast::Producer` behind every announced entry, while `origin.dynamic()` broadcasts are deliberately never announced (#1772). So "announced" implies "instantiated", and there is no way to advertise content we could create on demand without actually starting to create it. - -The concrete victim is any large on-demand catalog. A VOD library or transcoder that can serve 100k recordings must either instantiate 100k broadcast producers just to appear in announce streams, or stay invisible and force consumers to know paths out of band. #2610's dynamic announce interest (`requested_announce()`) surfaces the demand to the application, but its plan still has handlers respond by creating real broadcasts, which only moves the instantiation cost from attach time to interest time. Interest in a prefix should cost an enumeration, not a fleet of broadcasts. - -#### Design - -An **announce-only entry**: a path in the origin's announce tree with no front behind it. - -- It shows up in every announce consumer and on the wire like any other announcement. The wire genuinely cannot tell: announcements never carried broadcast state, and the epoch/ended fields planned in #2610 attach naturally (`ended: true` is the VOD enumeration case). No wire change. -- A subscribe or request against it falls through to the existing dynamic/request machinery (`origin.dynamic()`, `request_broadcast`), which materializes the real broadcast on first demand. The materialized broadcast attaches at the same path and epoch and supersedes the placeholder; when it closes, the announce-only entry remains until its own handle is dropped. - -Two API flavors that compose: - -1. **Standing**: `Producer::announce(path, ...) -> guard` (JS: `origin.announce(path)`). Advertise regardless of interest; drop the guard to retract. For small known catalogs. -2. **Interest-scoped**: the `requested_announce()` handle from #2610 gains an `announce(suffix, ...)` response method. Entries live only while the interest that solicited them does: the application enumerates its catalog when some peer opens an announce stream for the prefix, and the whole subtree of announce state evaporates when the last matching stream closes. - -Together with #2708 (lazy, prefix-scoped announce solicitation) this closes a fully demand-driven loop: interest flows upstream hop by hop as narrow ANNOUNCE\_REQUESTs, the application at the origin answers with cheap announce-only entries, and broadcasts only materialize when someone subscribes. - -#### Notes - -- The interest ledger proposed on #2708 is the shared foundation; flavor 2 depends on it and on #2610's `requested_announce()`. Flavor 1 is independent and could land first. -- Model hazard to design around: announce consumers hand out `broadcast::Consumer` fronts today. An announce-only entry has none, so either the announce event carries a lazily-materializing front (resolving through the request machinery on first track subscribe), or announce events decouple from fronts. The former keeps the consumer API unchanged and is the likely shape. -- Cross-package sync per the table: `js/net`, `moq-ffi` and wrappers if the surface reaches them, `doc/concept`, and `drafts/draft-lcurley-moq-lite.md` only if any wire clarification falls out (none expected). - -Related: #2610 (dynamic announce interest, epochs, ended broadcasts), #2708 (lazy announce solicitation), #2412 (moq-gst publish-on-demand), #1772 (dynamic broadcasts are never announced). - -## Closes - -- [#2756](https://github.com/moq-dev/moq/issues/2756) - close this issue when the quest finishes diff --git a/quest/m1/2873-moq-net-enumerate-and-resolve-historical-broadcast.md b/quest/m1/2873-moq-net-enumerate-and-resolve-historical-broadcast.md deleted file mode 100644 index 7b7ced59530..00000000000 --- a/quest/m1/2873-moq-net-enumerate-and-resolve-historical-broadcast.md +++ /dev/null @@ -1,66 +0,0 @@ -# [M] moq-net: enumerate and resolve historical broadcast generations by path and epoch - -## Goal - -Implement and verify the behavior tracked in [#2873](https://github.com/moq-dev/moq/issues/2873) -within the issue's stated scope and boundaries. - -## Plan - -Use the public issue's scope, implementation notes, and acceptance criteria -below as the starting plan. Reconcile paths and assumptions with the current -tree before implementation. - -### Issue context - -#### Problem - -Ordinary broadcast discovery has one current advertisement per path. With broadcast epochs, an origin can retain multiple generations under that path, but a new consumer still has no way to discover an older or ended generation. - -This matters for recordings and offline VOD. A client should be able to enumerate historical generations without changing the ordinary live path into `/`, and without instantiating a broadcast front for every stored recording. - -#### Contract - -Add a separate history or VOD discovery surface that enumerates entries by: - -- broadcast path -- non-zero epoch -- whether the generation is live or ended - -Ordinary announcements remain unchanged in purpose and expose only the highest currently available epoch per path. - -Resolution rules: - -- `TRACK` and `FETCH` may explicitly resolve any retained generation. -- `SUBSCRIBE` accepts only a live generation. -- An explicit epoch never silently resolves to another generation. -- Epoch `0` means the current highest generation. Resolution happens before validating whether the requested operation is allowed. -- If the current generation is ended, `SUBSCRIBE(epoch=0)` fails rather than searching backward for an older live generation. -- A generation may transition from live to ended, but not back to live. Publishing new live content at the path requires a greater epoch. - -Persistence, retention, and authorization remain application policy. The protocol and model only expose the retained generations that the application makes available. - -#### Architecture constraints - -This should build on the epoch model from #2610: path, then generation, then interchangeable routes. It should also compose with #2756 so a large offline catalog can advertise cheap entries without allocating a broadcast front per recording. - -Do not overload ordinary route selection with history. Historical generation enumeration is a distinct query/API and will need its own wire design before implementation. - -#### Acceptance cases - -- A path with one live generation and multiple ended generations is announced normally as only its highest epoch. -- History enumeration returns each retained `(path, epoch, ended)` entry. -- A client can fetch an explicitly selected ended generation. -- A subscription to an ended generation is rejected. -- An explicit unknown epoch fails without falling back to current. -- An offline catalog can expose many historical generations without eagerly instantiating broadcasts. - -Related: #2610, #2756, #2281, #2275, #2846. - -## Required - -- [Plan: historical broadcast generations](/quest/m1/plan-generations.md) - split into implementable quests first - -## Closes - -- [#2873](https://github.com/moq-dev/moq/issues/2873) - close this issue when the quest finishes diff --git a/quest/m1/README.md b/quest/m1/README.md index 7e48492ed32..4484a5dd46d 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -40,9 +40,6 @@ with the current dev tree before starting. - [#2985](/quest/m1/2985-js-net-path-keyed-publisher-state-goes-stale-when-a.md) - js/net: path-keyed publisher state goes stale when a broadcast is replaced - [#2991](/quest/m1/2991-net-coalesce-dynamic-tracks-and-preserve-sequences-across.md) - net: coalesce dynamic tracks and preserve sequences across replacements - [#2708](/quest/m1/2708-origin-open-the-announce-interest-lazily-on-first-consumer.md) - origin: open the announce interest lazily, on first consumer -- [#2756](/quest/m1/2756-origin-announce-only-entries-advertise-on-demand-content.md) - origin: announce-only entries, advertise on-demand content without instantiating broadcasts -- [Plan: generations](/quest/m1/plan-generations.md) - re-plan historical broadcast enumeration against the dev announce model via /plan-quest -- [#2873](/quest/m1/2873-moq-net-enumerate-and-resolve-historical-broadcast.md) - moq-net: enumerate and resolve historical broadcast generations by path and epoch - [Archive](/quest/m1/archive/README.md) - record selected tracks to any object_store and replay them over FETCH or derived HLS; gates the dev merge - [Playable](/quest/m1/hls-playable.md) - a 24/7 broadcast never becomes permanently unplayable over HLS - [#2848](/quest/m1/2848-follow-the-bandwidth-grant-in-moq-audio-instead-of.md) - Follow the bandwidth grant in moq-audio instead of holding a fixed reservation diff --git a/quest/m1/plan-generations.md b/quest/m1/plan-generations.md deleted file mode 100644 index 0b4c101ccdb..00000000000 --- a/quest/m1/plan-generations.md +++ /dev/null @@ -1,11 +0,0 @@ -# [S] Plan: historical broadcast generations - -## Goal - -A settled, per-PR split of [#2873](https://github.com/moq-dev/moq/issues/2873) -exists in the quest tree. Today the quest cannot land as one PR: its discovery contract is partly contradicted by dev's announce rework (the Ended flag was removed; recordings are discovered out of band). - -## Plan - -Run /plan-quest with the maintainer: re-plan the wire and API design against the current dev model and the broadcast epoch quest at /quest/m2/broadcast-epoch.md. The resulting quests replace -[the parent quest](/quest/m1/2873-moq-net-enumerate-and-resolve-historical-broadcast.md) and carry its Closes entries forward. diff --git a/quest/m2/README.md b/quest/m2/README.md index 63fdf31d956..ea43f8861a0 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -13,8 +13,8 @@ dashboards, fleet rollout) stay downstream. ## Quests -- [Broadcast epoch](/quest/m2/broadcast-epoch.md) - every broadcast carries a publisher-minted generation epoch, so restarts never splice -- [Generation](/quest/m2/hls-generation.md) - media URLs carry the resolved epoch so segment caching can be re-enabled +- [Plan: HLS identity](/quest/m2/plan-hls-identity.md) - settle where a cacheable generation ID comes from via /plan-quest +- [Generation](/quest/m2/hls-generation.md) - media URLs carry a generation ID so segment caching can be re-enabled - [Wildcard](/quest/m2/wildcard/README.md) - a service advertises a path pattern it could serve instead of enumerating broadcasts - [Path patterns](/quest/m2/path-patterns/README.md) - one versioned matcher for every predicate over broadcast paths: tokens, origins, interest - [Plan: keyframe request](/quest/m2/plan-keyframe-request.md) - settle the keyframe request wire design via /plan-quest diff --git a/quest/m2/broadcast-epoch.md b/quest/m2/broadcast-epoch.md deleted file mode 100644 index 1749fcb58c1..00000000000 --- a/quest/m2/broadcast-epoch.md +++ /dev/null @@ -1,47 +0,0 @@ -# [L] Broadcast epoch - -## Goal - -Every broadcast carries a generation epoch, so a restarted publisher is never -confused with its previous run. Consumers of that identity are separate quests; -this one is the identity itself. - -## Plan - -[#2610](https://github.com/moq-dev/moq/issues/2610) is an open proposal, not -landed work: no epoch field exists on the wire today, and this quest is what -changes that. It puts the identity on the WIRE rather than in the Hang catalog, -which is the whole reason this is its own quest: -`ANNOUNCE_START`/`ANNOUNCE_UPDATE` gain `Epoch` and `Ended`, `SUBSCRIBE`, `FETCH` -and `TRACK` gain `Epoch` (0 = current, mismatch = reset), and `TRACK_INFO` returns -the RESOLVED epoch so a consumer can key by generation without racing a -replacement. - -It replaces the first-hop route identity that stands in for content identity -today, which is why a restarting publisher with a stable origin id gets falsely -spliced. The resolution rules are the load-bearing part, because a consumer's -behaviour has to change with them: epoch is publisher-minted and strictly -increasing, forwarded unchanged, and **0 means unspecified**. Non-zero outranks 0, -higher wins over lower, equal non-zero epochs splice, and zero-vs-zero falls back -to today's first-entry identity. Receivers keep no high-water mark, which bounds -the damage of a bad epoch to its advertisement's lifetime. - -That 0 is why consumers cannot assume identity exists: a publisher that mints none -resolves to 0, and a consumer keying on it must keep its old behaviour rather than -treat 0 as a generation. - -`Ended` rides along and is in scope: it splits live from complete content (ended -broadcasts reject SUBSCRIBE and are read via FETCH), and is only announced to -streams whose `ANNOUNCE_REQUEST` opted in. Scope includes the IETF side, where -EPOCH is a negotiated moq-transport parameter, plus the bindings. - -That is a moq-lite-06 protocol change spanning the model, the wire codecs, and the -JS mirrors, so land it on its own terms rather than as a side effect of a -consumer's needs. Two consumers wait on it: cacheable HLS media URLs -([HLS generation](/quest/m2/hls-generation.md)), and recording keys (#2610 -proposes storing recordings at `/` instead of a UUID plus an -API lookup). - -## Closes - -- [#2610](https://github.com/moq-dev/moq/issues/2610) - close this issue when the quest finishes diff --git a/quest/m2/hls-generation.md b/quest/m2/hls-generation.md index 3a104ea158f..864eebd17a1 100644 --- a/quest/m2/hls-generation.md +++ b/quest/m2/hls-generation.md @@ -16,35 +16,37 @@ restarts at a new resolution decodes new segments against the old parameter sets; the segment window's `push` (`rs/moq-hls/src/export/segments.rs`) detects that restart already and resets only the playlist. -The identity itself is [Broadcast epoch](/quest/m2/broadcast-epoch.md)'s job, and it arrives from the -TRANSPORT rather than from the media: `TRACK_INFO` returns the resolved epoch. -Never substitute wall-clock or a relay-local counter, since a CDN needs one URL to -mean the same bytes on every edge. +The missing ingredient is the identity itself: nothing in the stack mints a +generation today. Where it comes from is +[Plan: HLS identity](/quest/m2/plan-hls-identity.md)'s job. Whatever it turns +out to be, it must be minted by the publisher and travel with the content; +never a wall-clock or a relay-local counter, since a CDN needs one URL to mean +the same bytes on every edge. So this quest is the consumption: the moq-hls playlist and MPD renderers EMIT -epoch-versioned `init.mp4` and segment URLs, and the cached init is invalidated -when the epoch changes. Both are CONDITIONAL on a non-zero resolved epoch: a -publisher that mints none resolves to 0, which cannot version anything, so -those broadcasts keep unversioned URLs rather than being cached against an -identity that does not exist. The cache-header half is moq.pro (downstream) -work: its edge parses the versioned paths and restores the long `max-age` -(where `immutable` is finally true) once the renderers emit them; versioning -the accepted paths without versioning the emitted ones changes nothing. - -**Open: the broadcast epoch does not cover a mid-broadcast rendition -reconfigure.** Only one of the two collision sources is a new generation: an -epoch is minted per path when the broadcast is announced, so a live publisher -that reconfigures a rendition (a resolution change, say) reuses the same -`init.mp4` URL and segment numbers for different bytes without the epoch moving. -Re-enabling caching on the epoch alone would let a CDN keep serving the previous -configuration. Resolve before caching is re-enabled, either by adding a -rendition-generation component to the URL or by requiring (and testing) an epoch -bump on every reconfigure. - -Acceptance: a publisher restart mid-playback exercised end to end, a zero-epoch -publisher still served with unversioned URLs, and a mid-broadcast reconfigure -that cannot be decoded against the previous configuration. +generation-versioned `init.mp4` and segment URLs, and the cached init is +invalidated when the generation changes. Both are CONDITIONAL on the identity +being present: a publisher that declares none cannot version anything, so those +broadcasts keep unversioned URLs rather than being cached against an identity +that does not exist. The cache-header half is moq.pro (downstream) work: its +edge parses the versioned paths and restores the long `max-age` (where +`immutable` is finally true) once the renderers emit them; versioning the +accepted paths without versioning the emitted ones changes nothing. + +There are two collision sources, and a per-broadcast identity only covers one +of them. A live publisher that reconfigures a rendition (a resolution change, +say) reuses the same `init.mp4` URL and segment numbers for different bytes +without restarting the broadcast, so caching on a restart-scoped generation +alone would let a CDN keep serving the previous configuration. Either the URL +gains a rendition-generation component or the identity is required (and tested) +to move on every reconfigure. Settle it in +[Plan: HLS identity](/quest/m2/plan-hls-identity.md); it is the reason that +plan exists rather than a detail of this one. + +Acceptance: a publisher restart mid-playback exercised end to end, a publisher +declaring no identity still served with unversioned URLs, and a mid-broadcast +reconfigure that cannot be decoded against the previous configuration. ## Required -- [Broadcast epoch](/quest/m2/broadcast-epoch.md) - a generation ID has to exist before a URL can carry it +- [Plan: HLS identity](/quest/m2/plan-hls-identity.md) - a generation ID has to exist before a URL can carry it diff --git a/quest/m2/plan-hls-identity.md b/quest/m2/plan-hls-identity.md new file mode 100644 index 00000000000..eb3d4da533f --- /dev/null +++ b/quest/m2/plan-hls-identity.md @@ -0,0 +1,25 @@ +# [S] Plan: a generation identity for cacheable media URLs + +## Goal + +A settled answer to where a generation ID comes from, so +[HLS generation](/quest/m2/hls-generation.md) can start. The broadcast epoch +was going to supply it; +[moq#3225](https://github.com/moq-dev/moq/pull/3225) removed Epoch from the +drafts as spec-only and never implemented, retired +`draft-lcurley-moq-broadcast`, and made announcements prefix routes with no +per-broadcast identity to hang a generation on. + +## Plan + +Run /plan-quest with the maintainer. The question is narrow: a CDN edge needs +one URL to mean the same bytes on every edge, so the ID has to be minted by +the publisher and travel with the content, not derived from wall-clock or a +relay-local counter. + +Candidate sources to settle between: a per-broadcast value returned on resolve +(`TRACK_INFO`), a publisher-declared field in the hang catalog, or something +the archive line already keys recordings by. Also settle the mid-broadcast +rendition reconfigure hole that +[HLS generation](/quest/m2/hls-generation.md) records, since a generation that +only moves on restart cannot version a live reconfigure either way.