Repository navigation
docs(quest): plan broadcast epochs #4004
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
49ed65a
21f66f8
cd527f2
0947b66
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| # Broadcast epochs | ||
|
|
||
| ## Goal | ||
|
|
||
| Reusing a broadcast name is never ambiguous. By default, each publish of | ||
| `demo/BBB.hang` goes out as `demo/BBB.hang/@<uuidv7>`, so the relay and viewers | ||
| can tell a new broadcast from a continuation. A viewer of the name follows | ||
| the newest live epoch as soon as it is announced, instead of waiting ~30s for | ||
| the old route's QUIC idle timeout. Moving to a new epoch is a clean boundary | ||
| (a fresh catalog and tracks, never spliced), and each run stays addressable | ||
| by its full path. | ||
|
|
||
| The epoch rides in the path, so it survives any moq-transport relay, and no | ||
| wire message changes. At an epoch-aware relay, a request for a bare name | ||
| resolves to its newest live epoch on every protocol version. | ||
|
|
||
| Non-goals: redundant publishers sharing one epoch, and failing over between | ||
| them faster than the keep-alive (the [redundant ingest](/quest/m2/redundant-ingest.md) | ||
| study). Also out of scope: trusting the publisher's clock (a far-future epoch | ||
| wins until its route goes away). | ||
|
|
||
| ## Plan | ||
|
|
||
| Decided: | ||
|
|
||
| - The marker is a child segment `@<uuidv7>`, parsed into | ||
| `Option<Epoch>` by [the primitive](/quest/m1/epoch.md). Not a lite-07 flag: | ||
| the path is the only carrier. | ||
| - The broadcast-publish path mints an epoch unless the path already carries | ||
| one. A caller who passes an explicit epoch, such as a redundant publisher, | ||
| keeps it. The prefix-route `announce(prefix, route)` stays raw, and that is | ||
| the opt-out. So prefix vs broadcast is an API distinction, not a wire flag. | ||
| - Viewers follow the greatest epoch with a live route. When it goes away and an | ||
| older one is still live, they fall back to it. | ||
| - A bare request with no route of its own resolves to that same epoch on every | ||
| version, so lite-06 and IETF clients keep working through an epoch-aware | ||
| relay. When a newer epoch appears, the bare subscription ends with a typed | ||
| reset, and the client's normal resubscribe lands on the new one. | ||
| - An unmodified third-party relay routes `foo/@<epoch>` but never resolves a | ||
| bare `foo`, since a route covers its descendants, not its parent. A | ||
| bare-name viewer behind one needs a publisher that opts out with the raw | ||
| prefix route. Document this rather than promise it works. | ||
| - Derived output lives under the epoch it came from | ||
| (`pid/foo.hang/@e/transcode.pro`), so nested epochs must parse. This moves | ||
| the [wildcard](/quest/m1/wildcard/README.md) line's derived-output example | ||
| down one segment, and its suffix patterns still match. | ||
|
|
||
| This README owns: | ||
|
|
||
| - An end-to-end relay test: republish a name while the old publisher's session | ||
| stays open. A new-API viewer and a lite-06 or IETF bare-path viewer both | ||
| reach the new epoch within one RTT-scale bound rather than the idle timeout. | ||
| Killing the newest epoch falls back to a still-live older one. | ||
| - A `doc/concept` page on broadcast naming: what an epoch is, publish and | ||
| consume behavior, takeover and fallback, bare-path resolution, and the | ||
| prefix-route opt-out. | ||
|
|
||
| ## Quests | ||
|
|
||
| - [Origin](/quest/m1/broadcast-epoch/origin.md) - moq-net publish mints an epoch, consumers follow the newest live one, and bare requests resolve to it on every version | ||
| - [Apps](/quest/m1/broadcast-epoch/apps.md) - moq-cli, the browser publish and watch components, and demo/web publish under epochs and play bare names | ||
| - [Gateways](/quest/m1/broadcast-epoch/gateways.md) - RTMP, SRT, and WHIP ingest mint an epoch per incoming connection, so an encoder reconnect is a clean takeover | ||
| - [Bindings](/quest/m1/broadcast-epoch/bindings.md) - moq-ffi, libmoq, and every wrapper expose the epoch and inherit the default | ||
| - [GStreamer and OBS](/quest/m1/broadcast-epoch/gst-obs.md) - moqsink and the OBS plugin publish each run under a fresh epoch | ||
|
|
||
| ## Required | ||
|
|
||
| - [Epoch primitive](/quest/m1/epoch.md) - the shared `Epoch` type and path split | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| # [M] Apps publish under epochs and play bare names | ||
|
|
||
| ## Goal | ||
|
|
||
| `moq-cli` publish and play, `@moq/publish`, `@moq/watch`, and `demo/web` use | ||
| the origin default. Each publish run is a new epoch, and watching a bare name | ||
| switches to a republish within an RTT rather than after the idle timeout. | ||
| The UI and logs show the full epoch path, and a watch link can pin one. | ||
|
|
||
| ## Plan | ||
|
|
||
| - Publish sides need little beyond passing bare names. Check that nothing | ||
| caches the announced path across a restart. | ||
| - Watch sides handle "the broadcast changed" as a fresh catalog and decoder | ||
| reset. Test a republish mid-playback in the browser and native players. | ||
| - Update `doc/bin/cli.md` and every example invocation that shows a published | ||
| path. | ||
|
|
||
| ## Required | ||
|
|
||
| - [Origin](/quest/m1/broadcast-epoch/origin.md) - the publish default and follow logic |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| # [M] Bindings expose epochs | ||
|
|
||
| ## Goal | ||
|
|
||
| `moq-ffi`, `libmoq`, and the py, swift, kt, go, and dart wrappers publish under | ||
| the default epoch and follow bare names like Rust. The epoch of a published or | ||
| consumed broadcast is readable, and a caller can pass an explicit one. The | ||
| reconnect counter `session.epoch()` is renamed so "epoch" has one meaning. | ||
|
|
||
| ## Plan | ||
|
|
||
| - Expose the parsed epoch (text and time) and an explicit-epoch publish | ||
| argument. Keep the surface to what a binding consumer needs. | ||
| - Rename `session.epoch()` in every binding (for example to `connects()`). | ||
| That is a break, so it lands on `dev`. | ||
| - Update `doc/lib/{py,swift,kt,go,dart,c}` per the cross-package sync table, | ||
| and run `just test smoke --all`. | ||
|
|
||
| ## Required | ||
|
|
||
| - [Origin](/quest/m1/broadcast-epoch/origin.md) - the behavior the bindings surface |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| # [S] Ingest gateways mint an epoch per connection | ||
|
|
||
| ## Goal | ||
|
|
||
| RTMP, SRT, and WHIP ingest publish each incoming connection under a fresh | ||
| epoch. An encoder that reconnects to the same stream key becomes a clean | ||
| takeover at once, instead of a second publisher racing the stale one until it | ||
| times out. | ||
|
|
||
| ## Plan | ||
|
|
||
| Use the origin default where the gateways create broadcasts (`moq-rtmp`, | ||
| `moq-srt`, `moq-rtc`, and the relay wiring). Test a reconnect under the same | ||
| key with the stale connection still open. Update `doc/bin/relay/` where it | ||
| describes ingest paths. | ||
|
|
||
| ## Required | ||
|
|
||
| - [Origin](/quest/m1/broadcast-epoch/origin.md) - the publish default |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| # [S] GStreamer and OBS publish under epochs | ||
|
|
||
| ## Goal | ||
|
|
||
| `moqsink` and the OBS plugin publish each run under a fresh epoch, so a | ||
| restarted pipeline or a stop and start in OBS is a clean takeover for viewers. | ||
|
|
||
| ## Plan | ||
|
|
||
| `moqsink` takes the origin default per session. When | ||
| [#3115](/quest/m2/3115-moqsink-the-publication-has-no-generation-so-a-flush.md) | ||
| lands, each of its publication generations is a new epoch. OBS gets it | ||
| through libmoq. Update `doc/bin/gstreamer.md` and `doc/bin/obs.md` if they | ||
| show paths. | ||
|
|
||
| ## Required | ||
|
|
||
| - [Bindings](/quest/m1/broadcast-epoch/bindings.md) - OBS publishes through libmoq |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| # [L] Origin: mint, follow, and resolve epochs | ||
|
|
||
| ## Goal | ||
|
|
||
| In `moq-net` (Rust and JS), publishing a broadcast at a bare path announces it | ||
| under a fresh epoch. A consumer of a bare name follows the newest epoch with | ||
| a live route and falls back to an older live one. A bare request resolves the | ||
| same way on every protocol version, so the relay serves old clients without | ||
| any wire change. | ||
|
|
||
| ## Plan | ||
|
|
||
| - Publish: the broadcast create-and-announce path appends `Epoch::mint()` | ||
| unless the path already has an epoch. `announce(prefix, route)` stays raw. | ||
|
Comment on lines
+13
to
+14
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
For a currently valid bare path containing the IETF maximum of 32 parts, appending this child produces 33 parts. Both Rust and JS encoders reject paths beyond 32, so the new default would make such broadcasts fail before reaching the wire. The plan should reserve capacity with an explicit compatibility decision or choose an encoding that does not add a part, and include this break in the stated API impact. AGENTS.md reference: AGENTS.md:L58-L60 Useful? React with 👍 / 👎.
Comment on lines
+13
to
+14
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There is no public Useful? React with 👍 / 👎. |
||
| A bare path already at the 32-part limit has no room for the epoch segment: | ||
| refuse it at publish, pointing at the raw route, rather than fail at encode. | ||
| - Auth: a grant that admits bare `foo` admits `foo/@<epoch>` for both publish | ||
| and subscribe, and a grant on one epoch never widens to the bare name or its | ||
| siblings. Check how exact grants and patterns match today, and test both | ||
| sides in moq-relay's auth tests. | ||
| - Consume: `routed` and `request_broadcast` on a bare name watch the routes | ||
| one `@` segment below it. They pick the greatest live epoch, and on a table | ||
|
Comment on lines
+13
to
+22
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Appending and resolving the child path crosses the existing authorization boundary: a literal grant such as AGENTS.md reference: AGENTS.md:L58-L60 Useful? React with 👍 / 👎. |
||
| change, re-select: move up at once, or fall back when the current one is | ||
| retracted. A path that names an epoch pins it and never moves. Each move is | ||
| a new broadcast to the caller, never a splice. The #3312 first-hop resume | ||
| rule still applies within one epoch path. | ||
| - Bare resolution: a bare request with no route of its own resolves to that | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When a derived output is served by the existing Useful? React with 👍 / 👎. |
||
| epoch. A takeover ends the bare subscription with a typed reset, never a | ||
| silent switch. Choose the code so existing clients resubscribe rather than | ||
| give up. | ||
| - A nested epoch (`name/@e/derived/@f`) resolves per level. Split-horizon | ||
| exclusion and the per-path fronts from #3312 stay intact. | ||
| - Benchmark resolution swept over epochs per name and names per origin, so | ||
| following does not scan the table. | ||
| - Update `doc/concept` and `drafts/draft-lcurley-moq-lite.md` wherever they | ||
| describe resolution or takeover. The rule is a relay behavior, so state it | ||
| in the draft even though no field changes. | ||
|
|
||
| Public API: behavior change on publish (the announced path gains an epoch, | ||
| and a 32-part bare path is refused), on bare-name consume, and on what an | ||
| exact grant admits. Decide at PR time whether that retargets to `dev`. | ||
| Wire: none. | ||
|
|
||
| ## Related | ||
|
|
||
| - [#2991](/quest/m1/2991-net-coalesce-dynamic-tracks-and-preserve-sequences-across.md) - a new epoch starts each track at sequence 0 | ||
| - [Broadcast route](/quest/m1/js-broadcast-route.md) - the JS claim matching the follow logic sits on | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| # [M] Broadcast epoch primitive | ||
|
|
||
| ## Goal | ||
|
|
||
| `moq-net` and `@moq/net` own one `Epoch` type: a publisher instance's | ||
| identity, carried as the last path segment `@<uuidv7>` (for example | ||
| `demo/BBB.hang/@0192a3f0-...`). It mints, parses, orders (newest greatest), | ||
| and reports its wall-clock time. A path splits into its epoch-free name and | ||
| an optional epoch. The epoch rides in the path, so it survives any | ||
| moq-transport relay with no wire change. | ||
|
|
||
| This is the shared primitive the [broadcast epochs](/quest/m1/broadcast-epoch/README.md) | ||
| line builds on, and it replaces the e2ee-local `moq_e2ee::Epoch`. | ||
|
|
||
| ## Plan | ||
|
|
||
| - Move `rs/moq-e2ee/src/epoch.rs` into `moq-net` and add the JS equivalent. | ||
| Parsing is strict: `@` followed by a lowercase hyphenated UUIDv7. Nothing else | ||
| counts as an epoch, so an app's own UUID segments never parse as one. | ||
|
Comment on lines
+17
to
+19
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The new strict-parser wording addresses AGENTS.md reference: AGENTS.md:L58-L60 Useful? React with 👍 / 👎. |
||
| - Path helpers split a path into name and epoch and join them back. A segment | ||
| like `@alice` is valid today and stays valid: strict parsing already keeps it | ||
| from reading as an epoch. Rejecting it instead would break the path contract | ||
| and land on `dev`. Check how the split interacts with | ||
| [path patterns](/quest/m1/path-patterns.md) and | ||
| [hidden broadcasts](/quest/m1/hidden-broadcasts.md) (a leading `.`). | ||
| - `moq-e2ee` uses the shared type. Update | ||
| [draft-lcurley-moq-e2ee](/drafts/draft-lcurley-moq-e2ee.md) so the path is | ||
| `<opaque>/@<epoch>`. Keep the HKDF input as the UUID text, so vectors change | ||
| only if they encode the path. Drop "deliberately e2ee-only" from the e2ee | ||
| README. | ||
| - Golden cross-language vectors for parse, reject, order, and time, beside | ||
| the existing path tests. | ||
| - Nothing mints by default here. Publish and consume behavior is | ||
| [origin](/quest/m1/broadcast-epoch/origin.md). | ||
|
|
||
| Public API: additive `Epoch` and path helpers in both languages. The | ||
| `moq_e2ee::Epoch` move is a break on an unpublished crate. Wire: none. | ||
|
|
||
| ## Related | ||
|
|
||
| - [E2EE](/quest/m1/e2ee/README.md) - the first consumer, whose TypeScript core requires this | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
quest readydoes not propagate a questline README'sRequiredentries to its children: it traverses the README'sQuests, then evaluates each plain quest's ownRequiredsection. Consequentlyorigin.md, which directly needsEpoch::mint(), is advertised as ready before the epoch primitive exists. Put this dependency on the Origin quest so the documented execution order is enforceable.Useful? React with 👍 / 👎.