From 49ed65a43dae78e602f29b41f889f97ff6450999 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 23 Sep 2026 22:23:32 +0000 Subject: [PATCH 1/4] docs(quest): plan broadcast epochs Reusing a broadcast name is ambiguous to the relay. Plan a trailing `@` epoch segment: a shared moq-net primitive (m1, required by e2ee), a m2 questline that mints on publish, follows the newest live epoch, and resolves bare names on every version, plus a redundant-ingest study and the #2985 js/net fix. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_016FYDouB6hmhgDSYKxgvygb --- ...keyed-publisher-state-goes-stale-when-a.md | 28 ++++++++ quest/m1/README.md | 2 + quest/m1/e2ee/README.md | 5 +- quest/m1/e2ee/typescript.md | 6 +- quest/m1/epoch.md | 40 ++++++++++++ quest/m1/wildcard/README.md | 2 + quest/m2/README.md | 2 + quest/m2/epoch/README.md | 65 +++++++++++++++++++ quest/m2/epoch/apps.md | 21 ++++++ quest/m2/epoch/bindings.md | 21 ++++++ quest/m2/epoch/gateways.md | 19 ++++++ quest/m2/epoch/gst-obs.md | 18 +++++ quest/m2/epoch/origin.md | 40 ++++++++++++ quest/m2/redundant-ingest.md | 23 +++++++ 14 files changed, 289 insertions(+), 3 deletions(-) create mode 100644 quest/m1/2985-js-net-path-keyed-publisher-state-goes-stale-when-a.md create mode 100644 quest/m1/epoch.md create mode 100644 quest/m2/epoch/README.md create mode 100644 quest/m2/epoch/apps.md create mode 100644 quest/m2/epoch/bindings.md create mode 100644 quest/m2/epoch/gateways.md create mode 100644 quest/m2/epoch/gst-obs.md create mode 100644 quest/m2/epoch/origin.md create mode 100644 quest/m2/redundant-ingest.md diff --git a/quest/m1/2985-js-net-path-keyed-publisher-state-goes-stale-when-a.md b/quest/m1/2985-js-net-path-keyed-publisher-state-goes-stale-when-a.md new file mode 100644 index 00000000000..8e9c5cf049a --- /dev/null +++ b/quest/m1/2985-js-net-path-keyed-publisher-state-goes-stale-when-a.md @@ -0,0 +1,28 @@ +# [S] js/net: path-keyed publisher state follows a same-path replacement + +## Goal + +Replacing a path's producer in `js/net` without closing the predecessor +behaves like a new broadcast. Discovery announces the replacement, `TRACK_INFO` +answers with the successor's metadata over both SUBSCRIBE and FETCH, and an +IETF `PUBLISH_NAMESPACE` refusal against the predecessor no longer suppresses +the successor. + +Epochs give each default publish a distinct path, but raw paths (the +prefix-route opt-out) still reuse names, so this still needs its own fix. + +## Plan + +Follow the local fix the issue proposes: reconcile announce state by producer +identity rather than by key, evict the path's `#trackInfo` entries on +`publish`, and key `refused` by producer identity. The issue lists the +regression tests. Check whether Rust has the same three gaps and fix any it +does in the same PR. Public API: none. Wire: none. + +## Closes + +- [#2985](https://github.com/moq-dev/moq/issues/2985) - close this issue when the quest finishes + +## Related + +- [#2991](/quest/m1/2991-net-coalesce-dynamic-tracks-and-preserve-sequences-across.md) - the dynamic-track half of the same replacement identity problem diff --git a/quest/m1/README.md b/quest/m1/README.md index 48cbd7cb52a..ab8cfb0eabe 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -98,7 +98,9 @@ transport, benchmark tooling); worktrees isolate commits, not semantics. - [JS abandonment](/quest/m1/js-subscribe-abandonment.md) - a viewer returning during IETF subscribe setup keeps its track across microtasks - [IETF stream types](/quest/m1/ietf-uni-stream-types.md) - padding streams are discarded stream-only and an unknown uni type closes the session, per draft-21 - [#2991](/quest/m1/2991-net-coalesce-dynamic-tracks-and-preserve-sequences-across.md) - one dynamic producer per track name in both languages, with the sequence namespace surviving a replacement +- [#2985](/quest/m1/2985-js-net-path-keyed-publisher-state-goes-stale-when-a.md) - js/net: replacing a path's producer re-announces it and invalidates the predecessor's TRACK_INFO and refusal state - [JS track end](/quest/m1/2318-js-net-remaining-capability-gaps-vs-rs-moq-net-setup-role.md) - js/net declares a track end ahead of the live edge and observes the publisher's SUBSCRIBE_END +- [Epoch primitive](/quest/m1/epoch.md) - one `Epoch` type in moq-net and @moq/net, carried as a trailing `@` path segment, shared by e2ee and broadcast epochs - [E2EE](/quest/m1/e2ee/README.md) - TypeScript and Rust peers interoperate over encrypted broadcasts no relay can decrypt - [Processor](/quest/m1/processor/README.md) - a customer-run worker publishes an on-demand contribution with scoped access - [#3056](/quest/m1/3056-watch-video-decoder-captures-the-rewind-generation-at.md) - watch: the video decoder resets on a declared discontinuity diff --git a/quest/m1/e2ee/README.md b/quest/m1/e2ee/README.md index aed70ba768f..0160e8535ee 100644 --- a/quest/m1/e2ee/README.md +++ b/quest/m1/e2ee/README.md @@ -19,8 +19,8 @@ The contract is [draft-lcurley-moq-e2ee](/drafts/draft-lcurley-moq-e2ee.md), pro ### Epoch and identity -- Every publisher instance mints an epoch, a UUIDv7 in lowercase text, and publishes at `/`, where `` derives from the credential and the semantic broadcast name without the epoch. The epoch is an input to every HKDF derivation, so a restart, takeover, or explicit group sequence cannot repeat a nonce under a key: nothing is persisted across instances and no generation counter is redistributed. Subscribers discover instances under the `/` prefix and take the greatest epoch, which sorts newest; a known full path carries its epoch in the last segment. -- The epoch is untrusted and unauthenticated. A wrong epoch fails authentication and a withheld one denies service; neither can make a nonce repeat, because only the publisher instance chooses what it encrypts under. This is the same trust a cache needs to serve the right instance, and it is deliberately e2ee-only: plaintext hang keeps its current paths. +- Every publisher instance mints an epoch, a UUIDv7 in lowercase text, and publishes at `/@`, where `` derives from the credential and the semantic broadcast name without the epoch. The epoch is an input to every HKDF derivation, so a restart, takeover, or explicit group sequence cannot repeat a nonce under a key: nothing is persisted across instances and no generation counter is redistributed. The epoch is the shared [`Epoch`](/quest/m1/epoch.md) primitive. Subscribers discover instances under the `/` prefix and take the greatest epoch, which sorts newest; a known full path carries its epoch in the last segment. +- The epoch is untrusted and unauthenticated. A wrong epoch fails authentication and a withheld one denies service; neither can make a nonce repeat, because only the publisher instance chooses what it encrypts under. This is the same trust a cache needs to serve the right instance. Plaintext broadcasts adopt the same epoch by default through [broadcast epochs](/quest/m2/epoch/README.md). - Nothing in the path says the bytes are encrypted. The format after decryption (`meeting.hang`) is inside the opaque name; a plaintext player, exporter, or matcher that opens a protected broadcast finds no catalog it can read and fails with its usual typed refusal, the same as for any format it does not support. - One 32-byte secret authorizes the whole broadcast; HKDF-SHA-256 derives separate AES-128-GCM keys for each physical track and for grouped-frame versus datagram domains. A grouped frame uses the 96-bit nonce `uint64_be(group) || uint32_be(frame)`; a datagram uses its sequence with frame zero under the datagram domain. Empty AAD: every immutable end-to-end field is in the HKDF info or the nonce. - Within an instance, group and datagram sequences are allocated monotonically and frames are numbered in write order, which is the whole reuse rule. There is no ciphertext retention or retransmission API: relays forward bytes unchanged, and an application that needs to resend encrypts nothing twice because it never gets the same identity twice. @@ -67,4 +67,5 @@ The Rust and TypeScript cores expose the same surface, and nothing else: ## Related +- [Epoch primitive](/quest/m1/epoch.md) - the shared `Epoch` type the protected path ends in - [archive](/quest/m1/archive/README.md) - protected broadcasts are deliberately outside recording and replay formats diff --git a/quest/m1/e2ee/typescript.md b/quest/m1/e2ee/typescript.md index 8a52351e62f..30c5cdc18a2 100644 --- a/quest/m1/e2ee/typescript.md +++ b/quest/m1/e2ee/typescript.md @@ -8,9 +8,13 @@ the Rust crate name for name, without putting keys or crypto policy in `@moq/net ## Plan -- Add the package above `@moq/net`. `Credential` accepts a 32-byte secret or a nonextractable HKDF `CryptoKey` with `deriveBits` usage, rejects extractable keys, and never serializes the secret. `credential.path(semantic)` returns the opaque broadcast name and `credential.generation(epoch)` returns a `Generation` with `name()`, `produce()`, `consume()`, and a `mint()` that returns a lowercase UUIDv7 from `Date.now()` plus `crypto.getRandomValues`. +- Add the package above `@moq/net`. `Credential` accepts a 32-byte secret or a nonextractable HKDF `CryptoKey` with `deriveBits` usage, rejects extractable keys, and never serializes the secret. `credential.path(semantic)` returns the opaque broadcast name and `credential.generation(epoch)` returns a `Generation` with `name()`, `produce()`, `consume()`, and epochs come from the shared `@moq/net` `Epoch`. - Mirror the Rust modules: `Track.Producer`, `Track.Consumer`, `Group.Producer`, `Group.Consumer`, `Datagram.Event`, and a `Failure` whose `code` is the draft's typed set. No catalog helpers, no exported constants beyond the profile limits an application sizes payloads with, no stateless `protect`/`open` with caller-chosen identities, and no test-only methods on public classes. - WebCrypto AES-GCM is async, so each producer and consumer runs a serial pump: one AEAD call in flight, completions in submit order, a bounded waiter queue that throws when full. Depth one keeps frame ordinals equal to the inner `@moq/net` write order without reservation sets; raise it only when a measured capture burst needs it. Propagate backpressure, cancellation, and authentication failure explicitly; never reorder media or fall back to plaintext. - Identity state is two counters per key (invocations, plaintext bytes) and one monotonic sequence per track. Check payload size before taking a frame ordinal so a rejected write never burns an identity. Datagram receivers keep a 1024-bit sliding bitmask marked only after a successful open; failed opens count against the key; a grouped authentication failure closes the ordered track. - Pass every vector in `drafts/moq-e2ee-00.json` against [draft-lcurley-moq-e2ee](/drafts/draft-lcurley-moq-e2ee.md), then cover pump ordering, saturation, and cancellation, protected payload ceilings, monotonic allocation, per-key exhaustion, bounded datagram suppression, bad-group termination, bad-datagram events, and a new epoch authenticating while the old keys do not. Exercise grouped tracks on both transports and datagrams on moq-lite via `insertDatagram`. - Add the package to `doc/lib/js/index.md` and the workspace; leave it unpublished until the browser components consume it. + +## Required + +- [Epoch primitive](/quest/m1/epoch.md) - the shared `Epoch` this package mints and binds diff --git a/quest/m1/epoch.md b/quest/m1/epoch.md new file mode 100644 index 00000000000..6a0c2dc943c --- /dev/null +++ b/quest/m1/epoch.md @@ -0,0 +1,40 @@ +# [M] Broadcast epoch primitive + +## Goal + +`moq-net` and `@moq/net` own one `Epoch` type: a publisher instance's +identity, carried as the last path segment `@` (for example +`demo/BBB.hang/@0192a3f0-...`). It mints, parses, orders (newest greatest), +and reports its wall-clock time. A path splits into its epoch-free name and +an optional epoch. The epoch rides in the path, so it survives any +moq-transport relay with no wire change. + +This is the shared primitive the [broadcast epochs](/quest/m2/epoch/README.md) +line builds on, and it replaces the e2ee-local `moq_e2ee::Epoch`. + +## Plan + +- Move `rs/moq-e2ee/src/epoch.rs` into `moq-net` and add the JS equivalent. + Parsing is strict: `@` followed by a lowercase hyphenated UUIDv7. Nothing else + counts as an epoch, so an app's own UUID segments never parse as one. +- Path helpers split a path into name and epoch and join them back. New path + construction rejects a segment that starts with `@` but is not a valid epoch, + so the marker stays unambiguous. Check how that interacts with + [path patterns](/quest/m1/path-patterns.md) and + [hidden broadcasts](/quest/m1/hidden-broadcasts.md) (a leading `.`). +- `moq-e2ee` uses the shared type. Update + [draft-lcurley-moq-e2ee](/drafts/draft-lcurley-moq-e2ee.md) so the path is + `/@`. Keep the HKDF input as the UUID text, so vectors change + only if they encode the path. Drop "deliberately e2ee-only" from the e2ee + README. +- Golden cross-language vectors for parse, reject, order, and time, beside + the existing path tests. +- Nothing mints by default here. Publish and consume behavior is + [origin](/quest/m2/epoch/origin.md). + +Public API: additive `Epoch` and path helpers in both languages. The +`moq_e2ee::Epoch` move is a break on an unpublished crate. Wire: none. + +## Related + +- [E2EE](/quest/m1/e2ee/README.md) - the first consumer, whose TypeScript core requires this diff --git a/quest/m1/wildcard/README.md b/quest/m1/wildcard/README.md index 03087f87cc1..c4e81fd0528 100644 --- a/quest/m1/wildcard/README.md +++ b/quest/m1/wildcard/README.md @@ -271,3 +271,5 @@ than announce state. pattern, and its catalog names the generations a wildcard cannot - [pop-skipping](/quest/m1/pop-skipping/README.md) - it owns the route cost and the rank hash this reuses +- [Broadcast epochs](/quest/m2/epoch/README.md) - derived output moves under the + source's `@` segment, which the suffix patterns still match diff --git a/quest/m2/README.md b/quest/m2/README.md index c93004a83a2..0bce2c1e0e3 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -42,7 +42,9 @@ upstream release waits in [m4](/quest/m4/README.md). - [vcpkg registry](/quest/m2/cpp-vcpkg.md) - a registry we own serves the prebuilt package to `vcpkg` manifests - [Conan remote](/quest/m2/cpp-conan.md) - a remote we own serves the same tarball to `conan install` - [Compressed tracks](/quest/m2/flate/README.md) - any track compresses per group from every language, not only the JSON modes +- [Broadcast epochs](/quest/m2/epoch/README.md) - each publish of a name gets a fresh `@` epoch, viewers follow the newest live one at once, and bare names still resolve on every version - [#3115](/quest/m2/3115-moqsink-the-publication-has-no-generation-so-a-flush.md) - moqsink: a flushing restart after EOS opens a new publication generation +- [Redundant ingest](/quest/m2/redundant-ingest.md) - decide whether two publishers sharing one epoch may splice, and who declares the incumbent dead before the keep-alive does - [Multipath spike](/quest/m2/multipath-spike.md) - whether bonded contribution over multipath QUIC is worth building, given it needs noq on both ends - [Receive timestamps](/quest/m2/quic-receive-ts.md) - per-packet arrival times in ACKs, the feedback GCC and deadlines need - [QUIC GCC](/quest/m2/quic-gcc.md) - a measured verdict on delay-based congestion control for media egress, shipping as `RealTime` diff --git a/quest/m2/epoch/README.md b/quest/m2/epoch/README.md new file mode 100644 index 00000000000..fde44400651 --- /dev/null +++ b/quest/m2/epoch/README.md @@ -0,0 +1,65 @@ +# Broadcast epochs + +## Goal + +Reusing a broadcast name is never ambiguous. By default, each publish of +`demo/BBB.hang` goes out as `demo/BBB.hang/@`, so the relay and viewers +can tell a new broadcast from a continuation. A viewer of the name follows +the newest live epoch as soon as it is announced, instead of waiting ~30s for +the old route's QUIC idle timeout. Moving to a new epoch is a clean boundary +(a fresh catalog and tracks, never spliced), and each run stays addressable +by its full path. + +The epoch rides in the path, so it survives any moq-transport relay, and no +wire message changes. A publisher that cannot or will not mint one is still +served: a request for a bare name resolves to its newest live epoch on every +protocol version. + +Non-goals: redundant publishers sharing one epoch, and failing over between +them faster than the keep-alive (the [redundant ingest](/quest/m2/redundant-ingest.md) +study). Also out of scope: trusting the publisher's clock (a far-future epoch +wins until its route goes away). + +## Plan + +Decided: + +- The marker is a child segment `@`, parsed into + `Option` by [the primitive](/quest/m1/epoch.md). Not a lite-07 flag: + the path is the only carrier. +- The broadcast-publish path mints an epoch unless the path already carries + one. A caller who passes an explicit epoch, such as a redundant publisher, + keeps it. The prefix-route `announce(prefix, route)` stays raw, and that is + the opt-out. So prefix vs broadcast is an API distinction, not a wire flag. +- Viewers follow the greatest epoch with a live route. When it goes away and an + older one is still live, they fall back to it. +- A bare request with no route of its own resolves to that same epoch on every + version, so lite-06 and IETF clients and third-party relays keep working. + When a newer epoch appears, the bare subscription ends with a typed reset, + and the client's normal resubscribe lands on the new one. +- Derived output lives under the epoch it came from + (`pid/foo.hang/@e/transcode.pro`), so nested epochs must parse. This moves + the [wildcard](/quest/m1/wildcard/README.md) line's derived-output example + down one segment, and its suffix patterns still match. + +This README owns: + +- An end-to-end relay test: republish a name while the old publisher's session + stays open. A new-API viewer and a lite-06 or IETF bare-path viewer both + reach the new epoch within one RTT-scale bound rather than the idle timeout. + Killing the newest epoch falls back to a still-live older one. +- A `doc/concept` page on broadcast naming: what an epoch is, publish and + consume behavior, takeover and fallback, bare-path resolution, and the + prefix-route opt-out. + +## Quests + +- [Origin](/quest/m2/epoch/origin.md) - moq-net publish mints an epoch, consumers follow the newest live one, and bare requests resolve to it on every version +- [Apps](/quest/m2/epoch/apps.md) - moq-cli, the browser publish and watch components, and demo/web publish under epochs and play bare names +- [Gateways](/quest/m2/epoch/gateways.md) - RTMP, SRT, and WHIP ingest mint an epoch per incoming connection, so an encoder reconnect is a clean takeover +- [Bindings](/quest/m2/epoch/bindings.md) - moq-ffi, libmoq, and every wrapper expose the epoch and inherit the default +- [GStreamer and OBS](/quest/m2/epoch/gst-obs.md) - moqsink and the OBS plugin publish each run under a fresh epoch + +## Required + +- [Epoch primitive](/quest/m1/epoch.md) - the shared `Epoch` type and path split diff --git a/quest/m2/epoch/apps.md b/quest/m2/epoch/apps.md new file mode 100644 index 00000000000..fb6472b4add --- /dev/null +++ b/quest/m2/epoch/apps.md @@ -0,0 +1,21 @@ +# [M] Apps publish under epochs and play bare names + +## Goal + +`moq-cli` publish and play, `@moq/publish`, `@moq/watch`, and `demo/web` use +the origin default. Each publish run is a new epoch, and watching a bare name +switches to a republish within an RTT rather than after the idle timeout. +The UI and logs show the full epoch path, and a watch link can pin one. + +## Plan + +- Publish sides need little beyond passing bare names. Check that nothing + caches the announced path across a restart. +- Watch sides handle "the broadcast changed" as a fresh catalog and decoder + reset. Test a republish mid-playback in the browser and native players. +- Update `doc/bin/cli.md` and every example invocation that shows a published + path. + +## Required + +- [Origin](/quest/m2/epoch/origin.md) - the publish default and follow logic diff --git a/quest/m2/epoch/bindings.md b/quest/m2/epoch/bindings.md new file mode 100644 index 00000000000..262b6b81374 --- /dev/null +++ b/quest/m2/epoch/bindings.md @@ -0,0 +1,21 @@ +# [M] Bindings expose epochs + +## Goal + +`moq-ffi`, `libmoq`, and the py, swift, kt, go, and dart wrappers publish under +the default epoch and follow bare names like Rust. The epoch of a published or +consumed broadcast is readable, and a caller can pass an explicit one. The +reconnect counter `session.epoch()` is renamed so "epoch" has one meaning. + +## Plan + +- Expose the parsed epoch (text and time) and an explicit-epoch publish + argument. Keep the surface to what a binding consumer needs. +- Rename `session.epoch()` in every binding (for example to `connects()`). + That is a break, so it lands on `dev`. +- Update `doc/lib/{py,swift,kt,go,dart,c}` per the cross-package sync table, + and run `just test smoke --all`. + +## Required + +- [Origin](/quest/m2/epoch/origin.md) - the behavior the bindings surface diff --git a/quest/m2/epoch/gateways.md b/quest/m2/epoch/gateways.md new file mode 100644 index 00000000000..e7ef109a503 --- /dev/null +++ b/quest/m2/epoch/gateways.md @@ -0,0 +1,19 @@ +# [S] Ingest gateways mint an epoch per connection + +## Goal + +RTMP, SRT, and WHIP ingest publish each incoming connection under a fresh +epoch. An encoder that reconnects to the same stream key becomes a clean +takeover at once, instead of a second publisher racing the stale one until it +times out. + +## Plan + +Use the origin default where the gateways create broadcasts (`moq-rtmp`, +`moq-srt`, `moq-rtc`, and the relay wiring). Test a reconnect under the same +key with the stale connection still open. Update `doc/bin/relay/` where it +describes ingest paths. + +## Required + +- [Origin](/quest/m2/epoch/origin.md) - the publish default diff --git a/quest/m2/epoch/gst-obs.md b/quest/m2/epoch/gst-obs.md new file mode 100644 index 00000000000..57626b62224 --- /dev/null +++ b/quest/m2/epoch/gst-obs.md @@ -0,0 +1,18 @@ +# [S] GStreamer and OBS publish under epochs + +## Goal + +`moqsink` and the OBS plugin publish each run under a fresh epoch, so a +restarted pipeline or a stop and start in OBS is a clean takeover for viewers. + +## Plan + +`moqsink` takes the origin default per session. When +[#3115](/quest/m2/3115-moqsink-the-publication-has-no-generation-so-a-flush.md) +lands, each of its publication generations is a new epoch. OBS gets it +through libmoq. Update `doc/bin/gstreamer.md` and `doc/bin/obs.md` if they +show paths. + +## Required + +- [Bindings](/quest/m2/epoch/bindings.md) - OBS publishes through libmoq diff --git a/quest/m2/epoch/origin.md b/quest/m2/epoch/origin.md new file mode 100644 index 00000000000..b5cba324869 --- /dev/null +++ b/quest/m2/epoch/origin.md @@ -0,0 +1,40 @@ +# [L] Origin: mint, follow, and resolve epochs + +## Goal + +In `moq-net` (Rust and JS), publishing a broadcast at a bare path announces it +under a fresh epoch. A consumer of a bare name follows the newest epoch with +a live route and falls back to an older live one. A bare request resolves the +same way on every protocol version, so the relay serves old clients without +any wire change. + +## Plan + +- Publish: the broadcast create-and-announce path appends `Epoch::mint()` + unless the path already has an epoch. `announce(prefix, route)` stays raw. +- Consume: `routed` and `request_broadcast` on a bare name watch the routes + one `@` segment below it. They pick the greatest live epoch, and on a table + change, re-select: move up at once, or fall back when the current one is + retracted. A path that names an epoch pins it and never moves. Each move is + a new broadcast to the caller, never a splice. The #3312 first-hop resume + rule still applies within one epoch path. +- Bare resolution: a bare request with no route of its own resolves to that + epoch. A takeover ends the bare subscription with a typed reset, never a + silent switch. Choose the code so existing clients resubscribe rather than + give up. +- A nested epoch (`name/@e/derived/@f`) resolves per level. Split-horizon + exclusion and the per-path fronts from #3312 stay intact. +- Benchmark resolution swept over epochs per name and names per origin, so + following does not scan the table. +- Update `doc/concept` and `drafts/draft-lcurley-moq-lite.md` wherever they + describe resolution or takeover. The rule is a relay behavior, so state it + in the draft even though no field changes. + +Public API: behavior change on publish (the announced path gains an epoch) +and on bare-name consume. Decide at PR time whether that retargets to `dev`. +Wire: none. + +## Related + +- [#2991](/quest/m1/2991-net-coalesce-dynamic-tracks-and-preserve-sequences-across.md) - a new epoch starts each track at sequence 0 +- [Broadcast route](/quest/m1/js-broadcast-route.md) - the JS claim matching the follow logic sits on diff --git a/quest/m2/redundant-ingest.md b/quest/m2/redundant-ingest.md new file mode 100644 index 00000000000..917ed230fc8 --- /dev/null +++ b/quest/m2/redundant-ingest.md @@ -0,0 +1,23 @@ +# [M] Redundant ingest under one epoch + +## Goal + +Decide whether and how two live publishers of identical content share one +broadcast, so viewers survive losing one faster than the QUIC keep-alive. +Epochs let the publishers claim one identity (the same `@`), but +today's #3312 rule splices only across routes with the same first hop, so +two ingest hosts are two identities. The study may end in a no-go. + +## Plan + +Open questions: whether an explicitly shared epoch may splice across first +hops at a group boundary, and what makes that safe (group sequences aligned +across encoders, a matching catalog). Also who declares the incumbent dead +early: a failover service that retracts it, or active-active delivery to the +relay. Weigh them against the moq-transport rule that multiple publishers of +a namespace must each be asked (#3697) and the cluster draft. Output: a +decision, with a quest for the chosen mechanism. + +## Related + +- [Broadcast epochs](/quest/m2/epoch/README.md) - explicit epochs are what a redundant pair would share From 21f66f85c6c2e2f1e24cd0bf43df980fc2de3977 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 23 Sep 2026 22:24:36 +0000 Subject: [PATCH 2/4] docs(quest): move broadcast epochs to m1 Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_016FYDouB6hmhgDSYKxgvygb --- quest/m1/README.md | 1 + quest/{m2/epoch => m1/broadcast-epoch}/README.md | 10 +++++----- quest/{m2/epoch => m1/broadcast-epoch}/apps.md | 2 +- quest/{m2/epoch => m1/broadcast-epoch}/bindings.md | 2 +- quest/{m2/epoch => m1/broadcast-epoch}/gateways.md | 2 +- quest/{m2/epoch => m1/broadcast-epoch}/gst-obs.md | 2 +- quest/{m2/epoch => m1/broadcast-epoch}/origin.md | 0 quest/m1/e2ee/README.md | 2 +- quest/m1/epoch.md | 4 ++-- quest/m1/wildcard/README.md | 2 +- quest/m2/README.md | 1 - quest/m2/redundant-ingest.md | 2 +- 12 files changed, 15 insertions(+), 15 deletions(-) rename quest/{m2/epoch => m1/broadcast-epoch}/README.md (79%) rename quest/{m2/epoch => m1/broadcast-epoch}/apps.md (89%) rename quest/{m2/epoch => m1/broadcast-epoch}/bindings.md (89%) rename quest/{m2/epoch => m1/broadcast-epoch}/gateways.md (89%) rename quest/{m2/epoch => m1/broadcast-epoch}/gst-obs.md (86%) rename quest/{m2/epoch => m1/broadcast-epoch}/origin.md (100%) diff --git a/quest/m1/README.md b/quest/m1/README.md index ab8cfb0eabe..3b50daa301e 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -102,6 +102,7 @@ transport, benchmark tooling); worktrees isolate commits, not semantics. - [JS track end](/quest/m1/2318-js-net-remaining-capability-gaps-vs-rs-moq-net-setup-role.md) - js/net declares a track end ahead of the live edge and observes the publisher's SUBSCRIBE_END - [Epoch primitive](/quest/m1/epoch.md) - one `Epoch` type in moq-net and @moq/net, carried as a trailing `@` path segment, shared by e2ee and broadcast epochs - [E2EE](/quest/m1/e2ee/README.md) - TypeScript and Rust peers interoperate over encrypted broadcasts no relay can decrypt +- [Broadcast epochs](/quest/m1/broadcast-epoch/README.md) - each publish of a name gets a fresh `@` epoch, viewers follow the newest live one at once, and bare names still resolve on every version - [Processor](/quest/m1/processor/README.md) - a customer-run worker publishes an on-demand contribution with scoped access - [#3056](/quest/m1/3056-watch-video-decoder-captures-the-rewind-generation-at.md) - watch: the video decoder resets on a declared discontinuity - [#933](/quest/m1/933-video-rotation-metadata-not-propagated-from-mobile-camera.md) - the catalog rotation follows the live camera's orientation diff --git a/quest/m2/epoch/README.md b/quest/m1/broadcast-epoch/README.md similarity index 79% rename from quest/m2/epoch/README.md rename to quest/m1/broadcast-epoch/README.md index fde44400651..6064aebdd94 100644 --- a/quest/m2/epoch/README.md +++ b/quest/m1/broadcast-epoch/README.md @@ -54,11 +54,11 @@ This README owns: ## Quests -- [Origin](/quest/m2/epoch/origin.md) - moq-net publish mints an epoch, consumers follow the newest live one, and bare requests resolve to it on every version -- [Apps](/quest/m2/epoch/apps.md) - moq-cli, the browser publish and watch components, and demo/web publish under epochs and play bare names -- [Gateways](/quest/m2/epoch/gateways.md) - RTMP, SRT, and WHIP ingest mint an epoch per incoming connection, so an encoder reconnect is a clean takeover -- [Bindings](/quest/m2/epoch/bindings.md) - moq-ffi, libmoq, and every wrapper expose the epoch and inherit the default -- [GStreamer and OBS](/quest/m2/epoch/gst-obs.md) - moqsink and the OBS plugin publish each run under a fresh epoch +- [Origin](/quest/m1/broadcast-epoch/origin.md) - moq-net publish mints an epoch, consumers follow the newest live one, and bare requests resolve to it on every version +- [Apps](/quest/m1/broadcast-epoch/apps.md) - moq-cli, the browser publish and watch components, and demo/web publish under epochs and play bare names +- [Gateways](/quest/m1/broadcast-epoch/gateways.md) - RTMP, SRT, and WHIP ingest mint an epoch per incoming connection, so an encoder reconnect is a clean takeover +- [Bindings](/quest/m1/broadcast-epoch/bindings.md) - moq-ffi, libmoq, and every wrapper expose the epoch and inherit the default +- [GStreamer and OBS](/quest/m1/broadcast-epoch/gst-obs.md) - moqsink and the OBS plugin publish each run under a fresh epoch ## Required diff --git a/quest/m2/epoch/apps.md b/quest/m1/broadcast-epoch/apps.md similarity index 89% rename from quest/m2/epoch/apps.md rename to quest/m1/broadcast-epoch/apps.md index fb6472b4add..93e4bc1e4bc 100644 --- a/quest/m2/epoch/apps.md +++ b/quest/m1/broadcast-epoch/apps.md @@ -18,4 +18,4 @@ The UI and logs show the full epoch path, and a watch link can pin one. ## Required -- [Origin](/quest/m2/epoch/origin.md) - the publish default and follow logic +- [Origin](/quest/m1/broadcast-epoch/origin.md) - the publish default and follow logic diff --git a/quest/m2/epoch/bindings.md b/quest/m1/broadcast-epoch/bindings.md similarity index 89% rename from quest/m2/epoch/bindings.md rename to quest/m1/broadcast-epoch/bindings.md index 262b6b81374..ed5567ef691 100644 --- a/quest/m2/epoch/bindings.md +++ b/quest/m1/broadcast-epoch/bindings.md @@ -18,4 +18,4 @@ reconnect counter `session.epoch()` is renamed so "epoch" has one meaning. ## Required -- [Origin](/quest/m2/epoch/origin.md) - the behavior the bindings surface +- [Origin](/quest/m1/broadcast-epoch/origin.md) - the behavior the bindings surface diff --git a/quest/m2/epoch/gateways.md b/quest/m1/broadcast-epoch/gateways.md similarity index 89% rename from quest/m2/epoch/gateways.md rename to quest/m1/broadcast-epoch/gateways.md index e7ef109a503..89a3df5c853 100644 --- a/quest/m2/epoch/gateways.md +++ b/quest/m1/broadcast-epoch/gateways.md @@ -16,4 +16,4 @@ describes ingest paths. ## Required -- [Origin](/quest/m2/epoch/origin.md) - the publish default +- [Origin](/quest/m1/broadcast-epoch/origin.md) - the publish default diff --git a/quest/m2/epoch/gst-obs.md b/quest/m1/broadcast-epoch/gst-obs.md similarity index 86% rename from quest/m2/epoch/gst-obs.md rename to quest/m1/broadcast-epoch/gst-obs.md index 57626b62224..5ae7824177d 100644 --- a/quest/m2/epoch/gst-obs.md +++ b/quest/m1/broadcast-epoch/gst-obs.md @@ -15,4 +15,4 @@ show paths. ## Required -- [Bindings](/quest/m2/epoch/bindings.md) - OBS publishes through libmoq +- [Bindings](/quest/m1/broadcast-epoch/bindings.md) - OBS publishes through libmoq diff --git a/quest/m2/epoch/origin.md b/quest/m1/broadcast-epoch/origin.md similarity index 100% rename from quest/m2/epoch/origin.md rename to quest/m1/broadcast-epoch/origin.md diff --git a/quest/m1/e2ee/README.md b/quest/m1/e2ee/README.md index 0160e8535ee..824379b5c54 100644 --- a/quest/m1/e2ee/README.md +++ b/quest/m1/e2ee/README.md @@ -20,7 +20,7 @@ The contract is [draft-lcurley-moq-e2ee](/drafts/draft-lcurley-moq-e2ee.md), pro ### Epoch and identity - Every publisher instance mints an epoch, a UUIDv7 in lowercase text, and publishes at `/@`, where `` derives from the credential and the semantic broadcast name without the epoch. The epoch is an input to every HKDF derivation, so a restart, takeover, or explicit group sequence cannot repeat a nonce under a key: nothing is persisted across instances and no generation counter is redistributed. The epoch is the shared [`Epoch`](/quest/m1/epoch.md) primitive. Subscribers discover instances under the `/` prefix and take the greatest epoch, which sorts newest; a known full path carries its epoch in the last segment. -- The epoch is untrusted and unauthenticated. A wrong epoch fails authentication and a withheld one denies service; neither can make a nonce repeat, because only the publisher instance chooses what it encrypts under. This is the same trust a cache needs to serve the right instance. Plaintext broadcasts adopt the same epoch by default through [broadcast epochs](/quest/m2/epoch/README.md). +- The epoch is untrusted and unauthenticated. A wrong epoch fails authentication and a withheld one denies service; neither can make a nonce repeat, because only the publisher instance chooses what it encrypts under. This is the same trust a cache needs to serve the right instance. Plaintext broadcasts adopt the same epoch by default through [broadcast epochs](/quest/m1/broadcast-epoch/README.md). - Nothing in the path says the bytes are encrypted. The format after decryption (`meeting.hang`) is inside the opaque name; a plaintext player, exporter, or matcher that opens a protected broadcast finds no catalog it can read and fails with its usual typed refusal, the same as for any format it does not support. - One 32-byte secret authorizes the whole broadcast; HKDF-SHA-256 derives separate AES-128-GCM keys for each physical track and for grouped-frame versus datagram domains. A grouped frame uses the 96-bit nonce `uint64_be(group) || uint32_be(frame)`; a datagram uses its sequence with frame zero under the datagram domain. Empty AAD: every immutable end-to-end field is in the HKDF info or the nonce. - Within an instance, group and datagram sequences are allocated monotonically and frames are numbered in write order, which is the whole reuse rule. There is no ciphertext retention or retransmission API: relays forward bytes unchanged, and an application that needs to resend encrypts nothing twice because it never gets the same identity twice. diff --git a/quest/m1/epoch.md b/quest/m1/epoch.md index 6a0c2dc943c..0323e9322c2 100644 --- a/quest/m1/epoch.md +++ b/quest/m1/epoch.md @@ -9,7 +9,7 @@ and reports its wall-clock time. A path splits into its epoch-free name and an optional epoch. The epoch rides in the path, so it survives any moq-transport relay with no wire change. -This is the shared primitive the [broadcast epochs](/quest/m2/epoch/README.md) +This is the shared primitive the [broadcast epochs](/quest/m1/broadcast-epoch/README.md) line builds on, and it replaces the e2ee-local `moq_e2ee::Epoch`. ## Plan @@ -30,7 +30,7 @@ line builds on, and it replaces the e2ee-local `moq_e2ee::Epoch`. - Golden cross-language vectors for parse, reject, order, and time, beside the existing path tests. - Nothing mints by default here. Publish and consume behavior is - [origin](/quest/m2/epoch/origin.md). + [origin](/quest/m1/broadcast-epoch/origin.md). Public API: additive `Epoch` and path helpers in both languages. The `moq_e2ee::Epoch` move is a break on an unpublished crate. Wire: none. diff --git a/quest/m1/wildcard/README.md b/quest/m1/wildcard/README.md index c4e81fd0528..9a07b0a66a5 100644 --- a/quest/m1/wildcard/README.md +++ b/quest/m1/wildcard/README.md @@ -271,5 +271,5 @@ than announce state. pattern, and its catalog names the generations a wildcard cannot - [pop-skipping](/quest/m1/pop-skipping/README.md) - it owns the route cost and the rank hash this reuses -- [Broadcast epochs](/quest/m2/epoch/README.md) - derived output moves under the +- [Broadcast epochs](/quest/m1/broadcast-epoch/README.md) - derived output moves under the source's `@` segment, which the suffix patterns still match diff --git a/quest/m2/README.md b/quest/m2/README.md index 0bce2c1e0e3..db6dd0c5249 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -42,7 +42,6 @@ upstream release waits in [m4](/quest/m4/README.md). - [vcpkg registry](/quest/m2/cpp-vcpkg.md) - a registry we own serves the prebuilt package to `vcpkg` manifests - [Conan remote](/quest/m2/cpp-conan.md) - a remote we own serves the same tarball to `conan install` - [Compressed tracks](/quest/m2/flate/README.md) - any track compresses per group from every language, not only the JSON modes -- [Broadcast epochs](/quest/m2/epoch/README.md) - each publish of a name gets a fresh `@` epoch, viewers follow the newest live one at once, and bare names still resolve on every version - [#3115](/quest/m2/3115-moqsink-the-publication-has-no-generation-so-a-flush.md) - moqsink: a flushing restart after EOS opens a new publication generation - [Redundant ingest](/quest/m2/redundant-ingest.md) - decide whether two publishers sharing one epoch may splice, and who declares the incumbent dead before the keep-alive does - [Multipath spike](/quest/m2/multipath-spike.md) - whether bonded contribution over multipath QUIC is worth building, given it needs noq on both ends diff --git a/quest/m2/redundant-ingest.md b/quest/m2/redundant-ingest.md index 917ed230fc8..6713d87dd3e 100644 --- a/quest/m2/redundant-ingest.md +++ b/quest/m2/redundant-ingest.md @@ -20,4 +20,4 @@ decision, with a quest for the chosen mechanism. ## Related -- [Broadcast epochs](/quest/m2/epoch/README.md) - explicit epochs are what a redundant pair would share +- [Broadcast epochs](/quest/m1/broadcast-epoch/README.md) - explicit epochs are what a redundant pair would share From cd527f21d53445064766cb49a78a8db00a0a9d86 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 24 Sep 2026 07:24:39 -0700 Subject: [PATCH 3/4] docs(quest): drop the re-added #2985 quest, which #3422 completed Co-Authored-By: Claude Opus 5.5 --- ...keyed-publisher-state-goes-stale-when-a.md | 28 ------------------- 1 file changed, 28 deletions(-) delete mode 100644 quest/m1/2985-js-net-path-keyed-publisher-state-goes-stale-when-a.md diff --git a/quest/m1/2985-js-net-path-keyed-publisher-state-goes-stale-when-a.md b/quest/m1/2985-js-net-path-keyed-publisher-state-goes-stale-when-a.md deleted file mode 100644 index 8e9c5cf049a..00000000000 --- a/quest/m1/2985-js-net-path-keyed-publisher-state-goes-stale-when-a.md +++ /dev/null @@ -1,28 +0,0 @@ -# [S] js/net: path-keyed publisher state follows a same-path replacement - -## Goal - -Replacing a path's producer in `js/net` without closing the predecessor -behaves like a new broadcast. Discovery announces the replacement, `TRACK_INFO` -answers with the successor's metadata over both SUBSCRIBE and FETCH, and an -IETF `PUBLISH_NAMESPACE` refusal against the predecessor no longer suppresses -the successor. - -Epochs give each default publish a distinct path, but raw paths (the -prefix-route opt-out) still reuse names, so this still needs its own fix. - -## Plan - -Follow the local fix the issue proposes: reconcile announce state by producer -identity rather than by key, evict the path's `#trackInfo` entries on -`publish`, and key `refused` by producer identity. The issue lists the -regression tests. Check whether Rust has the same three gaps and fix any it -does in the same PR. Public API: none. Wire: none. - -## Closes - -- [#2985](https://github.com/moq-dev/moq/issues/2985) - close this issue when the quest finishes - -## Related - -- [#2991](/quest/m1/2991-net-coalesce-dynamic-tracks-and-preserve-sequences-across.md) - the dynamic-track half of the same replacement identity problem From 0947b66c5992b12fedeb380a91cc5951b04711be Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 24 Sep 2026 07:30:04 -0700 Subject: [PATCH 4/4] docs(quest): scope epoch auth, relay compatibility, and path limits Co-Authored-By: Claude Opus 5.5 --- quest/m1/README.md | 1 - quest/m1/broadcast-epoch/README.md | 15 +++++++++------ quest/m1/broadcast-epoch/origin.md | 11 +++++++++-- quest/m1/epoch.md | 7 ++++--- 4 files changed, 22 insertions(+), 12 deletions(-) diff --git a/quest/m1/README.md b/quest/m1/README.md index 3b50daa301e..0fb7637c2db 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -98,7 +98,6 @@ transport, benchmark tooling); worktrees isolate commits, not semantics. - [JS abandonment](/quest/m1/js-subscribe-abandonment.md) - a viewer returning during IETF subscribe setup keeps its track across microtasks - [IETF stream types](/quest/m1/ietf-uni-stream-types.md) - padding streams are discarded stream-only and an unknown uni type closes the session, per draft-21 - [#2991](/quest/m1/2991-net-coalesce-dynamic-tracks-and-preserve-sequences-across.md) - one dynamic producer per track name in both languages, with the sequence namespace surviving a replacement -- [#2985](/quest/m1/2985-js-net-path-keyed-publisher-state-goes-stale-when-a.md) - js/net: replacing a path's producer re-announces it and invalidates the predecessor's TRACK_INFO and refusal state - [JS track end](/quest/m1/2318-js-net-remaining-capability-gaps-vs-rs-moq-net-setup-role.md) - js/net declares a track end ahead of the live edge and observes the publisher's SUBSCRIBE_END - [Epoch primitive](/quest/m1/epoch.md) - one `Epoch` type in moq-net and @moq/net, carried as a trailing `@` path segment, shared by e2ee and broadcast epochs - [E2EE](/quest/m1/e2ee/README.md) - TypeScript and Rust peers interoperate over encrypted broadcasts no relay can decrypt diff --git a/quest/m1/broadcast-epoch/README.md b/quest/m1/broadcast-epoch/README.md index 6064aebdd94..a0a9551ecc8 100644 --- a/quest/m1/broadcast-epoch/README.md +++ b/quest/m1/broadcast-epoch/README.md @@ -11,9 +11,8 @@ the old route's QUIC idle timeout. Moving to a new epoch is a clean boundary by its full path. The epoch rides in the path, so it survives any moq-transport relay, and no -wire message changes. A publisher that cannot or will not mint one is still -served: a request for a bare name resolves to its newest live epoch on every -protocol version. +wire message changes. At an epoch-aware relay, a request for a bare name +resolves to its newest live epoch on every protocol version. Non-goals: redundant publishers sharing one epoch, and failing over between them faster than the keep-alive (the [redundant ingest](/quest/m2/redundant-ingest.md) @@ -34,9 +33,13 @@ Decided: - Viewers follow the greatest epoch with a live route. When it goes away and an older one is still live, they fall back to it. - A bare request with no route of its own resolves to that same epoch on every - version, so lite-06 and IETF clients and third-party relays keep working. - When a newer epoch appears, the bare subscription ends with a typed reset, - and the client's normal resubscribe lands on the new one. + version, so lite-06 and IETF clients keep working through an epoch-aware + relay. When a newer epoch appears, the bare subscription ends with a typed + reset, and the client's normal resubscribe lands on the new one. +- An unmodified third-party relay routes `foo/@` but never resolves a + bare `foo`, since a route covers its descendants, not its parent. A + bare-name viewer behind one needs a publisher that opts out with the raw + prefix route. Document this rather than promise it works. - Derived output lives under the epoch it came from (`pid/foo.hang/@e/transcode.pro`), so nested epochs must parse. This moves the [wildcard](/quest/m1/wildcard/README.md) line's derived-output example diff --git a/quest/m1/broadcast-epoch/origin.md b/quest/m1/broadcast-epoch/origin.md index b5cba324869..571286a03b8 100644 --- a/quest/m1/broadcast-epoch/origin.md +++ b/quest/m1/broadcast-epoch/origin.md @@ -12,6 +12,12 @@ any wire change. - Publish: the broadcast create-and-announce path appends `Epoch::mint()` unless the path already has an epoch. `announce(prefix, route)` stays raw. + A bare path already at the 32-part limit has no room for the epoch segment: + refuse it at publish, pointing at the raw route, rather than fail at encode. +- Auth: a grant that admits bare `foo` admits `foo/@` for both publish + and subscribe, and a grant on one epoch never widens to the bare name or its + siblings. Check how exact grants and patterns match today, and test both + sides in moq-relay's auth tests. - Consume: `routed` and `request_broadcast` on a bare name watch the routes one `@` segment below it. They pick the greatest live epoch, and on a table change, re-select: move up at once, or fall back when the current one is @@ -30,8 +36,9 @@ any wire change. describe resolution or takeover. The rule is a relay behavior, so state it in the draft even though no field changes. -Public API: behavior change on publish (the announced path gains an epoch) -and on bare-name consume. Decide at PR time whether that retargets to `dev`. +Public API: behavior change on publish (the announced path gains an epoch, +and a 32-part bare path is refused), on bare-name consume, and on what an +exact grant admits. Decide at PR time whether that retargets to `dev`. Wire: none. ## Related diff --git a/quest/m1/epoch.md b/quest/m1/epoch.md index 0323e9322c2..ee20dd99d0c 100644 --- a/quest/m1/epoch.md +++ b/quest/m1/epoch.md @@ -17,9 +17,10 @@ line builds on, and it replaces the e2ee-local `moq_e2ee::Epoch`. - Move `rs/moq-e2ee/src/epoch.rs` into `moq-net` and add the JS equivalent. Parsing is strict: `@` followed by a lowercase hyphenated UUIDv7. Nothing else counts as an epoch, so an app's own UUID segments never parse as one. -- Path helpers split a path into name and epoch and join them back. New path - construction rejects a segment that starts with `@` but is not a valid epoch, - so the marker stays unambiguous. Check how that interacts with +- Path helpers split a path into name and epoch and join them back. A segment + like `@alice` is valid today and stays valid: strict parsing already keeps it + from reading as an epoch. Rejecting it instead would break the path contract + and land on `dev`. Check how the split interacts with [path patterns](/quest/m1/path-patterns.md) and [hidden broadcasts](/quest/m1/hidden-broadcasts.md) (a leading `.`). - `moq-e2ee` uses the shared type. Update