Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions quest/m1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,9 @@ transport, benchmark tooling); worktrees isolate commits, not semantics.
- [IETF stream types](/quest/m1/ietf-uni-stream-types.md) - padding streams are discarded stream-only and an unknown uni type closes the session, per draft-21
- [#2991](/quest/m1/2991-net-coalesce-dynamic-tracks-and-preserve-sequences-across.md) - one dynamic producer per track name in both languages, with the sequence namespace surviving a replacement
- [JS track end](/quest/m1/2318-js-net-remaining-capability-gaps-vs-rs-moq-net-setup-role.md) - js/net declares a track end ahead of the live edge and observes the publisher's SUBSCRIBE_END
- [Epoch primitive](/quest/m1/epoch.md) - one `Epoch` type in moq-net and @moq/net, carried as a trailing `@<uuidv7>` path segment, shared by e2ee and broadcast epochs
- [E2EE](/quest/m1/e2ee/README.md) - TypeScript and Rust peers interoperate over encrypted broadcasts no relay can decrypt
- [Broadcast epochs](/quest/m1/broadcast-epoch/README.md) - each publish of a name gets a fresh `@<uuidv7>` epoch, viewers follow the newest live one at once, and bare names still resolve on every version
- [Processor](/quest/m1/processor/README.md) - a customer-run worker publishes an on-demand contribution with scoped access
- [#3056](/quest/m1/3056-watch-video-decoder-captures-the-rewind-generation-at.md) - watch: the video decoder resets on a declared discontinuity
- [#933](/quest/m1/933-video-rotation-metadata-not-propagated-from-mobile-camera.md) - the catalog rotation follows the live camera's orientation
Expand Down
68 changes: 68 additions & 0 deletions quest/m1/broadcast-epoch/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# Broadcast epochs

## Goal

Reusing a broadcast name is never ambiguous. By default, each publish of
`demo/BBB.hang` goes out as `demo/BBB.hang/@<uuidv7>`, so the relay and viewers
can tell a new broadcast from a continuation. A viewer of the name follows
the newest live epoch as soon as it is announced, instead of waiting ~30s for
the old route's QUIC idle timeout. Moving to a new epoch is a clean boundary
(a fresh catalog and tracks, never spliced), and each run stays addressable
by its full path.

The epoch rides in the path, so it survives any moq-transport relay, and no
wire message changes. At an epoch-aware relay, a request for a bare name
resolves to its newest live epoch on every protocol version.

Non-goals: redundant publishers sharing one epoch, and failing over between
them faster than the keep-alive (the [redundant ingest](/quest/m2/redundant-ingest.md)
study). Also out of scope: trusting the publisher's clock (a far-future epoch
wins until its route goes away).

## Plan

Decided:

- The marker is a child segment `@<uuidv7>`, parsed into
`Option<Epoch>` by [the primitive](/quest/m1/epoch.md). Not a lite-07 flag:
the path is the only carrier.
- The broadcast-publish path mints an epoch unless the path already carries
one. A caller who passes an explicit epoch, such as a redundant publisher,
keeps it. The prefix-route `announce(prefix, route)` stays raw, and that is
the opt-out. So prefix vs broadcast is an API distinction, not a wire flag.
- Viewers follow the greatest epoch with a live route. When it goes away and an
older one is still live, they fall back to it.
- A bare request with no route of its own resolves to that same epoch on every
version, so lite-06 and IETF clients keep working through an epoch-aware
relay. When a newer epoch appears, the bare subscription ends with a typed
reset, and the client's normal resubscribe lands on the new one.
- An unmodified third-party relay routes `foo/@<epoch>` but never resolves a
bare `foo`, since a route covers its descendants, not its parent. A
bare-name viewer behind one needs a publisher that opts out with the raw
prefix route. Document this rather than promise it works.
- Derived output lives under the epoch it came from
(`pid/foo.hang/@e/transcode.pro`), so nested epochs must parse. This moves
the [wildcard](/quest/m1/wildcard/README.md) line's derived-output example
down one segment, and its suffix patterns still match.

This README owns:

- An end-to-end relay test: republish a name while the old publisher's session
stays open. A new-API viewer and a lite-06 or IETF bare-path viewer both
reach the new epoch within one RTT-scale bound rather than the idle timeout.
Killing the newest epoch falls back to a still-live older one.
- A `doc/concept` page on broadcast naming: what an epoch is, publish and
consume behavior, takeover and fallback, bare-path resolution, and the
prefix-route opt-out.

## Quests

- [Origin](/quest/m1/broadcast-epoch/origin.md) - moq-net publish mints an epoch, consumers follow the newest live one, and bare requests resolve to it on every version
- [Apps](/quest/m1/broadcast-epoch/apps.md) - moq-cli, the browser publish and watch components, and demo/web publish under epochs and play bare names
- [Gateways](/quest/m1/broadcast-epoch/gateways.md) - RTMP, SRT, and WHIP ingest mint an epoch per incoming connection, so an encoder reconnect is a clean takeover
- [Bindings](/quest/m1/broadcast-epoch/bindings.md) - moq-ffi, libmoq, and every wrapper expose the epoch and inherit the default
- [GStreamer and OBS](/quest/m1/broadcast-epoch/gst-obs.md) - moqsink and the OBS plugin publish each run under a fresh epoch

## Required

- [Epoch primitive](/quest/m1/epoch.md) - the shared `Epoch` type and path split
Comment on lines +66 to +68

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Put the epoch prerequisite on the first child

quest ready does not propagate a questline README's Required entries to its children: it traverses the README's Quests, then evaluates each plain quest's own Required section. Consequently origin.md, which directly needs Epoch::mint(), is advertised as ready before the epoch primitive exists. Put this dependency on the Origin quest so the documented execution order is enforceable.

Useful? React with 👍 / 👎.

21 changes: 21 additions & 0 deletions quest/m1/broadcast-epoch/apps.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# [M] Apps publish under epochs and play bare names

## Goal

`moq-cli` publish and play, `@moq/publish`, `@moq/watch`, and `demo/web` use
the origin default. Each publish run is a new epoch, and watching a bare name
switches to a republish within an RTT rather than after the idle timeout.
The UI and logs show the full epoch path, and a watch link can pin one.

## Plan

- Publish sides need little beyond passing bare names. Check that nothing
caches the announced path across a restart.
- Watch sides handle "the broadcast changed" as a fresh catalog and decoder
reset. Test a republish mid-playback in the browser and native players.
- Update `doc/bin/cli.md` and every example invocation that shows a published
path.

## Required

- [Origin](/quest/m1/broadcast-epoch/origin.md) - the publish default and follow logic
21 changes: 21 additions & 0 deletions quest/m1/broadcast-epoch/bindings.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# [M] Bindings expose epochs

## Goal

`moq-ffi`, `libmoq`, and the py, swift, kt, go, and dart wrappers publish under
the default epoch and follow bare names like Rust. The epoch of a published or
consumed broadcast is readable, and a caller can pass an explicit one. The
reconnect counter `session.epoch()` is renamed so "epoch" has one meaning.

## Plan

- Expose the parsed epoch (text and time) and an explicit-epoch publish
argument. Keep the surface to what a binding consumer needs.
- Rename `session.epoch()` in every binding (for example to `connects()`).
That is a break, so it lands on `dev`.
- Update `doc/lib/{py,swift,kt,go,dart,c}` per the cross-package sync table,
and run `just test smoke --all`.

## Required

- [Origin](/quest/m1/broadcast-epoch/origin.md) - the behavior the bindings surface
19 changes: 19 additions & 0 deletions quest/m1/broadcast-epoch/gateways.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# [S] Ingest gateways mint an epoch per connection

## Goal

RTMP, SRT, and WHIP ingest publish each incoming connection under a fresh
epoch. An encoder that reconnects to the same stream key becomes a clean
takeover at once, instead of a second publisher racing the stale one until it
times out.

## Plan

Use the origin default where the gateways create broadcasts (`moq-rtmp`,
`moq-srt`, `moq-rtc`, and the relay wiring). Test a reconnect under the same
key with the stale connection still open. Update `doc/bin/relay/` where it
describes ingest paths.

## Required

- [Origin](/quest/m1/broadcast-epoch/origin.md) - the publish default
18 changes: 18 additions & 0 deletions quest/m1/broadcast-epoch/gst-obs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# [S] GStreamer and OBS publish under epochs

## Goal

`moqsink` and the OBS plugin publish each run under a fresh epoch, so a
restarted pipeline or a stop and start in OBS is a clean takeover for viewers.

## Plan

`moqsink` takes the origin default per session. When
[#3115](/quest/m2/3115-moqsink-the-publication-has-no-generation-so-a-flush.md)
lands, each of its publication generations is a new epoch. OBS gets it
through libmoq. Update `doc/bin/gstreamer.md` and `doc/bin/obs.md` if they
show paths.

## Required

- [Bindings](/quest/m1/broadcast-epoch/bindings.md) - OBS publishes through libmoq
47 changes: 47 additions & 0 deletions quest/m1/broadcast-epoch/origin.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# [L] Origin: mint, follow, and resolve epochs

## Goal

In `moq-net` (Rust and JS), publishing a broadcast at a bare path announces it
under a fresh epoch. A consumer of a bare name follows the newest epoch with
a live route and falls back to an older live one. A bare request resolves the
same way on every protocol version, so the relay serves old clients without
any wire change.

## Plan

- Publish: the broadcast create-and-announce path appends `Epoch::mint()`
unless the path already has an epoch. `announce(prefix, route)` stays raw.
Comment on lines +13 to +14

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep valid 32-part paths publishable

For a currently valid bare path containing the IETF maximum of 32 parts, appending this child produces 33 parts. Both Rust and JS encoders reject paths beyond 32, so the new default would make such broadcasts fail before reaching the wire. The plan should reserve capacity with an explicit compatibility decision or choose an encoding that does not add a part, and include this break in the stated API impact.

AGENTS.md reference: AGENTS.md:L58-L60

Useful? React with 👍 / 👎.

Comment on lines +13 to +14

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Name the actual prefix-route API

There is no public announce(prefix, route) opt-out in either implementation: Rust's only method with that signature is #[cfg(test)] pub(crate) in rs/moq-net/src/model/origin.rs, while Rust and JS expose served prefix routes as dynamic(prefix, route). As written, callers cannot follow the documented opt-out and an implementer may accidentally change the exact-broadcast announce(route) API instead. Refer to dynamic here, or explicitly plan a new public API if that is intentional.

Useful? React with 👍 / 👎.

A bare path already at the 32-part limit has no room for the epoch segment:
refuse it at publish, pointing at the raw route, rather than fail at encode.
- Auth: a grant that admits bare `foo` admits `foo/@<epoch>` for both publish
and subscribe, and a grant on one epoch never widens to the bare name or its
siblings. Check how exact grants and patterns match today, and test both
sides in moq-relay's auth tests.
- Consume: `routed` and `request_broadcast` on a bare name watch the routes
one `@` segment below it. They pick the greatest live epoch, and on a table
Comment on lines +13 to +22

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve exact-path authorization across epoch resolution

Appending and resolving the child path crosses the existing authorization boundary: a literal grant such as publish: ["foo"] or subscribe: ["foo"] permits exactly foo, not foo/@<epoch>. Consequently the scoped origin will either reject the epoch route, breaking publication and viewing for valid tokens, or the implementation will have to bypass the grant. Define how epoch aliases interact with exact grants and cover both publish and subscribe authorization before treating this as only a behavior change.

AGENTS.md reference: AGENTS.md:L58-L60

Useful? React with 👍 / 👎.

change, re-select: move up at once, or fall back when the current one is
retracted. A path that names an epoch pins it and never moves. Each move is
a new broadcast to the caller, never a splice. The #3312 first-hop resume
rule still applies within one epoch path.
- Bare resolution: a bare request with no route of its own resolves to that

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Resolve epoch children before wildcard routes

When a derived output is served by the existing **/transcode.pro wildcard (quest/m1/wildcard/README.md:211-215), its bare path already has a matching dynamic route, so this condition never examines the advertised .../transcode.pro/@f children. Later bare viewers therefore keep reaching the wildcard handler instead of following the newest output epoch or receiving takeover resets. Specify that an existing epoch child takes precedence over a covering wildcard route, while the wildcard remains the cold-start fallback, and cover that ordering in the derived-output test.

Useful? React with 👍 / 👎.

epoch. A takeover ends the bare subscription with a typed reset, never a
silent switch. Choose the code so existing clients resubscribe rather than
give up.
- A nested epoch (`name/@e/derived/@f`) resolves per level. Split-horizon
exclusion and the per-path fronts from #3312 stay intact.
- Benchmark resolution swept over epochs per name and names per origin, so
following does not scan the table.
- Update `doc/concept` and `drafts/draft-lcurley-moq-lite.md` wherever they
describe resolution or takeover. The rule is a relay behavior, so state it
in the draft even though no field changes.

Public API: behavior change on publish (the announced path gains an epoch,
and a 32-part bare path is refused), on bare-name consume, and on what an
exact grant admits. Decide at PR time whether that retargets to `dev`.
Wire: none.

## Related

- [#2991](/quest/m1/2991-net-coalesce-dynamic-tracks-and-preserve-sequences-across.md) - a new epoch starts each track at sequence 0
- [Broadcast route](/quest/m1/js-broadcast-route.md) - the JS claim matching the follow logic sits on
5 changes: 3 additions & 2 deletions quest/m1/e2ee/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ The contract is [draft-lcurley-moq-e2ee](/drafts/draft-lcurley-moq-e2ee.md), pro

### Epoch and identity

- Every publisher instance mints an epoch, a UUIDv7 in lowercase text, and publishes at `<opaque>/<epoch>`, where `<opaque>` derives from the credential and the semantic broadcast name without the epoch. The epoch is an input to every HKDF derivation, so a restart, takeover, or explicit group sequence cannot repeat a nonce under a key: nothing is persisted across instances and no generation counter is redistributed. Subscribers discover instances under the `<opaque>/` prefix and take the greatest epoch, which sorts newest; a known full path carries its epoch in the last segment.
- The epoch is untrusted and unauthenticated. A wrong epoch fails authentication and a withheld one denies service; neither can make a nonce repeat, because only the publisher instance chooses what it encrypts under. This is the same trust a cache needs to serve the right instance, and it is deliberately e2ee-only: plaintext hang keeps its current paths.
- Every publisher instance mints an epoch, a UUIDv7 in lowercase text, and publishes at `<opaque>/@<epoch>`, where `<opaque>` derives from the credential and the semantic broadcast name without the epoch. The epoch is an input to every HKDF derivation, so a restart, takeover, or explicit group sequence cannot repeat a nonce under a key: nothing is persisted across instances and no generation counter is redistributed. The epoch is the shared [`Epoch`](/quest/m1/epoch.md) primitive. Subscribers discover instances under the `<opaque>/` prefix and take the greatest epoch, which sorts newest; a known full path carries its epoch in the last segment.
- The epoch is untrusted and unauthenticated. A wrong epoch fails authentication and a withheld one denies service; neither can make a nonce repeat, because only the publisher instance chooses what it encrypts under. This is the same trust a cache needs to serve the right instance. Plaintext broadcasts adopt the same epoch by default through [broadcast epochs](/quest/m1/broadcast-epoch/README.md).
- Nothing in the path says the bytes are encrypted. The format after decryption (`meeting.hang`) is inside the opaque name; a plaintext player, exporter, or matcher that opens a protected broadcast finds no catalog it can read and fails with its usual typed refusal, the same as for any format it does not support.
- One 32-byte secret authorizes the whole broadcast; HKDF-SHA-256 derives separate AES-128-GCM keys for each physical track and for grouped-frame versus datagram domains. A grouped frame uses the 96-bit nonce `uint64_be(group) || uint32_be(frame)`; a datagram uses its sequence with frame zero under the datagram domain. Empty AAD: every immutable end-to-end field is in the HKDF info or the nonce.
- Within an instance, group and datagram sequences are allocated monotonically and frames are numbered in write order, which is the whole reuse rule. There is no ciphertext retention or retransmission API: relays forward bytes unchanged, and an application that needs to resend encrypts nothing twice because it never gets the same identity twice.
Expand Down Expand Up @@ -67,4 +67,5 @@ The Rust and TypeScript cores expose the same surface, and nothing else:

## Related

- [Epoch primitive](/quest/m1/epoch.md) - the shared `Epoch` type the protected path ends in
- [archive](/quest/m1/archive/README.md) - protected broadcasts are deliberately outside recording and replay formats
6 changes: 5 additions & 1 deletion quest/m1/e2ee/typescript.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,13 @@ the Rust crate name for name, without putting keys or crypto policy in `@moq/net

## Plan

- Add the package above `@moq/net`. `Credential` accepts a 32-byte secret or a nonextractable HKDF `CryptoKey` with `deriveBits` usage, rejects extractable keys, and never serializes the secret. `credential.path(semantic)` returns the opaque broadcast name and `credential.generation(epoch)` returns a `Generation` with `name()`, `produce()`, `consume()`, and a `mint()` that returns a lowercase UUIDv7 from `Date.now()` plus `crypto.getRandomValues`.
- Add the package above `@moq/net`. `Credential` accepts a 32-byte secret or a nonextractable HKDF `CryptoKey` with `deriveBits` usage, rejects extractable keys, and never serializes the secret. `credential.path(semantic)` returns the opaque broadcast name and `credential.generation(epoch)` returns a `Generation` with `name()`, `produce()`, `consume()`, and epochs come from the shared `@moq/net` `Epoch`.
- Mirror the Rust modules: `Track.Producer`, `Track.Consumer`, `Group.Producer`, `Group.Consumer`, `Datagram.Event`, and a `Failure` whose `code` is the draft's typed set. No catalog helpers, no exported constants beyond the profile limits an application sizes payloads with, no stateless `protect`/`open` with caller-chosen identities, and no test-only methods on public classes.
- WebCrypto AES-GCM is async, so each producer and consumer runs a serial pump: one AEAD call in flight, completions in submit order, a bounded waiter queue that throws when full. Depth one keeps frame ordinals equal to the inner `@moq/net` write order without reservation sets; raise it only when a measured capture burst needs it. Propagate backpressure, cancellation, and authentication failure explicitly; never reorder media or fall back to plaintext.
- Identity state is two counters per key (invocations, plaintext bytes) and one monotonic sequence per track. Check payload size before taking a frame ordinal so a rejected write never burns an identity. Datagram receivers keep a 1024-bit sliding bitmask marked only after a successful open; failed opens count against the key; a grouped authentication failure closes the ordered track.
- Pass every vector in `drafts/moq-e2ee-00.json` against [draft-lcurley-moq-e2ee](/drafts/draft-lcurley-moq-e2ee.md), then cover pump ordering, saturation, and cancellation, protected payload ceilings, monotonic allocation, per-key exhaustion, bounded datagram suppression, bad-group termination, bad-datagram events, and a new epoch authenticating while the old keys do not. Exercise grouped tracks on both transports and datagrams on moq-lite via `insertDatagram`.
- Add the package to `doc/lib/js/index.md` and the workspace; leave it unpublished until the browser components consume it.

## Required

- [Epoch primitive](/quest/m1/epoch.md) - the shared `Epoch` this package mints and binds
41 changes: 41 additions & 0 deletions quest/m1/epoch.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# [M] Broadcast epoch primitive

## Goal

`moq-net` and `@moq/net` own one `Epoch` type: a publisher instance's
identity, carried as the last path segment `@<uuidv7>` (for example
`demo/BBB.hang/@0192a3f0-...`). It mints, parses, orders (newest greatest),
and reports its wall-clock time. A path splits into its epoch-free name and
an optional epoch. The epoch rides in the path, so it survives any
moq-transport relay with no wire change.

This is the shared primitive the [broadcast epochs](/quest/m1/broadcast-epoch/README.md)
line builds on, and it replaces the e2ee-local `moq_e2ee::Epoch`.

## Plan

- Move `rs/moq-e2ee/src/epoch.rs` into `moq-net` and add the JS equivalent.
Parsing is strict: `@` followed by a lowercase hyphenated UUIDv7. Nothing else
counts as an epoch, so an app's own UUID segments never parse as one.
Comment on lines +17 to +19

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Classify UUID-shaped path collisions as breaking

The new strict-parser wording addresses @alice, but it still reinterprets every existing trailing @<lowercase UUIDv7> segment as epoch metadata. Such strings are currently valid opaque path segments; after this change, publishing one skips minting a fresh epoch and consuming one pins rather than follows the literal broadcast. The API therefore is not purely additive: either choose a non-colliding representation or classify this reserved shape as a breaking path-contract change and target dev.

AGENTS.md reference: AGENTS.md:L58-L60

Useful? React with 👍 / 👎.

- Path helpers split a path into name and epoch and join them back. A segment
like `@alice` is valid today and stays valid: strict parsing already keeps it
from reading as an epoch. Rejecting it instead would break the path contract
and land on `dev`. Check how the split interacts with
[path patterns](/quest/m1/path-patterns.md) and
[hidden broadcasts](/quest/m1/hidden-broadcasts.md) (a leading `.`).
- `moq-e2ee` uses the shared type. Update
[draft-lcurley-moq-e2ee](/drafts/draft-lcurley-moq-e2ee.md) so the path is
`<opaque>/@<epoch>`. Keep the HKDF input as the UUID text, so vectors change
only if they encode the path. Drop "deliberately e2ee-only" from the e2ee
README.
- Golden cross-language vectors for parse, reject, order, and time, beside
the existing path tests.
- Nothing mints by default here. Publish and consume behavior is
[origin](/quest/m1/broadcast-epoch/origin.md).

Public API: additive `Epoch` and path helpers in both languages. The
`moq_e2ee::Epoch` move is a break on an unpublished crate. Wire: none.

## Related

- [E2EE](/quest/m1/e2ee/README.md) - the first consumer, whose TypeScript core requires this
2 changes: 2 additions & 0 deletions quest/m1/wildcard/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -271,3 +271,5 @@ than announce state.
pattern, and its catalog names the generations a wildcard cannot
- [pop-skipping](/quest/m1/pop-skipping/README.md) - it owns the route cost and
the rank hash this reuses
- [Broadcast epochs](/quest/m1/broadcast-epoch/README.md) - derived output moves under the
source's `@<epoch>` segment, which the suffix patterns still match
Loading
Loading