Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
5d4a7f5
chore(quest): claim quest/m0/hls-contributions.md
kixelated Nov 4, 2026
3401416
feat(hls): list contribution renditions with lazy media demand
kixelated Nov 4, 2026
1cbc370
Merge remote-tracking branch 'origin/main' into quest/m0/hls-contribu…
kixelated Nov 4, 2026
def0540
feat(hls): serve contribution catalogs and request-owned media
kixelated Nov 4, 2026
f903cd4
chore(hls): sort the added dependency
kixelated Nov 4, 2026
cee33f2
bench(hls): run contribution fanout measurements nightly
kixelated Nov 4, 2026
066ce77
fix(hls): finish lazy discovery and rendition handovers
kixelated Nov 4, 2026
45c5666
fix(hls): retain contribution failure diagnostics without HTTP
kixelated Nov 4, 2026
f97f700
Merge remote-tracking branch 'origin/main' into quest/m0/hls-contribu…
kixelated Nov 4, 2026
237e167
fix(hls): end contributions with their parent and keep ended playlist…
kixelated Nov 4, 2026
e0d9add
Merge remote-tracking branch 'origin/main' into quest/m0/hls-contribu…
kixelated Nov 4, 2026
f692a9c
test(net): skip the flaky wider-fill cases
pullfrog[bot] Nov 4, 2026
5cd8e2b
feat(hls): segment contributions on the root's grid and wait for thei…
kixelated Nov 4, 2026
0462d24
Merge remote-tracking branch 'origin/main' into quest/m0/hls-contribu…
kixelated Nov 4, 2026
f423877
fix(hls): satisfy clippy with and without the server feature
kixelated Nov 4, 2026
eb5333e
fix(hls): avoid a closed-set lookup deadlock and idle past the playli…
kixelated Nov 4, 2026
e7f14c3
Merge remote-tracking branch 'origin/main' into quest/m0/hls-contribu…
kixelated Nov 4, 2026
d6d44c1
Merge remote-tracking branch 'origin/quest/m0/hls-contributions' into…
kixelated Nov 4, 2026
54fd167
Merge remote-tracking branch 'origin/main' into quest/m0/hls-contribu…
kixelated Nov 4, 2026
d210d5a
fix(hls): address review threads on contributions
kixelated Nov 4, 2026
747c5fd
fix(hls,net): retry refusals on any route change, keep contributions …
kixelated Nov 4, 2026
fce20a1
Merge remote-tracking branch 'origin/main' into quest/m0/hls-contribu…
kixelated Nov 4, 2026
abfde93
fix(net): don't re-ask a refusal through a closed instance's handover
kixelated Nov 4, 2026
451d6dd
Merge remote-tracking branch 'origin/main' into quest/m0/hls-contribu…
kixelated Nov 4, 2026
c0b76df
fix(hls): list contributions only from resolved bindings, answer when…
kixelated Nov 4, 2026
9ae830b
Merge remote-tracking branch 'origin/main' into quest/m0/hls-contribu…
kixelated Nov 4, 2026
e991206
Merge remote-tracking branch 'origin/main' into quest/m0/hls-contribu…
kixelated Nov 4, 2026
c84cf08
Merge remote-tracking branch 'origin/main' into quest/m0/hls-contribu…
kixelated Nov 4, 2026
4a452d0
fix(quest): point the include quest at the HLS listing PR
pullfrog[bot] Nov 4, 2026
7da1567
Merge remote-tracking branch 'origin/main' into quest/m0/hls-contribu…
kixelated Nov 4, 2026
ec6d106
Merge remote-tracking branch 'origin/main' into quest/m0/hls-contribu…
kixelated Nov 4, 2026
42c5d27
fix(quest): unblock the refused-follower quest this PR satisfies
kixelated Nov 4, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ hang = { version = "0.21.11", path = "rs/hang" }
hashbrown = { version = "0.17.1", default-features = false }
hashlink = "0.12.2"
hex = "0.4"
http-body = "1.0.1"
humantime = "2.3"
humantime-serde = "1.1.1"
hyper = "1.11"
Expand Down
11 changes: 10 additions & 1 deletion doc/bin/hls.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ What that means for you:

- **The publisher declares the interval.** Only a broadcast whose catalog carries `media.join.interval` gets playlists. Rust publishers built on `moq-mux` (the CLI, the RTMP, SRT, and WebRTC gateways, and the native bindings) declare it: an encoder that knows its GOP passes it as a hint, and an ingest without one measures each track over its first second or group, whichever is longer, then declares the longest. `@moq/publish` declares its keyframe interval.
- **Every rendition shares the grid.** A group spans from its first frame's timestamp to the next group's and belongs to the first grid line it contains, or, containing none, the line it starts after. Segment N therefore starts at the group containing N times the segment length, every rendition numbers its segments alike, and each cuts its boundaries at its own keyframes. A grid line no group starts a segment at, because a GOP ran past it or the content paused, is listed as an `EXT-X-GAP` so the numbering never skips: on a 2 s grid, a GOP from 0 to 7 s is segment 0, segments 1 and 2 are gaps, and the next GOP starts segment 3. Timestamps that go back start a discontinuity on that rendition alone; `EXT-X-PROGRAM-DATE-TIME`, written after every discontinuity when the catalog has a clock, lets players resync.
- **Idle renditions cost nothing**, and a segment is servable for as long as the relay's [cache](/bin/relay/config#cache) retains it. Keep `--window` (the playlist length, default 16 s) within that retention and within the media's max age (30 s by default), after which a superseded segment's groups are stale.
- **Contribution media starts on selection.** The HTTP multivariant playlist reads each root `contributions` entry's catalog and lists its audio and video beside the broadcast's own. A contribution shares the broadcast's timeline: its renditions are segmented on the broadcast's join interval and clock, whatever its own catalog declares, so publish it on the same timestamps. One not yet routed is listed once its broadcast is announced. Reading its catalog starts no encoding: only a request to one of its rendition URLs subscribes to that rendition's media. Demand lasts through active responses and, after the last one finishes, for three target durations or the playlist window plus one target duration, whichever is longer (20 s by default); a later request starts it again. The broadcast's own renditions stay subscribed. A segment is servable for as long as the relay's [cache](/bin/relay/config#cache) retains it. Keep `--window` (the playlist length, default 16 s) within that retention and within the media's max age (30 s by default), after which a superseded segment's groups are stale.
- **Live joins cost the window.** The exporter subscribes 30 s behind the live edge and lists at most 256 segments. One joining partway through a segment lists nothing before the first segment it sees start.
- **Segments are timed by the grid.** A segment is listed once a later one starts or the track ends. It starts at its first group's timestamp, or at its line when that group starts after it (as one resuming from a pause does), or a gap at its line, and lasts until the next segment starts, so a segment's bytes never change with a catalog estimate. A GOP longer than the segment is listed at its first line, so its duration understates its media.
- **The target duration is twice the segment.** Every media playlist keeps one `EXT-X-TARGETDURATION` while its rendition stays in the catalog, even across a reconfigure: twice the segment length, rounded up to whole seconds (twice the interval for a GOP of a second or more, 2 s for 20 ms audio groups), since the GOP containing a grid line starts at most one interval before it, and a gap ends under two segment lengths after its line. Only a final GOP longer than that is listed past it, against RFC 8216's rule: hls.js plays it, but Apple's `mediastreamvalidator` flags it, so keep keyframe intervals within the declared interval.
Expand All @@ -65,6 +65,15 @@ What that means for you:
- **URLs name the stream.** Every URL the multivariant playlist and DASH manifest list sits under `@<instance>`, the stream's instance, so every edge and every reload derive the same URLs and a CDN in front of export caches them. A restarted publisher gets URLs of its own, whose numbering starts over. The multivariant playlist and manifest switch to it at once, while the old stream's playlists end at the segments they listed before the switch, even if its publisher keeps sending; a player already on it plays those out and then has to reload the multivariant playlist. A stream that reached export over moq-lite-06 or older, or moq-transport, has an instance its session minted, so only that export serves its URLs.
- **Only a clean finish ends a playlist.** `EXT-X-ENDLIST` and a `static` DASH manifest mean the media tracks finished, or, on a replaced stream's media playlists, that another publisher took the broadcast over. A rendition whose media track fails instead, such as a publisher dropping its session, answers HTTP 500 with `Cache-Control: no-store`, so a truncated recording never looks complete, and leaves the multivariant playlist and DASH manifest so players switch to a sibling. With none left, the DASH manifest answers the error too.

Contribution URLs nest under the served broadcast's (see
[`Contribution`](https://docs.rs/moq-hls/latest/moq_hls/export/struct.Contribution.html)), so
access middleware sees the parent broadcast's path, while the server resolves each
[contribution](/draft/moq-hang#contributions) through its own scoped origin and leaves out
one it may not read. They follow contribution restarts unless the entry pins an `instance`;
an unavailable contribution leaves the multivariant playlist until it returns. DASH and
`--output` include only the broadcast's own renditions; record a contribution as its own
broadcast.

A [broadcast archive](/concept/archives) is served the same
way, with no second stored copy: record a broadcast, republish it, and export
the republished broadcast. Through this project's publishers and relays,
Expand Down
6 changes: 5 additions & 1 deletion drafts/draft-lcurley-moq-hang.md
Original file line number Diff line number Diff line change
Expand Up @@ -244,7 +244,10 @@ Each key is a contribution ID.
Each entry MUST contain a string `broadcast`, resolved against the catalog's broadcast as in {{field-broadcast}}, and MAY contain an `instance` as in {{field-instance}}.
A consumer MUST reject a catalog with a malformed entry or a reference that escapes above the root.

Listing an entry MUST NOT subscribe to the contribution: a consumer opens its catalog only when it selects the contribution or one of its renditions, so a contribution produced on demand starts only for a viewer that wants it.
Listing an entry MUST NOT subscribe to the contribution: a consumer opens its catalog only when it selects the contribution or one of its renditions.
An HLS master selects every contribution it lists, reading their catalogs to describe their renditions, but MUST NOT subscribe to their media until requested by a player.
It segments a contribution's renditions on the listing catalog's `media` ({{media}}), whatever the contribution's own catalog declares.
A publisher producing media on demand MUST answer its catalog without starting media production.
A selected contribution resolves like a rendition's reference, under the same rules for `instance` and restarts ({{field-instance}}).
A contribution produced on demand has no stream until it resolves, so a publisher omits its `instance` until then.

Expand Down Expand Up @@ -1125,6 +1128,7 @@ This document has no IANA actions.
- Added the optional video `reorder` field: the most frames that can precede one in decode order and follow it in presentation, fixed for the track once declared. A consumer that authors decode times derives them from it and the current group alone, refusing deeper reordering; without it, a consumer infers them from timestamps.
- Added an optional `instance` beside every rendition and data track `broadcast` reference ({{field-instance}}). A consumer reads each referenced path from the first broadcast it resolves for a catalog instance; a present `instance` fails a rendition that resolves another stream, and is best effort against an Instance a receiver minted.
- Added the optional root `contributions` map ({{contributions}}): whole broadcasts named by a relative `broadcast` and an optional `instance`, opened only on selection, whose rendition IDs stay their own.
- An HLS master selects the contributions it lists by reading their catalogs, without demanding media, and segments them on the listing catalog's `media`; an on-demand publisher answers its catalog without starting media production.
- Added optional `stats` and `echo` catalog sections, publisher/viewer snapshots keyed by rendition ID, and the `.echo` feedback broadcast convention.
- An Opus frame timestamp counts the OpusHead pre-skip: a consumer presents each decoded sample the pre-skip earlier than its frame timestamp places it.

Expand Down
5 changes: 3 additions & 2 deletions js/hang/src/catalog/contribution.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,9 @@ import { InstanceSchema, RelativeBroadcastSchema } from "./path";
* A whole broadcast whose own catalog describes its renditions, listed without opening it.
*
* Listing one subscribes to nothing: a consumer opens the contribution's catalog only when it
* selects one of its renditions. Its rendition IDs belong to that broadcast, never the listing
* catalog.
* selects it, including an HLS master listing its renditions. An on-demand publisher answers
* the catalog without starting media. Its rendition IDs belong to that broadcast, never the
* listing catalog.
*/
export const ContributionSchema = z.object({
// The contribution's broadcast, relative to the broadcast serving this catalog.
Expand Down
1 change: 0 additions & 1 deletion quest/m0/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,5 +36,4 @@ edge, which embeds moq-relay; it moved there on 2026-09-28.
- [A waiter's record limit changes no behavior](/quest/m0/kio-waiter-records.md) - two moq-net tests stop failing when kio's waiters remember 16 lists, and the limit becomes a tuning knob worth 3.9% per delivered group
- [Backport the relay and auth hardening to release](/quest/m0/backport-release-hardening.md) - `release`'s relay refuses dot-segment paths, serves its HTTP routes with main's response hardening, and withdraws advertisements a peer starves of send credit; its auth client refuses redirects
- [Agent build output can't fill the desktop's disk](/quest/m0/agent-disk-budget.md) - finished agent checkouts reclaim their mbx targets automatically, so parallel agent builds stop filling the root disk
- [HLS lists contributions](/quest/m0/hls-contributions.md) - an HLS master lists each catalog contribution's renditions, and a contribution's media is subscribed only while players fetch it
- [@moq/net's WebSocket receive windows match Rust](/quest/m0/qmux-receive-windows.md) - WebSocket JS throughput stops being capped below Rust's
95 changes: 0 additions & 95 deletions quest/m0/hls-contributions.md

This file was deleted.

6 changes: 3 additions & 3 deletions quest/m1/catalog-include.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,12 +30,12 @@ the catalog root. Its why I wanted to call the section `include` originally."
- The field arrived in [moq-dev/moq#7312](https://github.com/moq-dev/moq/pull/7312)
(2026-11-01) and isn't on `release`, so no compatibility alias is needed and
the catalog `version` stays.
- Update the [HLS contributions](/quest/m0/hls-contributions.md) quest, or its
code if #7836 has landed, to the new name.
- Update the HLS listing's code, landed in
[moq-dev/moq#7836](https://github.com/moq-dev/moq/pull/7836), to the new name.

Public API: breaking but unreleased (Rust `hang`, `@moq/hang`, `@moq/watch`).
Wire: the catalog JSON key changes; unreleased.

## Related

- [HLS lists contributions](/quest/m0/hls-contributions.md) - the HLS consumer settled with the shared-timeline rule
- [HLS lists contributions](https://github.com/moq-dev/moq/pull/7836) - the HLS consumer settled with the shared-timeline rule
4 changes: 0 additions & 4 deletions quest/m2/follow-refused-retry.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,10 +54,6 @@ again.

Public API: none. Wire: none.

## Required

- [HLS lists a broadcast's contributions without starting their media](/quest/m0/hls-contributions.md) - its PR, [moq-dev/moq#7836](https://github.com/moq-dev/moq/pull/7836), adds the refused follower this extends

## Related

- [Listing and follow](/quest/m1/api-audit/route/events.md) - `follow` takes over instance changes from announce events
Expand Down
5 changes: 3 additions & 2 deletions rs/hang/src/catalog/contribution.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,9 @@ use serde::{Deserialize, Serialize};
/// A whole broadcast whose own catalog describes its renditions, listed without opening it.
///
/// Listing one subscribes to nothing: a consumer opens the contribution's catalog only when it
/// selects one of its renditions. Its rendition IDs belong to that broadcast, never the listing
/// catalog.
/// selects it, including an HLS master listing its renditions. An on-demand publisher answers
/// the catalog without starting media. Its rendition IDs belong to that broadcast, never the
/// listing catalog.
///
/// Marked `#[non_exhaustive]` so optional fields can be added later. External callers build one
/// with [`Contribution::new`].
Expand Down
9 changes: 8 additions & 1 deletion rs/moq-hls/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,14 @@ categories = ["multimedia", "network-programming", "web-programming"]

[features]
default = ["server"]
server = ["dep:axum"]
server = ["dep:axum", "dep:http-body"]

[dependencies]
# Only needed by the HTTP export server (gated by `server`).
axum = { workspace = true, optional = true }
bytes = { workspace = true }
hang = { workspace = true }
http-body = { workspace = true, optional = true }
humantime = { workspace = true }
kio = { workspace = true, features = ["runtime"] }
m3u8-rs = "6"
Expand All @@ -41,6 +42,7 @@ url = { workspace = true }

[dev-dependencies]
async-trait = { workspace = true }
criterion = { package = "bench-harness", path = "../bench-harness" }
futures = { workspace = true }
moq-archive = { path = "../moq-archive" }
moq-json = { path = "../moq-json" }
Expand All @@ -52,5 +54,10 @@ tower = { version = "0.5", features = ["util"] }
tracing-test = { workspace = true }
workspace-hack = { path = "../workspace-hack" }

[[bench]]
name = "contributions"
harness = false
required-features = ["server"]

[lints]
workspace = true
Loading
Loading