diff --git a/demo/web/src/index.ts b/demo/web/src/index.ts index 3701505bfae..55958c73391 100644 --- a/demo/web/src/index.ts +++ b/demo/web/src/index.ts @@ -79,6 +79,12 @@ const metaSignal = new Signals.Signal(undefined); // tile's follow it reactively. const relayUrl = new Signals.Signal(new URL(RELAY_URL)); +// This page's feedback name: a tile whose catalog solicits feedback publishes +// `/-.echo`, which `moq export echo` reads. Each tile takes its own name, +// since a feedback broadcast belongs to one catalog and two broadcasts may solicit the same path. +const VIEWER = `viewer-${crypto.randomUUID().slice(0, 8)}`; +let tileCount = 0; + // Discovery connection (the tiles each open their own connection internally). const connection = new Net.Connection({ url: relayUrl }); @@ -119,6 +125,7 @@ function createTile(name: string): WatchTile { // `broadcast` and decoders, so the shared inspector panel reflects the active tile. const watch = document.createElement("moq-watch") as MoqWatch; watch.name = name; + watch.echo = `${VIEWER}-${++tileCount}`; watch.muted = true; // unmuted only while active (see below) const canvas = document.createElement("canvas"); canvas.style.cssText = "width: 100%; height: auto;"; diff --git a/demo/web/src/publish.html b/demo/web/src/publish.html index 393eb02668b..bc131207374 100644 --- a/demo/web/src/publish.html +++ b/demo/web/src/publish.html @@ -29,7 +29,9 @@

MoQ Publish

cost of an extra encode + decode. --> - + + diff --git a/doc/concept/stats.md b/doc/concept/stats.md index f339cd6c765..1f022c7374a 100644 --- a/doc/concept/stats.md +++ b/doc/concept/stats.md @@ -36,7 +36,11 @@ measurement stays explicit, so it remains distinct from unavailable. [CLI](/bin/cli#reports-and-feedback), and `moq import --echo-path` solicits feedback. An import counts the frames and payload bytes it writes to each rendition, and video keyframes; an export counts what -it reads and records its newest arrival. See the +it reads and records its newest arrival. In the browser, `` +and `echo-path` do the same for the [publisher](/lib/js/publish#stats-and-feedback), +and `` reports what its [player](/lib/js/watch#feedback) +received, decoded, and played, with only the PROBE round-trip time as its +transport. See the [hang draft](/draft/moq-hang#media-stats) for track framing, field units, and the feedback convention. diff --git a/doc/lib/js/publish.md b/doc/lib/js/publish.md index 4d5119d6c6b..a14e533d6ac 100644 --- a/doc/lib/js/publish.md +++ b/doc/lib/js/publish.md @@ -35,6 +35,8 @@ WebCodecs, writes the catalog, and publishes a hang broadcast. | `preview` | What the nested element shows: the raw `source` (default), a decoded copy of the `encoded` stream to see what viewers get, or `none`. | | `backend` | How a `` preview is drawn; see [@moq/video](/lib/js/video). `auto` (default), `webgpu`, or `2d`. | | `announce` | When to advertise: once the `source` is live (default), or `always`. Nobody can see or subscribe to the broadcast until it is announced. To keep it out of discovery, use a hidden name with a `.`-prefixed segment; it is still subscribable by name. | +| `stats` | Publish a [stats track](/concept/stats#media-reports) named in the catalog. Off by default. | +| `echo-path` | Solicit viewer feedback under this path, relative to the broadcast as a rendition reference is: from `room/live`, `viewers` is `room/viewers`. Sets the catalog's `echo` section; absent solicits none. | Every attribute is also a reactive property, and property or UI changes reflect back into the attributes. An invalid value is refused rather than defaulted: the @@ -49,6 +51,23 @@ Capture failures, such as `NotAllowedError` when permission is refused, land in `el.sources.audio`, and `` shows them. A refused capture waits for a permission, device, or settings change instead of prompting again. +## Stats and feedback + +With `stats`, the catalog names a `stats.json` track that reports, about once a +second while someone subscribes, what each rendition has sent under its catalog +ID: frames, bytes, keyframes, frames skipped because the encoder was still +busy, and the target bitrate. Its `transport` carries only the PROBE round-trip +time, since browsers report no usable WebTransport statistics. The encoders' +`out.stats` and the stats panel read the same counters, and a custom producer +reports through its rendition's `stats` signal. The `Broadcast` takes the same +options as `stats`, `probe`, and `echo` inputs. + +`echo-path` only asks for feedback: a `` that reads the +catalog publishes `room/viewers/alice.echo`, and `moq export echo` or a dashboard +reads it there. Issue tokens that let viewers publish under that path. A path +that walks above the root from the broadcast name is left out of the catalog +with a warning, since every viewer would refuse it. + ## Encoding Video encodes while enabled, so its recent groups wait in the track's cache for diff --git a/doc/lib/js/watch.md b/doc/lib/js/watch.md index 465f73dda1e..4bfd2702924 100644 --- a/doc/lib/js/watch.md +++ b/doc/lib/js/watch.md @@ -39,6 +39,7 @@ in sync at the latency you ask for. | `backend` | How the canvas is drawn; see [@moq/video](/lib/js/video). `auto` (default), `webgpu`, or `2d`. | | `follow` | Follow the broadcast's announcements (default on): wait for one before subscribing, so a player can mount before the stream exists, and move to each restart. Off requests the name once. | | `catalog-format` | `hang`, `msf`, or `manual`; detected from the name by default. | +| `echo` | The viewer's name for [feedback](/concept/stats#media-reports), one path segment not starting with `.`. Once the catalog solicits feedback, the player publishes `/.echo`. Absent publishes none. | Every attribute is also a reactive property, and property or UI changes reflect back into the attributes. An invalid value is refused rather than defaulted: the @@ -84,6 +85,23 @@ name outside the origin's scope is an `error`. A restarted publisher at the same once, with a fresh catalog, and video, audio, and the clock start over on its timeline instead of waiting for it to catch up to the old one. +## Feedback + +A publisher solicits feedback with the catalog's `echo` section, such as +``. Given `echo="alice"`, the player resolves +that path against the broadcast and publishes `alice.echo` there, with one +`feedback.json` track reporting what each rendition was received and played +under its catalog ID: frames, bytes, decoded, late, decode errors, stalls and +stalled time, audio underruns, and the newest arrival, plus the PROBE +round-trip time. It writes about once a second while someone subscribes, and +follows the catalog: a removed or moved section retracts the broadcast. Each +player publishes its own broadcast, reporting only what it counted, so a second +player on one connection that resolves the same feedback path, for any catalog, +publishes nothing and warns until the first one leaves; give each player its own +name. The decoders' +`out.stats` and the stats panel read the same counters. The token must allow +publishing under the echo path. + ## Binding from a framework `import "@moq/watch/element"` registers `` when the module diff --git a/js/hang/api/index.api.md b/js/hang/api/index.api.md index b6bf37128f6..0cef8a39866 100644 --- a/js/hang/api/index.api.md +++ b/js/hang/api/index.api.md @@ -13,6 +13,7 @@ import { ZodMiniPipe } from "@zod/mini"; import { ZodMiniRecord } from "@zod/mini"; import { ZodMiniString } from "@zod/mini"; import { ZodMiniTransform } from "@zod/mini"; +import { ZodMiniType } from "@zod/mini"; import { infer } from "@zod/mini"; export * as Catalog from "@moq/hang/catalog"; @@ -160,6 +161,18 @@ export * as Hls from "@moq/hang/hls"; export * as Net from "@moq/net"; +export namespace Report { + export const INTERVAL: 1000; + export const PRIORITY: -128; + export class Writer { + constructor(broadcast: Net.Broadcast.Producer, name: string, schema?: ZodMiniType); + finish(): void; + readonly track: Net.Track.Producer; + update(value: T): void; + get wanted(): boolean; + } +} + export * as Signals from "@moq/signals"; export namespace Stats { diff --git a/js/hang/src/index.ts b/js/hang/src/index.ts index e614f079cad..7191432bc63 100644 --- a/js/hang/src/index.ts +++ b/js/hang/src/index.ts @@ -19,5 +19,7 @@ export * as Echo from "./echo"; export * as Health from "./health"; /** A rendition's HLS segment records, each segment's first group and pts. */ export * as Hls from "./hls"; +/** The compressed snapshot track a stats or feedback report is written to. */ +export * as Report from "./report"; /** Publisher media and transport snapshot schemas. */ export * as Stats from "./stats"; diff --git a/js/hang/src/report.ts b/js/hang/src/report.ts new file mode 100644 index 00000000000..7623034c67c --- /dev/null +++ b/js/hang/src/report.ts @@ -0,0 +1,63 @@ +/** + * The track a media report rides on: a publisher's stats or a viewer's feedback. + * @module + */ + +import * as Json from "@moq/json"; +import type * as Moq from "@moq/net"; +import type * as z from "@zod/mini"; + +/** How often a reporter writes, in milliseconds: about once a second, as the hang draft asks. */ +export const INTERVAL = 1000; + +/** The report track's priority: below every media track, so a report never delays the media it describes. */ +export const PRIORITY = -128; + +// Deltas compress against the snapshot in their group, and a joiner asks for a fresh group once +// they outweigh eight snapshots. Matches the Rust CLI's report tracks. +const REFRESH = 8; + +/** + * One compressed snapshot track, written only while someone subscribes, so an unobserved report + * never leaves the process. + */ +export class Writer { + /** The report track. */ + readonly track: Moq.Track.Producer; + + #producer: Json.Snapshot.Producer; + #closed = false; + + /** Create `name` on `broadcast`, validating each report against `schema` before it is written. */ + constructor(broadcast: Moq.Broadcast.Producer, name: string, schema?: z.ZodMiniType) { + this.track = broadcast.createTrack(name, { priority: PRIORITY, compress: true, refresh: REFRESH }); + this.#producer = new Json.Snapshot.Producer({ track: this.track, schema }); + } + + /** Whether anyone subscribes now, which is when {@link update} writes. Observe `track.demand().wanted` to follow it. */ + get wanted(): boolean { + return this.track.demand().wanted.peek() !== undefined; + } + + /** + * Publish `value` when someone subscribes; an unchanged value writes nothing. A track that refuses + * a write would refuse every later one, so it is aborted with the cause, which its subscribers read. + */ + update(value: T): void { + if (this.#closed || !this.wanted) return; + try { + this.#producer.update({ value }); + } catch (err) { + console.warn("a report track failed; closing it", err); + this.#closed = true; + this.track.abort(err instanceof Error ? err : new Error(String(err))); + } + } + + /** End the track cleanly, keeping the last report for a late subscriber. Idempotent. */ + finish(): void { + if (this.#closed) return; + this.#closed = true; + this.#producer.finish(); + } +} diff --git a/js/publish/README.md b/js/publish/README.md index f1f22f3d99a..c3655e7219a 100644 --- a/js/publish/README.md +++ b/js/publish/README.md @@ -72,6 +72,8 @@ The simplest way to publish a stream: | `preview` | string | `"source"` | What the preview renders: `"source"`, `"encoded"`, `"none"` | | `backend` | string | `"auto"` | Graphics API for a `` preview: `"auto"`, `"webgpu"`, `"2d"` | | `announce` | string | `"source"` | When to advertise: `"always"` or `"source"` (once media is captured). A camera source waits for every enabled track, and `"source"` waits until each captured track's config resolves or fails, so the first catalog lists every rendition. After its first announcement, a camera stays announced until deselected or disconnected, including mute, hide, device swaps, and capture retries. The broadcast is created while connected either way. To keep it out of discovery, use a hidden name with a `.`-prefixed segment; it is still subscribable by name. | +| `stats` | boolean | false | Publish a `stats.json` track, named in the catalog, reporting what each rendition sent and the PROBE round-trip time | +| `echo-path` | string | — | Solicit viewer feedback under this path, relative to the broadcast (from `room/live`, `viewers` is `room/viewers`) | Property and built-in UI changes reflect into attributes. Invalid URLs, version lists, and source/preview/backend/announce modes are refused: their property setters throw, and malformed markup diff --git a/js/publish/api/element.api.md b/js/publish/api/element.api.md index 54490b28255..8470d0b8f2b 100644 --- a/js/publish/api/element.api.md +++ b/js/publish/api/element.api.md @@ -17,7 +17,7 @@ export default class extends HTMLElement { constructor(); get announce(): AnnounceMode | undefined; set announce(value: AnnounceMode); - attributeChangedCallback(name: "source" | "url" | "announce" | "name" | "backend" | "versions" | "muted" | "invisible" | "preview", oldValue: string | null, newValue: string | null): void; + attributeChangedCallback(name: "source" | "url" | "stats" | "announce" | "name" | "backend" | "versions" | "muted" | "invisible" | "preview" | "echo-path", oldValue: string | null, newValue: string | null): void; audio: Audio.Encoder; get backend(): Preview.Backend; set backend(value: Preview.Backend); @@ -27,19 +27,23 @@ export default class extends HTMLElement { readonly controls: { announce: Signals.Signal; backend: Signals.Signal; + echoPath: Signals.Signal; invisible: Signals.Signal; muted: Signals.Signal; preview: Signals.Signal; source: Signals.Signal; + stats: Signals.Signal; }; disconnectedCallback(): void; + get echoPath(): Net.Path.Relative | undefined; + set echoPath(value: string | undefined); get invisible(): boolean; set invisible(value: boolean); get muted(): boolean; set muted(value: boolean); get name(): Net.Path.Valid; set name(value: string | Net.Path.Valid); - static observedAttributes: readonly ["url", "versions", "name", "muted", "invisible", "source", "preview", "backend", "announce"]; + static observedAttributes: readonly ["url", "versions", "name", "muted", "invisible", "source", "preview", "backend", "announce", "stats", "echo-path"]; get preview(): Preview.Mode; set preview(value: Preview.Mode); readonly signals: Signals.Effect; @@ -50,6 +54,8 @@ export default class extends HTMLElement { file: Signals.Signal; video: Signals.Signal; }>; + get stats(): boolean; + set stats(value: boolean); get url(): URL | undefined; set url(value: string | URL | undefined); get versions(): readonly string[] | undefined; diff --git a/js/publish/api/index.api.md b/js/publish/api/index.api.md index 496011d832f..1a6a58bed3d 100644 --- a/js/publish/api/index.api.md +++ b/js/publish/api/index.api.md @@ -91,6 +91,7 @@ export namespace Audio { track: Audio.StreamTrack; } export interface Stats { + bitrate?: number; bytes: number; frames: number; } @@ -117,6 +118,7 @@ export namespace Audio { export class Broadcast { constructor(props?: Signals.Inputs); static readonly CATALOG_TRACK: "catalog.json"; + static readonly STATS_TRACK: "stats.json"; audio(name: string): Rendition; readonly catalog: CatalogProducer; close(): void; @@ -132,6 +134,7 @@ export type BroadcastInput = { height: number; width: number; } | undefined>; + echo: Signals.Getter; enabled: Signals.Getter; flip: Signals.Getter; hls: Signals.Getter; @@ -139,7 +142,9 @@ export type BroadcastInput = { name: Signals.Getter; onDemand: Signals.Getter; origin: Signals.Getter; + probe: Signals.Getter; rotation: Signals.Getter; + stats: Signals.Getter; }; export class CatalogProducer { @@ -218,9 +223,18 @@ export class Rendition { readonly hlsHint: Signals.Signal; readonly kind: Kind; readonly name: string; + readonly stats: Signals.Signal; readonly track: Signals.Getter; } +export interface RenditionStats { + bitrate?: number; + bytes?: number; + frames?: number; + keyframes?: number; + skipped?: number; +} + export * as Signals from "@moq/signals"; export namespace Source { @@ -397,9 +411,11 @@ export namespace Video { track: Video.StreamTrack; } export interface Stats { + bitrate?: number; bytes: number; frames: number; keyframes: number; + skipped: number; } export interface StreamTrack extends MediaStreamTrack { clone(): Video.StreamTrack; diff --git a/js/publish/src/audio/capture.test.ts b/js/publish/src/audio/capture.test.ts index 330f295fc0d..a20806ef988 100644 --- a/js/publish/src/audio/capture.test.ts +++ b/js/publish/src/audio/capture.test.ts @@ -561,6 +561,7 @@ test("a failed encoder stays failed until the capture source changes", async () const rendition = { slot: new Signal(undefined), config: new Signal(undefined), + stats: new Signal(undefined), track: new Signal(track), hls: () => undefined, hlsHint: new Signal(undefined), diff --git a/js/publish/src/audio/encoder.test.ts b/js/publish/src/audio/encoder.test.ts index 513e9ee1e3a..a34e0407f55 100644 --- a/js/publish/src/audio/encoder.test.ts +++ b/js/publish/src/audio/encoder.test.ts @@ -293,6 +293,7 @@ async function setup(order = new Order(), codec?: Codec, groupDuration?: Milli) const rendition = { slot: new Signal(undefined), config: new Signal(undefined), + stats: new Signal(undefined), track: new Signal(track), hls: () => undefined, hlsHint: new Signal(undefined), diff --git a/js/publish/src/audio/encoder.ts b/js/publish/src/audio/encoder.ts index c824a0b0f12..1fbf77d4264 100644 --- a/js/publish/src/audio/encoder.ts +++ b/js/publish/src/audio/encoder.ts @@ -16,6 +16,7 @@ import type { Broadcast } from "../broadcast"; import { hlsHint, wanted, watched } from "../internal"; import { Estimator, type Order } from "../order"; import { Slot } from "../publication"; +import type { RenditionStats } from "../rendition"; import type { AudioFrame, Capture, Format } from "./capture"; import { Gain } from "./gain"; import { Resampler } from "./resampler"; @@ -83,6 +84,9 @@ export interface Stats { /** Total bytes encoded while serving. Monotonic; diff over an interval for an upload bitrate. */ bytes: number; + + /** The bitrate the encoder targets now, in bits per second, or undefined while none is resolved. */ + bitrate?: number; } /** Signals the encoder reads. */ @@ -130,7 +134,7 @@ export type EncoderOutput = { // True while encoding: enabled with a captured format and a track, which also takes a // subscriber to it or its HLS records unless the broadcast's `onDemand` is false. active: Signal; - // Cumulative output totals (frames, bytes) measured while serving. + // Cumulative output totals (frames, bytes) measured while serving, and the target bitrate. stats: Signal; }; @@ -303,6 +307,10 @@ export class Encoder { this.#signals.run(this.#runSettingsRecovery.bind(this)); this.#signals.run(this.#runCaptureRecovery.bind(this)); this.#signals.run(this.#runRegister.bind(this)); + this.#signals.run((effect) => { + const bitrate = effect.get(this.#config)?.encoder.bitrate; + this.#out.stats.update((stats) => ({ ...stats, bitrate })); + }); } // Pump PCM off the capture into whatever is currently publishing, applying the volume knobs on @@ -412,6 +420,7 @@ export class Encoder { // Publish the resolved config; undefined (no capture) drops it from the catalog. effect.proxy(rendition.config, this.out.catalog); + effect.proxy(rendition.stats, this.out.stats); // The pipeline outlives any one subscription: it is built as soon as capture runs and // #encode reads the live producer per frame rather than subscribing to it. Rebuilding on a @@ -683,6 +692,7 @@ export class Encoder { } this.#out.stats.update((stats) => ({ + ...stats, frames: stats.frames + 1, bytes: stats.bytes + frame.byteLength, })); diff --git a/js/publish/src/broadcast.test.ts b/js/publish/src/broadcast.test.ts index 5c4741078e0..eeaebeee3c4 100644 --- a/js/publish/src/broadcast.test.ts +++ b/js/publish/src/broadcast.test.ts @@ -1,5 +1,6 @@ import { expect, spyOn, test } from "bun:test"; import { Manual, Micro, Milli, Nano } from "@moq/clock"; +import { Stats } from "@moq/hang"; import * as Catalog from "@moq/hang/catalog"; import * as Container from "@moq/hang/container"; import * as Json from "@moq/json"; @@ -763,6 +764,112 @@ test("identity changes before the first announce take a fresh track name", async broadcast.close(); }); +test("stats names a track in the catalog and reports each listed rendition while subscribed", async () => { + const clock = new Manual(); + const installed = clock.install(); + const stats = new Signal(false); + const probe = new Signal({ rtt: Milli(12.7) }); + const broadcast = new Broadcast({ + enabled: true, + origin: new Origin.Producer(), + name: Path.from("test.hang"), + stats, + probe, + }); + try { + const video = broadcast.video("video"); + video.config.set(videoConfig); + video.stats.set({ frames: 3, bytes: 300, keyframes: 1, skipped: 2, bitrate: 1_000_000 }); + // A rendition without a config is not in the catalog, so it is not reported either. + broadcast.audio("audio").stats.set({ frames: 1 }); + await settle(); + expect((await readCatalog(broadcast))?.stats).toBeUndefined(); + + stats.set(true); + await settle(); + expect((await readCatalog(broadcast))?.stats).toEqual({ track: Broadcast.STATS_TRACK }); + + const net = broadcast.net.peek(); + if (!net) throw new Error("expected a network producer once connected"); + const subscriber = net.track(Broadcast.STATS_TRACK).subscribe(); + const consumer = new Json.Snapshot.Consumer({ + track: subscriber, + schema: Stats.SnapshotSchema, + }); + await settle(); + await clock.advance(Nano.fromMilli(Milli(1000))); + // Sub-millisecond durations are floored on the wire. + expect((await consumer.latest())?.value as unknown).toEqual({ + transport: { rtt: 12 }, + renditions: { video: { frames: 3, bytes: 300, keyframes: 1, skipped: 2, bitrate: 1_000_000 } }, + }); + subscriber.close(); + + stats.set(false); + await settle(); + expect((await readCatalog(broadcast))?.stats).toBeUndefined(); + } finally { + broadcast.close(); + installed.close(); + } +}); + +test("an unsubscribed stats track writes nothing", async () => { + const clock = new Manual(); + const installed = clock.install(); + const broadcast = new Broadcast({ + enabled: true, + origin: new Origin.Producer(), + name: Path.from("test.hang"), + stats: true, + }); + try { + const video = broadcast.video("video"); + video.config.set(videoConfig); + video.stats.set({ frames: 1 }); + await settle(); + await clock.advance(Nano.fromMilli(Milli(3000))); + + const net = broadcast.net.peek(); + if (!net) throw new Error("expected a network producer once connected"); + const subscriber = net.track(Broadcast.STATS_TRACK).subscribe(); + await settle(); + await clock.advance(Nano.fromMilli(Milli(1000))); + // Nothing was written while nobody subscribed, so the first report opens the first group. + expect((await subscriber.recvGroup())?.sequence).toBe(0); + subscriber.close(); + } finally { + broadcast.close(); + installed.close(); + } +}); + +test("echo solicits feedback in the catalog unless it walks above the root", async () => { + const echo = new Signal(Path.normalizeRelative("viewers")); + const name = new Signal(Path.from("room/live.hang")); + const broadcast = new Broadcast({ enabled: true, name, echo }); + try { + await settle(); + expect((await readCatalog(broadcast))?.echo as unknown).toEqual({ path: "viewers" }); + + // From `room/live.hang`, `../../viewers` walks above the root, which every viewer refuses. + echo.set(Path.normalizeRelative("../../viewers")); + await settle(); + expect((await readCatalog(broadcast))?.echo).toBeUndefined(); + + // A deeper name brings the same path back within the root. + name.set(Path.from("org/room/live.hang")); + await settle(); + expect((await readCatalog(broadcast))?.echo as unknown).toEqual({ path: "../../viewers" }); + + echo.set(undefined); + await settle(); + expect((await readCatalog(broadcast))?.echo).toBeUndefined(); + } finally { + broadcast.close(); + } +}); + test("each video and audio track publishes HLS records and advertises them once declared", async () => { const broadcast = new Broadcast({ enabled: true, origin: new Origin.Producer(), name: Path.from("hls.hang") }); const video = broadcast.video("video/hd"); diff --git a/js/publish/src/broadcast.ts b/js/publish/src/broadcast.ts index e0b046ebeeb..f43787edbe8 100644 --- a/js/publish/src/broadcast.ts +++ b/js/publish/src/broadcast.ts @@ -1,4 +1,5 @@ -import { Milli } from "@moq/clock"; +import { Milli, Nano } from "@moq/clock"; +import { Report, Stats } from "@moq/hang"; import * as Catalog from "@moq/hang/catalog"; import * as Container from "@moq/hang/container"; import * as Hls from "@moq/hang/hls"; @@ -9,7 +10,7 @@ import { CatalogProducer } from "./catalog"; import { wanted } from "./internal"; import { Order } from "./order"; import { type Identity, identity, type Listing, Publication, type Slot } from "./publication"; -import type { Kind, Rendition } from "./rendition"; +import type { Kind, Rendition, RenditionStats } from "./rendition"; import { makeRendition } from "./rendition-internal"; // The refresh ratio video tracks advertise: a joiner forces a keyframe once a group's deltas @@ -61,6 +62,27 @@ export type BroadcastInput = { */ maxAge: Getter; + /** + * Whether to publish a stats track and name it in the catalog's `stats` section. Defaults to + * false. The track reports each rendition's {@link Rendition.stats} under its ID, plus the + * connection's PROBE round-trip time, about once a second while someone subscribes. + */ + stats: Getter; + + /** + * The connection's PROBE estimates, whose round-trip time the stats track reports. Observed only + * while the stats track has a subscriber, since observing it asks the peer for PROBE updates. + */ + probe: Getter; + + /** + * Where viewers may publish feedback, relative to the broadcast as a rendition's `broadcast` + * reference is: from `room/live`, `viewers` is `room/viewers`. Sets the catalog's `echo` section, + * which solicits it; undefined (the default) solicits none. A path that walks above the root from + * the broadcast name is left out of the catalog with a warning, since every viewer would refuse it. + */ + echo: Getter; + /** * Override when renditions encode. Omitted: audio retains raw PCM until subscribed, while * video encodes continuously. False encodes both continuously; true gates video too, losing @@ -89,6 +111,9 @@ export class Broadcast { /** The catalog track name served to subscribers. */ static readonly CATALOG_TRACK = Catalog.TRACK; + /** The stats track name, named in the catalog's `stats` section while {@link BroadcastInput.stats} is set. */ + static readonly STATS_TRACK = "stats.json"; + /** The wired inputs, read-only. */ readonly in: Readonlys; @@ -152,11 +177,15 @@ export class Broadcast { flip: getter(props?.flip ?? false), rotation: getter(props?.rotation ?? 0), maxAge: getter(props?.maxAge), + stats: getter(props?.stats ?? false), + probe: getter(props?.probe), + echo: getter(props?.echo), onDemand: getter(props?.onDemand), hls: getter(props?.hls ?? true), }; this.#signals.run(this.#runCatalog.bind(this)); + this.#signals.run(this.#runEcho.bind(this)); this.#signals.run(this.#run.bind(this)); } @@ -319,6 +348,63 @@ export class Broadcast { if (publication) for (const [rendition, track, media] of served) publication.commit(rendition, track, media); } + // Solicit feedback in the catalog's `echo` section. A path above the root names no broadcast, and a + // viewer refuses the whole catalog for it, so it is left out instead. + #runEcho(effect: Effect) { + const path = effect.get(this.in.echo); + if (path === undefined) return; + const name = effect.get(this.in.name); + if (Moq.Path.tryResolve(name, path) === undefined) { + console.warn( + `not soliciting feedback: echo path ${JSON.stringify(path)} walks above the root from ${name}`, + ); + return; + } + this.catalog.mutate((catalog) => { + catalog.echo = { path }; + }); + effect.cleanup(() => + this.catalog.mutate((catalog) => { + delete catalog.echo; + }), + ); + } + + // Name the stats track in the catalog and write a report every interval while someone subscribes. + // The track lives as long as the network broadcast, so disabling stats only stops reporting. + #runStats(effect: Effect, report: () => Report.Writer) { + if (!effect.get(this.in.stats)) return; + const writer = report(); + + this.catalog.mutate((catalog) => { + catalog.stats = { track: Broadcast.STATS_TRACK }; + }); + effect.cleanup(() => + this.catalog.mutate((catalog) => { + delete catalog.stats; + }), + ); + + // Observing the probe asks the peer for PROBE updates, so only do it while someone reads them. + const watched = effect.computed((demand) => demand.get(writer.track.demand().wanted) !== undefined); + effect.run((demand) => { + if (demand.get(watched)) demand.get(this.in.probe); + }); + + effect.interval(() => writer.update(this.#stats() as Stats.Snapshot), Nano.fromMilli(Milli(Report.INTERVAL))); + } + + // What each listed rendition has sent, keyed by its catalog ID, and the connection's round-trip time. + #stats(): { transport?: { rtt: number }; renditions: Record } { + const renditions: Record = {}; + for (const rendition of Object.values(this.#renditions.peek())) { + const stats = rendition.stats.peek(); + if (stats && rendition.config.peek() !== undefined) renditions[rendition.name] = stats; + } + const rtt = this.in.probe.peek()?.rtt; + return rtt === undefined ? { renditions } : { transport: { rtt: Math.floor(rtt) }, renditions }; + } + // The names outside a publication's own that a replacement must avoid: every registered ID, and // every ID and track name in the catalog, including an application's own sections. #reserved(): Set { @@ -382,6 +468,15 @@ export class Broadcast { if (this.#publication.peek() === publication) this.#publication.set(undefined); }); + // Created on first use and kept with the broadcast, like a rendition's track. + let stats: Report.Writer | undefined; + const report = () => { + stats ??= new Report.Writer(broadcast, Broadcast.STATS_TRACK, Stats.SnapshotSchema); + return stats; + }; + effect.cleanup(() => stats?.finish()); + effect.run((inner) => this.#runStats(inner, report)); + let serving = false; effect.run((inner) => { if (!inner.get(this.in.announce)) { diff --git a/js/publish/src/element.ts b/js/publish/src/element.ts index fe2940119e0..68bc497ef3b 100644 --- a/js/publish/src/element.ts +++ b/js/publish/src/element.ts @@ -39,7 +39,19 @@ function parseVersions(value: string | null): string[] | undefined { } /** The attributes the element observes. */ -const OBSERVED = ["url", "versions", "name", "muted", "invisible", "source", "preview", "backend", "announce"] as const; +const OBSERVED = [ + "url", + "versions", + "name", + "muted", + "invisible", + "source", + "preview", + "backend", + "announce", + "stats", + "echo-path", +] as const; /** The built-in capture sources selectable via the `source` attribute. */ export type SourceType = "camera" | "screen" | "file"; @@ -83,6 +95,11 @@ function parsePreview(value: string | null): Preview.Mode { throw invalidAttribute("preview", value, '"encoded", "source", or "none"'); } +// Parse the `echo-path` attribute: a path relative to the broadcast, or undefined when absent. +function parseEchoPath(value: string | null): Moq.Path.Relative | undefined { + return value === null ? undefined : Moq.Path.normalizeRelative(value); +} + function parseBackend(value: string | null): Preview.Backend { if (value === null) return "auto"; const trimmed = value.trim(); @@ -111,6 +128,10 @@ export default class MoqPublish extends HTMLElement { // When to advertise the broadcast: always, or only once a source is live. Undefined while the // `announce` attribute is invalid, which never announces. announce: new Signal("source"), + // Whether to publish a stats track named in the catalog. + stats: new Signal(false), + // Where viewers may publish feedback, relative to the broadcast, or undefined to solicit none. + echoPath: new Signal(undefined), }; /** @@ -215,6 +236,9 @@ export default class MoqPublish extends HTMLElement { display: this.#capture.out.display, flip: this.#flip, rotation: this.#capture.out.rotation, + stats: this.controls.stats, + probe: this.connection.probe, + echo: this.controls.echoPath, }); this.signals.cleanup(() => this.broadcast.close()); @@ -335,6 +359,8 @@ export default class MoqPublish extends HTMLElement { this.signals.run((effect) => this.#reflect(effect, "preview", effect.get(this.controls.preview))); this.signals.run((effect) => this.#reflect(effect, "backend", effect.get(this.controls.backend))); this.signals.run((effect) => this.#reflect(effect, "announce", effect.get(this.controls.announce))); + this.signals.run((effect) => this.#reflect(effect, "stats", effect.get(this.controls.stats) ? "" : undefined)); + this.signals.run((effect) => this.#reflect(effect, "echo-path", effect.get(this.controls.echoPath))); this.signals.run(this.#runSource.bind(this)); } @@ -364,6 +390,8 @@ export default class MoqPublish extends HTMLElement { else if (name === "invisible") this.invisible = parseBoolean(newValue, false); else if (name === "preview") this.preview = parsePreview(newValue); else if (name === "backend") this.backend = parseBackend(newValue); + else if (name === "stats") this.stats = parseBoolean(newValue, false); + else if (name === "echo-path") this.echoPath = parseEchoPath(newValue); else { const exhaustive: never = name; throw new Error(`Invalid attribute: ${exhaustive}`); @@ -590,6 +618,34 @@ export default class MoqPublish extends HTMLElement { this.controls.announce.set(parseAnnounce(value)); this.#accept("announce"); } + + /** + * Whether to publish a stats track, mirrored by the `stats` attribute. The catalog names it in its + * `stats` section, and it reports what each rendition has sent plus the connection's PROBE + * round-trip time, about once a second while someone subscribes. + */ + get stats(): boolean { + return this.controls.stats.peek(); + } + + set stats(value: boolean) { + this.controls.stats.set(value); + this.#accept("stats"); + } + + /** + * Where viewers may publish feedback, relative to the broadcast as a rendition reference is (from + * `room/live`, `viewers` is `room/viewers`), mirrored by the `echo-path` attribute. Sets the + * catalog's `echo` section, so a `` publishes its feedback there. Undefined solicits none. + */ + get echoPath(): Moq.Path.Relative | undefined { + return this.controls.echoPath.peek(); + } + + set echoPath(value: string | undefined) { + this.controls.echoPath.set(parseEchoPath(value ?? null)); + this.#accept("echo-path"); + } } customElements.define("moq-publish", MoqPublish); diff --git a/js/publish/src/rendition-internal.ts b/js/publish/src/rendition-internal.ts index fad0053b62c..b66ef0d886a 100644 --- a/js/publish/src/rendition-internal.ts +++ b/js/publish/src/rendition-internal.ts @@ -7,6 +7,23 @@ import type { Slot } from "./publication"; /** The media kind of a rendition, selecting its catalog section (`video`, `audio`, or `text`). */ export type Kind = "video" | "audio" | "text"; +/** + * What a rendition's producer has sent, as the broadcast's stats track reports it. Counters are + * cumulative and a missing one means zero; `bitrate` is a gauge, absent when unknown. + */ +export interface RenditionStats { + /** Media frames sent. */ + frames?: number; + /** Media payload bytes sent. */ + bytes?: number; + /** Keyframes sent. */ + keyframes?: number; + /** Frames skipped before sending, such as those an overloaded encoder had no room for. */ + skipped?: number; + /** The latest target encoder bitrate in bits per second. */ + bitrate?: number; +} + /** @internal Register a handle whose track and teardown belong to its broadcast. */ export let makeRendition: ( name: string, @@ -48,6 +65,12 @@ export class Rendition { */ readonly hlsHint = new Signal(undefined); + /** + * What the producer has sent under this rendition, written by the producer and reported under + * its ID on the broadcast's stats track. `undefined` leaves the rendition out of the report. + */ + readonly stats = new Signal(undefined); + /** * The open track producer, or `undefined` while the network broadcast is down, after the track * closes, and, for non-audio tracks with the broadcast's `onDemand`, while no subscriber is attached diff --git a/js/publish/src/rendition.ts b/js/publish/src/rendition.ts index 597c59a3b40..04664179591 100644 --- a/js/publish/src/rendition.ts +++ b/js/publish/src/rendition.ts @@ -1,2 +1,2 @@ /** Registered media tracks owned by a broadcast. @module */ -export { type Kind, Rendition } from "./rendition-internal"; +export { type Kind, Rendition, type RenditionStats } from "./rendition-internal"; diff --git a/js/publish/src/ui/components/stats-tab.ts b/js/publish/src/ui/components/stats-tab.ts index f20cf1d0fc0..44872a228a0 100644 --- a/js/publish/src/ui/components/stats-tab.ts +++ b/js/publish/src/ui/components/stats-tab.ts @@ -25,6 +25,9 @@ export function statsTab(parent: Effect, publish: MoqPublish): HTMLElement { const videoCard = statCard("video", "Video", videoIcon); const vRes = statLine(videoCard.grid, "Resolution"); const vCodec = statLine(videoCard.grid, "Codec"); + // The same encoder stats the broadcast's stats track reports. + const vTarget = statLine(videoCard.grid, "Target"); + const vSkipped = statLine(videoCard.grid, "Skipped"); const vBitrateGraph = graph(parent, "Bitrate", { color: "#a855f7", format: formatBitrate }); const vFpsGraph = graph(parent, "Frame rate", { color: "#facc15", format: formatFps }); videoCard.el.append(vBitrateGraph.el, vFpsGraph.el); @@ -119,6 +122,10 @@ export function statsTab(parent: Effect, publish: MoqPublish): HTMLElement { prevFrames = frames; vFpsGraph.push(elapsed > 0 && frameDelta > 0 ? frameDelta / (elapsed / 1000) : undefined); + const stats = publish.video.out.stats.peek(); + vTarget.textContent = stats.bitrate !== undefined ? formatBitrate(stats.bitrate) : "—"; + vSkipped.textContent = `${stats.skipped}`; + const bytes = encoded(); const byteDelta = bytes - prevBytes; prevBytes = bytes; diff --git a/js/publish/src/video/capture.test.ts b/js/publish/src/video/capture.test.ts index cf8bdb8e347..002522bb18b 100644 --- a/js/publish/src/video/capture.test.ts +++ b/js/publish/src/video/capture.test.ts @@ -460,6 +460,7 @@ test("an encoder resuming on a still source encodes a keyframe of the current pi const rendition = { slot: new Signal(undefined), config: new Signal(undefined), + stats: new Signal(undefined), track: live, hls: () => undefined, hlsHint: new Signal(undefined), @@ -574,6 +575,7 @@ test("an encoder torn down while a frame is in flight closes that frame", async const rendition = { slot: new Signal(undefined), config: new Signal(undefined), + stats: new Signal(undefined), track: live, hls: () => undefined, hlsHint: new Signal(undefined), diff --git a/js/publish/src/video/encoder-stall.test.ts b/js/publish/src/video/encoder-stall.test.ts index 454a5593c47..d1f9eee1566 100644 --- a/js/publish/src/video/encoder-stall.test.ts +++ b/js/publish/src/video/encoder-stall.test.ts @@ -112,6 +112,7 @@ test("a slower-than-realtime encoder drops input instead of ratcheting catalog l const rendition = { slot: new Signal(undefined), config: new Signal(undefined), + stats: new Signal(undefined), track: new Signal(track), hls: () => undefined, hlsHint: new Signal(undefined), diff --git a/js/publish/src/video/encoder.test.ts b/js/publish/src/video/encoder.test.ts index fd0288c8010..ac833acaf6c 100644 --- a/js/publish/src/video/encoder.test.ts +++ b/js/publish/src/video/encoder.test.ts @@ -126,6 +126,7 @@ test("encoding tracks encoder config in its child effect", async () => { const rendition = { slot: new Signal(undefined), config: new Signal(undefined), + stats: new Signal(undefined), track: new Signal(track), hls: () => undefined, hlsHint: new Signal(undefined), @@ -170,6 +171,7 @@ test("an on-demand gap marks a discontinuity and leaves the broadcast-owned trac const rendition = { slot: new Signal(undefined), config: new Signal(undefined), + stats: new Signal(undefined), track: live, hls: () => undefined, hlsHint: new Signal(undefined), @@ -227,6 +229,7 @@ test("a bandwidth estimate updates the bitrate without blanking the config or re const rendition = { slot: new Signal(undefined), config: new Signal(undefined), + stats: new Signal(undefined), track: new Signal(track), hls: () => undefined, hlsHint: new Signal(undefined), @@ -315,6 +318,7 @@ test("every published config fits a probe for its own codec", async () => { const rendition = { slot: new Signal(undefined), config: new Signal(undefined), + stats: new Signal(undefined), track: new Signal(track), hls: () => undefined, hlsHint: new Signal(undefined), @@ -670,6 +674,7 @@ test("disabling encodes one black keyframe and keeps the rendition with enabled: const rendition = { slot: new Signal(undefined), config: new Signal(undefined), + stats: new Signal(undefined), track: new Signal(track), hls: () => undefined, hlsHint: new Signal(undefined), @@ -840,6 +845,7 @@ test("the probe encodes with the frame rate and bitrate ceiling", async () => { const rendition = { slot: new Signal(undefined), config: new Signal(undefined), + stats: new Signal(undefined), track: new Signal(track), hls: () => undefined, hlsHint: new Signal(undefined), @@ -1050,6 +1056,7 @@ async function recordKeyframes( const rendition = { slot: new Signal(undefined), config: new Signal(undefined), + stats: new Signal(undefined), track: new Signal(track), hls: () => undefined, hlsHint: new Signal(undefined), @@ -1512,6 +1519,7 @@ test("a shrinking capture resizes in band below the advertised ceiling", async ( const rendition = { slot: new Signal(undefined), config: new Signal(undefined), + stats: new Signal(undefined), track: new Signal(track), hls: () => undefined, hlsHint: new Signal(undefined), @@ -1587,6 +1595,7 @@ test("the catalog bitrate and frame rate only rise", async () => { const rendition = { slot: new Signal(undefined), config: new Signal(undefined), + stats: new Signal(undefined), track: new Signal(track), hls: () => undefined, hlsHint: new Signal(undefined), @@ -1648,6 +1657,7 @@ test("each slot keeps its own ceiling and maxima", async () => { const slot = new Signal(new Slot()); const rendition = () => ({ config: new Signal(undefined), + stats: new Signal(undefined), track: new Signal(undefined), slot, hls: () => undefined, diff --git a/js/publish/src/video/encoder.ts b/js/publish/src/video/encoder.ts index 5eee7e40d44..5c3fd3e6b83 100644 --- a/js/publish/src/video/encoder.ts +++ b/js/publish/src/video/encoder.ts @@ -17,6 +17,7 @@ import type { Broadcast } from "../broadcast"; import { hlsHint, watched } from "../internal"; import { Estimator, type Order } from "../order"; import { Slot } from "../publication"; +import type { RenditionStats } from "../rendition"; import { hardwareReliable } from "../support/video"; import type { Capture } from "./capture"; import { normalizeSource, type Source } from "./types"; @@ -34,6 +35,12 @@ export interface Stats { /** Total keyframes encoded while serving. Divide frames by this for the average GOP length. */ keyframes: number; + + /** Total captured frames skipped while serving because the encoder still had a frame queued. */ + skipped: number; + + /** The bitrate the encoder targets now, in bits per second, or undefined while none is resolved. */ + bitrate?: number; } // TODO support signals? @@ -116,7 +123,8 @@ export type EncoderOutput = { // True while encoding: enabled and the rendition has a track, which with the broadcast's // `onDemand` also takes a subscriber. active: Signal; - // Cumulative output totals (frames, bytes, keyframes) measured while serving. + // Cumulative output totals (frames, bytes, keyframes, skipped) measured while serving, and the + // target bitrate. stats: Signal; }; @@ -158,7 +166,7 @@ export class Encoder { catalog: new Signal(undefined), resolved: new Signal(undefined), active: new Signal(false), - stats: new Signal({ frames: 0, bytes: 0, keyframes: 0 }), + stats: new Signal({ frames: 0, bytes: 0, keyframes: 0, skipped: 0 }), }; /** The derived outputs, read-only. */ readonly out = readonlys(this.#out); @@ -278,6 +286,10 @@ export class Encoder { this.#signals.run(this.#runSettingsRecovery.bind(this)); this.#signals.run(this.#runCaptureRecovery.bind(this)); this.#signals.run(this.#runRegister.bind(this)); + this.#signals.run((effect) => { + const bitrate = effect.get(this.#out.resolved)?.bitrate; + this.#out.stats.update((stats) => ({ ...stats, bitrate })); + }); } // The current slot, read through `effect`. @@ -347,6 +359,7 @@ export class Encoder { // Publish the resolved catalog config; undefined (until one resolves) leaves it out of the catalog. effect.proxy(rendition.config, this.out.catalog); + effect.proxy(rendition.stats, this.out.stats); // Encode while enabled and the broadcast hands over the track (with `onDemand`, a subscriber is attached). effect.run((effect) => { @@ -431,6 +444,7 @@ export class Encoder { } this.#out.stats.update((stats) => ({ + ...stats, frames: stats.frames + 1, bytes: stats.bytes + frame.byteLength, keyframes: key ? stats.keyframes + 1 : stats.keyframes, @@ -496,7 +510,10 @@ export class Encoder { // A pending input means the codec is saturated. Keep capture live instead of // growing a stale encode queue whose lateness becomes permanent catalog delay. // Drop before answering a keyframe request or advancing the frame-rate cursor. - if (encoder.encodeQueueSize > 0) continue; + if (encoder.encodeQueueSize > 0) { + this.#out.stats.update((stats) => ({ ...stats, skipped: stats.skipped + 1 })); + continue; + } // This doesn't need to be reactive. const config = this.config.peek(); diff --git a/js/watch/README.md b/js/watch/README.md index 8a9111f6416..75e6eb483c0 100644 --- a/js/watch/README.md +++ b/js/watch/README.md @@ -76,9 +76,10 @@ The simplest way to watch a stream: | `buffer` | duration | `0ms` | Future-dated media held before playback skips ahead. | | `captions` | string | off | Text rendition to render. | | `catalog-format` | hang, msf, manual | auto-detected | The catalog format; detected from the name suffix unless set. | +| `echo` | path segment | — | The viewer's feedback name: once the catalog solicits feedback, publish `/.echo` reporting what each rendition received and played. | Property and built-in UI changes reflect into attributes. Invalid URLs, numbers, durations, -version lists, and rendering/catalog modes are refused: their property setters throw, and malformed markup +version lists, echo names, and rendering/catalog modes are refused: their property setters throw, and malformed markup reports a browser `error` event and suspends media until corrected or removed. Defaults apply when attributes are absent. diff --git a/js/watch/api/element.api.md b/js/watch/api/element.api.md index e3ea8a99964..0d0977889c8 100644 --- a/js/watch/api/element.api.md +++ b/js/watch/api/element.api.md @@ -16,7 +16,7 @@ import { Video } from "@moq/watch"; export default class extends HTMLElement { constructor(); - attributeChangedCallback(name: "delay" | "buffer" | "url" | "name" | "follow" | "volume" | "muted" | "paused" | "visible" | "backend" | "versions" | "announced" | "catalog-format" | "captions", oldValue: string | null, newValue: string | null): void; + attributeChangedCallback(name: "delay" | "buffer" | "url" | "echo" | "name" | "follow" | "volume" | "muted" | "paused" | "visible" | "backend" | "versions" | "announced" | "catalog-format" | "captions", oldValue: string | null, newValue: string | null): void; readonly audio: Audio.Decoder; get backend(): Video.Backend; set backend(value: Video.Backend); @@ -52,6 +52,8 @@ export default class extends HTMLElement { get delay(): Delay; set delay(value: Delay); disconnectedCallback(): void; + get echo(): string | undefined; + set echo(value: string | undefined); emitter: Audio.Emitter; get follow(): boolean; set follow(value: boolean); @@ -59,7 +61,7 @@ export default class extends HTMLElement { set muted(value: boolean); get name(): Net.Path.Valid; set name(value: string | Net.Path.Valid); - static observedAttributes: readonly ["url", "versions", "name", "paused", "volume", "muted", "visible", "backend", "follow", "announced", "delay", "buffer", "catalog-format", "captions"]; + static observedAttributes: readonly ["url", "versions", "name", "paused", "volume", "muted", "visible", "backend", "follow", "announced", "delay", "buffer", "catalog-format", "captions", "echo"]; get paused(): boolean; set paused(value: boolean); readonly player: Player; diff --git a/js/watch/api/index.api.md b/js/watch/api/index.api.md index 1fb1c574df2..14e18975ac4 100644 --- a/js/watch/api/index.api.md +++ b/js/watch/api/index.api.md @@ -85,7 +85,15 @@ export namespace Audio { track: Signals.Signal; }; export interface Stats { + arrival?: { + timestamp: Clock.Milli; + wall: Clock.Wall; + }; bytesReceived: number; + errors: number; + late: number; + stalled: Clock.Milli; + stalls: number; underruns: number; } export type Supported = (config: Hang.Catalog.Audio.AudioConfig) => Promise; @@ -311,8 +319,16 @@ export namespace Video { track: Signals.Signal; }; export interface Stats { + arrival?: { + timestamp: Clock.Milli; + wall: Clock.Wall; + }; bytesReceived: number; + errors: number; frameCount: number; + late: number; + stalled: Clock.Milli; + stalls: number; } export type Supported = (config: Hang.Catalog.Video.VideoConfig) => Promise; export type Target = { diff --git a/js/watch/src/audio/decoder.test.ts b/js/watch/src/audio/decoder.test.ts index f4fbfb6116b..a3962d2f6c3 100644 --- a/js/watch/src/audio/decoder.test.ts +++ b/js/watch/src/audio/decoder.test.ts @@ -318,7 +318,7 @@ async function play(initial: Delay, buffered = false, cmaf = false) { // The catalog broadcast's instance, identified by its `closed` handle. const active = new Signal({ closed: {} }); const broadcast = new Signal({ - in: { enabled }, + in: { enabled, name: new Signal(Moq.Path.from("watched")) }, out: { catalog, active }, relativeBroadcast, } as unknown as Broadcast); @@ -1069,7 +1069,7 @@ async function livePlayback(burst: number) { let producer = new Moq.Broadcast.Producer(); const media = new Signal(producer.consume()); const broadcast = new Signal({ - in: { enabled: new Signal(true) }, + in: { enabled: new Signal(true), name: new Signal(Moq.Path.from("watched")) }, out: { catalog, active: new Signal({ closed: {} }) }, relativeBroadcast: (effect: Effect) => effect.get(media), } as unknown as Broadcast); @@ -1461,7 +1461,7 @@ it("a rejected audio support probe does not leave buffered video waiting on a pe const logged = spyOn(console, "warn").mockImplementation(() => {}); const source = new Source({ broadcast: { - in: { enabled: new Signal(true) }, + in: { enabled: new Signal(true), name: new Signal(Moq.Path.from("watched")) }, out: { catalog: new Signal({ audio: { diff --git a/js/watch/src/audio/decoder.ts b/js/watch/src/audio/decoder.ts index 921ed8aafc1..c69ad26b844 100644 --- a/js/watch/src/audio/decoder.ts +++ b/js/watch/src/audio/decoder.ts @@ -1,4 +1,4 @@ -import { Micro, Milli, type Second } from "@moq/clock"; +import { Micro, Milli, type Second, type Wall } from "@moq/clock"; import * as Catalog from "@moq/hang/catalog"; import * as Container from "@moq/hang/container"; import * as Util from "@moq/hang/util"; @@ -16,6 +16,7 @@ import { import { Producer as ArrivalProducer, type Arrivals, gap } from "../arrivals"; import { hostedAssets } from "../assets"; import { base64ToBytes } from "../base64"; +import { arrivedAt, type Counter, type Key, type Received, Tally } from "../feedback"; import { type Failure, failureMoved, nextMedia, subscribeMedia } from "../media"; import { supportCacheKey } from "../support-probe"; import type { Sync } from "../sync"; @@ -95,8 +96,24 @@ export interface Stats { bytesReceived: number; /** How many times playback ran dry mid-stream; a rising count means the delay is too low. */ underruns: number; + /** Frames that arrived after their playout time. */ + late: number; + /** Renditions that failed to decode. */ + errors: number; + /** Times playback stalled waiting for audio once it had played. */ + stalls: number; + /** Total time spent in finished stalls. */ + stalled: Milli; + /** The newest received media timestamp and the wall time it arrived. */ + arrival?: { timestamp: Milli; wall: Wall }; } +// The running totals before anything is counted. +const EMPTY: Stats = { bytesReceived: 0, underruns: 0, late: 0, errors: 0, stalls: 0, stalled: Milli.zero }; + +// What a decode attempt reads, and where its counts go. +type Attempt = { config: DecoderConfig; failure: Signal; path: Moq.Path.Valid; key: Key }; + /** * Downloads audio from a track and emits it to an AudioContext. * @@ -168,6 +185,13 @@ export class Decoder { #signals = new Effect(); #failed?: Failure; + // What each rendition delivered, for the feedback a viewer reports. + #tally = new Tally(); + // The rendition decoding into the ring, which an underrun or a stall is counted against. + #playing?: Key; + // Whether audio played since the decoder was enabled, so the wait for the first is no stall. + #played = false; + // The catalog fields that require a replacement subscription or decoder. readonly #identity: Computed; @@ -224,6 +248,56 @@ export class Decoder { this.#signals.run(this.#runEnabled.bind(this)); this.#signals.run(this.#runLatency.bind(this)); this.#signals.run(this.#runDecoder.bind(this)); + this.#signals.run(this.#runStalls.bind(this)); + } + + /** + * What this decoder received and played of each rendition in the catalog of the broadcast at + * `path`, keyed by rendition ID, for the feedback a viewer reports. + * @internal + */ + feedback(path: Moq.Path.Valid): Record { + return this.#tally.report(path); + } + + // Count into a rendition's feedback and the decoder's running totals. + #count(key: Key | undefined, counts: Partial>, totals?: (stats: Stats) => Partial) { + if (key) this.#tally.count(key, counts); + if (totals) this.#out.stats.update((current) => ({ ...(current ?? EMPTY), ...totals(current ?? EMPTY) })); + } + + // Count a received frame and whether it arrived too late for playout. + #receive(key: Key, frame: Container.ReceivedFrame, timestamp: Milli, late: boolean): void { + const bytes = frame.payload.byteLength; + const arrived = arrivedAt(frame.arrival); + this.#tally.arrive(key, timestamp, arrived); + this.#count(key, { frames: 1, bytes, late: late ? 1 : 0 }, (stats) => ({ + bytesReceived: stats.bytesReceived + bytes, + late: stats.late + (late ? 1 : 0), + arrival: { timestamp, wall: arrived }, + })); + } + + // Count each stall once audio has played, and how long it lasted. A disabled decoder waits for + // audio again: resuming is a startup, not a stall. + #runStalls(effect: Effect): void { + if (!effect.get(this.in.enabled)) { + this.#played = false; + return; + } + if (!effect.get(this.#out.stalled)) { + this.#played = true; + return; + } + const key = this.#playing; + if (!this.#played || !key) return; + + const end = this.#tally.stall(key); + this.#count(undefined, {}, (stats) => ({ stalls: stats.stalls + 1 })); + effect.cleanup(() => { + const duration = end(); + this.#count(undefined, {}, (stats) => ({ stalled: Milli(stats.stalled + duration) })); + }); } // A group the relay expires is never observed, so a subscription cut to the target would cap the @@ -330,10 +404,7 @@ export class Decoder { const added = underruns - counted; counted = underruns; if (added <= 0) return; - this.#out.stats.update((stats) => ({ - bytesReceived: stats?.bytesReceived ?? 0, - underruns: (stats?.underruns ?? 0) + added, - })); + this.#count(this.#playing, { underruns: added }, (stats) => ({ underruns: stats.underruns + added })); }); effect.set(this.#out.root, worklet); @@ -420,10 +491,13 @@ export class Decoder { } // Closing this effect releases both its container reader and track subscription. const failure = new Signal(undefined); + const rendition: Key = { path: broadcast.in.name.peek(), id: track }; + this.#playing = rendition; let attempt: Effect | undefined; effect.run((inner) => { const error = inner.get(failure); if (!error) return; + this.#count(rendition, { errors: 1 }, (stats) => ({ errors: stats.errors + 1 })); this.#failed = { broadcast: broadcast.out.active.peek()?.closed, media: active.closed, @@ -468,7 +542,7 @@ export class Decoder { // A config the probe accepted can still fail to parse (a malformed init segment or Opus // header). Report it like a decode failure, which also retires the outgoing estimate. - const props = { config, failure, path: active.path }; + const props: Attempt = { config, failure, path: active.path, key: rendition }; spawnFatal(run, failure, async () => { if (config.container.kind === "cmaf") await this.#runCmafDecoder(run, sub, props); else this.#runLegacyDecoder(run, sub, props); @@ -476,11 +550,7 @@ export class Decoder { }); } - #runLegacyDecoder( - effect: Effect, - sub: Moq.Track.Subscriber, - { config, failure, path }: { config: DecoderConfig; failure: Signal; path: Moq.Path.Valid }, - ): void { + #runLegacyDecoder(effect: Effect, sub: Moq.Track.Subscriber, { config, failure, path, key }: Attempt): void { const preSkip = config.codec === "opus" && config.description ? Util.Opus.preSkip(Util.Hex.toBytes(config.description)) : 0; this.#terminal.clear(preSkip, Micro.fromMilli(Milli(config.warmup ?? 0))); @@ -495,7 +565,10 @@ export class Decoder { if (!loaded) return; // cancelled const decoder = new AudioDecoder({ - output: (data) => this.#emit(data), + output: (data) => { + this.#count(key, { decoded: 1 }); + this.#emit(data); + }, error: (error) => failure.set(error), }); effect.cleanup(() => { @@ -546,10 +619,7 @@ export class Decoder { const duration = packetDuration(config.codec, frame); if (duration !== undefined) this.#frame.set(duration); - this.#out.stats.update((stats) => ({ - bytesReceived: (stats?.bytesReceived ?? 0) + frame.payload.byteLength, - underruns: stats?.underruns ?? 0, - })); + this.#receive(key, frame, timestamp, late); if (!(await play(order.push(next)))) return; } @@ -559,7 +629,7 @@ export class Decoder { async #runCmafDecoder( effect: Effect, sub: Moq.Track.Subscriber, - { config, failure, path }: { config: DecoderConfig; failure: Signal; path: Moq.Path.Valid }, + { config, failure, path, key }: Attempt, ): Promise { if (config.container.kind !== "cmaf") return; // just to help typescript @@ -588,7 +658,10 @@ export class Decoder { if (!loaded) return; // cancelled const decoder = new AudioDecoder({ - output: (data) => this.#emit(data), + output: (data) => { + this.#count(key, { decoded: 1 }); + this.#emit(data); + }, error: (error) => failure.set(error), }); effect.cleanup(() => { @@ -632,10 +705,7 @@ export class Decoder { const duration = packetDuration(config.codec, frame); if (duration !== undefined) this.#frame.set(duration); - this.#out.stats.update((stats) => ({ - bytesReceived: (stats?.bytesReceived ?? 0) + frame.payload.byteLength, - underruns: stats?.underruns ?? 0, - })); + this.#receive(key, frame, timestamp, late); if (!(await play(order.push(next)))) return; } diff --git a/js/watch/src/echo.test.ts b/js/watch/src/echo.test.ts new file mode 100644 index 00000000000..dfb6af5bdb7 --- /dev/null +++ b/js/watch/src/echo.test.ts @@ -0,0 +1,192 @@ +import { expect, test } from "bun:test"; +import { Manual, Milli, Nano, yieldNow } from "@moq/clock"; +import { Echo as Schema } from "@moq/hang"; +import type * as Catalog from "@moq/hang/catalog"; +import * as Json from "@moq/json"; +import { Origin, Path } from "@moq/net"; +import { Signal } from "@moq/signals"; +import { Broadcast } from "./broadcast"; +import { checkName, Echo } from "./echo"; +import type { Received } from "./feedback"; + +const flush = async (): Promise => { + for (let i = 0; i < 5; i++) await yieldNow(); +}; + +// A watched broadcast whose catalog the test sets directly. +function watched(origin: Origin.Producer, name: string, catalog: Signal): Broadcast { + return new Broadcast({ origin, name: Path.from(name), follow: false, catalogFormat: "manual", catalog }); +} + +// A decoder that reports `received` for the catalog at `path`, and nothing for any other. +function source(path: string, received: Record) { + return { feedback: (base: Path.Valid) => (base === path ? received : {}) }; +} + +// The feedback broadcasts announced on `origin`, by path. +function announced(origin: Origin.Producer): Set { + const paths = new Set(); + const listing = origin.announced(Path.Pattern.subtree(Path.empty())); + void (async () => { + for (;;) { + const entry = await listing.next(); + if (!entry) break; + if (entry.kind === "end") paths.delete(entry.prefix); + else paths.add(entry.prefix); + } + })(); + return paths; +} + +// Read the newest feedback report published at `path`. +async function read(origin: Origin.Producer, path: string, clock: Manual): Promise { + const request = origin.request(Path.from(path)); + await flush(); + const broadcast = request.active.peek(); + if (!broadcast) throw new Error(`no feedback at ${path}`); + const track = broadcast.track(Schema.TRACK).subscribe(); + const consumer = new Json.Snapshot.Consumer({ track, schema: Schema.SnapshotSchema }); + await flush(); + await clock.advance(Nano.fromMilli(Milli(1000))); + const value = (await consumer.latest())?.value; + track.close(); + request.close(); + return value; +} + +test("refuses a name that is not one visible path segment", () => { + for (const name of ["", "a/b", ".", "..", "*", "**", ".hidden", "a\\b", "alice\\"]) { + expect(() => checkName(name)).toThrow(); + } + expect(checkName("alice")).toBe("alice"); + expect(checkName("a\\\\b")).toBe("a\\\\b"); +}); + +test("publishes feedback where the catalog solicits it and follows the solicitation", async () => { + const clock = new Manual(); + const installed = clock.install(); + const origin = new Origin.Producer(); + const paths = announced(origin); + const catalog = new Signal({}); + const broadcast = watched(origin, "room/live.hang", catalog); + const echo = new Echo({ + broadcast, + origin, + name: "alice", + probe: { rtt: Milli(20) }, + sources: [ + source("room/live.hang", { video: { frames: 2, bytes: 20 } }), + source("room/live.hang", { audio: { underruns: 1 } }), + ], + }); + try { + await flush(); + // Nothing is solicited, so nothing is published. + expect([...paths]).toEqual([]); + + catalog.set({ echo: { path: "viewers" } } as Catalog.Root); + await flush(); + expect([...paths]).toEqual(["room/viewers/alice.echo"]); + expect((await read(origin, "room/viewers/alice.echo", clock)) as unknown).toEqual({ + transport: { rtt: 20 }, + renditions: { video: { frames: 2, bytes: 20 }, audio: { underruns: 1 } }, + }); + + // A moved path retracts the broadcast it replaces. + catalog.set({ echo: { path: "live/viewers" } } as Catalog.Root); + await flush(); + expect([...paths]).toEqual(["room/live/viewers/alice.echo"]); + + // A removed section retracts it. + catalog.set({}); + await flush(); + expect([...paths]).toEqual([]); + } finally { + echo.close(); + broadcast.close(); + origin.close(); + installed.close(); + } +}); + +test("a second reporter at a feedback path in use publishes nothing until the holder leaves", async () => { + const clock = new Manual(); + const installed = clock.install(); + const origin = new Origin.Producer(); + const paths = announced(origin); + const catalog = new Signal({ echo: { path: "viewers" } } as Catalog.Root); + const first = watched(origin, "room/live.hang", catalog); + const second = watched(origin, "room/live.hang", catalog); + const enabled = new Signal(true); + const reporters = [ + new Echo({ + broadcast: first, + origin, + name: "alice", + sources: [source("room/live.hang", { video: { frames: 1 } })], + }), + new Echo({ + broadcast: second, + origin, + enabled, + name: "alice", + sources: [source("room/live.hang", { video: { frames: 10 } })], + }), + ]; + try { + await flush(); + expect([...paths]).toEqual(["room/viewers/alice.echo"]); + // Only the first reporter's own counts, never a sum. + expect((await read(origin, "room/viewers/alice.echo", clock)) as unknown).toEqual({ + renditions: { video: { frames: 1 } }, + }); + + // The refused one leaving changes nothing. + enabled.set(false); + await flush(); + expect([...paths]).toEqual(["room/viewers/alice.echo"]); + expect((await read(origin, "room/viewers/alice.echo", clock))?.renditions?.video.frames as unknown).toBe(1); + + // The holder leaving frees the path for a waiting reporter, which takes it over. + enabled.set(true); + await flush(); + reporters[0].close(); + await flush(); + expect([...paths]).toEqual(["room/viewers/alice.echo"]); + expect((await read(origin, "room/viewers/alice.echo", clock))?.renditions?.video.frames as unknown).toBe(10); + } finally { + for (const reporter of reporters) reporter.close(); + first.close(); + second.close(); + origin.close(); + installed.close(); + } +}); + +test("a reporter for another catalog at a path in use publishes nothing", async () => { + const clock = new Manual(); + const installed = clock.install(); + const origin = new Origin.Producer(); + const paths = announced(origin); + const catalog = new Signal({ echo: { path: "viewers" } } as Catalog.Root); + const a = watched(origin, "room/a.hang", catalog); + const b = watched(origin, "room/b.hang", catalog); + const reporters = [ + new Echo({ broadcast: a, origin, name: "alice", sources: [source("room/a.hang", { video: { frames: 2 } })] }), + new Echo({ broadcast: b, origin, name: "alice", sources: [source("room/b.hang", { video: { frames: 100 } })] }), + ]; + try { + await flush(); + expect([...paths]).toEqual(["room/viewers/alice.echo"]); + // Only the first catalog's counts, never a sum across catalogs. + expect((await read(origin, "room/viewers/alice.echo", clock)) as unknown).toEqual({ + renditions: { video: { frames: 2 } }, + }); + } finally { + for (const reporter of reporters) reporter.close(); + a.close(); + b.close(); + origin.close(); + installed.close(); + } +}); diff --git a/js/watch/src/echo.ts b/js/watch/src/echo.ts new file mode 100644 index 00000000000..f2d177b2da0 --- /dev/null +++ b/js/watch/src/echo.ts @@ -0,0 +1,177 @@ +import { Milli, Nano } from "@moq/clock"; +import { Report, Echo as Schema } from "@moq/hang"; +import * as Moq from "@moq/net"; +import { Effect, type Getter, getter, type Inputs, type Readonlys, Signal } from "@moq/signals"; +import type { Broadcast } from "./broadcast"; +import type { Received } from "./feedback"; + +/** A viewer report as written: per-rendition counters and the connection's round-trip time. */ +type Snapshot = { transport?: { rtt: number }; renditions: Record }; + +/** Signals the reporter reads. */ +export type EchoInput = { + /** The origin to publish feedback into. */ + origin: Getter; + /** Whether to publish. Defaults to true. */ + enabled: Getter; + /** The viewer's name, one path segment: feedback is published at `/.echo`. Undefined publishes none. */ + name: Getter; + /** The connection's PROBE estimates, whose round-trip time the report carries while someone reads it. */ + probe: Getter; +}; + +/** What a decoder received of each rendition in a catalog, keyed by rendition ID. */ +export interface Feedback { + /** Report what was received of the renditions in the catalog of the broadcast at `path`. */ + feedback(path: Moq.Path.Valid): Record; +} + +/** Constructor properties for {@link Echo}. */ +export type EchoProps = Inputs & { + /** The watched broadcast, whose catalog may solicit feedback. */ + broadcast: Broadcast; + /** The decoders whose counters the report carries. */ + sources: Feedback[]; +}; + +/** + * Refuse an echo name that is not one visible path segment: empty, containing `/`, a dot segment + * or glob, or starting with `.`, which would hide the feedback from discovery. Matches the Rust CLI. + * A `\` must escape another `\` in the published name, as in any path. + */ +export function checkName(name: string): string { + if (name === "" || name.includes("/") || name === "." || name === ".." || name === "*" || name === "**") { + throw new Error(`echo name ${JSON.stringify(name)} is not a single path segment`); + } + if (name.startsWith(".")) { + throw new Error(`echo name ${JSON.stringify(name)} starts with \`.\`, which hides its feedback from discovery`); + } + try { + Moq.Path.from(`${name}${Schema.SUFFIX}`); + } catch (err) { + throw new Error(`echo name ${JSON.stringify(name)} is not a valid path segment`, { cause: err }); + } + return name; +} + +/** + * Publishes a viewer's feedback where a watched catalog solicits it. + * + * Once the catalog carries an `echo` section, this resolves its path against the broadcast, as a + * rendition's `broadcast` reference resolves, and publishes `.echo` there with one + * `feedback.json` snapshot track: what each rendition was received and played, keyed by its ID in + * that catalog, and the connection's PROBE round-trip time. Each catalog update reconciles it: a + * removed section, or a changed path, retracts the broadcast it replaces. + * + * Each reporter publishes its own broadcast, so a report is only ever its own decoders' counts. A + * second broadcast at a path would replace the first, so a reporter that resolves a feedback path + * another reporter on the same origin already publishes publishes nothing and warns until that one + * releases it: give it another name. + */ +export class Echo { + /** The wired inputs, read-only. */ + readonly in: Readonlys; + /** The watched broadcast. */ + readonly broadcast: Broadcast; + + readonly #sources: Feedback[]; + #signals = new Effect(); + + constructor(props: EchoProps) { + this.in = { + origin: getter(props.origin), + enabled: getter(props.enabled ?? true), + name: getter(props.name), + probe: getter(props.probe), + }; + this.broadcast = props.broadcast; + this.#sources = props.sources; + + // Derived on its own, so an ordinary catalog update that keeps the echo path keeps the broadcast. + const target = this.#signals.computed((effect) => { + if (!effect.get(this.in.enabled)) return undefined; + const origin = effect.get(this.in.origin); + const name = effect.get(this.in.name); + const echo = effect.get(this.broadcast.out.catalog)?.echo; + if (!origin || name === undefined || !echo) return undefined; + + const base = effect.get(this.broadcast.in.name); + // The catalog consumer already refuses a path above the root; this only narrows the type. + const prefix = Moq.Path.tryResolve(base, echo.path); + if (prefix === undefined) return undefined; + return { origin, base, path: Moq.Path.join(prefix, Moq.Path.from(`${checkName(name)}${Schema.SUFFIX}`)) }; + }); + + this.#signals.run((effect) => { + const current = effect.get(target); + if (!current) return; + const { origin, path, base } = current; + + let claims = claimed.get(origin); + if (!claims) { + claims = { paths: new Set(), released: new Signal(0) }; + claimed.set(origin, claims); + } + if (claims.paths.has(path)) { + // Try again whenever a reporter on this origin releases its path. + effect.get(claims.released); + console.warn( + "feedback path already in use on this connection; give this player another echo name", + path, + ); + return; + } + + let broadcast: Moq.Broadcast.Producer | undefined; + let writer: Report.Writer; + try { + const publication = origin.createBroadcast(path); + broadcast = publication.broadcast; + writer = new Report.Writer(broadcast, Schema.TRACK, Schema.SnapshotSchema); + // A fresh instance, so a reader sees a replaced feedback broadcast as a restart. + publication.announce(); + } catch (err) { + broadcast?.close(); + console.warn("failed to publish feedback", path, err); + return; + } + const published = broadcast; + + const claim = claims; + claim.paths.add(path); + const write = () => writer.update(this.#report(base) as Schema.Snapshot); + // Teardown runs last in, first out: write the final counts, then retract and release the path. + effect.cleanup(() => { + claim.paths.delete(path); + claim.released.update((n) => n + 1); + }); + effect.cleanup(() => published.close()); + effect.cleanup(() => writer.finish()); + effect.cleanup(write); + effect.interval(write, Nano.fromMilli(Milli(Report.INTERVAL))); + + // Observing the probe asks the peer for PROBE updates, so only do it while someone reads them. + const demand = writer.track.demand(); + effect.run((inner) => { + if (inner.get(demand.wanted) !== undefined) inner.get(this.in.probe); + }); + }); + } + + // What the decoders received of the renditions in the catalog of the broadcast at `base`. + #report(base: Moq.Path.Valid): Snapshot { + const renditions: Record = {}; + for (const source of this.#sources) Object.assign(renditions, source.feedback(base)); + const rtt = this.in.probe.peek()?.rtt; + return rtt === undefined ? { renditions } : { transport: { rtt: Math.floor(rtt) }, renditions }; + } + + /** Stop publishing and release everything this holds. */ + close(): void { + this.#signals.close(); + } +} + +// The feedback paths published on each origin, so a second reporter never replaces the first's +// broadcast, and a count of releases, so a refused reporter tries again once one frees. +const claimed = new WeakMap; released: Signal }>(); diff --git a/js/watch/src/element.ts b/js/watch/src/element.ts index 4ec31b00af6..8c52f7d32e3 100644 --- a/js/watch/src/element.ts +++ b/js/watch/src/element.ts @@ -13,6 +13,7 @@ import { Derived, Effect, Signal } from "@moq/signals"; import type * as Audio from "./audio"; import { type Broadcast, CATALOG_FORMATS, type CatalogFormat } from "./broadcast"; import { formatDuration, parseDuration } from "./duration"; +import { checkName, Echo } from "./echo"; import { Player } from "./player"; import type { Delay, Sync } from "./sync"; import type * as Text from "./text"; @@ -65,6 +66,7 @@ const OBSERVED = [ "buffer", "catalog-format", "captions", + "echo", ] as const; // Parse visibility, with the default only when the attribute is absent. @@ -103,6 +105,16 @@ function parseBuffer(value: string | null): Milli { throw invalidAttribute("buffer", value, 'a duration like "30s"'); } +// Parse the `echo` attribute: the viewer's feedback name, one visible path segment, or undefined when absent. +function parseEcho(value: string | null): string | undefined { + if (value === null) return undefined; + try { + return checkName(value); + } catch { + throw invalidAttribute("echo", value, "one path segment not starting with `.`, any `\\` doubled"); + } +} + /** Parse the element's catalog-format attribute, refusing unknown formats. */ export function parseCatalogFormat(value: string | null): CatalogFormat | undefined { if (value === null) return undefined; @@ -195,6 +207,7 @@ export default class MoqWatch extends HTMLElement { #follow = new Signal(true); #catalogFormat = new Signal(undefined); #catalog = new Signal(undefined); + #echo = new Signal(undefined); // The canvas element to render into. #canvas = new Signal(undefined); @@ -255,6 +268,17 @@ export default class MoqWatch extends HTMLElement { text: { ...this.controls.text, container: this.#captionsOverlay }, }); this.signals.cleanup(() => this.player.close()); + + // Owned here rather than by the player, which only reads its origin: feedback publishes into it. + const echo = new Echo({ + broadcast: this.player.broadcast, + origin: this.connection.origin, + enabled: this.#enabled, + name: this.#echo, + probe: this.connection.probe, + sources: [this.player.video, this.player.audio], + }); + this.signals.cleanup(() => echo.close()); this.broadcast = this.player.broadcast; this.video = this.player.video; this.audio = this.player.audio; @@ -311,6 +335,7 @@ export default class MoqWatch extends HTMLElement { this.signals.run((effect) => this.#reflect(effect, "follow", effect.get(this.#follow) ? "" : "false")); this.signals.run((effect) => this.#reflect(effect, "catalog-format", effect.get(this.#catalogFormat))); this.signals.run((effect) => this.#reflect(effect, "captions", effect.get(this.controls.text.target))); + this.signals.run((effect) => this.#reflect(effect, "echo", effect.get(this.#echo))); this.signals.run((effect) => { const delay = effect.get(this.controls.delay); this.#reflect(effect, "delay", typeof delay === "number" ? formatDuration(delay) : delay); @@ -394,6 +419,7 @@ export default class MoqWatch extends HTMLElement { else if (name === "buffer") this.buffer = parseBuffer(newValue); else if (name === "catalog-format") this.catalogFormat = parseCatalogFormat(newValue); else if (name === "captions") this.captions = newValue || undefined; + else if (name === "echo") this.echo = parseEcho(newValue); else { const exhaustive: never = name; throw new Error(`Invalid attribute: ${exhaustive}`); @@ -602,6 +628,23 @@ export default class MoqWatch extends HTMLElement { this.controls.text.target.set(value || undefined); this.#accept("captions"); } + + /** + * The viewer's feedback name, mirrored by the `echo` attribute; undefined (the default) publishes + * no feedback. Once the watched catalog solicits feedback with an `echo` section, the element + * publishes `.echo` under its path, reporting what each rendition received and played. + * Each element publishes its own: a second element on one connection that resolves the same + * feedback path publishes nothing and warns until the first one leaves, so give each its own + * name. One visible path segment: a `/` or a leading `.` is refused. + */ + get echo(): string | undefined { + return this.#echo.peek(); + } + + set echo(value: string | undefined) { + this.#echo.set(parseEcho(value ?? null)); + this.#accept("echo"); + } } customElements.define("moq-watch", MoqWatch); diff --git a/js/watch/src/feedback.test.ts b/js/watch/src/feedback.test.ts new file mode 100644 index 00000000000..4e484250fd9 --- /dev/null +++ b/js/watch/src/feedback.test.ts @@ -0,0 +1,70 @@ +import { expect, test } from "bun:test"; +import { Manual, Milli, Nano, now, Wall, wall } from "@moq/clock"; +import * as Catalog from "@moq/hang/catalog"; +import { Path } from "@moq/net"; +import { arrivedAt, Tally } from "./feedback"; + +const live = { path: Path.from("room/live.hang"), id: "video" }; + +test("counts per rendition of each catalog broadcast, omitting zeros", () => { + const tally = new Tally(); + tally.count(live, { frames: 1, bytes: 10, late: 0 }); + tally.count(live, { frames: 1, bytes: 5 }); + tally.count({ ...live, id: "audio" }, { underruns: 2 }); + tally.count({ path: Path.from("other.hang"), id: "video" }, { frames: 7 }); + expect(tally.report(live.path)).toEqual({ video: { frames: 2, bytes: 15 }, audio: { underruns: 2 } }); + expect(tally.report(Path.from("unknown.hang"))).toEqual({}); +}); + +test("a stall reports its time so far and adds its duration once it ends", async () => { + const clock = new Manual(); + const installed = clock.install(); + try { + const tally = new Tally(); + const end = tally.stall(live); + await clock.advance(Nano.fromMilli(Milli(250.6))); + expect(tally.report(live.path).video).toEqual({ stalls: 1, stalled: 250 }); + await clock.advance(Nano.fromMilli(Milli(100))); + expect(end()).toBeCloseTo(350.6); + await clock.advance(Nano.fromMilli(Milli(1000))); + expect(tally.report(live.path).video).toEqual({ stalls: 1, stalled: 350 }); + } finally { + installed.close(); + } +}); + +test("an arrival pairs the media timestamp with milliseconds since the MoQ epoch", () => { + const tally = new Tally(); + tally.arrive(live, Milli(1234.9), Wall(Catalog.MOQ_EPOCH_UNIX_MILLIS + 5000.7)); + expect(tally.report(live.path).video.arrival as unknown).toEqual({ timestamp: 1234, wall: 5000 }); +}); + +test("an arrival's wall time is when the frame came off the transport, not when it was read", async () => { + const clock = new Manual(); + const installed = clock.install(); + try { + await clock.advance(Nano.fromMilli(Milli(1000))); + const arrival = Nano.toMilli(Nano(now())); + const before = wall(); + await clock.advance(Nano.fromMilli(Milli(300))); + expect(arrivedAt(arrival)).toBe(before); + } finally { + installed.close(); + } +}); + +test("keeps the most recent few catalog broadcasts", () => { + const tally = new Tally(); + for (let n = 0; n < 6; n++) tally.count({ path: Path.from(`b${n}.hang`), id: "video" }, { frames: 1 }); + expect(tally.report(Path.from("b0.hang"))).toEqual({}); + expect(tally.report(Path.from("b5.hang"))).toEqual({ video: { frames: 1 } }); +}); + +test("keeps a bounded number of rendition IDs per catalog broadcast, dropping the oldest", () => { + const tally = new Tally(); + for (let n = 0; n <= 1024; n++) tally.count({ ...live, id: `r${n}` }, { frames: 1 }); + const report = tally.report(live.path); + expect(Object.keys(report)).toHaveLength(1024); + expect(report.r0).toBeUndefined(); + expect(report.r1024).toEqual({ frames: 1 }); +}); diff --git a/js/watch/src/feedback.ts b/js/watch/src/feedback.ts new file mode 100644 index 00000000000..2c8916f64b0 --- /dev/null +++ b/js/watch/src/feedback.ts @@ -0,0 +1,125 @@ +import { Instant, type Milli, Nano, now, Wall, wall } from "@moq/clock"; +import * as Catalog from "@moq/hang/catalog"; +import type * as Moq from "@moq/net"; + +/** A cumulative counter a viewer reports for each rendition it read. */ +export type Counter = "frames" | "bytes" | "decoded" | "late" | "errors" | "stalls" | "stalled" | "underruns"; + +/** + * What a viewer received and played of one rendition, in the shape its feedback track reports. + * Counters are cumulative and a missing one means zero; `stalled` is in milliseconds. + */ +export type Received = Partial> & { + /** The newest received media timestamp in milliseconds, and its arrival in milliseconds since the MoQ epoch. */ + arrival?: { timestamp: number; wall: number }; +}; + +/** One rendition a decoder read: its ID in the catalog of the broadcast at `path`. */ +export interface Key { + /** The catalog broadcast, which a feedback report is published for. */ + path: Moq.Path.Valid; + /** The rendition's ID in that catalog, even when its media comes from another broadcast. */ + id: string; +} + +// The most catalog broadcasts a tally keeps counters for. A switch overlaps the outgoing and +// incoming one, so a few cover it without holding every broadcast ever watched. +const MAX_BROADCASTS = 4; + +/** + * The most rendition IDs a tally keeps per catalog broadcast, so a catalog that keeps renaming its + * renditions cannot grow it, or each report, without bound. Matches the Rust registry's limit. The + * oldest makes way; one still counting into it starts over, which a reader treats as a reset. + */ +const MAX_RENDITIONS = 1024; + +type Entry = { received: Received; stalling?: Instant }; + +/** + * Cumulative counters per rendition, for each catalog broadcast a decoder read. The decoder counts + * into it as frames arrive and play, and the feedback reporter reads it about once a second. + */ +export class Tally { + #broadcasts = new Map>(); + + #entry(key: Key): Entry { + let renditions = this.#broadcasts.get(key.path); + if (!renditions) { + renditions = new Map(); + this.#broadcasts.set(key.path, renditions); + for (const path of this.#broadcasts.keys()) { + if (this.#broadcasts.size <= MAX_BROADCASTS) break; + this.#broadcasts.delete(path); + } + } + let entry = renditions.get(key.id); + if (!entry) { + entry = { received: {} }; + renditions.set(key.id, entry); + if (renditions.size > MAX_RENDITIONS) { + const oldest = renditions.keys().next().value; + if (oldest !== undefined) renditions.delete(oldest); + } + } + return entry; + } + + /** Add `counts` to the rendition's counters. */ + count(key: Key, counts: Partial>): void { + const received = this.#entry(key).received; + for (const [counter, n] of Object.entries(counts) as [Counter, number][]) { + if (n !== 0) received[counter] = (received[counter] ?? 0) + n; + } + } + + /** Record a received frame at media `timestamp`, arriving at wall time `arrived`. */ + arrive(key: Key, timestamp: Milli, arrived: Wall): void { + const at = Math.floor(arrived - Catalog.MOQ_EPOCH_UNIX_MILLIS); + // A clock before the MoQ epoch has no wire encoding, so such an arrival is never recorded. + if (at < 0 || timestamp < 0) return; + this.#entry(key).received.arrival = { timestamp: Math.floor(timestamp), wall: at }; + } + + /** + * Count a stall that starts now, returning the function that ends it and reports how long it + * lasted in milliseconds. A report taken meanwhile includes the time stalled so far. + */ + stall(key: Key): () => number { + const entry = this.#entry(key); + const start = now(); + entry.received.stalls = (entry.received.stalls ?? 0) + 1; + entry.stalling = start; + return () => { + const duration = elapsed(start); + if (entry.stalling === start) entry.stalling = undefined; + entry.received.stalled = (entry.received.stalled ?? 0) + duration; + return duration; + }; + } + + /** What was received of each rendition of the broadcast at `path`, keyed by rendition ID. */ + report(path: Moq.Path.Valid): Record { + const report: Record = {}; + for (const [id, entry] of this.#broadcasts.get(path) ?? []) { + const received = { ...entry.received }; + if (entry.stalling !== undefined) received.stalled = (received.stalled ?? 0) + elapsed(entry.stalling); + if (received.stalled !== undefined) received.stalled = Math.floor(received.stalled); + if (received.arrival) received.arrival = { ...received.arrival }; + report[id] = received; + } + return report; + } +} + +/** + * The wall time of a frame's monotonic ingress time, as a received frame records it before + * reordering or decoder backpressure, so a frame read late still reports when it arrived. + */ +export function arrivedAt(arrival: Milli): Wall { + return Wall(wall() - Math.max(0, Nano.toMilli(Nano(now())) - arrival)); +} + +// Milliseconds since `start`. +function elapsed(start: Instant): number { + return Nano.toMilli(Instant.durationSince(now(), start)); +} diff --git a/js/watch/src/ui/stats.ts b/js/watch/src/ui/stats.ts index 56d0f773037..4e69d17930b 100644 --- a/js/watch/src/ui/stats.ts +++ b/js/watch/src/ui/stats.ts @@ -48,6 +48,9 @@ export function statsTab(parent: Effect, watch: MoqWatch): HTMLElement { const videoCard = statCard("video", "Video", videoIcon); const vRes = statLine(videoCard.grid, "Resolution"); const vCodec = statLine(videoCard.grid, "Codec"); + // The same decoder counters the viewer's feedback reports. + const vLate = statLine(videoCard.grid, "Late frames"); + const vStalls = statLine(videoCard.grid, "Stalls"); const vBitrateGraph = graph(parent, "Bitrate", { color: "#a855f7", format: formatBitrate }); const vFpsGraph = graph(parent, "Frame rate", { color: "#facc15", format: formatFps }); videoCard.el.append(vBitrateGraph.el, vFpsGraph.el); @@ -64,6 +67,7 @@ export function statsTab(parent: Effect, watch: MoqWatch): HTMLElement { const aChannels = statLine(audioCard.grid, "Channels"); const aBitrate = statLine(audioCard.grid, "Bitrate"); const aUnderruns = statLine(audioCard.grid, "Underruns"); + const aLate = statLine(audioCard.grid, "Late frames"); track(parent, audioCard, { catalog: watch.audio.source.out.catalog, flag: watch.controls.audio.muted, @@ -105,6 +109,8 @@ export function statsTab(parent: Effect, watch: MoqWatch): HTMLElement { vBitrateGraph.push(vBitrate); vFpsGraph.push(fps); if (vStats) vPrev = { frames: vStats.frameCount, bytes: vStats.bytesReceived, when: now }; + vLate.textContent = vStats ? `${vStats.late}` : "—"; + vStalls.textContent = vStats ? `${vStats.stalls} (${formatMillis(vStats.stalled)})` : "—"; // Audio. const aConf = watch.audio.source.out.config.peek(); @@ -119,6 +125,7 @@ export function statsTab(parent: Effect, watch: MoqWatch): HTMLElement { // A non-zero count means the target is below what arrivals actually need. const aUnder = aStats?.underruns ?? 0; aUnderruns.textContent = watch.audio.out.stalled.peek() ? `${aUnder} (buffering)` : `${aUnder}`; + aLate.textContent = aStats ? `${aStats.late}` : "—"; // Network. "Estimated max" is the congestion controller / PROBE estimate; // "Actual" is the goodput we measure from the video + audio byte counters. diff --git a/js/watch/src/video/decoder.test.ts b/js/watch/src/video/decoder.test.ts index fa32a7cd087..2e2b3ae04f0 100644 --- a/js/watch/src/video/decoder.test.ts +++ b/js/watch/src/video/decoder.test.ts @@ -36,6 +36,7 @@ let sources = 0; function watchBroadcast(catalog: Signal): Broadcast { const media = closedConsumer(`source-${sources++}`); return { + in: { name: new Signal(Moq.Path.from("watched")) }, out: { catalog, active: new Signal(undefined) }, relativeBroadcast: () => media, } as unknown as Broadcast; @@ -359,6 +360,7 @@ async function guardedPlayback( const media = new Signal(producer.consume()); const catalog = new Signal({ video: { renditions: { video } } }); const broadcast = new Signal({ + in: { name: new Signal(Moq.Path.from("watched")) }, out: { catalog, active }, relativeBroadcast: (effect: Effect) => effect.get(media), } as unknown as Broadcast); @@ -698,7 +700,13 @@ for (const kind of ["legacy", "cmaf"] as const) { try { expect(playback.submitted).toEqual([1000, 1100, 1200]); // Rejected payloads were still received, but never counted as frames. - expect(playback.decoder.out.stats.peek()).toEqual({ frameCount: 3, bytesReceived: 5 }); + expect(playback.decoder.out.stats.peek()).toMatchObject({ frameCount: 3, bytesReceived: 5 }); + // Feedback counts every frame received, under the rendition's ID in the watched catalog. + expect(playback.decoder.feedback(Moq.Path.from("watched")).video).toMatchObject({ + frames: 5, + bytes: 5, + decoded: 3, + }); } finally { playback.close(); } diff --git a/js/watch/src/video/decoder.ts b/js/watch/src/video/decoder.ts index 90bd143f0df..4ff9dfd8a86 100644 --- a/js/watch/src/video/decoder.ts +++ b/js/watch/src/video/decoder.ts @@ -1,4 +1,4 @@ -import { type Micro, Milli, Nano } from "@moq/clock"; +import { type Micro, Milli, Nano, type Wall } from "@moq/clock"; import * as Catalog from "@moq/hang/catalog"; import * as Container from "@moq/hang/container"; import * as Util from "@moq/hang/util"; @@ -17,6 +17,7 @@ import { } from "@moq/signals"; import { rotateVideoDimensions } from "@moq/video"; import { base64ToBytes } from "../base64"; +import { arrivedAt, type Counter, type Key, type Received, Tally } from "../feedback"; import { type Failure, failureMoved, nextMedia, subscribeMedia } from "../media"; import { supportCacheKey } from "../support-probe"; import type { Sync } from "../sync"; @@ -56,8 +57,26 @@ export interface Stats { /** Number of encoded bytes received. */ bytesReceived: number; + + /** Frames that arrived after their playout time. */ + late: number; + + /** Renditions that failed to decode. */ + errors: number; + + /** Times playback stalled: no new picture for half a second once one had played. */ + stalls: number; + + /** Total time spent in finished stalls. */ + stalled: Milli; + + /** The newest received media timestamp and the wall time it arrived. */ + arrival?: { timestamp: Milli; wall: Wall }; } +// The running totals before anything is counted. +const EMPTY: Stats = { frameCount: 0, bytesReceived: 0, late: 0, errors: 0, stalls: 0, stalled: Milli.zero }; + /** Signals the decoder derives. */ export type DecoderOutput = { /** Fatal decode failure, cleared when another source is selected. */ @@ -124,6 +143,11 @@ export class Decoder { #selected?: Rendition; #failed?: Failure; + // What each rendition delivered, for the feedback a viewer reports. + #tally = new Tally(); + // Whether a picture played since the decoder was enabled, so the wait for the first one is no stall. + #played = false; + #clearCurrentFrame(): void { this.#out.frame.update((prev) => { prev?.close(); @@ -175,6 +199,16 @@ export class Decoder { this.#signals.run(this.#runActive.bind(this)); this.#signals.run(this.#runDisplay.bind(this)); this.#signals.run(this.#runBuffering.bind(this)); + this.#signals.run(this.#runStalls.bind(this)); + } + + /** + * What this decoder received and played of each rendition in the catalog of the broadcast at + * `path`, keyed by rendition ID, for the feedback a viewer reports. + * @internal + */ + feedback(path: Moq.Path.Valid): Record { + return this.#tally.report(path); } // Read from the full catalog entry, which the decoder config omits, so a rising entry resizes Sync. @@ -265,6 +299,8 @@ export class Decoder { config: identity.decoder, identity: key, stats: this.#out.stats, + tally: this.#tally, + key: { path: broadcast.in.name.peek(), id: track }, catalog: broadcast.out.catalog, predecessor, }); @@ -337,6 +373,7 @@ export class Decoder { const frame = inner.get(active.frame); if (!frame) return; // A promoted track holds the previous picture until its first frame. if (active.error.peek()) return; + this.#played = true; if (this.#selected === active) { this.#failed = undefined; this.#out.error.set(undefined); @@ -353,6 +390,8 @@ export class Decoder { } #recordError(track: Rendition, error: Error): void { + this.#tally.count(track.key, { errors: 1 }); + this.#out.stats.update((stats) => ({ ...(stats ?? EMPTY), errors: (stats?.errors ?? 0) + 1 })); // An outgoing failure must not replace the selected rendition's failure latch. if (track !== this.#selected && this.#selected?.error.peek()) return; this.#failed = { @@ -400,6 +439,28 @@ export class Decoder { }, Nano.fromMilli(BUFFERING)); } + // Count each stall once a picture has played, and how long it lasted, against the rendition on + // screen. A disabled decoder waits for a new picture again: resuming is a startup, not a stall. + #runStalls(effect: Effect): void { + if (!effect.get(this.in.enabled)) { + this.#played = false; + return; + } + if (!effect.get(this.#out.stalled) || !this.#played) return; + const active = this.#active.peek(); + if (!active) return; + + const end = this.#tally.stall(active.key); + this.#out.stats.update((stats) => ({ ...(stats ?? EMPTY), stalls: (stats?.stalls ?? 0) + 1 })); + effect.cleanup(() => { + const duration = end(); + this.#out.stats.update((stats) => ({ + ...(stats ?? EMPTY), + stalled: Milli((stats?.stalled ?? 0) + duration), + })); + }); + } + /** Stop and release everything this holds. */ close() { this.#clearCurrentFrame(); @@ -434,6 +495,10 @@ interface DecoderTrackProps { // Absent for the max-delay decoder: the bytes arrive once and the min-delay decoder counts them. stats?: Signal; + // The rendition's feedback counters, under `key`: its ID in the catalog it was selected from. + // Absent for the max-delay decoder, like `stats`. + tally?: Tally; + key: Key; } class DecoderTrack { @@ -446,6 +511,8 @@ class DecoderTrack { identity: string; catalog: Getter; stats?: Signal; + tally?: Tally; + key: Key; timestamp = new Signal(undefined); frame = new Signal(undefined); @@ -509,6 +576,8 @@ class DecoderTrack { this.identity = props.identity; this.catalog = props.catalog; this.stats = props.stats; + this.tally = props.tally; + this.key = props.key; this.#warmup = props.warmup ?? getter(Milli.zero); this.#drives = props.warmup === undefined; this.#refresh = props.refresh ?? getter(false); @@ -517,6 +586,31 @@ class DecoderTrack { this.#signals.run(this.#run.bind(this)); } + // Count into this rendition's feedback and the decoder's running totals. + #count(counts: Partial>, totals?: (stats: Stats) => Partial): void { + this.tally?.count(this.key, counts); + if (totals) this.stats?.update((current) => ({ ...(current ?? EMPTY), ...totals(current ?? EMPTY) })); + } + + // Count a frame read from the track: its bytes even when rejected, and whether an accepted one + // arrived too late for playout. Classifies ingress before decoder backpressure changes the + // delivery clock. Only the min-delay decoder moves the clock and counts: the other reads the same + // frames behind it. + #receive(frame: Container.ReceivedFrame, accepted: boolean): void { + if (!this.#drives) return; + const timestamp = Milli.fromMicro(frame.timestamp as Micro); + const late = accepted && this.sync.received(timestamp, frame.arrival); + const bytes = frame.payload.byteLength; + const arrived = arrivedAt(frame.arrival); + this.tally?.arrive(this.key, timestamp, arrived); + this.#count({ frames: 1, bytes, late: late ? 1 : 0 }, (stats) => ({ + frameCount: stats.frameCount + (accepted ? 1 : 0), + bytesReceived: stats.bytesReceived + bytes, + late: stats.late + (late ? 1 : 0), + arrival: { timestamp, wall: arrived }, + })); + } + // Reorders groups and frames up to the shared max delay. A rendition switch is not a cold start: // the path's arrivals are what they were, so the estimate continues from whatever read before. #consume(effect: Effect, sub: Moq.Track.Subscriber, format: Container.Format): Container.Consumer { @@ -556,6 +650,7 @@ class DecoderTrack { // A cancelled codec must not recreate the poster its track already released. Firefox can // still deliver output after close(). if (signal.aborted) return; + this.#count({ decoded: 1 }); const generation = this.#generation; const group = this.#groups.get(frame.timestamp); // Pictures come out in presentation order, so a chunk at or before this one that @@ -683,16 +778,9 @@ class DecoderTrack { // resets this guard. Rejected payloads still count as received bytes. const stale = latest !== undefined && next.group < latest; const accepted = !stale && recovery.accept(frame); - this.stats?.update((current) => ({ - frameCount: (current?.frameCount ?? 0) + (accepted ? 1 : 0), - bytesReceived: (current?.bytesReceived ?? 0) + frame.payload.byteLength, - })); + this.#receive(frame, accepted); if (!accepted) continue; - // Classify ingress before decoder backpressure changes the delivery clock. Only the - // min-delay decoder moves the clock: the other reads frames already behind it. - if (this.#drives) this.sync.received(Milli.fromMicro(frame.timestamp as Micro), frame.arrival); - // Track decode buffer: frames sent to decoder but not yet rendered. Only bridge from // the previous frame when no gap or end came between them. Group ids can't answer that: // they aren't required to be sequential (some encoders derive them from DTS), so @@ -765,15 +853,9 @@ class DecoderTrack { // resets this guard. Rejected payloads still count as received bytes. const stale = latest !== undefined && next.group < latest; const accepted = !stale && recovery.accept(frame); - this.stats?.update((current) => ({ - frameCount: (current?.frameCount ?? 0) + (accepted ? 1 : 0), - bytesReceived: (current?.bytesReceived ?? 0) + frame.payload.byteLength, - })); + this.#receive(frame, accepted); if (!accepted) continue; - // Classify ingress before decoder backpressure changes the delivery clock (see #runLegacy). - if (this.#drives) this.sync.received(Milli.fromMicro(frame.timestamp), frame.arrival); - // Track decode buffer (see #runLegacy: bridge only when no gap or end came between, // never on group adjacency, which proves nothing about the timeline). if (previous !== undefined) { @@ -933,8 +1015,9 @@ class DecoderTrack { } /** Constructor properties for {@link Rendition}. */ -interface RenditionProps extends Omit { +interface RenditionProps extends Omit { stats: Signal; + tally: Tally; // The rendition on screen when this one was selected from the same path, whose decoders' arrival // estimates these continue. predecessor?: Rendition; @@ -958,6 +1041,8 @@ class Rendition { readonly config: DecoderConfig; readonly identity: string; readonly catalog: Getter; + // The rendition's feedback counters' key: its ID in the catalog it was selected from. + readonly key: Key; // The picture chosen between the two decoders, and its timestamp once it presented on the clock: // undefined while it is only a poster. @@ -981,6 +1066,7 @@ class Rendition { this.config = props.config; this.identity = props.identity; this.catalog = props.catalog; + this.key = props.key; // A missing or zero `warmup` keeps one decoder. The budgets follow a changed value without // reopening the max-delay decoder. @@ -1000,6 +1086,7 @@ class Rendition { const max = new DecoderTrack({ ...props, stats: undefined, + tally: undefined, warmup, refresh: dual, predecessor: predecessor?.max.peek()?.consumer ?? predecessor?.consumer, diff --git a/quest/m2/latency-ledger.md b/quest/m2/latency-ledger.md index afc6242490e..a9b15b73e79 100644 --- a/quest/m2/latency-ledger.md +++ b/quest/m2/latency-ledger.md @@ -48,7 +48,6 @@ probes, which was the right trade to get it running. This quest promotes them. ## Required -- [Browser reporters](/quest/m2/stats/js.md) - `` publishes the feedback snapshot these stages join - [Rust reporters](/quest/m2/stats/rust.md) - `moq play --echo` publishes the feedback snapshot these stages join ## Related diff --git a/quest/m2/stats/README.md b/quest/m2/stats/README.md index 1a64be617c6..c838ce836ae 100644 --- a/quest/m2/stats/README.md +++ b/quest/m2/stats/README.md @@ -68,6 +68,15 @@ kind. before the viewer read the catalog; one broadcast per catalog removes the collisions, the race, and the cap with less code, and needs no JS track request API. +- **One feedback writer per name** (2026-11-02, maintainer). Each player + publishes its own `.echo` broadcast with only its own counts. A second + player on the same connection that resolves the same feedback path, for + the same catalog or another, publishes nothing and warns until the first + one leaves; the application gives each player its own name. Reason: sharing one broadcast between + players needed a departed sum, a per-player ledger, and join-time baselines + to keep the counters cumulative, and four review rounds of + [moq-dev/moq#7222](https://github.com/moq-dev/moq/pull/7222) each found + another edge case in them. - **Trust is the token prefix** (2026-09-29). Whoever the application's tokens let publish under the echo path may report, and no report is authenticated beyond that. @@ -113,8 +122,6 @@ connection view and the dashboard flow. - [Rust reporters](/quest/m2/stats/rust.md) - the player and gateways publish stats and feedback, as the CLI remuxes and capture do -- [Browser reporters](/quest/m2/stats/js.md) - `` publishes - stats and `` publishes feedback - [Preflight](/quest/m2/stats/preflight.md) - a bounded test run over a broadcast reports which media layer is broken and why - [Client health in use](/quest/m2/stats/health-consumers.md) - a per-connection media counter, consumers of the grade, and re-measured limits diff --git a/quest/m2/stats/health-consumers.md b/quest/m2/stats/health-consumers.md index dc247051e11..7ac2ae86095 100644 --- a/quest/m2/stats/health-consumers.md +++ b/quest/m2/stats/health-consumers.md @@ -15,9 +15,6 @@ grade's limits are re-measured against browser reporters. only for a connection's sole broadcast until then. - Consumers: preflight is planned in [Preflight](/quest/m2/stats/preflight.md); add the demo dashboard and a `moq` CLI sink. -- Re-measure the thresholds in `rs/hang/src/health.rs` once - [Browser reporters](/quest/m2/stats/js.md) publish real numbers. - -## Required - -- [Browser reporters](/quest/m2/stats/js.md) - the browser numbers the limits are measured against +- Re-measure the thresholds in `rs/hang/src/health.rs` against the numbers + the browser reporters publish (`` and + ``). diff --git a/quest/m2/stats/js.md b/quest/m2/stats/js.md deleted file mode 100644 index bea3cad59d3..00000000000 --- a/quest/m2/stats/js.md +++ /dev/null @@ -1,46 +0,0 @@ -# [M] The browser publisher and player report stats and feedback - -## Goal - -`` announces a stats track in its catalog and fills it, and an -attribute solicits feedback by setting the catalog's `echo` section. -``, given an echo name, publishes an `.echo` broadcast under the -echo path of a catalog soliciting feedback. Both use the `@moq/hang` schemas. - -## Plan - -- `js/publish`: - - The encoder `Stats` signals are already per rendition (`frames`, - `bytes`, `keyframes`). Add drops and the target bitrate. - - Fold them into the stats snapshot keyed by rendition ID, and write it - through the `@moq/json` snapshot producer on one track with - `compress: true`, matching the [m1 compression migration](https://github.com/moq-dev/moq/pull/7046). - A `stats` attribute enables it. - - `transport` carries PROBE rtt only, because `WebTransport.getStats()` - returns nothing useful in Chromium or Firefox today. - - An `echo-path` attribute sets the catalog's `echo` section, matching - Rust's `moq import --echo-path` (decided 2026-10-22, maintainer). -- `js/watch`: - - The decoder `Stats` signals grow late frames, stalls and stalled - duration, underruns from the worklet's count, decode errors, and the - newest arrival. - - An `echo` attribute names the viewer, refused unless it is one path - segment, as in Rust. Once a watched catalog carries an - `echo` section, the element resolves the echo path against the broadcast, - appends `.echo`, and publishes there with the fixed feedback track, keyed by rendition ID, the same rule as - Rust, and reconciles it on each catalog update like Rust. - - Watch elements on one connection watching the same catalog under one - name share its producer, because a second `createBroadcast` at a path - closes the first. -- The existing UI stats panels read the same signals. -- The demo sets both attributes, so the media test can read a browser - viewer's feedback through `moq export echo`. -- `doc/lib/js` documents the attributes. -- Decided 2026-10-08: no E2EE refusal here. The elements have no encrypted - broadcast yet, so [Encrypted browser components](/quest/m2/e2ee/browser.md) refuses the - stats track and the `echo` attribute when it adds one: they would publish - rendition IDs and per-track counters in plaintext beside it. - -## Related - -- [Encrypted browser components](/quest/m2/e2ee/browser.md) - refuses stats and feedback on an encrypted broadcast