diff --git a/quest/m0/3477-watch-auto-latency.md b/quest/m0/3477-watch-auto-latency.md index efeb556cf58..5580ded276b 100644 --- a/quest/m0/3477-watch-auto-latency.md +++ b/quest/m0/3477-watch-auto-latency.md @@ -1,81 +1,47 @@ -# [M] js/watch: auto latency follows measured arrivals, and the audio ring holds slack and re-buffers +# [S] js/watch: the measured auto latency is proven clean in a browser against a real relay ## Goal -The "Real-time" / auto preset plays clean audio on a LAN and against a public -relay: no underrun episodes and no skip-aheads in steady state, at the lowest -delay the measured arrivals allow. The target is sized from what actually -arrives, the way WebRTC's NetEq does, and the round-trip time drops out of -the formula. The audio ring tolerates one chunk of slack above the target and -re-buffers after an underrun instead of playing the next chunk on an empty -cushion. Audio underruns are visible in the stats panel. +The "Real-time" / auto preset plays clean audio on a LAN and against the +public relay: zero underruns after convergence and no skip-aheads in steady +state, proven by the browser harness and by a manual run on Chrome and +Safari, not only by the unit replay. The publisher-side audio encoder lag the +same report observed is measured in the same manual run and either confirmed +fixed or turned into a root-cause quest. -Boundaries: convergence still uses skip-ahead and silence; time-stretching is -[Time stretch](/quest/m2/watch-audio-time-stretch.md). The TS importer's PES -bursts are advertised honestly by the jitter estimator quest, not re-paced. -The publisher-side encoder lag observed in the same report is its own quest. +Boundaries: the estimator and the ring re-buffer themselves are PR #3517 on +`dev`; this quest only verifies them where the PR could not. Convergence +still uses skip-ahead and silence; time-stretching is +[Time stretch](/quest/m2/watch-audio-time-stretch.md). ## Plan -Branch from main. The ring files are byte-identical on both branches; only -`sync.ts` differs (dev's `delay` + `buffer` split from #3396), so the target -estimator lands in whichever shape the branch has and merges forward. - -Measured in #3477 (traces and a replay harness on the reporter's fork): with -a 46 ms auto target and two-chunk pacing the shipped ring produces hundreds -of underruns and skips per minute; holding one chunk of slack and re-buffering -removes every underrun at the same target; only a target that follows the -arrival spread (270 ms on the public bursty stream) removes the skips. A -fixed 250 ms is clean everywhere, which is why 500 ms and 1 s always sounded -fine. - -### Target - -- Measure arrival jitter per track at container frame arrival in `js/hang`, - before decode: arrival wall time minus media timestamp, relative to its - running minimum. Keep a decaying histogram and read a high percentile - (p95 to start, NetEq's forgetting-factor shape), floored at one frame plus - the catalog `jitter`. `Sync` takes the maximum across audio and video and - adds nothing RTT-derived; `MIN_JITTER` and the `1.25 x minRtt` term go. -- The target starts from the catalog floor and converges as samples arrive; - it only lowers gradually, so a viewer's buffer shrinks in steps of at most - one chunk per interval and never underruns on a refinement. -- `maxAge` on the wire is `delay + buffer` as today; js/net rounds a - fractional value up, so a measured target needs no rounding here. - -### Ring - -- `shared-ring-buffer.ts` `read()`: skip only when buffered exceeds - `latency + chunk`, landing at `latency`; raise `STALLED` when the ring runs - dry on a non-empty timeline so `insert()` refills to the target before - playback resumes. That is a new worklet-to-main-thread write of the flag; - state the atomics discipline next to the existing epoch comments. -- `ring-buffer.ts` (the postMessage path every non-isolated page uses): - decouple capacity from the target in `#capacityFor`, apply the same band - in `write()` overflow, and set `#stalled` on underrun. -- `buffer.ts`: re-stalling flips `Backpressure.wait` in buffered mode, which - starts flushing waiters mid-playback. Reconcile with - `decoder.ts` `#runLatencyReanchor`, which already resets on a floor - increase, so there is one re-buffer path. -- Render underrun shortfall with a short ramp instead of a hard step. - -### Visibility - -- `audio.out.stalled` and an underrun counter reach the stats panel and the - buffering indicator, which today read video only; `demo/web` stops - hardcoding `stalled=false` for the audio bar. - -### Verification - -- Unit: replay the recorded arrival traces (attach a trimmed copy under - `js/watch`) through both rings and assert zero underruns at the converged - target; synthetic Gaussian arrivals with two-chunk bursts converge within a - bound. -- Browser: the `test/smoke` JS driver plays `bbb` from a local `moq import - ts` with ffmpeg's default PES packing at auto and asserts underruns stay - at zero after convergence, on the isolated and the postMessage path. -- Manually against the public relay on Chrome and Safari, the two rows the - issue measured. +Branch from dev. #3517 lands the measured arrival `spread` per track, the +`max(advertised, measured)` target, one chunk of ring slack, re-stall on +underrun, and an underrun counter in the stats panel, verified with synthetic +replay traces. It leaves three things undone, which are this quest: + +- Browser assertion: the `test/smoke` JS driver plays `bbb` from a local + `moq import ts` with ffmpeg's default PES packing at auto and asserts the + underrun counter stays at zero after convergence, on the isolated and the + postMessage ring paths. +- Recorded traces: the arrival traces from #3477 live on the reporter's fork. + Attach a trimmed copy under `js/watch` and replay them through both rings + in `replay.test.ts` in place of the synthetic traces of the same shape. +- Manual run against the public relay on Chrome and Safari, the two rows the + issue measured, with a real microphone and 40 ms or more of added RTT. + Measure the audio encoder's input-to-output lag in the same run using the + reporter's instrumented harness (fork branch `debug/rt-audio`); #3518 fixed + the known cause (the encoder was rebuilt on every subscriber churn), so the + 7.35 s lag and the 88 to 275 ms/s drift the issue reported stand + unconfirmed. If drift survives, the suspects are `writeFrame` opening a + group per audio frame under WebTransport stream credit and the main-thread + task queue that delivers encoder output; turn that into its own quest + rather than fixing it here. + +## Required + +- PR #3517 has merged to `dev` ## Closes @@ -84,6 +50,5 @@ fine. ## Related +- [Plan: A/V clock](/quest/m1/plan-av-clock.md) - the audio-master clock that follows the estimator - [Time stretch](/quest/m2/watch-audio-time-stretch.md) - convergence without skips or silence, on top of this -- [Encoder lag](/quest/m0/publish-audio-lag-measure.md) - the publisher-side lag the same report observed -- [Jitter estimator](/quest/m0/3479-mux-jitter-flush-span.md) - an honest catalog floor for bursty importers diff --git a/quest/m0/README.md b/quest/m0/README.md index fbb4350f302..5ed9fe532e9 100644 --- a/quest/m0/README.md +++ b/quest/m0/README.md @@ -15,10 +15,9 @@ regression test per Root Cause First. ## Quests -- [Auto latency](/quest/m0/3477-watch-auto-latency.md) - js/watch: auto latency follows measured arrivals, and the audio ring holds slack and re-buffers +- [Auto latency](/quest/m0/3477-watch-auto-latency.md) - js/watch: the measured auto latency (#3517 on dev) is proven clean in a browser and against the public relay - [Jitter estimator](/quest/m0/3479-mux-jitter-flush-span.md) - moq-mux: catalog jitter is the publisher's maximum flush span, never a running minimum - [WebKit gate](/quest/m0/webkit-webtransport-gate.md) - js/net: every WebKit engine takes the WebSocket path, not just the Safari brand, so iOS Chrome and Firefox stop freezing after two minutes -- [Encoder lag](/quest/m0/publish-audio-lag-measure.md) - js/publish: measure the audio encoder's input-to-output lag on a high-RTT path - [IETF error codes](/quest/m0/ietf-error-codes.md) - every code on a moq-transport wire is a registered value for the negotiated draft, requests and stream resets alike - [Publisher priority](/quest/m0/3534-ietf-publisher-priority.md) - moq-net: a track's publisher priority survives a moq-transport hop instead of being flattened to 0 - [TRACK_STATUS refusal](/quest/m0/3492-ietf-track-status-refusal.md) - moq-net: TRACK_STATUS gets a NOT_SUPPORTED refusal instead of a silent drop @@ -28,9 +27,8 @@ regression test per Root Cause First. - [TS restart stall](/quest/m0/3533-ts-export-restart-stall.md) - moq export ts: a content restart on a continuous timeline no longer fences video and primary audio for good - [Group charge](/quest/m0/group-charge.md) - charge real per-group cost so MOQ_CACHE_CAPACITY bounds real memory - [uring all-features](/quest/m0/uring-all-features-build.md) - moq-uring does not compile with `--all-features`, so the nightly features gate fails on it -- [Failure artifacts](/quest/m0/qa-failure-artifacts.md) - retain inspectable traces, logs, and rerun commands when QA fails -- [Browser permission QA](/quest/m0/browser-permission-qa.md) - nothing covers what the publisher does when the user denies the camera or microphone +- [Failure artifacts](/quest/m0/qa-failure-artifacts.md) - a failing harness run keeps its run directory and a Playwright trace, and CI uploads them +- [Harness drive-bys](/quest/m0/harness-drive-bys.md) - drop the just worktree recipe and decide the relay listening kind field that came with #3509 +- [Capture denial](/quest/m0/browser-permission-qa.md) - moq-publish swallows a refused camera or microphone and retries forever; it surfaces the denial and recovers on grant - [Publisher audio unlock](/quest/m0/publish-audio-unlock.md) - the publisher's capture AudioContext stays suspended when the page had no gesture, so no audio is ever encoded -- [Media QA on other engines](/quest/m0/browser-media-qa-engines.md) - the media output checks only run on the pinned Chromium -- [Teardown error classification](/quest/m0/js-teardown-error-classification.md) - a browser consumer cannot tell a requested end from a fault, so QA cannot be strict during one -- [Impaired-path profile](/quest/m0/transport-impairment-profile.md) - run the transport failure drills over a delayed, lossy, rate-limited QUIC path +- [Impaired path](/quest/m0/transport-impairment-profile.md) - the transport drills run over a seeded, impaired UDP path on any host diff --git a/quest/m0/browser-media-qa-engines.md b/quest/m0/browser-media-qa-engines.md deleted file mode 100644 index 04b11005dd1..00000000000 --- a/quest/m0/browser-media-qa-engines.md +++ /dev/null @@ -1,31 +0,0 @@ -# [M] Run the media QA harness on other browser engines - -## Goal - -The media output and lifecycle checks run on more than the pinned Chromium, and -an engine that cannot run a case says so by name instead of passing by omission. -A green run states which engines were measured and which APIs each one lacked. - -## Plan - -`just test smoke-media` measures presented frame progress, tone presence, and -audio/video skew against a deterministic fixture, but only on Playwright's -Chromium. Its capability probe already enumerates what the player needs -(WebTransport, WebCodecs encode and decode, AudioWorklet, -`MediaStreamTrackProcessor`, `canvas.captureStream`) and fails naming what is -missing, which is the groundwork this builds on. - -- Take the inventory first. Firefox has no WebTransport in Playwright and WebKit - has neither WebTransport nor the WebCodecs encoder, so decide per engine - whether it subscribes only, runs over the WebSocket fallback, or is out of - scope, and record why. -- Split the harness's publisher and subscriber roles so an engine can take one - side: a Chromium fixture with a Firefox player is the case worth having. -- Report a per-engine matrix. An engine that cannot publish is not a skipped - cell; it is a stated limit with the missing API named. -- Keep the tolerances shared. A skew bound that only holds on one engine is a - measurement of that engine, not of the player. - -## Related - -- [Failure artifacts](/quest/m0/qa-failure-artifacts.md) - shared trace and sample output diff --git a/quest/m0/browser-permission-qa.md b/quest/m0/browser-permission-qa.md index 77b3c751a8d..2db1d652275 100644 --- a/quest/m0/browser-permission-qa.md +++ b/quest/m0/browser-permission-qa.md @@ -1,38 +1,42 @@ -# [M] Cover denied capture permission in the browser +# [M] moq-publish surfaces a capture denial and recovers when permission is granted ## Goal -The browser harness proves what `` does when the user refuses the -camera or microphone: no broadcast is announced, the failure is visible through -the public API, and a subscriber gets a clean "nothing is published" rather than -a hang or a half-announced broadcast. Granting permission afterwards recovers -without a reload. +When the user refuses the camera or microphone, `` says so +through its public state, announces no broadcast, and a subscriber sees a +clean "nothing is published" rather than a hang or a half-announced +broadcast. Granting permission afterwards recovers without a reload. The +browser harness proves all of it. ## Plan -`test/smoke/clients/js/media.ts` already runs Chromium without -`--use-fake-ui-for-media-stream` and asserts the user-gesture gate on audio, but -every media case it drives uses the generated fixture, which never calls -`getUserMedia`. The permission path is still only exercised by the interop -matrix, which grants everything up front. +Today a denial is invisible. `js/publish/src/source/camera.ts` swallows the +`getUserMedia` rejection (`.catch(() => undefined)`) and hands it to the +retry loop; `microphone.ts` does the same, `device.ts` only warns. `Camera.out` +exposes `source` alone and `Retry` has no public failure signal, so the only +external symptom is a `source` that never becomes defined. -- Drive `` in a context where capture is denied. - `--use-fake-device-for-media-stream` keeps the device deterministic while - Playwright's permission state decides the verdict, so the case stays - repeatable. -- Assert `announce="source"` holds the broadcast back, that the element surfaces - the denial through its public state rather than only a console warning, and - that a subscriber sees no announcement. -- Grant the permission and assert the broadcast announces and encodes without - reloading the page. -- Publish what the case cannot claim: a fake device is not a real one, and a - headless permission decision is not a user clicking a prompt. -- Assert the audio gesture gate while there. `media.ts` clicks both pages and - requires audio afterwards, but does not assert silence beforehand: Chromium - enforces the gate on the fixture page and has been seen not enforcing it on the - player's, whose graph is built a second later. Find what actually decides it, - then assert the gate rather than only exercising it. +- Give the capture sources a public failure state: the denial (and any other + terminal `getUserMedia` error) reaches `` as an observable + status a page can render, and the element stops retrying a permission the + user refused. Refused is refused; a silent retry loop is warn-then-ignore. +- Keep `announce="source"` holding the broadcast back while there is no + source, so a subscriber sees no announcement. +- Regression in `test/smoke/clients/js/media.ts`: drive + `` with `--use-fake-device-for-media-stream` + so the device is deterministic while Playwright's permission state decides + the verdict. Assert the denial is visible through the element's state, no + announcement reaches the subscriber, then grant the permission and assert + the broadcast announces and encodes without reloading the page. State what + the case cannot claim: a fake device is not a real one and a headless + permission decision is not a user clicking a prompt. +- Assert the audio gesture gate while there. `media.ts` clicks both pages + and requires audio afterwards but does not assert silence beforehand; + Chromium enforces the gate on the fixture page and has been seen not + enforcing it on the player's, whose graph is built a second later. Find + what decides it, then assert the gate rather than only exercising it. ## Related +- [Publisher audio unlock](/quest/m0/publish-audio-unlock.md) - the other publisher path that fails silently without a gesture - [Failure artifacts](/quest/m0/qa-failure-artifacts.md) - shared trace and sample output diff --git a/quest/m0/harness-drive-bys.md b/quest/m0/harness-drive-bys.md new file mode 100644 index 00000000000..ed9eb9816ac --- /dev/null +++ b/quest/m0/harness-drive-bys.md @@ -0,0 +1,26 @@ +# [S] Trim what rode in with the harness isolation + +## Goal + +The test harness isolation from #3509 keeps its point (private run +directories, reserved ports, process-group teardown) and sheds the two +things that came along without a reason to live in this repository: the +`just worktree` recipe and, if nobody reads it, the `kind` field on the +relay's `listening` log line. + +## Plan + +- Delete the `worktree` recipe and its helpers from the root `justfile` + (about 265 lines: metadata-access probing, base recording, staleness + reports) and the section of `test/README.md` that documents it. Keep the + `_base` split that `_changed` shares, since `just check` scopes with it. +- Decide whether the relay's `kind = "quic" | "http" | "https"` on the + `listening` line and the two new web-listener lines stay. They are + operator-visible and documented in `doc/bin/relay/config.md`; keep them + only if the harness actually reads the bound address from them, otherwise + revert to the single line and drop the doc. +- `just clean` staying this-checkout-only is fine; leave it. + +## Related + +- [Failure artifacts](/quest/m0/qa-failure-artifacts.md) - the harness run directory this trims around diff --git a/quest/m0/js-teardown-error-classification.md b/quest/m0/js-teardown-error-classification.md deleted file mode 100644 index 6b2e2e7ffd4..00000000000 --- a/quest/m0/js-teardown-error-classification.md +++ /dev/null @@ -1,47 +0,0 @@ -# [M] Tell a teardown abort apart from a fault in the browser - -## Goal - -A browser consumer can tell "this subscription ended because you ended it" from -"something broke" without reading message text. Cutting a publisher off, or -pointing `` at another broadcast, produces a classified end rather -than a `console.error` a caller has to pattern-match. - -## Plan - -Today every path is the same path. A subscription aborted by a name change or a -departing publisher rethrows out of `js/hang`'s group reader, lands in -`Effect.spawn`'s catch-all, and prints `spawn error` plus whatever `Error` the -session happened to construct: `cancel`, `track is closed`, -`broadcast is closed: null`, `remote error: 0`. `js/watch/src/video/decoder.ts` -logs a bare `DOMException` with no prefix at all and carries a -`// TODO bubble up error`. Only `RemoteError` has a machine-checkable shape, and -its code is `0` whenever the transport dropped the stream without one. - -The consequence is a test that cannot be strict. `test/smoke/clients/js/media.ts` -drives unsubscribe/rejoin and publisher-stop/republish, both of which abort -subscriptions on purpose, so its waits tolerate every error the page reports for -the duration of the transition. Everything is still echoed to the log, and the -measurement window after each transition is fatal on any error, but during the -transition a genuine decoder or transport fault reads exactly like the abort the -case asked for. A message allowlist would not fix that: the truncated group a -publisher leaves behind reaches the decoder as the same `DOMException` a broken -decoder would. - -- Decide what a consumer should see when a track, broadcast, or session ends by - request rather than by fault. A distinct error type carrying that reason is the - obvious shape; whether it belongs in `@moq/net` alone or is mirrored in - `@moq/hang` and `@moq/watch` is the open question. -- Stop `Effect.spawn` being the reporting path for an expected end. A task whose - stream was cancelled has not failed and should not reach a catch-all logger. -- Give the video decoder's `error` callback the same treatment as the rest, so - the one unprefixed `console.error` in the player stops being unclassifiable. -- Mirror whatever lands in `rs/moq-net`, which already separates a close from an - abort, so the two languages agree on what a caller is told. -- Then tighten `media.ts`: tolerate the classified end and fail immediately on - anything else, and delete the blanket drain. - -## Related - -- [Media QA on other engines](/quest/m0/browser-media-qa-engines.md) - the same harness, another axis -- [Failure artifacts](/quest/m0/qa-failure-artifacts.md) - shared trace and sample output diff --git a/quest/m0/publish-audio-lag-measure.md b/quest/m0/publish-audio-lag-measure.md deleted file mode 100644 index a3115836027..00000000000 --- a/quest/m0/publish-audio-lag-measure.md +++ /dev/null @@ -1,32 +0,0 @@ -# [S] js/publish: measure the audio encoder's input-to-output lag on a high-RTT path - -## Goal - -A `` session on a real microphone against a relay with 40 ms or -more of added RTT emits every encoded audio chunk within one frame duration of -its input, measured over 25 seconds or more, with and without subscriber churn. - -## Plan - -One root cause is already fixed: the `AudioEncoder` was rebuilt whenever the -rendition's track producer changed, which discarded whatever the codec still -held and restarted the framer mid-frame, and it was built only once demand -arrived, putting `configure()` on the critical path of the first subscription. -That is covered by unit regressions in `js/publish/src/audio/encoder.test.ts`, -but it was never measured against a real relay, so the numbers #3477 reported -stand unconfirmed: a lag that grew to 7.35 s over 25 s on the public relay, and -a churn-free drift of 88 to 275 ms/s against 0.7 ms/s on localhost. - -- Measure with the reporter's instrumented harness (fork branch - `debug/rt-audio`: timestamps at capture, encoder input and output, and - `writeFrame`). Chrome's fake microphone is unusable, its encoder lags by - design. -- If drift survives, the suspects are what the fix did not touch: `writeFrame` - opening a group per audio frame under WebTransport stream credit, and the - main-thread task queue that delivers encoder output. -- Complete this when the measurement is clean, or turn what it finds into a - root-cause quest. - -## Related - -- [Auto latency](/quest/m0/3477-watch-auto-latency.md) - the report the numbers come from diff --git a/quest/m0/qa-failure-artifacts.md b/quest/m0/qa-failure-artifacts.md index a3161f66f02..5f1cf6057eb 100644 --- a/quest/m0/qa-failure-artifacts.md +++ b/quest/m0/qa-failure-artifacts.md @@ -1,43 +1,33 @@ -# [M] Preserve QA failures as reproducible debug bundles +# [S] A failing harness run leaves its run directory and a browser trace behind ## Goal -A failed or hung end-to-end run leaves enough evidence to diagnose and rerun -the failure without reconstructing its environment or asking for screenshots. -Agents can inspect test processes and traces through a documented debug path. +When smoke, WASM, or TS fails in CI, the run's logs, relay config, and a +Playwright trace of the failing page are downloadable from the workflow run. +Locally, the same directory survives the failure with the rerun command +printed. A passing run still cleans up. ## Plan -The smoke, WASM, and TS harnesses delete their temporary directories on exit. -The smoke and WASM workflows do not upload a diagnostic bundle. Browser console -output exists, but there is no retained Playwright trace for the failing action. - -- Add a common artifact-directory convention and retain failure evidence before - cleaning up processes. Include the command, versions, run identity, fixture - hashes, endpoint topology, negotiated protocol, per-process logs, and results. -- Capture Playwright traces with DOM snapshots/screenshots, page errors, and - supported browser network diagnostics. Ordinary HTTP traces do not expose - every QUIC/WebTransport event; collect relay qlog where supported and mark - missing backend trace capability explicitly. -- Bound logs and preserve a useful tail. On timeout, attempt a bounded stack - capture for owned native processes before terminating them; report debugger - permission failures without blocking cleanup. Retain matching symbols/build - identity for crash analysis. -- Provide a local retained-session option with the exact URL, PIDs, debugger - attach command, and teardown command. Add read-only CI artifact retrieval by - run ID so the agent can inspect the same failure that the reviewer sees. -- Upload bundles on failure in CI with finite retention. Use synthetic media - and test credentials by default; redact tokens, URL credentials, and headers - before upload. Packet payloads and full core dumps require explicit opt-in. - -Acceptance: inject a browser assertion failure, relay crash, and hung subscriber. -Each exits nonzero, leaves an inspectable bundle and rerun command, and reaps its -children. Verify a test token is absent from uploaded artifacts. Retrying for -diagnosis must retain the original failure rather than convert it into a pass. - -[Playwright's debugging guidance](https://playwright.dev/docs/best-practices) -supports trace-based inspection; use the existing pinned Playwright dependency. +The harness library already gives every run a private directory with each +process's log, and `MOQ_TEST_KEEP=1` retains it; on failure it prints the +rerun command but still deletes the directory unless the flag was set. The +browser drivers record nothing. `smoke.yml` and `wasm.yml` upload nothing. + +- Keep the run directory whenever the run fails, not only when asked. +- The Playwright drivers (`test/smoke/clients/js`, `test/wasm/driver.ts`) + start a trace and save it into the run directory on failure. Chromium's + console and page errors already reach the log. +- `smoke.yml` and `wasm.yml` upload the run directory as a workflow artifact + when the job fails, with a short retention. The relay config in it carries + only the harness's throwaway certificate and test tokens, so nothing needs + redacting; the TS harness joins if its outputs fit the same shape. + +That is the whole quest. Stack dumps of hung processes, relay qlog, HAR +capture, retained debug sessions, and fetching CI artifacts by run id were +the earlier plan, abandoned with #3506 as far more machinery than a failed +run needs; if one of them earns its place later it is its own quest. ## Related -- [qlog](/quest/m1/uring-qlog.md) - adds traces for the io_uring backend; other backends can ship first +- [Impaired path](/quest/m0/transport-impairment-profile.md) - records the profile and seed in the same directory diff --git a/quest/m0/transport-impairment-profile.md b/quest/m0/transport-impairment-profile.md index b125022c92d..499e71ddedd 100644 --- a/quest/m0/transport-impairment-profile.md +++ b/quest/m0/transport-impairment-profile.md @@ -1,43 +1,43 @@ -# [M] Impaired-path profile for the transport drills +# [M] The transport drills run over a seeded, impaired UDP path ## Goal The transport drills run against a bidirectionally impaired QUIC path (delay, -loss, and rate limit) with the impairment measured rather than assumed, and the -run removes every namespace and process it created on exit or cancellation. -The host's normal network path is never touched. +jitter, loss, reorder, and a rate limit) with the impairment asserted rather +than assumed, on macOS and Linux alike, with no capabilities and nothing +touching the host's network. A profile that fails to install fails the run. ## Plan -`test/drill` already owns the scenarios and the sensitivity proof; this quest -only adds the impaired path they run on. The drills drive real QUIC over -loopback inside one process, so a private network namespace with `netem` on its -own `lo` impairs them unmodified: no veth pair, no proxy, and no way to reach -the host's interfaces. `unshare -rn` grants `CAP_NET_ADMIN` inside that -namespace without root on stock kernels; report the missing capability rather -than escalating when it does not. - -- Linux only. On another host the recipe reports the profile as unavailable and - exits nonzero for a strict run, rather than silently running unimpaired. -- Apply `netem` (delay, jitter, loss, reorder) plus a `tbf` rate limit to `lo` - inside the namespace, so both directions are impaired. Never write a qdisc on - a host interface, and never require `sudo` for the default profile. -- Record the applied settings, the seed, the kernel and `tc` versions, and a - measured baseline (observed one-way delay, loss, and achieved rate) with the - run's artifacts. A seed does not make kernel scheduling deterministic, so the - record is evidence of what ran, not a promise of reproducibility. -- Do not substitute an HTTP interceptor or a TCP-only proxy: those cannot - impair QUIC. -- Prove teardown: cancel a run mid-drill and show the namespace, the qdiscs, - and every child process are gone, then rerun successfully on the same host. -- Assert the impairment actually applied before grading a drill. A profile that - silently failed to install turns an impaired run into an unimpaired pass. - -Acceptance: the three drills pass under a moderate profile and the recorded -baseline matches the requested one within a stated tolerance. A profile that -cannot install fails the run. Do not use retries to make intermittent failures green. +`rs/moq-relay/tests/drills.rs` already owns the scenarios: cancellation +under backpressure, relay death mid-group, and republish after an +interrupted publisher. They drive real QUIC over `127.0.0.1` inside one +process, relay and clients alike, so the impairment can be a userspace UDP +shaper the test itself owns: the client connects to the shaper's socket, +which forwards each datagram to the relay after the profile's treatment, and +back. That is not an HTTP interceptor or a TCP proxy, which cannot impair +QUIC; it is a datagram relay, and QUIC is indifferent to the extra hop. + +- A `Shaper` in the drill support code: per-direction delay and jitter, + loss, reorder, and a token-bucket rate limit, all driven by one seeded RNG + so a failing run's seed reproduces the same treatment. Kernel scheduling + still varies delivery timing; the seed makes the decisions reproducible, + not the clock. +- Assert the impairment applied before grading a drill: the shaper counts + what it dropped, delayed, and reordered, and a profile that treated nothing + fails the run. A profile that silently did nothing turns an impaired run + into an unimpaired pass. +- Record the profile, the seed, and the shaper's counters with the run's + artifacts. +- The drills pass unchanged under a moderate profile; add the impaired + variant as a second lane of each drill rather than a separate suite, so a + scenario cannot drift between the two. No retries to make an intermittent + failure green; a drill that only passes unimpaired is a finding. + +Kernel-real impairment (`netem` in a private network namespace) is out of +scope: it is Linux only, needs `CAP_NET_ADMIN`, and the drills test the +protocol's reaction to loss and delay, not the kernel's rendering of them. ## Related -- [Failure artifacts](/quest/m0/qa-failure-artifacts.md) - stores timelines, seeds, and traces -- [Runtime QA hosts](/quest/m2/runtime-qa-hosts.md) - provides Linux execution for the profile +- [Failure artifacts](/quest/m0/qa-failure-artifacts.md) - stores profiles, seeds, and traces diff --git a/quest/m1/2216-js-net-typed-announce-status.md b/quest/m1/2216-js-net-typed-announce-status.md deleted file mode 100644 index 542794cf609..00000000000 --- a/quest/m1/2216-js-net-typed-announce-status.md +++ /dev/null @@ -1,31 +0,0 @@ -# [S] js/net: an announcement event carries its typed status - -## Goal - -`Announced.Event` in js/net says whether a route is `active`, `restart`, or -`ended`, matching the three states Rust models and the wire carries. Today -the event is `{ prefix, active: boolean }`, so a restart is indistinguishable -from a steady active route and nothing downstream can react to it. - -## Plan - -Branch from dev: the event shape is public and this is breaking. - -- `js/net/src/announced.ts` `Event` gains a `status: "active" | "restart" | - "ended"` discriminant; `active` goes rather than lingering as a second - source of truth. The lite and IETF announce decoders map the wire states to - it, and the producer side (`origin.ts`, the announce handle from - [JS announce](/quest/m1/js-announce.md)) emits `restart` where Rust does. -- Consumers: `js/watch`, `js/publish`, `js/hang`, `demo/web`, and the - `doc/lib/js` pages that read `active`. -- Tests in `announced.test.ts` for each state and for a restart arriving on - a live route. - -## Closes - -- [#2216](https://github.com/moq-dev/moq/issues/2216) - close this issue when the quest finishes - -## Related - -- [JS announce](/quest/m1/js-announce.md) - the producer-side handle that emits these states -- [#2318](/quest/m1/2318-js-net-remaining-capability-gaps-vs-rs-moq-net-setup-role.md) - the other js/net gaps diff --git a/quest/m1/README.md b/quest/m1/README.md index f4e6d76b85e..39a2a15f9d7 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -52,12 +52,13 @@ with the current dev tree before starting. - [#3002](/quest/m1/3002-no-test-drives-a-late-group-through-the-ietf-dispatch-loop.md) - No test drives a late group through the IETF dispatch loop - [#3187](/quest/m1/3187-preserve-structured-protocol-error-codes-across-ffi-and-c.md) - Preserve structured protocol error codes across FFI and C bindings - [#2318](/quest/m1/2318-js-net-remaining-capability-gaps-vs-rs-moq-net-setup-role.md) - js/net: remaining capability gaps vs rs/moq-net (SETUP role, finish_at and final sequence, range controls, typed errors) -- [#2216](/quest/m1/2216-js-net-typed-announce-status.md) - js/net: an announcement event carries its typed status instead of an active boolean +- [Close classification](/quest/m1/js-close-classification.md) - a browser consumer tells a requested end from a fault, so the media harness fails on real errors during a transition - [#2774](/quest/m1/2774-collapse-reload-and-shared-into-one-connection-class.md) - Collapse Reload and Shared into one Connection class - [#2870](/quest/m1/2870-moq-hls-a-named-sibling-rendition-is-pinned-too-late-to.md) - moq-hls: a named sibling rendition is pinned too late to survive a same-path republish - [#2075](/quest/m1/2075-mirror-catalog-reservation-gating-in-moq-hang-js-hang.md) - Mirror catalog reservation gating in @moq/hang (js/hang) - [#933](/quest/m1/933-video-rotation-metadata-not-propagated-from-mobile-camera.md) - Video rotation metadata not propagated from mobile camera publish to watch renderer - [#3056](/quest/m1/3056-watch-video-decoder-captures-the-rewind-generation-at.md) - watch: video decoder captures the rewind generation at output time, not submit time +- [Plan: A/V clock](/quest/m1/plan-av-clock.md) - settle how the audio playhead drives Sync.reference while audio plays, then build it - [Config provenance](/quest/m1/config-provenance.md) - the merge records which source set a value, so TOML survives CLI defaults and empty lists, and env outranks the file - [Cluster construction](/quest/m1/cluster-construction.md) - construct one stable origin after its cache settings are known, deleting the rebuilding builder - [Abort on drop](/quest/m1/abort-on-drop.md) - one abort-on-drop task guard per crate, replacing five copies of the same Drop diff --git a/quest/m1/js-close-classification.md b/quest/m1/js-close-classification.md new file mode 100644 index 00000000000..22ea948cf9e --- /dev/null +++ b/quest/m1/js-close-classification.md @@ -0,0 +1,51 @@ +# [M] A browser consumer tells a requested end from a fault without reading message text + +## Goal + +Cutting a publisher off, or pointing `` at another broadcast, +produces a classified end in `@moq/hang` and `@moq/watch` rather than a +`console.error` a caller has to pattern-match. A task whose stream was +cancelled has not failed and never reaches a catch-all logger. The browser +harness then fails on any unclassified error during a transition instead of +tolerating every error the page reports. + +## Plan + +Branch from dev, which already has what main lacks: `js/net` on dev carries +`StreamError` and `SessionError` with code registries, and `StreamCode.Cancel` +is the routine unsubscribe. On main only `RemoteError` exists, with code `0` +whenever the transport dropped the stream without one, so the classification +cannot be built there. + +Today every path is the same path. A subscription aborted by a name change +or a departing publisher rethrows out of the `js/hang` group reader, lands in +`Effect.spawn`'s catch-all in `js/signals`, and prints `spawn error` plus +whatever the session constructed. `js/watch/src/video/decoder.ts` logs a bare +`DOMException` with no prefix and carries `// TODO bubble up error`. The +consequence is a test that cannot be strict: `test/smoke/clients/js/media.ts` +drives unsubscribe/rejoin and publisher-stop/republish, both of which abort +subscriptions on purpose, so `waitFrozen(.., tolerateErrors)` drains every +page error for the duration of the transition and a genuine decoder or +transport fault reads exactly like the abort the case asked for. + +- Decide what `@moq/hang` and `@moq/watch` expose when a track, broadcast, + or session ends by request: the dev `StreamError` with `Cancel` reaching + the consumer as a clean end, and everything else as a fault. Whether hang + and watch re-export the net types or wrap them is the open question; pick + the shape that mirrors `rs/moq-net`'s `Error::Cancel` plus `finish` versus + `abort`. +- Stop `Effect.spawn` being the reporting path for an expected end: a + cancelled task resolves, it does not throw into the catch-all. +- Give the video decoder's `error` callback the same treatment, so the one + unprefixed `console.error` in the player stops being unclassifiable. The + truncated group a departing publisher leaves behind reaches the decoder as + the same `DOMException` a broken decoder would, so the classification has + to come from the stream's end, not the exception. +- Tighten `media.ts`: tolerate the classified end, fail immediately on + anything else, and delete the blanket drain in `harness.ts`. + +## Related + +- [#2318](/quest/m1/2318-js-net-remaining-capability-gaps-vs-rs-moq-net-setup-role.md) - the other js/net gaps against the Rust model +- [Media QA on other engines](/quest/m2/browser-media-qa-engines.md) - the same harness, another axis +- [Failure artifacts](/quest/m0/qa-failure-artifacts.md) - shared trace and sample output diff --git a/quest/m1/perf/README.md b/quest/m1/perf/README.md index 5f439737da3..1bd8f1073e2 100644 --- a/quest/m1/perf/README.md +++ b/quest/m1/perf/README.md @@ -48,7 +48,7 @@ backend we ship today. - [#3199](/quest/m1/perf/3199-moq-uring-remove-sq-indirection-and-per-enter-ring-fd.md) - moq-uring: remove SQ indirection and per-enter ring fd lookup - [#3200](/quest/m1/perf/3200-moq-uring-batch-completion-wakeups-with-min-timeout.md) - moq-uring: batch completion wakeups with MIN_TIMEOUT - [#3129](/quest/m1/perf/3129-moq-uring-write-the-webtransport-stream-header-at-open.md) - moq-uring: write the WebTransport stream header at open time, so finish() never owes one -- [Driver allocs](/quest/m1/perf/driver-allocs.md) - the egress driver stops heap-allocating per train and per pump, and its one-train-per-turn cadence becomes a measured knob +- [Egress requeue](/quest/m1/perf/egress-requeue.md) - a requeue for the transmit pool stops re-walking every ready stream, and trains per turn becomes a measured budget - [#3201](/quest/m1/perf/3201-moq-uring-use-sendmsg-zc-for-large-udp-gso-trains.md) - moq-uring: use SENDMSG_ZC for large UDP GSO trains - [#3202](/quest/m1/perf/3202-moq-uring-use-fixed-file-slots-for-worker-udp-sockets.md) - moq-uring: use fixed-file slots for worker UDP sockets - [#3204](/quest/m1/perf/3204-moq-uring-register-tx-pool-buffers-for-zero-copy-sends.md) - moq-uring: register TX-pool buffers for zero-copy sends diff --git a/quest/m1/perf/driver-allocs.md b/quest/m1/perf/driver-allocs.md deleted file mode 100644 index 9b25fc559db..00000000000 --- a/quest/m1/perf/driver-allocs.md +++ /dev/null @@ -1,44 +0,0 @@ -# [S] Stop allocating per train and per pump in the egress driver - -## Goal - -Three steady-state costs in moq-uring's egress and completion loops: - -- `TxBuf::send` heap-allocates a `Box` (plus an `Rc` and - a slab insert) for every `sendmsg` train. -- `pump_inner` allocates a fresh `Vec` for every completion batch. -- `Driver::flush` stages one GSO train (up to 63 segments) per worker turn - and then self-wakes, so a connection with a deep backlog needs one full - loop turn per train. - -Kill the per-train and per-pump allocations, and turn the one-train-per-turn -cadence into a measured knob instead of a hardcoded fairness choice. - -## Plan - -- Pool the `SendHdr` allocations alongside the TX buffer leases they - describe: the pool already owns stable per-worker buffers, so the header - block can live with the buffer slot and be reused. -- Keep a persistent CQE scratch buffer on the worker, drained per pump, - instead of allocating per batch. -- Add a trains-per-turn budget to `Driver::flush`. The current - one-train-then-requeue shape is deliberate fairness across connections on - the shared socket; keep fairness by bounding the budget, and sweep 1, 2, - and 4 trains per turn under the fanout and single-heavy-connection shapes - to see whether the extra turn latency is real. -- Separate "there is more to send" from "readiness changed". The self-wake - after every train re-runs the whole of `Driver::poll`, walking quiche's - `readable()`/`writable()` iterators and removing and re-registering a - waiter per ready stream per train (#3120); a requeue for the transmit pool - should not re-walk readiness, or only the streams whose readiness changed - since the last turn should be woken. `state.finishing.retain(..)` calling - `stream_capacity` per finishing stream per turn goes with it. -- Each change stays independently ablatable. - -Acceptance: allocations per train and per pump (heaptrack or a debug -counter), CPU per Gbps and throughput ceiling via `just bench BASE` on -Linux. Latency must not regress at the swept budget. - -## Closes - -- [#3120](https://github.com/moq-dev/moq/issues/3120) - close this issue when the quest finishes diff --git a/quest/m1/perf/egress-requeue.md b/quest/m1/perf/egress-requeue.md new file mode 100644 index 00000000000..1f584a1e0ec --- /dev/null +++ b/quest/m1/perf/egress-requeue.md @@ -0,0 +1,39 @@ +# [S] The egress driver's requeue stops re-walking every ready stream + +## Goal + +Sending a deep backlog on one connection no longer re-runs the whole of +`Driver::poll` per GSO train. The driver separates "there is more to send" +from "readiness changed", so a requeue for the transmit pool walks nothing, +and the one-train-per-turn cadence becomes a measured budget instead of a +hardcoded fairness choice. + +## Plan + +Branch from dev. Profiled in #3120: `kio::waiter::WaiterList::register` is +the hottest symbol on the io_uring relay (4.6% video, 6.7% chat), because +`Connection::flush` stages one train (up to 63 segments) and then wakes +itself, and each of those turns walks quiche's `readable()` and `writable()` +iterators, removing and re-registering a waiter per ready stream. The same +sweep calls `stream_capacity` for every finishing stream per turn. + +- Separate the requeue from readiness. After a train, the driver asks only + for another transmit turn; the readiness walk runs when quiche reports a + readiness change, or only for the streams whose readiness changed since + the last turn. `state.finishing.retain(..)` stops calling + `stream_capacity` per finishing stream per turn. +- Add a trains-per-turn budget to `Driver::flush`. The one-train-then-requeue + shape is deliberate fairness across connections on the shared socket; keep + fairness by bounding the budget, and sweep 1, 2, and 4 trains per turn + under the fanout and single-heavy-connection shapes to see whether the + extra turn latency is real. +- Both changes stay independently ablatable, and the quinn driver gets the + same treatment where it has the same shape. + +Acceptance: `WaiterList::register` share in the `perf` profile, CPU per Gbps +and throughput ceiling via `just bench BASE` on Linux. Latency must not +regress at the swept budget. + +## Closes + +- [#3120](https://github.com/moq-dev/moq/issues/3120) - close this issue when the quest finishes diff --git a/quest/m1/plan-av-clock.md b/quest/m1/plan-av-clock.md new file mode 100644 index 00000000000..2e7750a2cb2 --- /dev/null +++ b/quest/m1/plan-av-clock.md @@ -0,0 +1,44 @@ +# [S] Plan: the audio playhead drives video sync while audio plays + +## Goal + +A written decision, recorded in this file and then executed as its own quest, +on how `js/watch` keeps audio and video in sync. Today the audio ring is +depth-driven and free-running on the AudioContext clock while video paces +against `Sync.reference`, a wall clock anchored at the earliest arrival. +Nothing ties them; A/V sync is emergent from both targeting the same delay, +so a ring that re-buffers or skips drifts against video until the next +re-anchor. The outcome is an audio-master clock: `Sync.reference` derives +from the audio playhead while audio plays and falls back to the wall clock +when muted or video-only. + +## Plan + +Branch from dev; the `Sync` inputs are public `@moq/watch` API and #3517 +already reshaped them there. + +Two API shapes were on the table on 2026-09-07 and neither was chosen: + +- Per-track handles: `sync.track("audio")` / `sync.track("video")`, each + reporting its own advertised delay and measured spread and one of them + nominated as the clock source. Recommended then, because it removes the + `audio`/`video`/`audioSpread`/`videoSpread` quartet of inputs and lets a + third track (text) join without another pair. +- `delay` as a floor only: keep the flat inputs, drop `"auto"`, and let the + audio playhead be the reference whenever an audio track is active. + +Decide with the estimator in hand: what the audio decoder can expose as its +playhead across the isolated and postMessage ring paths, how the video +decoder's per-frame `sync.wait()` reads it without a cross-thread hop per +frame, and what happens at the transitions (mute, audio track end, the ring +re-stalling). Record the verdict here, re-title this file as the +implementation quest, and re-estimate it. + +## Required + +- PR #3517 has merged to `dev` + +## Related + +- [Auto latency](/quest/m0/3477-watch-auto-latency.md) - the estimator this sits on +- [Time stretch](/quest/m2/watch-audio-time-stretch.md) - stretching needs a clock to converge toward diff --git a/quest/m2/3489-ts-import-stream-liveness.md b/quest/m2/3489-ts-import-stream-liveness.md index a74a2bea36f..d6990a5100e 100644 --- a/quest/m2/3489-ts-import-stream-liveness.md +++ b/quest/m2/3489-ts-import-stream-liveness.md @@ -21,17 +21,16 @@ audio half is visible, late, through #3372's resync line. that an absent PID is healthy. Give every elementary stream a row and retire the absence contract; the `lost.then_some(stats)` gate goes with it. `StreamStats` is `#[non_exhaustive]`, so the new fields are additive. -- Clock: parse PCR in the packet loop that already reads the adaptation-field - flags for duplicate detection. `last_pts` advances on video PES only and - freezes with the very stream this must catch. Record the clock at each - access unit in `flush`; the gap is the current clock minus that mark. Land - the adaptation-field parse once, shared with +- Clock: the packet loop reads the adaptation field for the discontinuity + indicator and skips the PCR bytes; parse the PCR there. `last_pts` advances + on video PES only and freezes with the very stream this must catch. Record + the clock at each access unit in `flush`; the gap is the current clock + minus that mark. Land the adaptation-field parse once, shared with [TS timebase discontinuity](/quest/m0/ts-forward-discontinuity.md). - Surface: `rs/moq-cli/src/publish.rs` `log_stats` gains a line for a stream whose count stopped advancing across a sample, distinct from the audio - resync message. Add a `stats` forwarder to `ContainerImpl` - (`rs/moq-mux/src/import/container.rs`) and a call in `rs/moq-srt/src/ts.rs`, - so SRT contribution reports the same counters instead of nothing. + resync message. The SRT gateway reports nothing today; that surface is + [SRT import stats](/quest/m2/srt-import-stats.md). - Name and shape the counters so the TR 101 290 quest adopts them as its `PID_error` check, and leave the catalog `stalled` bit alone; that is the ladder and publisher-stats work. @@ -45,6 +44,7 @@ audio half is visible, late, through #3372's resync line. ## Related +- [SRT import stats](/quest/m2/srt-import-stats.md) - the same rows read from the SRT gateway - [#1838](/quest/m3/1838-tr-101-290-monitoring-requirements-broadcast-contribution.md) - the monitoring model that subsumes this as `PID_error` - [Publisher stats](/quest/m2/qos/publisher-stats.md) - where per-rendition liveness would ride the catalog - [TS timebase discontinuity](/quest/m0/ts-forward-discontinuity.md) - shares the adaptation-field parse diff --git a/quest/m2/README.md b/quest/m2/README.md index 7f4fd701de0..abb085fbc7a 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -60,6 +60,7 @@ can act on. Each still carries its own plan and regression test. - [#2779](/quest/m2/2779-moq-export-ts-continuity-counters-are-numbered-from.md) - moq export ts: continuity counters are numbered from process state, so two exporters of the same broadcast emit streams that can never be compared - [#2829](/quest/m2/2829-moq-export-ts-the-audio-video-interleave-is-decided-by.md) - moq export ts: the audio/video interleave is decided by arrival timing, so two exporters of one broadcast render the same media in different orders - [#3489](/quest/m2/3489-ts-import-stream-liveness.md) - moq import ts: every elementary stream reports its access units and how long it has been quiet +- [SRT import stats](/quest/m2/srt-import-stats.md) - the SRT gateway reports the same per-stream counters instead of nothing - [Text availability](/quest/m2/text-schema.md) - a text track publishes its own coverage index instead of copying the media timeline - [SRT metadata parity](/quest/m2/srt-metadata.md) - the SRT publisher preserves MPEG-TS metadata byte-faithfully like the CLI importer - [ID3 catalog section](/quest/m2/id3.md) - timed ID3 as a first-class container-neutral catalog section @@ -82,6 +83,7 @@ can act on. Each still carries its own plan and regression test. - [#709](/quest/m2/709-automatic-letsencrypt-support.md) - the relay provisions and renews its own ACME certificate over HTTP-01, persisted on disk - [Room SDK](/quest/m2/room-sdk.md) - a headless room package: a room is a path prefix, no service, no storage - [Runtime QA hosts](/quest/m2/runtime-qa-hosts.md) - run exact source snapshots on accessible Linux and device hosts with retrievable debug evidence +- [Media QA on other engines](/quest/m2/browser-media-qa-engines.md) - the media harness measures a Firefox or WebKit player over the fallback and names what each engine lacks - [LiveKit shim](/quest/m2/livekit-shim.md) - a drop-in livekit-client-compatible package running rooms over MoQ - [Auth verdict](/quest/m2/auth-verdict.md) - the relay hands an opaque credential to its auth API and is told the grant; lands as the proxy mode in #3044 - [#3087](/quest/m2/3087-relay-mtls-peers-bypass-auth-api-mode-so-proxy-grants.md) - relay: mTLS peers bypass the auth API mode, so a proxy grant cannot refuse or scope them diff --git a/quest/m2/audio-codecs/README.md b/quest/m2/audio-codecs/README.md index bcc36d6d0b6..ebbabb185d0 100644 --- a/quest/m2/audio-codecs/README.md +++ b/quest/m2/audio-codecs/README.md @@ -35,11 +35,13 @@ mono/stereo. The layout and seam quests are independent and come first; each platform then lands as its own decode and encode quest so verification stays per host. The -HE-AAC refusal is a defect in what ships today and is ready now. +HE-AAC refusal and the PCE parse are defects in what ships today and are +ready now. ## Quests - [HE-AAC refusal](/quest/m2/audio-codecs/he-aac-refusal.md) - implicit-SBR HE-AAC over TS is refused instead of half-decoded as the LC core +- [AAC PCE](/quest/m2/audio-codecs/aac-pce.md) - a channel_config of 0 parses the program config element instead of guessing stereo - [Layout](/quest/m2/audio-codecs/layout.md) - a `Layout` type in one canonical order carries up to 7.1 through decode, resample, playback, and the FFI - [Decode seam](/quest/m2/audio-codecs/decode-backend.md) - `decode::backend` selects a platform decoder before symphonia, mirroring moq-video - [AudioToolbox decode](/quest/m2/audio-codecs/decode-audiotoolbox.md) - macOS and iOS decode HE-AAC, multichannel AAC, and what else the framework offers diff --git a/quest/m2/audio-codecs/aac-pce.md b/quest/m2/audio-codecs/aac-pce.md new file mode 100644 index 00000000000..d5eab1a4906 --- /dev/null +++ b/quest/m2/audio-codecs/aac-pce.md @@ -0,0 +1,34 @@ +# [S] moq-mux: an AAC channel_config of 0 parses the PCE or refuses the track + +## Goal + +The catalog never invents an AAC channel count. A stream whose +channelConfiguration is 0 gets its count from the program config element it +carries, and a reserved value refuses the track with a clear error. Today +both map to stereo with a warning. + +## Plan + +The ADTS and ASC importers in `rs/moq-mux/src/codec/aac` map +`channel_config == 0` and every value from 8 to 15 to stereo and warn. +Warn-then-continue is banned: supported or refused. + +- ASC: with `channel_config == 0` the GASpecificConfig carries a + `program_config_element`; parse its front, side, back, and LFE element + counts into the channel count (and, once [Layout](/quest/m2/audio-codecs/layout.md) + lands, into a layout). +- ADTS: channel_config 0 means the PCE is in the first raw data block. Parse + it there, once per track, the same way the HE-AAC sniff reads the first + block's fill elements. +- Give every ASC configuration from 8 to 15 an explicit disposition: implement + any supported channel mapping and return `Error::Unsupported` for every + remaining value, including reserved configurations. +- Regression: an ASC fixture with a PCE reports its real count; an ADTS + fixture with an in-band PCE does too; each value from 8 to 15 has its + supported count asserted or is refused; the + common configurations 1 to 7 are unchanged. + +## Related + +- [HE-AAC refusal](/quest/m2/audio-codecs/he-aac-refusal.md) - reads the same first block +- [Layout](/quest/m2/audio-codecs/layout.md) - what the parsed PCE eventually maps to diff --git a/quest/m2/audio-codecs/he-aac-refusal.md b/quest/m2/audio-codecs/he-aac-refusal.md index 6968965899b..4a82821ab3d 100644 --- a/quest/m2/audio-codecs/he-aac-refusal.md +++ b/quest/m2/audio-codecs/he-aac-refusal.md @@ -28,11 +28,7 @@ does too. the sniff belongs to the symphonia backend only; the OS decoders read SBR in band themselves. -Also fix the two places the catalog invents a channel count: the ADTS and -ASC importers in `rs/moq-mux/src/codec/aac` map `channel_config == 0` (a -program config element) and reserved values to stereo with a warning. -Warn-then-continue is banned: parse the PCE or refuse the track. - ## Related - [Decode seam](/quest/m2/audio-codecs/decode-backend.md) - the sniff becomes the software backend's contract +- [AAC PCE](/quest/m2/audio-codecs/aac-pce.md) - the other place the catalog lies about an AAC stream diff --git a/quest/m2/browser-media-qa-engines.md b/quest/m2/browser-media-qa-engines.md new file mode 100644 index 00000000000..08dc1c487c0 --- /dev/null +++ b/quest/m2/browser-media-qa-engines.md @@ -0,0 +1,39 @@ +# [M] Run the media QA harness with a Firefox or WebKit player + +## Goal + +The media output and lifecycle checks measure a player on more than the +pinned Chromium, and an engine that cannot run a case says so by name +instead of passing by omission. A green run states which engines were +measured and which APIs each one lacked. + +## Plan + +`just test smoke-media` measures presented frame progress, tone presence, +and audio/video skew against a deterministic fixture, but `harness.ts` +launches only Playwright's Chromium and nothing installs another engine. Its +capability probe already enumerates what the player needs (WebTransport, +WebCodecs encode and decode, AudioWorklet, `MediaStreamTrackProcessor`, +`canvas.captureStream`) and fails naming what is missing. + +Neither Playwright Firefox nor Playwright WebKit ships WebTransport, and +WebKit also lacks the WebCodecs encoder, so neither can publish and neither +can play over QUIC. What they can do is play over the WebSocket fallback, +which is the path a real Firefox viewer takes today. + +- Split the harness's publisher and subscriber roles so an engine can take + one side: a Chromium fixture with a Firefox or WebKit player over the + fallback is the case worth having. +- Install the engines the run asks for and report a per-engine matrix. An + engine that cannot publish is not a skipped cell; it is a stated limit with + the missing API named, and a green run says which engines it measured. +- Keep the tolerances shared. A skew bound that only holds on one engine is + a measurement of that engine, not of the player. +- Playwright WebKit is not Safari. Say so in the report; a Safari defect such + as #2812 still needs a manual run. + +## Related + +- [Runtime QA hosts](/quest/m2/runtime-qa-hosts.md) - where an engine that will not run on the CI image runs +- [Failure artifacts](/quest/m0/qa-failure-artifacts.md) - shared trace and sample output +- [Close classification](/quest/m1/js-close-classification.md) - the same harness, another axis diff --git a/quest/m2/mobile/README.md b/quest/m2/mobile/README.md index 9d711daf56c..046c34b35fe 100644 --- a/quest/m2/mobile/README.md +++ b/quest/m2/mobile/README.md @@ -10,16 +10,18 @@ video consumer, and capture exists on neither platform. ## Plan -Order follows #700's phases. The codec-only MVP first: a video consumer in -`moq-ffi` closes the one gap that blocks an app from playing a broadcast at -all. Then the ownership decision, because it decides whether the capture -backends below are built in Rust or left to Swift and Kotlin, and the -Android one is an XL bet. Capture and the Dart device proof follow. +The ownership decision comes first: it decides what a decoded frame is at +the FFI (a byte array or a native surface handle), and a consumer shipped +one way and reshaped the other is a breaking change in five bindings. It +also decides whether the capture backends below are built in Rust or left +to Swift and Kotlin, and the Android one is an XL bet. The video consumer +follows as the codec-only MVP #700 scoped; capture and the Dart device proof +follow that. ## Quests -- [FFI video consumer](/quest/m2/mobile/ffi-video-consumer.md) - an app decodes a subscribed video rendition through moq-ffi on every binding that ships codecs - [Ownership boundary](/quest/m2/mobile/ownership-boundary.md) - settle whether Rust or the platform owns capture, codecs, and rendering before growing either +- [FFI video consumer](/quest/m2/mobile/ffi-video-consumer.md) - an app decodes a subscribed video rendition through moq-ffi on every binding that ships codecs - [iOS capture](/quest/m2/mobile/video-ios.md) - moq-video captures the camera and screen on iOS, reusing the VideoToolbox backend - [Android capture](/quest/m2/mobile/video-android.md) - Camera2, MediaProjection and MediaCodec, a whole NDK/JNI backend family - [Dart on iOS](/quest/m2/mobile/dart-ios.md) - prove the shipped iOS native asset actually loads on a device, which no CI can diff --git a/quest/m2/mobile/ffi-video-consumer.md b/quest/m2/mobile/ffi-video-consumer.md index df5adbc211c..44160c7de1a 100644 --- a/quest/m2/mobile/ffi-video-consumer.md +++ b/quest/m2/mobile/ffi-video-consumer.md @@ -12,12 +12,18 @@ VideoToolbox and openh264 backends; nothing joins them. ## Plan +The decoded consumer below applies only if the ownership decision selects +Rust-owned codecs. If Swift and Kotlin own codecs, rescope this quest to +the existing encoded-access-unit boundary and abandon any redundant Rust +decode work before implementation. + - Add `MoqVideoConsumer` beside `MoqAudioConsumer` in `rs/moq-ffi/src`, selecting a rendition from the catalog, decoding with `moq-video::decode`, and delivering frames through the same callback-free handle shape the audio - consumer uses. Frames cross the boundary as I420 or RGBA byte arrays with - their timestamp and dimensions, the MVP #700 scoped; a zero-copy native - surface waits on the ownership decision. + consumer uses. Frames cross the boundary in the shape the ownership + decision chose: I420 or RGBA byte arrays with timestamp and dimensions if + the platform owns rendering, an opaque native surface handle if Rust does. + Choose once; reshaping it later breaks every binding. - Walk the Cross-Package Sync table: `rs/libmoq` and `moq.h`, the `py`, `swift`, `kt`, and `dart` wrappers, the Go wrapper, and `doc/lib` for each. The Dart binding alone has no codecs, so document the omission there rather @@ -25,8 +31,11 @@ VideoToolbox and openh264 backends; nothing joins them. - Verify on an iOS simulator and an Android emulator with the smoke media, and on macOS through libmoq so `just test smoke-full` covers it. +## Required + +- [Ownership boundary](/quest/m2/mobile/ownership-boundary.md) - decides what a frame is at the FFI + ## Related -- [Ownership boundary](/quest/m2/mobile/ownership-boundary.md) - decides whether frames stay byte arrays - [Decoder drain](/quest/m3/decode-drain.md) - flush and finish for pipelined decoders this will lean on - [Dart codec parity](/quest/m2/dart-codecs.md) - the one binding that cannot decode until it lands diff --git a/quest/m2/mobile/ownership-boundary.md b/quest/m2/mobile/ownership-boundary.md index 867ea3a72e7..f47bc10f006 100644 --- a/quest/m2/mobile/ownership-boundary.md +++ b/quest/m2/mobile/ownership-boundary.md @@ -25,4 +25,4 @@ capture quests in this questline start only once this is settled. ## Related -- [FFI video consumer](/quest/m2/mobile/ffi-video-consumer.md) - ships with byte-array frames either way +- [FFI video consumer](/quest/m2/mobile/ffi-video-consumer.md) - the first quest that ships a frame across the boundary, so it waits on this diff --git a/quest/m2/runtime-qa-hosts.md b/quest/m2/runtime-qa-hosts.md index 96a7f153516..5e8f3891708 100644 --- a/quest/m2/runtime-qa-hosts.md +++ b/quest/m2/runtime-qa-hosts.md @@ -41,4 +41,3 @@ is available, split that rollout into an explicit externally blocked follow-up. - [Video hardware validation](/quest/m3/video-hardware.md) - owns real encoder/capture/zero-copy verdicts - [Dart on iOS](/quest/m2/mobile/dart-ios.md) - owns simulator/device packaging validation -- [Impaired-path profile](/quest/m0/transport-impairment-profile.md) - consumes the Linux runner diff --git a/quest/m2/srt-import-stats.md b/quest/m2/srt-import-stats.md new file mode 100644 index 00000000000..cefe599a38a --- /dev/null +++ b/quest/m2/srt-import-stats.md @@ -0,0 +1,21 @@ +# [S] moq-srt reports the same import stats as moq import ts + +## Goal + +An operator running the SRT gateway sees the TS importer's per-stream +counters (access units, quiet gap, resyncs) the way `moq import ts` logs +them, instead of nothing. SRT contribution is where a silent PID matters +most, and today `rs/moq-srt/src/ts.rs` only calls `decode`. + +## Plan + +- Poll `self.importer.stats()` on the SRT publisher's existing + `ts::Import` at the same cadence as the CLI. +- Share the logging presentation with `rs/moq-cli/src/publish.rs` where + needed so the two front doors report the same rows. +- Test: the SRT harness feeds the suppressed-PID stimulus from the TS + liveness quest and asserts the gateway reports the stalled row. + +## Required + +- [#3489](/quest/m2/3489-ts-import-stream-liveness.md) - the rows this forwards diff --git a/quest/m2/watch-audio-time-stretch.md b/quest/m2/watch-audio-time-stretch.md index b72dff964f3..93b140d273c 100644 --- a/quest/m2/watch-audio-time-stretch.md +++ b/quest/m2/watch-audio-time-stretch.md @@ -9,11 +9,14 @@ audio or rendering silence. Convergence is inaudible at ordinary drift and burst sizes. Boundaries: no packet loss concealment; an underrun still renders a ramped -gap. The target estimator and the ring's hysteresis come from -[Auto latency](/quest/m0/3477-watch-auto-latency.md). +gap. The target estimator and the ring's slack and re-stall are #3517 on +dev; the clock the stretch converges toward is +[Plan: A/V clock](/quest/m1/plan-av-clock.md). ## Plan +Branch from dev. + - Implement WSOLA-style stretch and compress in `render-worklet.ts` on the PCM the ring hands out, bounded to a few percent per quantum, driven by the distance between buffered audio and the target. Skip-ahead remains only @@ -26,4 +29,6 @@ gap. The target estimator and the ring's hysteresis come from ## Required -- [Auto latency](/quest/m0/3477-watch-auto-latency.md) - the target and the ring slack this converges toward +- [Plan: A/V clock](/quest/m1/plan-av-clock.md) - stretching against a free-running ring only moves the drift + +- [Auto latency](/quest/m0/3477-watch-auto-latency.md) - the recorded traces this replays diff --git a/test/drill/README.md b/test/drill/README.md index da63a28a99c..f2fb5a0975f 100644 --- a/test/drill/README.md +++ b/test/drill/README.md @@ -106,8 +106,8 @@ corpus nobody replays. ## Not covered here - An impaired path (delay, loss, rate limits). Loopback is the only path these - drills see; `quest/m0/transport-impairment-profile.md` adds the Linux - network-namespace profile they run under. + drills see; `quest/m0/transport-impairment-profile.md` adds the seeded UDP + shaper they run under. - CI lane scheduling. - Failure bundles beyond what the test harness prints, which belongs to `quest/m0/qa-failure-artifacts.md`. diff --git a/test/smoke/clients/js/harness.ts b/test/smoke/clients/js/harness.ts index 1c027fbdfb7..5b24b908586 100644 --- a/test/smoke/clients/js/harness.ts +++ b/test/smoke/clients/js/harness.ts @@ -132,7 +132,7 @@ export function throwPageErrors(errors: BrowserErrors): void { * group reaches the video decoder as the same bare `DOMException` a broken decoder would. Nothing is * concealed - `open` echoes every console message and page error as it arrives - but during a * transition the two are indistinguishable. Classifying them is - * `/quest/m0/js-teardown-error-classification.md`, which ends with tightening this. + * `/quest/m1/js-close-classification.md`, which ends with tightening this. */ export function drainPageErrors(errors: BrowserErrors): string[] { return errors.page.splice(0).concat(errors.console.splice(0));