Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
710ae81
chore(quest): claim quest/m1/push-to-talk/README.md
kixelated Nov 3, 2026
cb02a1d
test(publish): measure push-to-talk across an impaired path
kixelated Nov 3, 2026
d58de97
test(publish): record the measured browser version
kixelated Nov 3, 2026
12163a2
style(test): format push-to-talk evidence output
kixelated Nov 3, 2026
144910e
test(publish): format runner and record the open recovery decision
kixelated Nov 3, 2026
f0b78ac
test(publish): preserve valid measurement evidence and record blockers
kixelated Nov 3, 2026
abd94b2
test(publish): fix local review findings in the measurement harness
kixelated Nov 3, 2026
3d3619a
Merge remote-tracking branch 'origin/main' into ptt-readme
kixelated Nov 4, 2026
320b2b1
test(publish): grade push-to-talk playout on top of one-way delivery
kixelated Nov 4, 2026
8a3f4a5
quest(push-to-talk): grade the merged head's runs against the restate…
kixelated Nov 4, 2026
0d9f90c
test(publish): split push-to-talk playout at the render onset
kixelated Nov 4, 2026
814f2f8
test(publish): read the push-to-talk render onset from the render clock
kixelated Nov 4, 2026
18250bb
quest(push-to-talk): grade on the render onset and count the tail race
kixelated Nov 4, 2026
ef9f48d
Merge remote-tracking branch 'origin/main' into ptt-readme
kixelated Nov 4, 2026
663eb94
test(publish): measure push-to-talk on an ideal path and split its la…
kixelated Nov 4, 2026
1cf73e5
test(publish): split the push-to-talk render hand-off from the jitter…
kixelated Nov 4, 2026
774548f
quest: finish push-to-talk and plan its latency follow-up
kixelated Nov 4, 2026
d2d74e0
test(publish): keep and print push-to-talk evidence, and test the ana…
kixelated Nov 4, 2026
5bdb303
test(publish): route the push-to-talk listener through the relay
kixelated Nov 4, 2026
ce6ea0d
quest(push-to-talk): keep the epic for a head lost at the listener
kixelated Nov 4, 2026
2d6d6b7
Merge remote-tracking branch 'origin/main' into ptt-readme
kixelated Nov 4, 2026
3155ec7
docs(audio-quality): describe the push-to-talk jitter buffer and refu…
kixelated Nov 4, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -496,6 +496,13 @@ jobs:
HARNESS_RUNS: ${{ runner.temp }}/moq-test-runs
# A passing run is kept too: budgets.json is measured from these nights.
HARNESS_KEEP: 1
- name: Push-to-talk
if: ${{ !cancelled() }}
run: nix develop --command just test push-to-talk
shell: bash -leo pipefail {0}
env:
HARNESS_RUNS: ${{ runner.temp }}/moq-test-runs
HARNESS_KEEP: 1
# The native player's rows, graded but not enforced: an intermittent device
# stall voids about one AAC row in seven, which would fail most nights until
# it is fixed. Kept beside the browser run so its ceilings can be re-measured.
Expand Down
3 changes: 2 additions & 1 deletion quest/m1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,8 @@ per-frame CPU growth since #5926.
- [A/V clock](/quest/m1/av-clock.md) - the audio playhead drives Sync.reference while audio plays, through per-track sync handles
- [Container reads drop backfill](/quest/m1/read-backfill.md) - the JS container consumer drops groups below its read cursor like Rust
- [Buffered playback subscribes in ascending order](/quest/m1/buffered-ascending.md) - a buffered `<moq-watch>` asks for the oldest group first, so a voice reply's opening arrives first, and audio's head hold shrinks to the floor
- [Push-to-talk loses no speech](/quest/m1/push-to-talk/README.md) - consumers hold a disabled rendition, and an unmute encodes at once with the mic still open
- [Push-to-talk loses no speech](/quest/m1/push-to-talk/README.md) - consumers hold a disabled rendition, and an unmute encodes at once with the mic still open; a reordered head after a long mute is still lost at the listener
- [Push-to-talk's press-to-audible latency is chosen, not inherited](/quest/m1/push-to-talk-latency.md) - each part of an ideal-path press's 469 ms, chiefly the 300 ms delay and 115 ms device output, is reduced through its knob or kept with a reason
- [TS fidelity and recovery](/quest/m1/ts-reliability/README.md) - both modes have explicit profiles, automated validation, loss tests, and a dedicated guide
- [WebGPU on Safari](/quest/m1/webgpu-safari.md) - the WebGPU renderer is verified on Safari 26 for macOS and iOS
- [CI housekeeping](/quest/m1/ci-housekeeping.md) - remove the actionlint queue ignore once a release after 1.7.12 reaches the dev shell
Expand Down
75 changes: 75 additions & 0 deletions quest/m1/push-to-talk-latency.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# [S] Push-to-talk's press-to-audible latency is chosen, not inherited

## Goal

Each part of a push-to-talk press's latency on an ideal path, from capture to
device output, is either reduced through the knob that controls it or kept
with a recorded reason. `just test push-to-talk` reports the result with the
same breakdown.

## Plan

Measured 2026-11-04 by
[moq-dev/moq#7760](https://github.com/moq-dev/moq/pull/7760), for
[Push-to-talk](/quest/m1/push-to-talk/README.md). The setup was `just test
push-to-talk` through a local relay, publisher and listener on separate
sessions, with no shaping or loss, 60 ms Opus at 128 kbit/s, and a fixed 300
ms `delay`. Chromium 156 ran headless on Linux. There were five runs of 22
presses each, 15 of them separated, so 75 separated presses in all; one press
lost its head (see Push-to-talk) and has no breakdown, leaving 74.
`summary.json` `path` splits each press at page-clock boundaries that sum
exactly to press → projected output.

| Part | Boundaries | Median ms | Max ms |
| --- | --- | --- | --- |
| capture | worklet delivery and framing | 29 | 110 |
| packetization | media the head frame still needed after the press | 39 | 60 |
| encode | encoder input to container write | 0.8 | 14 |
| network | container write to arrival, through the relay | 3.2 | 15 |
| decode | arrival to decoded PCM | 0.7 | 13 |
| jitter buffer | head's PCM to the first PCM that filled the delay | 252 | 268 |
| render | that PCM to the render onset | 21 | 54 |
| device | render onset to projected output | 115 | 119 |
| total | press to projected output | 469 | 576 |

The capture and packetization split is approximate, but their 66 ms median sum
is exact. Playout on top of one-way delivery was 261 ms median and 294 ms max
over all 75, inside the 300 ms the maintainer set.

Biggest contributors and their knobs:

- **Playout delay, about 300 ms.** The ring starts once it holds the delay in
decoded PCM, counted from the head's first sample. That is why the jitter
buffer plus packetization sum to about 291 ms. Knob: `<moq-watch>` `delay`.
This test fixes it at 300 ms. In `"auto"`, the default, the target is
`max(measured, advertised) + frame + catalog delay`
(`js/watch/src/audio/latency.ts`).
- **Device output, about 115 ms.** This matches Chromium's declared
`baseLatency + outputLatency` on this headless Linux host. Knob: the watch
AudioContext's `latencyHint` (`"interactive"` in
`js/watch/src/audio/decoder.ts`), otherwise the platform. A real device,
such as `kixtop`'s macOS output, should be measured before acting on it.
- **Capture and render, about 52 ms together.** Both follow the AudioContext
callback cadence on this host. The capture graph also delivers 60 ms frames
in bursts, 25-109 ms apart. Knob: the capture AudioContext's `latencyHint`
(`js/publish/src/audio/capture.ts`), otherwise the platform.
- **Opus frame, 60 ms.** This test chooses it; the publisher defaults to
20 ms. Under a fixed delay it is absorbed by the delay. In `"auto"` it adds
one frame to the target. Knob: the encoder's `frameDuration`.

No part looked like a bug. On an ideal path, encode and decode take about 1 ms
and the relay hop about 3 ms, and the head is not trimmed as decoder warmup.
The one lost head is an ordering defect, owned by
[Push-to-talk](/quest/m1/push-to-talk/README.md), not a latency part. The
fixed delay bypasses the jitter estimator. In `"auto"`, the 46 ms median
per-press arrival-lateness spread, from capture bursts rather than the
network, would make the measured term nonzero; that is the estimator working
as specified in [Audio jitter
target](/quest/m1/audio-jitter-target/README.md).

Open: decide which knob to turn first, and whether to measure on a real
device before deciding.

## Related

- [Audio jitter target](/quest/m1/audio-jitter-target/README.md) - the `"auto"` target this latency feeds
70 changes: 54 additions & 16 deletions quest/m1/push-to-talk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,23 +36,61 @@ Decided 2026-10-28 (maintainer):
replaces a publisher head buffer, which was considered and dropped as the
same idea special-cased.

The epic's own work: once the children land, measure push-to-talk (22
presses, 300 ms delay) behind bufferbloat: a rate limit below the media
bitrate, behind a deep but finite queue whose overflow drops packets. That
gates acceptance. A held listener's unmute pays no catalog round trip and no
resubscribe, so unmute → audible is the path's one-way delay plus the 300 ms.
Every frame of the utterance is encoded and sent; any frames dropped come
only from queue overflow. Rerun the reporter's setup (100 ms RTT, 2% random
loss) only as a reproduction of the original numbers, not as acceptance.
Record both in the PR.

Also check failure recovery (found 2026-10-28 by
[moq-dev/moq#7157](https://github.com/moq-dev/moq/pull/7157)): unmuting used to
reopen the microphone, which made a new source and so recovered a failed
encoder. Capture now stays open, so only a device swap, a settings change or a
new `source` recovers one. Decide whether re-enabling should retry a failed
encoder, and record why.
The epic's own work: the acceptance proof `just test push-to-talk` (22
presses, 60 ms Opus, 300 ms delay). A held listener's unmute pays no catalog
round trip and no resubscribe, and every frame of each utterance is encoded,
sent, received, and decoded from its first frame.

Decided with the maintainer on
[moq-dev/moq#7760](https://github.com/moq-dev/moq/pull/7760):

- ✅ **Ideal conditions only** (2026-11-04): "don't use random packet loss
either, assume ideal conditions." Both elements reach a local relay
directly, on separate sessions so the media crosses it, with no shaping,
loss, or queueing. The bufferbloat and 2% loss profiles were removed.
- ✅ **Latency: at most 300 ms of playout on top of one-way delivery**
(2026-11-03): "We can't ever expect 300ms latency if it takes more than
300ms to deliver a packet." No clocked live start or other startup
machinery. Five ideal runs on 2026-11-04 held it on all 75 separated presses
(median 261 ms, max 294 ms). The breakdown and its knobs are in
[Push-to-talk latency](/quest/m1/push-to-talk-latency.md).
- ✅ **No encoder recovery on re-enable** (2026-11-03). Asked first why a
fatal encoder error is expected at all: it is not. No push-to-talk run
reported one, and no issue reports a publisher encoder failure; the latch
came from a review finding
([moq-dev/moq#5441](https://github.com/moq-dev/moq/pull/5441)). WebCodecs
fails an encoder only for a config the platform rejects, an internal codec
error, or reclamation, which Chromium applies only to inactive, backgrounded
hardware codecs; Opus is software. A failure stays latched until a device,
settings or `source` change.

The publisher's tail drop is fixed:
[moq-dev/moq#7725](https://github.com/moq-dev/moq/pull/7725) flushes the codec
before a disable's discontinuity. A frame submitted under 1 ms before release
used to be lost in 3 of 7 runs; it was lost in none of 12 since.

What remains is a lost head at the listener. In one of 110 presses over the
2026-11-04 relay runs (run 3, press 20, after a 60 s mute), the
head's group 918 and the next frame's group 919, written at +72 and +73 ms,
reached the listener reordered: 919 at +187 ms and 918 at +192 ms, about 115
ms in transit against the usual 3 ms. The listener decoded 919 at +191.7 ms.
Group 918 arrived 0.4 ms later and never reached the decoder, so the utterance
played from its second frame, 60 ms late. The JS ordered read still delivers a
group below its cursor as a late backfill
(`js/hang/src/container/consumer.ts`), so the head was most likely dropped as
late between the read and the decoder (`js/watch/src/audio/decoder.ts`); pin
that down first. After a declared end, the next group is known to be the end's
group plus one. Holding for it within the delay budget would keep the head
without adding startup latency, but weigh that against the no-window decision
in [Container reads drop backfill](/quest/m1/read-backfill.md).

The runner fails on a head that is received but never decoded, so the proof
catches this. A complete set of runs finishes this epic.

## Closes

- [#7101](https://github.com/moq-dev/moq/issues/7101) - a disabled audio rendition is deselected, so the start of the next utterance is lost

## Related

- [Container reads drop backfill](/quest/m1/read-backfill.md) - changes how the ordered read treats a group below its cursor, where this head was lost
48 changes: 48 additions & 0 deletions test/audio-quality/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,54 @@ changed during the live run. The observer only
records frames delivered under the player's maximum age, so a replay cannot recover groups the
transport already discarded.

## Push-to-talk

```bash
just test push-to-talk
```

This separate, roughly nine-minute run drives the shipped publish and watch elements in one Chromium
page. A constant tone loops through the fake microphone, encoded as 128 kbit/s Opus with 60 ms
frames. The listener holds a fixed 300 ms delay. After warmup and a mute, it measures 22 presses,
alternating two and three seconds: the first after the warmup's five-second mute, seven with no
intervening wait, seven after five seconds, and seven after sixty seconds. Both elements reach a
local relay directly over WebTransport, with no shaping, loss, or queueing, so every millisecond
measured belongs to the media pipeline rather than the path.

`presses.json` records captured chunks at each press, WebCodecs encoder and decoder inputs and
outputs, container writes, frame arrivals before reordering, and audible output onsets. Publisher,
receiver, and worklet observations share the page's monotonic clock. A listening worklet detects
each nonzero output onset. Its render time is mapped through the context's render clock, read with
the page clock when the port message lands, so a late message moves neither boundary; its projected
output comes from `getOutputTimestamp`. This measures the browser's projected device output, not
acoustic output from a physical speaker. Zero-gap presses can remain continuously audible, and a
later onset can belong to the preceding utterance. Their audible latency is always null. Only
separated presses have an identifiable onset.

`summary.json` matches inputs, outputs, writes, and arrivals by media timestamp, so a late arrival
is still attributed to its utterance. The command checks pipeline completeness, not the epic's
latency verdict. It refuses missing encoder inputs, a head that never reaches the listener or never
decodes, submitted frames that never encode or reach the container, written frames that never reach
the listener, reopened capture, a new encoder (a probe or a rebuilt codec) or subscription, and
separated utterances with no audible output. It also fails on a broken driver or invalid capture.

The latency goal is at most 300 ms of playout added on top of one-way delivery. For each separated
press, `delivery` is the slowest of the utterance's frames that arrived before its onset, measured
from that frame's first sample (its media offset after the press). `playout` is what remains of the
render onset after the press and that delivery, and `output` is the projected device output after
that onset. `path` follows the head frame from the press to projected output in parts that sum to
it: `capture` (worklet delivery and framing, after the media the frame still needed),
`packetization` (that media, placed on the media clock by the newest chunk captured before the
press), `encode` (codec and container write), `network` (write to arrival, through the relay),
`decode` (arrival to decoded PCM), `jitter_buffer` (the head's PCM to the first PCM that fills the
playout delay), `render` (that PCM to the render onset: the worklet hand-off and the context's
render callback), and `device`, as the stages below name them. `latency` and `path` give each one's
min, median and max.

The command prints `latency`, `path`, and the completeness counts, and keeps its run directory
even when it passes (set `HARNESS_KEEP=0` to drop it). Nightly runs it and keeps the raw evidence
alongside the audio quality rows.

## Traces

The shaper's profiles are synthetic. [`traces/`](traces) holds real arrivals, each graded as a
Expand Down
Loading
Loading