Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions bun.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@
"test/interop/clients/js-native",
"test/drain",
"test/receive-credit",
"test/stream-credit",
"test/video",
"test/p2p",
"test/qmux",
Expand Down
21 changes: 16 additions & 5 deletions quest/m1/stream-credit/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# [L] Stream credit by priority
# Stream credit by priority

## Goal

Expand Down Expand Up @@ -54,7 +54,18 @@ wire message and multi-group streams. The 2026-11-04 choice of takeover
before cancelling is superseded. ✅ m2 no longer requires this epic
(rejected: keep the gate); it was an ordering rule, not a code dependency.

This epic owns the end-to-end test: a browser reader and a Rust relay over
WebTransport, 300 live tracks of mixed priority, the highest-priority ones
receiving every group, and nothing hanging. It builds its own harness, since
takeover no longer supplies one.
The end-to-end test is `just test stream-credit` (`test/stream-credit/`), from
[moq-dev/moq#8652](https://github.com/moq-dev/moq/pull/8652): a Chromium or
Firefox tab reads 300 live tracks of mixed priority through `moq-relay` over
WebTransport, on lite-07 and lite-06. The highest-priority tier receives every
group, the tiers read in priority order, and on lite-07 the relay continues
paused groups. It runs locally only until
[browser-resets](/quest/m1/stream-credit/browser-resets.md) makes it pass
reliably, since about one lite-07 cell in 20 still fails.

Decided 2026-11-13 (implementer, #8652, unattended): a lite-06 reader may lose a
starved track's final group at the track's end, and that is lite-06's limit,
not a defect: lite-06 has no drops, so a reader cannot wait for a group the
relay may have shed. The harness checks the final group on lite-07 only.
Rejected: holding the relay's lite-06 SUBSCRIBE_FINISH until no group waits for
credit, a heuristic for a version lite-07 replaces.
46 changes: 46 additions & 0 deletions quest/m1/stream-credit/browser-resets.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# [L] A browser group stream reset reaches the reader

## Goal

`just test stream-credit` passes reliably on lite-07 in Chromium and Firefox,
and runs in the nightly interop workflow. Each group stream the relay resets
reaches the tab's reader as a reset with its code, so a YIELDED group pauses
and a subscription ends once its groups are accounted for.

## Plan

Decided 2026-11-13 (maintainer, the pick round for [moq-dev/moq#8652](https://github.com/moq-dev/moq/pull/8652)): ✅ keep this follow-up (rejected: drop it and leave the harness a manual recipe).

[moq-dev/moq#8652](https://github.com/moq-dev/moq/pull/8652)'s harness found
two rare failures, both where a relay's reset of a tab's group stream meets the
browser. Heavy logging in the page hides them, so reproduce with the harness as
is and add tracing that buffers in memory.

- Chromium (about 1 lite-07 cell in 20): one subscription never ends. The page
read the header of a group the relay paused (YIELDED), continued, and then
dropped as stale (`SUBSCRIBE_DROP`, OLD), and that stream never ended or
errored in the page, so the lite-07 tail waits on it forever. No second header
for the group reached the page. Chromium 156 does not negotiate
RESET_STREAM_AT, so the relay sends a plain RESET_STREAM once the 3-byte
WebTransport header is acknowledged (`moq_quic` `take_held_reset`). Next: log
each group stream's ID in the relay and whether Chromium acknowledges its
RESET_STREAM. If Chromium acknowledges it, the loss is Chromium's: report it
upstream, and make the JS reader stop waiting on a stream whose group a drop
or a continuation accounted for. If not, fix the relay's reset.
- Firefox (1 lite-07 cell in about 25): a few starved tracks' final groups fail
with `TypeError: Error in input stream` and no stream error code, which the
reader cannot tell from a YIELDED reset, so the group aborts instead of
pausing. Firefox 157 negotiates RESET_STREAM_AT, and the relay resets with a
Reliable Size of 3. Next: reproduce in a minimal WebTransport page, and report
it upstream if Firefox drops the code.
- Then add `just test stream-credit` to `.github/workflows/interop.yml` after
"Stream limits over WebSocket", and delete the epic README, whose last work
this is.

To reproduce, loop `just test stream-credit --version moq-lite-07-wip --engine
chromium` until one fails; it keeps the run directory with the relay log, the
page console, and `pending.json`. The page's `console.debug` logs each pause.

## Related

- [moq-dev/moq#8643](https://github.com/moq-dev/moq/pull/8643) - a JS copy-fill duplicate group; the harness saw no duplicate group
1 change: 1 addition & 0 deletions sh/gh/test-inventory.allow
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ ignore moq-video::render::export::dx12::tests::exports_an_nv12_texture /quest/m5
ignore moq-video::render::export::dx12::tests::media_foundation_encodes_the_export_in_place /quest/m5/video-hardware.md

# Flaked under load; disabled until fixed at its cause.
recipe test::stream-credit /quest/m1/stream-credit/browser-resets.md

# Too slow for the merge gate; the quest runs it nightly once every scenario settles.
ignore moq-net::route::tests::cache::sweep_cache_routing_long /quest/m2/cache-routing-proof.md
1 change: 1 addition & 0 deletions test/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ questions to a harness. For builds and CI selection, see the
| [native-play](native-play/README.md) | `just test native-play` | the shipped headless player follows overlapping publishers and fallback through decoded PCM on a private Linux null audio device |
| [audio-quality](audio-quality/README.md) | `just test audio-quality` | browser and native audio playout stay within budget over an impaired path |
| [drain](drain/README.md) | `just test drain` | a JS viewer migrates off a draining relay without a dropped group |
| [stream-credit](stream-credit/README.md) | `just test stream-credit` | a browser tab reads 300 live tracks past its ~100-stream WebTransport cap, the highest priority first, and nothing hangs |
| qmux | `just test qmux` | `@moq/qmux`, as client and as server, interoperates with the Rust qmux 0.6.3 release on every QMux draft |
| video | `just test video` | the `@moq/video` renderer paints through WebGPU and falls back to Canvas2D after losing the GPU |
| [json-properties](json-properties.md) | `just test json-properties` | both Snapshot codecs reconstruct, and both Window codecs emit the exact events for, random JSON sequences with replayable seeds |
Expand Down
8 changes: 8 additions & 0 deletions test/justfile
Original file line number Diff line number Diff line change
Expand Up @@ -249,6 +249,14 @@ ts-controls:
drain *args:
./drain/run.sh {{ args }}

# A browser tab reads 300 live tracks of mixed priority through a Rust relay over WebTransport,
# past its ~100-stream cap: the top priority gets every group and nothing hangs. Runs lite-07 and
# lite-06 in Chromium and Firefox; `--version` and `--engine` pick one. See stream-credit/README.md.

# Stream credit by priority in the browser, end to end.
stream-credit *args:
./stream-credit/run.sh {{ args }}

# The @moq/video renderer in headless Chromium: WebGPU, a lost device, and the Canvas2D fallback.
video:
uv run --no-project python ../sh/js/install.py frozen
Expand Down
61 changes: 61 additions & 0 deletions test/stream-credit/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Stream credit harness

A browser tab reads 300 live tracks through a Rust relay over WebTransport,
three times the ~100 streams Chromium and Firefox let a server open to one
session. The relay hands the streams it can open to the highest-priority
groups and, on lite-07, makes room with a YIELDED reset that pauses a
lower-priority group for a later stream to continue.

## Running

```bash
just test stream-credit
```

```bash
just test stream-credit --version moq-lite-07-wip --engine chromium --timeout 120
```

Each run covers lite-07 and lite-06 in Chromium and Firefox, about 15 seconds
per cell. `cargo build -p moq-relay` builds the relay from this checkout.
`STREAM_CREDIT_PROFILE` picks its cargo profile, and `RELAY_BIN` points at a
prebuilt relay instead. A failing run keeps its run directory: the relay log
(with the publisher's scheduling at debug level), and per cell the page's
console, `report.json` with every group each track read, or `pending.json`
with the tracks still open.

## Shape

`run.sh` starts the relay, then runs `driver.ts` once per version and engine.
The driver plays every other part:

- **The publisher** is `@moq/net` in Bun, over WebSocket, whose qmux stream
limit is far above 300, so the browser's cap is the only one in play. Each
track keeps one group open at a time, as a live video track keeps its GOP
open: a frame at its start, one halfway through, and its end as the next
group starts, 40 groups at 200 ms.
- **The reader** is a page built from `src/` with Vite, in Playwright's
Chromium or Firefox. It dials the relay over WebTransport only and subscribes
to every track in three tiers of subscriber priority:

| Tier | Priority | Tracks | Delay budget |
| --- | --- | --- | --- |
| high | 2 | 30 | 500 ms |
| mid | 1 | 120 | live edge |
| low | 0 | 150 | live edge |

A cell passes when:

- the high tier read every group whole, every frame in order;
- every subscription ended cleanly within 10 s of the publisher finishing
(the relay drains the last groups in under a second on loopback);
- on lite-07, every track read its final group whole, since its readers wait
for every group the relay owes them. A lite-06 reader has no drops to wait
on, so it ends at the end it was told, and a starved track may lose the
final group still queued for credit;
- the tiers read frames in priority order, and the low tier lost some, so the
cap bound;
- on lite-07, the relay log shows paused groups continued, the YIELDED path.

With every tier at one priority, the high tier misses groups past its budget
and the relay continues no paused group, so the run fails.
Loading