Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions quest/m1/credit-model.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# [L] A TLA+ model shows receive credit never deadlocks

## Goal

A TLA+ model of one connection's flow control, run by TLC in CI like the
other `rs/moq-net/verify` models, checks every ordering of arrivals, reads,
pauses, and evictions for two properties:

- Safety: the credit every stream holds together (unread bytes, leased bytes
not yet released, unfinished frames, paused allowances) never exceeds the
connection window, and paused credit never exceeds its cap.
- Liveness: under weak fairness for the reader and the peer, every stream
eventually progresses, finishes, or is evicted, and a sender waiting for
stream credit eventually opens or gives up its group.

It models the design as [#9210](https://github.com/moq-dev/moq/pull/9210)
leaves it, and the code matches the model, or the PR fixes it.

## Plan

Decided 2026-11-20 (maintainer, /quest-plan of the pre-cut flow-control audit):

- ✅ The cut waits on a model (rejected: restoring the overcommitted benchmark
cells as the evidence, which only samples orderings; cutting without it).
At least four credit bugs landed in the two weeks before the audit, two of
them hangs: [#8794](https://github.com/moq-dev/moq/pull/8794) (a paused
stream granted credit until it held the window),
[#8808](https://github.com/moq-dev/moq/pull/8808) (QMux leaked receive
credit on `release`), [#8774](https://github.com/moq-dev/moq/pull/8774)
(a dropped group never reached the reader), and
[#8607](https://github.com/moq-dev/moq/pull/8607) (a group waiting for
credit held a higher-ranked one behind it).
- ✅ The model comes before #9210, which merges once the model passes on its
design (rejected: #9210 first, or both in parallel).
- ✅ Scope: receive credit, pausing, and the sender's credit wait (rejected:
receive side only; a relay chain end to end, since pausing is per hop).

What to model:

- Receive, `rs/moq-quic/src/connection/streams/lease.rs`: the connection and
stream windows, leases and their release at a segment seal, and
`evict_if_stuck` with its two waits: an evicted reader that has not stopped
its stream (`evicted_held`), and data a reader has not looked at
(`trim_unlooked`). The second is the likeliest hang: it holds eviction off
for as long as any stream has data no reader looked at, so liveness depends
on moq-net always polling every stream it accepted, including ones not yet
accepted.
- Pausing, `rs/moq-net/src/transport/boxed.rs` and `model/pace.rs`: the
half-window cap, #9210's slot allowances (`cap / ((i + 1) * (i + 2))`) and
`set_window`, the pause only at a segment seal, and the read-on-and-shed
path past the cap.
- Send, `rs/moq-net/src/lite/publisher.rs` `TrackRun::opening` and `stuck`,
and `transport/open.rs`: a group waiting for stream credit, by order and
rank.

Abstract bytes into a few units, and keep frames, streams, and slots small
enough for TLC to run in the merge gate, as the other models do; a larger
configuration can run nightly. Add a `verify-credit` skill beside
`verify-subscribe` and the others, naming the code each action maps to.

A violation in main's code is fixed in this PR. A violation in #9210's
design goes to #9210 before it merges.

Public API: none. Wire: none.

## Related

- [Paused-stream allowance](/quest/m1/paused-allowance.md) - the design this model checks before it merges
- [Overcommitted receive-window cells](/quest/m3/receive-windows-evicted.md) - the benchmark side of the same mechanism
- [Send-credit stall counters](/quest/m3/send-credit-stalls.md) - what an operator sees when the sender waits on credit
24 changes: 24 additions & 0 deletions quest/m1/js-frame-cap.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# [XS] @moq/net rejects a received frame over 32 MiB

## Goal

Both JS wire readers (`js/net/src/lite/group.ts` and
`js/net/src/ietf/object.ts`) reject a plain frame whose declared size passes
`MAX_FRAME_BYTES` at its header with `FrameTooLarge`, before reading any of
it, as Rust's `content.rs` admission does. A local writer is unchanged.

## Plan

Decided 2026-11-20 (maintainer, /quest-plan of the pre-cut flow-control
audit): ✅ additive, not a cut blocker (rejected: blocking the cut; dropping
it).

Today a received plain frame past one segment streams in at any size, and
`Consumer.readFrame()` (`#join` in `js/net/src/group.ts`) keeps every chunk
until it joins them, even after the group sheds them, and the browser
media path reads that way (`js/hang/src/container/consumer.ts`). Rust
relays already reject such frames, so only a browser connected directly to
a hostile peer is exposed. The 40 MiB tests in `group.test.ts` write
locally and stay.

Behavior change, no `!`. Public API: none. Wire: none.
24 changes: 24 additions & 0 deletions quest/m1/paused-allowance.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# [S] Any number of streams pause within half the connection window

## Goal

[#9210](https://github.com/moq-dev/moq/pull/9210) merges with its design
checked by the [credit model](/quest/m1/credit-model.md): each paused stream
holds a slot-sized allowance of credit instead of a stream window, so any
number of streams pause within half the connection window. #9210 deletes
this quest.

## Plan

Decided 2026-11-20 (maintainer, /quest-plan of the pre-cut flow-control
audit): ✅ #9210 ships in this cut (rejected: landing after the cut to soak).
✅ It merges once the credit model passes on its design (rejected: merging
first and modeling the result).

#9210's own design decisions, settled 2026-11-19, and its two open `Waiting`
items stay on the PR. When the model finds a violation in its design, fix it
there before merging.

## Required

- [Credit model](/quest/m1/credit-model.md) - checks the slot allowances and `set_window` before they ship
31 changes: 31 additions & 0 deletions quest/m1/relay-memory-default.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# [S] A stock moq-relay bounds its cache memory

## Goal

moq-relay with no cache settings sizes its group cache to keep 10% of system
memory free (the cgroup limit when set), as `cache.headroom = "10%"` does
today, so an authenticated publisher can't grow the relay without bound by
opening tracks. Setting `capacity` or `headroom` explicitly still works, and
an operator can still choose an unbounded pool explicitly.

## Plan

Decided 2026-11-20 (maintainer, /quest-plan of the pre-cut flow-control
audit): ✅ default `headroom` to 10% (rejected: a 50% `capacity`, which
ignores the rest of the process's memory; both).

Receive credit bounds only what a connection holds in the transport and in
unfinished frames. A finished frame moves into the shared cache, which is
unbounded in bytes when neither knob is set (`rs/moq-relay/src/cache.rs`).
Each track's live group is exempt from the 30 s idle window and holds up to
its 32 MiB tail, so a publisher holds about 32 MiB per track it opens. Under
pool debt a group sheds its front sooner, so a byte budget bounds it.

Decide how an operator spells "unbounded" without a magic value in the
setting, per AGENTS.md (for example an explicit `capacity = "100%"`, or
`headroom = "0%"`), and say it in the relay docs and the config reference.
Measure process RSS with several publishers opening many tracks against a
small headroom, with the default, so the bound is shown on the built binary.

Behavior change, no `!`. Public API: the relay config default. Wire: none.
`## Caused by`: as old as the cache pool, no single PR.
13 changes: 13 additions & 0 deletions quest/m3/release/cut.md
Original file line number Diff line number Diff line change
Expand Up @@ -231,6 +231,16 @@ Decided 2026-11-16 (maintainer, /quest-plan of the pre-cut audit):
cut gate's "hand-written `moq-c`" below reads `libmoq`, and the two
quests for a code-free `libmoq` stub were deleted.

Decided 2026-11-20 (maintainer, /quest-plan of the pre-cut flow-control
audit): ✅ the cut waits on the [credit model](/quest/m1/credit-model.md),
[#9210](https://github.com/moq-dev/moq/pull/9210)'s
[paused-stream allowance](/quest/m1/paused-allowance.md), which merges once
the model passes on it, and a [bounded relay cache by
default](/quest/m1/relay-memory-default.md). Rejected: cutting with flow
control unverified; landing #9210 after the cut. The
[JS frame cap](/quest/m1/js-frame-cap.md) ships with the cut only if it
lands first.

On the reconciled candidate, regenerate the Nix build plan with `just rs cargo-nix` after the
shared version bump. A dry run of the bump to 0.22.0 updated 52 packages and left Cargo.nix stale.
Repeat the two-relay smoke with the candidate and the last published relay. Put each binary upstream
Expand All @@ -249,4 +259,7 @@ check that Release remeasure opened or updated its pin PR, as RELEASE.md describ
- [Cargo feature review](/quest/m3/release/feature-flags.md) - every Cargo feature was judged against the feature rules, and approved changes landed
- [Interop and Video pass on main](/quest/m0/interop-green/README.md) - the cut PR's release gate runs `interop.yml`, which failed on `main` every day through 2026-11-12
- [Nightly passes on main](/quest/m0/nightly/README.md) - the release gate runs `features.yml`, red in the 2026-11-12 Nightly
- [Credit model](/quest/m1/credit-model.md) - receive credit, pausing, and the sender's credit wait never deadlock
- [Paused-stream allowance](/quest/m1/paused-allowance.md) - #9210 merges with its design checked
- [Relay memory default](/quest/m1/relay-memory-default.md) - a stock relay's cache is bounded
- [Claim new npm names](/quest/m3/release/npm-first-publish.md) - trusted publishing can publish every new `@moq/*` package