Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
8ef012a
chore(quest): claim quest/m1/drop-set.md
kixelated Nov 11, 2026
b267c9d
fix(net): a failed fetch below the end no longer holds it open
kixelated Nov 11, 2026
fd284ec
Merge remote-tracking branch 'origin/main' into quest/m1/drop-set
kixelated Nov 11, 2026
d3592ae
refactor(net): one record per reader holds what it handed out, skippe…
kixelated Nov 11, 2026
d73d5d8
refactor(net): a publisher's wire record is its reader's record
kixelated Nov 11, 2026
3a3ced4
feat(net): a reader's record is billed to the pool and folds or ends …
kixelated Nov 11, 2026
8b40da8
feat(net): bill a reader's given-up misses and a tail's finished streams
kixelated Nov 11, 2026
a57149a
feat(net): bill each js/net reader's record and fold or end it under …
kixelated Nov 11, 2026
ccba2f0
perf(net): js/net memoizes the stale runs a reader cannot trust
kixelated Nov 11, 2026
628251a
chore(net): format, and keep the JS wire record types internal
kixelated Nov 11, 2026
200e649
test(net): relay-level drop record tests, and name final drops held b…
kixelated Nov 11, 2026
afc7253
fix(net): keep final drops dropped, re-judge evicted stale runs, and …
kixelated Nov 11, 2026
d300ccd
quest: delete the drop set quest, complete
kixelated Nov 11, 2026
94926f0
Merge remote-tracking branch 'origin/main' into quest/m1/drop-set
kixelated Nov 11, 2026
7936302
fix(net): a lite-06 drop stays fetchable by a wider budget
kixelated Nov 11, 2026
33db1bb
fix(net): fold a tail's finished streams and a reader's refused fills…
kixelated Nov 11, 2026
4131850
fix(net): pay reader debt on every shrink, bill resumed JS readers, a…
kixelated Nov 11, 2026
3578dba
fix(net): a JS routed reader's bill follows its serving route
kixelated Nov 11, 2026
6fec489
fix(net): fetched groups owe nothing for the feed, and re-offers keep…
kixelated Nov 11, 2026
6dfa2ce
fix(net): a failed fill answer reaches no feed arrival through its drop
kixelated Nov 11, 2026
1e299eb
fix(net): a reclaimed failed fill answer accounts nothing for the feed
kixelated Nov 11, 2026
6d0c224
fix(net): a failed fill answer is no verdict and is served to no reader
kixelated Nov 11, 2026
846cc1c
fix(net): a failed fill answer leaves held readers their frames and f…
kixelated Nov 11, 2026
c2f8e81
fix(net): a failed fill answer stays cached and billed, withheld from…
kixelated Nov 11, 2026
7f97d9e
test(net): cover the late-subscriber guard and assert a pending order…
kixelated Nov 12, 2026
42f2094
Merge remote-tracking branch 'origin/main' into HEAD
kixelated Nov 12, 2026
78c7d44
fix(net): a fill's budget refusal is a stale verdict a widening reopens
kixelated Nov 12, 2026
5c05774
fix(net): withhold a failed fill answer as it aborts
kixelated Nov 12, 2026
98b5caf
perf(net): a drop updates a reader's unknown-run memo by what it changed
kixelated Nov 12, 2026
0a9abac
test(net): model the pool folding a subscription's record and tail
kixelated Nov 12, 2026
d498d51
style(net): sort the fill test's imports
kixelated Nov 12, 2026
288ba3e
test(net): count the updated reader's asks after the later reader's own
kixelated Nov 12, 2026
5f3977d
quest: a JS fill probe costs what changed, not every unknown run
kixelated Nov 12, 2026
6c60a9f
quest: say where the probe cost was measured
kixelated Nov 12, 2026
b672764
fix(net): an update owes nothing a tail's fold forgot finishing
kixelated Nov 12, 2026
2828cc6
style(net): the tail's folded span uses the sequence min and max
kixelated Nov 12, 2026
3499d64
fix(net): a tail's fold spares only the stale drops its ledger accoun…
kixelated Nov 12, 2026
4692f94
chore: drop TLC trace files committed by mistake
kixelated Nov 12, 2026
89ff1b1
quest: a FETCH of a failed group keeps its frames reclaimable
kixelated Nov 12, 2026
79e39da
fix(net): a closed JS reader bills nothing, and drop runs append with…
kixelated Nov 12, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions .claude/skills/verify-subscribe/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ description: Model-check how a lite-07 subscription accounts for each group (`rs

# Verify subscription accounting

`rs/moq-net/verify/subscribe/SubscribeAccounting.tla` models one group of one answered lite-07 subscription between the publisher's `TrackRun` and the subscriber's `TrackServe`. The draft promises that every sequence in range gets a Group Stream, a SUBSCRIBE_SKIP, or a SUBSCRIBE_DROP; that a stale (OLD) drop holds only under the update it names and the next SUBSCRIBE_UPDATE reopens it; and that a Group Stream wins over a stale drop, even one it overtook. Drops and skips travel in order on the Subscribe Stream, each Group Stream travels on its own, a reset can lose a stream's header, and a reset sent after the FIN may arrive after every byte. Each model step is one poll of the code that handles one event. The track cursor's verdicts (hand the group out, give it up stale or final, skip it) and whether the cache still holds a fresh copy on an update are free choices. TLC checks every ordering under the bounds in the config.
`rs/moq-net/verify/subscribe/SubscribeAccounting.tla` models one group of one answered lite-07 subscription between the publisher's `TrackRun` and the subscriber's `TrackServe`. The draft promises that every sequence in range gets a Group Stream, a SUBSCRIBE_SKIP, or a SUBSCRIBE_DROP; that a stale (OLD) drop holds only under the update it names and the next SUBSCRIBE_UPDATE reopens it; and that a Group Stream wins over a stale drop, even one it overtook. Drops and skips travel in order on the Subscribe Stream, each Group Stream travels on its own, a reset can lose a stream's header, and a reset sent after the FIN may arrive after every byte. Each model step is one poll of the code that handles one event. The track cursor's verdicts (hand the group out, give it up stale or final, skip it) and whether the cache still holds a fresh copy on an update are free choices. With `Pressure`, so is when the cache pool's debt folds what each side keeps of the group: the publisher's record and the subscriber's tail. TLC checks every ordering under the bounds in the config.

## Rules

Expand All @@ -25,21 +25,21 @@ just rs tla moq-net/verify/subscribe/SubscribeAccounting.tla
just rs tla moq-net/verify/subscribe/SubscribeAccounting.tla -config rs/moq-net/verify/subscribe/StreamWins.cfg
```

The first checks every rule but `StreamWins`: about 3.0M states, 20 to 30 seconds on 32 cores, and it passes at the sync commit. The second fails at the sync commit on the open finding below. `just test tla rs/moq-net/verify/subscribe` runs every config here, as Nightly does, and fails if a config with an `\* Open finding:` line passes. One group is enough: groups share only the order of the Subscribe Stream, and a second group multiplies the states (60M and more) without adding a race.
The first checks every rule but `StreamWins` under `Pressure`: about 13.1M states, a minute on 4 workers, and it passes at the sync commit. The second checks `StreamWins` without `Pressure` and fails on the open finding below. `just test tla rs/moq-net/verify/subscribe` runs every config here, as Nightly does, and fails if a config with an `\* Open finding:` line passes. One group is enough: groups share only the order of the Subscribe Stream, and a second group multiplies the states (60M and more) without adding a race.

Read a counterexample with `/statechart`, passing it the spec, a config, and `SubscribeAccounting.view.ts`, which labels what each side holds for the group, which update each side is on, and each step as the event that happens. It gives the shortest path to each broken rule in those words.

## Open finding: a group read whole is named in a stale drop

`StreamWins.cfg` fails. `GroupServe` in `GroupState::Closed` has written every frame and the FIN, and resets with OLD when the group goes stale before the acknowledgement; `settle_failed` then has `TrackRun::poll_step` name it in a stale drop under the update in effect, and record it in `expired`. The subscriber may already hold every byte. It ignores that drop only while the group is in `Tail::finished`, which `buffer_update` clears through `Tail::updated`, or while its copy still caches the group (`track::Producer::give_up` keeps an open group). So with an update sent between reading the FIN and the drop's arrival, and the copy having evicted the group, the subscriber records it stale and owes it again on the next update, while that update makes the publisher serve the whole group again (`reoffer_groups`) or drop it again under every later update. The draft says the publisher, having served a group, "neither serves it again nor names it in another drop". `residual_a_stale_unacknowledged_fin_drops_and_resends_a_served_group` in `lite/publisher.rs` replays the publisher's half, and `a_finished_group_wins_over_a_late_stale_drop_once_evicted` in `lite/subscriber.rs` pins the subscriber's half. Keeping `Tail::finished` across updates passes every rule in the model (about 1.9M states). Once it is fixed, flip the residual test, move `StreamWins` into `SubscribeAccounting.cfg`, and delete `StreamWins.cfg`.
`StreamWins.cfg` fails. `GroupServe` in `GroupState::Closed` has written every frame and the FIN, and resets with OLD when the group goes stale before the acknowledgement; `settle_failed` then has `TrackRun::poll_step` name it in a stale drop under the update in effect, and record it in `expired`. The subscriber may already hold every byte. It ignores that drop only while the group is in `Tail::finished`, which `buffer_update` clears through `Tail::updated`, or while its copy still caches the group (`track::Producer::give_up` keeps an open group). So with an update sent between reading the FIN and the drop's arrival, and the copy having evicted the group, the subscriber records it stale and owes it again on the next update, while that update makes the publisher serve the whole group again (`reoffer_groups`) or drop it again under every later update. The draft says the publisher, having served a group, "neither serves it again nor names it in another drop". `residual_a_stale_unacknowledged_fin_drops_and_resends_a_served_group` in `lite/publisher.rs` replays the publisher's half, and `a_finished_group_wins_over_a_late_stale_drop_once_evicted` in `lite/subscriber.rs` pins the subscriber's half. Keeping `Tail::finished` across updates passes every rule in the model without `Pressure` (about 1.9M states). Under it, `Tail::rebill` still forgets a finished run by design: the overtaken drop records the group stale, though `Tail::overtaken` keeps it out of what the next update owes, since the ledger accounted the group before the drop. Once it is fixed, flip the residual test and drop the `\* Open finding:` line from `StreamWins.cfg`, which stays without `Pressure`.

## Reading a failure

A counterexample is a path through the model, not yet a bug. Replay it on the code: the publisher's half with `lite07_run`, `write_group`, `run.update`, `SinkSession::with_unacked_fin`, and `drop_updates` in `lite/publisher.rs` tests; the subscriber's half with `lite07_serving`, `apply_response`, `buffer_update`, and `GroupRecv` in `lite/subscriber.rs` tests. If the code does not follow the path, the model is wrong: fix the model to match the code. If it does, report the path and the rule in the code's words. Do not weaken a rule, add a constraint, or shrink the bounds until the check passes.

## Coverage

Sync commit: `103052ed074a87f00269ecb0d639905482ecfb6f`. Run `git log 103052ed07.. -- rs/moq-net/src/lite/publisher.rs rs/moq-net/src/lite/subscriber.rs rs/moq-net/src/tail.rs rs/moq-net/src/model/track.rs` to see what changed since. After a change to a covered function, mirror it in the model, rerun both configs, and move the sync commit forward only once the model describes the code at the new commit. `just check` fails when a function or type the table names no longer exists in its file (`sh/coverage-tables.py`), so each Code cell starts with a repo-relative path, then the symbols in it.
Sync commit: `d9a88d2e3685f170e197c4e1623f26ccaa2c4c63`. Run `git log d9a88d2e36.. -- rs/moq-net/src/lite/publisher.rs rs/moq-net/src/lite/subscriber.rs rs/moq-net/src/tail.rs rs/moq-net/src/model/track.rs` to see what changed since. After a change to a covered function, mirror it in the model, rerun both configs, and move the sync commit forward only once the model describes the code at the new commit. `just check` fails when a function or type the table names no longer exists in its file (`sh/coverage-tables.py`), so each Code cell starts with a repo-relative path, then the symbols in it.

| Code | Model |
| --- | --- |
Expand All @@ -52,8 +52,9 @@ Sync commit: `103052ed074a87f00269ecb0d639905482ecfb6f`. Run `git log 103052ed07
| `rs/moq-net/src/lite/subscriber.rs` `ServeLoop::apply_response` (`Drop`, `Skip`) | `OnDrop`, `OnSkip`, `SubRecv` |
| `rs/moq-net/src/lite/subscriber.rs` `buffer_update`; `rs/moq-net/src/model/track.rs` `supersede_stale` | `SubUpdate` |
| `rs/moq-net/src/tail.rs` `Reading::open`, `Reading::finish`, `Tail::finished`, `updated`, `account`, `unaccount` | `acct`, `finished` |
| `rs/moq-net/src/tail.rs` `Tail::rebill`, `overtaken`, `updated`; `rs/moq-net/src/lite/subscriber.rs` `ServeLoop::apply_response`; `rs/moq-net/src/model/record.rs` `Record::folded` | `TailFold`, `forgot`, `PubFold`, `folded` |
| `rs/moq-net/src/model/track.rs` `Producer::give_up`, `skip_received` | `rec`, `cached` |

## Not covered

The subscription starts answered and covers the group, so SUBSCRIBE_OK, SUBSCRIBE_STARTED, ranges and Range IDs, fills (`RangeFills`), floors, and SUBSCRIBE_FINISH with `drop_unserved` are left out. The tail's grace and the ledger's folding are left out: `QuietIsSettled` asks for the publisher's answer, never the grace. So are priority and order (`Steer`), the Subscribe Stream's FIN and cancellation, sessions closing, versions before lite-07, relays forwarding drops downstream, datagrams, and the copy's readers (`poll_drop`). A route switch appears only as the cursor handing out once more a group it gave up. FETCH is left out: one request and one response, ended by a FIN or a reset, with nothing to race. A pass covers one group, four updates, and four Group Streams.
The subscription starts answered and covers the group, so SUBSCRIBE_OK, SUBSCRIBE_STARTED, ranges and Range IDs, fills (`RangeFills`), floors, and SUBSCRIBE_FINISH with `drop_unserved` are left out. The tail's grace and the ledger's folding are left out, so a group `Tail::overtaken` reports is one whose header arrived: `QuietIsSettled` asks for the publisher's answer, never the grace. So are priority and order (`Steer`), the Subscribe Stream's FIN and cancellation, sessions closing, versions before lite-07, relays forwarding drops downstream, datagrams, and the copy's readers (`poll_drop`). A route switch appears only as the cursor handing out once more a group it gave up. FETCH is left out: one request and one response, ended by a FIN or a reset, with nothing to race. A pass covers one group, four updates, and four Group Streams.
11 changes: 7 additions & 4 deletions js/net/src/broadcast.test.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { expect, test } from "bun:test";
import { Duration, Manual, yieldNow } from "@moq/clock";
import { Consumer as BroadcastConsumer, Producer as BroadcastProducer } from "./broadcast.ts";
import { NotFound, StreamCode } from "./error.ts";
import { NotFound, StreamCode, toStreamCode } from "./error.ts";
import { hooks } from "./internal.ts";
import { textFrame } from "./mock.ts";
import { Timescale, Timestamp } from "./time.ts";
Expand Down Expand Up @@ -666,9 +666,12 @@ test("fetch maxDelay tightens the publisher's maxAge without changing omission",
expect(omitted.sequence).toBe(0n);
omitted.close();

await expect(wireOf(broadcast).fetchGroup("video", 0n, { maxDelay: Duration.millis(0) })).rejects.toBeInstanceOf(
NotFound,
);
// Stale under its own budget, which says OLD as the Rust `fetch_group` does, so a relay asking
// for a narrow reader refuses it to that reader alone.
const stale = await wireOf(broadcast)
.fetchGroup("video", 0n, { maxDelay: Duration.millis(0) })
.catch((err: unknown) => err);
expect(toStreamCode(stale)).toBe(StreamCode.Old);
broadcast.close();
});

Expand Down
6 changes: 3 additions & 3 deletions js/net/src/broadcast.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
import { type GetPromise, Once, Signal } from "@moq/signals";
import * as cache from "./cache.ts";
import * as datagram from "./datagram.ts";
import { NotFound, StreamCode, StreamError, toStreamCode } from "./error.ts";
import { NotFound, StreamCode, StreamError } from "./error.ts";
import type { Consumer as GroupConsumer } from "./group.ts";
import type * as Instance from "./instance.ts";
import { hooks } from "./internal.ts";
Expand Down Expand Up @@ -327,8 +327,8 @@ async function fetchGroup(
void recv.group.closed.then(() => subscriber.close());
return recv.group;
case "drop":
// A fill's refusal says why; anything else given up is just gone.
if (toStreamCode(recv.error) === StreamCode.Old) throw new NotFound(`group ${sequence}`);
// Its own reason, so a stale verdict stays OLD: a relay's fill asking under this
// budget refuses it only to readers no more tolerant (see `fill.rejectGroups`).
throw recv.error;
case "error":
throw recv.error;
Expand Down
Loading