Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions quest/m1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,9 @@ Added 2026-10-27 (maintainer, from an external security audit): relay and
wire boundary fixes, importer streaming, moq-c handle wrap, browser input
checks, and CI tool pins, under one epic; its credential leaks went to m0.

Added 2026-11-01 (maintainer): datagram tracks, so lite-07 freezes with
datagrams as their own track kind instead of single-frame groups.

Added 2026-11-01 (maintainer, follow-up planning): byte-stream ladders, so
the binding API for `lower` breaks before the cut, `moq export` ending
with its broadcast, and a relay answering a SUBSCRIBE on a fetched copy.
Expand Down Expand Up @@ -150,6 +153,7 @@ per-frame CPU growth since #5926.
- [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - a lite-07 SUBSCRIBE asks for past and live ranges with absolute or relative ends, in either order, and replaces FETCH; merged ranges never starve a subscriber; relays fill misses by range, including over moq-transport
- [Drop set](/quest/m1/drop-set.md) - each cursor judges missing groups against its own budget, so a narrow reader drops promptly and a widening re-offers groups already reported dropped on lite-07
- [Binding subscriptions take a Delay { min, max }](/quest/m1/subscription-delay.md) - the bindings take the Rust and JS subscription Delay, and an omitted max means 0 in every one
- [Datagram tracks](/quest/m1/datagram-tracks/README.md) - a track is datagrams or streams; datagram tracks have their own request and sequence, no ranges, no cache, and map to object ID 0 on moq-transport
- [A step back smaller than the delay keeps the TS export](/quest/m1/ts-small-step-back.md) - audio restarts instead of regressing; video stays within its declared depth
- [A replaced copy's grace wakes its reader](/quest/m1/grace-deadline.md) - the grace arms a deadline, so a clean end is on time
- [moq-boy's status loop stops on an errored track](/quest/m1/moq-boy-status-loop.md) - only schema errors are caught
Expand Down
94 changes: 94 additions & 0 deletions quest/m1/datagram-tracks/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# Datagram tracks

## Goal

A track is either a datagram track or a stream track, fixed when it is
created. A datagram track:

- is subscribed through its own request, not SUBSCRIBE;
- carries frames that have only a sequence number and a timestamp, with no
group;
- takes no bounded ranges;
- is never cached, stored, or returned by FETCH.

moq-transport maps its sequence to a group ID with object ID 0. Rust and JS
speak it on lite-07, and the epic ends with a Rust-JS interop test of a
datagram track on lite-07, direct and through a relay. Older lite versions
keep today's datagrams.

## Plan

Decided 2026-11-01 (maintainer, in `/quest-plan`):

- **Datagrams are a track-level decision.** ✅ A track is datagrams or
streams, never a mix. This reverses the
[subscribe-ranges](/quest/m1/subscribe-ranges/README.md) decision of
2026-10-20 that any single-frame group may go as a datagram, including a
copy of a streamed one. With no mixing, the stream/datagram dedup rule, the
ordering of a datagram among its group's streams, NOT_FETCHABLE, and every
datagram case in range accounting all go from lite-07.
- Reason: datagrams are never cached or stored, so they can't take part in
ranges, fills, or FETCH. Mixing them into a stream track makes every one
of those rules carry a datagram exception.
- **Own request.** ✅ A datagram track is subscribed through its own lite-07
request, not SUBSCRIBE, and that request carries only a priority. ✅ It is
live from now: no start, end, order, or `Delay`
([Subscription delay](/quest/m1/subscription-delay.md) exempts it). An update changes only
the priority. A datagram still waits behind higher-priority traffic.
- **No group terminology.** ✅ A datagram carries a sequence number and a
timestamp, with no group. The sequence is the datagram track's own, not
the group sequence space. On moq-transport it maps to a group ID with
object ID 0.
- This exempts datagrams from
[lite-07 by start](/quest/m1/time-addressing/lite07.md): they keep a
sequence, while stream tracks address groups by start.
- The timestamp stays, as it does for every frame.
- **Discovery.** ✅ The application knows which kind a track is, through its
catalog or by convention. The wire carries no kind signal.
- ✅ Datagram tracks and stream tracks have separate namespaces, so
"video" as streams and "video" as datagrams are independent tracks.
- A datagram request for a name with no datagram track is answered as for
any missing track.
- A relay forwards the kind it receives and needs no catalog.
- **Model API.** ✅ Separate types in Rust and JS. A broadcast creates a
stream track or a datagram producer/consumer pair with its own sequence
counter. The datagram types have no group, frame, range, fetch, or cache
API, so the group terminology disappears for datagrams.
- **moq-transport mixing.** ✅ Keep the requested kind.
- A datagram-track subscription maps to IETF SUBSCRIBE and keeps only
datagrams (object 0). Stream objects arriving on it are dropped and
logged.
- A stream-track subscription drops datagrams.
- Outbound, a datagram track sends only OBJECT_DATAGRAM with object ID 0.
- **Archive.** ✅ moq-archive skips datagram tracks. A datagram track is
live-only, and an archive's playback doesn't include it.
- **Version.** ✅ lite-07, before it is finalized.
[Finalize moq-lite-07](/quest/m1/lite07-finalize.md) requires this epic.
lite-01..06 keep today's group datagrams.

Not settled by the maintainer, so the first child decides it and records why:
the datagram request's answer carries the track's timescale and
compression, as TRACK_INFO does for a stream track.

This README owns the work no child owns:

- the Rust-JS interop test of a datagram track on lite-07, direct and
through a relay;
- rewriting the Datagrams section of `doc/concept/moq-lite.md`.

The draft's own Datagrams section changes with the
[Rust child](/quest/m1/datagram-tracks/rust.md).

## Required

- [Rust datagram tracks](/quest/m1/datagram-tracks/rust.md) - Rust model types, the lite-07 datagram request and body, the draft, the relay, and moq-archive skipping datagram tracks
- [JS datagram tracks](/quest/m1/datagram-tracks/js.md) - `@moq/net` datagram types and the lite-07 datagram request
- [moq-transport datagram tracks](/quest/m1/datagram-tracks/ietf.md) - Rust and JS map a datagram track to SUBSCRIBE and OBJECT_DATAGRAM with object ID 0, keeping the requested kind
- [Bindings datagram tracks](/quest/m1/datagram-tracks/bindings.md) - moq-ffi, moq-c, and every wrapper expose the separate datagram types

## Related

- [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - ranges no longer cover datagrams
- [Address groups by timestamp](/quest/m1/time-addressing/README.md) - stream tracks address by start; datagram tracks keep a sequence
- [Wrapper traits over moq-net](/quest/m2/net-wrap/README.md) - the traits mirror the separate datagram types
- [moq-lite session tails count drops](/quest/m2/lite-tail-drop-count.md) - datagram holes matter only on lite-05/06
20 changes: 20 additions & 0 deletions quest/m1/datagram-tracks/bindings.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# [M] Bindings datagram tracks

## Goal

moq-ffi, moq-c, and every wrapper (Python, Swift, Kotlin, Go, Dart, C++)
expose the separate datagram producer and consumer types instead of
datagram methods on stream tracks, mirroring Rust's names.

## Plan

Follow the [epic](/quest/m1/datagram-tracks/README.md)'s decisions. Regenerate the
UniFFI outputs, update `go/wrapper` (`publish.go`, `subscribe.go`,
`records.go`) and `dart/moq`, and record the migration in each package's
`MIGRATE.md`. Update `doc/lib/*` where they show datagrams.

Public API: breaking for every binding's datagrams. Wire: none.

## Required

- [Rust datagram tracks](/quest/m1/datagram-tracks/rust.md) - the Rust types the bindings mirror
31 changes: 31 additions & 0 deletions quest/m1/datagram-tracks/ietf.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# [S] moq-transport datagram tracks

## Goal

Over moq-transport, Rust and JS map a datagram track to IETF SUBSCRIBE and
OBJECT_DATAGRAM, with its sequence as the group ID and object ID 0, and
keep the requested kind when a peer mixes datagrams and streams on one track.

## Plan

Decided 2026-11-01 (maintainer), in the
[epic](/quest/m1/datagram-tracks/README.md):

- A datagram-track subscription keeps only datagrams with object ID 0.
Stream objects arriving on it are dropped and logged once per
subscription.
- A stream-track subscription drops datagrams.
- Outbound, a datagram track sends only OBJECT_DATAGRAM with object ID 0.
A stream track never sends OBJECT_DATAGRAM.
- A datagram with a nonzero object ID on a datagram-track subscription is
dropped and logged.

Touches `rs/moq-net/src/ietf/datagram.rs` and the JS IETF adapter. Test
against moxygen in the interop lanes where it sends datagrams.

Public API: none beyond the model. Wire: no format change.

## Required

- [Rust datagram tracks](/quest/m1/datagram-tracks/rust.md) - the datagram types this maps
- [JS datagram tracks](/quest/m1/datagram-tracks/js.md) - the JS datagram types this maps
30 changes: 30 additions & 0 deletions quest/m1/datagram-tracks/js.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# [M] JS datagram tracks

## Goal

`@moq/net` has the same separate datagram types as Rust, and speaks the
lite-07 datagram request and datagram body against the Rust implementation.
Stream tracks lose `appendDatagram`, and datagram sequences no longer share
the group counter. Older lite versions keep today's group datagrams.

## Plan

Follow the [epic](/quest/m1/datagram-tracks/README.md)'s decisions and the
wire that [Rust datagram tracks](/quest/m1/datagram-tracks/rust.md) lands in
the draft.

- Remove from `js/net/src/track.ts` and the lite subscriber and publisher:
- `appendDatagram` and the shared sequence counter (`track.test.ts`
"appendDatagram shares the group sequence counter");
- the join and refresh rules for datagrams ("a join wants past a datagram
after the newest group");
- the datagram cases in ranges.
- Map lite-01..06 group datagrams the same way Rust does.
- Update any first-party consumer of JS datagrams.

Public API: breaking for `@moq/net` datagrams. Wire: lite-07-wip, matching
Rust.

## Required

- [Rust datagram tracks](/quest/m1/datagram-tracks/rust.md) - lands the lite-07 wire this matches
71 changes: 71 additions & 0 deletions quest/m1/datagram-tracks/rust.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# [L] Rust datagram tracks

## Goal

moq-net has separate datagram types: a broadcast creates a datagram
producer/consumer pair with its own sequence counter, and stream tracks
carry no datagram API. Lite-07 subscribes to a datagram track through its
own request and sends datagrams addressed by that sequence. The relay
forwards datagram tracks, and moq-archive skips them. lite-01..06 keep
today's group datagrams.

## Plan

The decisions are in the [epic](/quest/m1/datagram-tracks/README.md).

**Model**
- Add the datagram producer and consumer types to `rs/moq-net/src/model/`.
Each has its own sequence counter and a tiny send/receive buffer, with no
cache and no ranges.
- Datagram tracks have their own namespace in a broadcast.
- Remove the datagram paths from stream tracks:
- `insert_datagram` and the stored datagrams in `model/datagram.rs`;
- `poll_recv_datagram` and its resume handling in `model/resume.rs`;
- datagram cases in fills, misses, and range accounting.

**Wire (lite-07)**
- A datagram subscribe request carries only a priority, and an update
changes only the priority.
- A datagram carries its sequence, its timestamp, and the payload.
- Decide whether the request's answer carries the track's timescale and
compression as TRACK_INFO does, and record the reason in the epic.
- Every datagram write takes a timestamp, as every write does
([the model](/quest/m1/time-addressing/model.md), maintainer 2026-11-03).
On a scale-less track (`Timescale` 0) a stream frame's timestamp is its
group's key, and a datagram has no group: decide what its timestamp is
there, and record the reason in the epic.
- Remove the lite-07 rules that mixing needed:
- stream/datagram dedup;
- ordering a datagram among its group's streams;
- NOT_FETCHABLE;
- datagrams in SUBSCRIBE_DROP coverage;
- the datagram grace in the subscriber's end (`a_lost_datagram_delays_the_end`).
- On lite-01..06, one subscription to a name carries both group streams and
group datagrams. Map it the way moq-transport does, keeping the requested
kind:
- Inbound, a datagram-track subscription keeps only group datagrams and
drops group streams. A stream-track subscription drops group datagrams.
- Outbound, an old-version SUBSCRIBE for a name is served from the stream
track of that name. It is served from the datagram track, as group
datagrams keyed by its sequence, only when no stream track has that name.
- This outbound rule is the plan's default. This quest may revisit it, and
records why if it does.

**Elsewhere**
- The relay forwards each kind as it received it.
- moq-archive's producer and consumer drop their datagram form.
- moq-ffi's public datagram API (`append_datagram`, `recv_datagram`) is
unchanged in this quest, and is backed internally by the new types. The
[bindings](/quest/m1/datagram-tracks/bindings.md) quest then replaces it
with the separate types.
- Update `drafts/draft-lcurley-moq-lite.md`: the Datagrams section, the
request, the error table, and the lite-07 changelog. Then run
`just drafts check` and `just test interop --all`.
- The PR's Impact section says how to migrate.

Public API: breaking. Stream tracks lose their datagram methods, and the
datagram types are new. Wire: lite-07-wip datagrams and a new request.

## Related

- [Lite-07 by start](/quest/m1/time-addressing/lite07.md) - stream groups address by start; datagrams keep a sequence
1 change: 1 addition & 0 deletions quest/m1/lite07-finalize.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ timestamp bound in [#6168](https://github.com/moq-dev/moq/pull/6168).
- [Lite-07 by start](/quest/m1/time-addressing/lite07.md) - lite-07 carries group starts, previous-start links, GROUP_END, and time bounds instead of sequence numbers
- [@moq/net by start](/quest/m1/time-addressing/js.md) - JS speaks the same lite-07 wire
- [A job's exact route continues its claim's readers](/quest/m1/announce-takeover.md) - the job's route inherits the claim's instance, with no new wire message
- [Datagram tracks](/quest/m1/datagram-tracks/README.md) - lite-07 subscribes to datagram tracks through their own request, and datagrams carry their own sequence

## Related

Expand Down
16 changes: 5 additions & 11 deletions quest/m1/subscribe-ranges/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ relay bounds a sparse FETCH only if the model can request ranges:
lite-07; published versions keep it, answered through the same model.
- **Holes.** SUBSCRIBE_DROP (restored in lite-07) names every sequence in the
requested ranges the publisher won't deliver: never existed, expired, or
refused upstream. Datagram groups are always dropped in a past range (#4982). The
refused upstream. On lite-01..06 datagram groups are always dropped in a past range (#4982); on lite-07 ranges never cover datagrams ([Datagram tracks](/quest/m1/datagram-tracks/README.md)). The
publisher delivers what it has and fills the rest as upstream answers.
- **Model.** `Subscription` gains `ranges` beside its `order`, mirroring the wire;
`request_groups` as a separate API was rejected. A publisher's
Expand Down Expand Up @@ -79,16 +79,10 @@ relay bounds a sparse FETCH only if the model can request ranges:
[lite-07 finalizes](/quest/m1/lite07-finalize.md), which this line gates. This amends the 09-29
premise that ranges are group-only because a group's timestamp isn't always
known: timed tracks know it, and untimed ones refuse. The cache half landed in its own PR after #5929.
A datagram is judged on its own (maintainer, 2026-10-20): its timestamp
against the range's times, its sequence against the absolute and resolved
relative groups, never through the settle walk. Refusing time anchors on
datagram tracks was considered and can't work: nothing signals that a track
carries datagrams, and the lite draft lets a publisher send any single-frame
group as a datagram, including a copy of one it also streams. Datagrams stay
ephemeral (maintainer, 2026-10-20): a buffered datagram never indexes where
its group starts, so a time end settles across a sequence that arrived only
as a datagram, and a stream copy of that group arriving later is read even
if it starts past the end.
A datagram was judged on its own (maintainer, 2026-10-20). Superseded
2026-11-01 (maintainer): datagrams become their own track kind
([Datagram tracks](/quest/m1/datagram-tracks/README.md)), so on lite-07
a range never covers a datagram.
- Ranges are whole groups: a range always starts at a group's first frame
(maintainer, on [moq-dev/moq#6465](https://github.com/moq-dev/moq/pull/6465)). The IETF joining FETCH for the
live group's uncached prefix stays, because today's bridge relies on it
Expand Down
2 changes: 1 addition & 1 deletion quest/m1/subscribe-ranges/js.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ Ranges stay whole groups ([moq-dev/moq#6465](https://github.com/moq-dev/moq/pull
so the group model keeps refusing a nonzero lite `frameStart` as a protocol
violation, as `lite/subscriber.ts` `runGroup` does. Preserve the per-reader
group-bound rule for reliable reads, live-edge selection, subscription
updates, and publisher datagrams.
and subscription updates. Datagrams move to their own track kind ([Datagram tracks](/quest/m1/datagram-tracks/README.md)).

## Related

Expand Down
5 changes: 2 additions & 3 deletions quest/m1/subscribe-ranges/lite.md
Original file line number Diff line number Diff line change
Expand Up @@ -162,9 +162,8 @@ we should have a signal instead". Settled the same day:
- The subscriber FINs the stream once it wants nothing more, and the publisher
FINs in reply.
- Datagrams take no ranges, since they become their own track kind
(maintainer, 2026-11-01). The datagram-tracks plan
([moq-dev/moq#7575](https://github.com/moq-dev/moq/pull/7575)) touches this
quest too, and should take this design and the rename as given.
([Datagram tracks](/quest/m1/datagram-tracks/README.md), maintainer
2026-11-01), so datagram fills need no handling here.

The draft's exception for a start past the publisher's largest position should
also say that a relay knows that position only from its upstream's answer,
Expand Down
16 changes: 8 additions & 8 deletions quest/m1/subscribe-ranges/start-bookkeeping.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,14 @@ f235fcfcbd, df0bd5c36f, 342445f251, 06765f4c1a, 4927167904, f52848d634,
plus `cursor_floor` and a pre-06 re-pin branch, encoding three wire
generations of "start".

- On lite-07 a stream or a DROP accounts for every sequence except one sent
only as a datagram, which can be lost unseen
(`a_lost_datagram_delays_the_end` in `tests/integration/track_tail.rs`).
So the subscriber's end is "owed range covered and no stream active",
except that a hole a datagram may have left still settles within a bounded
grace; only reliable accounting of datagram sequences could remove that.
Keep `Settle`'s full grace for lite-05/06 and moq-transport behind one
version predicate.
- On lite-07 every owed range is covered by delivered stream groups or a
DROP, since
[Datagram tracks](/quest/m1/datagram-tracks/README.md) takes datagrams out
of stream tracks (maintainer, 2026-11-01). The subscriber's end is "owed
range covered and no stream active", with no datagram grace;
`a_lost_datagram_delays_the_end` becomes a lite-05/06 case. Keep
`Settle`'s full grace for lite-05/06 and moq-transport behind one version
predicate.
- Lite-07 is unpublished, so sending a fresh START per update can remove the
`requested == start` gate (`lite/subscriber.rs`, about line 4898).

Expand Down
1 change: 1 addition & 0 deletions quest/m1/subscription-delay.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ Decided 2026-10-30 (maintainer, follow-up planning): m1, because it breaks the p
- C: `max_delay_present` is false in a zeroed struct.

Each one's omitted or zeroed form becomes 0, and unlimited stays expressible. Dart defaults the parameter to `Duration.zero`, so an explicit `null` stays unlimited. Go's nil and C's unset `present` can't mean both, so each gains an explicit unlimited flag in the delay, as `moq_delay.max_unlimited` already does for decoders; the PR names it.
- **Not datagram subscriptions:** a datagram track's request carries only a priority and takes no `Delay` ([Datagram tracks](/quest/m1/datagram-tracks/README.md), maintainer 2026-11-01).
- Update `doc/lib/{py,swift,kt,go,dart,c,cpp}` and add MIGRATE notes under `## Unreleased` for each breaking binding change.

Public API: breaking in every binding. Wire: none.
Expand Down
Loading
Loading