From 00a73adaad485a21c10acd50d4a33d02c589b2b1 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 1 Oct 2026 15:06:16 -0700 Subject: [PATCH 1/3] quest: plan customer timestamp requests, make timestamps optional end to end Co-Authored-By: Claude Opus 5.5 --- quest/m1/README.md | 7 ++- quest/m1/data-consumer-timestamps.md | 19 +++--- quest/m1/js-data-consumer-timestamps.md | 31 ++++++++++ quest/m1/js-publish-timestamp.md | 26 ++++++--- quest/m1/plan-untimed-objects.md | 57 +++++++++--------- quest/m1/publish-timestamp.md | 77 +++++++++++++++---------- quest/m2/README.md | 1 + quest/{m3 => m2}/watch-data-sync.md | 10 ++-- quest/m3/README.md | 1 - 9 files changed, 144 insertions(+), 85 deletions(-) create mode 100644 quest/m1/js-data-consumer-timestamps.md rename quest/{m3 => m2}/watch-data-sync.md (63%) diff --git a/quest/m1/README.md b/quest/m1/README.md index fee0f21d4ef..1952c3efd27 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -72,10 +72,11 @@ QUIC studies there on that rule. - [Auth client CA](/quest/m1/relay-auth-client-ca.md) - on dev, `auth::Config::validate` and `init` take the client-CA flag, so no caller can skip the check - [Data track clock](/quest/m1/data-track-clock.md) - JSON and binary data tracks stamp on the catalog's clock at write time, matching the media's anchored clock - [Go and Dart doc samples](/quest/m1/doc-samples-go-dart.md) - Go and Dart doc samples compile against their wrappers -- [Publishing requires a timestamp](/quest/m1/publish-timestamp.md) - every Rust and binding publish API takes a timestamp; nothing fills in now for the caller -- [JS publishing requires a timestamp](/quest/m1/js-publish-timestamp.md) - the same in @moq/json, @moq/binary, and @moq/net -- [Plan: untimed peer objects](/quest/m1/plan-untimed-objects.md) - decide what a frame's timestamp is when a moq-transport or pre-lite-05 peer sends none +- [Publishing never invents a timestamp](/quest/m1/publish-timestamp.md) - no Rust or binding publish API fills in a timestamp; an untimed payload goes out untimed, and moq-mux data producers take broadcast-clock timestamps +- [JS publishing never invents a timestamp](/quest/m1/js-publish-timestamp.md) - the same in @moq/json, @moq/binary, and @moq/net +- [Plan: untimed objects](/quest/m1/plan-untimed-objects.md) - map faithful timestamp absence, publisher to consumer, into implementation quests - [Data consumer timestamps](/quest/m1/data-consumer-timestamps.md) - json and binary consumers return each value's timestamp, in Rust and every binding +- [JS data consumer timestamps](/quest/m1/js-data-consumer-timestamps.md) - @moq/json and @moq/binary consumers return each value's timestamp - [Nested data configs](/quest/m1/data-config-nesting.md) - docs nest `BinaryConfig`/`JsonConfig` in an application section instead of flattening it - [Omit empty catalog sections](/quest/m1/catalog-omit-empty.md) - a Rust catalog with no video or audio leaves those keys out, as JS does - [Delete SourceMap](/quest/m1/source-map-removal.md) - on dev, `Anchor` with its `Lane`s is the one way onto the broadcast clock diff --git a/quest/m1/data-consumer-timestamps.md b/quest/m1/data-consumer-timestamps.md index 98fcb1dc863..88d966b68e3 100644 --- a/quest/m1/data-consumer-timestamps.md +++ b/quest/m1/data-consumer-timestamps.md @@ -19,20 +19,19 @@ moq-json and moq-binary snapshot and stream consumers they wrap. A snapshot consumer returns the timestamp of the frame it decoded last. Decided: `next()` and `poll_next()` return `Timed`, the type the producers -take, whose `at` becomes required in -[Publishing requires a timestamp](/quest/m1/publish-timestamp.md). The -timestamp is the frame's media timestamp on the track's timescale. -[Plan: untimed peer objects](/quest/m1/plan-untimed-objects.md) decides -whether a peer can deliver a frame without one; if it can, `at` becomes an -`Option` here, so this waits for it and breaks once. +take in [Publishing never invents a timestamp](/quest/m1/publish-timestamp.md). +`at` is the frame's media timestamp on the track's timescale, and `None` for +an untimed frame: [Plan: untimed objects](/quest/m1/plan-untimed-objects.md) +settled (2026-10-01) that absence survives the wire rather than becoming +arrival time. A republisher passes `at` straight to a moq-mux data producer. moq-ffi's json/binary consumers return the timestamp too, and the py, swift, -kt, go, and dart wrappers and `doc/lib/*` follow. JS readers stay with -[Data sync in watch](/quest/m3/watch-data-sync.md). +kt, go, and dart wrappers and `doc/lib/*` follow. JS is +[JS data consumer timestamps](/quest/m1/js-data-consumer-timestamps.md). Public API: breaking, on `dev`. Wire: none. ## Required -- [Publishing requires a timestamp](/quest/m1/publish-timestamp.md) - makes `Timed.at` required, the type this returns -- [Plan: untimed peer objects](/quest/m1/plan-untimed-objects.md) - decides whether the returned timestamp can be absent +- [Publishing never invents a timestamp](/quest/m1/publish-timestamp.md) - gives `Timed.at` its untimed meaning, the type this returns +- [Plan: untimed objects](/quest/m1/plan-untimed-objects.md) - the model must carry an absent timestamp to consumers diff --git a/quest/m1/js-data-consumer-timestamps.md b/quest/m1/js-data-consumer-timestamps.md new file mode 100644 index 00000000000..e706c396b0c --- /dev/null +++ b/quest/m1/js-data-consumer-timestamps.md @@ -0,0 +1,31 @@ +# [S] JS data consumers return each value's timestamp + +## Goal + +`@moq/json` and `@moq/binary` snapshot and stream consumers return each value +with its frame's timestamp, so a browser can hold telemetry back until the +video playhead reaches it. Mirrors +[Data consumer timestamps](/quest/m1/data-consumer-timestamps.md) in Rust. + +## Plan + +Requested by OneTooMany, whose web frontend shows KLV and MAVLink telemetry +beside video and today reads raw frames to recover the timestamps. + +Decided (2026-10-01): `next()` and the async iterator yield `@moq/net`'s +`Timed` (`{ value, at? }`), the type the producers take after +[JS publishing never invents a timestamp](/quest/m1/js-publish-timestamp.md). +`at` is the frame's timestamp, absent for an untimed frame. A snapshot +consumer returns the timestamp of the frame it decoded last. Update +`doc/lib/js/{json,binary}.md`. + +Public API: breaking, on `dev`. Wire: none. + +## Required + +- [JS publishing never invents a timestamp](/quest/m1/js-publish-timestamp.md) - adds `Timed` to `@moq/net` +- [Plan: untimed objects](/quest/m1/plan-untimed-objects.md) - js/net stops filling arrival time, so `at` can be absent + +## Related + +- [Synced data playback](/quest/m2/watch-data-sync.md) - the js/watch reader built on these timestamps diff --git a/quest/m1/js-publish-timestamp.md b/quest/m1/js-publish-timestamp.md index 02549b2db61..388adbc49ff 100644 --- a/quest/m1/js-publish-timestamp.md +++ b/quest/m1/js-publish-timestamp.md @@ -1,18 +1,28 @@ -# [M] JS publishing requires a timestamp +# [M] JS publishing never invents a timestamp ## Goal -`@moq/json`, `@moq/binary`, and `@moq/net` publish APIs take a timestamp and -never fill in `Timestamp.now()` for the caller, mirroring -[Publishing requires a timestamp](/quest/m1/publish-timestamp.md). +`@moq/json`, `@moq/binary`, and `@moq/net` publish APIs never fill in +`Timestamp.now()` for the caller, mirroring +[Publishing never invents a timestamp](/quest/m1/publish-timestamp.md). A +payload published without a timestamp goes out untimed. ## Plan Today the json and binary snapshot and stream producers default `at = Timestamp.now()`, the json window producer always uses now, and js/net's -`writeString`, `writeJson`, and `writeBool` stamp now. Make the timestamp a -required argument on each, with the same shape as Rust's `Timed`. Callers in -the repository (js/hang catalog, js/publish, js/room) pass their clock's now -explicitly. Update `doc/lib/js/{json,binary,net}.md`. +`writeString`, `writeJson`, and `writeBool` stamp now. + +Decided (2026-10-01): `@moq/net` exports `Timed` +(`{ value: T; at?: Time.Timestamp }`), the Rust name and shape. Each producer +takes a `Timed`; an absent `at` is untimed, never now. The consumers return +the same type in +[JS data consumer timestamps](/quest/m1/js-data-consumer-timestamps.md). +Callers in the repository (js/hang catalog, js/publish, js/room) pass their +clock's now explicitly. Update `doc/lib/js/{json,binary,net}.md`. Public API: breaking, on `dev`. Wire: none. + +## Required + +- [Plan: untimed objects](/quest/m1/plan-untimed-objects.md) - an untimed payload must travel as untimed before producers stop filling in now diff --git a/quest/m1/plan-untimed-objects.md b/quest/m1/plan-untimed-objects.md index 33344c05b2e..c495e94c703 100644 --- a/quest/m1/plan-untimed-objects.md +++ b/quest/m1/plan-untimed-objects.md @@ -1,12 +1,12 @@ -# [S] Plan: untimed peer objects +# [S] Plan: untimed objects ## Goal -Decide what a frame's timestamp is when a peer sends none, and rewrite this -quest into implementation quests. That covers IETF objects on a track without -TIMESCALE or without a Timestamp property, and lite-01 to lite-04, which have -no timestamp on the wire. The decision must not invent a timeline that -changes with the relay path. +Rewrite this quest into implementation quests that carry an absent timestamp +faithfully, from publisher to consumer, through every relay. That covers +untimed publishes, IETF objects on a track without TIMESCALE or without a +Timestamp property, and lite-01 to lite-04, which have no timestamp on the +wire. Nothing may invent a timeline that changes with the relay path. ## Plan @@ -17,30 +17,33 @@ relay then forwards those stamps as if they were real. A failover to another first hop changes them, so the timeline jumps, and they break the "one clock per broadcast" property. The maintainer dislikes this. -Settled elsewhere: publishing always requires a timestamp -([Publishing requires a timestamp](/quest/m1/publish-timestamp.md)), and -lite-05 and lite-06 (published) can't encode an absent one. - -Candidates: - -- Faithful absence (leaning): the model and consumers carry - `Option` so a relay forwards "none" as none. lite-07-wip encodes - it by shifting the FRAME Timestamp Delta and the DATAGRAM Timestamp by one - (0 = absent); an absent frame doesn't move the delta baseline. A lite-05/06 - encoder writes its send time. Untimed groups are never media-stale; the - pool's wall-clock expiry still reclaims them, and start resolution picks the - latest group instead of replaying the cache. -- First-hop arrival, documented: no change, but the failover jump stays. -- 0 as a sentinel: rejected. It collides with a real pts of 0. -- `max_age` on max(wall, pts): the maintainer asked. Rust keeps `max_age` as - media-time staleness so a congestion stall can't age content out, with the - pool's expiry as the wall-clock bound (`rs/moq-net/src/model/cache.rs`). +Decided (2026-10-01): faithful absence, on both the publish and the subscribe +side. The model and every consumer carry `Option`, a relay forwards +"none" as none, and no layer substitutes arrival or wall time. +[Publishing never invents a timestamp](/quest/m1/publish-timestamp.md) stops +the producers filling in now. Untimed groups are never media-stale; the +pool's wall-clock expiry still reclaims them, and start resolution picks the +latest group instead of replaying the cache. + +- lite-07-wip encodes absence by shifting the FRAME Timestamp Delta and the + DATAGRAM Timestamp by one (0 = absent); an absent frame doesn't move the + delta baseline. +- lite-05 and lite-06 (published) can't encode absence, so their encoder + writes its send time, documented in the lite draft as a downgrade for old + peers. +- Rejected: first-hop arrival (the failover jump stays), 0 as a sentinel + (collides with a real pts of 0), and `max_age` on max(wall, pts) (Rust keeps + `max_age` as media-time staleness so a congestion stall can't age content + out, with the pool's expiry as the wall-clock bound, + `rs/moq-net/src/model/cache.rs`). + +What remains is mapping the fallout into implementation quests. Facts to gather: which IETF implementations send TIMESCALE and Timestamp (moxygen, the interop matrix), and every model decision that reads a timestamp (`track.rs` staleness and `GroupExpiry`, `resume.rs` successors, `group.rs` `poll_timestamp`, which treats "has a timestamp" as "has a frame"). -Output: implementation quests, and updates to -[Data consumer timestamps](/quest/m1/data-consumer-timestamps.md), whose -return type depends on this, and to the timestamp and lite drafts. +Output: implementation quests (the model and wire carrying `Option`, lite-07 +encoding, and the timestamp and lite drafts), and their places in the +`Required` lists of the publish and consumer timestamp quests. diff --git a/quest/m1/publish-timestamp.md b/quest/m1/publish-timestamp.md index edce12e6a48..a749f771e72 100644 --- a/quest/m1/publish-timestamp.md +++ b/quest/m1/publish-timestamp.md @@ -1,61 +1,74 @@ -# [L] Publishing requires a timestamp +# [L] Publishing never invents a timestamp ## Goal -Every publish API takes a timestamp, and nothing below it invents one. A -frame's timestamp is the publisher's statement of when the content happened. -A library never fills in "now" on the caller's behalf. Covers moq-net, +No publish API fills in a timestamp the caller didn't give. A frame's +timestamp is the publisher's statement of when the content happened; a payload +published without one goes out untimed and arrives untimed. Covers moq-net, moq-json, moq-binary, moq-mux, moq-ffi, and every binding wrapper (Python, Swift, Kotlin, Go, Dart). Not libmoq, which the [Generated C bindings](/quest/m1/c/README.md) replace. ## Plan -Decided (2026-09-30): the wire timestamp stays required, so the publish side -must be too. Today `Timed.at: None` means "stamp when written": moq-json and -moq-binary fill `Timestamp::now()` (moq-net's clock, not the broadcast's), and -moq-mux's `Clock::stamp` fills the broadcast clock's `now()`. The FFI raw +Decided (2026-10-01), reversing the 2026-09-30 "timestamp required" plan: the +timestamp is optional end to end, on the wire and in every API, and absent +means untimed. A library never substitutes now, wall clock, or arrival time. +How absence travels on the wire is +[Plan: untimed objects](/quest/m1/plan-untimed-objects.md); this quest +removes every place that fills one in. + +Today `Timed.at: None` means "stamp when written": moq-json and moq-binary +fill `Timestamp::now()` (moq-net's clock, not the broadcast's), and moq-mux's +`Clock::stamp` fills the broadcast clock's `now()`. The FFI raw `MoqFrame.timestamp_us` and `MoqDatagram.timestamp_us` default to 0. -- `moq_net::Timed` keeps its name and gets a required `at: T`, with no - `From`. The json/binary consumers return the same type (see - [Data consumer timestamps](/quest/m1/data-consumer-timestamps.md)). +- `moq_net::Timed` keeps its name and its `at: Option`; only the + meaning of `None` changes. The json/binary consumers return the same type + (see [Data consumer timestamps](/quest/m1/data-consumer-timestamps.md)). - moq-json (snapshot, stream, and window, whose `push` always stamps now) and - moq-binary producers take `Timed<_, Timestamp>`. The moq-mux data producers - take `Timed<_, Instant>` and map it onto the broadcast clock, refusing one - ahead of now (`Error::InvalidCapture`), as today; `Clock::stamp` stops - filling `None`. `catalog::data::Listing::record` then always has a capture - time for `delay` and `jitter`. + moq-binary producers take `Timed<_, Timestamp>` and write `None` as untimed. +- moq-mux data producers take `Timed<_, Timestamp>` on the broadcast clock, + the same contract as `container::Frame`, with no ahead-of-now check. A + caller already holding a broadcast-clock timestamp passes it as is: one + returned by a `clock::Lane` on the broadcast's `Anchor` (video + KLV from one + MPEG-TS program), or the `at` of a consumer's `Timed` it derived the value + from (MAVLink telemetry republished). A source clock running slightly fast + must not be refused. +- `Clock::capture(Instant)` becomes public for callers holding a capture + `Instant`; it keeps refusing one ahead of now or before the epoch + (`Error::InvalidCapture`). `Clock::stamp` is deleted. + `catalog::data::Listing::record` samples `delay` and `jitter` only for timed + writes, and records zero delay for a timestamp ahead of now rather than + refusing it. +- Document on the moq-mux producers that a carried-over timestamp lines up + with media only on a broadcast sharing the source's clock mapping. - Publishers inside the repository (hang catalog snapshots, MSF, stats, room chat, examples) pass their clock's now explicitly. -- moq-ffi: the - capture time is a media timestamp on the broadcast's timeline. moq-ffi - exposes the broadcast clock's `now()` as a timestamp; callers stamp payloads - with values taken from it, and moq refuses one ahead of now. That keeps a - device or process clock out, as the Rust `Instant` mapping does. Either map - the timestamp back through the clock inside moq-ffi or give the clock a - typed timestamp moq-mux accepts; keep a raw `Timestamp` from compiling - there. The json/binary `update` and `append` take it as a required - argument, and the raw frame and datagram records lose their `default = 0`. +- moq-ffi: data producers take an optional timestamp in microseconds on the + broadcast clock, unchecked, as Rust does. moq-ffi exposes the broadcast + clock's `now()` so callers have a value to stamp with. The raw frame and + datagram records' `timestamp_us` becomes optional instead of `default = 0`. - Go (`go/wrapper/moq`) and Python (`py/moq-rs`) wrap only the JSON producers; [#4137](https://github.com/moq-dev/moq/pull/4137) added `publish_binary_snapshot` / `publish_binary_stream` to moq-ffi without them. Add hand-written binary wrappers there. The maintainer asked for this. - Each wrapper gets a test that a past capture time is accepted and a future - one refused. + Each wrapper gets a test that a timestamp round-trips and an untimed payload + arrives untimed. - Docs: "stamped when written" in `doc/lib/rs/{moq-json,moq-binary}.md` and `rs/moq-net/src/model/timed.rs`, plus `doc/lib/{py,swift,kt,go,dart}`. -What a receiver fills in when a peer sends no timestamp is out of scope: -[Plan: untimed peer objects](/quest/m1/plan-untimed-objects.md). +The broadcast-clock input was requested by OneTooMany, who align KLV and +MAVLink-derived telemetry with video. -Public API: breaking, on `dev`. Wire: none. +Public API: breaking, on `dev`. Wire: none here; absence on the wire lands +with the untimed implementation quests. ## Required - [JSON and flate namespaces](/quest/m1/ffi-shape/json.md) - moves the data producers this changes, so the two breaks land in order rather than colliding +- [Plan: untimed objects](/quest/m1/plan-untimed-objects.md) - an untimed payload must travel as untimed before producers stop filling in now ## Related -- [Data track clock](/quest/m1/data-track-clock.md) - the moq-mux mapping this keeps reads the catalog's clock at write time -- [JS publishing requires a timestamp](/quest/m1/js-publish-timestamp.md) - the same change in the JS packages +- [JS publishing never invents a timestamp](/quest/m1/js-publish-timestamp.md) - the same change in the JS packages diff --git a/quest/m2/README.md b/quest/m2/README.md index 22c07b62be2..e566671e8c2 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -19,6 +19,7 @@ waiting on an upstream release waits in [m4](/quest/m4/README.md). - [One port](/quest/m2/one-port/README.md) - a relay speaks QUIC and STUN on one UDP port and HTTP, RTMP, and RTMPS on one TCP port; WebRTC media is an embedder hook - [Ladder](/quest/m2/ladder/README.md) - a transcode ladder adapts to the uplink it publishes over, instead of encoding every live rung at its ceiling - [Processor](/quest/m2/processor/README.md) - a customer-run worker publishes an on-demand contribution under its own service prefix with scoped access +- [Synced data playback](/quest/m2/watch-data-sync.md) - js/watch releases JSON and binary payloads on the media playhead, and a slow data track holds media back - [Stream sessions](/quest/m2/uring-tcp/README.md) - serve WebSocket and HTTP from the io_uring workers, where io_uring pays off most - [fMP4 emsg](/quest/m2/emsg.md) - settles the shared framing and missing-data semantics before this section adopts them - [ID3 catalog section](/quest/m2/id3.md) - timed ID3 as a first-class container-neutral catalog section diff --git a/quest/m3/watch-data-sync.md b/quest/m2/watch-data-sync.md similarity index 63% rename from quest/m3/watch-data-sync.md rename to quest/m2/watch-data-sync.md index 16a287344c3..557a53b7f6b 100644 --- a/quest/m3/watch-data-sync.md +++ b/quest/m2/watch-data-sync.md @@ -16,10 +16,12 @@ releases it. `delay` and `jitter` with `Sync` like a media rendition. - Snapshot tracks release the newest value at or before the playhead; stream tracks release every record in order. -- In m3 until an application needs synchronized data playback, such as - teleop telemetry beside video; until then, a raw consumer reads payloads as - they arrive. +- An untimed payload has no playhead position; release it on arrival. +- OneTooMany is the application this waited for (2026-10-01): their web + frontend holds KLV and MAVLink telemetry back to the video playhead with + its own sync code, which this replaces. In m2 rather than m1 because + they aren't blocked. ## Required -- [Data track clock](/quest/m1/data-track-clock.md) - data timestamps share the media clock mapping, so the playhead can release them +- [JS data consumer timestamps](/quest/m1/js-data-consumer-timestamps.md) - the reader releases each value by the timestamp its consumer returns diff --git a/quest/m3/README.md b/quest/m3/README.md index 05f2ed16ad4..02d73b4c1c6 100644 --- a/quest/m3/README.md +++ b/quest/m3/README.md @@ -29,7 +29,6 @@ deleted when it goes stale; git history keeps it. - [AF_XDP UDP path](/quest/m3/af-xdp.md) - the kernel-bypass verdict on today's virtio hosts that gates DPDK - [Unreal prototype](/quest/m3/unreal.md) - a UE5 module on the C++ package with exceptions disabled, rendering a subscribed broadcast to a texture - [Conan remote](/quest/m3/cpp-conan.md) - a remote we own serves the same tarball to `conan install` -- [Synced data playback](/quest/m3/watch-data-sync.md) - js/watch releases JSON and binary payloads on the media playhead, and a slow data track holds media back - [Linux OBS GPU input](/quest/m3/obs-linux-gpu.md) - publish OBS compositor frames without CPU readback on a validated Linux graphics/encoder combination - [Routing cost domains](/quest/m3/routing-cost-domains.md) - design operator boundaries and policy without adding incomparable costs - [The moq.pro mesh runs lite-07](/quest/m3/lite07-mesh.md) - the deployment that makes the exemption dead code From 07f46471bb618dcaf9a3aac6211bbc43bcbed668 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 1 Oct 2026 15:17:20 -0700 Subject: [PATCH 2/3] quest: snapshots stop coalescing; untimed records keep stream order Co-Authored-By: Claude Opus 5.5 --- quest/m1/README.md | 4 ++-- quest/m1/data-consumer-timestamps.md | 9 +++++++++ quest/m1/js-data-consumer-timestamps.md | 9 ++++++--- quest/m2/watch-data-sync.md | 11 ++++++++--- 4 files changed, 25 insertions(+), 8 deletions(-) diff --git a/quest/m1/README.md b/quest/m1/README.md index 1952c3efd27..15c8df6b502 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -75,8 +75,8 @@ QUIC studies there on that rule. - [Publishing never invents a timestamp](/quest/m1/publish-timestamp.md) - no Rust or binding publish API fills in a timestamp; an untimed payload goes out untimed, and moq-mux data producers take broadcast-clock timestamps - [JS publishing never invents a timestamp](/quest/m1/js-publish-timestamp.md) - the same in @moq/json, @moq/binary, and @moq/net - [Plan: untimed objects](/quest/m1/plan-untimed-objects.md) - map faithful timestamp absence, publisher to consumer, into implementation quests -- [Data consumer timestamps](/quest/m1/data-consumer-timestamps.md) - json and binary consumers return each value's timestamp, in Rust and every binding -- [JS data consumer timestamps](/quest/m1/js-data-consumer-timestamps.md) - @moq/json and @moq/binary consumers return each value's timestamp +- [Data consumer timestamps](/quest/m1/data-consumer-timestamps.md) - json and binary consumers return each value's timestamp, in Rust and every binding; snapshots yield every state +- [JS data consumer timestamps](/quest/m1/js-data-consumer-timestamps.md) - @moq/json and @moq/binary consumers return each value's timestamp, and snapshots stop coalescing - [Nested data configs](/quest/m1/data-config-nesting.md) - docs nest `BinaryConfig`/`JsonConfig` in an application section instead of flattening it - [Omit empty catalog sections](/quest/m1/catalog-omit-empty.md) - a Rust catalog with no video or audio leaves those keys out, as JS does - [Delete SourceMap](/quest/m1/source-map-removal.md) - on dev, `Anchor` with its `Lane`s is the one way onto the broadcast clock diff --git a/quest/m1/data-consumer-timestamps.md b/quest/m1/data-consumer-timestamps.md index 88d966b68e3..85834f0bd54 100644 --- a/quest/m1/data-consumer-timestamps.md +++ b/quest/m1/data-consumer-timestamps.md @@ -25,6 +25,15 @@ an untimed frame: [Plan: untimed objects](/quest/m1/plan-untimed-objects.md) settled (2026-10-01) that absence survives the wire rather than becoming arrival time. A republisher passes `at` straight to a moq-mux data producer. +Decided (2026-10-01): snapshot consumers stop coalescing. Today the moq-json +snapshot consumer applies every buffered delta but yields only the newest +state, and moq-binary jumps to the newest group, so a 9s state is lost when +11s is already buffered. A caller syncing to a playhead needs the newest +state at or before it, so every state is yielded in order with its `at`, and +a caller that wants only the latest keeps the last one. A reader that falls +behind the drift budget (`Lagged`) still jumps to the newest group, so a +slow reader stays bounded. Same in the moq-mux wrappers and moq-ffi. + moq-ffi's json/binary consumers return the timestamp too, and the py, swift, kt, go, and dart wrappers and `doc/lib/*` follow. JS is [JS data consumer timestamps](/quest/m1/js-data-consumer-timestamps.md). diff --git a/quest/m1/js-data-consumer-timestamps.md b/quest/m1/js-data-consumer-timestamps.md index e706c396b0c..1f655a4db41 100644 --- a/quest/m1/js-data-consumer-timestamps.md +++ b/quest/m1/js-data-consumer-timestamps.md @@ -1,4 +1,4 @@ -# [S] JS data consumers return each value's timestamp +# [M] JS data consumers return each value's timestamp ## Goal @@ -15,8 +15,11 @@ beside video and today reads raw frames to recover the timestamps. Decided (2026-10-01): `next()` and the async iterator yield `@moq/net`'s `Timed` (`{ value, at? }`), the type the producers take after [JS publishing never invents a timestamp](/quest/m1/js-publish-timestamp.md). -`at` is the frame's timestamp, absent for an untimed frame. A snapshot -consumer returns the timestamp of the frame it decoded last. Update +`at` is the frame's timestamp, absent for an untimed frame. Snapshot +consumers stop coalescing, as in Rust: `@moq/json` yields every +reconstructed state in order instead of draining to the latest, and +`@moq/binary` reads groups in order instead of skipping to the newest. A +reader that falls behind still jumps to the newest group. Update `doc/lib/js/{json,binary}.md`. Public API: breaking, on `dev`. Wire: none. diff --git a/quest/m2/watch-data-sync.md b/quest/m2/watch-data-sync.md index 557a53b7f6b..c63a06bf99c 100644 --- a/quest/m2/watch-data-sync.md +++ b/quest/m2/watch-data-sync.md @@ -14,9 +14,14 @@ releases it. (built-in section or an application section embedding the config), releases each payload when the playhead reaches its frame timestamp, and registers its `delay` and `jitter` with `Sync` like a media rendition. -- Snapshot tracks release the newest value at or before the playhead; stream - tracks release every record in order. -- An untimed payload has no playhead position; release it on arrival. +- Snapshot tracks release the newest state at or before the playhead, picked + from the in-order states the consumer yields; stream tracks release every + record in order. +- An untimed payload adds no timestamp wait of its own. On a stream track it + still waits for the records before it, so a timed record at 11s followed by + an untimed one releases both at an 11s playhead, in order. An untimed + snapshot state applies once the states before it have. Test both + sequences. - OneTooMany is the application this waited for (2026-10-01): their web frontend holds KLV and MAVLink telemetry back to the video playhead with its own sync code, which this replaces. In m2 rather than m1 because From 48aaa97e4e186de3859524df97c42df29ac93be7 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 1 Oct 2026 15:18:04 -0700 Subject: [PATCH 3/3] quest: snapshot consumers read with next() in order and latest() skipping ahead Co-Authored-By: Claude Opus 5.5 --- quest/m1/README.md | 4 ++-- quest/m1/data-consumer-timestamps.md | 25 +++++++++++++++---------- quest/m1/js-data-consumer-timestamps.md | 8 ++++---- 3 files changed, 21 insertions(+), 16 deletions(-) diff --git a/quest/m1/README.md b/quest/m1/README.md index 15c8df6b502..03cd97377b7 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -75,8 +75,8 @@ QUIC studies there on that rule. - [Publishing never invents a timestamp](/quest/m1/publish-timestamp.md) - no Rust or binding publish API fills in a timestamp; an untimed payload goes out untimed, and moq-mux data producers take broadcast-clock timestamps - [JS publishing never invents a timestamp](/quest/m1/js-publish-timestamp.md) - the same in @moq/json, @moq/binary, and @moq/net - [Plan: untimed objects](/quest/m1/plan-untimed-objects.md) - map faithful timestamp absence, publisher to consumer, into implementation quests -- [Data consumer timestamps](/quest/m1/data-consumer-timestamps.md) - json and binary consumers return each value's timestamp, in Rust and every binding; snapshots yield every state -- [JS data consumer timestamps](/quest/m1/js-data-consumer-timestamps.md) - @moq/json and @moq/binary consumers return each value's timestamp, and snapshots stop coalescing +- [Data consumer timestamps](/quest/m1/data-consumer-timestamps.md) - json and binary consumers return each value's timestamp, in Rust and every binding; snapshots add `latest()` beside an in-order `next()` +- [JS data consumer timestamps](/quest/m1/js-data-consumer-timestamps.md) - @moq/json and @moq/binary consumers return each value's timestamp, with snapshot `next()` and `latest()` - [Nested data configs](/quest/m1/data-config-nesting.md) - docs nest `BinaryConfig`/`JsonConfig` in an application section instead of flattening it - [Omit empty catalog sections](/quest/m1/catalog-omit-empty.md) - a Rust catalog with no video or audio leaves those keys out, as JS does - [Delete SourceMap](/quest/m1/source-map-removal.md) - on dev, `Anchor` with its `Lane`s is the one way onto the broadcast clock diff --git a/quest/m1/data-consumer-timestamps.md b/quest/m1/data-consumer-timestamps.md index 85834f0bd54..82e86a78fda 100644 --- a/quest/m1/data-consumer-timestamps.md +++ b/quest/m1/data-consumer-timestamps.md @@ -15,8 +15,8 @@ sync with video. Every `moq_net::Frame` has a timestamp, but the consumers decode only `frame.payload` and drop it: `moq_mux::{json,binary}::Consumer::next` and the -moq-json and moq-binary snapshot and stream consumers they wrap. A snapshot -consumer returns the timestamp of the frame it decoded last. +moq-json and moq-binary snapshot and stream consumers they wrap. Each snapshot +state carries the timestamp of the frame that produced it. Decided: `next()` and `poll_next()` return `Timed`, the type the producers take in [Publishing never invents a timestamp](/quest/m1/publish-timestamp.md). @@ -25,14 +25,19 @@ an untimed frame: [Plan: untimed objects](/quest/m1/plan-untimed-objects.md) settled (2026-10-01) that absence survives the wire rather than becoming arrival time. A republisher passes `at` straight to a moq-mux data producer. -Decided (2026-10-01): snapshot consumers stop coalescing. Today the moq-json -snapshot consumer applies every buffered delta but yields only the newest -state, and moq-binary jumps to the newest group, so a 9s state is lost when -11s is already buffered. A caller syncing to a playhead needs the newest -state at or before it, so every state is yielded in order with its `at`, and -a caller that wants only the latest keeps the last one. A reader that falls -behind the drift budget (`Lagged`) still jumps to the newest group, so a -slow reader stays bounded. Same in the moq-mux wrappers and moq-ffi. +Decided (2026-10-01): snapshot consumers get two reads, both returning +`Timed`. Today the moq-json snapshot consumer applies every buffered delta +but yields only the newest state, and moq-binary jumps to the newest group, so +a 9s state is lost when 11s is already buffered. A caller syncing to a +playhead needs the newest state at or before it. + +- `next()` (and `poll_next()`) yields every state in order, like a stream + consumer. A reader that falls behind the drift budget (`Lagged`) still + jumps to the newest group, so a slow reader stays bounded. +- `latest()` (and `poll_latest()`) skips to the newest state, today's + behavior, kept as the optimization for callers that only want the current + value. +- Same names in the moq-mux wrappers, moq-ffi, and every binding. moq-ffi's json/binary consumers return the timestamp too, and the py, swift, kt, go, and dart wrappers and `doc/lib/*` follow. JS is diff --git a/quest/m1/js-data-consumer-timestamps.md b/quest/m1/js-data-consumer-timestamps.md index 1f655a4db41..90657f35aa7 100644 --- a/quest/m1/js-data-consumer-timestamps.md +++ b/quest/m1/js-data-consumer-timestamps.md @@ -16,10 +16,10 @@ Decided (2026-10-01): `next()` and the async iterator yield `@moq/net`'s `Timed` (`{ value, at? }`), the type the producers take after [JS publishing never invents a timestamp](/quest/m1/js-publish-timestamp.md). `at` is the frame's timestamp, absent for an untimed frame. Snapshot -consumers stop coalescing, as in Rust: `@moq/json` yields every -reconstructed state in order instead of draining to the latest, and -`@moq/binary` reads groups in order instead of skipping to the newest. A -reader that falls behind still jumps to the newest group. Update +consumers get the same two reads as Rust: `next()` yields every state in +order (`@moq/json` stops draining to the latest, `@moq/binary` reads groups in +order), and `latest()` skips to the newest state, today's behavior. A reader +that falls behind still jumps to the newest group. Update `doc/lib/js/{json,binary}.md`. Public API: breaking, on `dev`. Wire: none.