diff --git a/doc/concept/moq-lite.md b/doc/concept/moq-lite.md index a05994b1124..8b23bace25c 100644 --- a/doc/concept/moq-lite.md +++ b/doc/concept/moq-lite.md @@ -349,13 +349,13 @@ frames go out stamped with their send time; the work-in-progress 07 carries it untimed. A publisher refuses a group that starts before a lower one, and so does a -moq-lite 07 receiver. Since moq-lite 05 and 06 send times look like media time, -a receiver drops every timestamp it gets over them and the track arrives -untimed. -Timestamps received over moq-transport, whose peers may stamp arrival times, -are not checked. Rust relays preserve them across moq-lite 07 when both peers -negotiate the [ietf extension](/draft/moq-ietf), which carries their origin and -keeps them unchecked. A moq-lite peer without it receives that track untimed. +moq-lite 07 receiver. A receiver passes on the timestamps it gets over moq-lite +05 and 06 without checking them, since their send times look like media time and +an older group fetched later would look like a rewind. Timestamps received over +moq-transport, whose peers may stamp arrival times, are not checked either. A +relay serves both on over every version. Over moq-lite 07 the +[ietf extension](/draft/moq-ietf) keeps moq-transport ones unchecked; without +it, the receiver checks them and may refuse a group that rewinds. A publisher may [compress](/concept/compression) a track, which pays off for repetitive data such as JSON and not for media. Each group's frames become one diff --git a/doc/setup/upgrade.md b/doc/setup/upgrade.md index b884b90e79c..bb20728bb01 100644 --- a/doc/setup/upgrade.md +++ b/doc/setup/upgrade.md @@ -83,10 +83,8 @@ These land with the next breaking release. receive.** Tracks from moq-lite before 05, and moq-transport tracks without `TIMESCALE` arrive untimed instead of stamped with their arrival time. moq-lite 05 and 06 can't encode absence and - carry a send time instead, so their receivers drop every timestamp and those - tracks arrive untimed too: read media time from the payload. moq-transport - timestamps are served to moq-lite untimed; see - [untimed tracks](/concept/moq-lite#subscriptions). A new subscriber with no + carry a send time instead, which their receivers pass through as the track's + time; see [untimed tracks](/concept/moq-lite#subscriptions). A new subscriber with no start on an untimed track starts at the latest group. - **Max age and max delay count wall-clock time too.** A superseded group is stale once its successor arrived a full budget ago, as well as by media diff --git a/drafts/draft-lcurley-moq-ietf.md b/drafts/draft-lcurley-moq-ietf.md index 9ef9dd9068a..bfb6fb929f8 100644 --- a/drafts/draft-lcurley-moq-ietf.md +++ b/drafts/draft-lcurley-moq-ietf.md @@ -143,10 +143,10 @@ The Version also gives each property its meaning: drafts 18 and 19 serialize a b Other properties are opaque to this extension. ## Timestamps {#timestamps} -A relay MUST preserve a relayed track's Timescale and timestamps across a negotiated hop, except in a listing ({{track}}). +A relay MUST preserve a relayed track's Timescale and timestamps across a negotiated hop. A nonzero Version identifies timestamps from MoQ Transport, which may be arrival times and need not rise with Group Sequence. The receiver MUST NOT apply moq-lite's group-start rewind check to such a track; a Version of zero retains that check. -Without this extension, a relay MUST serve a MoQ Transport track untimed as {{moql}} requires. +Without this extension, or in a listing ({{track}}), the receiver may check them like native moq-lite timestamps and refuse a Group that rewinds. On a timed track, the FRAME's Timestamp, or the datagram's for a datagram, is the object's time, and nothing else carries it. The Property Block MUST NOT contain a mutable Timestamp (0x10) there; a receiver MUST treat one as malformed. @@ -199,7 +199,7 @@ The receiver parses the keys inside the Immutable Properties from its value, whi It SHOULD read them from the value when needed rather than hold them beside it, as a list of IDs can cost several times the value. A TRACKS_ADD ({{moql}} Tracks Stream) embeds TRACK_INFO without TRACK_PROPERTIES, so when the extension is negotiated a subscriber MUST NOT skip the Track Stream for a track it learned from a listing. -Lacking the Version, a TRACKS_ADD cannot mark a {{moqt}} track's timestamps as unchecked ({{timestamps}}), so it lists that track untimed even when the extension is negotiated. +Lacking the Version, a TRACKS_ADD cannot mark a {{moqt}} track's timestamps as unchecked ({{timestamps}}). # Datagrams {#datagrams} @@ -264,7 +264,7 @@ A high, distinctive value is used to avoid collisions with the low values {{moql ## Since draft-lcurley-moq-ietf-00 (in progress) -- Preserve relayed track timestamps and their unchecked MoQ Transport provenance across negotiated hops. +- Preserve relayed track timestamps and their unchecked MoQ Transport provenance across negotiated hops. Without the extension a relay serves them as native moq-lite timestamps rather than untimed. - Initial version. # Acknowledgments diff --git a/drafts/draft-lcurley-moq-lite.md b/drafts/draft-lcurley-moq-lite.md index a43c4cce9a7..00117e52122 100644 --- a/drafts/draft-lcurley-moq-lite.md +++ b/drafts/draft-lcurley-moq-lite.md @@ -190,7 +190,7 @@ A Track without a `Timescale` is untimed: its frames and datagrams carry no time A timestamp MUST be less than 2^61, so every version can carry it; a receiver treats a larger one as a malformed Track. Unless a negotiated extension specifies otherwise, a publisher MUST NOT start a timed Group's frame 0 earlier than that of any lower-sequence Group it publishes. A receiver MAY check a received Group against this rule, refusing a violating Group's stream (STOP_SENDING) with TIMESTAMP_MISMATCH. -A relay that cannot vouch for a Track's timestamps, such as ones received over another protocol, MUST serve the Track untimed unless a negotiated extension identifies their unchecked provenance. +A relay serves a Track's timestamps on as it received them, including ones it did not check, unless a negotiated extension specifies otherwise. ## Positions {#positions} A Position is a (Group Sequence, Frame Index) pair identifying one frame within a Track. @@ -1832,7 +1832,7 @@ The `Message Length` describes the payload size on the wire. - Replaced ANNOUNCE_OK Active Count with an ordered, empty SYNCED event (0x4), marking the initial set without retaining routes that end before delivery. -- A publisher must not start a timed Group's frame 0 earlier than a lower-sequence Group's, and a receiver may refuse a Group that does. A relay serves a Track whose timestamps it cannot vouch for untimed unless a negotiated extension identifies their unchecked provenance. +- A publisher must not start a timed Group's frame 0 earlier than a lower-sequence Group's, and a receiver may refuse a Group that does. A relay serves every Track's timestamps on as received. - Bounded every timestamp below 2^61. A larger one is a malformed Track. diff --git a/js/e2ee/src/transport.test.ts b/js/e2ee/src/transport.test.ts index 6731284b1c0..42ae2e5ccaa 100644 --- a/js/e2ee/src/transport.test.ts +++ b/js/e2ee/src/transport.test.ts @@ -48,10 +48,10 @@ async function active(origin: Origin.Producer, path: Path.Valid): Promise WireFrame { +export function frameDecoder({ scale, compress = false }: FrameFormat): (c: Cursor) => WireFrame { const timescale = scale === 0 ? undefined : Time.Timescale(scale); let prevTs = 0n; return (c) => { @@ -100,8 +98,7 @@ export function frameDecoder({ scale, timed = true, compress = false }: FrameFor const payload = c.read(size); // Commit the delta only after the last read: an incomplete decode gets retried. prevTs += delta; - // Read to stay in step, but dropped where it may be a send time. - const timestamp = timescale === undefined || !timed ? undefined : new Time.Timestamp(Number(prevTs), timescale); + const timestamp = timescale === undefined ? undefined : new Time.Timestamp(Number(prevTs), timescale); return { payload, inflated, timestamp }; }; } diff --git a/js/net/src/lite/publisher.test.ts b/js/net/src/lite/publisher.test.ts index 03233fd214b..7f46c64c057 100644 --- a/js/net/src/lite/publisher.test.ts +++ b/js/net/src/lite/publisher.test.ts @@ -6,7 +6,6 @@ import { Producer as BroadcastProducer } from "../broadcast.ts"; import { fromTransport, SessionCode, StreamCode, StreamError, toStreamCode } from "../error.ts"; import type { Producer as GroupProducer } from "../group.ts"; import { randomHop } from "../hop.ts"; -import { Version as IetfVersion } from "../ietf/version.ts"; import { hooks } from "../internal.ts"; import { createMockTransportPair, textFrame } from "../mock.ts"; import { Producer as OriginProducer } from "../origin.ts"; @@ -599,15 +598,15 @@ test.each([0, 5, 6, 7])("a fetched cached tail keeps absolute indices from %s", } }); -// An unchecked track goes out untimed, so lite-06 carries its send time, never the frame's own -// timestamp, which may be a moq-transport arrival time. -test("lite draft-06: an unchecked track's fetched frames carry their send time", async () => { +// An unchecked track's frames go out with their own timestamps, which may be moq-transport +// arrival times, rather than their send time. +test("lite draft-06: an unchecked track's fetched frames carry their own timestamp", async () => { const pair = createMockTransportPair(ALPN_06); const origin = new OriginProducer(); const publisher = new Publisher(pair.server, Version.DRAFT_06, randomHop(), origin.consume()); const broadcast = publish(origin, Path.from("test")); - const track = broadcast.createTrack("video", { timescale: Timescale.MICRO, ietf: IetfVersion.DRAFT_22 }); + const track = broadcast.createTrack("video", { timescale: Timescale.MICRO, unchecked: true }); const group = track.createGroup(0); group.writeFrame(textFrame("hello", Timestamp.fromMillis(36_000_000))); @@ -619,14 +618,10 @@ test("lite draft-06: an unchecked track's fetched frames carry their send time", const msg = new Fetch({ broadcast: Path.from("test"), track: "video", priority: 0, group: 0 }); try { - const before = Math.floor(Timestamp.now().value); await publisher.runFetch(msg, server); - const after = Math.ceil(Timestamp.now().value); - // The first frame's zigzag delta is against 0, in the milliseconds TRACK_INFO declares. - const sent = Number((await client.reader.u62()) >> 1n); - expect(sent).toBeGreaterThanOrEqual(before); - expect(sent).toBeLessThanOrEqual(after); + // The first frame's zigzag delta is against 0, in the microseconds TRACK_INFO declares. + expect(await client.reader.u62()).toBe(36_000_000_000n << 1n); } finally { publisher.close(); client.close(); diff --git a/js/net/src/lite/publisher.ts b/js/net/src/lite/publisher.ts index 9d710009f7f..b4727c9731f 100644 --- a/js/net/src/lite/publisher.ts +++ b/js/net/src/lite/publisher.ts @@ -97,17 +97,12 @@ interface Stamp { timescale: Timescale; /** * Whether each frame's own timestamp goes out. Lite-05/06 cannot encode an untimed track, so - * one served untimed carries its send time instead, unchecked ones included. + * an untimed one carries its send time instead. */ timed: boolean; } -/** The timescale a moq-lite hop serves a track at: none for an {@link track.Info.ietf} one. */ -function liteTimescale(info: track.Info): Timescale | undefined { - return info.ietf === undefined ? (info.timescale ?? undefined) : undefined; -} - -/** How `version` stamps a track served at `timescale` (see `liteTimescale`), or undefined when it carries none. */ +/** How `version` stamps a track with `timescale`, or undefined when it carries none. */ function stamping(version: Version, timescale: Timescale | undefined): Stamp | undefined { const wire = wireTimescale(version, timescale); return wire === undefined ? undefined : { timescale: wire, timed: timescale !== undefined }; @@ -134,9 +129,8 @@ function trackInfoMessage(info: track.Info, version: Version): TrackInfoMessage // Publisher Max Age: the publisher's retention bound, advertised so relays re-serve // with the same window. maxAge: info.maxAge, - // `#serveGroup` emits each frame converted to it. Unchecked timestamps never enter - // moq-lite, whose receivers check them. - timescale: liteTimescale(info), + // `#serveGroup` emits each frame converted to it. + timescale: info.timescale ?? undefined, // An older peer is served the frames inflated. compress: hasCompression(version) && !!info.compress, }); @@ -943,7 +937,7 @@ export class Publisher { // The timescale is an immutable property, so serving MUST use exactly // what TRACK_INFO advertised. It comes from the producer's accept(), so // they always agree. - stamp = stamping(this.version, liteTimescale(info)); + stamp = stamping(this.version, info.timescale ?? undefined); compress = hasCompression(this.version) && !!info.compress; } else { // Older drafts acknowledge with SUBSCRIBE_OK and stream frames verbatim. diff --git a/js/net/src/lite/subscriber.test.ts b/js/net/src/lite/subscriber.test.ts index d03d309e5af..c2d7299554b 100644 --- a/js/net/src/lite/subscriber.test.ts +++ b/js/net/src/lite/subscriber.test.ts @@ -22,6 +22,7 @@ import { hooks } from "../internal.ts"; import { GRACE } from "../ledger.ts"; import * as Path from "../path.ts"; import { Reader, Writer } from "../stream.ts"; +import { Timescale } from "../time.ts"; import type { Producer } from "../track.ts"; import { type AnnounceBroadcast, AnnounceInit, AnnounceOk, encodeAnnounceBroadcast } from "./announce.ts"; import { Datagram as DatagramMessage } from "./datagram.ts"; @@ -2259,9 +2260,9 @@ async function groupStream(timestamp: number, version: Version = Version.DRAFT_0 } // Lite-05/06 stamp an untimed track's frames with their send time, which a receiver cannot tell -// from media time, so it drops every timestamp there: a group that rewinds or lands out of order is -// accepted untimed. -test.each([Version.DRAFT_05, Version.DRAFT_06])("version %p drops received timestamps", async (version) => { +// from media time, so it keeps every timestamp there unchecked: a group that rewinds or lands out +// of order is accepted with its timestamp. +test.each([Version.DRAFT_05, Version.DRAFT_06])("version %p passes received timestamps through", async (version) => { const { quic, streams, opened } = fakeSession(); const subscriber = new Subscriber(quic, version, HopSchema.parse(1n)); const broadcast = subscriber.consume(Path.from("room")); @@ -2280,14 +2281,14 @@ test.each([Version.DRAFT_05, Version.DRAFT_06])("version %p drops received times for (let i = 0; i < 3; i++) { const group = await track.recvGroup(); const frame = await group?.readFrame(); - got.push([group?.sequence, frame?.payload.length, frame?.timestamp]); + got.push([group?.sequence, frame?.payload.length, frame?.timestamp?.value]); } expect(got).toEqual([ - [3, 1, undefined], - [4, 1, undefined], - [5, 1, undefined], + [3, 1, 2000], + [4, 1, 1000], + [5, 1, 500], ]); - expect((await track.info()).timescale).toBeUndefined(); + expect((await track.info()).timescale).toBe(Timescale.MILLI); expect(track.closed.peek()).toBeUndefined(); } finally { track.close(); diff --git a/js/net/src/lite/subscriber.ts b/js/net/src/lite/subscriber.ts index 3230fb97838..ee8d36fc45d 100644 --- a/js/net/src/lite/subscriber.ts +++ b/js/net/src/lite/subscriber.ts @@ -1212,14 +1212,15 @@ export class Subscriber { // Map the wire TRACK_INFO onto the model track.Info a producer/consumer holds. #toModelInfo(info: TrackInfo): track.Info { return { - // Lite-05/06 timestamps may be send times, so the copy is untimed and the ingest drops them. - timescale: hasMediaTime(this.version) ? info.timescale : undefined, + timescale: info.timescale, // Publisher Max Age rides on the wire, so the local media-time budget // matches what the upstream advertises (relays re-serve with the same bound). maxAge: info.maxAge === undefined ? undefined : Milli(info.maxAge), priority: info.priority, refresh: info.refresh, compress: info.compress, + // Lite-05/06 timestamps may be send times, so the copy's groups skip the rewind check. + ...(hasMediaTime(this.version) ? {} : { unchecked: true }), }; } @@ -1426,7 +1427,6 @@ export class Subscriber { try { const decode = frameDecoder({ scale: info.timescale ?? 0, - timed: hasMediaTime(this.version), compress: info.compress, }); @@ -1738,7 +1738,6 @@ export class Subscriber { await readFrames(stream, producer, { scale: timescale, - timed: hasMediaTime(this.version), compress: entry.compress, }); @@ -1820,11 +1819,8 @@ export class Subscriber { const scale = entry.timescale; - // Dropped where it may be a send time, like a frame's. const timestamp = - dg.timestamp === undefined || !hasMediaTime(this.version) - ? undefined - : new Time.Timestamp(dg.timestamp, Time.Timescale(scale)); + dg.timestamp === undefined ? undefined : new Time.Timestamp(dg.timestamp, Time.Timescale(scale)); // A datagram's sequence is never owed a stream, so it never holds the tail open. entry.tail.account(dg.sequence, dg.sequence + 1); hooks.writeDatagram(entry.track, { sequence: dg.sequence, timestamp, payload: dg.payload }, dg.inflated); diff --git a/js/net/src/lite/track_stream.test.ts b/js/net/src/lite/track_stream.test.ts index 41210a837db..2a07d272e32 100644 --- a/js/net/src/lite/track_stream.test.ts +++ b/js/net/src/lite/track_stream.test.ts @@ -4,7 +4,6 @@ import { describe, expect, test } from "bun:test"; import { Elapsed, Milli, Nano, timeout } from "@moq/clock"; import { randomHop } from "../hop.ts"; -import { Version as IetfVersion } from "../ietf/version.ts"; import { createMockTransportPair } from "../mock.ts"; import { Producer as OriginProducer } from "../origin.ts"; import * as Path from "../path.ts"; @@ -382,10 +381,8 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver origin.close(); }); - // Timestamps received over moq-transport may be arrival times, so they never enter moq-lite, - // whose receivers check every timestamp: the track is served untimed (milliseconds of send - // time before lite-07). - test("an unchecked track is served untimed", async () => { + // An unchecked track's timestamps go out as received, so TRACK_INFO declares its own scale. + test("an unchecked track is served with its timescale", async () => { const pair = createMockTransportPair(ALPN_05); const origin = new OriginProducer(); const publisher = new Publisher(pair.server, version, randomHop(), origin.consume()); @@ -396,7 +393,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver for (;;) { const request = await wireOf(broadcast).requested(); if (!request) return; - request.accept({ timescale: Timescale.MICRO, ietf: IetfVersion.DRAFT_22 }); + request.accept({ timescale: Timescale.MICRO, unchecked: true }); } })(); @@ -406,7 +403,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver void publisher.runTrackInfo(new TrackMessage(Path.from("room"), "video"), server); const info = await TrackInfo.decode(client.reader, version); - expect(info.timescale).toBe(version === Version.DRAFT_07 ? undefined : Timescale.MILLI); + expect(info.timescale).toBe(Timescale.MICRO); client.writer.close(); publisher.close(); diff --git a/js/net/src/lite/version.ts b/js/net/src/lite/version.ts index 15bf5727be3..af46a947bb8 100644 --- a/js/net/src/lite/version.ts +++ b/js/net/src/lite/version.ts @@ -481,8 +481,8 @@ export function hasRefresh(version: Version): boolean { } /** Whether received frame timestamps are the publisher's media time. Lite-05/06 stamp an untimed - * track's payloads with their send time, which a receiver cannot tell from it, so it drops every - * timestamp received on them and the track arrives untimed. */ + * track's payloads with their send time, which a receiver cannot tell from it, so it passes them + * through as the track's time without checking them for a rewind. */ export function hasMediaTime(version: Version): boolean { // Explicitly list older versions so future versions keep the lite-07+ behavior. switch (version) { diff --git a/js/net/src/resume.test.ts b/js/net/src/resume.test.ts index e1c7f590a66..74030c17fb0 100644 --- a/js/net/src/resume.test.ts +++ b/js/net/src/resume.test.ts @@ -7,6 +7,7 @@ import { type Consumer as GroupConsumer, Producer as GroupProducer } from "./gro import { Route } from "./hop.ts"; import type { Session as IetfSession } from "./ietf/session.ts"; import { Subscriber as IetfSubscriber } from "./ietf/subscriber.ts"; +import { Version as IetfVersion } from "./ietf/version.ts"; import { hooks, MAX_DATAGRAMS } from "./internal.ts"; import type { Lister } from "./listing.ts"; import { Producer } from "./origin.ts"; @@ -640,19 +641,43 @@ test("a group no route continues is given up once a newer one supersedes it", as close(); }); -test("a replacement claiming different track properties is refused once the old copy runs out", async () => { +// Moq-transport provenance is content, like Rust's `properties`: a copy over another draft differs. +for (const [name, before, after] of [ + ["track properties", { priority: 1 }, { priority: 2 }], + ["moq-transport provenance", { ietf: IetfVersion.DRAFT_22 }, {}], +] as const) { + test(`a replacement claiming different ${name} is refused once the old copy runs out`, async () => { + const { a, b, swap, active, close } = await setup(); + const old = a.createTrack("seq", before); + b.createTrack("seq", after); + const sub = active.track("seq").subscribe(); + expect(await sub.info()).toMatchObject(before); + await swap(); + expect(sub.closed.peek()).toBeUndefined(); + + // The old copy failing with its route is what ends the track. + old.abort(new StreamError(StreamCode.SessionClosed)); + await settle(); + expect(() => sub.tryRecvGroup()).toThrow("changed its properties"); + + sub.close(); + close(); + }); +} + +// `unchecked` records how a copy arrived (lite-05/06), so it splices beside a lite-07 copy. +test("a replacement differing only in unchecked takes over", async () => { const { a, b, swap, active, close } = await setup(); - const old = a.createTrack("seq", { priority: 1 }); - b.createTrack("seq", { priority: 2 }); + const old = a.createTrack("seq", { unchecked: true }); + const tb = copy(b); const sub = active.track("seq").subscribe(); - expect((await sub.info()).priority).toBe(1); + await sub.info(); await swap(); - expect(sub.closed.peek()).toBeUndefined(); - - // The old copy failing with its route is what ends the track. + tb.write(0); old.abort(new StreamError(StreamCode.SessionClosed)); await settle(); - expect(() => sub.tryRecvGroup()).toThrow("changed its properties"); + expect(sub.closed.peek()).toBeUndefined(); + expect(drain(sub)).toEqual([0]); sub.close(); close(); diff --git a/js/net/src/resume.ts b/js/net/src/resume.ts index 92f4ca46105..b7450b95376 100644 --- a/js/net/src/resume.ts +++ b/js/net/src/resume.ts @@ -104,7 +104,9 @@ export function routeFailed(err: Error): boolean { return err instanceof SessionError || sessionClosed(err); } -// Every `track.Info` field, so one added later fails to compile until it is listed here. +// Every `track.Info` field but `unchecked`, so one added later fails to compile until it is +// listed here. `unchecked` says how a copy arrived, not what it holds, so copies over different +// lite versions still match. `ietf` is compared, like Rust's moq-transport `properties`. const INFO_FIELDS = Object.keys({ compress: true, timescale: true, @@ -112,7 +114,7 @@ const INFO_FIELDS = Object.keys({ priority: true, refresh: true, ietf: true, -} satisfies Record) as (keyof track.Info)[]; +} satisfies Record, true>) as (keyof track.Info)[]; /** Two copies claiming the same content must agree on all of it, like Rust's `Info` equality. */ function compatible(a: track.Info, b: track.Info): boolean { diff --git a/js/net/src/track.test.ts b/js/net/src/track.test.ts index 7abd4e4a41a..b2fb40a7fa9 100644 --- a/js/net/src/track.test.ts +++ b/js/net/src/track.test.ts @@ -3408,22 +3408,27 @@ test("an endless track with gaps remembers only those within the budget", () => } }); -test("IETF provenance survives normalization on timed and untimed tracks", () => { - for (const timescale of [undefined, Timescale.MILLI]) { - const info = infoDefaults({ ietf: IetfVersion.DRAFT_22, timescale }); - expect(info.ietf).toBe(IetfVersion.DRAFT_22); - expect(info.timescale).toBe(timescale); - } - const producer = new TrackProducer("ietf").accept({ timescale: Timescale.MILLI, ietf: IetfVersion.DRAFT_22 }); - try { - stampGroup(producer, 0, 1000); - stampGroup(producer, 1, 500); - producer.writeFrame({ payload: enc.encode("arrival"), timestamp: Timestamp.fromMillis(250) }); - } finally { - producer.finish(); - producer.close(); - } -}); +for (const [name, extra] of [ + ["unchecked", { unchecked: true }], + ["IETF provenance", { ietf: IetfVersion.DRAFT_22 }], +] as const) { + test(`${name} survives normalization and skips the rewind check`, () => { + for (const timescale of [undefined, Timescale.MILLI]) { + const info = infoDefaults({ ...extra, timescale }); + expect(info).toMatchObject(extra); + expect(info.timescale).toBe(timescale); + } + const producer = new TrackProducer("rewind").accept({ timescale: Timescale.MILLI, ...extra }); + try { + stampGroup(producer, 0, 1000); + stampGroup(producer, 1, 500); + producer.writeFrame({ payload: enc.encode("arrival"), timestamp: Timestamp.fromMillis(250) }); + } finally { + producer.finish(); + producer.close(); + } + }); +} test("an appended rewind is refused and an equal start is not", () => { const producer = new TrackProducer("test").accept({ timescale: Timescale.MILLI }); diff --git a/js/net/src/track.ts b/js/net/src/track.ts index ecac970601a..14076c384e0 100644 --- a/js/net/src/track.ts +++ b/js/net/src/track.ts @@ -157,6 +157,12 @@ export interface Info { refresh?: number; /** The original moq-transport publisher's draft, whose timestamps may rewind. @internal */ ietf?: IetfVersion; + /** + * Whether this copy's groups skip the rewind check, since its timestamps may be lite-05/06 send + * times. An {@link Info.ietf} track skips it too. Mirrors the Rust `track::Info::unchecked`. + * @internal + */ + unchecked?: boolean; } // Normalize a duration for the wire, which carries it as an unsigned varint. @@ -195,6 +201,7 @@ export function infoDefaults(info: Partial = {}): Info { priority: priorityValue(info.priority ?? DEFAULT_PRIORITY), refresh: info.refresh == null ? undefined : refreshRatio(info.refresh), ...(info.ietf === undefined ? {} : { ietf: info.ietf }), + ...(info.unchecked ? { unchecked: true } : {}), }; } @@ -1474,9 +1481,10 @@ export class Producer { return group; } - // Whether this track's timestamps may rewind (see {@link Info.ietf}). + // Whether this track's timestamps may rewind (see {@link Info.unchecked} and {@link Info.ietf}). #unchecked(): boolean { - return this.#state.info.peek()?.ietf !== undefined; + const info = this.#state.info.peek(); + return info?.unchecked === true || info?.ietf !== undefined; } // Fan a datagram out to every live subscriber, dropping the oldest once the ring is full. diff --git a/js/net/src/untimed.test.ts b/js/net/src/untimed.test.ts index 3579fbf1b8d..8339b7cec7c 100644 --- a/js/net/src/untimed.test.ts +++ b/js/net/src/untimed.test.ts @@ -1,5 +1,6 @@ -// A track keeps its timedness end to end: no receive path fills in arrival time, and a -// wire that can't declare a timeline delivers frames untimed. Mirrors rs/moq-net/tests/integration/untimed.rs. +// A track keeps its timedness end to end: no receive path fills in arrival time, a wire that +// can't declare a timeline delivers frames untimed, and lite-05/06, which can't declare an +// untimed one, deliver its send times. Mirrors rs/moq-net/tests/integration/untimed.rs. import { expect, test } from "bun:test"; import { type Consumer as BroadcastConsumer, Producer as BroadcastProducer } from "./broadcast.ts"; import { accept, connect } from "./connection/index.ts"; @@ -19,18 +20,20 @@ interface Wire { name: string; protocol: string; version?: number; - /** Whether a timed track arrives timed. Lite-05/06 receivers drop timestamps, which may be send times. */ + /** Whether a timed track arrives timed. */ declares: boolean; + /** Whether an untimed track arrives timed in milliseconds with its send times, as on lite-05/06. */ + sends: boolean; } const WIRES: Wire[] = [ - { name: "lite-01", protocol: "", version: Lite.Version.DRAFT_01, declares: false }, - { name: "lite-03", protocol: Lite.ALPN_03, declares: false }, - { name: "lite-04", protocol: Lite.ALPN_04, declares: false }, - { name: "lite-05", protocol: Lite.ALPN_05, declares: false }, - { name: "lite-06", protocol: Lite.ALPN_06, declares: false }, - { name: "lite-07-wip", protocol: Lite.ALPN_07_WIP, declares: true }, - { name: "ietf-22", protocol: Ietf.ALPN.DRAFT_22, declares: true }, + { name: "lite-01", protocol: "", version: Lite.Version.DRAFT_01, declares: false, sends: false }, + { name: "lite-03", protocol: Lite.ALPN_03, declares: false, sends: false }, + { name: "lite-04", protocol: Lite.ALPN_04, declares: false, sends: false }, + { name: "lite-05", protocol: Lite.ALPN_05, declares: true, sends: true }, + { name: "lite-06", protocol: Lite.ALPN_06, declares: true, sends: true }, + { name: "lite-07-wip", protocol: Lite.ALPN_07_WIP, declares: true, sends: false }, + { name: "ietf-22", protocol: Ietf.ALPN.DRAFT_22, declares: true, sends: false }, ]; /** Serve one group of `timestamps` on a track with `timescale`, and read it back over `wire`. */ @@ -155,11 +158,11 @@ test("a forwarded untimed track stays untimed", async () => { }); for (const wire of WIRES) { - test(`${wire.name}: an untimed track arrives untimed`, async () => { + test(`${wire.name}: an untimed track arrives ${wire.sends ? "with its send times" : "untimed"}`, async () => { const { info, frames } = await roundTrip(wire, undefined, [undefined, undefined]); expect(frames.map((frame) => frame.payload[0])).toEqual([0, 1]); - expect(info.timescale).toBeUndefined(); - for (const frame of frames) expect(frame.timestamp).toBeUndefined(); + expect(info.timescale).toBe(wire.sends ? Timescale.MILLI : undefined); + for (const frame of frames) expect(frame.timestamp !== undefined).toBe(wire.sends); }); test(`${wire.name}: a timed track keeps its timestamps only where the wire delivers them`, async () => { @@ -181,7 +184,7 @@ for (const wire of WIRES) { } for (const wire of WIRES.filter((wire) => wire.name === "lite-06" || wire.name === "lite-07-wip")) { - test(`${wire.name}: an untimed datagram arrives untimed`, async () => { + test(`${wire.name}: an untimed datagram arrives ${wire.sends ? "with its send time" : "untimed"}`, async () => { const pair = createMockTransportPair(wire.protocol); const origin = new OriginProducer(); const [client, server] = await Promise.all([ @@ -205,7 +208,7 @@ for (const wire of WIRES.filter((wire) => wire.name === "lite-06" || wire.name = const datagram = await subscriber.recvDatagram(); expect(datagram?.payload).toEqual(new Uint8Array([7])); - expect(datagram?.timestamp).toBeUndefined(); + expect(datagram?.timestamp !== undefined).toBe(wire.sends); broadcast.close(); subscriber.close(); diff --git a/quest/m1/README.md b/quest/m1/README.md index 3e1fec30632..1f21f64ea8b 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -129,7 +129,6 @@ removed). - [Cluster routing](/quest/m1/cluster-routing/README.md) - prove shared routing contracts, implement per-origin routes, and connect outbound local clusters - [Routing guide](/quest/m1/routing-guide.md) - concepts and practical outbound federation setup - [moq-c at parity](/quest/m1/c/README.md) - the hand-written `moq-c` ships at the cut with every m1 binding feature moq-ffi gains -- [Lite-05/06 timestamps pass through](/quest/m1/lite-timestamps-through.md) - a timed track received over lite-05/06 stays timed and is forwarded with its timestamps over every version - [Exact timestamps in every binding](/quest/m1/binding-timestamps.md) - bindings carry `{value, scale}` timestamps instead of lossy microseconds - [Bindings see a moq-transport run end as an end](/quest/m1/ffi-ietf-run-end.md) - an IETF publisher ending its run reaches moq-ffi consumers as an end, not `Internal` - [A broadcast stamps everything from one clock](/quest/m1/broadcast-clock.md) - catalog, MSF, and HLS record groups are stamped from the broadcast's clock, never the ambient one diff --git a/quest/m1/lite-timestamps-through.md b/quest/m1/lite-timestamps-through.md deleted file mode 100644 index a2cfbb5e52c..00000000000 --- a/quest/m1/lite-timestamps-through.md +++ /dev/null @@ -1,41 +0,0 @@ -# [M] Lite-05/06 timestamps pass through - -## Goal - -A timed track received over moq-lite-05/06 keeps its frame and datagram -timestamps, and a relay forwards them over every version, so a lite-06 -reader of a timed feed (such as a stats feed) sees timed frames instead of -untimed ones. Partly reverses -[moq-dev/moq#6150](https://github.com/moq-dev/moq/pull/6150), which drops -every lite-05/06 timestamp a receiver reads. - -## Plan - -Decided 2026-10-25 (maintainer): "We dropped lite timestamps because they -defaulted to wall clock timestamps, but we should pass them through instead." - -- A track received over lite-05/06 is timed and marked crate-private - `unchecked`, like a moq-transport one: its groups skip the - [moq-dev/moq#5803](https://github.com/moq-dev/moq/pull/5803) rewind check - on ingress, because an untimed origin stamps send times at serve time and - an older group fetched later would look like a rewind. -- Unchecked tracks, lite-05/06 and moq-transport alike, go out with their own - timestamps over every version: lite-05/06, lite-07, and moq-transport. A - moq-transport track served to a lite-05/06 peer stops being stamped with - send times. -- A lite-07 receiver keeps checking every received group (no wire flag), so a - send-time origin's out-of-order group may be refused downstream; rewinds - are unsupported, and the cost was accepted. -- A lite-05/06 sender serving an untimed track keeps the published rule: - timescale 1000 and each payload's send time in milliseconds. Receivers now - pass that through as the track's time. - -Update the draft's lite-05/06 receiver text and changelog, -`doc/concept/moq-lite.md`, and the PR's Impact migration note, in Rust -and `@moq/net` alike. The cut's migration audit reconciles the existing #6150 -notes with that Impact section. Test a lite-06 hop between two relays: a timed track -arrives timed with its values, and an out-of-order fetched group from an -untimed origin is accepted on lite-05/06. - -Public API: none (crate-private `unchecked`). Wire: none for lite-05/06; -receiver behavior changes, and the lite draft text says so. diff --git a/quest/m1/time-addressing/model.md b/quest/m1/time-addressing/model.md index 074d17841f5..04f3e4b9fdb 100644 --- a/quest/m1/time-addressing/model.md +++ b/quest/m1/time-addressing/model.md @@ -120,5 +120,4 @@ Wire: no format change; legacy and lite-07-wip group IDs become start ticks. - [Fetch as a range](/quest/m1/subscribe-ranges/fetch-range.md) - the one-range reads this rework moves onto times - [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - the range ends and drops this rework moves onto times - [Delete the old start bookkeeping](/quest/m1/subscribe-ranges/start-bookkeeping.md) - the lite cursor state lives in the model before the cut rekeys it -- [Lite-05/06 timestamps pass through](/quest/m1/lite-timestamps-through.md) - lite-05/06 tracks stay timed, so their ingest keeps real starts - [Drop set](/quest/m1/drop-set.md) - its dropped and expired runs convert mechanically from sequences to starts diff --git a/rs/moq-cli/src/publish.rs b/rs/moq-cli/src/publish.rs index b2040055271..df34d3c0642 100644 --- a/rs/moq-cli/src/publish.rs +++ b/rs/moq-cli/src/publish.rs @@ -1769,9 +1769,8 @@ pub(crate) mod tests { // Joining once every group is out, with a budget half the jump, reads every group // after it past the marker, and none of the groups the jump left more than // the budget behind. An IETF subscription starts at the newest group, so only lite - // has the groups behind it to read, and a lite-06 subscriber drops the timestamps - // that carry the jump. - if version == "moq-lite-07-wip" { + // has the groups behind it to read. + if version.starts_with("moq-lite") { let late = read(start(moq_net::track::Subscription::default().with_max_delay(Duration::from_secs(10))).await) .await diff --git a/rs/moq-net/MIGRATE.md b/rs/moq-net/MIGRATE.md index eede46d4122..b1d9c4efbc3 100644 --- a/rs/moq-net/MIGRATE.md +++ b/rs/moq-net/MIGRATE.md @@ -368,8 +368,8 @@ let rate = bandwidth.as_ref().and_then(|b| b.peek()); A local write whose first frame starts before a lower-sequence group's, or after a higher one's, fails with `TimestampMismatch`; `@moq/net` throws it the same way. A group received over moq-lite-07 is checked the same way. -A track received over moq-lite-05/06 arrives untimed, since its timestamps may be send times: read times from the payload, such as the hang container's, instead of `frame::Frame::timestamp`. -One received over moq-transport keeps its timestamps unchecked, and a relay serves it to moq-lite untimed. +A track received over moq-lite-05/06 keeps its timestamps unchecked, since they may be an untimed origin's send times, and so does one received over moq-transport. +A relay serves both on over every version, where a moq-lite-07 receiver checks them and may refuse a group that rewinds. A publisher whose timeline restarts, such as an HLS `EXT-X-DISCONTINUITY` that resets decode times, publishes a new broadcast instead. ### Broadcast `finish` is gone diff --git a/rs/moq-net/src/fuzz/ingest.rs b/rs/moq-net/src/fuzz/ingest.rs index f44e7a4aabd..6ce9547b480 100644 --- a/rs/moq-net/src/fuzz/ingest.rs +++ b/rs/moq-net/src/fuzz/ingest.rs @@ -87,8 +87,9 @@ pub fn ingest(data: &[u8]) -> bool { let (session, info, body) = match selector % 2 { 0 => { let version = super::LITE_VERSIONS[(*selector as usize / 2) % super::LITE_VERSIONS.len()]; - // Lite05/06 timestamps may be send times, so their tracks arrive untimed. - let timed = timed && version.has_media_time(); + // The track arrives timed exactly when the wire stamps it: never before Lite05, + // always on Lite05/06 (an untimed one carries its send times), as declared after. + let timed = version.wire_timescale(timed.then_some(Timescale::MILLI)).is_some(); let session = Session::Lite { version, records: settings & 0x10 != 0 && version.has_ietf(), @@ -220,7 +221,7 @@ fn run(session: Session, info: &track::Info, body: &[u8], chunk: usize) -> Optio let consumer = producer.consume(); let mut group = crate::recv::Group::new(producer); let framing = lite::Framing::new(info, version); - let mut ingest = lite::FrameIngest::new(framing, version, records, frame_start); + let mut ingest = lite::FrameIngest::new(framing, records, frame_start); let end = drive(body.len(), |waiter| ingest.poll(&mut reader, &mut group, waiter)); (end.map(|()| "fin".to_string()), (group, consumer)) } diff --git a/rs/moq-net/src/lite/ietf.rs b/rs/moq-net/src/lite/ietf.rs index c2a9c3b2c91..64d97ad40f5 100644 --- a/rs/moq-net/src/lite/ietf.rs +++ b/rs/moq-net/src/lite/ietf.rs @@ -1138,7 +1138,7 @@ mod hop_tests { // The upstream group is still open. let frame = next(&mut received).await.unwrap(); assert_eq!(&frame.payload.contiguous(0)[..], b"a"); - assert_eq!(frame.timestamp, negotiated.then(|| ts(0))); + assert_eq!(frame.timestamp, Some(ts(0))); match negotiated { true => { assert_eq!(fields(&frame), Some((first.id, first.properties.clone())), "{offer:?}"); @@ -1177,12 +1177,7 @@ mod hop_tests { let info = kio::clock::timeout(TIMEOUT, copy.info()).await.unwrap().unwrap(); assert_eq!( (info.max_age, info.timescale, info.priority, info.refresh), - ( - Some(Duration::from_secs(7)), - negotiated.then_some(Timescale::MILLI), - 9, - Some(0) - ), + (Some(Duration::from_secs(7)), Some(Timescale::MILLI), 9, Some(0)), "{offer:?}" ); match negotiated { @@ -1438,9 +1433,9 @@ mod hop_tests { } } - /// Provenance keeps IETF times unchecked after a hop, while a peer declining the - /// extension gets no timeline. SUBSCRIBE, FETCH, and datagrams must agree with TRACK_INFO, - /// while a listing stays untimed. + /// Every hop carries IETF times, and provenance keeps them unchecked after a hop that + /// negotiated the extension. A peer declining it checks them like native moq-lite, as a + /// listing's copy does. SUBSCRIBE, FETCH, and datagrams must agree with TRACK_INFO. #[kio::test] async fn ietf_timestamps_keep_their_provenance_across_a_hop() { for offer in [[true, true], [true, false], [false, true]] { @@ -1457,8 +1452,7 @@ mod hop_tests { head.finish().unwrap(); let (downstream, sessions) = test_hop(&upstream, "room", offer).await; let broadcast = downstream.consume().request("room", None).await.unwrap(); - // A listing cannot carry the provenance, so it lists the track untimed even when - // negotiated: a relay re-listing it would otherwise advertise it as native lite. + // A listing carries no provenance, so it lists the track as native lite. let mut tracks = broadcast.tracks().unwrap(); let listed = loop { match kio::clock::timeout(TIMEOUT, tracks.next()).await.unwrap().unwrap() { @@ -1467,10 +1461,14 @@ mod hop_tests { None => panic!("listing ended before the track"), } }; - assert_eq!((listed.timescale, listed.ietf()), (None, None), "{offer:?}"); + assert_eq!( + (listed.timescale, listed.ietf()), + (Some(Timescale::MILLI), None), + "{offer:?}" + ); let copy = broadcast.track("video").unwrap(); let info = kio::clock::timeout(TIMEOUT, copy.info()).await.unwrap().unwrap(); - assert_eq!(info.timescale, negotiated.then_some(Timescale::MILLI), "{offer:?}"); + assert_eq!(info.timescale, Some(Timescale::MILLI), "{offer:?}"); assert_eq!( info.ietf(), negotiated.then_some(crate::ietf::Version::Draft22), @@ -1482,7 +1480,7 @@ mod hop_tests { .await .unwrap() .unwrap(); - assert_eq!(next(&mut fetched).await.unwrap().timestamp, negotiated.then(|| ts(100))); + assert_eq!(next(&mut fetched).await.unwrap().timestamp, Some(ts(100))); assert!(next(&mut fetched).await.is_none()); let mut subscriber = kio::clock::timeout(TIMEOUT, copy.subscribe(track::Subscription::default().with_start(1))) @@ -1490,23 +1488,26 @@ mod hop_tests { .unwrap() .unwrap(); // A peer may have stamped arrival times. The second group's rewind must not - // acquire native moq-lite checks merely because it crossed a relay hop. + // acquire native moq-lite checks merely because it crossed a negotiated hop; + // without the extension, the copy is checked, so it advances instead. + let start = if negotiated { ts(50) } else { ts(150) }; let mut group = track.create_group(1u64.into()).unwrap(); - group.write_frame(ts(50), b"frame".to_vec()).unwrap(); + group.write_frame(start, b"frame".to_vec()).unwrap(); group.finish().unwrap(); let mut received = recv(&mut subscriber).await; assert_eq!(received.sequence, 1); let frame = next(&mut received).await.unwrap(); - assert_eq!(frame.timestamp, negotiated.then(|| ts(50)), "{offer:?}"); + assert_eq!(frame.timestamp, Some(start), "{offer:?}"); assert_eq!(&frame.payload.contiguous(0)[..], b"frame"); assert!(next(&mut received).await.is_none()); - track.insert_datagram(2, ts(25), b"datagram".to_vec()).unwrap(); + let time = if negotiated { ts(25) } else { ts(200) }; + track.insert_datagram(2, time, b"datagram".to_vec()).unwrap(); let datagram = kio::clock::timeout(TIMEOUT, kio::wait(|waiter| subscriber.poll_recv_wire_datagram(waiter))) .await .unwrap() .unwrap() .unwrap(); - assert_eq!(datagram.timestamp, negotiated.then(|| ts(25)), "{offer:?}"); + assert_eq!(datagram.timestamp, Some(time), "{offer:?}"); assert_eq!(&datagram.payload[..], b"datagram"); drop(sessions); } diff --git a/rs/moq-net/src/lite/publisher.rs b/rs/moq-net/src/lite/publisher.rs index 7be2006dd75..59c4e16b368 100644 --- a/rs/moq-net/src/lite/publisher.rs +++ b/rs/moq-net/src/lite/publisher.rs @@ -1904,7 +1904,7 @@ impl Request for TrackInfoServe { crate::ietf::flate::egress(&info, shared.version.has_compression())?; // TRACK_INFO only flows on Lite05+ (the encode errors otherwise). - writer.buffer(&lite::TrackInfo::serve(&info, shared.version, ietf))?; + writer.buffer(&lite::TrackInfo::serve(&info, shared.version))?; if ietf { writer.buffer(&lite::ietf::TrackProperties(info.properties.clone()))?; } @@ -1964,9 +1964,7 @@ impl Request for TracksServe { let event = match ready!(self.tracks.poll_next(waiter))? { Some(crate::broadcast::TracksEvent::Added { name, info }) => lite::TracksEvent::Added { name: name.to_string().into(), - // Untimed even to an extension peer: TRACKS_ADD has no TRACK_PROPERTIES to - // carry the unchecked provenance, so a relay would re-list it as native lite. - info: lite::TrackInfo::serve(&info, shared.version, false), + info: lite::TrackInfo::serve(&info, shared.version), }, Some(crate::broadcast::TracksEvent::Removed { name }) => lite::TracksEvent::Removed { name: name.to_string().into(), @@ -2076,7 +2074,6 @@ impl Request for SubscribeServe { loop { match self { Self::Query { querying, .. } => { - let ietf = ready!(shared.peer_setup.poll_ietf(waiter)); let info = ready!(querying.poll_ok(waiter))?; let Self::Query { msg, @@ -2094,7 +2091,7 @@ impl Request for SubscribeServe { else { unreachable!() }; - let bounds = Bounds::new(&msg.ranges, info.lite_timescale(ietf))?; + let bounds = Bounds::new(&msg.ranges, info.timescale)?; *self = Self::subscribe(shared, *msg, bounds, &track, latest, update); } Self::Confirm { subscribing, .. } => { @@ -2121,7 +2118,7 @@ impl Request for SubscribeServe { // `None` omits the per-frame timestamp prefix: an untimed track on // Lite07+, or any track before Lite05 (whose frames still carry // timestamps in the model, just not on this wire). - let stamp = Stamp::new(shared.version, track.info().lite_timescale(ietf)); + let stamp = Stamp::new(shared.version, track.info().timescale); // Lite05+ accepts implicitly: no SUBSCRIBE_OK, the immutable properties // live in TRACK_INFO, and the resolved range arrives as @@ -2335,7 +2332,7 @@ impl Request for FetchServe { let mut group = group.with_end(msg.end_frame.and_then(|end| end.checked_add(1))); // Matches the timescale TRACK_INFO declared for the track. - let stamp = Stamp::new(shared.version, group.lite_timescale(ietf)); + let stamp = Stamp::new(shared.version, group.timescale()); // And its compression, which a lite-07 peer receives as stored. if shared.version.has_compression() { group = group.compressed(); @@ -3448,13 +3445,12 @@ struct Stamp { /// The wire timescale TRACK_INFO declared (see [`Version::wire_timescale`]). timescale: crate::Timescale, /// Whether each frame's own timestamp goes out. Lite05/06 cannot encode an untimed - /// track, so one served untimed carries its send time instead, unchecked ones included. + /// track, so an untimed one carries its send time instead. timed: bool, } impl Stamp { - /// How `version` stamps a track served at `timescale` (see - /// [`track::Info::lite_timescale`]), or `None` when it carries no timestamps. + /// How `version` stamps a track with `timescale`, or `None` when it carries no timestamps. fn new(version: Version, timescale: Option) -> Option { Some(Self { timescale: version.wire_timescale(timescale)?, @@ -5177,29 +5173,25 @@ mod serve_group_test { } } - /// A track served untimed to lite-05/06 carries its send time, never its frames' own - /// timestamps: those of an unchecked track may be moq-transport arrival times. + /// Lite-05/06 cannot encode an untimed track, so one carries its send time there and + /// nothing on lite-07, while a timed track carries its own timestamps on every version. #[kio::test] - async fn an_unchecked_track_goes_out_with_send_times() { + async fn only_an_untimed_track_goes_out_with_send_times() { let runtime = crate::time::sim(); let stamped = Timestamp::from_millis(36_000_000).unwrap(); let sent = Timestamp::from(runtime.now()).value(); - let checked = track::Info::timed(); - let unchecked = track::Info { - properties: Some(std::sync::Arc::new(crate::model::object::TrackProperties { - version: crate::ietf::Version::Draft22, - values: Vec::new(), - })), - ..track::Info::timed() - }; - let stamp = Stamp::new(Version::Lite06, checked.lite_timescale(false)).unwrap(); - assert_eq!(stamp.wire(Some(stamped), &runtime).unwrap(), stamped.value()); - let stamp = Stamp::new(Version::Lite06, unchecked.lite_timescale(false)).unwrap(); - assert_eq!(stamp.wire(Some(stamped), &runtime).unwrap(), sent); - assert!(Stamp::new(Version::Lite07, unchecked.lite_timescale(false)).is_none()); - let stamp = Stamp::new(Version::Lite07, unchecked.lite_timescale(true)).unwrap(); - assert_eq!(stamp.wire(Some(stamped), &runtime).unwrap(), stamped.value()); + for version in [Version::Lite06, Version::Lite07] { + let stamp = Stamp::new(version, Some(crate::Timescale::MILLI)).unwrap(); + assert_eq!( + stamp.wire(Some(stamped), &runtime).unwrap(), + stamped.value(), + "{version}" + ); + } + let stamp = Stamp::new(Version::Lite06, None).unwrap(); + assert_eq!(stamp.wire(None, &runtime).unwrap(), sent); + assert!(Stamp::new(Version::Lite07, None).is_none()); } #[kio::test] diff --git a/rs/moq-net/src/lite/subscriber.rs b/rs/moq-net/src/lite/subscriber.rs index 6b48f7767e5..c2965b71114 100644 --- a/rs/moq-net/src/lite/subscriber.rs +++ b/rs/moq-net/src/lite/subscriber.rs @@ -594,9 +594,7 @@ impl Subscriber { .zip(entry.framing.timescale) .map(|(value, scale)| Timestamp::new(value, scale)) .transpose() - .map_err(|_| Error::BoundsExceeded(crate::coding::BoundsExceeded))? - // Dropped where it may be a send time, like a frame's. - .filter(|_| self.version.has_media_time()); + .map_err(|_| Error::BoundsExceeded(crate::coding::BoundsExceeded))?; // A datagram is never owed a stream, so it never holds the subscription's end open. if let Ok(mut tail) = entry.tail.write() { @@ -1097,7 +1095,7 @@ impl GroupRecv { // Guarded: dropping this machine mid-group is a cancellation, not a clean end. let group: crate::recv::Group = crate::recv::Group::new(group); let track: track::Producer = track; - let ingest: FrameIngest = FrameIngest::new(framing, self.subscriber.version, ietf, hdr.frame_start); + let ingest: FrameIngest = FrameIngest::new(framing, ietf, hdr.frame_start); let reading: Reading = reading; // Filling the group, bailing if the track or group dies first. The track or group @@ -1149,9 +1147,6 @@ pub(crate) struct FrameIngest { /// `Some` once the Group State of the `ietf` extension arrived: every frame follows an /// object record. records: Option, - /// Whether the session's timestamps are media time. Lite-05/06 ones may be send times, - /// so they are read to stay in step and dropped. - media_time: bool, } /// A group's object records on ingest, under the `ietf` extension. @@ -1197,7 +1192,7 @@ enum IngestPhase { impl FrameIngest { /// `ietf` when the session negotiated the extension, so the stream starts with a Group State. - pub(crate) fn new(framing: lite::Framing, version: Version, ietf: bool, frame_start: u64) -> Self { + pub(crate) fn new(framing: lite::Framing, ietf: bool, frame_start: u64) -> Self { Self { framing, prev_ts: 0, @@ -1207,7 +1202,6 @@ impl FrameIngest { }, frame_start, records: None, - media_time: version.has_media_time(), } } @@ -1329,9 +1323,8 @@ impl FrameIngest { self.prev_ts = next; let timestamp = Timestamp::new(next, scale) .map_err(|_| Error::BoundsExceeded(crate::coding::BoundsExceeded))?; - // Read to stay in step, but dropped where it may be a send time. self.phase = IngestPhase::Inflated { - timestamp: self.media_time.then_some(timestamp), + timestamp: Some(timestamp), }; } IngestPhase::Inflated { timestamp } => { @@ -3005,11 +2998,10 @@ mod tests { going_away: Default::default(), auth: crate::auth::Link::new(false), }); - // The track is untimed where the subscriber drops received timestamps. let track = track::Producer::new( std::sync::Arc::new(crate::broadcast::Info::default()), "video", - crate::track::Info::default().with_timescale(version.has_media_time().then_some(Timescale::MILLI)), + crate::track::Info::timed(), ); let mut consumer = track.consume().subscribe(None).await.unwrap(); subscriber.subscribes.lock().insert( @@ -3352,16 +3344,15 @@ mod tests { auth: crate::auth::Link::new(false), }); - // The copy is untimed where the wire's timestamps may be send times. - let timed = version.has_media_time(); - let info = match timed { - true => crate::track::Info::timed(), - false => crate::track::Info::default(), + // Unchecked where the wire's timestamps may be send times, as the copy is accepted. + let info = crate::track::Info { + unchecked: !version.has_media_time(), + ..crate::track::Info::timed() }; let track = track::Producer::new(std::sync::Arc::new(crate::broadcast::Info::default()), "video", info); let mut before = track.create_group(4u64.into()).unwrap(); before - .write_frame(timed.then(|| Timestamp::from_millis(1000).unwrap()), &b"before"[..]) + .write_frame(Timestamp::from_millis(1000).unwrap(), &b"before"[..]) .unwrap(); before.finish().unwrap(); subscriber.subscribes.lock().insert( @@ -3388,10 +3379,10 @@ mod tests { (res, track) } - /// Lite-05/06 timestamps may be send times, so a received frame is read past its - /// timestamp and lands untimed: nothing rewinds, and the track stays open. + /// Lite-05/06 timestamps may be send times, so a received group that rewinds keeps its + /// timestamp unchecked, and the track stays open. #[kio::test] - async fn lite06_timestamps_are_dropped() { + async fn a_lite06_rewind_passes_through() { for version in [Version::Lite05, Version::Lite06] { // Frame 0 at 500ms, before group 4's 1000ms. let (res, track) = receive_group(version, &[500]).await; @@ -3402,7 +3393,7 @@ mod tests { assert_eq!(group.sequence, 5, "{version}"); let frame = group.read_frame().now_or_never().unwrap().unwrap().unwrap(); assert_eq!(frame.payload.as_ref(), b"frame", "{version}"); - assert_eq!(frame.timestamp, None, "{version}"); + assert_eq!(frame.timestamp, Timestamp::from_millis(500).ok(), "{version}"); } } @@ -3417,7 +3408,6 @@ mod tests { /// A received timestamp past 2^61 - 1 is refused at every version, so a relay never /// holds one that lite-05/06 could not carry as a group's first frame. lite-06 reaches /// it with a second delta, lite-07 with a first frame its 64-bit varint can encode. - /// lite-05/06 drop the timestamp, but a stream carrying one past the bound is still malformed. #[kio::test] async fn a_received_timestamp_past_2_61_is_refused() { let max: i64 = (1 << 61) - 1; @@ -3426,8 +3416,7 @@ mod tests { res.unwrap_or_else(|err| panic!("{version}: {err:?}")); let mut group = track.consume().peek_group(5).unwrap(); let frame = group.read_frame().now_or_never().unwrap().unwrap().unwrap(); - let kept = version.has_media_time().then_some(max as u64); - assert_eq!(frame.timestamp.map(Timestamp::value), kept, "{version}"); + assert_eq!(frame.timestamp.map(Timestamp::value), Some(max as u64), "{version}"); let (res, _) = receive_group(version, &[max, 1]).await; assert!(matches!(res, Err(Error::BoundsExceeded(_))), "{version}: {res:?}"); @@ -4005,15 +3994,8 @@ mod tests { let broadcast = crate::broadcast::Info::new().produce(); // The broadcast keeps only a weak handle, so the map below owns the only strong // `track::Producer`: dropping it is what ends the track. - // Earlier versions drop their wire timestamps. let mut producer = broadcast - .create_track( - "datagrams", - match version.has_media_time() { - true => crate::track::Info::default().with_timescale(Timescale::MILLI), - false => crate::track::Info::default(), - }, - ) + .create_track("datagrams", crate::track::Info::timed()) .unwrap(); producer.set_idle(); let mut received = producer.subscribe(None); @@ -7602,11 +7584,9 @@ impl kio::Task for TrackServeRun { let request = request.expect("request pending"); match res { Ok((mut info, held)) => { - // Lite05/06 timestamps may be send times, so the copy is untimed - // and the ingest reads past them (see `lite::Framing::new`). - if !self.serve.subscriber.version.has_media_time() { - info.timescale = None; - } + // Lite05/06 timestamps may be send times, so the copy's groups + // skip the rewind check. + info.unchecked = !self.serve.subscriber.version.has_media_time(); let mut serve_loop = ServeLoop::adopt(&self.serve, request, info.clone(), dynamic, fetches); serve_loop.held = Some(held); // Only once accepted: a fetch answered now caches into the accepted copy. @@ -8927,7 +8907,6 @@ impl kio::Task for FetchServeRun { readers, ingest: Box::new(FrameIngest::new( lite::Framing::new(&info, self.serve.subscriber.version), - self.serve.subscriber.version, ietf, frame_start, )), diff --git a/rs/moq-net/src/lite/track.rs b/rs/moq-net/src/lite/track.rs index 43ecbd2951b..cc493363af4 100644 --- a/rs/moq-net/src/lite/track.rs +++ b/rs/moq-net/src/lite/track.rs @@ -90,13 +90,12 @@ pub struct TrackInfo { impl TrackInfo { /// The TRACK_INFO a publisher serves for `info` on `version`. - pub(crate) fn serve(info: &crate::track::Info, version: Version, ietf: bool) -> Self { + pub(crate) fn serve(info: &crate::track::Info, version: Version) -> Self { Self { refresh: info.refresh, priority: info.priority, max_age: info.max_age, - // Only an extension peer knows the original publisher may stamp arrival times. - timescale: info.lite_timescale(ietf), + timescale: info.timescale, // An older peer is served the frames inflated. compress: info.compress && version.has_compression(), } @@ -133,8 +132,8 @@ pub(crate) struct Framing { } impl Framing { - /// How `version` frames a track accepted as `info`. Lite05/06 stamp every frame, so a - /// track received untimed there still has timestamps to read past. + /// How `version` frames a track accepted as `info`. Lite05/06 stamp every frame, an + /// untimed track's with its send time in milliseconds. pub fn new(info: &crate::track::Info, version: Version) -> Self { Self { timescale: version.wire_timescale(info.timescale), @@ -330,7 +329,7 @@ mod test { #[test] fn track_info_defaults_match_cross_language_wire_bytes() { // The default track is untimed, which lite-05 declares as milliseconds. - let info = TrackInfo::serve(&crate::track::Info::default(), Version::Lite05, false); + let info = TrackInfo::serve(&crate::track::Info::default(), Version::Lite05); let mut buf = Vec::new(); info.encode(&mut Encoder::new(&mut buf, Version::Lite05.into()), Version::Lite05) .unwrap(); diff --git a/rs/moq-net/src/lite/version.rs b/rs/moq-net/src/lite/version.rs index 32d7256757f..54c0629746a 100644 --- a/rs/moq-net/src/lite/version.rs +++ b/rs/moq-net/src/lite/version.rs @@ -343,7 +343,7 @@ impl Version { /// Whether received frame timestamps are the publisher's media time. Lite05/06 stamp an /// untimed track's payloads with their send time, which a receiver cannot tell from it, - /// so it drops every timestamp received on them and the track arrives untimed. + /// so it passes them through as the track's time without checking them for a rewind. pub(crate) fn has_media_time(self) -> bool { !matches!( self, diff --git a/rs/moq-net/src/model/group.rs b/rs/moq-net/src/model/group.rs index 7e34950a582..d66c6450ca9 100644 --- a/rs/moq-net/src/model/group.rs +++ b/rs/moq-net/src/model/group.rs @@ -3592,13 +3592,6 @@ impl Consumer { self.arrived } - /// The timescale a moq-lite hop serves the parent track at (see - /// [`track::Info::lite_timescale`]). - #[cfg_attr(not(feature = "async"), allow(dead_code, reason = "async-only"))] - pub(crate) fn lite_timescale(&self, ietf: bool) -> Option { - self.track.lite_timescale(ietf) - } - /// The draft the parent track's moq-transport publisher spoke, which serialized every /// object's properties, or `None` for a moq-lite one. #[cfg(feature = "async")] diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 601bed0758b..f00c325a3f5 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -102,7 +102,7 @@ pub(crate) const IDLE_LINGER: Duration = Duration::from_secs(30); // Deliberately not `Copy`, even though it's now a plain value: adding `Copy` turns // every existing `info.clone()` in a consumer's code into a `clippy::clone_on_copy` // error under `-D warnings`. -#[derive(Clone, Debug, PartialEq, Eq)] +#[derive(Clone, Debug)] #[non_exhaustive] pub struct Info { /// Units per second for per-frame timestamps on this track, or `None` for an untimed @@ -175,8 +175,37 @@ pub struct Info { /// Shared, since every group holds a copy of the info. #[cfg_attr(not(feature = "async"), allow(dead_code, reason = "async-only"))] pub(crate) properties: Option>, + /// Whether this copy's groups skip the rewind check: set on a lite-05/06 copy, whose + /// untimed origin stamps send times, so an older group fetched later looks like a rewind. + /// A track with moq-transport provenance ([`Self::ietf`]) skips it too. + #[cfg_attr(not(feature = "async"), allow(dead_code, reason = "async-only"))] + pub(crate) unchecked: bool, +} + +// `unchecked` says how this copy arrived, not what it holds, so copies of one track over +// different versions still match. +impl PartialEq for Info { + fn eq(&self, other: &Self) -> bool { + let Self { + timescale, + max_age, + priority, + compress, + refresh, + properties, + unchecked: _, + } = self; + *timescale == other.timescale + && *max_age == other.max_age + && *priority == other.priority + && *compress == other.compress + && *refresh == other.refresh + && *properties == other.properties + } } +impl Eq for Info {} + impl Default for Info { fn default() -> Self { Self { @@ -186,6 +215,7 @@ impl Default for Info { compress: false, refresh: None, properties: None, + unchecked: false, } } } @@ -225,13 +255,6 @@ impl Info { self } - /// The timescale served on a moq-lite hop, preserving moq-transport timestamps only - /// when the hop negotiated the `ietf` extension. - #[cfg_attr(not(feature = "async"), allow(dead_code, reason = "async-only"))] - pub(crate) fn lite_timescale(&self, ietf: bool) -> Option { - self.timescale.filter(|_| self.ietf().is_none() || ietf) - } - /// The original moq-transport publisher's draft, whose timestamps may rewind, or /// `None` for a native moq-lite track. pub(crate) fn ietf(&self) -> Option { @@ -884,9 +907,13 @@ impl TrackState { /// so the watermark caps it instead. /// /// Usually one group lookup each way; only unstarted or partial neighbors are skipped. - /// `None` on a track with moq-transport provenance, whose timestamps may rewind. + /// `None` on an [unchecked](Info::unchecked) track, whose timestamps may rewind. pub(super) fn neighbors(&self, sequence: u64) -> Option { - if self.info.as_ref().is_some_and(|info| info.ietf().is_some()) { + if self + .info + .as_ref() + .is_some_and(|info| info.unchecked || info.ietf().is_some()) + { return None; } let bounds = self.bounds.lock().unwrap_or_else(PoisonError::into_inner); diff --git a/rs/moq-net/tests/integration/datagram.rs b/rs/moq-net/tests/integration/datagram.rs index 200cf92a824..0e8087bd623 100644 --- a/rs/moq-net/tests/integration/datagram.rs +++ b/rs/moq-net/tests/integration/datagram.rs @@ -48,7 +48,6 @@ async fn connect_datagram_track() -> Fixture { let producer = broadcast.create_track("datagrams", timed()).unwrap(); broadcast.announce(Default::default()).unwrap(); - // Lite-07, since a lite-05/06 subscriber drops every timestamp it receives. let mut options = MockConnectOptions::new("moq-lite-07-wip".parse::().unwrap()); options.server_publish = Some(publisher.consume()); options.client_subscribe = Some(consumer_origin.clone()); diff --git a/rs/moq-net/tests/integration/range_anchors.rs b/rs/moq-net/tests/integration/range_anchors.rs index 796cb5ddee7..6d0f4b1c707 100644 --- a/rs/moq-net/tests/integration/range_anchors.rs +++ b/rs/moq-net/tests/integration/range_anchors.rs @@ -120,14 +120,11 @@ fn anchored() -> [(Range, Vec); 6] { #[kio::test] async fn each_wire_refuses_only_the_anchors_it_drops() { for version in std::iter::once(None).chain(VERSIONS.iter().copied().map(Some)) { - // Lite-07 reports the largest and media time and asks for everything the budget - // allows, so its copy resolves every anchor where it is held. + // Lite-07 reports the largest and asks for everything the budget allows, so its copy + // resolves every anchor where it is held. let carried = matches!(version, None | Some("moq-lite-07-wip")); // A wire without timestamps refuses a time anchor for that first. - let timed = !matches!( - version, - Some("moq-lite-03" | "moq-lite-04" | "moq-lite-05" | "moq-lite-06") - ); + let timed = !matches!(version, Some("moq-lite-03" | "moq-lite-04")); for (range, held) in anchored() { let mut result = read(version, range).await; let time = matches!(range.start, Anchor::Time(_)) || matches!(range.end, Some(Anchor::Time(_))); diff --git a/rs/moq-net/tests/integration/received_rewind.rs b/rs/moq-net/tests/integration/received_rewind.rs index 7e7618817d4..e9e43ad837b 100644 --- a/rs/moq-net/tests/integration/received_rewind.rs +++ b/rs/moq-net/tests/integration/received_rewind.rs @@ -1,11 +1,12 @@ #![cfg(feature = "async")] -//! A relay drops the timestamps it receives over lite-05/06. +//! A relay passes on the timestamps it receives over lite-05/06, unchecked. //! //! Lite-06 stamps an untimed track's frames with their send time and declares //! milliseconds, so a group sent after a higher one carries the later stamp. The relay -//! cannot tell that from media time, so its copy is untimed and every hop past it sees an -//! untimed track that nothing can refuse. +//! cannot tell that from media time, so its copy skips the rewind check, as a +//! moq-transport one does, and serves the timestamps on over every version. A lite-07 +//! receiver checks them, so it refuses that out-of-order group. use crate::support; @@ -17,6 +18,13 @@ use support::harness::{MockConnectOptions, connect_mock}; const TIMEOUT: Duration = Duration::from_secs(10); +/// A group's sequence and its first frame's timestamp, or the error reading it. +type Read = (u64, Result, moq_net::Error>); + +fn millis(ms: u64) -> moq_net::Timestamp { + moq_net::Timestamp::from_millis(ms).unwrap() +} + /// Long enough that both groups are fresh. const BUDGET: Duration = Duration::from_secs(60); @@ -29,22 +37,20 @@ fn produce_origin(hop: u64) -> moq_net::origin::Producer { fn write_group(track: &moq_net::track::Producer, sequence: u64) { // A timed track's group starts at its sequence in seconds, so none rewinds. - let timestamp = track - .info() - .timescale - .map(|_| moq_net::Timestamp::from_millis(sequence * 1000).unwrap()); + let timestamp = track.info().timescale.map(|_| millis(sequence * 1000)); let mut group = track.create_group(moq_net::group::Info { sequence }).unwrap(); group.write_frame(timestamp, &b"x"[..]).unwrap(); group.finish().unwrap(); } /// Groups 1 then 0 of a track at `timescale`, into a relay over `upstream` and out over -/// `downstream`. Returns the scale the subscriber learned and the groups it read. +/// `downstream`. Returns the scale the subscriber learned and, per group read, its sequence +/// and its first frame's timestamp or the error reading it. async fn relay( upstream: &str, downstream: &str, timescale: Option, -) -> (Option, Vec) { +) -> (Option, Vec) { let publisher = produce_origin(1); let broadcast = publisher.create_broadcast("bcast").unwrap(); let track = broadcast @@ -87,13 +93,11 @@ async fn relay( let Some(mut group) = next.expect("recv_group") else { break; }; - while kio::clock::timeout(TIMEOUT, group.read_frame()) + let read = kio::clock::timeout(TIMEOUT, group.read_frame()) .await .expect("read_frame timed out") - .expect("read_frame") - .is_some() - {} - got.push(group.sequence); + .map(|frame| frame.expect("a frame").timestamp); + got.push((group.sequence, read)); } (timescale, got) }); @@ -117,32 +121,47 @@ async fn relay( got } +/// A relay passes on lite-06 send times unchecked, so a lite-06 or moq-transport reader +/// reads both groups timed and the track stays open. A lite-07 reader checks them and +/// refuses the group that rewinds, the cost of an untimed origin on lite-06. #[kio::test] async fn a_relay_forwards_descending_send_times_without_closing_the_track() { let mut failures = Vec::new(); for downstream in ["moq-lite-06", "moq-lite-07-wip", "moq-transport-22"] { - let got = relay("moq-lite-06", downstream, None).await; - if got.0.is_some() || got.1 != [1, 0] { - failures.push(format!("{downstream}: {got:?}")); + let (timescale, got) = relay("moq-lite-06", downstream, None).await; + let sequences: Vec<_> = got.iter().map(|(sequence, _)| *sequence).collect(); + let refused = downstream == "moq-lite-07-wip"; + let ok = timescale == Some(moq_net::Timescale::MILLI) + && sequences == [1, 0] + && matches!(got[0].1, Ok(Some(_))) + && match &got[1].1 { + Err(moq_net::Error::TimestampMismatch) => refused, + Ok(Some(_)) => !refused, + _ => false, + }; + if !ok { + failures.push(format!("{downstream}: {timescale:?} {got:?}")); } } assert!(failures.is_empty(), "{failures:#?}"); } -/// A moq-transport peer may stamp arrival times, so a relay keeps a timed track's timestamps -/// unchecked and passes them on only over moq-transport: a moq-lite hop, which checks every -/// timestamp it receives, is served the track untimed. +/// A timed track's timestamps cross a lite-06 or moq-transport hop between two origins with +/// their values, and the relay serves them on over every version. #[kio::test] -async fn moq_transport_timestamps_stay_on_moq_transport() { +async fn a_timed_track_keeps_its_timestamps_through_a_relay() { let mut failures = Vec::new(); - for (downstream, expected) in [ - ("moq-lite-06", None), - ("moq-lite-07-wip", None), - ("moq-transport-22", Some(moq_net::Timescale::MILLI)), - ] { - let got = relay("moq-transport-22", downstream, Some(moq_net::Timescale::MILLI)).await; - if got.0 != expected || got.1 != [1, 0] { - failures.push(format!("{downstream}: {got:?}")); + for upstream in ["moq-lite-06", "moq-transport-22"] { + for downstream in ["moq-lite-06", "moq-lite-07-wip", "moq-transport-22"] { + let (timescale, got) = relay(upstream, downstream, Some(moq_net::Timescale::MILLI)).await; + let ok = timescale == Some(moq_net::Timescale::MILLI) + && matches!( + got.as_slice(), + [(1, Ok(Some(one))), (0, Ok(Some(zero)))] if *one == millis(1000) && *zero == millis(0) + ); + if !ok { + failures.push(format!("{upstream} -> {downstream}: {timescale:?} {got:?}")); + } } } assert!(failures.is_empty(), "{failures:#?}"); @@ -150,8 +169,9 @@ async fn moq_transport_timestamps_stay_on_moq_transport() { /// Fetch groups 1 then 0 of a finished untimed track over `version`, with no live /// subscription. Each answer carries its send time, so group 0 lands after group 1; the -/// copy the fetches fill drops them, so neither is refused. -async fn fetch_only(version: &str) -> Vec> { +/// copy the fetches fill is unchecked, so neither is refused. Returns each first frame's +/// timestamp. +async fn fetch_only(version: &str) -> Vec, moq_net::Error>> { let publisher = produce_origin(1); let broadcast = publisher.create_broadcast("bcast").unwrap(); let track = broadcast.create_track("data", Info::default()).unwrap(); @@ -176,8 +196,9 @@ async fn fetch_only(version: &str) -> Vec> { for sequence in [1, 0] { let read = async { let mut group = remote.track("data").unwrap().fetch_group(sequence, None).await?; + let timestamp = group.read_frame().await?.expect("a frame").timestamp; while group.read_frame().await?.is_some() {} - Ok(group.sequence) + Ok(timestamp) }; got.push(kio::clock::timeout(TIMEOUT, read).await.expect("fetch timed out")); // Answered later, so group 0's send time lands after group 1's. @@ -192,7 +213,8 @@ async fn fetches_alone_accept_descending_send_times() { let mut failures = Vec::new(); for version in ["moq-lite-05", "moq-lite-06"] { let got = fetch_only(version).await; - if got.iter().any(Result::is_err) { + // Both arrive timed with their send times, the later fetch's last. + if !matches!(got.as_slice(), [Ok(Some(one)), Ok(Some(zero))] if zero > one) { failures.push(format!("{version}: {got:?}")); } } diff --git a/rs/moq-net/tests/integration/untimed.rs b/rs/moq-net/tests/integration/untimed.rs index 2cf8e5d08a5..8beca065822 100644 --- a/rs/moq-net/tests/integration/untimed.rs +++ b/rs/moq-net/tests/integration/untimed.rs @@ -4,8 +4,8 @@ //! that can say so, and a timed track keeps its timestamps. //! //! No receiver fills in arrival time. Lite-05 and lite-06 cannot encode absence, so their -//! encoder writes its send time instead, and their receivers drop every timestamp, which -//! leaves a timed track untimed too. +//! encoder writes its send time instead, which their receivers pass through as the track's +//! time. use crate::support; @@ -27,14 +27,17 @@ const TIMEOUT: Duration = Duration::from_secs(10); enum Arrives { /// Both keep what the publisher wrote. Faithful, - /// The wire carries no trustworthy timescale, so even the timed track arrives untimed. + /// The timed track keeps its timestamps, but the untimed one arrives timed in + /// milliseconds with its send times. + SendTimes, + /// The wire carries no timescale, so even the timed track arrives untimed. Untimed, } fn expected(version: &str) -> Arrives { match version { - // Lite-05 and lite-06 stamp untimed tracks with send times, so their receivers drop them. - "moq-lite-03" | "moq-lite-04" | "moq-lite-05" | "moq-lite-06" => Arrives::Untimed, + "moq-lite-03" | "moq-lite-04" => Arrives::Untimed, + "moq-lite-05" | "moq-lite-06" => Arrives::SendTimes, _ => Arrives::Faithful, } } @@ -130,6 +133,11 @@ async fn frames_arrive_as_published() { assert_eq!(untimed, None, "{version}: absence survives"); assert_eq!(untimed_info.timescale, None, "{version}: no timeline is declared"); } + Arrives::SendTimes => { + assert_eq!(timed, Some(millis(7)), "{version}: the timestamp survives"); + assert!(untimed.is_some(), "{version}: the send time arrives"); + assert_eq!(untimed_info.timescale, Some(Timescale::MILLI), "{version}"); + } Arrives::Untimed => { assert_eq!(timed, None, "{version}: no units, so no timestamp"); assert_eq!(timed_info.timescale, None, "{version}: no timeline is declared"); @@ -197,8 +205,8 @@ async fn an_explicit_start_holds_on_an_untimed_track() { } } -/// An untimed track's datagram arrives untimed, though lite-05 and lite-06 carry a send -/// time that the receiver drops. +/// An untimed track's datagram arrives untimed, except that lite-05 and lite-06 carry its +/// send time. #[kio::test] async fn datagrams_arrive_as_published() { for version in ["moq-lite-05", "moq-lite-07-wip", "moq-transport-22"] { @@ -219,11 +227,13 @@ async fn datagrams_arrive_as_published() { .expect("datagram timeout") .unwrap() .expect("a datagram"); - assert_eq!(datagram.timestamp, None, "{version}"); + let sent = expected(version) == Arrives::SendTimes; + assert_eq!(datagram.timestamp.is_some(), sent, "{version}"); } } -/// A standalone FETCH of an untimed track answers with untimed objects. +/// A standalone FETCH of an untimed track answers with untimed objects, except that lite-05 +/// and lite-06 carry each one's send time. #[kio::test] async fn a_fetched_untimed_frame_arrives_untimed() { for version in ["moq-lite-06", "moq-lite-07-wip", "moq-transport-18"] { @@ -251,6 +261,7 @@ async fn a_fetched_untimed_frame_arrives_untimed() { let mut fetched = waiting.await.unwrap(); let frame = fetched.read_frame().await.unwrap().unwrap(); - assert_eq!(frame.timestamp, None, "{version}"); + let sent = expected(version) == Arrives::SendTimes; + assert_eq!(frame.timestamp.is_some(), sent, "{version}"); } } diff --git a/rs/moq-net/tests/integration/wide_sequence.rs b/rs/moq-net/tests/integration/wide_sequence.rs index 730498cf4ec..3dfb14cd918 100644 --- a/rs/moq-net/tests/integration/wide_sequence.rs +++ b/rs/moq-net/tests/integration/wide_sequence.rs @@ -57,8 +57,8 @@ async fn largest_quic_sequence_reaches_the_subscriber() { assert_eq!(got.sequence, sequence, "{alpn} relayed={relayed}"); let frame = got.read_frame().await.unwrap().unwrap(); assert_eq!(frame.payload, b"ladder".as_ref()); - // Lite-02 carries no timestamps, and a lite-06 subscriber drops them. - let timed = alpn == "moq-transport-18"; + // Lite-02 carries no timestamps. + let timed = alpn != "moq-lite-02"; assert_eq!(frame.timestamp, timed.then_some(timestamp), "{alpn} relayed={relayed}"); } } diff --git a/rs/moq-net/tests/integration/widened_max_age.rs b/rs/moq-net/tests/integration/widened_max_age.rs index 64667869119..477b4953646 100644 --- a/rs/moq-net/tests/integration/widened_max_age.rs +++ b/rs/moq-net/tests/integration/widened_max_age.rs @@ -4,8 +4,8 @@ //! relay would: the widened SUBSCRIBE_UPDATE has the publisher re-offer the cached groups //! it skipped as stale, below where the copy started. //! -//! Lite-06 relies on that re-offer alone: its copy is untimed and its FETCH carries no max -//! age, so a reader asks no fills below the start the publisher resolved from the budget. +//! Lite-06 relies on that re-offer alone: its FETCH carries no max age, so a reader asks no +//! fills below the start the publisher resolved from the budget. use std::collections::BTreeSet; use std::time::Duration; @@ -186,14 +186,10 @@ async fn a_warm_relay_serves_a_wider_newcomer_what_a_cold_one_would() { let mut wide = kio::clock::timeout(TIMEOUT, warm.subscribe(WIDE)).await.unwrap(); let got = kio::clock::timeout(TIMEOUT, received(&mut wide)).await.unwrap(); assert_eq!(got, want, "{version}: warm"); - // A lite-06 copy has no media time to find the re-offer stale by, and each group in - // it arrives just now, so the narrow reader takes it too, once per widening. + // The copy keeps the publisher's media time, lite-06's included, so the narrow + // reader finds the re-offered groups stale and takes none of them. let late = kio::clock::timeout(TIMEOUT, received(&mut narrow)).await.unwrap(); - let reoffered = match version { - "moq-lite-06" => (2..6).collect(), - _ => BTreeSet::new(), - }; - assert_eq!(late, reoffered, "{version}: what the narrow reader takes late"); + assert_eq!(late, BTreeSet::new(), "{version}: what the narrow reader takes late"); } } diff --git a/rs/moq-relay/tests/integration/connection.rs b/rs/moq-relay/tests/integration/connection.rs index ab611c887d8..3fe6240a06a 100644 --- a/rs/moq-relay/tests/integration/connection.rs +++ b/rs/moq-relay/tests/integration/connection.rs @@ -62,8 +62,8 @@ async fn largest_quic_sequence_reaches_the_subscriber() { assert_eq!(got.sequence, sequence, "{alpn}"); let frame = got.read_frame().await.unwrap().unwrap(); assert_eq!(frame.payload, b"ladder".as_ref()); - // Lite-02 carries no timestamps, and a lite-06 subscriber drops them. - let timed = alpn == "moq-transport-22"; + // Lite-02 carries no timestamps. + let timed = alpn != "moq-lite-02"; assert_eq!(frame.timestamp, timed.then_some(timestamp), "{alpn}"); } } diff --git a/rs/moq-tokio/tests/integration/broadcast.rs b/rs/moq-tokio/tests/integration/broadcast.rs index 5ef6505e9ea..3be0359c53e 100644 --- a/rs/moq-tokio/tests/integration/broadcast.rs +++ b/rs/moq-tokio/tests/integration/broadcast.rs @@ -135,9 +135,7 @@ async fn broadcast_test(scheme: &str, client_version: Option<&str>, server_versi } /// Lite05 publisher↔subscriber round-trip exercising the per-frame timestamp -/// delta encoding, including negative deltas (B-frame ordering). The subscriber decodes -/// each delta to stay in step with the payloads, then drops it, since a lite-05 timestamp -/// may be a send time. +/// delta encoding, including negative deltas (B-frame ordering). async fn lite05_timestamp_roundtrip(scheme: &str) { use moq_tokio::moq_net::{Timescale, Timestamp}; @@ -231,7 +229,10 @@ async fn lite05_timestamp_roundtrip(scheme: &str) { .expect("next_frame failed") .expect("group closed prematurely"); - assert_eq!(frame_sub.timestamp, None, "a lite-05 subscriber drops timestamps"); + let ts = frame_sub.timestamp.expect("timed frame"); + assert_eq!(ts.scale(), Timescale::MICRO); + assert_eq!(ts.value(), expected_us); + let payload = frame_sub.read_all().await.expect("failed to read frame"); assert_eq!(payload, bytes::Bytes::from(format!("frame@{expected_us}"))); } @@ -250,7 +251,7 @@ async fn broadcast_moq_lite_05_timestamps_webtransport() { /// Lite05 FETCH round-trip: retrieve a past group by sequence without holding a /// subscription, exercising the bare-FRAME fetch response and per-frame timestamp -/// decoding on the fetch stream, whose timestamps the subscriber then drops. The track is also `compress`-hinted, so the +/// decoding on the fetch stream. The track is also `compress`-hinted, so the /// fetched frames are Deflate-compressed (matching TRACK_INFO) and inflated by the /// subscriber, exercising fetch/TRACK_INFO codec consistency. async fn lite05_fetch_roundtrip(scheme: &str) { @@ -339,7 +340,9 @@ async fn lite05_fetch_roundtrip(scheme: &str) { .expect("next_frame failed") .expect("group closed prematurely"); - assert_eq!(frame_sub.timestamp, None, "a lite-05 subscriber drops timestamps"); + let ts = frame_sub.timestamp.expect("timed frame"); + assert_eq!(ts.scale(), Timescale::MICRO); + assert_eq!(ts.value(), expected_us); let payload = frame_sub.read_all().await.expect("failed to read frame"); assert_eq!(payload, bytes::Bytes::from(format!("frame@{expected_us}"))); @@ -604,10 +607,12 @@ async fn broadcast_moq_lite_05_fetch_during_subscribe_webtransport() { } /// Lite05 can't declare an untimed track, so a track that sets no timescale goes out -/// in milliseconds, its frames stamped with their send time. The subscriber drops them, -/// so the track arrives untimed. +/// in milliseconds, its frames stamped with their send time. The subscriber receives +/// `Some(ts)` at that scale. #[tokio::test] async fn broadcast_moq_lite_05_untimed_track() { + use moq_tokio::moq_net::Timescale; + let pub_origin = moq_tokio::origin::spawn(); let broadcast = pub_origin.create_broadcast("test").expect("create broadcast"); broadcast.announce(Default::default()).expect("create broadcast"); @@ -679,8 +684,13 @@ async fn broadcast_moq_lite_05_untimed_track() { .expect("next_frame failed") .expect("group closed"); - assert_eq!(frame_sub.timestamp, None, "the send time is dropped"); - assert_eq!(track_sub.info().timescale, None, "the track arrives untimed"); + let ts = frame_sub.timestamp.expect("timed frame"); + assert_eq!( + ts.scale(), + Timescale::MILLI, + "an untimed track's send times are milliseconds" + ); + assert_eq!(track_sub.info().timescale, Some(Timescale::MILLI)); drop(connection); server_handle diff --git a/test/interop/clients/c/subscribe.c b/test/interop/clients/c/subscribe.c index ddd9d91dabb..f0c6ca8d617 100644 --- a/test/interop/clients/c/subscribe.c +++ b/test/interop/clients/c/subscribe.c @@ -383,9 +383,10 @@ static void on_finish_health_frame(void *ud, int32_t frame) { int32_t read = moq_consume_track_frame((uint32_t)frame, &info); const char payload[] = "session-alive"; pthread_mutex_lock(&c->mu); + // Its timestamp goes unchecked: lite-05/06 carry an untimed track's send time. if (read < 0 || info.payload_size != sizeof(payload) - 1 || memcmp(info.payload, payload, sizeof(payload) - 1) != 0 || - info.timestamp_present || ++c->finish.health_frames != 1) c->finish.failed = 1; + ++c->finish.health_frames != 1) c->finish.failed = 1; pthread_cond_broadcast(&c->cv); pthread_mutex_unlock(&c->mu); moq_consume_track_frame_free((uint32_t)frame); diff --git a/test/interop/clients/cpp/main.cpp b/test/interop/clients/cpp/main.cpp index 89a3dfad8d5..c91a89abc6e 100644 --- a/test/interop/clients/cpp/main.cpp +++ b/test/interop/clients/cpp/main.cpp @@ -410,8 +410,9 @@ int subscribe( auto group = until(track->next_group(), read_deadline, "health group"); if (!group) fail("missing health group"); auto frame = until(group->read_frame(), read_deadline, "health frame"); - if (!frame || std::string(frame->payload.begin(), frame->payload.end()) != "session-alive" || frame->timestamp_us) { - fail("health payload or timestamp"); + // Its timestamp goes unchecked: lite-05/06 carry an untimed track's send time. + if (!frame || std::string(frame->payload.begin(), frame->payload.end()) != "session-alive") { + fail("health payload"); } if (until(group->read_frame(), read_deadline, "health group EOF") || until(track->next_group(), read_deadline, "health track EOF") || session->state().connects != 1) { diff --git a/test/interop/clients/go/main.go b/test/interop/clients/go/main.go index 8e44a841896..27bd64942a9 100644 --- a/test/interop/clients/go/main.go +++ b/test/interop/clients/go/main.go @@ -432,8 +432,9 @@ func finishSubscribe(ctx context.Context, client *moq.Client, broadcast string, if err != nil { return err } - if frame == nil || string(frame.Payload) != "session-alive" || frame.Timestamp != nil { - return errors.New("unexpected session health payload or timestamp") + // Its timestamp goes unchecked: lite-05/06 carry an untimed track's send time. + if frame == nil || string(frame.Payload) != "session-alive" { + return errors.New("unexpected session health payload") } frame, err = group.ReadFrame(ctx) if err != nil { diff --git a/test/interop/clients/js-native/finish.ts b/test/interop/clients/js-native/finish.ts index 6a02a7f167a..9449a496070 100644 --- a/test/interop/clients/js-native/finish.ts +++ b/test/interop/clients/js-native/finish.ts @@ -69,11 +69,8 @@ export async function readFinish( if (!group) throw new Error("no health group"); try { const frame = await group.readFrame(); - if ( - !frame || - new TextDecoder().decode(frame.payload) !== "session-alive" || - frame.timestamp !== undefined - ) + // Its timestamp goes unchecked: lite-05/06 carry an untimed track's send time. + if (!frame || new TextDecoder().decode(frame.payload) !== "session-alive") throw new Error("health frame"); if ((await group.readFrame()) !== undefined || (await track.recvGroup()) !== undefined) throw new Error("health EOF"); diff --git a/test/interop/clients/python/interop.py b/test/interop/clients/python/interop.py index a02c125d6b0..2ba8042d34b 100644 --- a/test/interop/clients/python/interop.py +++ b/test/interop/clients/python/interop.py @@ -235,8 +235,9 @@ async def _finish_subscribe(client: moq.Client, broadcast: str, frames: int) -> async for group in live: async with group: async for frame in group: - if seen or frame.payload != b"session-alive" or frame.timestamp is not None: - raise RuntimeError("unexpected session health payload or timestamp") + # Its timestamp goes unchecked: lite-05/06 carry an untimed track's send time. + if seen or frame.payload != b"session-alive": + raise RuntimeError("unexpected session health payload") seen = True if not seen: raise RuntimeError("missing session health payload") diff --git a/test/interop/clients/rust/finish.rs b/test/interop/clients/rust/finish.rs index 3c13ac09920..23df9ab5187 100644 --- a/test/interop/clients/rust/finish.rs +++ b/test/interop/clients/rust/finish.rs @@ -169,10 +169,8 @@ async fn main() -> anyhow::Result<()> { let mut track = health.track("live")?.subscribe(None).await?; let mut group = track.recv_group().await?.context("no health group")?; let frame = group.read_frame().await?.context("no health frame")?; - ensure!( - frame.payload.as_ref() == b"session-alive" && frame.timestamp.is_none(), - "health payload or timestamp" - ); + // Its timestamp goes unchecked: lite-05/06 carry an untimed track's send time. + ensure!(frame.payload.as_ref() == b"session-alive", "health payload"); ensure!( group.read_frame().await?.is_none() && track.recv_group().await?.is_none(), "health EOF"