Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions doc/concept/moq-lite.md
Original file line number Diff line number Diff line change
Expand Up @@ -376,14 +376,15 @@ A track is timed or untimed: it declares a timescale or none, and a frame that
doesn't match is refused. No receiver fills in an arrival time, so an untimed
track is judged by the wall clock alone. Tracks from moq-lite before 05, or from
moq-transport without a `TIMESCALE`, arrive
untimed. moq-lite 05 and 06 cannot declare an untimed track, so there its
frames go out stamped with their send time; the work-in-progress 07 carries it
without a timescale, each frame stamped with its group's start.
untimed. moq-lite 05 and 06 cannot declare an untimed track, so there it goes
out as milliseconds, each frame stamped with its group's start; the
work-in-progress 07 carries it without a timescale, with the same stamps.

A publisher refuses a group that starts before a lower one, and so does a
moq-lite 07 receiver. A receiver passes on the timestamps it gets over moq-lite
05 and 06 without checking them, since their send times look like media time and
an older group fetched later would look like a rewind. A relay serves them on
05 and 06 without checking them, since an older publisher stamps an untimed
track there with send times, which look like media time, and an older group
fetched later would look like a rewind. A relay serves them on
over every version. A moq-transport frame's time is its Group ID, so its
timestamps cannot rewind and are checked like any other.

Expand Down
4 changes: 2 additions & 2 deletions doc/setup/upgrade.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,8 +112,8 @@ These land with the next breaking release.
receive.** Tracks from moq-lite before 05, and moq-transport tracks without
`TIMESCALE` arrive untimed instead of
stamped with their arrival time. moq-lite 05 and 06 can't encode absence and
carry a send time instead, which their receivers pass through as the track's
time; see [untimed tracks](/concept/moq-lite#subscriptions). A new subscriber with no
carry each group's start as milliseconds instead, which their receivers pass
through as the track's time; see [untimed tracks](/concept/moq-lite#subscriptions). A new subscriber with no
start on an untimed track starts at the latest group.

- **Max age and max delay count wall-clock time too.** A superseded group is
Expand Down
4 changes: 2 additions & 2 deletions drafts/draft-lcurley-moq-lite.md
Original file line number Diff line number Diff line change
Expand Up @@ -824,7 +824,7 @@ A group is expired once either of two ages meets the relevant `Max Age`:
- Its **timestamp age** is the difference between the *newest* frame timestamp of the latest group that has at least one frame, and this group's **reach**, defined below.
- Its **arrival age** is the time elapsed since its successor arrived at this hop.

On a scale-less Track, timestamp age counts bare units as the milliseconds of a Max Age.
A scale-less Track has no timestamp age: its groups expire by arrival age alone.

By timestamp age, a group is expired only once it **provably cannot overlap the Max Age window**: every frame it could still present is older than the budget allows.
Arrival age needs no such proof: it bounds how long a superseded group may still wait to be delivered.
Expand Down Expand Up @@ -2365,7 +2365,7 @@ A stream that FINs without a GROUP_END carries a closed Group, which may have he
- FRAME and datagrams always carry a timestamp, a bare number on a scale-less Track.
- Replaced GROUP's `Group Sequence` with `Group Start`, added `Previous Start`, the start of the Group before it, so a subscriber sees the gap before a Group without a SUBSCRIBE_SKIP.
- Added GROUP_END, a `Payload Length` of 0 followed by the Group's exclusive end: a Group or Fetch Stream ends with it when its Group finished, a FIN alone ends a closed Group whose frames may not all exist, and a reset aborts. A Group's own end bounds its reach.
- Expiration also measures a group's arrival age, the time since its successor arrived at this hop.
- Expiration also measures a group's arrival age, the time since its successor arrived at this hop, and only that on a scale-less Track.
- A relay closes a Group it holds only the start of, without a GROUP_END. A publisher resets the Group Stream of a group that expires with OLD, even one whose GROUP_END it sent but is not yet acknowledged, and names the group in a SUBSCRIBE_DROP with OLD.
- FRAME leads with `Payload Length`, the payload size plus one, in place of the trailing `Message Length`; 0 is malformed.
- A publisher SHOULD drop a datagram rather than block when the transport has no room for it, where it previously sent one only if the congestion controller could transmit it immediately.
Expand Down
7 changes: 4 additions & 3 deletions js/net/src/group-id.test.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
// Group IDs past 2^53 decode, survive a JS forwarding node, and encode again on every wire
// version, as a third-party moq-transport publisher's or a fine timescale's start ticks may be.
// version that can carry them, as a third-party moq-transport publisher's or a fine timescale's start
// ticks may be.
import { expect, test } from "bun:test";
import { type Consumer as BroadcastConsumer, Producer as BroadcastProducer } from "./broadcast.ts";
import { accept, connect } from "./connection/index.ts";
Expand Down Expand Up @@ -33,8 +34,8 @@ const WIRES: Wire[] = [
{ name: "lite-02", protocol: "", version: Lite.Version.DRAFT_02 },
{ name: "lite-03", protocol: Lite.ALPN_03 },
{ name: "lite-04", protocol: Lite.ALPN_04 },
{ name: "lite-05", protocol: Lite.ALPN_05, datagrams: true },
{ name: "lite-06", protocol: Lite.ALPN_06, datagrams: true },
// Lite-05 and lite-06 are missing: they stamp a scale-less track's frames and datagrams with
// their keys as milliseconds, which a JS hop's timestamps hold only below 2^53.
{ name: "lite-07-wip", protocol: Lite.ALPN_07_WIP, datagrams: true },
{ name: "ietf-18", protocol: Ietf.ALPN.DRAFT_18 },
{ name: "ietf-20", protocol: Ietf.ALPN.DRAFT_20 },
Expand Down
60 changes: 59 additions & 1 deletion js/net/src/lite/publisher.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,16 @@ import {
SubscribeUpdate,
} from "./subscribe.ts";
import { TrackInfo, Track as TrackMessage } from "./track.ts";
import { ALPN_05, ALPN_06, ALPN_07_WIP, hasLengthFirst, hasRanges, keepsSubscribeOpen, Version } from "./version.ts";
import {
ALPN_05,
ALPN_06,
ALPN_07_WIP,
hasLengthFirst,
hasRanges,
keepsSubscribeOpen,
Version,
versionName,
} from "./version.ts";

function publish(origin: OriginProducer, path: Path.Valid) {
const broadcastRoute = origin.route(path);
Expand Down Expand Up @@ -642,6 +651,55 @@ test("lite draft-06: an unchecked track's fetched frames carry their own timesta
}
});

// Replays `rs/moq-net/fixtures/stamps.json`, which moq-net replays too: lite-05/06 stamp a
// scale-less frame with its group's start, never a clock, so a frame carries the same timestamp
// on every hop and fetch, and its Group Stream matches byte for byte in both languages.
const STAMPS = (await Bun.file(new URL("../../../../rs/moq-net/fixtures/stamps.json", import.meta.url)).json()) as {
name: string;
version: string;
start: number;
payloads: string[];
hex: string;
}[];

test.each(STAMPS.map((stamp) => [stamp.name, stamp] as const))("%s", async (_name, stamp) => {
using clock = manual();
const version = defined([Version.DRAFT_05, Version.DRAFT_06].find((v) => versionName(v) === stamp.version));
const pair = createMockTransportPair(stamp.version);
const origin = new OriginProducer();
const publisher = new Publisher(pair.server, version, randomHop(), origin.consume());
const broadcast = publish(origin, Path.from("test"));
const track = defined(broadcast.createTrack("video", {}));

const client = await Stream.open(pair.client, { version });
const server = await Stream.accept(pair.server, version);
if (!server) throw new Error("publisher never accepted the subscribe stream");
void publisher.runSubscribe(
new Subscribe({ id: 0n, broadcast: Path.from("test"), track: "video", priority: 0 }),
server,
);

const start = BigInt(stamp.start);
const group = track.createGroup(start);
for (const payload of stamp.payloads) group.writeFrame(textFrame(payload, Timestamp.unscaled(start)));
group.finish();
await clock.advance(Duration.ZERO);

const opened = pair.client.incomingUnidirectionalStreams.getReader();
try {
const next = await opened.read();
if (next.done) throw new Error("publisher never opened the group stream");
const bytes = await new Response(next.value).arrayBuffer();
expect(Buffer.from(bytes).toString("hex")).toBe(stamp.hex);
} finally {
await opened.cancel();
publisher.close();
client.close();
broadcast.close();
origin.close();
}
});

// How long a served-subscription test waits on its manual clock for the next group stream
// before calling the publisher idle.
const IDLE_MS = Duration.millis(500);
Expand Down
44 changes: 13 additions & 31 deletions js/net/src/lite/publisher.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,8 @@ import type { Consumer as OriginConsumer } from "../origin.ts";
import * as Path from "../path.ts";
import { readSize, sendPayload } from "../payload.ts";
import { Priority, sendOrder } from "../priority.ts";
import { rescale } from "../rescale.ts";
import { concat, encodeVarint, type Reader, type Stream, type StreamVersion, Writer } from "../stream.ts";
import { floorAt, type Timescale, Timestamp } from "../time.ts";
import { floorAt, type Timescale, type Timestamp } from "../time.ts";
import type * as track from "../track.ts";
import type { Reported } from "../track.ts";
import { untilAborted } from "../util/abort.ts";
Expand Down Expand Up @@ -150,42 +149,33 @@ function groupEnd(version: StreamVersion, end: bigint): Uint8Array {
interface Stamp {
/**
* The wire timescale TRACK_INFO declared (see `wireTimescale`), or undefined on a scale-less
* lite-07 track, whose timestamps go out as bare numbers: a frame's its group's start.
* lite-07 track, whose timestamps go out as bare numbers.
*/
timescale?: Timescale;
/**
* The timed track whose frames' own timestamps go out, at its timescale, which the wire's
* equals. Lite-05/06 cannot encode a scale-less track, so a scale-less one carries its send
* time instead.
*/
timed?: { timescale?: Timescale | null };
}

/**
* How `version` stamps a track with `info`, or undefined when it carries no timestamps: before
* lite-05. Lite-05/06 declare milliseconds for a scale-less track, and lite-07 stamps it bare.
*/
function stamping(version: Version, info: { timescale?: Timescale | null }): Stamp | undefined {
const timescale = info.timescale ?? undefined;
const wire = wireTimescale(version, timescale);
const wire = wireTimescale(version, info.timescale);
if (wire === undefined) return hasLengthFirst(version) ? {} : undefined;
return timescale === undefined ? { timescale: wire } : { timescale: wire, timed: info };
return { timescale: wire };
}

/**
* A frame or datagram's `timestamp` as its raw wire value: bare on a scale-less lite-07 track, or
* its send time when a lite-05/06 track goes out scale-less.
* A frame or datagram's `timestamp` as its raw wire value. A scale-less track's is its group's
* start, or its datagram's sequence, which goes out verbatim on every version: nothing is stamped
* with a clock, so a frame carries the same timestamp through every hop and fetch. Mirrors the
* Rust `Stamp::wire`.
*/
function wireTime(timestamp: Timestamp, stamp: Stamp): bigint {
if (stamp.timescale === undefined) return bareTime(timestamp);
return BigInt(stamp.timed ? rescale(stamp.timed, timestamp).value : floorAt(Timestamp.now(), stamp.timescale));
}

/** A scale-less lite-07 datagram's timestamp as it goes out: the bare value its producer gave. */
function bareTime(timestamp: Timestamp): bigint {
const key = timestamp.key;
if (key === undefined) throw new TimestampMismatch();
return key;
if (key !== undefined) return key;
// A timed timestamp on a track declared scale-less: the model never writes one.
if (stamp.timescale === undefined) throw new TimestampMismatch();
return BigInt(floorAt(timestamp, stamp.timescale));
}

/**
Expand Down Expand Up @@ -2423,15 +2413,7 @@ export class Publisher {
end = max(end, datagram.sequence + 1n);
// An old subscription uses its stream track's timescale for both kinds. Without one
// the track is a scale-less lite-07 one, whose timestamp goes out bare.
const timescale = options.stamp?.timescale;
const timestamp =
timescale === undefined
? bareTime(datagram.timestamp)
: BigInt(
datagram.timestamp.scale === undefined
? floorAt(Timestamp.now(), timescale)
: floorAt(datagram.timestamp, timescale),
);
const timestamp = wireTime(datagram.timestamp, options.stamp ?? {});
const body = new DatagramMessage(options.sub, datagram.sequence, timestamp, datagram.payload).encode(
this.version,
);
Expand Down
2 changes: 1 addition & 1 deletion js/net/src/lite/subscriber.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2854,7 +2854,7 @@ async function groupStream(timestamp: number, version: Version = Version.DRAFT_0
);
}

// Lite-05/06 stamp an untimed track's frames with their send time, which a receiver cannot tell
// A released lite-05/06 publisher stamps an untimed track's frames with their send time, which a receiver cannot tell
// from media time, so it keeps every timestamp there unchecked: a group that rewinds or lands out
// of order is accepted with its timestamp.
test.each([Version.DRAFT_05, Version.DRAFT_06])("version %p passes received timestamps through", async (version) => {
Expand Down
8 changes: 4 additions & 4 deletions js/net/src/lite/version.ts
Original file line number Diff line number Diff line change
Expand Up @@ -483,9 +483,9 @@ export function hasRefresh(version: Version): boolean {
}
}

/** Whether received frame timestamps are the publisher's media time. Lite-05/06 stamp an untimed
* track's payloads with their send time, which a receiver cannot tell from it, so it passes them
* through as the track's time without checking them for a rewind. */
/** Whether received frame timestamps are the publisher's media time. A released lite-05/06
* publisher may stamp an untimed track's payloads with their send time, which a receiver cannot
* tell from it, so it passes them through as the track's time without checking them for a rewind. */
export function hasMediaTime(version: Version): boolean {
// Explicitly list older versions so future versions keep the lite-07+ behavior.
switch (version) {
Expand Down Expand Up @@ -637,7 +637,7 @@ export function hasEndError(version: Version): boolean {
/**
* The timescale the wire carries for a track with `timescale`, or `undefined` when its frames
* and datagrams carry no timestamp. Lite-05 and lite-06 always carry one, so an untimed track
* declares milliseconds there and its payloads carry their send times instead.
* declares milliseconds there and its payloads carry their group's start, or their sequence.
*/
export function wireTimescale(version: Version, timescale: Timescale | null | undefined): Timescale | undefined {
switch (version) {
Expand Down
7 changes: 4 additions & 3 deletions js/net/src/resume.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2005,8 +2005,8 @@ test.each([
])(
"a group fetched over a wire goes stale by the fetch's own budget ($budget)",
async ({ maxDelay, stale, bounded }) => {
// Still, so only presentation time judges.
const { installed } = manual();
// Still until the test moves it: a scale-less track ages by arrival alone.
const { clock, installed } = manual();
const front = new Front();
const a = new BroadcastProducer();
const ta = copy(a);
Expand Down Expand Up @@ -2037,8 +2037,9 @@ test.each([
await settle();
expect(group.expired.peek()).toBeUndefined();

// The copy's live edge runs a second past group 0's successor.
// Group 0's successor arrived at the copy a second ago.
ta.write(2000n);
await clock.advance(SECOND);
if (stale) expect(await verdict).toMatchObject({ code: StreamCode.Old });
else expect(await isPending(verdict)).toBe(true);

Expand Down
27 changes: 27 additions & 0 deletions js/net/src/track.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1334,6 +1334,33 @@ test("a scale-less join reaches back by arrival", () => {
}
});

// A scale-less track's starts and ends are bare numbers, never milliseconds, so groups whose
// starts jump age by arrival alone, anywhere in the key range, and a group finished whole waits
// for a successor. Mirrors the Rust `a_scale_less_track_ages_by_arrival_alone`.
test("a scale-less track ages by arrival alone", () => {
const clock = manualTime(0);
try {
for (const [i, base] of [0n, 1n << 61n, 2n ** 64n - 30_001n].entries()) {
clock.set(i * 1000);
const producer = new TrackProducer("test").accept({ maxAge: null });
for (const offset of [0n, 10_000n, 20_000n]) {
const group = producer.createGroup(base + offset);
group.writeFrame(textFrame("x", Timestamp.unscaled(base + offset)));
group.finishAt(base + offset + 1n);
}
const subscribe = () => producer.subscribe({ delay: { max: Duration.millis(5) } });
// All three arrived at once, so however far apart their starts, none is stale yet.
expect(readable(subscribe())).toEqual([base, base + 10_000n, base + 20_000n]);
// Their successors arrived a full budget ago, and the newest has none to age it.
clock.set(i * 1000 + 10);
expect(readable(subscribe())).toEqual([base + 20_000n]);
producer.close();
}
} finally {
clock.installed.close();
}
});

// A time start resolves on keys alone: groups opened before their first frames, below a
// reached end, are read from the one containing it. Mirrors the Rust `a_time_start_needs_no_frame`.
test("a time start resolves on keys alone, before any group's first frame", () => {
Expand Down
Loading