Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 8 additions & 6 deletions doc/bin/relay/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -659,12 +659,14 @@ slack below physical memory (or just use `headroom`, which measures actual
available memory). `duration` is the age counterpart: it stops idle history
from pinning memory when the byte budget alone leaves room for it.

All eviction happens as tracks write (there is no background reaper), so both
`duration` and the byte budget cap how much history *active* publishers build
up. A publisher that stops writing but stays connected keeps what it had cached
until it resumes or the broadcast closes; under memory pressure the byte budget
is repaid by the tracks that are still writing. A publisher that disconnects has
its groups released as soon as the broadcast closes with it.
The byte budget is repaid as tracks write, so it caps how much history *active*
publishers build up: a publisher that stops writing pays none of it down, and
under memory pressure the budget is repaid by the tracks that are still writing.
`duration` does not depend on writes. The relay sweeps every cached track on a
wall-clock cadence, so a publisher that stalls but stays connected still has its
idle groups reclaimed, including one it left a group open in, and a subscriber
parked inside that group is told rather than waiting forever. A publisher that
Comment on lines +665 to +668

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Synchronize the cache-duration CLI help

The updated relay documentation now promises write-independent sweeping, but the --cache-duration help generated from rs/moq-relay/src/cache.rs:57-61 still says expiry runs only on writes and that stalled publishers retain their cache. Operators inspecting moq-relay --help therefore receive the opposite behavior from this page; update that field's doc comment along with this behavioral change. (Written by GPT-5.6 Sol)

AGENTS.md reference: AGENTS.md:L128-L128

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, fixed. This is the Cross-Package Sync rule and I missed it: doc/bin/relay/config.md was updated for the new behavior while the --help text it mirrors was not, so the two gave opposite answers.

rs/moq-relay/src/cache.rs now says the same thing as the docs page: the window holds whether or not a publisher is still writing, since the relay sweeps every cached track on a wall-clock cadence, and capacity is the budget that depends on writes. The live-edge caveat stays.

(Written by claude-opus-5[1m])

disconnects has its groups released as soon as the broadcast closes with it.

All three flags also accept CLI arguments (`--cache-capacity`,
`--cache-headroom`, `--cache-duration`) and environment variables
Expand Down
14 changes: 14 additions & 0 deletions js/net/src/group.ts
Original file line number Diff line number Diff line change
Expand Up @@ -137,11 +137,24 @@ export class Producer {
// so this is what stops a reader-less fetch instead of the stream ending on its own.
#used = new Signal<boolean>(false);

// When the group was created or last written. Retention ages a group from this,
// not from its creation, so a group still being filled is never reclaimed.
#activity = performance.now();

constructor(sequence: number) {
this.#state = new GroupState(sequence);
this.sequence = sequence;
}

/**
* When the group was created or last written, in `performance.now()` milliseconds.
*
* @internal Track retention only.
*/
get activity(): number {
return this.#activity;
}

/**
* Settles once the group closes: `null` on a clean close, or the abort {@link Error}.
* Peek it synchronously (`undefined` while open), observe it reactively, or `await` it.
Expand Down Expand Up @@ -228,6 +241,7 @@ export class Producer {
// `Error::FrameTooLarge`; do the same rather than silently dropping it.
if (frame.payload.byteLength > MAX_GROUP_CACHE_BYTES) throw new FrameTooLarge();

this.#activity = performance.now();
appendFrame(this.#state, frame);

if (this.#mirrors) {
Expand Down
72 changes: 72 additions & 0 deletions js/net/src/track.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -804,6 +804,78 @@ test("retention pruning aborts a held mirror without a new live edge", async ()
}
});

test("retention reclaims a group the publisher abandoned open", async () => {
const clock = mockMonotonicTime(10_000);
try {
const producer = new TrackProducer("test").accept({ maxAge: 100 });
const track = producer.subscribe({ maxAge: 100 });
const stalled = producer.appendGroup();
stalled.writeString("first");
// A successor, so the stalled group is not the live edge the publisher is
// still filling. It is never closed: the publisher simply walked away.
producer.appendGroup();

const group = await track.recvGroup();
if (!group) throw new Error("missing group");
expect(await group.readString()).toBe("first");

clock.set(10_200);
producer.subscribe({ maxAge: 100 });

// A gap, not a clean finish: the publisher never ended this group.
await expect(group.readFrame()).rejects.toBeInstanceOf(Lagged);
} finally {
clock.restore();
}
});

test("an abandoned open group ages out with no further write", async () => {
// Real time, since this is about the wakeup: nothing writes to the track again, so
// without a timer the read below parks forever.
const producer = new TrackProducer("test").accept({ maxAge: 30 });
const track = producer.subscribe({ maxAge: 30 });
const stalled = producer.appendGroup();
stalled.writeString("first");
producer.appendGroup();

const group = await track.recvGroup();
if (!group) throw new Error("missing group");
expect(await group.readString()).toBe("first");

const read = group.readFrame().then(
() => "clean end",
(err: unknown) => err,
);
const timeout = new Promise((resolve) => setTimeout(() => resolve("still parked"), 1000));
expect(await Promise.race([read, timeout])).toBeInstanceOf(Lagged);
});

test("the prune wakeup never exceeds the setTimeout limit", () => {
// A window past 2^31 ms truncates the delay to a signed 32-bit int, so the wakeup
// fires immediately and re-arms the same oversized delay: a millisecond timer loop
// instead of a wakeup. Capped, it just wakes early and finds nothing due.
const real = globalThis.setTimeout;
const delays: number[] = [];
// @ts-expect-error a stub, not a full setTimeout
globalThis.setTimeout = (_fn: () => void, delay: number) => {
delays.push(delay);
return { unref: () => {} };
};

try {
const producer = new TrackProducer("test").accept({ maxAge: 2 ** 31 + 10_000 });
const stalled = producer.appendGroup();
stalled.writeString("first");
producer.appendGroup(); // the live edge, so the stalled group is prunable
producer.close();
} finally {
globalThis.setTimeout = real;
}

expect(delays.length).toBeGreaterThan(0);
for (const delay of delays) expect(delay).toBeLessThanOrEqual(2 ** 31 - 1);
});

test("retention pruning preserves clean EOF for a drained mirror", async () => {
const clock = mockMonotonicTime(10_000);
try {
Expand Down
67 changes: 64 additions & 3 deletions js/net/src/track.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ import { Timescale, type Timestamp } from "./time.ts";

export type { Datagram } from "./datagram.ts";

// The largest delay setTimeout keeps: anything more truncates to a signed 32-bit int
// and fires right away.
const MAX_TIMEOUT_MS = 2 ** 31 - 1;

/** Default {@link Info.maxAge} window (milliseconds) when the publisher does not set one. */
export const DEFAULT_MAX_AGE_MS = 5000;

Expand Down Expand Up @@ -365,11 +369,16 @@ export class Producer {
#sequence: TrackSequence = { next: 0 };

// Recently written source groups, retained for replay to late subscribers and
// pruned once closed and older than the cache window. Each entry tracks the mirror
// pruned once idle for longer than the cache window. Each entry tracks the mirror
// it handed to every sink so eviction can drop them too: otherwise a slow consumer
// that never reads would pin old groups (and their frame bytes) forever.
#cache: CachedGroup[] = [];

// Wakeup for the next entry due to age out. Writes settle retention inline, but a
// publisher that stalls stops writing, so without this an abandoned group (and any
// reader parked in it) would wait for a write that never comes.
#pruneTimer?: ReturnType<typeof setTimeout>;

// One independent downstream state per live subscriber.
#sinks = new Set<TrackState>();

Expand Down Expand Up @@ -533,6 +542,11 @@ export class Producer {

// Drop a cached group's mirror from every sink so no consumer can pin it.
#evict(entry: CachedGroup): void {
// Reclaiming a group the publisher never closed ends it as the gap it is, before
// the mirrors below would otherwise report a clean finish to a reader that had
// drained it. The usual case, an already-closed group aging out, keeps its own
// terminal state.
if (!entry.group.isClosed) entry.group.close(new Lagged());
for (const [sink, mirror] of entry.mirrors) {
hooks.evictGroup(mirror);
sink.groups.mutate((groups) => {
Expand All @@ -545,20 +559,65 @@ export class Producer {
entry.mirrors.clear();
}

// Evict cached groups that are closed and older than the cache window.
// The one group retention never takes: the newest, while it is still open. That is
// the live edge a publisher is appending to, and a track may legitimately keep it
// open across a long quiet stretch (a catalog snapshot, a JSON stream). Every other
// open group is an abandoned one, and ages out like a closed one.
#liveEdge(): GroupProducer | undefined {
const latest = this.#cache.at(-1)?.group;
return latest?.closed.peek() === undefined ? latest : undefined;
Comment on lines +566 to +568

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Select the live edge by sequence

When writeGroup() receives groups out of order, as the repo's relay tests explicitly allow for newest-first upstream delivery, the last cache entry is not necessarily the newest sequence. If group 2 is open and group 1 arrives afterward, this protects group 1 and makes the actual live group 2 eligible for idle eviction; after maxAge of quiet, group 2 is aborted even though the publisher may legitimately continue it. Track the highest cached sequence, matching sink.latest and the Rust model, rather than using insertion order. (Written by GPT-5.6 Sol)

AGENTS.md reference: AGENTS.md:L152-L156

Useful? React with 👍 / 👎.

}

// Evict cached groups idle for longer than the cache window. Idle means nothing
// written, so an abandoned open group ages out instead of pinning its buffer (and
// any reader parked in it) forever.
#prune(): void {
const maxAgeMs = this.#state.info.peek()?.maxAge ?? DEFAULT_MAX_AGE_MS;
const cutoff = performance.now() - maxAgeMs;
const live = this.#liveEdge();

const retained: CachedGroup[] = [];
for (const entry of this.#cache) {
if (entry.time >= cutoff || entry.group.closed.peek() === undefined) {
if (entry.group === live || entry.group.activity >= cutoff) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep in-progress frame downloads from expiring

When an incoming non-latest group takes longer than maxAge to receive one frame, this branch evicts it even while payload chunks are still arriving. Both lite/subscriber.ts::runGroup and ietf/subscriber.ts::handleGroup await the entire payload through stream.read(size) or Frame.decode before calling producer.writeFrame, so group.activity remains stale throughout the transfer; #evict then aborts the producer, causing the receive loop to exit and discard the partial frame. Refresh activity as chunks arrive or otherwise mark an in-progress frame as non-idle, with a regression test for a slow fragmented payload. (Written by GPT-5.6 Sol)

AGENTS.md reference: AGENTS.md:L152-L156

Useful? React with 👍 / 👎.

retained.push(entry);
continue;
}
this.#evict(entry);
}
this.#cache = retained;
this.#schedulePrune();
}

// Arm the wakeup for the next entry due to age out, replacing any pending one.
// Writes settle retention inline, so this only has to cover the case no write
// follows. Cheap to over-arm: an entry written since is retained and re-armed.
#schedulePrune(): void {
clearTimeout(this.#pruneTimer);
this.#pruneTimer = undefined;
if (this.#state.closed.peek() !== undefined) return;

// One pass, no intermediate arrays: this runs on every publish, and a spread
// over the cache would also cap how many groups a track can hold.
const live = this.#liveEdge();
let oldest: number | undefined;
for (const entry of this.#cache) {
if (entry.group === live) continue;
if (oldest === undefined || entry.group.activity < oldest) oldest = entry.group.activity;
}
if (oldest === undefined) return;
Comment on lines +603 to +607

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reschedule pruning when the live group closes

When the cache contains only its open live group, this skips the sole entry and returns without arming a timer. Normal helpers such as writeString() publish the group while it is open and close it afterward, but GroupProducer.close() never notifies the track, so with no later publish or subscription that now-closed group remains cached past maxAge indefinitely. Observe the close or otherwise schedule a wakeup, with coverage for a single closed group and no subsequent track operation. (Written by GPT-5.6 Sol)

AGENTS.md reference: AGENTS.md:L152-L156

Useful? React with 👍 / 👎.


const maxAgeMs = this.#state.info.peek()?.maxAge ?? DEFAULT_MAX_AGE_MS;
// setTimeout truncates its delay to a signed 32-bit int, so a longer window
// would fire immediately and spin. Wake at the cap instead and re-arm: #prune
// retains anything still fresh, so the extra wakeups are the only cost.
const delay = Math.min(MAX_TIMEOUT_MS, Math.max(0, oldest + maxAgeMs - performance.now()));
const timer = setTimeout(() => {
this.#pruneTimer = undefined;
this.#prune();
}, delay);
// A cache prune is never a reason to hold a Node/Bun process open.
(timer as unknown as { unref?: () => void }).unref?.();
this.#pruneTimer = timer;
}

// Retain a source group and fan it out to every live sink.
Expand Down Expand Up @@ -675,6 +734,8 @@ export class Producer {
for (const sink of this.#sinks) sink.final = this.#state.final;
}
closeTrackState(this.#state, abort);
clearTimeout(this.#pruneTimer);
this.#pruneTimer = undefined;
for (const { group } of this.#cache) group.close(abort);
for (const sink of this.#sinks) {
for (const group of sink.groups.peek()) group.close(abort);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,4 +34,3 @@ consumer's `discontinuity()` bumps once. Fix on `dev`.
## Related

- [Monotonic timeline](/quest/m1/monotonic-timeline.md) - makes the declared marker the only reset signal
- [#3161](/quest/m1/3161-retention-should-reclaim-idle-open-groups-now-that-expiry.md) - the other open-group lifecycle rule in the same area

This file was deleted.

1 change: 0 additions & 1 deletion quest/m1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ with the current dev tree before starting.
- [#2979](/quest/m1/2979-moq-tokio-does-not-compile-with-no-default-features-and.md) - moq-tokio compiles with any subset of transport features, and nightly checks it per crate
- [#2853](/quest/m1/2853-quiche-with-a-pinned-source-port-can-dial-only-a-broken.md) - quiche with a pinned source port can dial only a broken IPv4 address
- [#2624](/quest/m1/2624-moq-native-goaway-redirect-guard-classifies-hosts-by-name.md) - moq-native: GOAWAY redirect guard classifies hosts by name, not by resolved address
- [#3161](/quest/m1/3161-retention-should-reclaim-idle-open-groups-now-that-expiry.md) - Retention should reclaim idle open groups now that expiry is timestamp-only
- [Monotonic timeline](/quest/m1/monotonic-timeline.md) - a track's timestamps never fall below its live edge; publishers declare a discontinuity and continue forward, consumers stop detecting rewinds
- [Group overflow](/quest/m1/group-overflow-abort.md) - an oversized open group aborts for every reader instead of shedding its head
- [#2895](/quest/m1/2895-add-an-atomic-readiness-gate-for-origin-broadcasts.md) - Add an atomic readiness gate for Origin broadcasts
Expand Down
7 changes: 3 additions & 4 deletions quest/m1/group-overflow-abort.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,9 @@ evicted prefix.
- **A new error variant**, `Error::GroupTooLarge`, mirroring the existing
`FrameTooLarge` naming. `Lagged` is named from the consumer's side and would
keep blaming the reader for the writer's overrun; `Evicted` already means the
pool dropped a whole group under external memory pressure, and
[#3161](/quest/m1/3161-retention-should-reclaim-idle-open-groups-now-that-expiry.md)
will abort idle open groups, so these three need to stay distinguishable.
pool dropped a whole group under external memory pressure, and the cache
pool's idle sweep aborts an idle open group with `Old`, so these three need to
stay distinguishable.
- **The writer learns synchronously.** The write that pushes the group past its
budget returns `Err(Error::GroupTooLarge)` and aborts the group, which is the
shape `FrameTooLarge` already has in `write_frame`. Today `evict()` returns
Expand Down Expand Up @@ -126,6 +126,5 @@ already claims its top end "intentionally reaches the raised
## Related

- [Group charge](/quest/m0/group-charge.md) - pool-level budget accounting, unaffected by this change
- [#3161](/quest/m1/3161-retention-should-reclaim-idle-open-groups-now-that-expiry.md) - also turns "open group hit a limit" into an abort, so the two must not collide on the error variant
- [JS stream codes](/quest/m1/js-net-stream-error-codes.md) - without it a JS publisher sends this new code to the wire as Internal
- [#3001](/quest/m1/3001-ietf-stream-resets-send-moq-lite-error-codes-so-routine.md) - the moq-transport half of the code mapping
Loading
Loading