diff --git a/cpp/moq/MIGRATE.md b/cpp/moq/MIGRATE.md index 1f13c3321fe..025937be93a 100644 --- a/cpp/moq/MIGRATE.md +++ b/cpp/moq/MIGRATE.md @@ -4,6 +4,13 @@ How to update your code for each breaking change, newest first. ## Unreleased +### A client asks only about what it watches (#6165) + +On moq-lite, a client session asks its relay for announcements only under paths something watches, through `announced` or `announced_broadcast`, instead of under its whole scope at connect. +`OriginConsumer::request_broadcast` watches nothing, so on a client it fails with an unroutable error for a path nothing watches, even when the relay announces it. +Resolve the path with `announced_broadcast`, or hold `announced` over its prefix while requesting. +moq-transport and moq-lite-03/04 sessions are unchanged. + ### `moq::ContainerFormat::kFmp4` unwraps to Legacy (#6250) A container import with `moq::ContainerFormat::kFmp4` now publishes one Legacy frame per sample instead of one CMAF frame per fragment. diff --git a/dart/moq/MIGRATE.md b/dart/moq/MIGRATE.md index 43386ac161c..2c305a2cb5a 100644 --- a/dart/moq/MIGRATE.md +++ b/dart/moq/MIGRATE.md @@ -4,6 +4,13 @@ How to update your code for each breaking change, newest first. ## Unreleased +### A client asks only about what it watches (#6165) + +On moq-lite, a client session asks its relay for announcements only under paths something watches, through `announced()` or `announcedBroadcast()`, instead of under its whole scope at connect. +`requestBroadcast()` watches nothing, so on a client it fails with an unroutable error for a path nothing watches, even when the relay announces it. +Resolve the path with `announcedBroadcast()`, or hold `announced()` over its prefix while requesting. +moq-transport and moq-lite-03/04 sessions are unchanged. + ### `ContainerFormat.fmp4` unwraps to Legacy (#6250) A container import with `ContainerFormat.fmp4` now publishes one Legacy frame per sample instead of one CMAF frame per fragment. diff --git a/dart/moq_ffi/MIGRATE.md b/dart/moq_ffi/MIGRATE.md index e0ee6d6efd6..d8d452d9e39 100644 --- a/dart/moq_ffi/MIGRATE.md +++ b/dart/moq_ffi/MIGRATE.md @@ -4,6 +4,14 @@ How to update your code for each breaking change, newest first. ## Unreleased +### A client asks only about what it watches (#6165) + +On moq-lite, a client session asks its relay for announcements only under paths something watches, through `announced()` or `announcedBroadcast()`, instead of under its whole scope at connect. +`MoqOriginConsumer.requestBroadcast()` watches nothing, so on a client it fails with an unroutable error for a path nothing watches, even when the relay announces it. +Resolve the path with `announcedBroadcast()`, or hold `announced()` over its prefix while requesting. +moq-transport and moq-lite-03/04 sessions are unchanged. +Regenerate the bindings for the changed `request_broadcast` checksum. + ### Catalog references carry their epoch `MoqVideo` and `MoqAudio` gain `epoch`, the publisher instance their `broadcast` reference names. diff --git a/dart/moq_ffi/lib/src/moq.dart b/dart/moq_ffi/lib/src/moq.dart index 3e8d9dbd938..05440ca2688 100644 --- a/dart/moq_ffi/lib/src/moq.dart +++ b/dart/moq_ffi/lib/src/moq.dart @@ -19609,7 +19609,7 @@ void _checkApiChecksums() { throw UniffiInternalError.panicked("UniFFI API checksum mismatch"); } if (uniffi_moq_ffi_checksum_method_moqoriginconsumer_request_broadcast() != - 64026) { + 65225) { throw UniffiInternalError.panicked("UniFFI API checksum mismatch"); } if (uniffi_moq_ffi_checksum_method_moqorigindynamic_cancel() != 47453) { diff --git a/doc/lib/dart/index.md b/doc/lib/dart/index.md index 6565953d44d..84f76c2113c 100644 --- a/doc/lib/dart/index.md +++ b/doc/lib/dart/index.md @@ -66,7 +66,8 @@ moq default: } }); -final broadcast = await moq.requestBroadcast('live/camera'); +// Wait for a publisher at one path; call available() again for the next one. +final broadcast = await moq.announcedBroadcast('live/camera').available(); final catalog = await media.CatalogConsumer.subscribe(broadcast: broadcast); print(await catalog.next()); ``` @@ -99,7 +100,7 @@ encoder.finish(); // Decode a subscribed rendition. Dispose each frame promptly. import 'package:moq/video.dart' as video; -final remote = await moq.requestBroadcast('live/camera'); +final remote = await moq.announcedBroadcast('live/camera').available(); final renditions = await media.CatalogConsumer.subscribe(broadcast: remote); final decoder = await video.Consumer.subscribe( broadcast: remote, diff --git a/doc/lib/rs/moq-net.md b/doc/lib/rs/moq-net.md index a815bb91777..d9295f9ad4a 100644 --- a/doc/lib/rs/moq-net.md +++ b/doc/lib/rs/moq-net.md @@ -19,6 +19,7 @@ What you use it for, beyond what the concept page already describes: - **Requests can pin an epoch.** `consumer.request_broadcast(path, Some(epoch))` resolves only through a route announcing that [publisher epoch](/concept/moq-lite#publisher-epochs); `None` takes whichever route wins. The resolved consumer names its epoch in `info().epoch`, pinned or not. - **Rank several CDNs.** Dial each with its own subscribe handle, `origin.clone().with_preference(origin::Preference::new(rank))` (higher wins; 0 is neutral and a negative rank demotes), all on one origin. Requests use the preferred link that has a route and fail over to the next; see [multiple CDNs](/concept/moq-lite#multiple-cdns). - **Follow restarts.** `consumer.announced_broadcast(path, None)` hands out each publisher instance serving a path from `available()`: the first once announced, then each one that takes over or returns after an end, like JS `request(path, { announced: true })`. A broadcast already resolved stays on the old instance until dropped or its route goes. `consumer.follow(path)` yields the underlying start, update, restart, and end of the route a request resolves. +- **Discovery follows what you watch.** On moq-lite, a client session asks the relay for announcements only under the prefixes its subscribe origin's readers watch, with `announced()`, `routed_broadcast`, or `announced_broadcast`, stops asking about a prefix 2s after nothing at or beneath it is watched, and asks again on reconnect. A broadcast resolved through a discovered route keeps its path watched until dropped. `request_broadcast` watches nothing, so a path nobody watches is unroutable. A server session, or a client fed into a `peer()` or `upstream()` origin handle, asks for everything it may, so a relay discovers everything. An HTTP gateway (the moq-hls `Server`, the moq-rtc WHEP router) watches its whole scope while it lives, so the paths its callers name never open a request each. moq-transport sessions, and moq-lite-03/04 sessions (whose announcements mark no initial set), still ask for everything. - **One track per name.** Concurrent subscriptions share one request and producer. The application owns sequence numbers: a producer that replaces an ended one gets `Error::Duplicate` from an append that would reuse a sequence the name already wrote, so it starts with `create_group` at an explicit sequence. Only a new broadcast starts over. Readers of an aborted track resume onto the replacement only when the broadcast announces an epoch; otherwise their subscription ends with the error. - **End a broadcast with `finish()`.** When the input ends, finish the tracks, then call `broadcast.finish()`. The returned handle keeps the broadcast announced and serving its finished tracks for `linger()`, their longest max-age, so a subscriber still catching up can drain them. Await `wait()` on a runtime, or drop the handle once your own clock passes `linger()`. Dropping it early is the interrupt, and `close()` on another handle ends it at once. - **Publish only while someone is watching.** `demand()` on a track, group, or broadcast says whether a subscriber is attached, which is how capture and transcode skip work nobody asked for. Holding a broadcast or track consumer is not demand: a track consumer requests nothing until it queries, subscribes, or fetches, and those count until dropped. A shared fetch stays up until its last reader leaves. diff --git a/go/ffi/MIGRATE.md b/go/ffi/MIGRATE.md index 85200148d87..c0dd098ee1a 100644 --- a/go/ffi/MIGRATE.md +++ b/go/ffi/MIGRATE.md @@ -4,6 +4,14 @@ How to update your code for each breaking change, newest first. ## Unreleased +### A client asks only about what it watches (#6165) + +On moq-lite, a client session asks its relay for announcements only under paths something watches, through `Announced()` or `AnnouncedBroadcast()`, instead of under its whole scope at connect. +`MoqOriginConsumer.RequestBroadcast()` watches nothing, so on a client it fails with an unroutable error for a path nothing watches, even when the relay announces it. +Resolve the path with `AnnouncedBroadcast()`, or hold `Announced()` over its prefix while requesting. +moq-transport and moq-lite-03/04 sessions are unchanged. +Regenerate the bindings for the changed `request_broadcast` checksum. + ### Catalog references carry their epoch `MoqVideo` and `MoqAudio` gain `Epoch *string`, the publisher instance their `Broadcast` reference names. diff --git a/go/wrapper/MIGRATE.md b/go/wrapper/MIGRATE.md index a72b416c1a6..fda653cd3a4 100644 --- a/go/wrapper/MIGRATE.md +++ b/go/wrapper/MIGRATE.md @@ -4,6 +4,13 @@ How to update your code for each breaking change, newest first. ## Unreleased +### A client asks only about what it watches (#6165) + +On moq-lite, a client session asks its relay for announcements only under paths something watches, through `Announced()` or `AnnouncedBroadcast()`, instead of under its whole scope at connect. +`OriginConsumer.RequestBroadcast()` watches nothing, so on a client it fails with an unroutable error for a path nothing watches, even when the relay announces it. +Resolve the path with `AnnouncedBroadcast()`, or hold `Announced()` over its prefix while requesting. +moq-transport and moq-lite-03/04 sessions are unchanged. + ### `media.ContainerFormatFmp4` unwraps to Legacy (#6250) A container import with `media.ContainerFormatFmp4` now publishes one Legacy frame per sample instead of one CMAF frame per fragment. diff --git a/kt/moq-ffi/MIGRATE.md b/kt/moq-ffi/MIGRATE.md index f9c39e69de4..a87e06daea2 100644 --- a/kt/moq-ffi/MIGRATE.md +++ b/kt/moq-ffi/MIGRATE.md @@ -4,6 +4,14 @@ How to update your code for each breaking change, newest first. ## Unreleased +### A client asks only about what it watches (#6165) + +On moq-lite, a client session asks its relay for announcements only under paths something watches, through `announced()` or `announcedBroadcast()`, instead of under its whole scope at connect. +`MoqOriginConsumer.requestBroadcast()` watches nothing, so on a client it fails with an unroutable error for a path nothing watches, even when the relay announces it. +Resolve the path with `announcedBroadcast()`, or hold `announced()` over its prefix while requesting. +moq-transport and moq-lite-03/04 sessions are unchanged. +Regenerate the bindings for the changed `request_broadcast` checksum. + ### Catalog references carry their epoch `MoqVideo` and `MoqAudio` gain `epoch`, the publisher instance their `broadcast` reference names; it defaults to `null`, so only positional construction changes. diff --git a/kt/moq/MIGRATE.md b/kt/moq/MIGRATE.md index cdba9fdcbf2..0cfe339b860 100644 --- a/kt/moq/MIGRATE.md +++ b/kt/moq/MIGRATE.md @@ -4,6 +4,13 @@ How to update your code for each breaking change, newest first. ## Unreleased +### A client asks only about what it watches (#6165) + +On moq-lite, a client session asks its relay for announcements only under paths something watches, through `announced()` or `announcedBroadcast()`, instead of under its whole scope at connect. +`requestBroadcast()` watches nothing, so on a client it fails with an unroutable error for a path nothing watches, even when the relay announces it. +Resolve the path with `announcedBroadcast()`, or hold `announced()` over its prefix while requesting. +moq-transport and moq-lite-03/04 sessions are unchanged. + ### `ContainerFormat.FMP4` unwraps to Legacy (#6250) A container import with `ContainerFormat.FMP4` now publishes one Legacy frame per sample instead of one CMAF frame per fragment. diff --git a/py/moq-ffi/MIGRATE.md b/py/moq-ffi/MIGRATE.md index 13fb1158a89..3390d71fffd 100644 --- a/py/moq-ffi/MIGRATE.md +++ b/py/moq-ffi/MIGRATE.md @@ -4,6 +4,14 @@ How to update your code for each breaking change, newest first. ## Unreleased +### A client asks only about what it watches (#6165) + +On moq-lite, a client session asks its relay for announcements only under paths something watches, through `announced()` or `announced_broadcast()`, instead of under its whole scope at connect. +`MoqOriginConsumer.request_broadcast()` watches nothing, so on a client it fails with an unroutable error for a path nothing watches, even when the relay announces it. +Resolve the path with `announced_broadcast()`, or hold `announced()` over its prefix while requesting. +moq-transport and moq-lite-03/04 sessions are unchanged. +Regenerate the bindings for the changed `request_broadcast` checksum. + ### Catalog references carry their epoch `MoqVideo` and `MoqAudio` gain `epoch`, the publisher instance their `broadcast` reference names; it defaults to `None`, so only positional construction changes. diff --git a/py/moq-rs/MIGRATE.md b/py/moq-rs/MIGRATE.md index 47d7733b7bb..39798153e67 100644 --- a/py/moq-rs/MIGRATE.md +++ b/py/moq-rs/MIGRATE.md @@ -4,6 +4,13 @@ How to update your code for each breaking change, newest first. ## Unreleased +### A client asks only about what it watches (#6165) + +On moq-lite, a client session asks its relay for announcements only under paths something watches, through `announced()` or `announced_broadcast()`, instead of under its whole scope at connect. +`OriginConsumer.request_broadcast()` watches nothing, so on a client it fails with an unroutable error for a path nothing watches, even when the relay announces it. +Resolve the path with `announced_broadcast()`, or hold `announced()` over its prefix while requesting. +moq-transport and moq-lite-03/04 sessions are unchanged. + ### `ContainerFormat.FMP4` unwraps to Legacy (#6250) A container import with `ContainerFormat.FMP4` now publishes one Legacy frame per sample instead of one CMAF frame per fragment. diff --git a/quest/m1/README.md b/quest/m1/README.md index 9054c433d52..8d853307b57 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -81,7 +81,6 @@ lite-07 stream type here. - [Finalize moq-lite-07](/quest/m1/lite07-finalize.md) - when the maintainer cuts it, lite-07 negotiates as `moq-lite-07` and the next release ships it - [Moxygen minimal interop](/quest/m1/smoke-moxygen.md) - waiting on Moxygen to send a registered filter type, then the smoke lane stops hiding its failure - [More tests under load](/quest/m1/test-flakes-2/README.md) - the second round of load-only failures, one quest per flake, fixed at the cause -- [Lazy discovery](/quest/m1/lazy-discovery/README.md) - a `consume` session requests announcements only for prefixes the origin is watching, in JS then Rust; `follow`, `broadcasts`, and the `discovery` option go away - [A public model](/quest/m1/public-model/README.md) - every model item a built-in session reaches is public, unnecessary, or marked deliberately private, so a custom publisher, subscriber, or relay builds from the public model alone - [FETCH_OK whole group](/quest/m1/ietf-fetch-end-location.md) - on drafts 14 to 19, FETCH_OK End Object 0 means the whole group in both directions, and a FETCH ending at the maximum group is refused as unsupported - [Pause an IETF SUBSCRIBE](/quest/m1/ietf-subscribe-pause.md) - Forward=0 pauses a SUBSCRIBE without demand upstream and Forward=1 resumes it, instead of ending it diff --git a/quest/m1/lazy-discovery/README.md b/quest/m1/lazy-discovery/README.md deleted file mode 100644 index ebdc00f7849..00000000000 --- a/quest/m1/lazy-discovery/README.md +++ /dev/null @@ -1,49 +0,0 @@ -# Lazy discovery - -## Goal - -A session fed into a local origin (`consume`) asks its peer for -announcements only under the prefixes something on that origin is watching, -opening and closing ANNOUNCE_REQUEST / SUBSCRIBE_NAMESPACE as watchers come -and go and re-opening them on reconnect, in JS and the Rust client. An app -that watches a few topics never receives announcements for the rest of what -its token allows. Subscriptions are already lazy; this makes discovery match. - -Non-goals: no wire change, relay announce behavior is unchanged (a relay -watches the root, so it still discovers everything), and token claims still -cap what a peer returns. - -## Plan - -Decided 2026-10-10 in a `/quest-plan` interview (paper trail in the PR that -added this line). Motivated by a customer on `@moq/net` 0.4.2 who wants a -changing set of topics and today sets `discovery: false` (meant for relays -that cannot do discovery) to suppress the root ANNOUNCE_REQUEST, then opens -per-topic `session.announced(...)` streams by hand, bypassing the origin. - -Shared rules for the children: - -- Wire requests are a covering set: only the broadest watched prefixes go - out, and a narrower watch rides a broader request. When a broader watch - closes while a narrower one remains, the narrower request opens before the - broader one closes, so nothing retracts and re-announces. -- A pattern watch requests its literal head (`room/*/chat` watches `room/`; - `**` watches the root), the rule `interests()` uses today; the origin - filters locally. -- When the last watcher of a prefix leaves, its request lingers for a fixed - 2s before closing, matching `LINGER` in `js/net/src/connection/forward.ts` - and the Rust child's plan. This discovery linger is separate from the - configurable shared-connection linger; one-shot sessions have no such - option. No new option. -- A plain JS `request(path)` stays blind: it opens no announce request and - subscribes at once. Rust has no blind requests or answers (decided - 2026-10-20, recorded in [the Rust child](/quest/m1/lazy-discovery/rust.md)). -- Docs and examples that a change makes stale are updated in that change. - -End to end, once all children land: the customer's flow (one connection, -topics added and removed later) works through `origin` alone, with only -per-topic requests on the wire. - -## Required - -- [Rust lazy discovery](/quest/m1/lazy-discovery/rust.md) - the Rust client mirrors watch-driven discovery and the follow fold diff --git a/quest/m1/lazy-discovery/rust.md b/quest/m1/lazy-discovery/rust.md deleted file mode 100644 index 71791c22d58..00000000000 --- a/quest/m1/lazy-discovery/rust.md +++ /dev/null @@ -1,60 +0,0 @@ -# [L] Rust lazy discovery - -## Goal - -The Rust client session requests announcements only for prefixes its -`consume` origin is watching, matching the -[JS behavior](/doc/lib/js/net.md) and the -[line's shared rules](/quest/m1/lazy-discovery/README.md). Rust's `follow` -folds into its request path as in the -[JS read API](/doc/lib/js/net.md). Relays keep watching the -root, so relay behavior and cluster routing are unchanged. - -## Plan - -Decided 2026-10-10 in the [lazy discovery](/quest/m1/lazy-discovery/README.md) -interview: mirror JS, and settle the Rust API mapping in this quest's own -`/quest-plan` round before implementing. - -Facts (2026-10-10, `origin/main` e5446b036): - -- The lite subscriber builds one `AnnouncePrefix` per - `interest_prefixes(&origin.allowed())` at session start - (`rs/moq-net/src/lite/subscriber.rs:601`); the IETF subscriber has the - equivalent. -- `OriginConsumer::follow` exists (`rs/moq-net/src/model/origin.rs:4473`); - `origin::Consumer::announced_broadcast` is Rust's announced request. -- `origin::Consumer::announced_broadcast` registers no discovery interest - yet. This quest makes it register interest for its path, as JS does, since - the session stops asking for its whole scope. -- moq-relay reads `origin.consume().announced()` for the root in its web, - internal, and cluster paths, so a relay's watch set is the root. - -The JS shape to mirror (`js/net/src/connection/forward.ts`): the origin -counts watched literal heads in a trie and hands each session only the -covering-set changes a watch causes, so a change costs the prefixes it covers -or uncovers and stays flat over unrelated held watches -(`js/net/bench/forward.ts`); an announce request stays open while anything at -or beneath its prefix is watched, so a narrower watch rides it and never takes -its routes over (nothing tells when a fresh request has synced); a narrower -request already open when a broader watch arrives stays open until the broader -request goes; a route reported by overlapping requests is inserted once, -driven by the first reporter, and if it changes hands anyway (a request -failing) it restarts unless one epoch proves both saw one instance; closing an announce request ends its stream; a plain request -resolved through a discovered route watches its path until closed; a request -closes 2s after nothing beneath it is watched (a fixed constant, since one-shot -connections have no `linger`); a refused or reset request downgrades the -whole session to blind answers. - -Decided 2026-10-20 (maintainer): no blind answers in Rust. moq-net has no -blind requests, and `origin::Consumer::announced_broadcast` -already waits on an announced route rather than falling back, so a refused or -reset announce request does not downgrade a Rust session; the request fails -like any other. This is the one place Rust does not mirror JS. - -Open for the plan round: how the watch set crosses the poll-based session driver, and moq-ffi and binding -fallout (see the Cross-Package Sync table in `AGENTS.md`). - -Tests must pin the relay side: an origin in relay mode (cluster and upstream -sessions on the client subscriber path) still sends a root request, so -lazy discovery never narrows what a relay discovers. diff --git a/quest/m1/public-model/README.md b/quest/m1/public-model/README.md index 019c8064fbd..a1c82a05e45 100644 --- a/quest/m1/public-model/README.md +++ b/quest/m1/public-model/README.md @@ -44,8 +44,7 @@ Rules every child follows: on one naming a deleted quest. - Items replaced by another quest's new API wait for that quest instead of growing a public shape it deletes: fetch and per-group requests - ([Dynamic range requests](/quest/m1/subscribe-ranges/dynamic.md)), announce - interest ([Rust lazy discovery](/quest/m1/lazy-discovery/rust.md)). + ([Dynamic range requests](/quest/m1/subscribe-ranges/dynamic.md)). Inventory method, to rerun before each child starts (counts drift). Rust: replace every `pub(crate)` under `rs/moq-net/src/model/` with diff --git a/quest/m1/public-model/answer.md b/quest/m1/public-model/answer.md index c57e0f8ccb9..fdafa2a343f 100644 --- a/quest/m1/public-model/answer.md +++ b/quest/m1/public-model/answer.md @@ -75,7 +75,7 @@ Rust items (`rs/moq-net/src/model/`), as they stood at b4eab0db1: answer source give a peer's route a public shape, and move the out-of-crate tests and benches onto it. - `origin::interest_prefixes` (both subscribers): remove the need through - [Rust lazy discovery](/quest/m1/lazy-discovery/rust.md), which gives Rust + Rust lazy discovery ([moq-dev/moq#6165](https://github.com/moq-dev/moq/pull/6165)), which gives Rust the origin's watched interests the way JS has them; make that public in both languages there or here, whichever lands second. - `origin::Consumer::discovery`, `includes_hidden` (IETF publisher): fold @@ -115,11 +115,10 @@ JS items (`js/net/src/`): - **Blind answers** (the same loop, raised in review of #5717). Every JS session answers each request no route covers by subscribing on its peer: always, and as the only path after a peer refuses or resets discovery. - [Lazy discovery](/quest/m1/lazy-discovery/README.md) keeps a plain JS + Lazy discovery keeps a plain JS `request(path)` blind, so blind answering stays in JS and needs a public shape before the loop goes. Rust has no blind answers (decided 2026-10-20, - maintainer, recorded in [Rust lazy discovery](/quest/m1/lazy-discovery/rust.md)), - so no Rust session needs this shape. + maintainer), so no Rust session needs this shape. **Decision** (settled 2026-10-16, narrowed to JS 2026-10-20): - ✅ A fallback answerer, `Origin.Producer.fallback()` in JS. The name is still open: propose alternatives in this quest's PR before shipping it. It returns the same @@ -188,10 +187,6 @@ Tests: each public call gets a mock-time test in its language, and the built-in sessions move onto the public calls, so their existing suites cover the behavior. -## Required - -- [Rust lazy discovery](/quest/m1/lazy-discovery/rust.md) - settles the interest and discovery items before they become public - ## Related - [Public model after ranges, handover, and the JS tail](/quest/m1/public-model/deferred.md) - the track info and fetch answers, split out of this quest diff --git a/quest/m2/ietf-lazy-discovery.md b/quest/m2/ietf-lazy-discovery.md index 1d11d192fcc..e9a2cc2f401 100644 --- a/quest/m2/ietf-lazy-discovery.md +++ b/quest/m2/ietf-lazy-discovery.md @@ -22,11 +22,6 @@ unsolicited PUBLISH_NAMESPACE. Public API: none expected. Wire: no new messages. -## Required - -- [Lazy discovery in Rust](/quest/m1/lazy-discovery/rust.md) - the lite path this mirrors - ## Related -- [Lazy discovery](/quest/m1/lazy-discovery/README.md) - the moq-lite epic - [Keyed prefix polling](/quest/m2/keyed-prefix-polling.md) - the per-wake cost of many open prefixes diff --git a/quest/m2/keyed-prefix-polling.md b/quest/m2/keyed-prefix-polling.md index cb5a3f61455..5d04843514a 100644 --- a/quest/m2/keyed-prefix-polling.md +++ b/quest/m2/keyed-prefix-polling.md @@ -20,10 +20,6 @@ Decided 2026-10-21 with the maintainer: gated on a measurement. Public API: none. Wire: none. -## Required - -- [Lazy discovery in Rust](/quest/m1/lazy-discovery/rust.md) - lands the per-prefix requests this polls - ## Related - [IETF lazy discovery](/quest/m2/ietf-lazy-discovery.md) - more prefixes per session once IETF is lazy too diff --git a/rs/moq-c/MIGRATE.md b/rs/moq-c/MIGRATE.md index 9c8718d1616..6335f6c8e98 100644 --- a/rs/moq-c/MIGRATE.md +++ b/rs/moq-c/MIGRATE.md @@ -4,6 +4,13 @@ How to update your code for each breaking change, newest first. ## Unreleased +### A client asks only about what it watches (#6165) + +On moq-lite, a client session asks its relay for announcements only under paths something watches, through `moq_origin_announced` or `moq_origin_announced_broadcast`, instead of under its whole scope at connect. +`moq_origin_request` watches nothing, so on a client it fails with a negative code for a path nothing watches, even when the relay announces it. +Resolve the path with `moq_origin_announced_broadcast`, or hold `moq_origin_announced` over its prefix while requesting. +moq-transport and moq-lite-03/04 sessions are unchanged. + ### `MOQ_CONTAINER_FORMAT_FMP4` unwraps to Legacy (#6250) A container import with `MOQ_CONTAINER_FORMAT_FMP4` now publishes one Legacy frame per sample instead of one CMAF frame per fragment. diff --git a/rs/moq-c/src/api.rs b/rs/moq-c/src/api.rs index d01021e42fb..0ff9e6c2550 100644 --- a/rs/moq-c/src/api.rs +++ b/rs/moq-c/src/api.rs @@ -2100,7 +2100,9 @@ pub extern "C" fn moq_origin_announced_broadcast_cancel(task: u32) -> i32 { /// Resolves against what is announced *now*, where [moq_origin_announced_broadcast] waits /// indefinitely: it returns an announced broadcast at once, and fails when none is reachable, /// including a broadcast created but not announced. It does NOT wait for a later -/// announcement. Serve on-demand paths with [moq_origin_dynamic]. +/// announcement. Serve on-demand paths with [moq_origin_dynamic]. A moq-lite client session +/// only asks its peer about paths something watches, through [moq_origin_announced] or +/// [moq_origin_announced_broadcast], so a path nothing watches is unreachable on it. /// /// `on_broadcast` is invoked with a positive broadcast handle once served, then exactly once more /// with a terminal code: `0` (finished, including after [moq_origin_request_cancel]) or a negative diff --git a/rs/moq-c/src/test.rs b/rs/moq-c/src/test.rs index f3e9a10f215..828a2b2c1a7 100644 --- a/rs/moq-c/src/test.rs +++ b/rs/moq-c/src/test.rs @@ -5629,8 +5629,22 @@ fn server_accepts_a_session() { let bandwidth = id(moq_session_bandwidth(session)); assert_eq!(moq_bandwidth_close(bandwidth), 0); - // The broadcast the server publishes reaches the client. - let consumed = request_broadcast(received, b"demo"); + // The broadcast the server publishes reaches the client. A client only asks the + // server about what it watches, so wait on the path rather than request it once. + let path = b"demo"; + let announced_cb = Callback::new(); + let task = id(unsafe { + moq_origin_announced_broadcast( + received, + path.as_ptr() as *const c_char, + path.len(), + Some(channel_callback), + announced_cb.ptr, + ) + }); + let consumed = id(announced_cb.recv()); + assert_eq!(moq_origin_announced_broadcast_cancel(task), 0); + assert_eq!(announced_cb.recv_terminal(), 0); assert_eq!(moq_consume_close(consumed), 0); // Closing the accepted session is a clean terminal close. diff --git a/rs/moq-ffi/MIGRATE.md b/rs/moq-ffi/MIGRATE.md index d66b79717fd..124800378b7 100644 --- a/rs/moq-ffi/MIGRATE.md +++ b/rs/moq-ffi/MIGRATE.md @@ -5,6 +5,13 @@ These are the UniFFI names; each language wrapper has its own `MIGRATE.md` in it ## Unreleased +### A client asks only about what it watches (#6165) + +On moq-lite, a client session asks its relay for announcements only under paths something watches, through `announced` or `announced_broadcast`, instead of under its whole scope at connect. +`MoqOriginConsumer::request_broadcast` watches nothing, so on a client it fails with `MoqError::Unroutable` for a path nothing watches, even when the relay announces it. +Resolve the path with `announced_broadcast`, or hold `announced` over its prefix while requesting. +moq-transport and moq-lite-03/04 sessions are unchanged. + ### `MoqContainerFormat::Fmp4` unwraps to Legacy (#6250) A container import with `MoqContainerFormat::Fmp4` now publishes one Legacy frame per sample instead of one CMAF frame per fragment. diff --git a/rs/moq-ffi/src/origin.rs b/rs/moq-ffi/src/origin.rs index 9c27aee52ed..6bbc6af4af4 100644 --- a/rs/moq-ffi/src/origin.rs +++ b/rs/moq-ffi/src/origin.rs @@ -497,7 +497,9 @@ impl MoqOriginConsumer { /// cancel. /// /// Calling this straight after connecting therefore races the session's announcements - /// and can report a live broadcast as unroutable. Await `announced_broadcast` first. + /// and can report a live broadcast as unroutable. Await `announced_broadcast` first. A + /// moq-lite client only asks the relay about paths something watches, through `announced` + /// or `announced_broadcast`, so a path nothing watches stays unroutable. pub async fn request_broadcast(&self, path: String) -> Result, MoqError> { let broadcast = self .driver diff --git a/rs/moq-hls/src/server/mod.rs b/rs/moq-hls/src/server/mod.rs index edba91ba14e..e9ad5b40a22 100644 --- a/rs/moq-hls/src/server/mod.rs +++ b/rs/moq-hls/src/server/mod.rs @@ -76,16 +76,22 @@ pub struct Server { struct Inner { origin: moq_net::origin::Consumer, + /// Watches the whole scope while the server lives, so a lazily discovering client + /// session asks its peer once rather than once per requested path, which an HTTP + /// client could otherwise name without bound. + _watch: moq_net::announce::Consumer, config: Config, broadcasters: Mutex>>, } impl Server { - /// Build a server reading broadcasts from `origin`. Every request is served; - /// gate access by layering middleware onto [`router`](Self::router). + /// Build a server reading broadcasts from `origin`, which it watches whole while it + /// lives. Every request is served; gate access by layering middleware onto + /// [`router`](Self::router). pub fn new(origin: moq_net::origin::Consumer, config: Config) -> Self { Self { inner: Arc::new(Inner { + _watch: origin.clone().with_hidden(true).announced(), origin, config, broadcasters: Mutex::new(HashMap::new()), diff --git a/rs/moq-hls/src/server/routes.rs b/rs/moq-hls/src/server/routes.rs index 0fb30596c17..08b28075ccd 100644 --- a/rs/moq-hls/src/server/routes.rs +++ b/rs/moq-hls/src/server/routes.rs @@ -1048,6 +1048,71 @@ mod tests { pair.accept.abort(); } + /// The server watches its whole scope, so a lookup rides one request to the peer + /// rather than opening its own for every path an HTTP client names. + #[tokio::test] + async fn lookups_ride_the_servers_scope_watch() { + let pair = lite_pair().await; + let other = pair.pub_origin.create_broadcast("room/other").expect("other"); + other.announce(Default::default()).expect("announce other"); + + let server = Server::new(pair.sub_origin.consume(), crate::export::Config::default()); + let missing = tokio::time::timeout(TIMEOUT, server.broadcaster("room/missing")) + .await + .expect("lookup timed out"); + assert!(missing.is_none()); + + // Nothing watches `room/other` on its own, yet the synced lookup already discovered it. + pair.sub_origin + .consume() + .request_broadcast("room/other", None) + .await + .expect("discovered through the scope watch"); + + drop(other); + pair.accept.abort(); + } + + /// A client asks its peer only about watched paths, so a sibling no listing covers is + /// asked about before it is requested, on every lite version, including those whose + /// announce stream marks no initial set. + #[tokio::test] + async fn a_sibling_resolves_through_a_client_session() { + for version in [ + "moq-lite-01", + "moq-lite-02", + "moq-lite-03", + "moq-lite-04", + "moq-lite-05", + "moq-lite-06", + "moq-lite-07-wip", + ] { + let pair = lite_pair_on(version, moq_net::origin::Config::default()).await; + let catalog = pair.pub_origin.create_broadcast("room/live").expect("catalog"); + catalog.announce(Default::default()).expect("announce catalog"); + let media = pair.pub_origin.create_broadcast("room/source").expect("media"); + media.announce(Default::default()).expect("announce media"); + + let origin = pair.sub_origin.consume(); + let broadcast = tokio::time::timeout(TIMEOUT, origin.routed_broadcast("room/live")) + .await + .expect("catalog resolve timed out") + .expect("catalog broadcast"); + let source = moq_mux::Source::new(origin, broadcast); + let sibling = tokio::time::timeout( + TIMEOUT, + source.resolve(Some(&moq_net::path::Relative::new("./source")), None), + ) + .await + .expect("sibling resolve timed out") + .unwrap_or_else(|err| panic!("{version}: sibling broadcast: {err}")); + assert!(!sibling.is_closed(), "{version}"); + + drop((catalog, media)); + pair.accept.abort(); + } + } + #[tokio::test] async fn a_disconnected_sibling_publisher_answers_404() { let pair = lite_pair().await; @@ -1077,13 +1142,11 @@ mod tests { let app = Server::new(pair.sub_origin.consume(), crate::export::Config::default()).router(); wait_listed(&app, "/room/live/video/video0/media.m3u8", "seg/656e3d1b.0.m4s").await; - let remote_media = tokio::time::timeout( - TIMEOUT, - pair.sub_origin.consume().request_broadcast("room/source", None), - ) - .await - .expect("sibling resolve timed out") - .expect("sibling broadcast"); + // Waited on rather than requested once: a client only asks about what it watches. + let remote_media = tokio::time::timeout(TIMEOUT, pair.sub_origin.consume().routed_broadcast("room/source")) + .await + .expect("sibling resolve timed out") + .expect("sibling broadcast"); // Only the sibling goes: the catalog keeps listing the rendition, so the request // reaches the media fetch rather than missing the rendition. media_broadcast.close(); diff --git a/rs/moq-mux/src/container/test_util.rs b/rs/moq-mux/src/container/test_util.rs index b69610165dc..f0bb717a357 100644 --- a/rs/moq-mux/src/container/test_util.rs +++ b/rs/moq-mux/src/container/test_util.rs @@ -248,6 +248,7 @@ impl Link { let server_transport = Gated { inner: server_transport, gate: gate.clone(), + answers: true, }; let client = moq_net::Client::new() .with_versions(version.into()) @@ -280,6 +281,51 @@ impl Link { } } + /// A lazily discovering client session subscribing into `to` from a server publishing + /// `from`. A server that never `answers` accepts no request stream, as a stalled peer. + pub(crate) async fn lazy(from: &moq_net::origin::Producer, to: &moq_net::origin::Producer, answers: bool) -> Self { + use moq_net::transport::memory; + + let version: moq_net::Version = "moq-lite-07-wip".parse().unwrap(); + let mut config = memory::Config::default(); + config.protocol = Some(version.alpn().to_string()); + let (client_transport, server_transport) = memory::pair(config); + let gate = Gate(std::sync::Arc::new(std::sync::Mutex::new((usize::MAX, None)))); + let server_transport = Gated { + inner: server_transport, + gate: gate.clone(), + answers, + }; + let client = moq_net::Client::new() + .with_versions(version.into()) + .with_subscriber(to.clone()); + let server = moq_net::Server::new() + .with_versions(version.into()) + .with_publisher(from.consume()); + let sessions = tokio::join!( + async { + let (session, driver) = client + .connect_lite(moq_net::time::now(), client_transport) + .await + .unwrap(); + tokio::spawn(moq_net::time::run(driver)); + session + }, + async { + let (session, driver) = server + .accept_lite(moq_net::time::now(), server_transport) + .await + .unwrap(); + tokio::spawn(moq_net::time::run(driver)); + session + } + ); + Self { + _sessions: sessions, + gate, + } + } + /// Accept every stream from now on, the held ones first. pub(crate) fn open(&self) { let mut gate = self.gate.0.lock().unwrap(); @@ -299,6 +345,8 @@ struct Gate(std::sync::Arc)>>) struct Gated { inner: moq_net::transport::memory::Session, gate: Gate, + /// Whether it accepts the peer's request (bidi) streams at all. + answers: bool, } impl moq_net::transport::poll::Session for Gated { @@ -327,6 +375,9 @@ impl moq_net::transport::poll::Session for Gated { &mut self, cx: &mut std::task::Context<'_>, ) -> std::task::Poll, Self::Error>> { + if !self.answers { + return std::task::Poll::Pending; + } self.inner.poll_accept_bi(cx) } diff --git a/rs/moq-mux/src/source.rs b/rs/moq-mux/src/source.rs index 565911b29bf..c64a28c44d0 100644 --- a/rs/moq-mux/src/source.rs +++ b/rs/moq-mux/src/source.rs @@ -141,7 +141,7 @@ impl Source { // Unpinned even with an epoch: an epochless route still answers, best effort, and // the read refuses a route that proves another instance. None => Bound::Requested { - request: self.origin.request_broadcast(path, None).into_inner(), + lookup: Box::new(Mutex::new(Lookup::new(&self.origin, path))), pins: self.pins.clone(), path: path.clone(), }, @@ -248,7 +248,9 @@ impl Source { /// Start one broadcast request now and retain its result for repeated reads. /// /// Unlike [`resolve`](Self::resolve), this issues the request without awaiting or polling. - /// A remote or dynamic handler may answer later. The binding never retries a failed request + /// A remote or dynamic handler may answer later. A client session that only discovers + /// what is watched is asked about the path first, and the request goes out once it + /// answers. The binding never retries a failed request /// or resolves the path again after a publisher is replaced. /// /// A path this source already resolved binds to that broadcast; otherwise the origin is @@ -306,9 +308,9 @@ pub struct Binding { enum Bound { /// A broadcast the caller already holds. Ready(moq_net::broadcast::Consumer), - /// A request issued when the binding was made. + /// A lookup started when the binding was made. Requested { - request: moq_net::origin::Requesting, + lookup: Box>, /// Where the answer is pinned, the first for its path winning. pins: Pins, path: moq_net::PathOwned, @@ -330,8 +332,8 @@ impl Binding { pub fn poll_broadcast(&self, waiter: &kio::Waiter) -> Poll> { let broadcast = match &self.bound { Bound::Ready(broadcast) => broadcast.clone(), - Bound::Requested { request, pins, path } => { - let broadcast = ready!(request.poll_ok(waiter))?; + Bound::Requested { lookup, pins, path } => { + let broadcast = ready!(lookup.lock().expect("lookup poisoned").poll(waiter))?; // A request that raced another for the path takes the one pinned first. The path's // first answer is pinned even when it fails this binding's epoch, so no later // reference resolves the path independently. @@ -361,6 +363,76 @@ impl Binding { } } +/// One request for a path, made once the origin has heard whether anything serves it. +/// +/// A client session only asks its peer about paths something watches, so a sibling no +/// listing covers would be unroutable when asked at once. Following the path first and +/// asking once a route serves it, or once every session has synced without one, answers +/// from what the peers announce. An origin that already knows its routes answers at once, +/// so the request still goes out on binding. +struct Lookup { + origin: moq_net::origin::Consumer, + path: moq_net::PathOwned, + /// Dropped once the request settles: a broadcast resolved through a discovered route + /// keeps its own path watched. + watch: Option, + request: Option, +} + +impl Lookup { + fn new(origin: &moq_net::origin::Consumer, path: &moq_net::PathOwned) -> Self { + // A path no pattern can spell, or one outside the scope, fails on the request instead. + let watch = origin.follow(path).ok(); + let mut lookup = Self { + origin: origin.clone(), + path: path.clone(), + watch, + request: None, + }; + // Ask now when the origin already knows, so a dynamic handler hears it on binding. + let _ = lookup.poll(&kio::Waiter::noop()); + lookup + } + + fn poll(&mut self, waiter: &kio::Waiter) -> Poll> { + loop { + if self.request.is_none() { + if let Some(watch) = &mut self.watch { + // The follower reduces the covering routes to the one serving the path, so its + // latest event is whether a route serves it now. Ask as soon as one does: + // waiting for every session to sync would let one that never answers hold the + // lookup up. Without a route, only sync concludes that none will. + let mut routed = false; + loop { + match watch.poll_next(waiter) { + Poll::Ready(Some(moq_net::announce::Event::End(_))) => routed = false, + Poll::Ready(Some(_)) => routed = true, + Poll::Ready(None) | Poll::Pending => break, + } + } + if !routed { + ready!(watch.poll_synced(waiter)); + } + } + self.request = Some(self.origin.request_broadcast(&self.path, None).into_inner()); + } + let result = ready!(self.request.as_ref().expect("requested above").poll_ok(waiter)); + // The route can retract between the follower reporting it and the request reading + // the table. Until every session has synced, keep following for another. + let unsynced = self + .watch + .as_ref() + .is_some_and(|watch| watch.poll_synced(&kio::Waiter::noop()).is_pending()); + if matches!(result, Err(moq_net::Error::Unroutable)) && unsynced { + self.request = None; + continue; + } + self.watch = None; + return Poll::Ready(result); + } + } +} + /// Test helper: build an origin producer, spawning its driver on the ambient runtime. #[cfg(test)] pub(crate) fn produce_origin() -> moq_net::origin::Producer { @@ -490,6 +562,46 @@ mod tests { ); } + /// A sibling resolves through the session that announces it, even while another lazily + /// discovering session on the origin never answers its announce request. + #[tokio::test(start_paused = true)] + async fn a_stalled_session_does_not_hold_up_a_sibling() { + use crate::container::test_util::Link; + + let healthy = produce_origin(); + let _catalog = healthy.publish("a/pub", Default::default()).unwrap(); + let _sibling = healthy.publish("a/source", Default::default()).unwrap(); + let stalled = produce_origin(); + let subscriber = produce_origin(); + let _healthy = Link::lazy(&healthy, &subscriber, true).await; + let _stalled = Link::lazy(&stalled, &subscriber, false).await; + + let timeout = std::time::Duration::from_secs(5); + let consumer = subscriber.consume(); + let catalog = tokio::time::timeout(timeout, consumer.routed_broadcast("a/pub")) + .await + .expect("the catalog resolves") + .unwrap(); + let source = Source::new(consumer, catalog); + let sibling = tokio::time::timeout(timeout, source.resolve(Some(&Relative::new("./source")), None)) + .await + .expect("the sibling resolves without the stalled session") + .unwrap(); + assert!(!sibling.is_closed()); + + // Nothing serves a later sibling yet and the stalled session never syncs, so its + // lookup keeps waiting rather than failing, and resolves once it is published. + let binding = source.bind(Some(&Relative::new("./late")), None).unwrap(); + settle().await; + assert!(binding.poll_broadcast(&kio::Waiter::noop()).is_pending()); + let _late = healthy.publish("a/late", Default::default()).unwrap(); + let late = tokio::time::timeout(timeout, binding.broadcast()) + .await + .expect("the later sibling resolves") + .unwrap(); + assert!(!late.is_closed()); + } + /// A sibling reference stays on the broadcast it first resolved too. #[tokio::test(start_paused = true)] async fn a_source_keeps_the_sibling_it_resolved() { diff --git a/rs/moq-net/Cargo.toml b/rs/moq-net/Cargo.toml index ab0875da5fb..ae52af6e34b 100644 --- a/rs/moq-net/Cargo.toml +++ b/rs/moq-net/Cargo.toml @@ -105,6 +105,12 @@ name = "synced" harness = false required-features = ["fuzz"] +# Reaches the crate-private session interests through the hidden `fuzz` module. +[[bench]] +name = "interests" +harness = false +required-features = ["fuzz"] + # Reaches the private message codecs through the hidden `fuzz` module. [[bench]] name = "codec" diff --git a/rs/moq-net/MIGRATE.md b/rs/moq-net/MIGRATE.md index 332ae29e923..1653e37cc01 100644 --- a/rs/moq-net/MIGRATE.md +++ b/rs/moq-net/MIGRATE.md @@ -4,6 +4,17 @@ How to update your code for each breaking change, newest first. ## Unreleased +### A client session asks only about what its readers watch (#6165) + +On moq-lite, a client session no longer asks its peer for announcements under its whole subscribe scope at connect. +It asks only under the prefixes its subscribe origin's readers watch, with `announced()`, `follow`, `routed`, `routed_broadcast`, or `announced_broadcast`, and stops asking 2s after nothing at or beneath a prefix is watched. +`request_broadcast` watches nothing, so on a client it fails with `Error::Unroutable` for a path no reader watches, even when the peer announces it. +Resolve a path with `routed_broadcast` or `announced_broadcast`, or hold an `announced()` stream over its prefix while requesting, the way `routed` already did after connecting. +A broadcast resolved through a discovered route keeps its path watched until dropped. +A server session, and a client fed into a `Producer::peer` or `Producer::upstream` handle, still asks for everything it may, so a relay discovers as before. +Serving another session's request is not a watch, so a client that relays what it discovers to its own sessions should feed its client session `origin.upstream()` (or `peer()`) instead of a plain handle. +moq-transport sessions, and moq-lite-03/04 sessions (whose announce streams mark no initial set), are unchanged. + ### Frames carry their arrival time `frame::Frame` gains a private arrival stamp, read with `Frame::arrived()`, so it can no longer be built with a struct literal. diff --git a/rs/moq-net/benches/interests.rs b/rs/moq-net/benches/interests.rs new file mode 100644 index 00000000000..505aecf666e --- /dev/null +++ b/rs/moq-net/benches/interests.rs @@ -0,0 +1,95 @@ +//! What it costs an origin to tell its client sessions which prefixes to ask about. +//! +//! Every announcement stream on the origin counts its prefix as wanted, and each client +//! session fed into the origin hears the changes to the broadest wanted prefixes. Both +//! shapes are swept over the unrelated watches held elsewhere and the sessions +//! listening, so a cost that grows with the held watches rather than the touched path +//! shows up as a slope. Hearing one change costs each session its own entry, so the +//! sessions axis is expected to grow; the watches axis is not. +//! +//! - uncovered: a watch on a fresh prefix comes and goes, which every session hears. +//! - covered: a watch beneath one already held comes and goes, which no session hears. +//! +//! The session side is crate-private, so this reaches it through the hidden `fuzz` module. +//! +//! Run with `cargo bench -p moq-net --features fuzz --bench interests`. + +use criterion::{BenchmarkId, Criterion, criterion_group, criterion_main}; +use moq_net::{Hop, Pattern, Patterns, announce, fuzz, origin}; + +/// `(watches, sessions)` shapes. +const SHAPES: [(usize, usize); 4] = [(10, 10), (1_000, 10), (10, 1_000), (1_000, 1_000)]; + +/// An origin with `watches` announcement streams on prefixes of their own under +/// `other/`, one on `room/`, and `sessions` client sessions listening. The handles +/// are held for the whole run. +struct Table { + producer: origin::Producer, + _driver: origin::Driver, + _watches: Vec, + sessions: Vec, +} + +fn table(watches: usize, sessions: usize) -> Table { + let (producer, driver) = origin::Producer::new(origin::Config::new(Hop::new(1).unwrap())); + let mut held: Vec<_> = (0..watches) + .map(|i| scoped(&producer, &format!("other/{i}")).announced()) + .collect(); + held.push(scoped(&producer, "room").announced()); + let sessions: Vec<_> = (0..sessions).map(|_| fuzz::interests(&producer)).collect(); + for session in &sessions { + session.drain(); + } + Table { + producer, + _driver: driver, + _watches: held, + sessions, + } +} + +fn scoped(producer: &origin::Producer, prefix: &str) -> origin::Consumer { + producer + .consume() + .scope(prefix, &Patterns::from(Pattern::all())) + .unwrap() +} + +fn bench_uncovered(c: &mut Criterion) { + let mut group = c.benchmark_group("interests/uncovered"); + for (watches, sessions) in SHAPES { + let id = BenchmarkId::from_parameter(format!("{watches}w_{sessions}s")); + group.bench_function(id, |b| { + let table = table(watches, sessions); + let incoming = scoped(&table.producer, "incoming"); + b.iter(|| { + drop(incoming.announced()); + for session in &table.sessions { + assert_eq!(session.drain(), 2); + } + }); + }); + } + group.finish(); +} + +fn bench_covered(c: &mut Criterion) { + let mut group = c.benchmark_group("interests/covered"); + for (watches, sessions) in SHAPES { + let id = BenchmarkId::from_parameter(format!("{watches}w_{sessions}s")); + group.bench_function(id, |b| { + let table = table(watches, sessions); + let incoming = scoped(&table.producer, "room/incoming"); + b.iter(|| { + drop(incoming.announced()); + }); + for session in &table.sessions { + assert_eq!(session.drain(), 0); + } + }); + } + group.finish(); +} + +criterion_group!(benches, bench_uncovered, bench_covered); +criterion_main!(benches); diff --git a/rs/moq-net/src/discovery.rs b/rs/moq-net/src/discovery.rs new file mode 100644 index 00000000000..7fa8163c41b --- /dev/null +++ b/rs/moq-net/src/discovery.rs @@ -0,0 +1,276 @@ +//! Which prefixes a session asks its peer to announce. +//! +//! A client session fed into an origin asks only about the prefixes the origin's +//! readers want discovered, as [`origin::Producer`] reports them, and stops asking +//! [`LINGER`] after nothing at or beneath a prefix is wanted. A relay or a server asks +//! about everything its scope may reach, once, for the session's lifetime. + +use std::{ + collections::{BTreeSet, HashMap, HashSet, VecDeque}, + task::{Poll, ready}, + time::Duration, +}; + +use crate::{ + Path, PathOwned, + model::{InterestChange, Interests, Snapshot}, + origin, + time::{Deadline, Deadlines, Instant}, +}; + +/// How long an announce request outlives the last reader at or beneath its prefix, +/// so a reader torn down and rebuilt reuses it. +pub(crate) const LINGER: Duration = Duration::from_secs(2); + +/// What the session does with its announce requests next. +pub(crate) enum Change { + /// Ask the peer about the prefix, holding the snapshot until the request's + /// initial set is in. + Open(PathOwned, Snapshot), + /// Stop asking about the prefix: end its request, retracting what it reported. + Close(PathOwned), +} + +/// The announce requests a session keeps open, as a covering set of what is wanted. +/// +/// A request stays open while anything at or beneath its prefix is wanted, so a +/// narrower want rides it rather than opening a request that would later have to take +/// its routes over. A narrower request already open when a broader one opens, wanted or +/// lingering, stays open until the broader one goes, so neither takes the other's routes +/// over before it has synced. Nothing here ever reopens a request that ended on its own: the next want +/// that nothing covers does. +pub(crate) struct Discovery { + /// `None` for a session that asks about everything once, or once the origin closed. + interests: Option, + runtime: Deadlines, + /// Decided but not yet handed to the session, in order. + changes: VecDeque, + /// How many wants name each prefix: overlapping scopes can name one twice. + wanted: HashMap, + /// How many wanted prefixes sit at or beneath each prefix, so a request learns in a + /// walk up its path whether anything beneath it is still wanted. + beneath: HashMap, + /// The requests the session holds open, by prefix. + requests: HashMap, + /// The requests nothing beneath wants any more, by when they close. + lingering: BTreeSet<(Instant, PathOwned)>, + /// Fires for the first of `lingering`. + timer: Deadline, +} + +#[derive(Default)] +struct Request { + /// When it closes, while nothing at or beneath its prefix is wanted. + linger: Option, + /// Narrower requests nothing wants any more, kept open until this one goes. + held: HashSet, +} + +impl Discovery { + /// Decide for a session feeding `origin`: a client on a local link asks only about + /// what is wanted, while a server, or any session on a link between relays, asks + /// about everything it may, so discovery never narrows what a relay learns. + pub fn new(origin: &origin::Producer, client: bool, runtime: Deadlines) -> Self { + let mut discovery = Self { + interests: None, + timer: Deadline::new(&runtime), + runtime, + changes: VecDeque::new(), + wanted: HashMap::new(), + beneath: HashMap::new(), + requests: HashMap::new(), + lingering: BTreeSet::new(), + }; + match client && !origin.relays() { + true => discovery.interests = Some(origin.interests()), + // The wire speaks announce interest by prefix: ask for each granted pattern's + // literal head and let the origin's scope filter what arrives. Each snapshot is + // taken now, so a reader never reads as synced before the request goes out. + false => { + for prefix in crate::model::interest_prefixes(&origin.allowed()) { + let snapshot = origin.snapshot(&prefix); + discovery.requests.insert(prefix.clone(), Request::default()); + discovery.changes.push_back(Change::Open(prefix, snapshot)); + } + } + } + discovery + } + + /// The next change to apply, registering `waiter` for the next one otherwise. + pub fn poll(&mut self, waiter: &kio::Waiter) -> Poll { + loop { + if let Some(change) = self.changes.pop_front() { + return Poll::Ready(change); + } + if let Some(interests) = &self.interests { + match interests.poll_next(waiter) { + Poll::Ready(Some(change)) => { + self.interest(change); + continue; + } + // The origin closed: its readers want nothing more. + Poll::Ready(None) => { + self.interests = None; + continue; + } + Poll::Pending => {} + } + } + ready!(self.timer.poll(waiter)); + let now = self.runtime.now(); + while let Some((at, prefix)) = self.lingering.first().cloned() + && at <= now + { + self.lingering.pop_first(); + self.expire(prefix); + } + self.timer.set(self.lingering.first().map(|(at, _)| *at)); + } + } + + /// The request at `prefix` ended on its own: the peer finished it. + pub fn ended(&mut self, prefix: &PathOwned) { + if let Some(request) = self.requests.remove(prefix) { + self.forget(prefix, request); + } + } + + fn interest(&mut self, change: InterestChange) { + let InterestChange { + prefix, + wanted: want, + snapshot, + } = change; + + let count = self.wanted.get(&prefix).copied().unwrap_or(0); + let count = match want { + true => count + 1, + false => count + .checked_sub(1) + .expect("an interest released more often than gained"), + }; + match count { + 0 => self.wanted.remove(&prefix), + count => self.wanted.insert(prefix.clone(), count), + }; + // Another want already named the prefix, or one still does. + if count != usize::from(want) { + return; + } + + for above in covering(&prefix) { + let held = self.beneath.get(&above).copied().unwrap_or(0); + match want { + true => self.beneath.insert(above, held + 1), + false if held > 1 => self.beneath.insert(above, held - 1), + false => self.beneath.remove(&above), + }; + } + + let mut covered = false; + for above in covering(&prefix) { + if !self.requests.contains_key(&above) { + continue; + } + covered = true; + match self.beneath.contains_key(&above) { + true => self.stay(&above), + false => self.release(above), + } + } + + if want && !covered { + let snapshot = snapshot.expect("a wanted prefix comes with its snapshot"); + let request = self.hold_lingering(&prefix); + self.requests.insert(prefix.clone(), request); + self.changes.push_back(Change::Open(prefix, snapshot)); + } + } + + /// A new request at `prefix` holds the narrower ones still lingering beneath it, as it + /// would had they been released after it opened, so none closes before it has synced. + fn hold_lingering(&mut self, prefix: &PathOwned) -> Request { + let mut request = Request::default(); + let beneath: Vec<_> = self + .lingering + .iter() + .filter(|(_, other)| other.has_prefix(prefix)) + .cloned() + .collect(); + for (at, other) in beneath { + self.lingering.remove(&(at, other.clone())); + if let Some(narrower) = self.requests.get_mut(&other) { + narrower.linger = None; + } + request.held.insert(other); + } + self.timer.set(self.lingering.first().map(|(at, _)| *at)); + request + } + + /// Keep the request at `prefix` open: something beneath it is wanted again. + fn stay(&mut self, prefix: &PathOwned) { + let Some(request) = self.requests.get_mut(prefix) else { + return; + }; + if let Some(at) = request.linger.take() { + self.lingering.remove(&(at, prefix.clone())); + self.timer.set(self.lingering.first().map(|(at, _)| *at)); + } + } + + /// Settle a request nothing beneath it wants any more. A broader open request that + /// is still wanted holds it until that one goes; otherwise it lingers out. + fn release(&mut self, prefix: PathOwned) { + for above in covering(&prefix).into_iter().skip(1) { + if !self.beneath.contains_key(&above) { + continue; + } + if let Some(holder) = self.requests.get_mut(&above) { + holder.held.insert(prefix); + return; + } + } + let Some(request) = self.requests.get_mut(&prefix) else { + return; + }; + if request.linger.is_some() { + return; + } + let at = self.runtime.now() + LINGER; + request.linger = Some(at); + self.lingering.insert((at, prefix)); + self.timer.set(self.lingering.first().map(|(at, _)| *at)); + } + + /// Close a request that lingered out. + fn expire(&mut self, prefix: PathOwned) { + let Some(request) = self.requests.remove(&prefix) else { + return; + }; + self.forget(&prefix, request); + self.changes.push_back(Change::Close(prefix)); + } + + /// What a request that went held now settles on its own, unless it is wanted again. + fn forget(&mut self, prefix: &PathOwned, request: Request) { + if let Some(at) = request.linger { + self.lingering.remove(&(at, prefix.to_owned())); + } + for held in request.held { + if self.requests.contains_key(&held) && !self.beneath.contains_key(&held) { + self.release(held); + } + } + } +} + +/// `path` and every prefix above it, up to the root, narrowest first. +fn covering(path: &Path) -> Vec { + std::iter::successors(Some(path.as_str()), |path| { + (!path.is_empty()).then(|| path.rfind('/').map_or("", |slash| &path[..slash])) + }) + .map(|path| Path::new(path).to_owned()) + .collect() +} diff --git a/rs/moq-net/src/fuzz.rs b/rs/moq-net/src/fuzz.rs index bee20a0edef..60fc8c4a6cc 100644 --- a/rs/moq-net/src/fuzz.rs +++ b/rs/moq-net/src/fuzz.rs @@ -313,6 +313,26 @@ pub fn snapshot(origin: &crate::origin::Producer, prefix: &str) -> Snapshot { Snapshot(origin.snapshot(prefix)) } +/// What a client session fed into an origin hears it should ask its peer about, for +/// the interests bench. +pub struct Interests(crate::model::Interests); + +/// Register as a client session fed into `origin` does. +pub fn interests(origin: &crate::origin::Producer) -> Interests { + Interests(origin.interests()) +} + +impl Interests { + /// Take every change heard so far, returning how many there were. + pub fn drain(&self) -> usize { + let mut changes = 0; + while let std::task::Poll::Ready(Some(_)) = self.0.poll_next(&kio::Waiter::noop()) { + changes += 1; + } + changes + } +} + /// Add `n` publisher-side payload bytes on `(path, tier)`, creating the entry /// when the path is tracked. The stats producer bench marks a share of held /// paths changed this way, without a media frame. A later [`crate::stats::Registry::report`] diff --git a/rs/moq-net/src/ietf/publisher.rs b/rs/moq-net/src/ietf/publisher.rs index 8e3ce155bca..75d2ae64de3 100644 --- a/rs/moq-net/src/ietf/publisher.rs +++ b/rs/moq-net/src/ietf/publisher.rs @@ -765,7 +765,8 @@ where runtime, session, self_origin: origin.hop(), - origin, + // Serving the peer is not a local reader asking to discover anything. + origin: origin.serving(), control, peer_hop, peer_setup, diff --git a/rs/moq-net/src/ietf/session.rs b/rs/moq-net/src/ietf/session.rs index 962b792bd9d..579cf6025b8 100644 --- a/rs/moq-net/src/ietf/session.rs +++ b/rs/moq-net/src/ietf/session.rs @@ -1497,7 +1497,8 @@ async fn enforce_grant( mut session: S, ) -> Result<(), Error> { let origin = publisher.serving_origin().await; - let mut announced = origin.announced(); + // Checking what we publish is not a local reader asking to discover anything. + let mut announced = origin.clone().serving().announced(); let mut check = crate::auth::Enforce::default(); let Some(path) = kio::wait(|waiter| check.poll(&auth, &mut announced, waiter)).await else { return Ok(()); diff --git a/rs/moq-net/src/lib.rs b/rs/moq-net/src/lib.rs index b1a2cd5b38e..b7388952407 100644 --- a/rs/moq-net/src/lib.rs +++ b/rs/moq-net/src/lib.rs @@ -87,6 +87,8 @@ pub mod auth; mod client; mod coding; #[cfg(feature = "async")] +mod discovery; +#[cfg(feature = "async")] mod driver; mod epoch; mod error; diff --git a/rs/moq-net/src/lite/publisher.rs b/rs/moq-net/src/lite/publisher.rs index 773fc365331..5278ac62f09 100644 --- a/rs/moq-net/src/lite/publisher.rs +++ b/rs/moq-net/src/lite/publisher.rs @@ -194,7 +194,8 @@ impl Publisher { client: config.client, withdrawal: Default::default(), session: config.session, - origin: config.origin, + // Serving the peer is not a local reader asking to discover anything. + origin: config.origin.serving(), self_origin, peer_setup: config.peer_setup, peer_hop: config.peer_hop, diff --git a/rs/moq-net/src/lite/session.rs b/rs/moq-net/src/lite/session.rs index 5fa6494167b..a9852496dd0 100644 --- a/rs/moq-net/src/lite/session.rs +++ b/rs/moq-net/src/lite/session.rs @@ -346,7 +346,7 @@ where session_stream: setup_stream, ending: None, publisher, - subscriber: SubscriberDriver::new(subscriber, early), + subscriber: SubscriberDriver::new(subscriber, early, client), session, }; diff --git a/rs/moq-net/src/lite/subscriber.rs b/rs/moq-net/src/lite/subscriber.rs index 2f0a7262164..43305166cf3 100644 --- a/rs/moq-net/src/lite/subscriber.rs +++ b/rs/moq-net/src/lite/subscriber.rs @@ -598,9 +598,11 @@ pub(super) struct SubscriberDriver { subscriber: Subscriber, /// Aborts whatever is still subscribed when the session ends. cleanup: SubscriptionCleanup, - /// One machine per permitted prefix. Only an error ends the session; a + /// Which prefixes to ask the peer about. + discovery: crate::discovery::Discovery, + /// One machine per prefix asked about. Only an error ends the session; a /// prefix finishing cleanly (publisher FIN) just retires. - prefixes: Vec>, + prefixes: HashMap>, uni: UniAccept, /// PROBE feedback; finishes quietly when unsupported or given up on. bandwidth: Option>, @@ -611,16 +613,13 @@ pub(super) struct SubscriberDriver { } impl SubscriberDriver { - pub fn new(subscriber: Subscriber, early: Vec>) -> Self { - // The wire speaks announce interest by prefix: ask for each granted - // pattern's literal head and let the origin's scope filter what arrives. - let prefixes = crate::model::interest_prefixes(&subscriber.origin.allowed()) - .into_iter() - .map(|prefix| AnnouncePrefix::new(subscriber.clone(), prefix)) - .collect(); - + pub fn new(subscriber: Subscriber, early: Vec>, client: bool) -> Self { + // Lite03/04 mark no initial set, so a watch would read as synced before the peer + // answers it. Asking for everything at connect keeps their discovery eager instead. + let lazy = client && !matches!(subscriber.version, Version::Lite03 | Version::Lite04); Self { - prefixes, + discovery: crate::discovery::Discovery::new(&subscriber.origin, lazy, subscriber.runtime.clone()), + prefixes: HashMap::new(), cleanup: SubscriptionCleanup(subscriber.subscribes.clone()), uni: UniAccept::new(subscriber.clone(), early), bandwidth: Some(RecvBandwidth::new(subscriber.clone())), @@ -660,14 +659,34 @@ impl SubscriberDriver { } pub fn poll(&mut self, waiter: &kio::Waiter) -> Poll> { - let mut i = 0; - while i < self.prefixes.len() { - match self.prefixes[i].poll(waiter) { - Poll::Ready(Ok(())) => { - self.prefixes.swap_remove(i); + loop { + while let Poll::Ready(change) = self.discovery.poll(waiter) { + match change { + crate::discovery::Change::Open(prefix, snapshot) => { + let machine = AnnouncePrefix::new(self.subscriber.clone(), prefix.clone(), snapshot); + self.prefixes.insert(prefix, machine); + } + // Dropping the machine resets its stream, which the peer reads as the + // request ending, and retracts every route it reported. + crate::discovery::Change::Close(prefix) => { + self.prefixes.remove(&prefix); + } } - Poll::Ready(Err(err)) => return Poll::Ready(Err(err)), - Poll::Pending => i += 1, + } + let mut ended = Vec::new(); + for (prefix, machine) in &mut self.prefixes { + match machine.poll(waiter) { + Poll::Ready(Ok(())) => ended.push(prefix.clone()), + Poll::Ready(Err(err)) => return Poll::Ready(Err(err)), + Poll::Pending => {} + } + } + if ended.is_empty() { + break; + } + for prefix in ended { + self.prefixes.remove(&prefix); + self.discovery.ended(&prefix); } } if let Poll::Ready(res) = self.uni.poll(waiter) { @@ -1584,15 +1603,14 @@ struct PrefixRun { } impl AnnouncePrefix { - fn new(subscriber: Subscriber, prefix: PathOwned) -> Self { - // Taken now, while the session is being set up, so a consumer of the origin never - // reads as synced before the request even goes out. - let snapshot = Some(subscriber.origin.snapshot(&prefix)); + /// `snapshot` was taken when the prefix was first wanted, so a consumer of the + /// origin never reads as synced before the request even goes out. + fn new(subscriber: Subscriber, prefix: PathOwned, snapshot: crate::model::Snapshot) -> Self { Self { subscriber, prefix, state: PrefixState::Open, - snapshot, + snapshot: Some(snapshot), } } @@ -5198,7 +5216,11 @@ mod tests { going_away: Default::default(), auth: crate::auth::Handle::new(false), }); - let mut prefix = AnnouncePrefix::new(subscriber, Path::new("").to_owned()); + let mut prefix = AnnouncePrefix::new( + subscriber.clone(), + Path::new("").to_owned(), + subscriber.origin.snapshot(""), + ); let mut cursor = consumer.announced(); let mut run = std::pin::pin!(kio::wait(|waiter| prefix.poll(waiter))); @@ -5265,7 +5287,14 @@ mod tests { going_away: Default::default(), auth: crate::auth::Handle::new(false), }); - (AnnouncePrefix::new(subscriber, Path::new("").to_owned()), consumer) + ( + AnnouncePrefix::new( + subscriber.clone(), + Path::new("").to_owned(), + subscriber.origin.snapshot(""), + ), + consumer, + ) } /// The origin's consumers sync once the counted starts are in and read, never before. @@ -5438,7 +5467,11 @@ mod tests { }); let slots = crate::session::Slots::new(2); subscriber.announces = slots.clone(); - let mut prefix = AnnouncePrefix::new(subscriber, Path::new("").to_owned()); + let mut prefix = AnnouncePrefix::new( + subscriber.clone(), + Path::new("").to_owned(), + subscriber.origin.snapshot(""), + ); let err = kio::wait(|waiter| prefix.poll(waiter)) .await diff --git a/rs/moq-net/src/model/mod.rs b/rs/moq-net/src/model/mod.rs index 63e3891e27d..10a7da6f77d 100644 --- a/rs/moq-net/src/model/mod.rs +++ b/rs/moq-net/src/model/mod.rs @@ -80,6 +80,11 @@ pub(crate) use origin_impl::interest_prefixes; // pending: /quest/m1/public-model/answer.md pub(crate) use origin_impl::Snapshot; +// What a client session asks its peer to announce, as the origin's readers change. +#[cfg(feature = "async")] +// pending: /quest/m1/public-model/answer.md +pub(crate) use origin_impl::{InterestChange, Interests}; + // The advertise-only route guard, for tests shaping the route table. #[cfg(all(test, feature = "async"))] // private: test diff --git a/rs/moq-net/src/model/origin.rs b/rs/moq-net/src/model/origin.rs index 999d6e6de46..4229dc2d341 100644 --- a/rs/moq-net/src/model/origin.rs +++ b/rs/moq-net/src/model/origin.rs @@ -1351,6 +1351,8 @@ struct TableCursor { /// For a cursor reading through a mount: the mount and the handle's own /// patterns, so a wildcard captures what the handle names rather than the target. named: Option<(Mount, Patterns)>, + /// Whether the cursor counts an [`Interest`] at each of its heads. + interested: bool, /// The last delivered best route per presented (relative) prefix, for change /// detection. Servability is part of the dedupe key but never leaves the model. current: HashMap, @@ -2062,32 +2064,58 @@ impl Producer { // pending: /quest/m1/public-model/answer.md pub(crate) fn snapshot(&self, prefix: impl AsPath) -> Snapshot { let prefix = self.root.join(prefix.as_path()).to_owned(); - // A prefix past the depth limit can carry no route, so nothing waits on it. - let Ok(claim) = prefix_claim(&prefix) else { - return Snapshot::landed(); - }; + self.shared.lock().snapshot(&self.shared, prefix, self.link) + } + + /// Hear what this handle's session should ask its peer about: the broadest + /// prefixes readers of the origin want discovered, mapped onto this handle's + /// scope, starting with the current ones. + #[cfg_attr(not(feature = "async"), allow(dead_code))] + // pending: /quest/m1/public-model/answer.md + pub(crate) fn interests(&self) -> Interests { + let queue = kio::Queue::new(); let mut state = self.shared.lock(); - if state.closed { - return Snapshot::landed(); + let id = state.next_listener; + state.next_listener += 1; + let listener = Listener { + id, + root: self.root.clone(), + heads: interest_prefixes(&self.allowed()), + allowed: self.scope.allowed.clone(), + link: self.link, + queue: queue.clone(), + }; + match state.closed { + true => queue.close(), + false => { + for prefix in state.routes.interests_from(&Path::empty()) { + for relative in listener.asks(&prefix) { + let snapshot = state.snapshot(&self.shared, self.root.join(&relative), self.link); + let _ = queue.try_push(InterestChange { + prefix: relative, + wanted: true, + snapshot: Some(snapshot), + }); + } + } + state.listeners.push(listener); + } } - let id = state.next_snapshot; - state.next_snapshot += 1; - state.add_snapshot( - &prefix, - SnapshotEntry { - id, - link: self.link, - claim, - prefix: prefix.clone(), - }, - ); - Snapshot { - shared: Some(self.shared.clone()), - prefix, + Interests { + shared: self.shared.clone(), id, + queue, } } + /// Whether routes arrive here on a link between relays ([`Self::peer`] or + /// [`Self::upstream`]), whose sessions discover everything they may. + #[cfg_attr(not(feature = "async"), allow(dead_code))] + // pending: /quest/m1/public-model/answer.md + pub(crate) fn relays(&self) -> bool { + self.link != Link::Local + } + /// Advertise a route over `prefix` and serve the requests beneath it. /// /// A route is always a prefix: it claims `prefix` and every path beneath it @@ -2709,6 +2737,10 @@ impl DriverState { .collect(); let cursors: Vec<_> = shared.cursors.values().map(|cursor| cursor.state.clone()).collect(); let fronts: Vec<_> = shared.fronts.values().map(|front| front.request.clone()).collect(); + // Sessions stop hearing interests; their queued snapshots release as they drop. + for listener in &shared.listeners { + listener.queue.close(); + } (servers, cursors, fronts) }; @@ -3031,13 +3063,18 @@ async fn serve_front(task: FrontTask) -> Vec { // starting with the request that minted the front. let mut held = true; let mut events: VecDeque = VecDeque::new(); + // Once the front picks a route a session received, it asks sessions to keep + // discovering its path for as long as it lives, so the route outlives whatever + // reader found it, such as a listing that has since closed. + let mut discovered: Option = None; // Read the table for the machine: the best route and whether // the serving source is on its way out. Also what the watch wakes for. let select = |front: &mut Front, sources: &HashMap, seen: &mut u64, - resolved: &mut Option| + resolved: &mut Option, + discovered: &mut Option| -> Event { let table = shared.read(); if table.closed { @@ -3047,7 +3084,10 @@ async fn serve_front(task: FrontTask) -> Vec { *seen = watch.seen(); front.retain_routes(|route| table.routes.covers(&path.as_path(), route)); let instance = resolved.as_ref().map(|resolved| &resolved.instance); - let best = pick(&table, &path.as_path(), horizon, front, instance).map(|entry| Candidate { + let best = pick(&table, &path.as_path(), horizon, front, instance); + // Only a received route carries a hop chain: a local one needs no discovery. + let received = best.is_some_and(|entry| !entry.hops.is_empty()); + let best = best.map(|entry| Candidate { route: entry.id, local: entry.local, epoch: entry.epoch.is_some(), @@ -3068,6 +3108,10 @@ async fn serve_front(task: FrontTask) -> Vec { table.renew(&prefix); } } + if received && discovered.is_none() { + let mut table = shared.lock(); + *discovered = Some(Interest::new(&mut table, &shared, path.clone())); + } Event::Selected { best, serving_closing } }; @@ -3099,7 +3143,7 @@ async fn serve_front(task: FrontTask) -> Vec { } }; - events.push_back(select(&mut front, &sources, &mut seen, &mut resolved)); + events.push_back(select(&mut front, &sources, &mut seen, &mut resolved, &mut discovered)); loop { while let Some(event) = events.pop_front() { @@ -3109,7 +3153,9 @@ async fn serve_front(task: FrontTask) -> Vec { }; for action in front.step(event) { match action { - Action::Reselect => events.push_back(select(&mut front, &sources, &mut seen, &mut resolved)), + Action::Reselect => { + events.push_back(select(&mut front, &sources, &mut seen, &mut resolved, &mut discovered)) + } Action::Request { route } => { // The entry and what it serves. let found = { @@ -3714,7 +3760,7 @@ async fn serve_front(task: FrontTask) -> Vec { deadline.set(None); Event::Deadline { now: timers.now() } } - Step::Table => select(&mut front, &sources, &mut seen, &mut resolved), + Step::Table => select(&mut front, &sources, &mut seen, &mut resolved, &mut discovered), Step::Listing => { // A route claiming a prefix names no broadcast, so a path it serves on request // has no inventory to list. @@ -3827,6 +3873,12 @@ struct RouteNode { /// Snapshots at this node or below, so a cursor registering walks only the /// subtrees holding one. snapshots_below: usize, + /// Readers asking sessions to discover routes at or beneath exactly this + /// prefix; see [`Interest`]. + interests: usize, + /// Interests at this node or below, so finding the narrower ones a change + /// covers or uncovers walks only the subtrees holding one. + interests_below: usize, children: HashMap, } @@ -4027,12 +4079,118 @@ impl Drop for Snapshot { } } +/// Counts an absolute prefix as one readers want discovered until dropped: a +/// session fed into the origin asks its peer about the broadest such prefixes. +/// Dropping it takes the table lock: never drop one while holding it. +struct Interest { + shared: SendCell>, + prefix: PathOwned, +} + +impl Interest { + /// Count `prefix` under the held table lock. + fn new(state: &mut OriginState, shared: &SendCell>, prefix: PathOwned) -> Self { + state.add_interest(shared, &prefix); + Self { + shared: shared.clone(), + prefix, + } + } +} + +impl Drop for Interest { + fn drop(&mut self) { + self.shared.lock().remove_interest(&self.shared, &self.prefix); + } +} + +/// A session registered for [`Interests`], as the table holds it. +struct Listener { + id: u64, + /// The session's root, absolute: what it asks about is named relative to it. + root: PathOwned, + /// The broadest prefixes the session may ask about, relative to `root`. + heads: Vec, + /// The absolute patterns the session may publish under. + allowed: Patterns, + /// The link the session's routes arrive on, which its snapshots wait for. + link: Link, + queue: kio::Queue, +} + +impl Listener { + /// What the session asks about, relative to its root, for an interest at the + /// absolute `prefix`: each head an interest above it covers, since the scope + /// may ask for no more, or the prefix itself inside a head. + fn asks(&self, prefix: &Path) -> Vec { + let Some(relative) = prefix.strip_prefix(&self.root) else { + return match self.root.has_prefix(prefix) { + true => self.heads.clone(), + false => Vec::new(), + }; + }; + let mut out = Vec::new(); + for head in &self.heads { + if head.has_prefix(&relative) { + out.push(head.clone()); + } else if relative.has_prefix(head) { + let reaches = Pattern::subtree(prefix.as_str()).is_ok_and(|subtree| self.allowed.overlaps(&subtree)); + return match reaches { + true => vec![relative.to_owned()], + false => Vec::new(), + }; + } + } + out + } +} + +/// What a session asks its peer about, as it changes: the broadest prefixes +/// readers of the origin want discovered, mapped onto the session's scope. +// pending: /quest/m1/public-model/answer.md +pub(crate) struct Interests { + shared: SendCell>, + id: u64, + queue: kio::Queue, +} + +impl Interests { + /// The next change, or `None` once the origin closed. + #[cfg_attr(not(feature = "async"), allow(dead_code))] + // pending: /quest/m1/public-model/answer.md + pub(crate) fn poll_next(&self, waiter: &kio::Waiter) -> Poll> { + Poll::Ready(ready!(self.queue.poll_pop(waiter)).ok()) + } +} + +impl Drop for Interests { + fn drop(&mut self) { + // Only the table's copy of the queue goes here; the changes still queued drop with + // this handle's, after the lock is released, since each may hold a snapshot. + self.shared.lock().listeners.retain(|listener| listener.id != self.id); + } +} + +/// One change to what a session asks its peer about; see [`Interests`]. +// pending: /quest/m1/public-model/answer.md +pub(crate) struct InterestChange { + /// The prefix, relative to the session's root. + pub prefix: PathOwned, + /// Whether readers now want the prefix discovered, or no longer do. + pub wanted: bool, + /// Set when wanted: holds the readers under the prefix short of synced from + /// the moment they asked, until the request the session opens for it + /// delivers the peer's initial set, or the session drops it unasked. + pub snapshot: Option, +} + /// What a registration adds to the subtree counts on its walk. #[derive(Clone, Copy)] struct Below { cursors: usize, watches: usize, snapshots: usize, + interests: usize, } impl Below { @@ -4040,21 +4198,23 @@ impl Below { cursors: 0, watches: 0, snapshots: 0, + interests: 0, }; const CURSOR: Self = Self { cursors: 1, - watches: 0, - snapshots: 0, + ..Self::NONE }; const WATCH: Self = Self { - cursors: 0, watches: 1, - snapshots: 0, + ..Self::NONE }; const SNAPSHOT: Self = Self { - cursors: 0, - watches: 0, snapshots: 1, + ..Self::NONE + }; + const INTEREST: Self = Self { + interests: 1, + ..Self::NONE }; } @@ -4065,6 +4225,7 @@ impl RouteNode { && self.cursors.is_empty() && self.watches.is_empty() && self.snapshots.is_empty() + && self.interests == 0 && self.children.is_empty() } @@ -4082,6 +4243,7 @@ impl RouteNode { self.cursors_below += below.cursors; self.watches_below += below.watches; self.snapshots_below += below.snapshots; + self.interests_below += below.interests; match parts.next() { None => self, Some(part) => self.children.entry(part.to_string()).or_default().reach(parts, below), @@ -4111,6 +4273,7 @@ impl RouteNode { self.cursors_below -= below.cursors; self.watches_below -= below.watches; self.snapshots_below -= below.snapshots; + self.interests_below -= below.interests; Some(result) } @@ -4191,6 +4354,21 @@ impl RouteNode { child.collect_cursors(out); } } + + /// Collect the broadest interests strictly beneath this node, which sits at + /// `at`: each one no other interest beneath this node covers. + fn collect_interests(&self, at: &Path, out: &mut Vec) { + for (part, child) in &self.children { + if child.interests_below == 0 { + continue; + } + let path = at.join(part.as_str()); + match child.interests > 0 { + true => out.push(path), + false => child.collect_interests(&path, out), + } + } + } } impl RouteTable { @@ -4361,6 +4539,35 @@ impl RouteTable { }); } + /// Whether an interest strictly above `path` covers it. + fn interest_above(&self, path: &Path) -> bool { + let mut node = &self.root; + for part in path.parts() { + if node.interests > 0 { + return true; + } + match node.children.get(part) { + Some(child) => node = child, + None => return false, + } + } + false + } + + /// The interests at `path` or, failing that, the broadest beneath it: what a + /// session joining now asks about under `path`. + fn interests_from(&self, path: &Path) -> Vec { + let Some(node) = self.root.find(path.parts()) else { + return Vec::new(); + }; + if node.interests > 0 { + return vec![path.to_owned()]; + } + let mut out = Vec::new(); + node.collect_interests(path, &mut out); + out + } + /// Wake the watches of every path a route at `prefix` covers. fn poke_below(&self, prefix: &Path, hold: &Hold) { if let (_, Some(node)) = self.split(prefix) { @@ -4429,6 +4636,11 @@ struct OriginState { // Set when the origin's driver dropped: new requests fail with `Closed` // immediately and handlers observe the end instead of parking forever. closed: bool, + + // The sessions asking their peer about only what readers here want + // discovered; see [`Interests`]. + listeners: Vec, + next_listener: u64, } impl OriginState { @@ -4444,6 +4656,8 @@ impl OriginState { fronts: WeakCache::default(), withdrawn: HashMap::new(), closed: false, + listeners: Vec::new(), + next_listener: 0, } } @@ -4572,6 +4786,102 @@ impl OriginState { } } + /// Start a [`Snapshot`] at the absolute `prefix` for the routes arriving on `link`. + fn snapshot(&mut self, shared: &SendCell>, prefix: PathOwned, link: Link) -> Snapshot { + // A prefix past the depth limit can carry no route, so nothing waits on it. + let Ok(claim) = prefix_claim(&prefix) else { + return Snapshot::landed(); + }; + if self.closed { + return Snapshot::landed(); + } + let id = self.next_snapshot; + self.next_snapshot += 1; + self.add_snapshot( + &prefix, + SnapshotEntry { + id, + link, + claim, + prefix: prefix.clone(), + }, + ); + Snapshot { + shared: Some(shared.clone()), + prefix, + id, + } + } + + /// Count the absolute `head` as a prefix readers want discovered, telling the + /// listeners when that changes the covering set. + fn add_interest(&mut self, shared: &SendCell>, head: &Path) { + let node = self.routes.root.reach(head.parts(), Below::INTEREST); + node.interests += 1; + if node.interests > 1 || self.listeners.is_empty() { + return; + } + let mut narrower = Vec::new(); + node.collect_interests(head, &mut narrower); + if self.routes.interest_above(head) { + return; + } + // Asked for before the narrower ones it covers are let go, so nothing retracts. + self.emit(shared, head, true); + for prefix in narrower { + self.emit(shared, &prefix, false); + } + } + + /// Undo one [`Self::add_interest`] of `head`. + fn remove_interest(&mut self, shared: &SendCell>, head: &Path) { + let listening = !self.listeners.is_empty(); + let narrower = self.routes.root.edit(head.parts(), Below::INTEREST, |node| { + node.interests -= 1; + (node.interests == 0 && listening).then(|| { + let mut narrower = Vec::new(); + node.collect_interests(head, &mut narrower); + narrower + }) + }); + let Some(Some(narrower)) = narrower else { + return; + }; + if self.routes.interest_above(head) { + return; + } + for prefix in narrower { + self.emit(shared, &prefix, true); + } + self.emit(shared, head, false); + } + + /// Tell every listener the covering set of interests gained or lost the + /// absolute `prefix`. + fn emit(&mut self, shared: &SendCell>, prefix: &Path, wanted: bool) { + let mut changes = Vec::new(); + for listener in &self.listeners { + for relative in listener.asks(prefix) { + changes.push(( + listener.queue.clone(), + listener.root.join(&relative), + relative, + listener.link, + )); + } + } + for (queue, absolute, prefix, link) in changes { + let snapshot = wanted.then(|| self.snapshot(shared, absolute, link)); + // A listener still registered has an open queue, except after the teardown, + // whose snapshots are landed: dropping a refused one never takes this lock. + let _ = queue.try_push(InterestChange { + prefix, + wanted, + snapshot, + }); + } + } + /// Register a [`Watch`] on the routes covering `path`. fn watch(&mut self, shared: &SendCell>, path: &Path) -> Watch { let id = self.next_watch; @@ -4615,7 +4925,20 @@ impl OriginState { } /// Register a cursor and replay the current best route per presented prefix. - fn register_cursor(&mut self, id: ConsumerId, mut cursor: TableCursor) { + fn register_cursor( + &mut self, + shared: &SendCell>, + id: ConsumerId, + mut cursor: TableCursor, + ) { + // First, so the snapshots a session takes for these interests hold the cursor + // unsynced from the start, through the scan below. + if cursor.interested { + for head in &cursor.heads { + self.add_interest(shared, head); + } + } + // The routes a cursor can see sit on the walk down to one of its heads or // somewhere beneath it, so only those subtrees are replayed. let mut presented: BTreeSet = BTreeSet::new(); @@ -5337,6 +5660,8 @@ impl kio::Task for Requesting { pub struct Routing { consumer: Consumer, path: PathOwned, + // Asks sessions to discover the path for as long as this waits on it. + interest: Option, // The covering routes, watched since the generation the last ask saw. watch: Option<(Watch, u64)>, // The ask in flight, dropped once it fails `Unroutable` until the routes move. @@ -5379,6 +5704,10 @@ impl kio::Task for Routing { }; let watch = table.watch(&self.consumer.shared, &resolved); let seen = watch.seen(); + if self.interest.is_none() && self.consumer.interested { + let resolved = resolved.to_owned(); + self.interest = Some(Interest::new(&mut table, &self.consumer.shared, resolved)); + } drop(table); self.watch = Some((watch, seen)); self.request = Some( @@ -5483,6 +5812,11 @@ pub struct Consumer { // Which routes beneath a hidden (`.`-prefixed) segment `announced` reports. hidden: Hidden, + // Whether this handle's announcement streams and routed requests count as + // interests, so a session fed into the origin asks its peer about them. Off + // for a session serving its peer (see `Self::serving`). + interested: bool, + // The cache policy remote fronts inherit, mirroring what // `create_broadcast` gives a local front. pool: cache::Pool, @@ -5511,6 +5845,7 @@ impl Consumer { ..Horizon::default() }, hidden: Hidden::default(), + interested: true, pool: producer.pool.clone(), cache_duration: producer.cache_duration, tasks: producer.tasks.downgrade(), @@ -5548,6 +5883,15 @@ impl Consumer { self } + /// A clone for a session serving its peer: its reads are the peer's, not a + /// local reader's, so they never make a session ask its own peer about them. + #[cfg_attr(not(feature = "async"), allow(dead_code))] + // pending: /quest/m1/public-model/answer.md + pub(crate) fn serving(mut self) -> Self { + self.interested = false; + self + } + /// A clone whose [`announced`](Self::announced) also reports hidden routes: /// those with a segment starting with `.` below the requested prefix. /// Hidden routes are left out by default, so a platform can add `.`-named @@ -5634,6 +5978,8 @@ impl Consumer { /// relative to this consumer's root; its patterns only filter visibility. /// Routes with a segment starting with `.` below the literal head of those /// patterns are hidden unless [`with_hidden`](Self::with_hidden) opted in. + /// The stream also watches its scope: a moq-lite client session fed by this + /// origin asks its peer about the scope while any stream watches it. /// Drop the returned [`AnnounceConsumer`] to unregister. pub fn announced(&self) -> AnnounceConsumer { let state = kio::Producer::::default(); @@ -5652,6 +5998,7 @@ impl Consumer { under, holes, named: mount.map(|mount| (mount.clone(), self.scope.allowed.clone())), + interested: self.interested, current: HashMap::new(), }; @@ -5838,6 +6185,7 @@ impl Consumer { Routing { consumer: self.clone(), path, + interest: None, watch: None, request, epoch, @@ -5923,7 +6271,11 @@ impl Consumer { /// /// The returned future fails with [`Error::Unroutable`] at once when no /// announced route covers the path, including a broadcast created on this - /// origin but not announced. + /// origin but not announced. A moq-lite client session only discovers what this + /// origin's readers watch, so a path no [`Self::announced`], + /// [`Self::routed_broadcast`], or [`Self::announced_broadcast`] watches is + /// unroutable on it. A broadcast resolved through a discovered route keeps its + /// path watched until dropped. /// A route claims capability, not inventory: resolving a covered path /// succeeds optimistically, and a path that names nothing surfaces as /// [`Error::NotFound`] on its tracks instead. @@ -6181,7 +6533,7 @@ impl AnnounceConsumer { } else { for cursor in cursors { let id = ConsumerId::new(); - table.register_cursor(id, cursor); + table.register_cursor(shared, id, cursor); ids.push(id); } } @@ -6511,11 +6863,14 @@ impl futures::Stream for AnnounceConsumer { impl Drop for AnnounceConsumer { fn drop(&mut self) { - let mut shared = self.shared.lock(); + let mut state = self.shared.lock(); for id in &self.ids { - if let Some(cursor) = shared.cursors.remove(id) { + if let Some(cursor) = state.cursors.remove(id) { for head in &cursor.heads { - shared.routes.remove_cursor(head, *id); + state.routes.remove_cursor(head, *id); + if cursor.interested { + state.remove_interest(&self.shared, head); + } } } } diff --git a/rs/moq-net/tests/integration/idle_session.rs b/rs/moq-net/tests/integration/idle_session.rs index 215fca18189..593ee2ead4f 100644 --- a/rs/moq-net/tests/integration/idle_session.rs +++ b/rs/moq-net/tests/integration/idle_session.rs @@ -39,7 +39,11 @@ async fn viewer( ) -> (MockPair, moq_net::bandwidth::Consumer) { let mut options = MockConnectOptions::new(version.parse::().unwrap()); options.server_publish = Some(origin.consume()); - options.client_subscribe = Some(origin::Producer::new(origin::Config::new(Hop::new(2).unwrap())).0); + // Driven, so its readers' interests reach the session and the viewer asks for what + // the server publishes. + let (subscribe, driver) = origin::Producer::new(origin::Config::new(Hop::new(2).unwrap())); + support::harness::spawn(driver); + options.client_subscribe = Some(subscribe); options.stats = STATS; options.server_polls = polls; let pair = connect_mock(options).await; diff --git a/rs/moq-net/tests/integration/lazy_discovery.rs b/rs/moq-net/tests/integration/lazy_discovery.rs new file mode 100644 index 00000000000..92cbc1ca2e4 --- /dev/null +++ b/rs/moq-net/tests/integration/lazy_discovery.rs @@ -0,0 +1,360 @@ +#![cfg(feature = "async")] + +//! A client session asks its peer to announce only the prefixes its origin's readers +//! watch, keeps a request open while anything at or beneath it is watched, and lets +//! it go once nothing has been for a linger. A relay's link still asks for everything. + +use std::time::Duration; + +use moq_net::{Error, Hop, Pattern, Patterns, Version, announce, origin}; + +use crate::support; +use support::harness::{MockConnectOptions, MockPair, connect_mock}; + +// Versions that mark where the initial set ends, so a synced read holds all of it. +const VERSIONS: &[&str] = &["moq-lite-05", "moq-lite-06", "moq-lite-07-wip"]; + +/// Long enough, in virtual time, for anything in flight to reach the far side. +const SETTLE: Duration = Duration::from_millis(500); + +/// Past the 2s an announce request outlives its last watcher. +const LINGERED: Duration = Duration::from_secs(3); + +/// The first byte of a moq-lite ANNOUNCE stream. +const ANNOUNCE: u8 = 1; + +fn produce_origin(hop: u64) -> origin::Producer { + let (producer, driver) = origin::Producer::new(origin::Config::new(Hop::new(hop).unwrap())); + support::harness::spawn(driver); + producer +} + +/// A server publishing `a/x` and `b/y`, and a client fed into `subscriber` asking +/// only about what its readers watch. +struct Lazy { + subscriber: origin::Producer, + pair: MockPair, + _broadcasts: Vec, +} + +impl Lazy { + async fn new(version: &str) -> Self { + Self::with(version, produce_origin(2)).await + } + + async fn with(version: &str, subscriber: origin::Producer) -> Self { + let publisher = produce_origin(1); + let broadcasts = ["a/x", "b/y"] + .into_iter() + .map(|path| { + let broadcast = publisher.create_broadcast(path).unwrap(); + broadcast.announce(Default::default()).unwrap(); + broadcast + }) + .collect(); + + let mut options = MockConnectOptions::new(version.parse::().unwrap()); + options.server_publish = Some(publisher.consume()); + options.client_subscribe = Some(subscriber.clone()); + options.client_discovers_all = false; + let pair = connect_mock(options).await; + moq_clock::sleep(SETTLE).await; + Self { + subscriber, + pair, + _broadcasts: broadcasts, + } + } + + /// How many announce requests the client has opened. + fn requests(&self) -> usize { + self.pair + .client_transport + .bidi_types() + .into_iter() + .filter(|kind| *kind == ANNOUNCE) + .count() + } + + /// Watch everything under `prefix`. + fn watch(&self, prefix: &str) -> announce::Consumer { + self.subscriber + .consume() + .scope(prefix, &Patterns::from(Pattern::all())) + .unwrap() + .announced() + } + + /// Whether the client's origin routes `path` right now. + async fn routes(&self, path: &str) -> bool { + match self.subscriber.consume().request_broadcast(path, None).await { + Ok(_) => true, + Err(Error::Unroutable) => false, + Err(err) => panic!("{path}: {err}"), + } + } +} + +/// Read until the stream syncs, returning the prefixes it then holds. +async fn synced(announced: &mut announce::Consumer) -> Vec { + let mut seen = Vec::new(); + while !announced.is_synced() { + let event = moq_clock::timeout(SETTLE, announced.next()) + .await + .expect("synced in time") + .expect("the stream stays open"); + match event { + announce::Event::Start(announce) | announce::Event::Restart(announce) => { + seen.push(announce.prefix.to_string()) + } + announce::Event::End(announce) => seen.retain(|prefix| *prefix != announce.prefix.as_str()), + announce::Event::Update(_) => {} + } + } + seen.sort(); + seen +} + +/// Nothing watched, nothing asked: the peer's broadcasts stay undiscovered. +#[moq_clock::test] +async fn asks_nothing_until_watched() { + for version in VERSIONS { + let lazy = Lazy::new(version).await; + assert_eq!(lazy.requests(), 0, "{version}"); + assert!(!lazy.routes("a/x").await, "{version}"); + } +} + +/// A watch asks about its own prefix and nothing else. +#[moq_clock::test] +async fn asks_only_about_the_watched_prefix() { + for version in VERSIONS { + let lazy = Lazy::new(version).await; + let mut watch = lazy.watch("a"); + assert_eq!(synced(&mut watch).await, ["x"], "{version}"); + assert_eq!(lazy.requests(), 1, "{version}"); + assert!(lazy.routes("a/x").await, "{version}"); + assert!(!lazy.routes("b/y").await, "{version}"); + } +} + +/// A watch made before the session exists is asked about once it connects. +#[moq_clock::test] +async fn a_new_session_asks_about_existing_watches() { + let subscriber = produce_origin(2); + let mut watch = subscriber + .consume() + .scope("b", &Patterns::from(Pattern::all())) + .unwrap() + .announced(); + let lazy = Lazy::with("moq-lite-07-wip", subscriber).await; + assert_eq!(synced(&mut watch).await, ["y"]); + assert_eq!(lazy.requests(), 1); + assert!(!lazy.routes("a/x").await); +} + +/// The request outlives its last watcher by the linger, then closes and retracts. +#[moq_clock::test] +async fn lingers_after_the_last_watcher() { + for version in VERSIONS { + let lazy = Lazy::new(version).await; + let mut watch = lazy.watch("a"); + synced(&mut watch).await; + drop(watch); + + moq_clock::sleep(Duration::from_secs(1)).await; + assert!(lazy.routes("a/x").await, "{version}: still lingering"); + + // Watched again within the linger: the same request carries on. + let again = lazy.watch("a"); + moq_clock::sleep(LINGERED).await; + assert!(lazy.routes("a/x").await, "{version}: rewatched"); + assert_eq!(lazy.requests(), 1, "{version}"); + + drop(again); + moq_clock::sleep(LINGERED).await; + assert!(!lazy.routes("a/x").await, "{version}: closed"); + } +} + +/// A narrower watch rides an open broader request, which stays open while it does. +#[moq_clock::test] +async fn a_narrower_watch_rides_the_broader_request() { + for version in VERSIONS { + let lazy = Lazy::new(version).await; + let mut root = lazy.watch(""); + assert_eq!(synced(&mut root).await, ["a/x", "b/y"], "{version}"); + let mut narrower = lazy.watch("a"); + assert_eq!(synced(&mut narrower).await, ["x"], "{version}"); + assert_eq!(lazy.requests(), 1, "{version}"); + + drop(root); + moq_clock::sleep(LINGERED).await; + assert_eq!(lazy.requests(), 1, "{version}: nothing reopened"); + assert!(lazy.routes("a/x").await, "{version}"); + assert!(narrower.try_next().is_none(), "{version}: nothing retracted"); + } +} + +/// A narrower request open before a broader one stays until the broader one goes. +#[moq_clock::test] +async fn a_broader_request_holds_the_narrower_open() { + for version in VERSIONS { + let lazy = Lazy::new(version).await; + let mut narrower = lazy.watch("a"); + synced(&mut narrower).await; + let mut root = lazy.watch(""); + assert_eq!(synced(&mut root).await, ["a/x", "b/y"], "{version}"); + assert_eq!(lazy.requests(), 2, "{version}"); + + drop(narrower); + moq_clock::sleep(LINGERED).await; + assert!(root.try_next().is_none(), "{version}: nothing retracted"); + + // The broader request lingers out, then the narrower one it held does. + drop(root); + moq_clock::sleep(LINGERED).await; + assert!(!lazy.routes("b/y").await, "{version}"); + assert!(lazy.routes("a/x").await, "{version}"); + moq_clock::sleep(LINGERED).await; + assert!(!lazy.routes("a/x").await, "{version}"); + } +} + +/// A narrower request still lingering when a broader one opens is held like one still +/// watched, rather than lingering out on its own timer. +#[moq_clock::test] +async fn a_broader_request_holds_a_lingering_narrower_open() { + for version in VERSIONS { + let lazy = Lazy::new(version).await; + let mut narrower = lazy.watch("a"); + synced(&mut narrower).await; + drop(narrower); + + moq_clock::sleep(Duration::from_secs(1)).await; + let mut root = lazy.watch(""); + assert_eq!(synced(&mut root).await, ["a/x", "b/y"], "{version}"); + assert_eq!(lazy.requests(), 2, "{version}"); + + // Past the narrower request's own linger: it is held, not closed. + moq_clock::sleep(LINGERED).await; + assert!(root.try_next().is_none(), "{version}: nothing retracted"); + + // The broader request lingers out, then the narrower one it held does. + drop(root); + moq_clock::sleep(LINGERED).await; + assert!(!lazy.routes("b/y").await, "{version}"); + assert!(lazy.routes("a/x").await, "{version}"); + moq_clock::sleep(LINGERED).await; + assert!(!lazy.routes("a/x").await, "{version}"); + } +} + +/// A broadcast resolved through a discovered route keeps it discovered after the +/// listing that found it closes. +#[moq_clock::test] +async fn a_resolved_broadcast_keeps_its_route() { + for version in VERSIONS { + let lazy = Lazy::new(version).await; + let mut watch = lazy.watch("a"); + synced(&mut watch).await; + let broadcast = lazy.subscriber.consume().request_broadcast("a/x", None).await.unwrap(); + drop(watch); + + moq_clock::sleep(LINGERED).await; + assert!(lazy.routes("a/x").await, "{version}"); + assert!(!lazy.routes("b/y").await, "{version}"); + + drop(broadcast); + } +} + +/// Waiting on a path asks about it. +#[moq_clock::test] +async fn waiting_on_a_path_asks_about_it() { + for version in VERSIONS { + let lazy = Lazy::new(version).await; + let consumer = lazy.subscriber.consume(); + moq_clock::timeout(SETTLE, consumer.routed_broadcast("a/x")) + .await + .expect("routed in time") + .unwrap(); + let mut followed = consumer.announced_broadcast("b/y", None).unwrap(); + moq_clock::timeout(SETTLE, followed.available()) + .await + .expect("announced in time") + .unwrap(); + assert_eq!(lazy.requests(), 2, "{version}"); + } +} + +/// A relay's link asks for everything, watched or not, so discovery never narrows +/// what a relay learns. +#[moq_clock::test] +async fn a_relay_link_asks_for_everything() { + for version in VERSIONS { + for upstream in [false, true] { + let relay = produce_origin(2); + let link = match upstream { + true => relay.clone().upstream(), + false => relay.clone().peer(), + }; + let lazy = Lazy::with(version, link).await; + assert_eq!(lazy.requests(), 1, "{version}"); + // Read through a plain handle: one taken from an upstream link hides its routes. + for path in ["a/x", "b/y"] { + relay.consume().request_broadcast(path, None).await.unwrap(); + } + } + } +} + +/// A server asks a client for everything it may, so a relay ingests what clients publish. +#[moq_clock::test] +async fn a_server_asks_for_everything() { + for version in VERSIONS { + let publisher = produce_origin(1); + let broadcast = publisher.create_broadcast("a/x").unwrap(); + broadcast.announce(Default::default()).unwrap(); + let relay = produce_origin(2); + let mut options = MockConnectOptions::new(version.parse::().unwrap()); + options.client_publish = Some(publisher.consume()); + options.server_subscribe = Some(relay.clone()); + let _pair = connect_mock(options).await; + moq_clock::sleep(SETTLE).await; + relay.consume().request_broadcast("a/x", None).await.unwrap(); + } +} + +/// Serving the peer's own request is no local reader: a client publishing and +/// consuming through one origin still asks about only what its readers watch. +#[moq_clock::test] +async fn serving_the_peer_asks_nothing() { + for version in VERSIONS { + let local = produce_origin(2); + let remote = produce_origin(1); + let broadcast = remote.create_broadcast("a/x").unwrap(); + broadcast.announce(Default::default()).unwrap(); + + let mut options = MockConnectOptions::new(version.parse::().unwrap()); + options.client_publish = Some(local.consume()); + options.client_subscribe = Some(local.clone()); + options.client_discovers_all = false; + options.server_publish = Some(remote.consume()); + options.server_subscribe = Some(remote.clone()); + let pair = connect_mock(options).await; + moq_clock::sleep(SETTLE).await; + + let requests = pair + .client_transport + .bidi_types() + .into_iter() + .filter(|kind| *kind == ANNOUNCE) + .count(); + assert_eq!(requests, 0, "{version}"); + assert!(matches!( + local.consume().request_broadcast("a/x", None).await, + Err(Error::Unroutable) + )); + } +} diff --git a/rs/moq-net/tests/integration/main.rs b/rs/moq-net/tests/integration/main.rs index 58f5881ff01..1001d50ea54 100644 --- a/rs/moq-net/tests/integration/main.rs +++ b/rs/moq-net/tests/integration/main.rs @@ -32,6 +32,7 @@ mod history_groups; mod idle_cancel_refetch; mod idle_session; mod late_lower_group; +mod lazy_discovery; mod mesh_withdraw; mod model; mod multi_cdn; diff --git a/rs/moq-net/tests/integration/support/harness.rs b/rs/moq-net/tests/integration/support/harness.rs index 73007de13ee..c46b388f920 100644 --- a/rs/moq-net/tests/integration/support/harness.rs +++ b/rs/moq-net/tests/integration/support/harness.rs @@ -96,6 +96,11 @@ pub struct MockConnectOptions { pub server_stats: moq_net::stats::Session, /// Count server driver polls into this counter when supplied. pub server_polls: Option>, + /// Hold an announcement stream over the client's whole subscribe origin for + /// the pair's lifetime, so the client asks the server about everything it may, + /// the way a reader listing everything would. Clear it to test discovery + /// following the readers. + pub client_discovers_all: bool, /// Pause the receiving driver while its transport keeps delivering data. pub server_gate: Option>, } @@ -118,6 +123,7 @@ impl MockConnectOptions { stats: MockStats::default(), server_stats: Default::default(), server_polls: None, + client_discovers_all: true, server_gate: None, } } @@ -132,6 +138,8 @@ pub struct MockPair { pub client_transport: MockSession, /// The server's end of the mock transport, for steering delivery. pub server_transport: MockSession, + /// See [`MockConnectOptions::client_discovers_all`]. + pub client_discovery: Option, } /// Run the MoQ handshake over the mock transport, returning connected sessions. @@ -162,6 +170,11 @@ pub async fn connect_mock(opts: MockConnectOptions) -> MockPair { if let Some(publish) = &opts.client_publish { client = client.with_publisher(publish); } + let client_discovery = opts + .client_subscribe + .as_ref() + .filter(|_| opts.client_discovers_all) + .map(|subscribe| subscribe.consume().announced()); if let Some(subscribe) = opts.client_subscribe { client = client.with_subscriber(subscribe); } @@ -211,6 +224,7 @@ pub async fn connect_mock(opts: MockConnectOptions) -> MockPair { server: server_session, client_transport: transports.0, server_transport: transports.1, + client_discovery, } } diff --git a/rs/moq-relay/tests/integration/goaway_cluster.rs b/rs/moq-relay/tests/integration/goaway_cluster.rs index 921713dea74..3206c7f618f 100644 --- a/rs/moq-relay/tests/integration/goaway_cluster.rs +++ b/rs/moq-relay/tests/integration/goaway_cluster.rs @@ -516,7 +516,8 @@ async fn cluster_diamond_goaway_seamless_failover_inner() { let (_mid_a_upstream_client, mid_a_upstream) = within( "MID-A connects to TOP", connect_once( - mid_a_client.with_subscriber(mid_a_origin.clone()), + // An upstream link: MID-A relays for BOTTOM, so it asks TOP for everything. + mid_a_client.with_subscriber(mid_a_origin.clone().upstream()), top_url.parse().expect("parse top url"), ), ) diff --git a/rs/moq-rtc/Cargo.toml b/rs/moq-rtc/Cargo.toml index 64728015a93..323ebf4f332 100644 --- a/rs/moq-rtc/Cargo.toml +++ b/rs/moq-rtc/Cargo.toml @@ -45,7 +45,7 @@ uuid = { version = "1", features = ["v4"] } # Signs the MESSAGE-INTEGRITY on a forged STUN binding request: str0m rejects a # request without one, and its own SHA1 provider is private. aws-lc-rs = { workspace = true } -# The tests host their broadcasts on `origin::spawn`. -moq-tokio = { workspace = true, features = ["net"] } +# The tests host their broadcasts on `origin::spawn`, and dial a session over TCP. +moq-tokio = { workspace = true, features = ["net", "tcp"] } tokio = { workspace = true, features = ["test-util"] } tracing-test = { workspace = true } diff --git a/rs/moq-rtc/src/server/whep.rs b/rs/moq-rtc/src/server/whep.rs index 694b679540a..46006d4cf5a 100644 --- a/rs/moq-rtc/src/server/whep.rs +++ b/rs/moq-rtc/src/server/whep.rs @@ -25,6 +25,10 @@ pub use crate::server::Response; struct RouterState { server: Server, subscriber: moq_net::origin::Consumer, + /// Watches the whole subscribe scope while the router lives, so a lazily discovering + /// client session asks its peer once rather than once per offered path, which an + /// unauthenticated caller could otherwise name without bound. + _watch: std::sync::Arc, } /// How long WHEP negotiation waits for the broadcast's first catalog snapshot @@ -38,7 +42,11 @@ const CATALOG_TIMEOUT: Duration = Duration::from_secs(5); pub fn router(server: Server, subscriber: moq_net::origin::Consumer) -> Router { Router::new() .route("/{*path}", post(handle).delete(delete)) - .with_state(RouterState { server, subscriber }) + .with_state(RouterState { + _watch: std::sync::Arc::new(subscriber.clone().with_hidden(true).announced()), + server, + subscriber, + }) } #[cfg(feature = "server")] @@ -113,10 +121,14 @@ async fn delete(State(state): State, Path(path): Path) -> S /// header. The caller must run the returned [`Response`] to drive the MoQ->RTP /// session. Mirrors [`whip::accept`](super::whip::accept). /// +/// Each call watches `broadcast` while it resolves. A caller serving many paths from one +/// lazily discovering client session should hold an `announced()` stream over its scope, +/// as [`router`] does, so each call rides that one watch. +/// /// `offer` is the raw SDP body; the caller is responsible for checking the /// `Content-Type: application/sdp` request header. Fails with [`Error::InvalidSdp`] -/// on a malformed offer, and surfaces a not-announced broadcast (or one outside -/// `subscriber`'s scope) as [`Error::Moq`]. +/// on a malformed offer, and surfaces a broadcast still unannounced after 5s (or one +/// outside `subscriber`'s scope) as [`Error::Moq`]. pub async fn accept( server: &Server, subscriber: &moq_net::origin::Consumer, @@ -125,19 +137,7 @@ pub async fn accept( ) -> Result { let offer = sdp::parse_offer(offer)?; - // Resolve the broadcast on the subscribe origin by path. `request_broadcast` resolves an - // announced broadcast immediately and falls back to a dynamic handler; with neither it - // errors and the WHEP client retries (typical). The `Source` resolves any sibling broadcast - // a rendition's catalog `broadcast` field references (e.g. `./source`) the same way. - // - // Bound the wait for resolution + the first catalog: an unannounced broadcast, or an - // announced-but-catalog-less one, would otherwise park this handler forever. - let source = moq_clock::timeout(CATALOG_TIMEOUT, async { - let broadcast = subscriber.request_broadcast(broadcast.as_path(), None).await?; - EgressSource::new(moq_mux::Source::new(subscriber.clone(), broadcast)).await - }) - .await - .map_err(|_| Error::CatalogTimeout)??; + let source = resolve(subscriber, broadcast.as_path()).await?; let codecs = source.catalog_codecs(); if codecs.is_empty() { return Err(Error::NoRenditions); @@ -194,6 +194,29 @@ fn is_sdp(headers: &HeaderMap) -> bool { .unwrap_or(false) } +/// Resolve `path` on the subscribe origin and wait for its first catalog. +/// +/// `routed_broadcast` watches the path while it resolves, which is what makes a lazily +/// discovering client session ask its peer about it; a plain request would be unroutable +/// there. The `Source` resolves any sibling broadcast a rendition's catalog `broadcast` field +/// references (e.g. `./source`) through the same origin. +/// +/// Both waits share one bound, or an unannounced broadcast, or an announced-but-catalog-less +/// one, would park the handler forever. No route by the deadline is +/// [`moq_net::Error::Unroutable`], so the WHEP client sees a 404 and retries (typical). +async fn resolve(subscriber: &moq_net::origin::Consumer, path: moq_net::Path<'_>) -> Result { + let deadline = moq_clock::now() + CATALOG_TIMEOUT; + let broadcast = moq_clock::timeout_at(deadline, subscriber.routed_broadcast(path)) + .await + .map_err(|_| Error::Moq(moq_net::Error::Unroutable))??; + moq_clock::timeout_at( + deadline, + EgressSource::new(moq_mux::Source::new(subscriber.clone(), broadcast)), + ) + .await + .map_err(|_| Error::CatalogTimeout)? +} + #[cfg(feature = "server")] fn status_for(err: &Error) -> StatusCode { match err { @@ -205,3 +228,116 @@ fn status_for(err: &Error) -> StatusCode { _ => StatusCode::INTERNAL_SERVER_ERROR, } } + +#[cfg(test)] +mod tests { + use std::time::Duration; + + use bytes::Bytes; + + use crate::codec::{Bridge, Frame}; + + const TIMEOUT: Duration = Duration::from_secs(10); + + /// A plain (lazily discovering) moq-lite client subscribing from a server that + /// publishes `pub_origin`, over loopback TCP. + async fn lazy_client( + pub_origin: moq_net::origin::Producer, + ) -> ( + moq_net::origin::Producer, + moq_tokio::Connection, + tokio::task::JoinHandle<()>, + ) { + let sub_origin = moq_tokio::origin::spawn(); + + let mut listen = moq_tokio::listen::Config::default(); + listen.tcp.bind = Some("127.0.0.1:0".parse().expect("addr")); + let mut server = listen + .init(Default::default()) + .expect("server") + .listen() + .await + .expect("listen"); + let addr = server.tcp_local_addr().expect("tcp listener bound"); + let accept = tokio::spawn(async move { + let request = server.accept().await.expect("accept"); + let (session, driver) = request + .with_publisher(&pub_origin) + .ok() + .await + .expect("publisher session"); + let _ = driver.await; + let _ = session.closed().await; + }); + + let client = moq_tokio::connect::Config::default() + .init(Default::default()) + .expect("client") + .with_subscriber(sub_origin.clone()) + .with_reconnect(false); + let url: url::Url = format!("tcp://{addr}/").parse().expect("url"); + let connection = moq_clock::timeout(TIMEOUT, client.connect(url).established()) + .await + .expect("connect timed out") + .expect("connect"); + (sub_origin, connection, accept) + } + + /// A plain moq-lite client origin asks its peer only about watched paths, so resolving + /// a WHEP play must watch the broadcast rather than request it blind. + #[tokio::test] + async fn a_broadcast_resolves_through_a_lazy_client_session() { + let pub_origin = moq_tokio::origin::spawn(); + + let mut source = pub_origin.create_broadcast("cam").expect("create broadcast"); + source + .announce(moq_net::origin::Route::default()) + .expect("announce broadcast"); + let catalog = + moq_mux::catalog::Producer::new(&mut source, moq_mux::catalog::Config::default()).expect("create catalog"); + let mut opus = crate::codec::opus::Bridge::new(source, catalog, 48_000, 2).expect("create Opus bridge"); + opus.push(Frame { + timestamp_us: 20_000, + payload: Bytes::from_static(&[0xfc, 0xff, 0xfe]), + }) + .expect("publish packet"); + + let (sub_origin, _connection, accept) = lazy_client(pub_origin).await; + + let resolved = super::resolve(&sub_origin.consume(), moq_net::Path::new("cam")) + .await + .expect("the broadcast resolves"); + assert!(!resolved.catalog_codecs().is_empty()); + + accept.abort(); + } + + /// The router watches its whole subscribe scope, so an offer's lookup rides one + /// request to the peer rather than opening its own for every path a caller names. + #[cfg(feature = "server")] + #[tokio::test] + async fn offers_ride_the_routers_scope_watch() { + let pub_origin = moq_tokio::origin::spawn(); + let other = pub_origin.create_broadcast("other").expect("create broadcast"); + other + .announce(moq_net::origin::Route::default()) + .expect("announce broadcast"); + let (sub_origin, _connection, accept) = lazy_client(pub_origin).await; + + let _router = super::router(crate::server::Server::new(Default::default()), sub_origin.consume()); + let missing = sub_origin.consume().follow("missing").expect("follow"); + moq_clock::timeout(TIMEOUT, missing.synced()) + .await + .expect("sync timed out"); + + // Nothing watches `other` on its own, yet the synced lookup already discovered it. + sub_origin + .consume() + .request_broadcast("other", None) + .await + .expect("discovered through the scope watch"); + + drop(other); + accept.abort(); + } +} diff --git a/swift/MIGRATE.md b/swift/MIGRATE.md index 01b84eb5db6..3402b613c65 100644 --- a/swift/MIGRATE.md +++ b/swift/MIGRATE.md @@ -4,6 +4,13 @@ How to update your code for each breaking change, newest first. ## Unreleased +### A client asks only about what it watches (#6165) + +On moq-lite, a client session asks its relay for announcements only under paths something watches, through `announced()` or `announcedBroadcast()`, instead of under its whole scope at connect. +`OriginConsumer.requestBroadcast()` watches nothing, so on a client it fails with an unroutable error for a path nothing watches, even when the relay announces it. +Resolve the path with `announcedBroadcast()`, or hold `announced()` over its prefix while requesting. +moq-transport and moq-lite-03/04 sessions are unchanged. + ### `ContainerFormat.fmp4` unwraps to Legacy (#6250) A container import with `ContainerFormat.fmp4` now publishes one Legacy frame per sample instead of one CMAF frame per fragment. diff --git a/swift/ffi/MIGRATE.md b/swift/ffi/MIGRATE.md index f52e36a3a93..e50bf21fcd7 100644 --- a/swift/ffi/MIGRATE.md +++ b/swift/ffi/MIGRATE.md @@ -4,6 +4,14 @@ How to update your code for each breaking change, newest first. ## Unreleased +### A client asks only about what it watches (#6165) + +On moq-lite, a client session asks its relay for announcements only under paths something watches, through `announced()` or `announcedBroadcast()`, instead of under its whole scope at connect. +`MoqOriginConsumer.requestBroadcast()` watches nothing, so on a client it fails with an unroutable error for a path nothing watches, even when the relay announces it. +Resolve the path with `announcedBroadcast()`, or hold `announced()` over its prefix while requesting. +moq-transport and moq-lite-03/04 sessions are unchanged. +Regenerate the bindings for the changed `request_broadcast` checksum. + ### Catalog references carry their epoch `MoqVideo` and `MoqAudio` gain `epoch`, the publisher instance their `broadcast` reference names; the memberwise initializer defaults it to `nil`.