Repository navigation
quest: plan routes as the origin API #7420
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
62c2832
quest: plan routes as the origin API
kixelated 3d0a9cb
quest: address review on the route epic
kixelated e23efda
quest: routes own their broadcasts, identity is one stream
kixelated dc300c9
quest: keep publisher identity apart from route-layer node identity
kixelated f844fcb
Merge remote-tracking branch 'origin/main' into quest-plan/route-orig…
kixelated 25ed72c
quest: plan the route epic follow-ups
kixelated 1ef50da
Merge origin/main: identity replaces the landed ANNOUNCE_TAKEOVER
kixelated fd067f0
quest land: merge tested base
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,61 +1,132 @@ | ||
| # [M] Announcements take over a claim's existing instances | ||
| # [XL] A job's exact route continues its claim's readers | ||
|
|
||
| ## Goal | ||
|
|
||
| A worker advertises an epochless prefix to accept lazy work, then announces | ||
| an independent, more-specific route with a unique job epoch without resetting | ||
| readers already served by that same job. A different worker or replacement | ||
| job never inherits those readers. Rust and JS expose the transition as | ||
| `Takeover`, distinct from `Restart`. | ||
| A worker advertises a prefix route to accept lazy work. When a request | ||
| starts a job, the job's exact route carries the prefix route's identity, so | ||
| a reader first served through the claim continues on the exact route with | ||
| no cold-start reset, across relays, on lite-07. A different worker, or a | ||
| restarted job, never continues those readers: each is another identity. | ||
| Readers holding `follow` or `request` simply continue, and see a new | ||
| broadcast only when the identity changes. Rust and JS players keep playing | ||
| through a claim. | ||
|
|
||
| The ANNOUNCE_TAKEOVER work that landed in | ||
| [moq-dev/moq#6994](https://github.com/moq-dev/moq/pull/6994) is removed: the | ||
| lite-07 ANNOUNCE_TAKEOVER message (0x5), the origin, front and relay | ||
| handling of predecessors, the `Takeover { announce, predecessor }` event, the | ||
| predecessor fields in moq-ffi and the C ABI, and the takeover text in the | ||
| lite draft and the concept docs. No wire message or public event replaces it. | ||
|
|
||
| Non-goals: moq-transport, which keeps the cold-start reset | ||
| ([moq-transport carries no epoch](/quest/m1/ietf-epoch-cut.md), and an | ||
| edge mints a new identity for each announcement, | ||
| [Identity and epoch](/quest/m1/api-audit/route/identity.md)). | ||
|
|
||
| ## Plan | ||
|
|
||
| Rust, the lite-07 wire (ANNOUNCE_TAKEOVER, 0x5), the lite draft, the concept | ||
| docs, and the FFI bindings landed in | ||
| [moq-dev/moq#6994](https://github.com/moq-dev/moq/pull/6994): the origin | ||
| records which incarnation a route continues at install, fronts and session | ||
| selections follow each takeover as it lands (so a chain of them too), | ||
| cursors deliver `Takeover` only where every route they saw at the | ||
| predecessor's prefix serves that incarnation, and relays forward it on that | ||
| same condition, falling back to Start or Restart. The identity guide | ||
| describes it. | ||
| Keep the rest in m1 as a prerequisite of the lite-07 cut. moq-transport gets | ||
| no counterpart, since [moq-transport carries no epoch](/quest/m1/ietf-epoch-cut.md) | ||
| (decided 2026-10-28), so a moq-transport hop uses Start/Restart. | ||
|
|
||
| Decided 2026-10-29 (maintainer, settling #6994): the event stays | ||
| `Takeover { announce, predecessor }`, with `predecessor == announce.prefix` | ||
| in place; the C ABI's `moq_announce_event` carries the predecessor fields | ||
| rather than gaining an accessor; and a reader whose cursor saw a takeover | ||
| withdrawn before delivery gets the claim's later answers as Start/Restart. | ||
| A chain of in-place takeovers landing before a reader looks again is fixed | ||
| in that PR, without keeping history of retired routes. | ||
| JS continuity and epoch-bound requests through an adopted source moved here. | ||
|
|
||
| What remains: | ||
|
|
||
| - JS continuity. `js/net` decodes ANNOUNCE_TAKEOVER and keeps the Announce | ||
| IDs in step, but surfaces it as a `start` (or a `restart` at the | ||
| predecessor's own prefix), as an older wire would. Add a `takeover` kind | ||
| carrying the predecessor's prefix to `announced.Event`, and make the lite | ||
| subscriber's `BroadcastCache` re-key a consumer resolved through the | ||
| predecessor's current advertisement, beneath the new prefix, under the new | ||
| route instead of invalidating it. Handle the kind in | ||
| `connection/forward.ts` and any player that follows announcements, so a | ||
| Re-planned 2026-10-30 (maintainer, the [route epic](/quest/m1/api-audit/route/README.md)): | ||
| continuity is the identity promise, not an `ANNOUNCE_TAKEOVER` predecessor | ||
| ID. An identity is one continuous sequence space at a path, so the exact | ||
| route and the claim's answer are the same consumer route, | ||
| origin → route(path, identity) → broadcast. The 2026-10-10 plan (TAKEOVER as | ||
| START plus a predecessor Announce ID, with per-stream ID translation in | ||
| relays) is dropped. Its safety requirements carry over, as follows. | ||
|
|
||
| Decided 2026-10-31 (maintainer, merging main into | ||
| [moq-dev/moq#7420](https://github.com/moq-dev/moq/pull/7420)): identity | ||
| wins over the predecessor design that landed in | ||
| [moq-dev/moq#6994](https://github.com/moq-dev/moq/pull/6994) meanwhile. | ||
| Keeping both would give two continuity rules, one by identity and one by | ||
| predecessor prefix, that every origin, relay and binding must agree on. The | ||
| identity rule already covers the claim case and needs no wire message, so | ||
| this quest removes the predecessor machinery and grows to [XL]. | ||
|
|
||
| - **Removing ANNOUNCE_TAKEOVER.** Delete the 0x5 message and its codec in | ||
| `rs/moq-net/src/lite/announce.rs`, `compress.rs`, `publisher.rs`, | ||
| `subscriber.rs` and `replay.rs`; the predecessor tracking in | ||
| `rs/moq-net/src/model/origin.rs` and `front.rs` (the `seen` stamps in | ||
| `OriginConsumerState` and `PendingFront::via`); relay forwarding in | ||
| `rs/moq-relay`; and `rs/moq-net/tests/integration/announce_takeover.rs`. | ||
| Remove the `Takeover` event from Rust (unless | ||
| [Listing and follow](/quest/m1/api-audit/route/events.md) has already | ||
| removed the event kinds), `TAKEOVER` and the predecessor fields from | ||
| `MoqAnnounceEvent` in `rs/moq-ffi/src/origin.rs` and from | ||
| `moq_announce_event` in `rs/moq-c/src/api.rs`, and their wrapper and | ||
| binding-doc mentions. In `js/net`, drop the ANNOUNCE_TAKEOVER decoding in | ||
| `lite/announce.ts` and `lite/subscriber.ts`. Remove the takeover sections | ||
| from `drafts/draft-lcurley-moq-lite.md`, `doc/concept/identity.md` and | ||
| `doc/concept/moq-lite.md`. | ||
| - **Inheriting the identity.** `request.route(options)` from | ||
| [One Route handle](/quest/m1/api-audit/route/handle.md) creates the | ||
| path's exact route under the prefix route's identity by default, and the | ||
| answer it gives the waiting request is that route's broadcast. A job | ||
| that serves a held broadcast through [serve](/quest/m1/api-audit/route/serve.md) | ||
| never announces an exact route, so it needs none of this. | ||
| - **One stream per path and identity.** A claim may answer a path under | ||
| its identity only while that stream is live. Once the job's route ends, | ||
| the next request for that path gets a fresh identity, because its | ||
| sequences restart. A request pinned to the old identity is refused as | ||
| ended, never answered with new bytes. This is the old rule "a worker | ||
| restarting a job under an unchanged claim cannot relabel the previous | ||
| job's cached content", now enforced in the origin. Prove it in the | ||
| model, including retained generations and pending answers, before | ||
| relying on it. | ||
| - **Relays.** An aggregated prefix can be served by several workers, but | ||
| each worker has its own identity. A relay can therefore only continue a | ||
| reader on routes carrying the identity it resolved, and readers of A | ||
| never move to B. A relay that cannot keep the identity, such as an | ||
| older-lite hop, mints a new one, so its readers see a new broadcast | ||
| rather than a splice. | ||
| - **What a continued reader keeps.** Handles, group positions, metadata, | ||
| object and negative caches, catalog pins, and outstanding | ||
| TRACK/SUBSCRIBE/FETCH requests. Unrelated sibling paths are unchanged. A | ||
| continued stream does not renew a composed catalog. | ||
| - **JS players.** `js/net`'s lite subscriber continues a consumer resolved | ||
| through the claim on the exact route with the same identity instead of | ||
| invalidating it, in `BroadcastCache` and `connection/forward.ts`, so a | ||
| `@moq/watch` reader through a claim keeps playing. | ||
| - Epoch-bound requests through an adopted source. A Rust front or selection | ||
| that adopts a takeover keeps the epochless upstream SUBSCRIBE it already | ||
| sent; a later request naming the new epoch joins it without restamping. | ||
| This is accepted for now, since the answering worker is the instance by | ||
| construction. Decide whether that join needs the upstream's answer to | ||
| confirm the epoch. | ||
| - Tests not yet written, with controlled time: a SUBSCRIBE or FETCH pending | ||
| at the front between its resolve and the takeover (`PendingFront::via`), | ||
| a takeover naming a predecessor already restarted (the `seen` stamps in | ||
| `OriginConsumerState`), conflicting epochs, a diamond of | ||
| relays, a claim ending before or after its child, and held catalog pins | ||
| through a real player rather than route-table checks. | ||
| - **Epoch-bound requests through a continued route.** A front or selection | ||
| that continues on the exact route keeps the upstream SUBSCRIBE it already | ||
| sent under the claim's route; a later request naming the exact route's | ||
| epoch joins it without restamping. This is accepted while the answering | ||
| worker is the same identity by construction. Decide whether that join | ||
| needs the upstream's answer to confirm the epoch. | ||
| - **Docs.** Update the lazy-job lifecycle in `doc/concept/identity.md`: | ||
| the cold-start step no longer resets on lite-07. Also update the lite | ||
| draft's announce section where it mentions claims. | ||
|
|
||
| Tests with controlled time cover: | ||
| - the first reader before and after the exact route lands, with no reader | ||
| at all, and with two workers at one prefix; | ||
| - a job restarting on one worker, and a request pinned to an ended stream; | ||
| - pending TRACK/SUBSCRIBE/FETCH at the front between its resolve and the | ||
| exact route landing, and late groups across the transition; | ||
| - conflicting epochs within one identity; | ||
| - independently ending prefix and exact routes (a claim ending before or | ||
| after its child), and late snapshots; | ||
| - multi-hop and diamond relays, and mixed versions with an older-lite hop; | ||
| - held catalog and media pins through a real player rather than route-table | ||
| checks, and the composer. | ||
|
|
||
| Remove the cold-start warm-up reads (`claimed` in | ||
| `rs/moq-net-tests/tests/integration/restart.rs` and the | ||
| `rs/moq-net/tests/integration/drained_claim.rs` helper) for lite-07 and | ||
| assert that the first subscription succeeds. Keep them for moq-transport. | ||
| Run `just check`, `just test interop --all` and `just drafts check`. | ||
|
|
||
| Public API: breaking; removes `Takeover` and the predecessor fields in every | ||
| language. Wire: removes lite-07's ANNOUNCE_TAKEOVER (0x5), adding nothing | ||
| beyond [Identity and epoch](/quest/m1/api-audit/route/identity.md)'s fields. | ||
|
|
||
| ## Required | ||
|
|
||
| - [Identity and epoch](/quest/m1/api-audit/route/identity.md) - the identity a job's exact route inherits | ||
| - [One Route handle](/quest/m1/api-audit/route/handle.md) - `request.route` creates the exact route under the claim's identity | ||
|
|
||
| ## Related | ||
|
|
||
| - [Routes and announces](/quest/m1/cluster-routing/routes.md) - the route layer whose ANNOUNCE_START layout overlaps this one's | ||
| - [Listing and follow](/quest/m1/api-audit/route/events.md) - continuity shows up inside `follow`, not as an event | ||
| - [Claim answer epochs](https://github.com/moq-dev/moq/pull/5678) - why a restarted job must never be stitched to its predecessor | ||
| - [Retired requests](https://github.com/moq-dev/moq/pull/5294) - replacements never substitute content into held handles | ||
| - [Routes and announces](/quest/m1/cluster-routing/routes.md) - the route layer whose ANNOUNCE_START layout carries the identity |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| # [S] JS follow takes FollowOptions | ||
|
|
||
| ## Goal | ||
|
|
||
| In `@moq/net`, `Origin.Consumer.follow` (and the `Producer` and `Table` | ||
| passthroughs) takes its own `FollowOptions`, and `request` keeps | ||
| `RequestOptions`. Each options type documents only what its method does. | ||
| Every caller in `js/` compiles against the split, and the API report is | ||
| regenerated. | ||
|
|
||
| ## Plan | ||
|
|
||
| Planned 2026-10-31 (`/quest-plan`, maintainer), a follow-up from | ||
| [JS follow returns a Following](/quest/m1/api-audit/resolve/js-following.md), | ||
| which split the return types but kept one `RequestOptions` for both | ||
| methods. Keep the shared fields' names. Drop a field from `FollowOptions` | ||
| only if `follow` ignores it today, and say which in the PR. If the two | ||
| types still end up identical, a shared base with two aliases is fine; the | ||
| point is that each method's docs describe one behavior. | ||
|
|
||
| Public API: breaking for TypeScript type annotations that name | ||
| `RequestOptions` for a `follow` call. Wire: none. | ||
|
|
||
| ## Required | ||
|
|
||
| - [JS follow returns a Following](/quest/m1/api-audit/resolve/js-following.md) - the `Following` return type this finishes |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| # [S] JS follow returns a Following | ||
|
|
||
| ## Goal | ||
|
|
||
| In `@moq/net`, `Origin.Consumer.follow(path, ...)` returns its own | ||
| `Following` type, matching Rust and every binding, and `request` keeps | ||
| `Requesting`. Each type documents one meaning. js/watch's annotation | ||
| (`js/watch/src/broadcast.ts:284`) and the other callers compile against it. js/watch's cache | ||
| entry (`js/watch/src/broadcast.ts:172`) and the local at `:285` hold either | ||
| result, so they take a shared read-side type (the common `active`/`closed` | ||
| shape) or `Requesting | Following`. | ||
|
|
||
| ## Plan | ||
|
|
||
| Found by the 2026-10-30 origin API audit | ||
| ([route epic](/quest/m1/api-audit/route/README.md)). Today both methods | ||
| return `Requesting`, whose docs (`js/net/src/origin.ts:1624-1650`) describe | ||
| both behaviors. Keep the `active`/`closed` shape, and rename only where a | ||
| member means something different for `follow`. A local helper in | ||
| `js/net/src/connection/forward.ts:121` is also named `follow` but | ||
| re-receives a route; rename it so it stops reading as `Consumer.follow`. | ||
|
|
||
| Public API: breaking for TypeScript type annotations only. Wire: none. |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.