Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
0e39990
chore(quest): claim quest/m1/poll-first/async-feature.md
kixelated Nov 1, 2026
b536715
chore(bench): spike async wrapper generators before rollout
kixelated Nov 1, 2026
a7c1c7c
chore(bench): validate spike failures and recording metadata
kixelated Nov 1, 2026
608408a
chore(bench): measure the pinned compiler without host cache shims
kixelated Nov 1, 2026
d27e368
chore(bench): invalidate a method body for incremental measurements
kixelated Nov 1, 2026
2c3a5dd
fix(bench): delimit declarative type fragments and verify before timing
kixelated Nov 1, 2026
263e56b
fix(bench): inspect broadcast request-handle documentation
kixelated Nov 1, 2026
91b1ec6
docs(quest): record generator measurements and the open maintainer ch…
kixelated Nov 1, 2026
669bf07
Merge remote-tracking branch 'origin/main' into worktree-agent-a4f323…
kixelated Nov 1, 2026
ebd7b35
feat(kio)!: generate async wrappers behind an async feature
kixelated Nov 1, 2026
befa9f0
fix(sh): brace the allowed name in the coroutine scan's pattern
kixelated Nov 1, 2026
398809e
Merge remote-tracking branch 'origin/main' into worktree-agent-a4f323…
kixelated Nov 1, 2026
802507f
Merge remote-tracking branch 'origin/main' into iterate-7506
kixelated Nov 1, 2026
4b0e877
fix(kio-macros): reject a wrapper name that cannot be an identifier
kixelated Nov 1, 2026
1f79714
fix(auth): enable kio's async feature for Shared::wait
kixelated Nov 1, 2026
1702e94
Merge origin/main into quest/m1/poll-first/async-feature
kixelated Nov 1, 2026
8d61f77
quest land: merge tested base
Nov 1, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 36 additions & 9 deletions quest/m1/poll-first/async-feature.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,34 @@ and the bindings and JS keep their APIs.
Decided with the maintainer on 2026-10-25: poll is canonical, async is the
ergonomic layer, and the gating proceeds regardless of the build measurement.

- **Spike the generator first** on kio and one moq-net module, then pick one:
- a `macro_rules!` over a list of signatures (no proc-macro cost);
- a proc macro attribute on the `poll_*` method or its impl block;
- a rust2-driver lowering.
Compare compile time (clean and edit, on moq-net), error spans, and docs output.
A wrapper's docs come from its `poll_*` method.
Decided with the maintainer on 2026-11-01, from the
[generator spike](https://github.com/moq-dev/moq/blob/91b1ec62e1e35295ef74d0f7cee1afc49739bf00/bench/async-wrappers/RESULTS.md):
the generator is a method attribute in kio-macros, `#[kio::wrapper]`. The poll
stays ordinary Rust, the wrapper's name, signature, and docs are derived from it,
and a standalone kio build with `async` pays the measured proc-macro cost (about
3.5 seconds clean).

Done in [moq-dev/moq#7506](https://github.com/moq-dev/moq/pull/7506):
`#[kio::wrapper]` with its contract in kio-macros' behavior and UI tests; kio's
default-off `async` feature, under which kio compiles no coroutines; moq-net's
`async` enabling `kio/async`; moq-net's four `broadcast` wrappers generated; and
`just rs coroutines`, in the nightly feature matrix, which scans the MIR of kio's
and moq-net's poll-only builds. Callers gate the attribute with
`#[cfg_attr(feature = "async", kio::wrapper)]`, so a poll-only build skips the
proc macro, and the scan catches a wrapper left ungated.

What remains:

- **moq-net's remaining wrappers:** about 47 hand-written
`kio::wait(|waiter| self.poll_x(waiter, ..)).await` wrappers become the attribute
on their polls, each poll's docs rewritten to read for both forms. A few stay
hand-written: those that take a closure by value and poll it by `&mut`, and the
`#[cfg(test)]` helpers the poll-only tests use.
- **moq-net's poll-only coroutines:** `auth::until` (behind `Grant::guard`), and
`model::origin`'s `run_front` and `serve_front`, still compile without `async`.
Gate `until` with its async callers, or make it a poll, and turn the front
driver into a hand-written poll machine (its body is one large `kio::wait`).
Then drop their `--allow`s from `just rs coroutines`.
- **Logic in `async fn` bodies:** an `async fn` that does more than call one
`poll_*` becomes a `poll_*` first, or moves out of the library to the edges. List
each one in the PR. moq-net's straight-line ones became `#[kio::machine]`s in
Expand All @@ -28,14 +50,19 @@ ergonomic layer, and the gating proceeds regardless of the build measurement.
bodies with a loop or a branch around a wait (about 59 in moq-net) are
converted by hand, since the macro stays straight-line only (decided
2026-10-28).
- **Sibling libraries:** the poll-based libraries with hand-written wrappers
(moq-mux, moq-hls, moq-auth, moq-stats, moq-room, moq-json, moq-sign, moq-e2ee,
qmux, and others) gain an `async` feature, generate their wrappers, and join
`just rs coroutines`.
- **Default off:** moq-net's `async` feature is still on by default. Turn it off,
and enable it at the edges: binaries, tests, tokio glue, and the bindings.
- **Tests:** `#[kio::test]` and `#[tokio::test]` bodies stay async, so test
targets enable the feature.
- **Cargo:** feature unification means one dependent enabling `async` enables it
for the whole build. Make sure library-only builds, such as `cargo check -p
moq-net` and the rust2 bench's library-only build, leave it off.
- **Docs:** update `rs/AGENTS.md`'s rule ("New logic is a `poll_*` with an `async`
helper") to say the helper is generated and feature-gated.

Public API: every library crate gains an `async` feature, and its async wrappers
move behind it. Add an Impact migration note for the published crates whenever
a consumer must enable `async`, including with `default-features = false`.
a consumer must enable `async`, including with `default-features = false`, as
`rs/kio/MIGRATE.md` does for kio.
4 changes: 2 additions & 2 deletions rs/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,9 @@ Most crates use a split-handle pattern: a `Producer` writes, any number of `Cons
Two ways to drive things, both backed by `kio`:

- `poll_*` functions take a `&kio::Waiter` and return `Poll<...>`, drivable from any executor or synchronously. Like `Future`, but wakers are cleaned up on drop.
- `async fn` wrappers, usually `kio::wait(|waiter| self.poll_x(waiter))`.
- `async fn` wrappers, `kio::wait(|waiter| self.poll_x(waiter))`, generated by `#[cfg_attr(feature = "async", kio::wrapper)]` on the poll (kio's `macros` feature) and compiled only with the crate's `async` feature. Write the poll's docs for both forms: what it resolves to, not how it registers.

Prefer poll. New logic is a `poll_*` with an `async` helper, not the other way around. `pin!` then polling inside `kio::wait` is a code smell, as is `tokio::spawn` / `select!` inside a library; use `kio` primitives.
Prefer poll. New logic is a `poll_*` with a generated `async` wrapper, not the other way around. A library build without `async` compiles no coroutines; `just rs coroutines` checks kio's and moq-net's. `pin!` then polling inside `kio::wait` is a code smell, as is `tokio::spawn` / `select!` inside a library; use `kio` primitives.

- A `poll_*` that waits on several things in a fixed order is a `#[kio::machine]` (kio's `macros` feature): straight-line code with a `wait!` at each wait, from which it writes the state enum and its transitions. Hand-write the machine when a wait sits in a loop or branch, or when it yields more than once (a stream's `poll_next`). A conditional wait is one whose poll is ready at once.
- Use `ready!(...)` instead of `Poll::Pending => return Poll::Pending`.
Expand Down
10 changes: 10 additions & 0 deletions rs/justfile
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,7 @@ c-tests:
# Compile the full feature matrix, including isolated combinations and their tests.
features:
just rs net-features --all
just rs coroutines
just rs quic-features --all
just rs tokio-features --all
just rs media-features --all
Expand All @@ -226,6 +227,8 @@ features:
just rs test -p moq-cli --no-default-features --features capture,pipewire,websocket -E 'test(capture_window)'
bash sh/local.sh cargo check --locked -p moq-bench --no-default-features
bash sh/local.sh cargo check --locked -p moq-boy --no-default-features
# Alone, since the workspace's `kio/sim` dev-dependencies would unify `kio/async` back in.
bash sh/local.sh cargo check --locked --manifest-path rs/moq-auth/Cargo.toml --lib --all-features
bash sh/local.sh cargo check --locked -p moq-hls -p moq-rtmp -p moq-rtc --no-default-features --all-targets
bash sh/local.sh cargo clippy --locked --workspace {{ no_fuzz }} --all-targets --all-features -- -D warnings
RUSTDOCFLAGS="-D warnings" bash sh/local.sh cargo doc --locked --workspace {{ no_fuzz }} --all-features --no-deps
Expand All @@ -236,6 +239,13 @@ features:
net-features *args:
bash sh/rs/net-features.sh "$@"

# Prove the poll-only library builds compile no coroutines and document cleanly.
# moq-net allows the async fns still waiting on a poll conversion.
coroutines:
bash sh/rs/coroutines.sh kio -- --no-default-features --features macros,runtime,manual,serde
RUSTDOCFLAGS="-D warnings" bash sh/local.sh cargo doc --locked -p kio --no-deps --no-default-features --features macros,runtime,manual,serde
bash sh/rs/coroutines.sh moq-net --allow until --allow run_front --allow serve_front -- --no-default-features

# Prove moq-quic leaves kio's dev-only clock runtime to its consumers, and lints without its default features; a workspace build unifies both back in.
[positional-arguments]
quic-features *args:
Expand Down
20 changes: 13 additions & 7 deletions rs/kio/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,26 +13,32 @@ keywords = ["async", "producer", "consumer", "clock", "simulation"]
categories = ["asynchronous", "concurrency", "date-and-time", "development-tools::testing"]

[package.metadata.docs.rs]
features = ["runtime", "manual", "sim", "serde", "macros"]
features = ["async", "runtime", "manual", "sim", "serde", "macros"]

# The self dev-dependency below only enables features; no code names it.
[package.metadata.cargo-shear]
ignored = ["kio"]

[features]
# Lean by default: without `runtime`, the clock compiles no host timer backend.
# Lean by default: without `runtime`, the clock compiles no host timer backend, and
# without `async`, no coroutines.
default = []
# The `async fn` beside each `poll_*` method, generated by `#[kio::wrapper]`, and the
# few `async` helpers with no poll of their own (`Queue::push`, the `wait` predicates,
# `clock::drive`, `Interval::tick`). Off by default: enable it where you `.await`.
async = ["macros"]
# Host timers on tokio's clock.
runtime = ["dep:tokio"]
# The deterministic single-threaded executor, `#[kio::test]`, and per-task Ready
# poll accounting. Meant for dev-dependencies, so production builds never compile it.
sim = ["dep:futures", "dep:kio-macros"]
sim = ["async", "dep:futures"]
# `clock::Manual::install`: a clock that only a test moves, as one tokio runtime's
# ambient clock. Meant for test builds, so production builds never compile it.
manual = ["runtime"]
# Numeric Unix-second JSON adapters for public wall times.
serde = ["dep:serde", "dep:serde_with"]
# `#[kio::machine]`, which writes a `poll_*` state machine from straight-line code.
# `#[kio::machine]`, which writes a `poll_*` state machine from straight-line code, and
# `#[kio::wrapper]`, which writes the `async fn` that awaits one.
macros = ["dep:kio-macros"]

[dependencies]
Expand All @@ -51,9 +57,9 @@ tokio = { workspace = true, features = ["rt", "time"], optional = true }
# Bench-only: the FuturesUnordered comparison in benches/tasks.rs.
criterion = { package = "bench-harness", path = "../bench-harness" }
futures = "0.3"
# Enable the simulator and the installed manual clock so the clock's tests run in
# the merge gate.
kio = { path = ".", features = ["manual", "sim", "serde"] }
# Enable the async wrappers, the simulator, and the installed manual clock so the
# tests that await them run in the merge gate.
kio = { path = ".", features = ["async", "manual", "sim", "serde"] }
serde_json = { workspace = true }
test-env = { path = "../test-env" }
tokio = { workspace = true, features = ["macros", "rt", "rt-multi-thread", "test-util", "time"] }
Expand Down
8 changes: 8 additions & 0 deletions rs/kio/MIGRATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,14 @@ How to update your code for each breaking change, newest first.

## Unreleased

### `async` methods moved behind the `async` feature

`Queue::{push, pop}`, `Producer::{wait, closed, wanted_changed}`, `Consumer::{wait,
closed}`, `Shared::wait`, `ConsumerWeak::closed`, `ProducerWeak::closed`,
`clock::drive`, and `clock::Interval::tick` now compile only with the new `async`
feature, which is off by default; `sim` enables it. Add `features = ["async"]` to
the `kio` dependency of each crate that `.await`s them. Each `poll_*` method is unchanged, and still drives the same wait without it.

### `Pollable` merged into `Task`, and unused surface removed (#4156)

`Fan`, `Hold`, `WeakLock`, `Pollable`, and the public `Lock::downgrade` are gone.
Expand Down
62 changes: 62 additions & 0 deletions rs/kio/macros/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,68 @@ pub fn machine(args: proc_macro::TokenStream, item: proc_macro::TokenStream) ->
}

mod sim_test;
mod wrapper;

/// Generate the `async fn` that waits on a `poll_*` method.
///
/// On `poll_<name>`, it adds `async fn <name>` beside it, `kio::wait`ing on the poll.
/// The wrapper takes the poll's receiver and arguments, minus the waiter, and returns
/// what the poll is ready with. It copies the poll's visibility, generics, docs, `cfg`,
/// and deprecation, then links back to the poll. The poll stays as written, so its
/// docs describe both forms: what the call is ready with, not how it registers.
///
/// ```
/// use std::task::Poll;
///
/// struct Counter {
/// values: kio::Queue<u64>,
/// }
///
/// impl Counter {
/// /// The next value at or above `min`, skipping smaller ones.
/// #[kio::wrapper]
/// fn poll_at_least(&self, waiter: &kio::Waiter, min: u64) -> Poll<Result<u64, kio::Closed>> {
/// loop {
/// match self.values.poll_pop(waiter) {
/// Poll::Ready(Ok(value)) if value < min => continue,
/// other => return other,
/// }
/// }
/// }
/// }
///
/// # async fn demo(counter: Counter) -> Result<u64, kio::Closed> {
/// // Generated: `async fn at_least(&self, min: u64) -> Result<u64, kio::Closed>`.
/// counter.at_least(10).await
/// # }
/// ```
///
/// A library keeps its wrappers behind its `async` feature, and its poll-only build
/// free of this macro, with `#[cfg_attr(feature = "async", kio::wrapper)]`.
///
/// The poll must:
///
/// - Be a method taking `&self` or `&mut self`, not `async`, `const`, `unsafe`, or
/// `extern`.
/// - Take exactly one `&Waiter` (any path ending in `Waiter`), in any position.
/// - Bind every other argument to a plain name (`mut` is dropped). The wrapper passes
/// each one to every poll unchanged, so it must be `Copy` or a reference; rustc
/// reports the move from the closure otherwise.
/// - Return `Poll<T>` (any path ending in `Poll`). The wrapper returns `T`.
///
/// Other attributes, such as `#[track_caller]` or `#[inline]`, stay on the poll alone.
#[proc_macro_attribute]
pub fn wrapper(args: proc_macro::TokenStream, item: proc_macro::TokenStream) -> proc_macro::TokenStream {
let method = parse_macro_input!(item as syn::ImplItemFn);
// A rejected poll still compiles, so the error is the only one reported.
wrapper::expand(args.into(), &method)
.unwrap_or_else(|err| {
let mut out = method.to_token_stream();
out.extend(err.into_compile_error());
out
})
.into()
}

/// Run an `async fn` test to completion on `kio::clock::sim::run`.
#[proc_macro_attribute]
Expand Down
Loading