From 0e399906c9bad87fed81d1f2d4facbd4853a6fc5 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 31 Oct 2026 23:42:17 -0700 Subject: [PATCH 01/12] chore(quest): claim quest/m1/poll-first/async-feature.md Quest-Claim: bdbdb42af8e85df3 From b53671512a536485285aa8955048c9a4f157f9d9 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 31 Oct 2026 23:48:50 -0700 Subject: [PATCH 02/12] chore(bench): spike async wrapper generators before rollout Co-Authored-By: GPT-6.1 Sol --- bench/async-wrappers/README.md | 91 ++++++++++++ bench/async-wrappers/attribute.rs | 118 ++++++++++++++++ bench/async-wrappers/rules.rs | 26 ++++ bench/async-wrappers/run.py | 203 +++++++++++++++++++++++++++ quest/m1/poll-first/async-feature.md | 4 + 5 files changed, 442 insertions(+) create mode 100644 bench/async-wrappers/README.md create mode 100644 bench/async-wrappers/attribute.rs create mode 100644 bench/async-wrappers/rules.rs create mode 100644 bench/async-wrappers/run.py diff --git a/bench/async-wrappers/README.md b/bench/async-wrappers/README.md new file mode 100644 index 00000000000..8aa9b42fc7d --- /dev/null +++ b/bench/async-wrappers/README.md @@ -0,0 +1,91 @@ +# Async-wrapper generator spike + +This checkpoint compares generation on `kio::Queue::pop` and all four wrappers in +`moq_net::broadcast`: `Producer::{requested_track, closed}`, `Consumer::closed`, +and `Demand::wanted_changed`. It does not pick a generator or change production +crates. The remaining work belongs to +[async-feature](/quest/m1/poll-first/async-feature.md). + +Run in the repository's Nix shell: + +```sh +quest gate python3 bench/async-wrappers/run.py --rounds 3 +``` + +Use `--output .tmp/another-run` to keep earlier results. The runner refuses to +reuse variant directories. `--verify-only` skips timings on a fresh snapshot. +The source archive comes from `HEAD`, while the generator fixtures come from +beside the runner. Commit before measuring so the checkpoint is reproducible. +It requires the workspace lockfile's Cargo dependencies in the local cache; +`--offline` refuses a missing dependency instead of including downloads in timings. + +The runner creates source snapshots under `.tmp/`, never edits the checkout, +and gives each variant its own target directory. It compares: + +- **Handwritten:** the same five wrappers with poll docs copied to them, gated + behind `async`. This controls for equivalent generated code and documentation. +- **Declarative:** `rules.rs` replaces each poll declaration with one macro + invocation containing its docs, signature, and body. The explicit `poll => async` + names avoid an identifier-concatenation dependency. The bracketed attributes + apply only to the poll method, preserving `_sim`'s `track_caller`. +- **Attribute:** `attribute.rs` is appended to the existing kio-macros crate, and + re-exported through kio only in the snapshots. A method attribute derives the + wrapper name, return type, receiver, visibility, and docs from ordinary Rust. + `cfg_attr(feature = "async", ...)` avoids building the generator in a standalone + poll-only kio build. moq-net already enables kio's `macros` feature for machines. + +Both prototypes intentionally support only safe, non-generic methods taking a +borrowed receiver and waiter. The enclosing impl may be generic (`Queue`). +They are comparison fixtures, not ready-to-publish macro APIs. Generic methods, +extra arguments, explicit lifetimes, conditional method attributes, renamed +crate dependencies, and composition with `#[kio::machine]` still need design +and coverage after the maintainer chooses. An impl-level attribute would avoid +repeating `cfg_attr`, but would parse entire impls and need explicit opt-in for +which methods are wrappers. It is not measured here. + +## Method + +Three rounds rotate the variant order. Each round measures wall time for: + +1. A clean standalone `cargo check --lib -p kio --no-default-features --features + kio/async`, exposing the proc-macro dependency cost. +2. A fresh-target `cargo check --lib -p moq-net --no-default-features --features + moq-net/async`, including its dependency graph. +3. The same moq-net command after a comment edit in `model/broadcast.rs`, with + incremental compilation enabled. This measures a recompilation of the changed + module, not an idle Cargo check or an algorithm change. + +The runner disables compiler wrappers and custom Rust flags, uses two Cargo +jobs for every variant, and records all samples plus toolchain and source SHA in +`results.json`. OS file caches remain warm, and host contention can affect wall +time. This small spike measures generator overhead for five wrappers, not the +savings from eliminating every coroutine in the quest. Gating remains the +maintainer's settled direction regardless of speed. + +Verification checks feature-on and feature-off library builds, Rustdoc with +warnings denied, and identical rendered poll/wrapper docs for `Queue::pop` and +`Producer::requested_track`. It also compiles an owned-receiver mistake and +requires failure; inspect `*-bad-receiver.log` for error spans. Other wrappers +and unrelated kio async bodies remain untouched, so a feature-off success here +does **not** establish the quest's eventual no-coroutines condition. + +## rust2-driver + +Upstream was inspected at +[06614b5a4577a9230206a5babf75c592b1e07efc](https://github.com/moq-dev/rust2/tree/06614b5a4577a9230206a5babf75c592b1e07efc). +Its [front end](https://github.com/moq-dev/rust2/blob/06614b5a4577a9230206a5babf75c592b1e07efc/driver/src/lib.rs) +and [compiler callbacks](https://github.com/moq-dev/rust2/blob/06614b5a4577a9230206a5babf75c592b1e07efc/driver/src/rustc.rs) +provide compiler integration, not async-wrapper generation. The driver uses +`rustc_private` and its own nightly toolchain; moq pins stable 1.98.1. A new +lowering would therefore be a compiler-toolchain project, including distributing +it to library consumers and supporting rustdoc and IDEs. No working lowering +was available to time or inspect for wrapper diagnostics/docs. This is an +unmeasured option requiring a larger spike, not a zero-cost measured alternative. + +Rust's stable +[attribute macros](https://doc.rust-lang.org/reference/procedural-macros.html#attribute-macros) +can emit adjacent methods and preserve input spans. Stable +[declarative macros](https://doc.rust-lang.org/reference/macros-by-example.html) +can also emit both methods, but consume a dialect rather than annotate a normal +method. Both fixtures copy docs from the poll declaration instead of maintaining +a second signature/doc list. diff --git a/bench/async-wrappers/attribute.rs b/bench/async-wrappers/attribute.rs new file mode 100644 index 00000000000..4b85074c354 --- /dev/null +++ b/bench/async-wrappers/attribute.rs @@ -0,0 +1,118 @@ +// Appended only to the snapshot's existing kio-macros crate. +#[proc_macro_attribute] +pub fn wrapper( + args: proc_macro::TokenStream, + item: proc_macro::TokenStream, +) -> proc_macro::TokenStream { + fn expand( + args: proc_macro2::TokenStream, + method: syn::ImplItemFn, + ) -> syn::Result { + if !args.is_empty() { + return Err(syn::Error::new_spanned(args, "wrapper takes no arguments")); + } + let sig = &method.sig; + let name = sig.ident.to_string(); + let Some(name) = name.strip_prefix("poll_").filter(|name| !name.is_empty()) else { + return Err(syn::Error::new_spanned( + &sig.ident, + "wrapper requires a poll_* method", + )); + }; + let Some(syn::FnArg::Receiver(receiver)) = sig.inputs.first() else { + return Err(syn::Error::new_spanned( + &sig.inputs, + "wrapper requires a borrowed receiver", + )); + }; + if receiver.reference.is_none() || receiver.colon_token.is_some() { + return Err(syn::Error::new_spanned( + receiver, + "wrapper requires a borrowed receiver", + )); + } + let Some(syn::FnArg::Typed(waiter)) = sig.inputs.iter().nth(1) else { + return Err(syn::Error::new_spanned( + &sig.inputs, + "wrapper requires &Waiter after self", + )); + }; + let syn::Type::Reference(waiter_ref) = waiter.ty.as_ref() else { + return Err(syn::Error::new_spanned( + &waiter.ty, + "wrapper requires &Waiter after self", + )); + }; + let syn::Type::Path(waiter_path) = waiter_ref.elem.as_ref() else { + return Err(syn::Error::new_spanned( + &waiter.ty, + "wrapper requires &Waiter after self", + )); + }; + if waiter_ref.mutability.is_some() + || !waiter_path.path.is_ident("Waiter") + && waiter_path + .path + .segments + .last() + .is_none_or(|segment| segment.ident != "Waiter") + { + return Err(syn::Error::new_spanned( + &waiter.ty, + "wrapper requires &Waiter after self", + )); + } + let syn::ReturnType::Type(_, ty) = &sig.output else { + return Err(syn::Error::new_spanned( + &sig.output, + "wrapper requires Poll", + )); + }; + let syn::Type::Path(path) = ty.as_ref() else { + return Err(syn::Error::new_spanned(ty, "wrapper requires Poll")); + }; + let segment = path.path.segments.last().unwrap(); + let syn::PathArguments::AngleBracketed(arguments) = &segment.arguments else { + return Err(syn::Error::new_spanned(ty, "wrapper requires Poll")); + }; + let Some(syn::GenericArgument::Type(output)) = arguments + .args + .first() + .filter(|_| arguments.args.len() == 1 && segment.ident == "Poll") + else { + return Err(syn::Error::new_spanned(ty, "wrapper requires Poll")); + }; + // This checkpoint deliberately refuses more than the receiver and waiter. + if sig.inputs.len() != 2 + || !sig.generics.params.is_empty() + || sig.asyncness.is_some() + || sig.unsafety.is_some() + { + return Err(syn::Error::new_spanned( + sig, + "spike supports safe, non-generic methods with only self and waiter", + )); + } + let mut wrapper = sig.clone(); + wrapper.ident = syn::Ident::new(name, sig.ident.span()); + wrapper.asyncness = Some(Default::default()); + wrapper.inputs = std::iter::once(sig.inputs.first().unwrap().clone()).collect(); + wrapper.output = syn::parse_quote!(-> #output); + let docs = method + .attrs + .iter() + .filter(|attr| attr.path().is_ident("doc")); + let vis = &method.vis; + let poll = &sig.ident; + Ok(quote! { + #method + #(#docs)* + #[cfg(feature = "async")] + #vis #wrapper { ::kio::wait(|waiter| self.#poll(waiter)).await } + }) + } + let method = syn::parse_macro_input!(item as syn::ImplItemFn); + expand(args.into(), method) + .unwrap_or_else(syn::Error::into_compile_error) + .into() +} diff --git a/bench/async-wrappers/rules.rs b/bench/async-wrappers/rules.rs new file mode 100644 index 00000000000..52a9da7baca --- /dev/null +++ b/bench/async-wrappers/rules.rs @@ -0,0 +1,26 @@ +// Spike dialect: docs and signature live once, alongside the canonical poll body. +macro_rules! wrapper { + ($(#[doc = $doc:literal])* [$($attr:tt)*] + $vis:vis fn $poll:ident => $name:ident (&$this:ident, $waiter:ident: &$waiter_ty:ty) + -> $out:ty $body:block) => { + wrapper!(@emit [$(#[doc = $doc])*] [$($attr)*] $vis $poll $name + [&$this] $this $waiter $waiter_ty $out $body); + }; + ($(#[doc = $doc:literal])* [$($attr:tt)*] + $vis:vis fn $poll:ident => $name:ident (&mut $this:ident, $waiter:ident: &$waiter_ty:ty) + -> $out:ty $body:block) => { + wrapper!(@emit [$(#[doc = $doc])*] [$($attr)*] $vis $poll $name + [&mut $this] $this $waiter $waiter_ty $out $body); + }; + (@emit [$($doc:tt)*] [$($attr:tt)*] $vis:vis $poll:ident $name:ident + [$($receiver:tt)*] $this:ident $waiter:ident $waiter_ty:ty $out:ty $body:block) => { + $($doc)* $($attr)* + $vis fn $poll($($receiver)*, $waiter: &$waiter_ty) -> ::std::task::Poll<$out> $body + + $($doc)* + #[cfg(feature = "async")] + $vis async fn $name($($receiver)*) -> $out { + ::kio::wait(|waiter| $this.$poll(waiter)).await + } + }; +} diff --git a/bench/async-wrappers/run.py b/bench/async-wrappers/run.py new file mode 100644 index 00000000000..09809475746 --- /dev/null +++ b/bench/async-wrappers/run.py @@ -0,0 +1,203 @@ +#!/usr/bin/env python3 +"""Materialize and compare wrapper generators without changing production crates.""" +import argparse +import json +import os +from pathlib import Path +import re +import shutil +import statistics +import subprocess +import tarfile +import time + +ROOT = Path(__file__).resolve().parents[2] +HERE = Path(__file__).resolve().parent +OUT = ROOT / ".tmp/async-wrappers" +VARIANTS = ("handwritten", "rules", "attribute") +MODULES = { + "rs/kio/src/queue.rs": ("pop",), + "rs/moq-net/src/model/broadcast.rs": ("requested_track", "closed", "wanted_changed"), +} + + +def command(args, cwd, log, env=None, success=True): + start = time.monotonic() + with log.open("w") as output: + result = subprocess.run(args, cwd=cwd, env=env, stdout=output, stderr=subprocess.STDOUT) + if (result.returncode == 0) != success: + raise RuntimeError(f"unexpected exit {result.returncode}: {args}; see {log}") + return round(time.monotonic() - start, 3) + + +def replace(path, old, new): + text = path.read_text() + if old not in text: + raise RuntimeError(f"source moved: {old!r} in {path}") + path.write_text(text.replace(old, new)) + + +def prepare(variant, archive): + tree = OUT / variant + if tree.exists(): + raise RuntimeError(f"{tree} exists; choose a new --output directory") + tree.mkdir() + with tarfile.open(archive) as source: + source.extractall(tree, filter="data") + replace(tree / "rs/kio/Cargo.toml", "[features]\n", '[features]\nasync = []\n') + replace(tree / "rs/moq-net/Cargo.toml", 'async = [', 'async = ["kio/async", ') + replace(tree / "rs/kio/src/lib.rs", '#![warn(missing_docs)]', + '#![warn(missing_docs)]\nextern crate self as kio;') + if variant == "attribute": + replace(tree / "rs/kio/Cargo.toml", 'async = []', 'async = ["macros"]') + with (tree / "rs/kio/macros/src/lib.rs").open("a") as target: + target.write("\n" + (HERE / "attribute.rs").read_text()) + replace(tree / "rs/kio/src/lib.rs", "pub use kio_macros::machine;", + "pub use kio_macros::{machine, wrapper};") + for module, names in MODULES.items(): + path = tree / module + text = path.read_text() + # Only top-level impl methods in these two known source files. Fail if + # their spelling or inventory changes, rather than measuring a partial edit. + pattern = r"(?:\t///[^\n]*\n)*(?:\t#\[[^\n]*\]\n)*\tpub async fn (" + "|".join(names) + r")\([^\n]*\)[^\n]*\{\n.*?\n\t}\n" + expected = 1 if "kio/" in module else 4 + text, count = re.subn(pattern, "", text, flags=re.S) + if count != expected: + raise RuntimeError(f"expected {expected} wrappers, found {count}: {module}") + seen = 0 + def lower(match): + nonlocal seen + seen += 1 + docs, attrs, name, receiver, waiter, waiter_ty, output, body = match.groups() + async_name = name.removeprefix("poll_") + if variant == "attribute": + return f'{docs}{attrs}\t#[cfg_attr(feature = "async", kio::wrapper)]\n\tpub fn {name}({receiver}, {waiter}: &{waiter_ty}) -> Poll<{output}> {body}\n' + if variant == "rules": + return f'{docs}\t[{attrs.strip()}]\n\tpub fn {name} => {async_name}({receiver}, {waiter}: &{waiter_ty}) -> {output} {body}\n' + return f'{docs}{attrs}\tpub fn {name}({receiver}, {waiter}: &{waiter_ty}) -> Poll<{output}> {body}\n{docs}\t#[cfg(feature = "async")]\n\tpub async fn {async_name}({receiver}) -> {output} {{\n\t\t::kio::wait(|waiter| self.{name}(waiter)).await\n\t}}\n' + pattern = (r"((?:\t///[^\n]*\n)*)((?:\t#\[[^\n]*\]\n)*)\tpub fn " + r"(poll_(?:" + "|".join(names) + r"))\((&(?:mut )?self), (\w+): &([^\n]+?)\) -> Poll<([^\n]+)> (\{\n.*?\n\t})\n") + if variant == "rules": + # Wrap each declaration/body as a single macro input. + def rules(match): + return "\twrapper! {\n" + lower(match) + "\t}\n" + text = re.sub(pattern, rules, text, flags=re.S) + offset = re.match(r"(?://![^\n]*\n)*", text).end() + text = text[:offset] + (HERE / "rules.rs").read_text() + "\n" + text[offset:] + else: + text = re.sub(pattern, lower, text, flags=re.S) + if seen != expected: + raise RuntimeError(f"expected {expected} polls, found {seen}: {module}") + path.write_text(text) + return tree + + +def environment(tree): + env = os.environ.copy() + # Every variant owns its cache. No remote compiler cache or enclosing wrapper + # may turn a clean measurement into a cache hit. + for key in ("RUSTC_WRAPPER", "RUSTC_WORKSPACE_WRAPPER", "CARGO_ENCODED_RUSTFLAGS", "RUSTFLAGS", "RUSTDOCFLAGS"): + env.pop(key, None) + env.update(CARGO_TARGET_DIR=str(tree / "target"), CARGO_INCREMENTAL="1", CARGO_BUILD_JOBS="2") + return env + + +def check(tree, package, label, features=True, success=True): + args = ["cargo", "check", "--locked", "--offline", "--lib", "-p", package, "--no-default-features"] + if features: + args += ["--features", f"{package}/async"] + return command(args, tree, OUT / f"{tree.name}-{package}-{label}.log", environment(tree), success) + + +def verify(tree): + for package in ("kio", "moq-net"): + check(tree, package, "on") + check(tree, package, "off", False) + args = ["cargo", "doc", "--locked", "--offline", "--no-deps", "--lib", "-p", package, + "--no-default-features", "--features", f"{package}/async"] + env = environment(tree) + env["RUSTDOCFLAGS"] = "-D warnings" + command(args, tree, OUT / f"{tree.name}-{package}-docs.log", env) + for file, poll, wrapper, phrase in ( + ("kio/struct.Queue.html", "poll_pop", "pop", "Poll for the next item."), + ("moq_net/broadcast/struct.Producer.html", "poll_requested_track", "requested_track", "Poll"), + ): + html = (tree / "target/doc" / file).read_text() + # Compare actual rendered documentation, not just matching source attrs. + docs = [] + for name in (poll, wrapper): + section = html.split(f'id="method.{name}"', 1)[1].split('', 1)[0] + docs.append(section.split('
', 1)[1]) + if docs[0] != docs[1] or phrase not in docs[0]: + raise RuntimeError(f"docs differ: {file} {poll}/{wrapper}") + # Malformed receiver: useful diagnostics for a mistake in the generator input. + path = tree / "rs/kio/src/queue.rs" + original = path.read_text() + if tree.name == "attribute": + bad = original.replace("pub fn poll_pop(&self,", "pub fn poll_pop(self,") + elif tree.name == "rules": + bad = original.replace("pub fn poll_pop => pop(&self,", "pub fn poll_pop => pop(self,") + else: + bad = original.replace("pub fn poll_pop(&self,", "pub fn poll_pop(self,") + if bad == original: + raise RuntimeError("diagnostic injection missed its target") + path.write_text(bad) + try: + check(tree, "kio", "bad-receiver", success=False) + finally: + path.write_text(original) + print(f"verified {tree.name}: feature on/off, rustdoc, rejected owned receiver", flush=True) + + +def main(): + global OUT + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--output", type=Path, default=OUT) + parser.add_argument("--rounds", type=int, default=3) + parser.add_argument("--verify-only", action="store_true") + args = parser.parse_args() + OUT = args.output.resolve() + if not OUT.is_relative_to(ROOT / ".tmp"): + parser.error("--output must be inside this checkout's .tmp/") + if args.rounds < 1: + parser.error("--rounds must be positive") + OUT.mkdir(parents=True, exist_ok=True) + archive = OUT / "source.tar" + with archive.open("wb") as output: + subprocess.run(["git", "archive", "HEAD"], cwd=ROOT, stdout=output, check=True) + trees = {variant: prepare(variant, archive) for variant in VARIANTS} + results = {variant: {"kio_clean": [], "net_clean": [], "net_edit": []} for variant in VARIANTS} + metadata = { + "head": subprocess.check_output(["git", "rev-parse", "HEAD"], cwd=ROOT, text=True).strip(), + "rustc": subprocess.check_output(["rustc", "--version"], text=True).strip(), + "jobs": 2, "incremental": True, "rounds": args.rounds, + "system": os.uname()._asdict(), + } + if not args.verify_only: + for round_number in range(args.rounds): + order = VARIANTS[round_number % 3:] + VARIANTS[:round_number % 3] + for variant in order: + tree = trees[variant] + shutil.rmtree(tree / "target", ignore_errors=True) + # kio separately exposes the dependency cost hidden by moq-net's + # existing proc-macro graph. Then discard it for the net clean run. + results[variant]["kio_clean"].append(check(tree, "kio", f"clean-{round_number}")) + shutil.rmtree(tree / "target") + results[variant]["net_clean"].append(check(tree, "moq-net", f"clean-{round_number}")) + path = tree / "rs/moq-net/src/model/broadcast.rs" + original = path.read_text() + path.write_text(original + f"\n// edit round {round_number}\n") + try: + results[variant]["net_edit"].append(check(tree, "moq-net", f"edit-{round_number}")) + finally: + path.write_text(original) + print(variant, round_number, results[variant], flush=True) + (OUT / "results.json").write_text(json.dumps({"metadata": metadata, "seconds": results}, indent=2) + "\n") + for variant, metrics in results.items(): + print(variant, {key: statistics.median(values) for key, values in metrics.items()}, flush=True) + for tree in trees.values(): + verify(tree) + + +if __name__ == "__main__": + main() diff --git a/quest/m1/poll-first/async-feature.md b/quest/m1/poll-first/async-feature.md index 766ef491fc1..7e1a598c805 100644 --- a/quest/m1/poll-first/async-feature.md +++ b/quest/m1/poll-first/async-feature.md @@ -14,6 +14,10 @@ and the bindings and JS keep their APIs. Decided with the maintainer on 2026-10-25: poll is canonical, async is the ergonomic layer, and the gating proceeds regardless of the build measurement. +- **Generator checkpoint:** [the reproducible spike](/bench/async-wrappers/README.md) + targets `kio::Queue::pop` and the four broadcast wrappers without changing + production crates. The generator choice remains the maintainer's; finish the + comparison and choose before broad rollout. - **Spike the generator first** on kio and one moq-net module, then pick one: - a `macro_rules!` over a list of signatures (no proc-macro cost); - a proc macro attribute on the `poll_*` method or its impl block; From a7c1c7c4a7b00cded24e5e94681ee137c0e2c091 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 31 Oct 2026 23:51:52 -0700 Subject: [PATCH 03/12] chore(bench): validate spike failures and recording metadata Co-Authored-By: GPT-6.1 Sol --- bench/async-wrappers/run.py | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/bench/async-wrappers/run.py b/bench/async-wrappers/run.py index 09809475746..0a447111db5 100644 --- a/bench/async-wrappers/run.py +++ b/bench/async-wrappers/run.py @@ -144,6 +144,10 @@ def verify(tree): path.write_text(bad) try: check(tree, "kio", "bad-receiver", success=False) + diagnostic = (OUT / f"{tree.name}-kio-bad-receiver.log").read_text() + expected = {"handwritten": "cannot move out", "rules": "no rules expected", "attribute": "wrapper requires a borrowed receiver"}[tree.name] + if expected not in diagnostic: + raise RuntimeError(f"missing expected diagnostic: {expected}") finally: path.write_text(original) print(f"verified {tree.name}: feature on/off, rustdoc, rejected owned receiver", flush=True) @@ -171,14 +175,15 @@ def main(): "head": subprocess.check_output(["git", "rev-parse", "HEAD"], cwd=ROOT, text=True).strip(), "rustc": subprocess.check_output(["rustc", "--version"], text=True).strip(), "jobs": 2, "incremental": True, "rounds": args.rounds, - "system": os.uname()._asdict(), + "system": {key: getattr(os.uname(), key) for key in ("sysname", "release", "machine")}, } if not args.verify_only: for round_number in range(args.rounds): order = VARIANTS[round_number % 3:] + VARIANTS[:round_number % 3] for variant in order: tree = trees[variant] - shutil.rmtree(tree / "target", ignore_errors=True) + if (tree / "target").exists(): + shutil.rmtree(tree / "target") # kio separately exposes the dependency cost hidden by moq-net's # existing proc-macro graph. Then discard it for the net clean run. results[variant]["kio_clean"].append(check(tree, "kio", f"clean-{round_number}")) From 608408a1844db3b59a3f08ccb5ee50029eeded69 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 31 Oct 2026 23:52:42 -0700 Subject: [PATCH 04/12] chore(bench): measure the pinned compiler without host cache shims Co-Authored-By: GPT-6.1 Sol --- bench/async-wrappers/run.py | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/bench/async-wrappers/run.py b/bench/async-wrappers/run.py index 0a447111db5..eb4aeb0f79e 100644 --- a/bench/async-wrappers/run.py +++ b/bench/async-wrappers/run.py @@ -14,6 +14,7 @@ ROOT = Path(__file__).resolve().parents[2] HERE = Path(__file__).resolve().parent OUT = ROOT / ".tmp/async-wrappers" +TOOLCHAIN = Path(subprocess.check_output(["rustc", "--print", "sysroot"], text=True).strip()) / "bin" VARIANTS = ("handwritten", "rules", "attribute") MODULES = { "rs/kio/src/queue.rs": ("pop",), @@ -98,7 +99,9 @@ def environment(tree): # may turn a clean measurement into a cache hit. for key in ("RUSTC_WRAPPER", "RUSTC_WORKSPACE_WRAPPER", "CARGO_ENCODED_RUSTFLAGS", "RUSTFLAGS", "RUSTDOCFLAGS"): env.pop(key, None) - env.update(CARGO_TARGET_DIR=str(tree / "target"), CARGO_INCREMENTAL="1", CARGO_BUILD_JOBS="2") + env.update(CARGO_TARGET_DIR=str(tree / "target"), CARGO_INCREMENTAL="1", CARGO_BUILD_JOBS="2", + PATH=f"{TOOLCHAIN}:{env['PATH']}", BASH_ENV="/dev/null", + RUSTC=str(TOOLCHAIN / "rustc"), RUSTDOC=str(TOOLCHAIN / "rustdoc")) return env @@ -174,7 +177,7 @@ def main(): metadata = { "head": subprocess.check_output(["git", "rev-parse", "HEAD"], cwd=ROOT, text=True).strip(), "rustc": subprocess.check_output(["rustc", "--version"], text=True).strip(), - "jobs": 2, "incremental": True, "rounds": args.rounds, + "jobs": 2, "incremental": True, "rounds": args.rounds, "toolchain": str(TOOLCHAIN), "system": {key: getattr(os.uname(), key) for key in ("sysname", "release", "machine")}, } if not args.verify_only: From d27e3683e18f9eaf86fd36e837e531bd60435a48 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 31 Oct 2026 23:54:38 -0700 Subject: [PATCH 05/12] chore(bench): invalidate a method body for incremental measurements Co-Authored-By: GPT-6.1 Sol --- bench/async-wrappers/README.md | 7 ++++--- bench/async-wrappers/run.py | 5 ++++- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/bench/async-wrappers/README.md b/bench/async-wrappers/README.md index 8aa9b42fc7d..7fb5009f205 100644 --- a/bench/async-wrappers/README.md +++ b/bench/async-wrappers/README.md @@ -51,11 +51,12 @@ Three rounds rotate the variant order. Each round measures wall time for: kio/async`, exposing the proc-macro dependency cost. 2. A fresh-target `cargo check --lib -p moq-net --no-default-features --features moq-net/async`, including its dependency graph. -3. The same moq-net command after a comment edit in `model/broadcast.rs`, with +3. The same moq-net command after an assertion-message edit in `model/broadcast.rs`, with incremental compilation enabled. This measures a recompilation of the changed - module, not an idle Cargo check or an algorithm change. + method body, not an idle Cargo check, a comment-only edit, or an algorithm change. -The runner disables compiler wrappers and custom Rust flags, uses two Cargo +The runner invokes Cargo/rustc/rustdoc from the pinned toolchain's sysroot, +disables compiler wrappers, shell startup injection, and custom Rust flags, uses two Cargo jobs for every variant, and records all samples plus toolchain and source SHA in `results.json`. OS file caches remain warm, and host contention can affect wall time. This small spike measures generator overhead for five wrappers, not the diff --git a/bench/async-wrappers/run.py b/bench/async-wrappers/run.py index eb4aeb0f79e..30b612ef782 100644 --- a/bench/async-wrappers/run.py +++ b/bench/async-wrappers/run.py @@ -194,7 +194,10 @@ def main(): results[variant]["net_clean"].append(check(tree, "moq-net", f"clean-{round_number}")) path = tree / "rs/moq-net/src/model/broadcast.rs" original = path.read_text() - path.write_text(original + f"\n// edit round {round_number}\n") + edited = original.replace("predicate guaranteed a request", f"spike edit {round_number}: request predicate guaranteed an entry") + if edited == original: + raise RuntimeError("body edit missed its target") + path.write_text(edited) try: results[variant]["net_edit"].append(check(tree, "moq-net", f"edit-{round_number}")) finally: From 2c3a5dd5729ddfcf175928b505fc0bbdd35a8f32 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 31 Oct 2026 23:55:17 -0700 Subject: [PATCH 06/12] fix(bench): delimit declarative type fragments and verify before timing Co-Authored-By: GPT-6.1 Sol --- bench/async-wrappers/rules.rs | 6 +++--- bench/async-wrappers/run.py | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/bench/async-wrappers/rules.rs b/bench/async-wrappers/rules.rs index 52a9da7baca..271fdb56b0d 100644 --- a/bench/async-wrappers/rules.rs +++ b/bench/async-wrappers/rules.rs @@ -4,16 +4,16 @@ macro_rules! wrapper { $vis:vis fn $poll:ident => $name:ident (&$this:ident, $waiter:ident: &$waiter_ty:ty) -> $out:ty $body:block) => { wrapper!(@emit [$(#[doc = $doc])*] [$($attr)*] $vis $poll $name - [&$this] $this $waiter $waiter_ty $out $body); + [&$this] $this $waiter $waiter_ty, $out $body); }; ($(#[doc = $doc:literal])* [$($attr:tt)*] $vis:vis fn $poll:ident => $name:ident (&mut $this:ident, $waiter:ident: &$waiter_ty:ty) -> $out:ty $body:block) => { wrapper!(@emit [$(#[doc = $doc])*] [$($attr)*] $vis $poll $name - [&mut $this] $this $waiter $waiter_ty $out $body); + [&mut $this] $this $waiter $waiter_ty, $out $body); }; (@emit [$($doc:tt)*] [$($attr:tt)*] $vis:vis $poll:ident $name:ident - [$($receiver:tt)*] $this:ident $waiter:ident $waiter_ty:ty $out:ty $body:block) => { + [$($receiver:tt)*] $this:ident $waiter:ident $waiter_ty:ty, $out:ty $body:block) => { $($doc)* $($attr)* $vis fn $poll($($receiver)*, $waiter: &$waiter_ty) -> ::std::task::Poll<$out> $body diff --git a/bench/async-wrappers/run.py b/bench/async-wrappers/run.py index 30b612ef782..87b3862ed4e 100644 --- a/bench/async-wrappers/run.py +++ b/bench/async-wrappers/run.py @@ -180,6 +180,8 @@ def main(): "jobs": 2, "incremental": True, "rounds": args.rounds, "toolchain": str(TOOLCHAIN), "system": {key: getattr(os.uname(), key) for key in ("sysname", "release", "machine")}, } + for tree in trees.values(): + verify(tree) if not args.verify_only: for round_number in range(args.rounds): order = VARIANTS[round_number % 3:] + VARIANTS[:round_number % 3] @@ -206,8 +208,6 @@ def main(): (OUT / "results.json").write_text(json.dumps({"metadata": metadata, "seconds": results}, indent=2) + "\n") for variant, metrics in results.items(): print(variant, {key: statistics.median(values) for key, values in metrics.items()}, flush=True) - for tree in trees.values(): - verify(tree) if __name__ == "__main__": From 263e56b0fcabce1402cc4a405bad06bfa22c8fa3 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 1 Nov 2026 00:00:36 -0700 Subject: [PATCH 07/12] fix(bench): inspect broadcast request-handle documentation Co-Authored-By: GPT-6.1 Sol --- bench/async-wrappers/README.md | 8 +++++--- bench/async-wrappers/run.py | 2 +- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/bench/async-wrappers/README.md b/bench/async-wrappers/README.md index 7fb5009f205..ff99c40b784 100644 --- a/bench/async-wrappers/README.md +++ b/bench/async-wrappers/README.md @@ -1,7 +1,7 @@ # Async-wrapper generator spike This checkpoint compares generation on `kio::Queue::pop` and all four wrappers in -`moq_net::broadcast`: `Producer::{requested_track, closed}`, `Consumer::closed`, +`moq_net::broadcast`: `Requests::{requested_track, closed}`, `Consumer::closed`, and `Demand::wanted_changed`. It does not pick a generator or change production crates. The remaining work belongs to [async-feature](/quest/m1/poll-first/async-feature.md). @@ -26,7 +26,9 @@ and gives each variant its own target directory. It compares: behind `async`. This controls for equivalent generated code and documentation. - **Declarative:** `rules.rs` replaces each poll declaration with one macro invocation containing its docs, signature, and body. The explicit `poll => async` - names avoid an identifier-concatenation dependency. The bracketed attributes + names avoid an identifier-concatenation dependency. A direct stable 1.98.1 + probe rejects `${concat(poll_, $name)}` with E0658, so that spelling cannot + remove the duplicated names on the supported toolchain. The bracketed attributes apply only to the poll method, preserving `_sim`'s `track_caller`. - **Attribute:** `attribute.rs` is appended to the existing kio-macros crate, and re-exported through kio only in the snapshots. A method attribute derives the @@ -65,7 +67,7 @@ maintainer's settled direction regardless of speed. Verification checks feature-on and feature-off library builds, Rustdoc with warnings denied, and identical rendered poll/wrapper docs for `Queue::pop` and -`Producer::requested_track`. It also compiles an owned-receiver mistake and +`Requests::requested_track`. It also compiles an owned-receiver mistake and requires failure; inspect `*-bad-receiver.log` for error spans. Other wrappers and unrelated kio async bodies remain untouched, so a feature-off success here does **not** establish the quest's eventual no-coroutines condition. diff --git a/bench/async-wrappers/run.py b/bench/async-wrappers/run.py index 87b3862ed4e..cd823105c03 100644 --- a/bench/async-wrappers/run.py +++ b/bench/async-wrappers/run.py @@ -123,7 +123,7 @@ def verify(tree): command(args, tree, OUT / f"{tree.name}-{package}-docs.log", env) for file, poll, wrapper, phrase in ( ("kio/struct.Queue.html", "poll_pop", "pop", "Poll for the next item."), - ("moq_net/broadcast/struct.Producer.html", "poll_requested_track", "requested_track", "Poll"), + ("moq_net/broadcast/struct.Requests.html", "poll_requested_track", "requested_track", "Poll"), ): html = (tree / "target/doc" / file).read_text() # Compare actual rendered documentation, not just matching source attrs. From 91b1ec62e1e35295ef74d0f7cee1afc49739bf00 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 1 Nov 2026 00:13:05 -0700 Subject: [PATCH 08/12] docs(quest): record generator measurements and the open maintainer choice Co-Authored-By: GPT-6.1 Sol --- bench/async-wrappers/README.md | 6 +- bench/async-wrappers/RESULTS.md | 103 +++++++++++++++++++++++++++ bench/async-wrappers/results.json | 68 ++++++++++++++++++ quest/m1/poll-first/async-feature.md | 9 ++- 4 files changed, 182 insertions(+), 4 deletions(-) create mode 100644 bench/async-wrappers/RESULTS.md create mode 100644 bench/async-wrappers/results.json diff --git a/bench/async-wrappers/README.md b/bench/async-wrappers/README.md index ff99c40b784..097e51dbd77 100644 --- a/bench/async-wrappers/README.md +++ b/bench/async-wrappers/README.md @@ -3,7 +3,7 @@ This checkpoint compares generation on `kio::Queue::pop` and all four wrappers in `moq_net::broadcast`: `Requests::{requested_track, closed}`, `Consumer::closed`, and `Demand::wanted_changed`. It does not pick a generator or change production -crates. The remaining work belongs to +crates. See the [measured results and recommendation](RESULTS.md). The remaining work belongs to [async-feature](/quest/m1/poll-first/async-feature.md). Run in the repository's Nix shell: @@ -29,7 +29,9 @@ and gives each variant its own target directory. It compares: names avoid an identifier-concatenation dependency. A direct stable 1.98.1 probe rejects `${concat(poll_, $name)}` with E0658, so that spelling cannot remove the duplicated names on the supported toolchain. The bracketed attributes - apply only to the poll method, preserving `_sim`'s `track_caller`. + apply only to the poll method, preserving `_sim`'s `track_caller`. A separate + list of wrapper signatures cannot inspect existing poll docs, so the candidate + owns the poll declaration/body too; a duplicate signature/doc list was dropped. - **Attribute:** `attribute.rs` is appended to the existing kio-macros crate, and re-exported through kio only in the snapshots. A method attribute derives the wrapper name, return type, receiver, visibility, and docs from ordinary Rust. diff --git a/bench/async-wrappers/RESULTS.md b/bench/async-wrappers/RESULTS.md new file mode 100644 index 00000000000..f1acdddbf89 --- /dev/null +++ b/bench/async-wrappers/RESULTS.md @@ -0,0 +1,103 @@ +# Generator choice checkpoint + +Recommendation for the maintainer: use a method attribute in the existing +kio-macros crate. This is a recommendation, **not a settled choice**. Keep the +[async-feature quest](/quest/m1/poll-first/async-feature.md) and its PR open until +the maintainer picks a generator and the rollout finishes. + +## Measurements + +Measured on 2026-11-01 from +[263e56b0fcabce1402cc4a405bad06bfa22c8fa3](https://github.com/moq-dev/moq/commit/263e56b0fcabce1402cc4a405bad06bfa22c8fa3), +with pinned Rust 1.98.1 on Linux x86\_64, two Cargo jobs, separate fresh target +directories, compiler caches bypassed, and three rotated rounds. +[Raw samples](results.json) retain the exact toolchain and source identity. +These are `cargo check --lib` wall times, not release builds or runtime benchmarks. + +| Variant | kio clean, seconds | moq-net clean, seconds | moq-net body edit, seconds | +| --- | ---: | ---: | ---: | +| Handwritten control | 1.749 (1.265–2.010) | 28.966 (24.279–36.841) | 1.993 (1.754–2.001) | +| Declarative | 1.609 (1.402–2.114) | 24.447 (23.468–37.555) | 1.751 (1.610–2.241) | +| Attribute | 5.302 (4.591–6.470) | 29.567 (27.239–35.154) | 1.721 (1.461–2.089) | + +Cells show median (minimum–maximum). Host contention and warm OS caches remain: +moq-net's ranges overlap, and even the handwritten control's clean checks span +12.562 seconds. Do not rank moq-net build speed from this sample. The attribute's +standalone kio cost is clear: its clean median adds 3.553 seconds by enabling the +existing syn/quote/machine macro graph. moq-net already builds that graph. The +five-wrapper spike does not measure whole-library coroutine removal savings; +the maintainer already decided that gating proceeds regardless of those savings. + +Reproduce with the [runner and method](README.md). The successful command was: + +```sh +quest gate python3 bench/async-wrappers/run.py --rounds 3 --output .tmp/async-wrappers-run3 +``` + +## Diagnostics and docs + +All three variants passed kio and moq-net library checks with the selected +wrappers' feature enabled and disabled. Rustdoc passed with warnings denied. +The actual rendered docs for `Queue::pop` and `Requests::requested_track` match +their poll methods exactly in every variant, including resolved links. + +An owned receiver deliberately substituted for `&self` produces these diagnostics +in the generated snapshots: + +| Variant | Primary diagnostic | Span | +| --- | --- | --- | +| Handwritten | `cannot move out of *self, as self is a captured variable in an FnMut closure` | The wrapper's `self.poll_pop(waiter)` call | +| Declarative | `no rules expected keyword self` | `self` in the macro invocation; a note points to the matcher's `&` | +| Attribute | `wrapper requires a borrowed receiver` | `self` in the original Rust method | + +The runner requires both failure and the expected diagnostic, then restores the +input. The attribute can report its contract directly; the declarative version +can improve its messages with catch-all rules, at the cost of more dialect code. + +## Options for the maintainer + +1. **Method attribute, recommended.** Keep canonical poll methods as ordinary + Rust, derive the async name/signature/docs, and reuse kio-macros. This avoids a + second signature list and keeps method bodies readable to Rust tooling before + expansion. Accept the measured standalone kio async-build cost. Generalize the + prototype for the real signature inventory and test macro composition before + publishing it; this checkpoint does not claim IDE completion was tested. +2. **Declarative macro.** Avoid the standalone proc-macro cost. The prototype is + smaller (26 lines versus 118 for the attribute), but moves the poll declaration + and body into a dialect and spells both method names. A separate wrapper + signature list cannot read poll docs; that duplicate source was deliberately + dropped. Stable 1.98.1 rejects `${concat(...)}` with E0658. This remains viable + if minimum kio build cost outweighs ordinary Rust source and targeted errors. +3. **rust2-driver lowering.** Fund a larger compiler spike first. The pinned + [upstream inspection](README.md#rust2-driver) found no wrapper lowering to + measure. Its nightly `rustc_private` integration would need consumer, + rustdoc, and IDE delivery work. No timing, diagnostic, or docs result is + claimed for a nonexistent implementation. + +No newer settled generator choice was found in the current quest or the merged +[machine-bodies PR](https://github.com/moq-dev/moq/pull/7284). Its straight-line +machine conversion is present in the spike's base. No public library API, default +features, wire format, bindings, or JS API changes in this checkpoint. + +## Checks and remaining work + +- The full spike passed: all feature builds, Rustdoc/doc comparisons, expected + diagnostics, and nine timed variant rounds. +- `quest check --base origin/main` passed (651 documents). +- `just ci origin/main` passed with pinned Nix binaries, `BASH_ENV=/dev/null`, + and a scratch `CARGO_TARGET_DIR` independent of the host mbx target symlink. + The ordinary host invocation first failed in the existing test-lint fixture + because mbx returned `Argument list too long`; the same host failure is already + recorded in [moq-dev/moq#7464](https://github.com/moq-dev/moq/pull/7464). The + fixture also passed independently with the pinned toolchain. No repository + workaround was added for the host launcher. +- Local author review checks the snapshots, feature wiring, diagnostic spans, + rendered docs, raw samples, and the limits of the performance claim. Independent + cross-vendor review remains for the implementation after the maintainer's pick. + +After the pick: design the final macro contract against the full signature +inventory, put its verification in CI, convert remaining async logic to polls, +roll out generated wrappers and default-disabled `async` features, enable them at +application/test/binding edges, and verify library-only builds contain no +coroutines. Keep the migration notes and instruction/doc updates with that +rollout. These remain work in this quest, not completed work in this checkpoint. diff --git a/bench/async-wrappers/results.json b/bench/async-wrappers/results.json new file mode 100644 index 00000000000..f2de2cbdc26 --- /dev/null +++ b/bench/async-wrappers/results.json @@ -0,0 +1,68 @@ +{ + "metadata": { + "head": "263e56b0fcabce1402cc4a405bad06bfa22c8fa3", + "rustc": "rustc 1.98.1 (48a229cea 2026-09-01)", + "jobs": 2, + "incremental": true, + "rounds": 3, + "toolchain": "/nix/store/4klbl675argl2irhl89zrh2n14wilzx9-rust-minimal-1.98.1/bin", + "system": { + "sysname": "Linux", + "release": "7.0.0-42-generic", + "machine": "x86_64" + } + }, + "seconds": { + "handwritten": { + "kio_clean": [ + 2.01, + 1.749, + 1.265 + ], + "net_clean": [ + 36.841, + 28.966, + 24.279 + ], + "net_edit": [ + 1.993, + 2.001, + 1.754 + ] + }, + "rules": { + "kio_clean": [ + 2.114, + 1.402, + 1.609 + ], + "net_clean": [ + 37.555, + 24.447, + 23.468 + ], + "net_edit": [ + 2.241, + 1.751, + 1.61 + ] + }, + "attribute": { + "kio_clean": [ + 5.302, + 4.591, + 6.47 + ], + "net_clean": [ + 35.154, + 27.239, + 29.567 + ], + "net_edit": [ + 2.089, + 1.721, + 1.461 + ] + } + } +} diff --git a/quest/m1/poll-first/async-feature.md b/quest/m1/poll-first/async-feature.md index 7e1a598c805..bb0fc35b992 100644 --- a/quest/m1/poll-first/async-feature.md +++ b/quest/m1/poll-first/async-feature.md @@ -16,8 +16,13 @@ ergonomic layer, and the gating proceeds regardless of the build measurement. - **Generator checkpoint:** [the reproducible spike](/bench/async-wrappers/README.md) targets `kio::Queue::pop` and the four broadcast wrappers without changing - production crates. The generator choice remains the maintainer's; finish the - comparison and choose before broad rollout. + production crates. The [completed comparison](/bench/async-wrappers/RESULTS.md) + recommends a method attribute reusing kio-macros: ordinary Rust poll source, + automatic names, and targeted errors, at a measured standalone kio build cost. + This is not a maintainer decision. Choose the generator before broad rollout; + declarative generation remains viable and rust2 needs a new compiler spike. + Generalize the chosen prototype, add CI coverage for its contract, then complete + the coroutine conversion, feature wiring, and library-only verification below. - **Spike the generator first** on kio and one moq-net module, then pick one: - a `macro_rules!` over a list of signatures (no proc-macro cost); - a proc macro attribute on the `poll_*` method or its impl block; From ebd7b35276d8d5ae948b70eb7501815fb2902b69 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 1 Nov 2026 10:46:26 -0800 Subject: [PATCH 09/12] feat(kio)!: generate async wrappers behind an async feature The maintainer picked the method attribute from the generator spike. `#[kio::wrapper]` on a `poll_*` method adds the `async fn` that awaits it, deriving its name, signature, generics, and docs. Its contract is covered by kio-macros' behavior and UI tests. kio's async methods move behind a new default-off `async` feature, so a poll-only kio compiles no coroutines. moq-net's `async` enables it, and moq-net's broadcast wrappers become generated. `just rs coroutines` scans the MIR of kio's and moq-net's poll-only builds in the nightly feature matrix. The spike fixtures are removed: the runner appended its own `wrapper` to kio-macros, and its results stay linked from the quest by commit. Co-Authored-By: Claude Opus 5.5 --- bench/async-wrappers/README.md | 96 -------- bench/async-wrappers/RESULTS.md | 103 -------- bench/async-wrappers/attribute.rs | 118 --------- bench/async-wrappers/results.json | 68 ------ bench/async-wrappers/rules.rs | 26 -- bench/async-wrappers/run.py | 214 ---------------- quest/m1/poll-first/async-feature.md | 54 ++-- rs/AGENTS.md | 4 +- rs/justfile | 8 + rs/kio/Cargo.toml | 20 +- rs/kio/MIGRATE.md | 8 + rs/kio/macros/src/lib.rs | 62 +++++ rs/kio/macros/src/wrapper.rs | 231 ++++++++++++++++++ .../macros/tests/ui/wrapper_moved_argument.rs | 13 + .../tests/ui/wrapper_moved_argument.stderr | 21 ++ rs/kio/macros/tests/ui/wrapper_name.rs | 13 + rs/kio/macros/tests/ui/wrapper_name.stderr | 5 + rs/kio/macros/tests/ui/wrapper_no_receiver.rs | 13 + .../tests/ui/wrapper_no_receiver.stderr | 5 + rs/kio/macros/tests/ui/wrapper_output.rs | 15 ++ rs/kio/macros/tests/ui/wrapper_output.stderr | 11 + .../macros/tests/ui/wrapper_owned_receiver.rs | 13 + .../tests/ui/wrapper_owned_receiver.stderr | 5 + rs/kio/macros/tests/ui/wrapper_pattern.rs | 13 + rs/kio/macros/tests/ui/wrapper_pattern.stderr | 5 + rs/kio/macros/tests/ui/wrapper_qualifiers.rs | 28 +++ .../macros/tests/ui/wrapper_qualifiers.stderr | 23 ++ rs/kio/macros/tests/ui/wrapper_waiter.rs | 23 ++ rs/kio/macros/tests/ui/wrapper_waiter.stderr | 23 ++ rs/kio/macros/tests/wrapper.rs | 191 +++++++++++++++ rs/kio/src/clock/mod.rs | 8 +- rs/kio/src/clock/timer.rs | 2 + rs/kio/src/consumer.rs | 19 +- rs/kio/src/lib.rs | 13 +- rs/kio/src/producer.rs | 21 +- rs/kio/src/queue.rs | 17 +- rs/kio/src/shared.rs | 1 + rs/kio/src/weak.rs | 18 +- rs/moq-net/Cargo.toml | 2 +- rs/moq-net/src/model/broadcast.rs | 44 +--- rs/moq-shaper/Cargo.toml | 2 +- sh/rs/coroutines.sh | 69 ++++++ 42 files changed, 905 insertions(+), 743 deletions(-) delete mode 100644 bench/async-wrappers/README.md delete mode 100644 bench/async-wrappers/RESULTS.md delete mode 100644 bench/async-wrappers/attribute.rs delete mode 100644 bench/async-wrappers/results.json delete mode 100644 bench/async-wrappers/rules.rs delete mode 100644 bench/async-wrappers/run.py create mode 100644 rs/kio/macros/src/wrapper.rs create mode 100644 rs/kio/macros/tests/ui/wrapper_moved_argument.rs create mode 100644 rs/kio/macros/tests/ui/wrapper_moved_argument.stderr create mode 100644 rs/kio/macros/tests/ui/wrapper_name.rs create mode 100644 rs/kio/macros/tests/ui/wrapper_name.stderr create mode 100644 rs/kio/macros/tests/ui/wrapper_no_receiver.rs create mode 100644 rs/kio/macros/tests/ui/wrapper_no_receiver.stderr create mode 100644 rs/kio/macros/tests/ui/wrapper_output.rs create mode 100644 rs/kio/macros/tests/ui/wrapper_output.stderr create mode 100644 rs/kio/macros/tests/ui/wrapper_owned_receiver.rs create mode 100644 rs/kio/macros/tests/ui/wrapper_owned_receiver.stderr create mode 100644 rs/kio/macros/tests/ui/wrapper_pattern.rs create mode 100644 rs/kio/macros/tests/ui/wrapper_pattern.stderr create mode 100644 rs/kio/macros/tests/ui/wrapper_qualifiers.rs create mode 100644 rs/kio/macros/tests/ui/wrapper_qualifiers.stderr create mode 100644 rs/kio/macros/tests/ui/wrapper_waiter.rs create mode 100644 rs/kio/macros/tests/ui/wrapper_waiter.stderr create mode 100644 rs/kio/macros/tests/wrapper.rs create mode 100755 sh/rs/coroutines.sh diff --git a/bench/async-wrappers/README.md b/bench/async-wrappers/README.md deleted file mode 100644 index 097e51dbd77..00000000000 --- a/bench/async-wrappers/README.md +++ /dev/null @@ -1,96 +0,0 @@ -# Async-wrapper generator spike - -This checkpoint compares generation on `kio::Queue::pop` and all four wrappers in -`moq_net::broadcast`: `Requests::{requested_track, closed}`, `Consumer::closed`, -and `Demand::wanted_changed`. It does not pick a generator or change production -crates. See the [measured results and recommendation](RESULTS.md). The remaining work belongs to -[async-feature](/quest/m1/poll-first/async-feature.md). - -Run in the repository's Nix shell: - -```sh -quest gate python3 bench/async-wrappers/run.py --rounds 3 -``` - -Use `--output .tmp/another-run` to keep earlier results. The runner refuses to -reuse variant directories. `--verify-only` skips timings on a fresh snapshot. -The source archive comes from `HEAD`, while the generator fixtures come from -beside the runner. Commit before measuring so the checkpoint is reproducible. -It requires the workspace lockfile's Cargo dependencies in the local cache; -`--offline` refuses a missing dependency instead of including downloads in timings. - -The runner creates source snapshots under `.tmp/`, never edits the checkout, -and gives each variant its own target directory. It compares: - -- **Handwritten:** the same five wrappers with poll docs copied to them, gated - behind `async`. This controls for equivalent generated code and documentation. -- **Declarative:** `rules.rs` replaces each poll declaration with one macro - invocation containing its docs, signature, and body. The explicit `poll => async` - names avoid an identifier-concatenation dependency. A direct stable 1.98.1 - probe rejects `${concat(poll_, $name)}` with E0658, so that spelling cannot - remove the duplicated names on the supported toolchain. The bracketed attributes - apply only to the poll method, preserving `_sim`'s `track_caller`. A separate - list of wrapper signatures cannot inspect existing poll docs, so the candidate - owns the poll declaration/body too; a duplicate signature/doc list was dropped. -- **Attribute:** `attribute.rs` is appended to the existing kio-macros crate, and - re-exported through kio only in the snapshots. A method attribute derives the - wrapper name, return type, receiver, visibility, and docs from ordinary Rust. - `cfg_attr(feature = "async", ...)` avoids building the generator in a standalone - poll-only kio build. moq-net already enables kio's `macros` feature for machines. - -Both prototypes intentionally support only safe, non-generic methods taking a -borrowed receiver and waiter. The enclosing impl may be generic (`Queue`). -They are comparison fixtures, not ready-to-publish macro APIs. Generic methods, -extra arguments, explicit lifetimes, conditional method attributes, renamed -crate dependencies, and composition with `#[kio::machine]` still need design -and coverage after the maintainer chooses. An impl-level attribute would avoid -repeating `cfg_attr`, but would parse entire impls and need explicit opt-in for -which methods are wrappers. It is not measured here. - -## Method - -Three rounds rotate the variant order. Each round measures wall time for: - -1. A clean standalone `cargo check --lib -p kio --no-default-features --features - kio/async`, exposing the proc-macro dependency cost. -2. A fresh-target `cargo check --lib -p moq-net --no-default-features --features - moq-net/async`, including its dependency graph. -3. The same moq-net command after an assertion-message edit in `model/broadcast.rs`, with - incremental compilation enabled. This measures a recompilation of the changed - method body, not an idle Cargo check, a comment-only edit, or an algorithm change. - -The runner invokes Cargo/rustc/rustdoc from the pinned toolchain's sysroot, -disables compiler wrappers, shell startup injection, and custom Rust flags, uses two Cargo -jobs for every variant, and records all samples plus toolchain and source SHA in -`results.json`. OS file caches remain warm, and host contention can affect wall -time. This small spike measures generator overhead for five wrappers, not the -savings from eliminating every coroutine in the quest. Gating remains the -maintainer's settled direction regardless of speed. - -Verification checks feature-on and feature-off library builds, Rustdoc with -warnings denied, and identical rendered poll/wrapper docs for `Queue::pop` and -`Requests::requested_track`. It also compiles an owned-receiver mistake and -requires failure; inspect `*-bad-receiver.log` for error spans. Other wrappers -and unrelated kio async bodies remain untouched, so a feature-off success here -does **not** establish the quest's eventual no-coroutines condition. - -## rust2-driver - -Upstream was inspected at -[06614b5a4577a9230206a5babf75c592b1e07efc](https://github.com/moq-dev/rust2/tree/06614b5a4577a9230206a5babf75c592b1e07efc). -Its [front end](https://github.com/moq-dev/rust2/blob/06614b5a4577a9230206a5babf75c592b1e07efc/driver/src/lib.rs) -and [compiler callbacks](https://github.com/moq-dev/rust2/blob/06614b5a4577a9230206a5babf75c592b1e07efc/driver/src/rustc.rs) -provide compiler integration, not async-wrapper generation. The driver uses -`rustc_private` and its own nightly toolchain; moq pins stable 1.98.1. A new -lowering would therefore be a compiler-toolchain project, including distributing -it to library consumers and supporting rustdoc and IDEs. No working lowering -was available to time or inspect for wrapper diagnostics/docs. This is an -unmeasured option requiring a larger spike, not a zero-cost measured alternative. - -Rust's stable -[attribute macros](https://doc.rust-lang.org/reference/procedural-macros.html#attribute-macros) -can emit adjacent methods and preserve input spans. Stable -[declarative macros](https://doc.rust-lang.org/reference/macros-by-example.html) -can also emit both methods, but consume a dialect rather than annotate a normal -method. Both fixtures copy docs from the poll declaration instead of maintaining -a second signature/doc list. diff --git a/bench/async-wrappers/RESULTS.md b/bench/async-wrappers/RESULTS.md deleted file mode 100644 index f1acdddbf89..00000000000 --- a/bench/async-wrappers/RESULTS.md +++ /dev/null @@ -1,103 +0,0 @@ -# Generator choice checkpoint - -Recommendation for the maintainer: use a method attribute in the existing -kio-macros crate. This is a recommendation, **not a settled choice**. Keep the -[async-feature quest](/quest/m1/poll-first/async-feature.md) and its PR open until -the maintainer picks a generator and the rollout finishes. - -## Measurements - -Measured on 2026-11-01 from -[263e56b0fcabce1402cc4a405bad06bfa22c8fa3](https://github.com/moq-dev/moq/commit/263e56b0fcabce1402cc4a405bad06bfa22c8fa3), -with pinned Rust 1.98.1 on Linux x86\_64, two Cargo jobs, separate fresh target -directories, compiler caches bypassed, and three rotated rounds. -[Raw samples](results.json) retain the exact toolchain and source identity. -These are `cargo check --lib` wall times, not release builds or runtime benchmarks. - -| Variant | kio clean, seconds | moq-net clean, seconds | moq-net body edit, seconds | -| --- | ---: | ---: | ---: | -| Handwritten control | 1.749 (1.265–2.010) | 28.966 (24.279–36.841) | 1.993 (1.754–2.001) | -| Declarative | 1.609 (1.402–2.114) | 24.447 (23.468–37.555) | 1.751 (1.610–2.241) | -| Attribute | 5.302 (4.591–6.470) | 29.567 (27.239–35.154) | 1.721 (1.461–2.089) | - -Cells show median (minimum–maximum). Host contention and warm OS caches remain: -moq-net's ranges overlap, and even the handwritten control's clean checks span -12.562 seconds. Do not rank moq-net build speed from this sample. The attribute's -standalone kio cost is clear: its clean median adds 3.553 seconds by enabling the -existing syn/quote/machine macro graph. moq-net already builds that graph. The -five-wrapper spike does not measure whole-library coroutine removal savings; -the maintainer already decided that gating proceeds regardless of those savings. - -Reproduce with the [runner and method](README.md). The successful command was: - -```sh -quest gate python3 bench/async-wrappers/run.py --rounds 3 --output .tmp/async-wrappers-run3 -``` - -## Diagnostics and docs - -All three variants passed kio and moq-net library checks with the selected -wrappers' feature enabled and disabled. Rustdoc passed with warnings denied. -The actual rendered docs for `Queue::pop` and `Requests::requested_track` match -their poll methods exactly in every variant, including resolved links. - -An owned receiver deliberately substituted for `&self` produces these diagnostics -in the generated snapshots: - -| Variant | Primary diagnostic | Span | -| --- | --- | --- | -| Handwritten | `cannot move out of *self, as self is a captured variable in an FnMut closure` | The wrapper's `self.poll_pop(waiter)` call | -| Declarative | `no rules expected keyword self` | `self` in the macro invocation; a note points to the matcher's `&` | -| Attribute | `wrapper requires a borrowed receiver` | `self` in the original Rust method | - -The runner requires both failure and the expected diagnostic, then restores the -input. The attribute can report its contract directly; the declarative version -can improve its messages with catch-all rules, at the cost of more dialect code. - -## Options for the maintainer - -1. **Method attribute, recommended.** Keep canonical poll methods as ordinary - Rust, derive the async name/signature/docs, and reuse kio-macros. This avoids a - second signature list and keeps method bodies readable to Rust tooling before - expansion. Accept the measured standalone kio async-build cost. Generalize the - prototype for the real signature inventory and test macro composition before - publishing it; this checkpoint does not claim IDE completion was tested. -2. **Declarative macro.** Avoid the standalone proc-macro cost. The prototype is - smaller (26 lines versus 118 for the attribute), but moves the poll declaration - and body into a dialect and spells both method names. A separate wrapper - signature list cannot read poll docs; that duplicate source was deliberately - dropped. Stable 1.98.1 rejects `${concat(...)}` with E0658. This remains viable - if minimum kio build cost outweighs ordinary Rust source and targeted errors. -3. **rust2-driver lowering.** Fund a larger compiler spike first. The pinned - [upstream inspection](README.md#rust2-driver) found no wrapper lowering to - measure. Its nightly `rustc_private` integration would need consumer, - rustdoc, and IDE delivery work. No timing, diagnostic, or docs result is - claimed for a nonexistent implementation. - -No newer settled generator choice was found in the current quest or the merged -[machine-bodies PR](https://github.com/moq-dev/moq/pull/7284). Its straight-line -machine conversion is present in the spike's base. No public library API, default -features, wire format, bindings, or JS API changes in this checkpoint. - -## Checks and remaining work - -- The full spike passed: all feature builds, Rustdoc/doc comparisons, expected - diagnostics, and nine timed variant rounds. -- `quest check --base origin/main` passed (651 documents). -- `just ci origin/main` passed with pinned Nix binaries, `BASH_ENV=/dev/null`, - and a scratch `CARGO_TARGET_DIR` independent of the host mbx target symlink. - The ordinary host invocation first failed in the existing test-lint fixture - because mbx returned `Argument list too long`; the same host failure is already - recorded in [moq-dev/moq#7464](https://github.com/moq-dev/moq/pull/7464). The - fixture also passed independently with the pinned toolchain. No repository - workaround was added for the host launcher. -- Local author review checks the snapshots, feature wiring, diagnostic spans, - rendered docs, raw samples, and the limits of the performance claim. Independent - cross-vendor review remains for the implementation after the maintainer's pick. - -After the pick: design the final macro contract against the full signature -inventory, put its verification in CI, convert remaining async logic to polls, -roll out generated wrappers and default-disabled `async` features, enable them at -application/test/binding edges, and verify library-only builds contain no -coroutines. Keep the migration notes and instruction/doc updates with that -rollout. These remain work in this quest, not completed work in this checkpoint. diff --git a/bench/async-wrappers/attribute.rs b/bench/async-wrappers/attribute.rs deleted file mode 100644 index 4b85074c354..00000000000 --- a/bench/async-wrappers/attribute.rs +++ /dev/null @@ -1,118 +0,0 @@ -// Appended only to the snapshot's existing kio-macros crate. -#[proc_macro_attribute] -pub fn wrapper( - args: proc_macro::TokenStream, - item: proc_macro::TokenStream, -) -> proc_macro::TokenStream { - fn expand( - args: proc_macro2::TokenStream, - method: syn::ImplItemFn, - ) -> syn::Result { - if !args.is_empty() { - return Err(syn::Error::new_spanned(args, "wrapper takes no arguments")); - } - let sig = &method.sig; - let name = sig.ident.to_string(); - let Some(name) = name.strip_prefix("poll_").filter(|name| !name.is_empty()) else { - return Err(syn::Error::new_spanned( - &sig.ident, - "wrapper requires a poll_* method", - )); - }; - let Some(syn::FnArg::Receiver(receiver)) = sig.inputs.first() else { - return Err(syn::Error::new_spanned( - &sig.inputs, - "wrapper requires a borrowed receiver", - )); - }; - if receiver.reference.is_none() || receiver.colon_token.is_some() { - return Err(syn::Error::new_spanned( - receiver, - "wrapper requires a borrowed receiver", - )); - } - let Some(syn::FnArg::Typed(waiter)) = sig.inputs.iter().nth(1) else { - return Err(syn::Error::new_spanned( - &sig.inputs, - "wrapper requires &Waiter after self", - )); - }; - let syn::Type::Reference(waiter_ref) = waiter.ty.as_ref() else { - return Err(syn::Error::new_spanned( - &waiter.ty, - "wrapper requires &Waiter after self", - )); - }; - let syn::Type::Path(waiter_path) = waiter_ref.elem.as_ref() else { - return Err(syn::Error::new_spanned( - &waiter.ty, - "wrapper requires &Waiter after self", - )); - }; - if waiter_ref.mutability.is_some() - || !waiter_path.path.is_ident("Waiter") - && waiter_path - .path - .segments - .last() - .is_none_or(|segment| segment.ident != "Waiter") - { - return Err(syn::Error::new_spanned( - &waiter.ty, - "wrapper requires &Waiter after self", - )); - } - let syn::ReturnType::Type(_, ty) = &sig.output else { - return Err(syn::Error::new_spanned( - &sig.output, - "wrapper requires Poll", - )); - }; - let syn::Type::Path(path) = ty.as_ref() else { - return Err(syn::Error::new_spanned(ty, "wrapper requires Poll")); - }; - let segment = path.path.segments.last().unwrap(); - let syn::PathArguments::AngleBracketed(arguments) = &segment.arguments else { - return Err(syn::Error::new_spanned(ty, "wrapper requires Poll")); - }; - let Some(syn::GenericArgument::Type(output)) = arguments - .args - .first() - .filter(|_| arguments.args.len() == 1 && segment.ident == "Poll") - else { - return Err(syn::Error::new_spanned(ty, "wrapper requires Poll")); - }; - // This checkpoint deliberately refuses more than the receiver and waiter. - if sig.inputs.len() != 2 - || !sig.generics.params.is_empty() - || sig.asyncness.is_some() - || sig.unsafety.is_some() - { - return Err(syn::Error::new_spanned( - sig, - "spike supports safe, non-generic methods with only self and waiter", - )); - } - let mut wrapper = sig.clone(); - wrapper.ident = syn::Ident::new(name, sig.ident.span()); - wrapper.asyncness = Some(Default::default()); - wrapper.inputs = std::iter::once(sig.inputs.first().unwrap().clone()).collect(); - wrapper.output = syn::parse_quote!(-> #output); - let docs = method - .attrs - .iter() - .filter(|attr| attr.path().is_ident("doc")); - let vis = &method.vis; - let poll = &sig.ident; - Ok(quote! { - #method - #(#docs)* - #[cfg(feature = "async")] - #vis #wrapper { ::kio::wait(|waiter| self.#poll(waiter)).await } - }) - } - let method = syn::parse_macro_input!(item as syn::ImplItemFn); - expand(args.into(), method) - .unwrap_or_else(syn::Error::into_compile_error) - .into() -} diff --git a/bench/async-wrappers/results.json b/bench/async-wrappers/results.json deleted file mode 100644 index f2de2cbdc26..00000000000 --- a/bench/async-wrappers/results.json +++ /dev/null @@ -1,68 +0,0 @@ -{ - "metadata": { - "head": "263e56b0fcabce1402cc4a405bad06bfa22c8fa3", - "rustc": "rustc 1.98.1 (48a229cea 2026-09-01)", - "jobs": 2, - "incremental": true, - "rounds": 3, - "toolchain": "/nix/store/4klbl675argl2irhl89zrh2n14wilzx9-rust-minimal-1.98.1/bin", - "system": { - "sysname": "Linux", - "release": "7.0.0-42-generic", - "machine": "x86_64" - } - }, - "seconds": { - "handwritten": { - "kio_clean": [ - 2.01, - 1.749, - 1.265 - ], - "net_clean": [ - 36.841, - 28.966, - 24.279 - ], - "net_edit": [ - 1.993, - 2.001, - 1.754 - ] - }, - "rules": { - "kio_clean": [ - 2.114, - 1.402, - 1.609 - ], - "net_clean": [ - 37.555, - 24.447, - 23.468 - ], - "net_edit": [ - 2.241, - 1.751, - 1.61 - ] - }, - "attribute": { - "kio_clean": [ - 5.302, - 4.591, - 6.47 - ], - "net_clean": [ - 35.154, - 27.239, - 29.567 - ], - "net_edit": [ - 2.089, - 1.721, - 1.461 - ] - } - } -} diff --git a/bench/async-wrappers/rules.rs b/bench/async-wrappers/rules.rs deleted file mode 100644 index 271fdb56b0d..00000000000 --- a/bench/async-wrappers/rules.rs +++ /dev/null @@ -1,26 +0,0 @@ -// Spike dialect: docs and signature live once, alongside the canonical poll body. -macro_rules! wrapper { - ($(#[doc = $doc:literal])* [$($attr:tt)*] - $vis:vis fn $poll:ident => $name:ident (&$this:ident, $waiter:ident: &$waiter_ty:ty) - -> $out:ty $body:block) => { - wrapper!(@emit [$(#[doc = $doc])*] [$($attr)*] $vis $poll $name - [&$this] $this $waiter $waiter_ty, $out $body); - }; - ($(#[doc = $doc:literal])* [$($attr:tt)*] - $vis:vis fn $poll:ident => $name:ident (&mut $this:ident, $waiter:ident: &$waiter_ty:ty) - -> $out:ty $body:block) => { - wrapper!(@emit [$(#[doc = $doc])*] [$($attr)*] $vis $poll $name - [&mut $this] $this $waiter $waiter_ty, $out $body); - }; - (@emit [$($doc:tt)*] [$($attr:tt)*] $vis:vis $poll:ident $name:ident - [$($receiver:tt)*] $this:ident $waiter:ident $waiter_ty:ty, $out:ty $body:block) => { - $($doc)* $($attr)* - $vis fn $poll($($receiver)*, $waiter: &$waiter_ty) -> ::std::task::Poll<$out> $body - - $($doc)* - #[cfg(feature = "async")] - $vis async fn $name($($receiver)*) -> $out { - ::kio::wait(|waiter| $this.$poll(waiter)).await - } - }; -} diff --git a/bench/async-wrappers/run.py b/bench/async-wrappers/run.py deleted file mode 100644 index cd823105c03..00000000000 --- a/bench/async-wrappers/run.py +++ /dev/null @@ -1,214 +0,0 @@ -#!/usr/bin/env python3 -"""Materialize and compare wrapper generators without changing production crates.""" -import argparse -import json -import os -from pathlib import Path -import re -import shutil -import statistics -import subprocess -import tarfile -import time - -ROOT = Path(__file__).resolve().parents[2] -HERE = Path(__file__).resolve().parent -OUT = ROOT / ".tmp/async-wrappers" -TOOLCHAIN = Path(subprocess.check_output(["rustc", "--print", "sysroot"], text=True).strip()) / "bin" -VARIANTS = ("handwritten", "rules", "attribute") -MODULES = { - "rs/kio/src/queue.rs": ("pop",), - "rs/moq-net/src/model/broadcast.rs": ("requested_track", "closed", "wanted_changed"), -} - - -def command(args, cwd, log, env=None, success=True): - start = time.monotonic() - with log.open("w") as output: - result = subprocess.run(args, cwd=cwd, env=env, stdout=output, stderr=subprocess.STDOUT) - if (result.returncode == 0) != success: - raise RuntimeError(f"unexpected exit {result.returncode}: {args}; see {log}") - return round(time.monotonic() - start, 3) - - -def replace(path, old, new): - text = path.read_text() - if old not in text: - raise RuntimeError(f"source moved: {old!r} in {path}") - path.write_text(text.replace(old, new)) - - -def prepare(variant, archive): - tree = OUT / variant - if tree.exists(): - raise RuntimeError(f"{tree} exists; choose a new --output directory") - tree.mkdir() - with tarfile.open(archive) as source: - source.extractall(tree, filter="data") - replace(tree / "rs/kio/Cargo.toml", "[features]\n", '[features]\nasync = []\n') - replace(tree / "rs/moq-net/Cargo.toml", 'async = [', 'async = ["kio/async", ') - replace(tree / "rs/kio/src/lib.rs", '#![warn(missing_docs)]', - '#![warn(missing_docs)]\nextern crate self as kio;') - if variant == "attribute": - replace(tree / "rs/kio/Cargo.toml", 'async = []', 'async = ["macros"]') - with (tree / "rs/kio/macros/src/lib.rs").open("a") as target: - target.write("\n" + (HERE / "attribute.rs").read_text()) - replace(tree / "rs/kio/src/lib.rs", "pub use kio_macros::machine;", - "pub use kio_macros::{machine, wrapper};") - for module, names in MODULES.items(): - path = tree / module - text = path.read_text() - # Only top-level impl methods in these two known source files. Fail if - # their spelling or inventory changes, rather than measuring a partial edit. - pattern = r"(?:\t///[^\n]*\n)*(?:\t#\[[^\n]*\]\n)*\tpub async fn (" + "|".join(names) + r")\([^\n]*\)[^\n]*\{\n.*?\n\t}\n" - expected = 1 if "kio/" in module else 4 - text, count = re.subn(pattern, "", text, flags=re.S) - if count != expected: - raise RuntimeError(f"expected {expected} wrappers, found {count}: {module}") - seen = 0 - def lower(match): - nonlocal seen - seen += 1 - docs, attrs, name, receiver, waiter, waiter_ty, output, body = match.groups() - async_name = name.removeprefix("poll_") - if variant == "attribute": - return f'{docs}{attrs}\t#[cfg_attr(feature = "async", kio::wrapper)]\n\tpub fn {name}({receiver}, {waiter}: &{waiter_ty}) -> Poll<{output}> {body}\n' - if variant == "rules": - return f'{docs}\t[{attrs.strip()}]\n\tpub fn {name} => {async_name}({receiver}, {waiter}: &{waiter_ty}) -> {output} {body}\n' - return f'{docs}{attrs}\tpub fn {name}({receiver}, {waiter}: &{waiter_ty}) -> Poll<{output}> {body}\n{docs}\t#[cfg(feature = "async")]\n\tpub async fn {async_name}({receiver}) -> {output} {{\n\t\t::kio::wait(|waiter| self.{name}(waiter)).await\n\t}}\n' - pattern = (r"((?:\t///[^\n]*\n)*)((?:\t#\[[^\n]*\]\n)*)\tpub fn " - r"(poll_(?:" + "|".join(names) + r"))\((&(?:mut )?self), (\w+): &([^\n]+?)\) -> Poll<([^\n]+)> (\{\n.*?\n\t})\n") - if variant == "rules": - # Wrap each declaration/body as a single macro input. - def rules(match): - return "\twrapper! {\n" + lower(match) + "\t}\n" - text = re.sub(pattern, rules, text, flags=re.S) - offset = re.match(r"(?://![^\n]*\n)*", text).end() - text = text[:offset] + (HERE / "rules.rs").read_text() + "\n" + text[offset:] - else: - text = re.sub(pattern, lower, text, flags=re.S) - if seen != expected: - raise RuntimeError(f"expected {expected} polls, found {seen}: {module}") - path.write_text(text) - return tree - - -def environment(tree): - env = os.environ.copy() - # Every variant owns its cache. No remote compiler cache or enclosing wrapper - # may turn a clean measurement into a cache hit. - for key in ("RUSTC_WRAPPER", "RUSTC_WORKSPACE_WRAPPER", "CARGO_ENCODED_RUSTFLAGS", "RUSTFLAGS", "RUSTDOCFLAGS"): - env.pop(key, None) - env.update(CARGO_TARGET_DIR=str(tree / "target"), CARGO_INCREMENTAL="1", CARGO_BUILD_JOBS="2", - PATH=f"{TOOLCHAIN}:{env['PATH']}", BASH_ENV="/dev/null", - RUSTC=str(TOOLCHAIN / "rustc"), RUSTDOC=str(TOOLCHAIN / "rustdoc")) - return env - - -def check(tree, package, label, features=True, success=True): - args = ["cargo", "check", "--locked", "--offline", "--lib", "-p", package, "--no-default-features"] - if features: - args += ["--features", f"{package}/async"] - return command(args, tree, OUT / f"{tree.name}-{package}-{label}.log", environment(tree), success) - - -def verify(tree): - for package in ("kio", "moq-net"): - check(tree, package, "on") - check(tree, package, "off", False) - args = ["cargo", "doc", "--locked", "--offline", "--no-deps", "--lib", "-p", package, - "--no-default-features", "--features", f"{package}/async"] - env = environment(tree) - env["RUSTDOCFLAGS"] = "-D warnings" - command(args, tree, OUT / f"{tree.name}-{package}-docs.log", env) - for file, poll, wrapper, phrase in ( - ("kio/struct.Queue.html", "poll_pop", "pop", "Poll for the next item."), - ("moq_net/broadcast/struct.Requests.html", "poll_requested_track", "requested_track", "Poll"), - ): - html = (tree / "target/doc" / file).read_text() - # Compare actual rendered documentation, not just matching source attrs. - docs = [] - for name in (poll, wrapper): - section = html.split(f'id="method.{name}"', 1)[1].split('', 1)[0] - docs.append(section.split('
', 1)[1]) - if docs[0] != docs[1] or phrase not in docs[0]: - raise RuntimeError(f"docs differ: {file} {poll}/{wrapper}") - # Malformed receiver: useful diagnostics for a mistake in the generator input. - path = tree / "rs/kio/src/queue.rs" - original = path.read_text() - if tree.name == "attribute": - bad = original.replace("pub fn poll_pop(&self,", "pub fn poll_pop(self,") - elif tree.name == "rules": - bad = original.replace("pub fn poll_pop => pop(&self,", "pub fn poll_pop => pop(self,") - else: - bad = original.replace("pub fn poll_pop(&self,", "pub fn poll_pop(self,") - if bad == original: - raise RuntimeError("diagnostic injection missed its target") - path.write_text(bad) - try: - check(tree, "kio", "bad-receiver", success=False) - diagnostic = (OUT / f"{tree.name}-kio-bad-receiver.log").read_text() - expected = {"handwritten": "cannot move out", "rules": "no rules expected", "attribute": "wrapper requires a borrowed receiver"}[tree.name] - if expected not in diagnostic: - raise RuntimeError(f"missing expected diagnostic: {expected}") - finally: - path.write_text(original) - print(f"verified {tree.name}: feature on/off, rustdoc, rejected owned receiver", flush=True) - - -def main(): - global OUT - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--output", type=Path, default=OUT) - parser.add_argument("--rounds", type=int, default=3) - parser.add_argument("--verify-only", action="store_true") - args = parser.parse_args() - OUT = args.output.resolve() - if not OUT.is_relative_to(ROOT / ".tmp"): - parser.error("--output must be inside this checkout's .tmp/") - if args.rounds < 1: - parser.error("--rounds must be positive") - OUT.mkdir(parents=True, exist_ok=True) - archive = OUT / "source.tar" - with archive.open("wb") as output: - subprocess.run(["git", "archive", "HEAD"], cwd=ROOT, stdout=output, check=True) - trees = {variant: prepare(variant, archive) for variant in VARIANTS} - results = {variant: {"kio_clean": [], "net_clean": [], "net_edit": []} for variant in VARIANTS} - metadata = { - "head": subprocess.check_output(["git", "rev-parse", "HEAD"], cwd=ROOT, text=True).strip(), - "rustc": subprocess.check_output(["rustc", "--version"], text=True).strip(), - "jobs": 2, "incremental": True, "rounds": args.rounds, "toolchain": str(TOOLCHAIN), - "system": {key: getattr(os.uname(), key) for key in ("sysname", "release", "machine")}, - } - for tree in trees.values(): - verify(tree) - if not args.verify_only: - for round_number in range(args.rounds): - order = VARIANTS[round_number % 3:] + VARIANTS[:round_number % 3] - for variant in order: - tree = trees[variant] - if (tree / "target").exists(): - shutil.rmtree(tree / "target") - # kio separately exposes the dependency cost hidden by moq-net's - # existing proc-macro graph. Then discard it for the net clean run. - results[variant]["kio_clean"].append(check(tree, "kio", f"clean-{round_number}")) - shutil.rmtree(tree / "target") - results[variant]["net_clean"].append(check(tree, "moq-net", f"clean-{round_number}")) - path = tree / "rs/moq-net/src/model/broadcast.rs" - original = path.read_text() - edited = original.replace("predicate guaranteed a request", f"spike edit {round_number}: request predicate guaranteed an entry") - if edited == original: - raise RuntimeError("body edit missed its target") - path.write_text(edited) - try: - results[variant]["net_edit"].append(check(tree, "moq-net", f"edit-{round_number}")) - finally: - path.write_text(original) - print(variant, round_number, results[variant], flush=True) - (OUT / "results.json").write_text(json.dumps({"metadata": metadata, "seconds": results}, indent=2) + "\n") - for variant, metrics in results.items(): - print(variant, {key: statistics.median(values) for key, values in metrics.items()}, flush=True) - - -if __name__ == "__main__": - main() diff --git a/quest/m1/poll-first/async-feature.md b/quest/m1/poll-first/async-feature.md index cc68154a1e6..8d7b31b3c20 100644 --- a/quest/m1/poll-first/async-feature.md +++ b/quest/m1/poll-first/async-feature.md @@ -14,21 +14,34 @@ and the bindings and JS keep their APIs. Decided with the maintainer on 2026-10-25: poll is canonical, async is the ergonomic layer, and the gating proceeds regardless of the build measurement. -- **Generator checkpoint:** [the reproducible spike](/bench/async-wrappers/README.md) - targets `kio::Queue::pop` and the four broadcast wrappers without changing - production crates. The [completed comparison](/bench/async-wrappers/RESULTS.md) - recommends a method attribute reusing kio-macros: ordinary Rust poll source, - automatic names, and targeted errors, at a measured standalone kio build cost. - This is not a maintainer decision. Choose the generator before broad rollout; - declarative generation remains viable and rust2 needs a new compiler spike. - Generalize the chosen prototype, add CI coverage for its contract, then complete - the coroutine conversion, feature wiring, and library-only verification below. -- **Spike the generator first** on kio and one moq-net module, then pick one: - - a `macro_rules!` over a list of signatures (no proc-macro cost); - - a proc macro attribute on the `poll_*` method or its impl block; - - a rust2-driver lowering. - Compare compile time (clean and edit, on moq-net), error spans, and docs output. - A wrapper's docs come from its `poll_*` method. +Decided with the maintainer on 2026-11-01, from the +[generator spike](https://github.com/moq-dev/moq/blob/91b1ec62e1e35295ef74d0f7cee1afc49739bf00/bench/async-wrappers/RESULTS.md): +the generator is a method attribute in kio-macros, `#[kio::wrapper]`. The poll +stays ordinary Rust, the wrapper's name, signature, and docs are derived from it, +and a standalone kio build with `async` pays the measured proc-macro cost (about +3.5 seconds clean). + +Done in [moq-dev/moq#7506](https://github.com/moq-dev/moq/pull/7506): +`#[kio::wrapper]` with its contract in kio-macros' behavior and UI tests; kio's +default-off `async` feature, under which kio compiles no coroutines; moq-net's +`async` enabling `kio/async`; moq-net's four `broadcast` wrappers generated; and +`just rs coroutines`, in the nightly feature matrix, which scans the MIR of kio's +and moq-net's poll-only builds. Callers gate the attribute with +`#[cfg_attr(feature = "async", kio::wrapper)]`, so a poll-only build skips the +proc macro, and the scan catches a wrapper left ungated. + +What remains: + +- **moq-net's remaining wrappers:** about 47 hand-written + `kio::wait(|waiter| self.poll_x(waiter, ..)).await` wrappers become the attribute + on their polls, each poll's docs rewritten to read for both forms. A few stay + hand-written: those that take a closure by value and poll it by `&mut`, and the + `#[cfg(test)]` helpers the poll-only tests use. +- **moq-net's poll-only coroutines:** `auth::until` (behind `Grant::guard`), and + `model::origin`'s `run_front` and `serve_front`, still compile without `async`. + Gate `until` with its async callers, or make it a poll, and turn the front + driver into a hand-written poll machine (its body is one large `kio::wait`). + Then drop their `--allow`s from `just rs coroutines`. - **Logic in `async fn` bodies:** an `async fn` that does more than call one `poll_*` becomes a `poll_*` first, or moves out of the library to the edges. List each one in the PR. moq-net's straight-line ones became `#[kio::machine]`s in @@ -37,14 +50,19 @@ ergonomic layer, and the gating proceeds regardless of the build measurement. bodies with a loop or a branch around a wait (about 59 in moq-net) are converted by hand, since the macro stays straight-line only (decided 2026-10-28). +- **Sibling libraries:** the poll-based libraries with hand-written wrappers + (moq-mux, moq-hls, moq-auth, moq-stats, moq-room, moq-json, moq-sign, moq-e2ee, + qmux, and others) gain an `async` feature, generate their wrappers, and join + `just rs coroutines`. +- **Default off:** moq-net's `async` feature is still on by default. Turn it off, + and enable it at the edges: binaries, tests, tokio glue, and the bindings. - **Tests:** `#[kio::test]` and `#[tokio::test]` bodies stay async, so test targets enable the feature. - **Cargo:** feature unification means one dependent enabling `async` enables it for the whole build. Make sure library-only builds, such as `cargo check -p moq-net` and the rust2 bench's library-only build, leave it off. -- **Docs:** update `rs/AGENTS.md`'s rule ("New logic is a `poll_*` with an `async` - helper") to say the helper is generated and feature-gated. Public API: every library crate gains an `async` feature, and its async wrappers move behind it. Add an Impact migration note for the published crates whenever -a consumer must enable `async`, including with `default-features = false`. +a consumer must enable `async`, including with `default-features = false`, as +`rs/kio/MIGRATE.md` does for kio. diff --git a/rs/AGENTS.md b/rs/AGENTS.md index 4b3cd8bb03e..9d406f4cf5c 100644 --- a/rs/AGENTS.md +++ b/rs/AGENTS.md @@ -20,9 +20,9 @@ Most crates use a split-handle pattern: a `Producer` writes, any number of `Cons Two ways to drive things, both backed by `kio`: - `poll_*` functions take a `&kio::Waiter` and return `Poll<...>`, drivable from any executor or synchronously. Like `Future`, but wakers are cleaned up on drop. -- `async fn` wrappers, usually `kio::wait(|waiter| self.poll_x(waiter))`. +- `async fn` wrappers, `kio::wait(|waiter| self.poll_x(waiter))`, generated by `#[cfg_attr(feature = "async", kio::wrapper)]` on the poll (kio's `macros` feature) and compiled only with the crate's `async` feature. Write the poll's docs for both forms: what it resolves to, not how it registers. -Prefer poll. New logic is a `poll_*` with an `async` helper, not the other way around. `pin!` then polling inside `kio::wait` is a code smell, as is `tokio::spawn` / `select!` inside a library; use `kio` primitives. +Prefer poll. New logic is a `poll_*` with a generated `async` wrapper, not the other way around. A library build without `async` compiles no coroutines; `just rs coroutines` checks kio's and moq-net's. `pin!` then polling inside `kio::wait` is a code smell, as is `tokio::spawn` / `select!` inside a library; use `kio` primitives. - A `poll_*` that waits on several things in a fixed order is a `#[kio::machine]` (kio's `macros` feature): straight-line code with a `wait!` at each wait, from which it writes the state enum and its transitions. Hand-write the machine when a wait sits in a loop or branch, or when it yields more than once (a stream's `poll_next`). A conditional wait is one whose poll is ready at once. - Use `ready!(...)` instead of `Poll::Pending => return Poll::Pending`. diff --git a/rs/justfile b/rs/justfile index b6b36e0c9ce..f26ba8760ae 100644 --- a/rs/justfile +++ b/rs/justfile @@ -215,6 +215,7 @@ c-tests: # Compile the full feature matrix, including isolated combinations and their tests. features: just rs net-features --all + just rs coroutines just rs quic-features --all just rs tokio-features --all just rs media-features --all @@ -236,6 +237,13 @@ features: net-features *args: bash sh/rs/net-features.sh "$@" +# Prove the poll-only library builds compile no coroutines and document cleanly. +# moq-net allows the async fns still waiting on a poll conversion. +coroutines: + bash sh/rs/coroutines.sh kio -- --no-default-features --features macros,runtime,manual,serde + RUSTDOCFLAGS="-D warnings" bash sh/local.sh cargo doc --locked -p kio --no-deps --no-default-features --features macros,runtime,manual,serde + bash sh/rs/coroutines.sh moq-net --allow until --allow run_front --allow serve_front -- --no-default-features + # Prove moq-quic leaves kio's dev-only clock runtime to its consumers, and lints without its default features; a workspace build unifies both back in. [positional-arguments] quic-features *args: diff --git a/rs/kio/Cargo.toml b/rs/kio/Cargo.toml index 1287cd12bb9..6e74d9e91b8 100644 --- a/rs/kio/Cargo.toml +++ b/rs/kio/Cargo.toml @@ -13,26 +13,32 @@ keywords = ["async", "producer", "consumer", "clock", "simulation"] categories = ["asynchronous", "concurrency", "date-and-time", "development-tools::testing"] [package.metadata.docs.rs] -features = ["runtime", "manual", "sim", "serde", "macros"] +features = ["async", "runtime", "manual", "sim", "serde", "macros"] # The self dev-dependency below only enables features; no code names it. [package.metadata.cargo-shear] ignored = ["kio"] [features] -# Lean by default: without `runtime`, the clock compiles no host timer backend. +# Lean by default: without `runtime`, the clock compiles no host timer backend, and +# without `async`, no coroutines. default = [] +# The `async fn` beside each `poll_*` method, generated by `#[kio::wrapper]`, and the +# few `async` helpers with no poll of their own (`Queue::push`, the `wait` predicates, +# `clock::drive`, `Interval::tick`). Off by default: enable it where you `.await`. +async = ["macros"] # Host timers on tokio's clock. runtime = ["dep:tokio"] # The deterministic single-threaded executor, `#[kio::test]`, and per-task Ready # poll accounting. Meant for dev-dependencies, so production builds never compile it. -sim = ["dep:futures", "dep:kio-macros"] +sim = ["async", "dep:futures"] # `clock::Manual::install`: a clock that only a test moves, as one tokio runtime's # ambient clock. Meant for test builds, so production builds never compile it. manual = ["runtime"] # Numeric Unix-second JSON adapters for public wall times. serde = ["dep:serde", "dep:serde_with"] -# `#[kio::machine]`, which writes a `poll_*` state machine from straight-line code. +# `#[kio::machine]`, which writes a `poll_*` state machine from straight-line code, and +# `#[kio::wrapper]`, which writes the `async fn` that awaits one. macros = ["dep:kio-macros"] [dependencies] @@ -50,9 +56,9 @@ tokio = { workspace = true, features = ["rt", "time"], optional = true } # Bench-only: the FuturesUnordered comparison in benches/tasks.rs. criterion = { package = "bench-harness", path = "../bench-harness" } futures = "0.3" -# Enable the simulator and the installed manual clock so the clock's tests run in -# the merge gate. -kio = { path = ".", features = ["manual", "sim", "serde"] } +# Enable the async wrappers, the simulator, and the installed manual clock so the +# tests that await them run in the merge gate. +kio = { path = ".", features = ["async", "manual", "sim", "serde"] } serde_json = { workspace = true } test-env = { path = "../test-env" } tokio = { workspace = true, features = ["macros", "rt", "rt-multi-thread", "test-util", "time"] } diff --git a/rs/kio/MIGRATE.md b/rs/kio/MIGRATE.md index 2810c099b36..33830c627fc 100644 --- a/rs/kio/MIGRATE.md +++ b/rs/kio/MIGRATE.md @@ -4,6 +4,14 @@ How to update your code for each breaking change, newest first. ## Unreleased +### `async` methods moved behind the `async` feature + +`Queue::{push, pop}`, `Producer::{wait, closed, wanted_changed}`, `Consumer::{wait, +closed}`, `Shared::wait`, `ConsumerWeak::closed`, `ProducerWeak::closed`, +`clock::drive`, and `clock::Interval::tick` now compile only with the new `async` +feature, which is off by default; `sim` enables it. Add `features = ["async"]` to +the `kio` dependency of each crate that `.await`s them. Each `poll_*` method is unchanged, and still drives the same wait without it. + ### `Pollable` merged into `Task`, and unused surface removed (#4156) `Fan`, `Hold`, `WeakLock`, `Pollable`, and the public `Lock::downgrade` are gone. diff --git a/rs/kio/macros/src/lib.rs b/rs/kio/macros/src/lib.rs index c0127347d67..b28c7e341fd 100644 --- a/rs/kio/macros/src/lib.rs +++ b/rs/kio/macros/src/lib.rs @@ -89,6 +89,68 @@ pub fn machine(args: proc_macro::TokenStream, item: proc_macro::TokenStream) -> } mod sim_test; +mod wrapper; + +/// Generate the `async fn` that waits on a `poll_*` method. +/// +/// On `poll_`, it adds `async fn ` beside it, `kio::wait`ing on the poll. +/// The wrapper takes the poll's receiver and arguments, minus the waiter, and returns +/// what the poll is ready with. It copies the poll's visibility, generics, docs, `cfg`, +/// and deprecation, then links back to the poll. The poll stays as written, so its +/// docs describe both forms: what the call is ready with, not how it registers. +/// +/// ``` +/// use std::task::Poll; +/// +/// struct Counter { +/// values: kio::Queue, +/// } +/// +/// impl Counter { +/// /// The next value at or above `min`, skipping smaller ones. +/// #[kio::wrapper] +/// fn poll_at_least(&self, waiter: &kio::Waiter, min: u64) -> Poll> { +/// loop { +/// match self.values.poll_pop(waiter) { +/// Poll::Ready(Ok(value)) if value < min => continue, +/// other => return other, +/// } +/// } +/// } +/// } +/// +/// # async fn demo(counter: Counter) -> Result { +/// // Generated: `async fn at_least(&self, min: u64) -> Result`. +/// counter.at_least(10).await +/// # } +/// ``` +/// +/// A library keeps its wrappers behind its `async` feature, and its poll-only build +/// free of this macro, with `#[cfg_attr(feature = "async", kio::wrapper)]`. +/// +/// The poll must: +/// +/// - Be a method taking `&self` or `&mut self`, not `async`, `const`, `unsafe`, or +/// `extern`. +/// - Take exactly one `&Waiter` (any path ending in `Waiter`), in any position. +/// - Bind every other argument to a plain name (`mut` is dropped). The wrapper passes +/// each one to every poll unchanged, so it must be `Copy` or a reference; rustc +/// reports the move from the closure otherwise. +/// - Return `Poll` (any path ending in `Poll`). The wrapper returns `T`. +/// +/// Other attributes, such as `#[track_caller]` or `#[inline]`, stay on the poll alone. +#[proc_macro_attribute] +pub fn wrapper(args: proc_macro::TokenStream, item: proc_macro::TokenStream) -> proc_macro::TokenStream { + let method = parse_macro_input!(item as syn::ImplItemFn); + // A rejected poll still compiles, so the error is the only one reported. + wrapper::expand(args.into(), &method) + .unwrap_or_else(|err| { + let mut out = method.to_token_stream(); + out.extend(err.into_compile_error()); + out + }) + .into() +} /// Run an `async fn` test to completion on `kio::clock::sim::run`. #[proc_macro_attribute] diff --git a/rs/kio/macros/src/wrapper.rs b/rs/kio/macros/src/wrapper.rs new file mode 100644 index 00000000000..b848fe591a6 --- /dev/null +++ b/rs/kio/macros/src/wrapper.rs @@ -0,0 +1,231 @@ +//! The `#[kio::wrapper]` expansion: an `async fn` beside the `poll_*` method it waits on. + +use proc_macro2::{Span, TokenStream}; +use quote::quote; +use syn::{ + Attribute, FnArg, GenericArgument, GenericParam, Ident, ImplItemFn, Pat, PathArguments, ReturnType, Type, + punctuated::Punctuated, spanned::Spanned, +}; + +pub(crate) fn expand(args: TokenStream, method: &ImplItemFn) -> syn::Result { + if !args.is_empty() { + return Err(syn::Error::new_spanned(args, "#[kio::wrapper] takes no arguments")); + } + let sig = &method.sig; + if let Some(token) = &sig.asyncness { + return Err(syn::Error::new_spanned( + token, + "#[kio::wrapper] goes on a poll method, not an `async fn`", + )); + } + if let Some(token) = &sig.constness { + return Err(syn::Error::new_spanned( + token, + "#[kio::wrapper] cannot wrap a `const fn`", + )); + } + if let Some(token) = &sig.unsafety { + return Err(syn::Error::new_spanned( + token, + "#[kio::wrapper] cannot wrap an `unsafe fn`", + )); + } + if let Some(abi) = &sig.abi { + return Err(syn::Error::new_spanned( + abi, + "#[kio::wrapper] cannot wrap an `extern fn`", + )); + } + if let Some(variadic) = &sig.variadic { + return Err(syn::Error::new_spanned( + variadic, + "#[kio::wrapper] cannot wrap a variadic fn", + )); + } + + let poll = &sig.ident; + let name = poll.to_string(); + let Some(name) = name.strip_prefix("poll_").filter(|name| !name.is_empty()) else { + return Err(syn::Error::new_spanned( + poll, + "#[kio::wrapper] needs a `poll_` method", + )); + }; + // `poll_match` waits as `r#match`. + let name = match syn::parse_str::(name) { + Ok(_) => Ident::new(name, poll.span()), + Err(_) => Ident::new_raw(name, poll.span()), + }; + + let mut inputs = sig.inputs.iter(); + let receiver = match inputs.next() { + Some(FnArg::Receiver(receiver)) if receiver.reference.is_some() && receiver.colon_token.is_none() => receiver, + Some(FnArg::Receiver(receiver)) => { + return Err(syn::Error::new_spanned( + receiver, + "#[kio::wrapper] needs a borrowed receiver, `&self` or `&mut self`", + )); + } + _ => { + return Err(syn::Error::new( + sig.paren_token.span.join(), + "#[kio::wrapper] needs a borrowed receiver, `&self` or `&mut self`", + )); + } + }; + + // Every other argument is passed through to each poll; the waiter is the wrapper's own. + let waiter = Ident::new("waiter", Span::mixed_site()); + let mut params = Punctuated::::new(); + let mut call = Vec::new(); + let mut found = None; + for input in inputs { + let FnArg::Typed(input) = input else { + return Err(syn::Error::new_spanned(input, "`self` must be the first argument")); + }; + if let Some(mutable) = waiter_ref(&input.ty) { + if mutable { + return Err(syn::Error::new_spanned( + &input.ty, + "#[kio::wrapper] needs the waiter as `&Waiter`, not `&mut Waiter`", + )); + } + if let Some(first) = found.replace(input) { + let mut err = + syn::Error::new_spanned(&input.ty, "#[kio::wrapper] needs exactly one `&Waiter` argument"); + err.combine(syn::Error::new_spanned(&first.ty, "the first `&Waiter` is here")); + return Err(err); + } + call.push(quote! { #waiter }); + continue; + } + let Pat::Ident(pat) = input.pat.as_ref() else { + return Err(syn::Error::new_spanned( + &input.pat, + "#[kio::wrapper] needs each argument bound to a plain name", + )); + }; + if pat.by_ref.is_some() || pat.subpat.is_some() { + return Err(syn::Error::new_spanned( + &input.pat, + "#[kio::wrapper] needs each argument bound to a plain name", + )); + } + // `mut` only affects the poll's own binding: the wrapper never rebinds it. + let mut param = input.clone(); + param.pat = Box::new(Pat::Ident(syn::PatIdent { + mutability: None, + ..pat.clone() + })); + params.push(FnArg::Typed(param)); + let ident = &pat.ident; + call.push(quote! { #ident }); + } + if found.is_none() { + return Err(syn::Error::new( + sig.paren_token.span.join(), + "#[kio::wrapper] needs a `&Waiter` argument", + )); + } + + let output = ready_type(sig)?; + + // Type and const parameters, named so the wrapper never relies on inference to pick them. + let turbofish = sig + .generics + .params + .iter() + .filter_map(|param| match param { + GenericParam::Type(param) => Some(¶m.ident), + GenericParam::Const(param) => Some(¶m.ident), + GenericParam::Lifetime(_) => None, + }) + .collect::>(); + let turbofish = match turbofish.is_empty() { + true => quote! {}, + false => quote! { ::<#(#turbofish),*> }, + }; + + let mut wrapper = sig.clone(); + wrapper.ident = name; + wrapper.asyncness = Some(syn::Token![async](poll.span())); + wrapper.inputs = std::iter::once(FnArg::Receiver(receiver.clone())) + .chain(params) + .collect(); + wrapper.output = match output { + Type::Tuple(unit) if unit.elems.is_empty() => ReturnType::Default, + output => syn::parse_quote_spanned!(sig.output.span()=> -> #output), + }; + + let attrs = method.attrs.iter().filter(|attr| copied(attr)).collect::>(); + let link = format!("Awaits [`{poll}`](Self::{poll})."); + let link = match attrs.iter().any(|attr| attr.path().is_ident("doc")) { + true => quote! { #[doc = ""] #[doc = #link] }, + false => quote! { #[doc = #link] }, + }; + // A deprecated wrapper calls its deprecated poll without a warning of its own. + let allow = match attrs.iter().any(|attr| attr.path().is_ident("deprecated")) { + true => quote! { #[allow(deprecated, reason = "a deprecated wrapper of a deprecated poll")] }, + false => quote! {}, + }; + let vis = &method.vis; + // The receiver's own `self`, which a `self` spanned anywhere else would not resolve to. + let this = &receiver.self_token; + + Ok(quote! { + #method + + #(#attrs)* + #link + #allow + #vis #wrapper { + ::kio::wait(|#waiter| #this.#poll #turbofish(#(#call),*)).await + } + }) +} + +/// Whether `ty` is a `&mut` if it is a reference to `Waiter` (by any path ending in it). +fn waiter_ref(ty: &Type) -> Option { + let Type::Reference(reference) = ty else { return None }; + let Type::Path(path) = reference.elem.as_ref() else { + return None; + }; + let waiter = path.qself.is_none() + && path + .path + .segments + .last() + .is_some_and(|segment| segment.ident == "Waiter" && segment.arguments.is_none()); + waiter.then_some(reference.mutability.is_some()) +} + +/// `T` from `-> Poll`, by any path ending in `Poll`. +fn ready_type(sig: &syn::Signature) -> syn::Result<&Type> { + const MESSAGE: &str = "#[kio::wrapper] needs a method returning `Poll`"; + let output = &sig.output; + let error = || syn::Error::new_spanned(output, MESSAGE); + let ReturnType::Type(_, ty) = output else { + // No return type to point at: point where it would go. + return Err(syn::Error::new(sig.paren_token.span.close(), MESSAGE)); + }; + let Type::Path(path) = ty.as_ref() else { + return Err(error()); + }; + let Some(segment) = path.path.segments.last().filter(|segment| segment.ident == "Poll") else { + return Err(error()); + }; + let PathArguments::AngleBracketed(arguments) = &segment.arguments else { + return Err(error()); + }; + match arguments.args.first() { + Some(GenericArgument::Type(ty)) if arguments.args.len() == 1 => Ok(ty), + _ => Err(error()), + } +} + +/// What the wrapper inherits: docs, the conditions it compiles under, and deprecation. +fn copied(attr: &Attribute) -> bool { + ["doc", "cfg", "deprecated"] + .iter() + .any(|name| attr.path().is_ident(name)) +} diff --git a/rs/kio/macros/tests/ui/wrapper_moved_argument.rs b/rs/kio/macros/tests/ui/wrapper_moved_argument.rs new file mode 100644 index 00000000000..18572e63618 --- /dev/null +++ b/rs/kio/macros/tests/ui/wrapper_moved_argument.rs @@ -0,0 +1,13 @@ +// An argument that is neither `Copy` nor a reference: every poll would move it. +use std::task::Poll; + +struct S; + +impl S { + #[kio::wrapper] + fn poll_len(&self, _waiter: &kio::Waiter, name: String) -> Poll { + Poll::Ready(name.len()) + } +} + +fn main() {} diff --git a/rs/kio/macros/tests/ui/wrapper_moved_argument.stderr b/rs/kio/macros/tests/ui/wrapper_moved_argument.stderr new file mode 100644 index 00000000000..4987429988c --- /dev/null +++ b/rs/kio/macros/tests/ui/wrapper_moved_argument.stderr @@ -0,0 +1,21 @@ +error[E0507]: cannot move out of value, a captured variable in an `FnMut` closure + --> tests/ui/wrapper_moved_argument.rs:8:44 + | +7 | #[kio::wrapper] + | --------------- captured by this `FnMut` closure +8 | fn poll_len(&self, _waiter: &kio::Waiter, name: String) -> Poll { + | ^^^^ + | | + | captured outer variable + | move occurs because value has type `String`, which does not implement the `Copy` trait + | value is moved here + | +help: `Fn` and `FnMut` closures require captured values to be able to be consumed multiple times, but `FnOnce` closures may consume them only once + --> $WORKSPACE/rs/kio/src/waiter.rs + | + | F: FnMut(&Waiter) -> Poll + Unpin, + | ^^^^^^^^^^^^^^^^^^^^^^^^^ +help: consider cloning the value if the performance cost is acceptable + | +8 | fn poll_len(&self, _waiter: &kio::Waiter, name.clone(): String) -> Poll { + | ++++++++ diff --git a/rs/kio/macros/tests/ui/wrapper_name.rs b/rs/kio/macros/tests/ui/wrapper_name.rs new file mode 100644 index 00000000000..7bc1e4002e2 --- /dev/null +++ b/rs/kio/macros/tests/ui/wrapper_name.rs @@ -0,0 +1,13 @@ +// A method without the `poll_` prefix. +use std::task::Poll; + +struct S(kio::Queue); + +impl S { + #[kio::wrapper] + fn pop(&self, waiter: &kio::Waiter) -> Poll> { + self.0.poll_pop(waiter) + } +} + +fn main() {} diff --git a/rs/kio/macros/tests/ui/wrapper_name.stderr b/rs/kio/macros/tests/ui/wrapper_name.stderr new file mode 100644 index 00000000000..ff6c34a2dfa --- /dev/null +++ b/rs/kio/macros/tests/ui/wrapper_name.stderr @@ -0,0 +1,5 @@ +error: #[kio::wrapper] needs a `poll_` method + --> tests/ui/wrapper_name.rs:8:5 + | +8 | fn pop(&self, waiter: &kio::Waiter) -> Poll> { + | ^^^ diff --git a/rs/kio/macros/tests/ui/wrapper_no_receiver.rs b/rs/kio/macros/tests/ui/wrapper_no_receiver.rs new file mode 100644 index 00000000000..3a2e40753f4 --- /dev/null +++ b/rs/kio/macros/tests/ui/wrapper_no_receiver.rs @@ -0,0 +1,13 @@ +// An associated function, with no receiver to borrow. +use std::task::Poll; + +struct S; + +impl S { + #[kio::wrapper] + fn poll_ready(_waiter: &kio::Waiter) -> Poll<()> { + Poll::Ready(()) + } +} + +fn main() {} diff --git a/rs/kio/macros/tests/ui/wrapper_no_receiver.stderr b/rs/kio/macros/tests/ui/wrapper_no_receiver.stderr new file mode 100644 index 00000000000..3ebc739bab9 --- /dev/null +++ b/rs/kio/macros/tests/ui/wrapper_no_receiver.stderr @@ -0,0 +1,5 @@ +error: #[kio::wrapper] needs a borrowed receiver, `&self` or `&mut self` + --> tests/ui/wrapper_no_receiver.rs:8:15 + | +8 | fn poll_ready(_waiter: &kio::Waiter) -> Poll<()> { + | ^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/rs/kio/macros/tests/ui/wrapper_output.rs b/rs/kio/macros/tests/ui/wrapper_output.rs new file mode 100644 index 00000000000..b6f3b560503 --- /dev/null +++ b/rs/kio/macros/tests/ui/wrapper_output.rs @@ -0,0 +1,15 @@ +// A method that is not ready with a value: no return type, or not a `Poll`. + +struct S; + +impl S { + #[kio::wrapper] + fn poll_nothing(&self, _waiter: &kio::Waiter) {} + + #[kio::wrapper] + fn poll_option(&self, _waiter: &kio::Waiter) -> Option<()> { + None + } +} + +fn main() {} diff --git a/rs/kio/macros/tests/ui/wrapper_output.stderr b/rs/kio/macros/tests/ui/wrapper_output.stderr new file mode 100644 index 00000000000..d8caf52258d --- /dev/null +++ b/rs/kio/macros/tests/ui/wrapper_output.stderr @@ -0,0 +1,11 @@ +error: #[kio::wrapper] needs a method returning `Poll` + --> tests/ui/wrapper_output.rs:7:46 + | +7 | fn poll_nothing(&self, _waiter: &kio::Waiter) {} + | ^ + +error: #[kio::wrapper] needs a method returning `Poll` + --> tests/ui/wrapper_output.rs:10:47 + | +10 | fn poll_option(&self, _waiter: &kio::Waiter) -> Option<()> { + | ^^^^^^^^^^^^^ diff --git a/rs/kio/macros/tests/ui/wrapper_owned_receiver.rs b/rs/kio/macros/tests/ui/wrapper_owned_receiver.rs new file mode 100644 index 00000000000..4f5a313b289 --- /dev/null +++ b/rs/kio/macros/tests/ui/wrapper_owned_receiver.rs @@ -0,0 +1,13 @@ +// An owned receiver, which every poll would consume. +use std::task::Poll; + +struct S(kio::Queue); + +impl S { + #[kio::wrapper] + fn poll_pop(self, waiter: &kio::Waiter) -> Poll> { + self.0.poll_pop(waiter) + } +} + +fn main() {} diff --git a/rs/kio/macros/tests/ui/wrapper_owned_receiver.stderr b/rs/kio/macros/tests/ui/wrapper_owned_receiver.stderr new file mode 100644 index 00000000000..e42d963a545 --- /dev/null +++ b/rs/kio/macros/tests/ui/wrapper_owned_receiver.stderr @@ -0,0 +1,5 @@ +error: #[kio::wrapper] needs a borrowed receiver, `&self` or `&mut self` + --> tests/ui/wrapper_owned_receiver.rs:8:14 + | +8 | fn poll_pop(self, waiter: &kio::Waiter) -> Poll> { + | ^^^^ diff --git a/rs/kio/macros/tests/ui/wrapper_pattern.rs b/rs/kio/macros/tests/ui/wrapper_pattern.rs new file mode 100644 index 00000000000..ef9662dd3e2 --- /dev/null +++ b/rs/kio/macros/tests/ui/wrapper_pattern.rs @@ -0,0 +1,13 @@ +// An argument destructured in place: the wrapper has no name to pass it by. +use std::task::Poll; + +struct S; + +impl S { + #[kio::wrapper] + fn poll_pair(&self, _waiter: &kio::Waiter, (a, b): (u8, u8)) -> Poll { + Poll::Ready(a + b) + } +} + +fn main() {} diff --git a/rs/kio/macros/tests/ui/wrapper_pattern.stderr b/rs/kio/macros/tests/ui/wrapper_pattern.stderr new file mode 100644 index 00000000000..0f1d5497743 --- /dev/null +++ b/rs/kio/macros/tests/ui/wrapper_pattern.stderr @@ -0,0 +1,5 @@ +error: #[kio::wrapper] needs each argument bound to a plain name + --> tests/ui/wrapper_pattern.rs:8:45 + | +8 | fn poll_pair(&self, _waiter: &kio::Waiter, (a, b): (u8, u8)) -> Poll { + | ^^^^^^ diff --git a/rs/kio/macros/tests/ui/wrapper_qualifiers.rs b/rs/kio/macros/tests/ui/wrapper_qualifiers.rs new file mode 100644 index 00000000000..ecfb4e0d8dd --- /dev/null +++ b/rs/kio/macros/tests/ui/wrapper_qualifiers.rs @@ -0,0 +1,28 @@ +// Qualifiers an `async fn` wrapper cannot carry, and an argument to the attribute. +use std::task::Poll; + +struct S; + +impl S { + #[kio::wrapper] + async fn poll_async(&self, _waiter: &kio::Waiter) -> Poll<()> { + Poll::Ready(()) + } + + #[kio::wrapper] + const fn poll_const(&self, _waiter: &kio::Waiter) -> Poll<()> { + Poll::Ready(()) + } + + #[kio::wrapper] + unsafe fn poll_unsafe(&self, _waiter: &kio::Waiter) -> Poll<()> { + Poll::Ready(()) + } + + #[kio::wrapper(name = ready)] + fn poll_named(&self, _waiter: &kio::Waiter) -> Poll<()> { + Poll::Ready(()) + } +} + +fn main() {} diff --git a/rs/kio/macros/tests/ui/wrapper_qualifiers.stderr b/rs/kio/macros/tests/ui/wrapper_qualifiers.stderr new file mode 100644 index 00000000000..83ca87975df --- /dev/null +++ b/rs/kio/macros/tests/ui/wrapper_qualifiers.stderr @@ -0,0 +1,23 @@ +error: #[kio::wrapper] goes on a poll method, not an `async fn` + --> tests/ui/wrapper_qualifiers.rs:8:2 + | +8 | async fn poll_async(&self, _waiter: &kio::Waiter) -> Poll<()> { + | ^^^^^ + +error: #[kio::wrapper] cannot wrap a `const fn` + --> tests/ui/wrapper_qualifiers.rs:13:2 + | +13 | const fn poll_const(&self, _waiter: &kio::Waiter) -> Poll<()> { + | ^^^^^ + +error: #[kio::wrapper] cannot wrap an `unsafe fn` + --> tests/ui/wrapper_qualifiers.rs:18:2 + | +18 | unsafe fn poll_unsafe(&self, _waiter: &kio::Waiter) -> Poll<()> { + | ^^^^^^ + +error: #[kio::wrapper] takes no arguments + --> tests/ui/wrapper_qualifiers.rs:22:17 + | +22 | #[kio::wrapper(name = ready)] + | ^^^^^^^^^^^^ diff --git a/rs/kio/macros/tests/ui/wrapper_waiter.rs b/rs/kio/macros/tests/ui/wrapper_waiter.rs new file mode 100644 index 00000000000..7a2fe365a4c --- /dev/null +++ b/rs/kio/macros/tests/ui/wrapper_waiter.rs @@ -0,0 +1,23 @@ +// No waiter, two, and a mutable one. +use std::task::Poll; + +struct S; + +impl S { + #[kio::wrapper] + fn poll_none(&self) -> Poll<()> { + Poll::Ready(()) + } + + #[kio::wrapper] + fn poll_two(&self, _a: &kio::Waiter, _b: &kio::Waiter) -> Poll<()> { + Poll::Ready(()) + } + + #[kio::wrapper] + fn poll_mutable(&self, _waiter: &mut kio::Waiter) -> Poll<()> { + Poll::Ready(()) + } +} + +fn main() {} diff --git a/rs/kio/macros/tests/ui/wrapper_waiter.stderr b/rs/kio/macros/tests/ui/wrapper_waiter.stderr new file mode 100644 index 00000000000..4eb1a45ac5d --- /dev/null +++ b/rs/kio/macros/tests/ui/wrapper_waiter.stderr @@ -0,0 +1,23 @@ +error: #[kio::wrapper] needs a `&Waiter` argument + --> tests/ui/wrapper_waiter.rs:8:14 + | +8 | fn poll_none(&self) -> Poll<()> { + | ^^^^^^^ + +error: #[kio::wrapper] needs exactly one `&Waiter` argument + --> tests/ui/wrapper_waiter.rs:13:43 + | +13 | fn poll_two(&self, _a: &kio::Waiter, _b: &kio::Waiter) -> Poll<()> { + | ^^^^^^^^^^^^ + +error: the first `&Waiter` is here + --> tests/ui/wrapper_waiter.rs:13:25 + | +13 | fn poll_two(&self, _a: &kio::Waiter, _b: &kio::Waiter) -> Poll<()> { + | ^^^^^^^^^^^^ + +error: #[kio::wrapper] needs the waiter as `&Waiter`, not `&mut Waiter` + --> tests/ui/wrapper_waiter.rs:18:34 + | +18 | fn poll_mutable(&self, _waiter: &mut kio::Waiter) -> Poll<()> { + | ^^^^^^^^^^^^^^^^ diff --git a/rs/kio/macros/tests/wrapper.rs b/rs/kio/macros/tests/wrapper.rs new file mode 100644 index 00000000000..f62cbd89da6 --- /dev/null +++ b/rs/kio/macros/tests/wrapper.rs @@ -0,0 +1,191 @@ +//! What `#[kio::wrapper]` generates for each poll shape it accepts. + +use std::{ + pin::pin, + task::{Context, Poll, Waker}, +}; + +/// One poll of `fut` on a waker that does nothing. +fn poll_once(fut: F) -> Poll { + pin!(fut).poll(&mut Context::from_waker(Waker::noop())) +} + +struct Values { + queue: kio::Queue, + taken: Vec, +} + +impl Values { + fn new(values: &[u64]) -> Self { + let queue = kio::Queue::new(); + for value in values { + queue.try_push(*value).unwrap(); + } + Self { + queue, + taken: Vec::new(), + } + } + + /// The next value. + #[kio::wrapper] + fn poll_next(&self, waiter: &kio::Waiter) -> Poll> { + self.queue.poll_pop(waiter) + } + + /// The next value at or above `min`, keeping the smaller ones it skips. + #[kio::wrapper] + pub fn poll_at_least(&mut self, w: &kio::Waiter, mut min: u64) -> Poll> { + min = min.max(1); + loop { + match self.queue.poll_pop(w) { + Poll::Ready(Ok(value)) if value < min => self.taken.push(value), + other => return other, + } + } + } + + /// Moves the next value into `into`, with the waiter last. + #[kio::wrapper] + fn poll_move_into(&self, into: &mut Vec, waiter: &kio::Waiter) -> Poll<()> { + if let Poll::Ready(Ok(value)) = self.queue.poll_pop(waiter) { + into.push(value); + } + Poll::Ready(()) + } + + /// The next value, converted: `T` appears only in the return type. + #[kio::wrapper] + fn poll_convert>(&self, waiter: &kio::Waiter) -> std::task::Poll> { + self.queue.poll_pop(waiter).map(|value| value.ok().map(T::from)) + } + + /// Kept for compatibility. + #[deprecated = "use `poll_next`"] + #[kio::wrapper] + fn poll_old(&self, waiter: &kio::Waiter) -> Poll> { + self.queue.poll_pop(waiter) + } + + /// The next value, under a keyword's name. + #[kio::wrapper] + fn poll_match(&self, waiter: &kio::Waiter) -> Poll> { + self.queue.poll_pop(waiter) + } + + /// Only compiled where the poll is. + #[cfg(test)] + #[kio::wrapper] + fn poll_conditional(&self, waiter: &kio::Waiter) -> Poll> { + self.queue.poll_pop(waiter) + } + + /// Never compiled: a wrapper without the poll's `cfg` would call a missing method. + #[cfg(not(test))] + #[kio::wrapper] + fn poll_missing(&self, waiter: &kio::Waiter) -> Poll> { + self.queue.poll_pop(waiter) + } +} + +#[test] +fn awaits_the_poll() { + let values = Values::new(&[1]); + assert_eq!(poll_once(values.next()), Poll::Ready(Ok(1))); + assert_eq!(poll_once(values.next()), Poll::Pending); + values.queue.close(); + assert_eq!(poll_once(values.next()), Poll::Ready(Err(kio::Closed))); +} + +#[test] +fn passes_arguments_to_every_poll() { + let mut values = Values::new(&[1, 2]); + let mut at_least = Box::pin(values.at_least(3)); + assert_eq!(poll_once(at_least.as_mut()), Poll::Pending); + drop(at_least); + assert_eq!(values.taken, [1, 2]); + + values.queue.try_push(4).unwrap(); + assert_eq!(poll_once(values.at_least(3)), Poll::Ready(Ok(4))); + + let mut into = Vec::new(); + values.queue.try_push(5).unwrap(); + assert_eq!(poll_once(values.move_into(&mut into)), Poll::Ready(())); + assert_eq!(into, [5]); +} + +#[test] +fn names_the_generic_parameters() { + let values = Values::new(&[7]); + assert_eq!(poll_once(values.convert::()), Poll::Ready(Some(7u128))); +} + +#[test] +#[allow(deprecated, reason = "awaits the deprecated wrapper")] +fn keeps_attributes() { + let values = Values::new(&[1, 2]); + assert_eq!(poll_once(values.old()), Poll::Ready(Ok(1))); + assert_eq!(poll_once(values.conditional()), Poll::Ready(Ok(2))); +} + +#[test] +fn escapes_a_keyword_name() { + let values = Values::new(&[3]); + assert_eq!(poll_once(values.r#match()), Poll::Ready(Ok(3))); +} + +/// A wrapper on a generic impl, beside a machine that waits on it. +struct Pair { + queue: kio::Queue, +} + +impl Pair { + /// The next item. + #[kio::wrapper] + fn poll_item(&self, waiter: &kio::Waiter) -> Poll> { + self.queue.poll_pop(waiter) + } +} + +struct Sum { + pair: Pair, + state: SumState, +} + +#[kio::machine(self.state: SumState)] +impl Sum { + /// Two items, added. + #[kio::wrapper] + fn poll_sum(&mut self, waiter: &kio::Waiter) -> Poll> { + let first: u64 = wait!(First => self.pair.poll_item(waiter))?; + let second = wait!(Second => self.pair.poll_item(waiter))?; + Poll::Ready(Ok(first + second)) + } +} + +#[test] +fn composes_with_a_machine() { + let mut sum = Sum { + pair: Pair { + queue: kio::Queue::new(), + }, + state: SumState::Start, + }; + sum.pair.queue.try_push(1).unwrap(); + assert_eq!(poll_once(sum.sum()), Poll::Pending); + assert_eq!(sum.state.name(), "Second"); + sum.pair.queue.try_push(2).unwrap(); + assert_eq!(poll_once(sum.sum()), Poll::Ready(Ok(3))); +} + +#[test] +fn is_send_when_its_borrows_are() { + fn send(fut: F) -> F { + fut + } + let pair = Pair { + queue: kio::Queue::new(), + }; + pair.queue.try_push(1u64).unwrap(); + assert_eq!(poll_once(send(pair.item())), Poll::Ready(Ok(1))); +} diff --git a/rs/kio/src/clock/mod.rs b/rs/kio/src/clock/mod.rs index 10f50046cc6..e799716a45f 100644 --- a/rs/kio/src/clock/mod.rs +++ b/rs/kio/src/clock/mod.rs @@ -8,8 +8,8 @@ //! [`Deadline`]s on a [`Deadlines`] queue, and tests step them with a //! [`Manual`] clock. //! - Async code reads the ambient clock: [`now`], [`wall`], [`sleep`], -//! [`sleep_until`], [`timeout`], [`interval`], and [`drive`] resolve to the -//! current runtime's clock. That is tokio's (paused tokio included), the +//! [`sleep_until`], [`timeout`], [`interval`], and `drive` (with the `async` +//! feature) resolve to the current runtime's clock. That is tokio's (paused tokio included), the //! simulator inside `sim::run`, and a [`Deadlines`] queue on a thread that //! [entered](Deadlines::enter) it, which is how a runtime that reads the host //! clock itself (an io_uring worker) becomes a backend. @@ -46,8 +46,10 @@ pub use deadline::{Deadline, Deadlines, Entered}; pub use installed::Installed; pub use instant::Instant; pub use manual::Manual; +#[cfg(feature = "async")] +pub use timer::drive; pub use timer::{ - Elapsed, Interval, MissedTickBehavior, Sleep, Timeout, drive, interval, interval_at, sleep, sleep_until, timeout, + Elapsed, Interval, MissedTickBehavior, Sleep, Timeout, interval, interval_at, sleep, sleep_until, timeout, timeout_at, }; pub use wall::{OutOfRange, Wall, WallMap}; diff --git a/rs/kio/src/clock/timer.rs b/rs/kio/src/clock/timer.rs index 8cadf58ef66..c296390792f 100644 --- a/rs/kio/src/clock/timer.rs +++ b/rs/kio/src/clock/timer.rs @@ -229,6 +229,7 @@ pub fn interval_at(start: Instant, period: Duration) -> Interval { impl Interval { /// Wait for the next tick, returning the instant it was scheduled for. + #[cfg(feature = "async")] pub async fn tick(&mut self) -> Instant { std::future::poll_fn(|cx| self.poll_tick(cx)).await } @@ -275,6 +276,7 @@ impl Interval { /// /// `poll` is called with the current instant and returns when it next wants to be /// polled: `Ok(Some(at))` by `at` or on a wake, `Ok(None)` only on a wake. +#[cfg(feature = "async")] pub async fn drive(mut poll: impl FnMut(Instant, &crate::Waiter) -> Result, E> + Unpin) -> E { let mut timer: Option = None; crate::wait(move |waiter| { diff --git a/rs/kio/src/consumer.rs b/rs/kio/src/consumer.rs index 8d47ad38bec..b55b0be7ee2 100644 --- a/rs/kio/src/consumer.rs +++ b/rs/kio/src/consumer.rs @@ -3,14 +3,14 @@ use std::{ task::Poll, }; -use crate::{Closed, Counts, State, lock::*, producer::Ref, waiter::*, weak::ConsumerWeak}; +use crate::{Counts, State, lock::*, producer::Ref, waiter::*, weak::ConsumerWeak}; /// The consuming side of a shared state channel. /// /// Consumers have read-only access to the shared value and are notified when /// a producer modifies it. Cloning a consumer increments the consumer reference /// count. When the last consumer is dropped, the consumer-count waiters -/// (e.g. [`Producer::wanted_changed`](crate::Producer::wanted_changed)) are notified. +/// (e.g. [`Producer::poll_wanted_changed`](crate::Producer::poll_wanted_changed)) are notified. #[derive(Debug)] pub struct Consumer { pub(crate) state: Lock>, @@ -98,7 +98,8 @@ impl Consumer { Poll::Pending } - /// Poll for channel closure, registering the waiter if still open. + /// Resolves once the channel is closed. + #[cfg_attr(feature = "async", kio_macros::wrapper)] #[cfg_attr(feature = "sim", track_caller)] pub fn poll_closed(&self, waiter: &Waiter) -> Poll<()> { let mut state = self.state.lock(); @@ -112,22 +113,18 @@ impl Consumer { /// Wait for the closure to return [`Poll::Ready`], re-polling on each state change. /// - /// Returns `Ok(R)` when the closure returns [`Poll::Ready`], or [`Closed`] if the + /// Returns `Ok(R)` when the closure returns [`Poll::Ready`], or [`Closed`](crate::Closed) if the /// channel closes first. Unlike [`poll`](Self::poll) this hands back no [`Ref`], so /// no lock guard can be held across a later `.await`; call [`read`](Self::read) if /// you need the final state. - pub async fn wait(&self, mut f: F) -> Result + #[cfg(feature = "async")] + pub async fn wait(&self, mut f: F) -> Result where F: FnMut(&Ref<'_, T>) -> Poll + Unpin, { // The `Ref` is dropped here inside the closure, releasing the lock before the // caller ever sees the result. - crate::wait(move |waiter| self.poll(waiter, &mut f).map(|res| res.map_err(|_| Closed))).await - } - - /// Wait until the channel is closed. - pub async fn closed(&self) { - crate::wait(move |waiter| self.poll_closed(waiter)).await + crate::wait(move |waiter| self.poll(waiter, &mut f).map(|res| res.map_err(|_| crate::Closed))).await } /// Get read-only access to the shared state. diff --git a/rs/kio/src/lib.rs b/rs/kio/src/lib.rs index 734fac1f482..40cc551e23e 100644 --- a/rs/kio/src/lib.rs +++ b/rs/kio/src/lib.rs @@ -24,8 +24,12 @@ //! [`clock`] is the time those waits run on: logical instants, wall time, //! deadlines, timers, and a deterministic simulator for tests. //! +//! Every wait is a `poll_*` method first. The `async` feature adds the `async fn` that +//! awaits each one, so a build without it compiles no coroutines (`async` bodies). +//! //! With the `macros` feature, `#[kio::machine]` writes a `poll_*` state machine from -//! straight-line code with explicit waits. +//! straight-line code with explicit waits, and `#[kio::wrapper]` generates the `async +//! fn` that awaits a `poll_*` method. #![warn(missing_docs)] @@ -55,7 +59,7 @@ mod weak; mod tests; #[cfg(feature = "macros")] -pub use kio_macros::machine; +pub use kio_macros::{machine, wrapper}; pub use bridge::Bridge; pub use consumer::Consumer; @@ -72,8 +76,9 @@ pub use weak::{ConsumerWeak, ProducerWeak, Weak}; #[cfg(feature = "sim")] pub use kio_macros::test; -// So `#[kio::test]` resolves inside this crate's own tests. -#[cfg(test)] +// So `#[kio::test]` resolves inside this crate's own tests, and the wrappers +// `#[kio::wrapper]` generates here reach `::kio::wait`. +#[cfg(any(test, feature = "async"))] extern crate self as kio; /// Bytes an `Arc` allocation occupies: `T` behind two reference counts, padded to diff --git a/rs/kio/src/producer.rs b/rs/kio/src/producer.rs index 8784fd153c1..d376d33b0d3 100644 --- a/rs/kio/src/producer.rs +++ b/rs/kio/src/producer.rs @@ -176,6 +176,7 @@ impl Producer { /// [`Poll::Ready`], or [`Closed`] if the channel closes first. The `Ok` guard is the /// write access you asked for, so it's yours to hold; the `Err` case hands back no /// guard at all. Call [`read`](Self::read) if you need the final state. + #[cfg(feature = "async")] pub async fn wait(&self, mut f: F) -> Result, Closed> where F: FnMut(&Ref<'_, T>) -> Poll<()> + Unpin, @@ -185,13 +186,8 @@ impl Producer { crate::wait(move |waiter| self.poll(waiter, &mut f).map(|res| res.map_err(|_| Closed))).await } - /// Wait until the channel is closed. - pub async fn closed(&self) { - crate::wait(move |waiter| self.poll_closed(waiter)).await - } - - /// Poll for channel closure (an explicit [`Mut::close`], e.g. an abort), - /// registering the waiter if still open. + /// Resolves once the channel is closed (an explicit [`Mut::close`], e.g. an abort). + #[cfg_attr(feature = "async", kio_macros::wrapper)] #[cfg_attr(feature = "sim", track_caller)] pub fn poll_closed(&self, waiter: &Waiter) -> Poll<()> { let mut state = self.state.lock(); @@ -211,14 +207,9 @@ impl Producer { self.counts.consumers.load(Ordering::Relaxed) > 0 } - /// Wait until [`Self::wanted`] differs from `last`, returning the new value, or - /// [`Closed`] if the channel closes first. - pub async fn wanted_changed(&self, last: bool) -> Result { - crate::wait(move |waiter| self.poll_wanted_changed(waiter, last)).await - } - - /// Poll for [`Self::wanted`] to differ from `last`: `Ready(Ok)` with the new value once - /// it does, [`Closed`] if the channel closed first, else `Pending`. + /// Resolves to the new [`Self::wanted`] once it differs from `last`, or [`Closed`] if + /// the channel closes first. + #[cfg_attr(feature = "async", kio_macros::wrapper)] #[cfg_attr(feature = "sim", track_caller)] pub fn poll_wanted_changed(&self, waiter: &Waiter, last: bool) -> Poll> { // Closure is checked first, so a closed channel reports `Closed` from every handle diff --git a/rs/kio/src/queue.rs b/rs/kio/src/queue.rs index d750affbddb..e84138b797d 100644 --- a/rs/kio/src/queue.rs +++ b/rs/kio/src/queue.rs @@ -68,8 +68,8 @@ impl State { /// Role-less like [`Shared`](crate::Shared): every clone can push, pop, or close, /// and there is no liveness of its own: closure is explicit via /// [`close`](Self::close), never implied by dropping handles. Bounded queues park -/// pushes at capacity ([`poll_push_with`](Self::poll_push_with) / -/// [`push`](Self::push)) or reject them ([`try_push`](Self::try_push)); pops park +/// pushes at capacity ([`poll_push_with`](Self::poll_push_with), or `push` with the +/// `async` feature) or reject them ([`try_push`](Self::try_push)); pops park /// while empty. After close, pops drain what's queued before reporting [`Closed`]. /// /// An event wakes *every* waiter parked on the affected side (there is no @@ -153,6 +153,7 @@ impl Queue { /// /// Returns [`Closed`] if the queue closes first (or already was); the item is /// dropped in that case; use [`try_push`](Self::try_push) to get it back. + #[cfg(feature = "async")] pub async fn push(&self, item: T) -> Result<(), Closed> { let mut item = Some(item); // Capture the slot by `&mut` so the closure is `Unpin` regardless of `T`. @@ -177,10 +178,11 @@ impl Queue { Ok(Some(item)) } - /// Poll for the next item. + /// Pop, waiting for an item while the queue is empty. /// /// Queued items drain before closure is reported, so [`Closed`] means empty - /// *and* closed. Registers `waiter` while empty; a push or close re-polls it. + /// *and* closed. A push or close wakes the waiter. + #[cfg_attr(feature = "async", kio_macros::wrapper)] #[cfg_attr(feature = "sim", track_caller)] pub fn poll_pop(&self, waiter: &Waiter) -> Poll> { let (item, mut waiters) = { @@ -198,13 +200,6 @@ impl Queue { crate::coop::ready(Ok(item)) } - /// Pop, waiting for an item while the queue is empty. - /// - /// Returns [`Closed`] once the queue is both closed and drained. - pub async fn pop(&self) -> Result { - crate::wait(move |waiter| self.poll_pop(waiter)).await - } - /// Close the queue: pushes fail from here on, pops drain what's queued and then /// report [`Closed`]. Idempotent. pub fn close(&self) { diff --git a/rs/kio/src/shared.rs b/rs/kio/src/shared.rs index dfb850a4e0b..9728ff48019 100644 --- a/rs/kio/src/shared.rs +++ b/rs/kio/src/shared.rs @@ -87,6 +87,7 @@ impl Shared { /// /// The returned [`Mut`] holds the lock, so do your mutation and drop it rather than /// carrying it across another `.await`, which would block every other handle. + #[cfg(feature = "async")] pub async fn wait(&self, mut f: F) -> Mut<'_, T> where F: FnMut(&Ref<'_, T>) -> Poll<()> + Unpin, diff --git a/rs/kio/src/weak.rs b/rs/kio/src/weak.rs index 097db7bc0f3..deab9d53500 100644 --- a/rs/kio/src/weak.rs +++ b/rs/kio/src/weak.rs @@ -150,12 +150,8 @@ impl ProducerWeak { self.state.lock().closed } - /// Wait until the channel is closed. - pub async fn closed(&self) { - crate::wait(move |waiter| self.poll_closed(waiter)).await - } - - /// Poll for channel closure, registering the waiter if still open. + /// Resolves once the channel is closed. + #[cfg_attr(feature = "async", kio_macros::wrapper)] #[cfg_attr(feature = "sim", track_caller)] pub fn poll_closed(&self, waiter: &Waiter) -> Poll<()> { let mut state = self.state.lock(); @@ -242,7 +238,7 @@ impl ConsumerWeak { /// Poll the shared state with a closure, exactly as [`Consumer::poll`] does. /// /// Available here so a watcher can register for changes without joining the - /// consumer count, which is what [`Producer::wanted_changed`](crate::Producer::wanted_changed) + /// consumer count, which is what [`Producer::poll_wanted_changed`](crate::Producer::poll_wanted_changed) /// keys off. #[cfg_attr(feature = "sim", track_caller)] pub fn poll(&self, waiter: &Waiter, f: F) -> Poll>> @@ -271,12 +267,8 @@ impl ConsumerWeak { self.state.lock().closed } - /// Wait until the channel is closed. - pub async fn closed(&self) { - crate::wait(move |waiter| self.poll_closed(waiter)).await - } - - /// Poll for channel closure, registering the waiter if still open. + /// Resolves once the channel is closed. + #[cfg_attr(feature = "async", kio_macros::wrapper)] #[cfg_attr(feature = "sim", track_caller)] pub fn poll_closed(&self, waiter: &Waiter) -> Poll<()> { let mut state = self.state.lock(); diff --git a/rs/moq-net/Cargo.toml b/rs/moq-net/Cargo.toml index d0f3460b673..405dc55c83e 100644 --- a/rs/moq-net/Cargo.toml +++ b/rs/moq-net/Cargo.toml @@ -15,7 +15,7 @@ categories = ["multimedia", "network-programming", "web-programming"] [features] default = ["async"] # Runtime-backed helpers and Lite/IETF sessions; explicit-time model drivers stay available. -async = ["dep:futures", "kio/runtime"] +async = ["dep:futures", "kio/async", "kio/runtime"] # Exposes the wire codecs to the `fuzz/` harness and the announce, codec, path, varint, frame, and canonical storage benches, # and session views of an origin to the viewers bench, through the hidden `fuzz` module. Off by diff --git a/rs/moq-net/src/model/broadcast.rs b/rs/moq-net/src/model/broadcast.rs index 6150a5bff0d..b6a4a8e16b2 100644 --- a/rs/moq-net/src/model/broadcast.rs +++ b/rs/moq-net/src/model/broadcast.rs @@ -785,10 +785,11 @@ impl Requests { &self.info } - /// Poll for the next consumer-requested track, without blocking. + /// The next track a consumer requests, as a [`track::Request`] to serve. /// /// Returns [`Error::Closed`] once the broadcast has ended, so a serving loop /// knows to stop and release its handle. + #[cfg_attr(feature = "async", kio::wrapper)] pub fn poll_requested_track(&mut self, waiter: &kio::Waiter) -> Poll> { // A source that no longer names the instance that minted it takes no new track: // ending it refuses what is queued with `Unroutable`, while the tracks it handed @@ -817,12 +818,6 @@ impl Requests { Poll::Ready(Ok(pending.claim().with_stats(self.stats.clone()))) } - /// Block until a consumer requests a track, returning a [`track::Request`] to serve. - #[cfg(feature = "async")] - pub async fn requested_track(&mut self) -> Result { - kio::wait(|waiter| self.poll_requested_track(waiter)).await - } - /// Create a consumer that can subscribe to tracks in this broadcast. pub fn consume(&self) -> Consumer { Consumer { @@ -849,13 +844,8 @@ impl Requests { Lister::new(self.state.clone()) } - /// Block until the broadcast ends, by [`Producer::close`] or every producer dropping. - #[cfg(feature = "async")] - pub async fn closed(&self) { - kio::wait(|waiter| self.poll_closed(waiter)).await - } - - /// Poll until the producer closes. + /// Resolves once the broadcast ends, by [`Producer::close`] or every producer dropping. + #[cfg_attr(feature = "async", kio::wrapper)] pub fn poll_closed(&self, waiter: &kio::Waiter) -> Poll<()> { self.alive.token.poll_closed(waiter) } @@ -1053,12 +1043,6 @@ impl Consumer { BroadcastState::peek(&mut state, name).or_else(|| ended.filter(|_| state.closing).cloned()) } - /// Block until the broadcast ends, by [`Producer::close`] or every producer dropping. - #[cfg(feature = "async")] - pub async fn closed(&self) { - self.alive.closed().await - } - /// Returns true once the broadcast has ended. pub fn is_closed(&self) -> bool { self.alive.is_closed() @@ -1071,11 +1055,11 @@ impl Consumer { self.state.read().closing } - /// Register a [`kio::Waiter`] that fires when the broadcast closes. + /// Resolves once the broadcast ends, by [`Producer::close`] or every producer dropping. /// - /// Returns [`Poll::Ready`] if already closed, otherwise [`Poll::Pending`] after - /// arming the waiter. Useful for composing close-detection into a larger poll - /// without spawning a task per broadcast. + /// The poll composes close-detection into a larger one without spawning a task + /// per broadcast. + #[cfg_attr(feature = "async", kio::wrapper)] pub fn poll_closed(&self, waiter: &kio::Waiter) -> Poll<()> { self.alive.poll_closed(waiter) } @@ -1191,17 +1175,13 @@ impl Demand { !state.requests.is_empty() || state.tracks.iter().any(|track| track.is_wanted()) } - /// Return [`wanted`](Self::wanted) once it differs from what this handle last returned: - /// immediately on the first call, then on each change. Errors once the broadcast closes. + /// Resolves to [`wanted`](Self::wanted) once it differs from what this handle last + /// returned: immediately on the first call, then on each change. Errors once the + /// broadcast closes. /// /// So `while !demand.wanted_changed().await? {}` waits for a reader, and /// `while demand.wanted_changed().await? {}` waits for the last one to leave. - #[cfg(feature = "async")] - pub async fn wanted_changed(&mut self) -> Result { - kio::wait(|waiter| self.poll_wanted_changed(waiter)).await - } - - /// Poll for [`wanted`](Self::wanted) to differ from what this handle last returned. + #[cfg_attr(feature = "async", kio::wrapper)] pub fn poll_wanted_changed(&mut self, waiter: &kio::Waiter) -> Poll> { if self.alive.poll_closed(waiter).is_ready() { return Poll::Ready(Err(Error::Dropped)); diff --git a/rs/moq-shaper/Cargo.toml b/rs/moq-shaper/Cargo.toml index 6194b81cc0a..1cbd2e29855 100644 --- a/rs/moq-shaper/Cargo.toml +++ b/rs/moq-shaper/Cargo.toml @@ -25,7 +25,7 @@ anyhow = { workspace = true } clap = { workspace = true } humantime = { workspace = true } humantime-serde = { workspace = true } -kio = { workspace = true, features = ["runtime"] } +kio = { workspace = true, features = ["async", "runtime"] } rand = { workspace = true } serde = { workspace = true } serde_json = { workspace = true } diff --git a/sh/rs/coroutines.sh b/sh/rs/coroutines.sh new file mode 100755 index 00000000000..64f9e1f9609 --- /dev/null +++ b/sh/rs/coroutines.sh @@ -0,0 +1,69 @@ +#!/usr/bin/env bash +# Prove a poll-only library build compiles no coroutines: no `async fn` body, +# `async` block, or `async` closure in the crate's own MIR. A library's `async` +# feature carries its wrappers, so a build without it should have none. Each +# `--allow NAME` excuses the `async fn NAME` bodies still waiting on a poll +# conversion, and fails once none is left, so the list only shrinks. +# +# usage: coroutines.sh PACKAGE [--allow NAME]... [-- CARGO_ARGS...] +set -euo pipefail + +if [[ "${GITHUB_ACTIONS:-}" != true && "${LOCAL_SUPERVISED:-}" != 1 ]]; then + exec bash "$(dirname "$0")/../local.sh" bash "$0" "$@" +fi + +usage='usage: coroutines.sh PACKAGE [--allow NAME]... [-- CARGO_ARGS...]' +if [[ $# == 0 ]]; then + echo "$usage" >&2 + exit 2 +fi +package=$1 +shift +allowed=() +while [[ $# != 0 ]]; do + case "$1" in + --allow) + [[ $# -ge 2 && "$2" =~ ^[A-Za-z_][A-Za-z0-9_]*$ ]] || { + echo "$usage" >&2 + exit 2 + } + allowed+=("$2") + shift 2 + ;; + --) + shift + break + ;; + *) + echo "$usage" >&2 + exit 2 + ;; + esac +done + +cd "$(git rev-parse --show-toplevel)" +mir=$(mktemp) +trap 'rm -f "$mir"' EXIT + +# Only the final crate gets `--emit`, so its dependencies build as usual. +cargo rustc --locked -p "$package" --lib "$@" -- --emit "mir=$mir" + +# rustc names each coroutine by its body, as `{async fn body of queue::Queue::pop()}`, +# sometimes without the module path, so an allowed name matches as the last segment. +found=$(grep -oE '\{async (fn body of [^{}]*|block@[^{}]*|closure@[^{}]*)\}' "$mir" | sort -u || true) +unexpected=$found +for name in ${allowed[@]+"${allowed[@]}"}; do + pattern="^\{async fn body of ([A-Za-z0-9_<>, ]+::)*$name[<(]" + if ! grep -qE -- "$pattern" <<<"$found"; then + echo "error: $package no longer compiles async fn $name; remove its --allow" >&2 + exit 1 + fi + unexpected=$(grep -vE -- "$pattern" <<<"$unexpected" || true) +done +if [[ -n "$unexpected" ]]; then + echo "error: $package compiles coroutines without its async feature:" >&2 + echo "$unexpected" >&2 + echo "Gate each behind the feature, or generate it from a poll with #[kio::wrapper]." >&2 + exit 1 +fi +echo "$package: no coroutines beyond ${#allowed[@]} allowed async fn names" From befa9f04bec538811384b3486e3f0275adfb24fc Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 1 Nov 2026 10:50:18 -0800 Subject: [PATCH 10/12] fix(sh): brace the allowed name in the coroutine scan's pattern Co-Authored-By: Claude Opus 5.5 --- sh/rs/coroutines.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sh/rs/coroutines.sh b/sh/rs/coroutines.sh index 64f9e1f9609..d8cbc994a32 100755 --- a/sh/rs/coroutines.sh +++ b/sh/rs/coroutines.sh @@ -53,7 +53,7 @@ cargo rustc --locked -p "$package" --lib "$@" -- --emit "mir=$mir" found=$(grep -oE '\{async (fn body of [^{}]*|block@[^{}]*|closure@[^{}]*)\}' "$mir" | sort -u || true) unexpected=$found for name in ${allowed[@]+"${allowed[@]}"}; do - pattern="^\{async fn body of ([A-Za-z0-9_<>, ]+::)*$name[<(]" + pattern="^\{async fn body of ([A-Za-z0-9_<>, ]+::)*${name}[<(]" if ! grep -qE -- "$pattern" <<<"$found"; then echo "error: $package no longer compiles async fn $name; remove its --allow" >&2 exit 1 From 4b0e8774c5bdd315b961916211feeccd41bdcc37 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 1 Nov 2026 11:46:52 -0800 Subject: [PATCH 11/12] fix(kio-macros): reject a wrapper name that cannot be an identifier `poll_self`, `poll__`, and `poll_1` reached `Ident::new_raw`, which panics for the keywords that cannot be raw and for a digit-led name. Report a spanned error on the poll instead. Co-Authored-By: Claude Opus 5.5 --- rs/kio/macros/src/wrapper.rs | 19 +++++++++++---- rs/kio/macros/tests/ui/wrapper_unnameable.rs | 23 +++++++++++++++++++ .../macros/tests/ui/wrapper_unnameable.stderr | 17 ++++++++++++++ 3 files changed, 54 insertions(+), 5 deletions(-) create mode 100644 rs/kio/macros/tests/ui/wrapper_unnameable.rs create mode 100644 rs/kio/macros/tests/ui/wrapper_unnameable.stderr diff --git a/rs/kio/macros/src/wrapper.rs b/rs/kio/macros/src/wrapper.rs index b848fe591a6..28d13400590 100644 --- a/rs/kio/macros/src/wrapper.rs +++ b/rs/kio/macros/src/wrapper.rs @@ -4,7 +4,7 @@ use proc_macro2::{Span, TokenStream}; use quote::quote; use syn::{ Attribute, FnArg, GenericArgument, GenericParam, Ident, ImplItemFn, Pat, PathArguments, ReturnType, Type, - punctuated::Punctuated, spanned::Spanned, + ext::IdentExt, punctuated::Punctuated, spanned::Spanned, }; pub(crate) fn expand(args: TokenStream, method: &ImplItemFn) -> syn::Result { @@ -51,10 +51,19 @@ pub(crate) fn expand(args: TokenStream, method: &ImplItemFn) -> syn::Result` method", )); }; - // `poll_match` waits as `r#match`. - let name = match syn::parse_str::(name) { - Ok(_) => Ident::new(name, poll.span()), - Err(_) => Ident::new_raw(name, poll.span()), + // `poll_match` waits as `r#match`. `poll_self`, `poll__`, and `poll_1` have no + // such name: those keywords cannot be raw, and a digit cannot start one. + let name = if syn::parse_str::(name).is_ok() { + Ident::new(name, poll.span()) + } else if syn::parse::Parser::parse_str(Ident::parse_any, name).is_ok() + && !matches!(name, "_" | "self" | "Self" | "super" | "crate") + { + Ident::new_raw(name, poll.span()) + } else { + return Err(syn::Error::new_spanned( + poll, + format!("#[kio::wrapper] cannot name the wrapper `{name}`, even as a raw identifier"), + )); }; let mut inputs = sig.inputs.iter(); diff --git a/rs/kio/macros/tests/ui/wrapper_unnameable.rs b/rs/kio/macros/tests/ui/wrapper_unnameable.rs new file mode 100644 index 00000000000..51861f94d70 --- /dev/null +++ b/rs/kio/macros/tests/ui/wrapper_unnameable.rs @@ -0,0 +1,23 @@ +// A poll whose wrapper name is no identifier, even raw. +use std::task::Poll; + +struct S; + +impl S { + #[kio::wrapper] + fn poll_self(&self, _waiter: &kio::Waiter) -> Poll<()> { + Poll::Ready(()) + } + + #[kio::wrapper] + fn poll__(&self, _waiter: &kio::Waiter) -> Poll<()> { + Poll::Ready(()) + } + + #[kio::wrapper] + fn poll_1(&self, _waiter: &kio::Waiter) -> Poll<()> { + Poll::Ready(()) + } +} + +fn main() {} diff --git a/rs/kio/macros/tests/ui/wrapper_unnameable.stderr b/rs/kio/macros/tests/ui/wrapper_unnameable.stderr new file mode 100644 index 00000000000..a54c02a2573 --- /dev/null +++ b/rs/kio/macros/tests/ui/wrapper_unnameable.stderr @@ -0,0 +1,17 @@ +error: #[kio::wrapper] cannot name the wrapper `self`, even as a raw identifier + --> tests/ui/wrapper_unnameable.rs:8:5 + | +8 | fn poll_self(&self, _waiter: &kio::Waiter) -> Poll<()> { + | ^^^^^^^^^ + +error: #[kio::wrapper] cannot name the wrapper `_`, even as a raw identifier + --> tests/ui/wrapper_unnameable.rs:13:5 + | +13 | fn poll__(&self, _waiter: &kio::Waiter) -> Poll<()> { + | ^^^^^^ + +error: #[kio::wrapper] cannot name the wrapper `1`, even as a raw identifier + --> tests/ui/wrapper_unnameable.rs:18:5 + | +18 | fn poll_1(&self, _waiter: &kio::Waiter) -> Poll<()> { + | ^^^^^^ From 1f79714e4f42ff065929bf5488471527a902d23c Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 1 Nov 2026 11:46:54 -0800 Subject: [PATCH 12/12] fix(auth): enable kio's async feature for Shared::wait moq-auth depends on kio directly and awaits `Shared::wait`, now behind `kio/async`. The workspace build unified the feature in through the `kio/sim` dev-dependencies, so only a lone library build broke. `just rs features` now checks that build. Co-Authored-By: Claude Opus 5.5 --- rs/justfile | 2 ++ rs/moq-auth/Cargo.toml | 3 ++- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/rs/justfile b/rs/justfile index 971e1a581e6..019382be7a0 100644 --- a/rs/justfile +++ b/rs/justfile @@ -227,6 +227,8 @@ features: just rs test -p moq-cli --no-default-features --features capture,pipewire,websocket -E 'test(capture_window)' bash sh/local.sh cargo check --locked -p moq-bench --no-default-features bash sh/local.sh cargo check --locked -p moq-boy --no-default-features + # Alone, since the workspace's `kio/sim` dev-dependencies would unify `kio/async` back in. + bash sh/local.sh cargo check --locked --manifest-path rs/moq-auth/Cargo.toml --lib --all-features bash sh/local.sh cargo check --locked -p moq-hls -p moq-rtmp -p moq-rtc --no-default-features --all-targets bash sh/local.sh cargo clippy --locked --workspace {{ no_fuzz }} --all-targets --all-features -- -D warnings RUSTDOCFLAGS="-D warnings" bash sh/local.sh cargo doc --locked --workspace {{ no_fuzz }} --all-features --no-deps diff --git a/rs/moq-auth/Cargo.toml b/rs/moq-auth/Cargo.toml index ccfe2f04a03..43eb004cf91 100644 --- a/rs/moq-auth/Cargo.toml +++ b/rs/moq-auth/Cargo.toml @@ -22,7 +22,8 @@ axum = { workspace = true, optional = true } base64 = { workspace = true } data-encoding = { workspace = true } jsonwebtoken = { workspace = true, features = ["aws_lc_rs"] } -kio = { workspace = true, features = ["serde"] } +# `async` for `Shared::wait`, until moq-auth grows an `async` feature of its own. +kio = { workspace = true, features = ["async", "serde"] } moq-pattern = { workspace = true, features = ["serde"] } p256 = "0.14.0" p384 = "0.14.0"