diff --git a/bench/ffi/README.md b/bench/ffi/README.md index 957020807..c00a4d3df 100644 --- a/bench/ffi/README.md +++ b/bench/ffi/README.md @@ -146,9 +146,14 @@ frame path: asyncio's, as BoltFFI's Python measured 10 us. This case starts a task per read; a reader looping on one task would skip the start, and is what to measure before changing a generator. -4. **A future per read.** A ready read costs C 155 ns over the write where Rust costs 73: - the future's three allocations and three crossings. Each language adds 160 (Swift) to 420 (Go) ns - more in its own wrapper, so the C ABI's share is not yet where a per-handle poll would pay. +4. **A future per read.** Since `_start` makes a future and polls it in one call, a + ready read costs C 126 ns over the write where Rust costs 93: one crossing and one + allocation, down from 176 with four allocations and three crossings. Measured 2026-11-13 + on an AMD Ryzen 9 9950X (32 threads) at a load average of 26, 12 rounds alternating with + the commit before, so the tables above predate it. Over their writes, Swift pays 289, + Go 414, Dart 522, Python 445, and Kotlin 590 (from 381, 556, 868, 659, and 615). Go's and + Dart's include the returned frame's ownership, about 190 and 215 ns over C's object return; + the rest is each language's own await. 5. **Kotlin's record codec.** A record return costs Kotlin 113 ns wall against C's 31, through the byte-array codec; Swift, which reads the C struct, costs 37. 6. **The handle borrow at the C ABI.** The u64 getter costs Rust 0.4 ns and C 10: the diff --git a/ffi/README.md b/ffi/README.md index 07563cb96..8126117a6 100644 --- a/ffi/README.md +++ b/ffi/README.md @@ -84,7 +84,7 @@ re-exports its items instead, which export. A type no path reaches is refused. | `Result` | `bool f(..., T *out, E *err)`; an `E` of another crate that implements `Display`, such as `anyhow::Error` or `io::Error`, is `String`, its message | | a returned tuple `(A, B)` | one out pointer each: `void f(..., A *out0, B *out1)` | | a tuple inside another type, or as a field or parameter | `_tuple__`: a struct of fields `_0`, `_1`, and so on; of references only returned | -| `async fn`, or a return that implements `Future` | a future: `f` makes it, `f_poll` polls it, `f_future_free` drops it; a function that is not `async` runs at the call, as in Rust | +| `async fn`, or a return that implements `Future` | a future: `f` makes it, `f_poll` polls it, `f_future_free` drops it, and `f_start` makes it and polls it once, returning a handle only if it pends; a function that is not `async` runs at the call, as in Rust | | a function returning `Poll` that polls through a `&Context`, a `&mut Context`, or a reference to an object exactly one associated function makes from a `Waker` alone, of a type with no generic parameters, such as kio's `Waiter`, which may be the method's own `&self`, as `Waiter::poll_future`'s is | a future of `T`, as an `async fn`'s: each poll calls the function again with the poll's context, or with an object made from its waker and kept until the next poll, so what it registers stays registered; each other parameter taken by value must be `Copy`; a method polling through its own `&self` is a function without one | | `poll_x`, when `x` exports as a future | left out: the future is its poll | | `bool`, integers but 128-bit, floats, `char` | `bool`, `` types, `float`, `double`, a `uint32_t` code point | @@ -257,7 +257,10 @@ where it was while its file is unchanged. A poll that finishes writes the result through the poll's own out pointers, so a cancelled read never loses a value. The future reuses the last waker when given the same one again, and owns what it borrowed: the handle it was called on stays borrowed until - it finishes or is freed. + it finishes or is freed. `_start` is the call and a first poll with no waker in one: + a future ready at once costs one call and no handle, and one that pends writes its + handle through the last parameter, which must not be null. A null waker wakes nothing + and costs no allocation, and a future is one allocation. - **Wakers** move into Rust, which calls `drop` once it lets go of the last clone, so a state machine polled with a waker, such as moq's `Driver`s through kio's `Waiter`, runs from the caller's event loop. Rust may call `wake` and `drop` from any thread, as a @@ -663,9 +666,9 @@ and, for `Swift.Duration`, macOS 13 or iOS 16, and builds in either language mod as its `io::Error`'s message. - **Futures** are `async throws` functions, which resume on the task's executor, never inside a wake. Cancelling the task frees the future, losing nothing, and throws - `CancellationError`. Each call is its own future, so it has one poller. The call polls it - once before suspending, and makes the signal it parks on only if that poll pends; the - signal latches a wake that lands before it parks. + `CancellationError`. Each call is its own future, so it has one poller. The call makes + and polls it in one crossing, `_start`, before suspending, and makes the signal it parks + on only if that poll pends; the signal latches a wake that lands before it parks. - A **future Swift implements** is an `@escaping @Sendable () async -> Void`, which Rust's first poll runs as a task of its own. Rust dropping it cancels the task, and one dropped before it starts never runs. A poll lent one, such as `Waiter.pollFuture`, is a static @@ -780,8 +783,8 @@ reads a go.mod. a reader that is also an `rt.StartSeeker`, as `io.Copy` uses an `io.WriterTo`, and fails in any other. Their errors cross as their messages. - **Futures** are calls taking a `context.Context` first, which block until the future is - ready. The call polls it once before it waits, and makes the waker it waits on only if - that poll pends; a wake from any thread signals the waiting goroutine, which polls + ready. The call makes and polls it in one crossing, `_start`, before it waits, and + makes the waker it waits on only if that poll pends; a wake from any thread signals the waiting goroutine, which polls again, so a wake never polls inside Rust, and one that lands before the wait is kept. A context done before the call, or while it waits, returns its error and frees the future, which cancels it in Rust and loses nothing: the next call takes what it would have. @@ -872,8 +875,8 @@ the library's own URI, mapped to the library by a build hook or by `--native-ass throws aborts. - **Futures** are `Cancelable`, a `Future` completed on the isolate that made it: an extension type on Dart's own future, which `await` takes its shortest path through. - The call polls it once, returning it complete if that poll finished, and makes a waker - only if it pends. A wake, from any + The call makes and polls it in one crossing, `_start`, returning it complete if that + poll finished, and makes a waker only if it pends. A wake, from any thread, posts to the isolate's port, and its event loop polls again, so a wake never polls inside Rust, and one that lands during a poll is kept. `cancel()` frees the future unless it finished, which cancels it in Rust and loses nothing: the next call diff --git a/ffi/fixtures/api/src/lib.rs b/ffi/fixtures/api/src/lib.rs index 5faf446d3..2f256c2f9 100644 --- a/ffi/fixtures/api/src/lib.rs +++ b/ffi/fixtures/api/src/lib.rs @@ -1015,6 +1015,18 @@ fn on_thread(log: &(impl Log + ?Sized), lines: &[u8]) -> u32 { }) } +/// A record named as Swift's own error a cancelled task throws, which the binding renames. +pub struct CancellationError { + pub code: u32, +} + +/// Each of `0..n` to `count` before it returns its future, then `n` doubled once polled, +/// or `Overflow`: an eager, fallible future that lends a closure. +pub fn counted(n: u32, mut count: impl FnMut(u32)) -> std::future::Ready> { + (0..n).for_each(&mut count); + std::future::ready(n.checked_mul(2).ok_or(Error::Overflow)) +} + /// Each line to `keep` on another thread, returning how many it kept: a lent closure may /// run on any thread until the call returns. pub fn filter(lines: Vec, mut keep: impl FnMut(u8, &str) -> bool + Send) -> u32 { diff --git a/ffi/moq/bench/export.c b/ffi/moq/bench/export.c index 01e679f46..268c54a95 100644 --- a/ffi/moq/bench/export.c +++ b/ffi/moq/bench/export.c @@ -65,14 +65,14 @@ static moq_group_Consumer *recv_group(moq_track_Subscriber *subscriber) { return reader; } +/* A read ready at once: made and polled in one call, with no handle. */ static moq_frame_Frame *read_frame(moq_group_Consumer *reader) { - moq_group_Consumer_read_frame_future *reading = moq_group_Consumer_read_frame(reader); + moq_group_Consumer_read_frame_future *reading = NULL; moq_frame_Frame *read = NULL; moq_Error *err = NULL; - if (moq_group_Consumer_read_frame_poll(reading, waker, &read, &err) != MOQ_POLL_READY || read == NULL) { + if (moq_group_Consumer_read_frame_start(reader, &read, &err, &reading) != MOQ_POLL_READY || read == NULL) { fail("read_frame"); } - moq_group_Consumer_read_frame_future_free(reading); return read; } diff --git a/ffi/runtime/ffi_cgo.go b/ffi/runtime/ffi_cgo.go index 96be64b94..4a209e718 100644 --- a/ffi/runtime/ffi_cgo.go +++ b/ffi/runtime/ffi_cgo.go @@ -362,19 +362,13 @@ func reader_(k *cgo.Handle) cForeignReader { return cForeignReader{read: goRead, seek: goSeek, data: unsafe.Pointer(k)} } -// await_ polls a future until poll reports it finished, returning its error, or ctx's -// once ctx is done: then the caller frees the future unfinished, which cancels it in Rust -// and loses nothing, and the next call sees what it would have taken. Its first poll makes -// no waker, which only a future that pends needs; a wake that lands between a poll that -// pends and the wait is kept, so the wait returns at once. +// await_ polls a future whose call pended until poll reports it finished, returning its +// error, or ctx's once ctx is done: then the caller frees the future unfinished, which +// cancels it in Rust and loses nothing, and the next call sees what it would have taken. +// The call polled it first with no waker, which only a future that pends needs, and +// checked ctx before its first poll; a wake that lands between a poll that pends and the +// wait is kept, so the wait returns at once. func await_(ctx context.Context, poll func(cWaker) (bool, error)) error { - // A context done already takes nothing, even what is ready. - if err := ctx.Err(); err != nil { - return err - } - if done, err := poll(cWaker{}); done { - return err - } woken := make(chan struct{}, 1) waker := waker_(func() { select { diff --git a/ffi/runtime/rt.dart b/ffi/runtime/rt.dart index af74e1d08..f4c9973a9 100644 --- a/ffi/runtime/rt.dart +++ b/ffi/runtime/rt.dart @@ -62,14 +62,9 @@ extension type Cancelable._(Future _future) implements Future { void cancel() => _n.calls[_future]?.cancel(); } -/// Poll `f` once with no waker, and if it pends, again with one: its future. -Cancelable _call( - ffi.Pointer f, - void Function(ffi.Pointer, ffi.Pointer) free, - bool Function(ffi.Pointer, _Waker, _Call) poll, -) { - final call = _Call._(f, free, poll); - call._step(_n.idle); +/// The future of `call`, which its start completed if its first poll finished, or else +/// left the future in Rust that pended, polled again here with a waker. +Cancelable _call(_Call call) { final first = call._first; if (first != null) return Cancelable._(first); final done = call._done = dasync.Completer(); @@ -83,7 +78,8 @@ Cancelable _call( /// A [Cancelable]'s future in Rust, until it finishes. final class _Call { - ffi.Pointer _f; + /// The future in Rust, once its start pended, until it finishes. + ffi.Pointer _f = ffi.nullptr; final void Function(ffi.Pointer, ffi.Pointer) _free; /// Poll once, completing this if it finished: whether it did. @@ -99,7 +95,7 @@ final class _Call { bool _polling = false; bool _cancelled = false; - _Call._(this._f, this._free, this._poll); + _Call._(this._free, this._poll); void _wake() { if (_f != ffi.nullptr) _step(_waker!); @@ -524,7 +520,6 @@ final class _Native { late final _Stack stack = _Stack(); late final isolate.RawReceivePort _port; late final ffi.Pointer home; - final _Waker idle = ffi.Struct.create<_Waker>(); /// What a token's wake calls, and its drop. final Map _wakes = {}; diff --git a/ffi/runtime/rt.rs b/ffi/runtime/rt.rs index 4c52657ad..e37c9834f 100644 --- a/ffi/runtime/rt.rs +++ b/ffi/runtime/rt.rs @@ -1035,7 +1035,11 @@ impl Waker { /// returned stays wrapped too, so its `data` names it for as long as a future may /// compare it. pub fn lift(self) -> std::task::Waker { - std::task::Waker::from(Arc::new(Foreign(self))) + match (self.wake, self.drop) { + // Nothing to wake or release: a ready future's first poll takes no allocation. + (None, None) => std::task::Waker::noop().clone(), + _ => std::task::Waker::from(Arc::new(Foreign(self))), + } } /// Wake it, as C does. @@ -1308,30 +1312,53 @@ impl std::io::Seek for Reader { } } -/// Writes a finished future's result through the poll's out pointers, one per value it -/// returns, and its error pointer, returning the poll. The poll that finishes the future -/// calls it, so it never crosses threads. -pub type Done = Box i32>; - pub const PENDING: i32 = 0; pub const READY: i32 = 1; pub const ERROR: i32 = 2; -/// A future behind a handle, polled from C. +/// A future behind a handle, polled from C: the head of the one allocation holding the +/// future and the function that lowers its result, which [`Future::new`] makes. +#[repr(C)] pub struct Future { - future: Option + Send>>>, + /// Poll the allocation, lowering the result through the poll's out pointers once ready. + poll: unsafe fn(*mut Future, Waker, &[*mut c_void], *mut c_void, &str) -> i32, + /// Drop the allocation. + drop: unsafe fn(*mut Future), /// The last foreign waker, and the Rust waker around it. waker: Option<(Waker, std::task::Waker)>, polling: AtomicBool, } +/// The allocation behind a handle: its head first, so the handle points at both. +#[repr(C)] +struct Typed { + head: Future, + /// The future, pinned here until it finishes and drops in place. + future: Option, + /// Writes the finished future's result through the poll's out pointers, one per value + /// it returns, and its error pointer, returning the poll. The poll that finishes the + /// future calls it, so it never crosses threads. + lower: L, +} + impl Future { - pub fn new(future: impl std::future::Future + Send + 'static) -> *mut Self { - Box::into_raw(Box::new(Self { - future: Some(Box::pin(future)), - waker: None, - polling: AtomicBool::new(false), - })) + pub fn new(future: F, lower: L) -> *mut Self + where + F: std::future::Future + Send + 'static, + // Send, as any poll may finish the future on another thread. + L: Fn(F::Output, &[*mut c_void], *mut c_void) -> i32 + Send + 'static, + { + let typed = Box::new(Typed { + head: Self { + poll: poll_typed::, + drop: drop_typed::, + waker: None, + polling: AtomicBool::new(false), + }, + future: Some(future), + lower, + }); + Box::into_raw(typed).cast() } /// Poll it once with `waker`, writing the result through `outs` and `err` when ready. @@ -1344,29 +1371,51 @@ impl Future { fail(what, "a future is null"); } // SAFETY: the caller's contract; only the flag is touched before it is held. - if unsafe { &*future }.polling.swap(true, Ordering::Acquire) { + let head = unsafe { &*future }; + if head.polling.swap(true, Ordering::Acquire) { fail(what, "a future is polled concurrently"); } - // SAFETY: the flag makes this the only access. - let this = unsafe { &mut *future }; - let Some(inner) = this.future.as_mut() else { - fail(what, "a future is polled after it finished"); - }; - if !this.waker.as_ref().is_some_and(|(last, _)| last.same(&waker)) { - this.waker = Some((waker, waker.lift())); + // SAFETY: the flag makes this the only access, and `new` made the allocation. + let result = unsafe { (head.poll)(future, waker, outs, err, what) }; + // SAFETY: as above; the allocation outlives the poll. + unsafe { &*future }.polling.store(false, Ordering::Release); + result + } + + /// Make the future's first poll, with no waker: on `READY` or `ERROR` the result is + /// written, the future freed, and null written through `handle`; on `PENDING` its + /// handle goes there, for C to poll with a waker and free. + /// + /// # Safety + /// `future` is a handle `new` just made, `handle` is null, which aborts, or points + /// where a handle goes, and each of `outs` and `err` is null or points where the function's result + /// types go. + pub unsafe fn start( + future: *mut Self, + outs: &[*mut c_void], + err: *mut c_void, + handle: *mut *mut Self, + what: &str, + ) -> i32 { + if handle.is_null() { + fail(what, "a future's handle pointer is null"); } - let (_, rust) = this.waker.as_ref().expect("set above"); - let mut cx = Context::from_waker(rust); - let poll = inner.as_mut().poll(&mut cx); - let result = match poll { - std::task::Poll::Pending => PENDING, - std::task::Poll::Ready(done) => { - this.future = None; - this.waker = None; - done(outs, err) - } + let none = Waker { + wake: None, + drop: None, + data: std::ptr::null_mut(), }; - this.polling.store(false, Ordering::Release); + // SAFETY: the caller's contract. + let result = unsafe { Self::poll(future, none, outs, err, what) }; + match result { + // SAFETY: the caller's contract. + PENDING => unsafe { handle.write(future) }, + // SAFETY: it finished; null tells C there is no handle, and `free` takes it. + _ => unsafe { + handle.write(std::ptr::null_mut()); + Self::free(future) + }, + } result } @@ -1377,11 +1426,53 @@ impl Future { pub unsafe fn free(future: *mut Self) { if !future.is_null() { // SAFETY: the caller's contract. - drop(unsafe { Box::from_raw(future) }); + unsafe { ((*future).drop)(future) }; } } } +/// [`Future`]'s poll of a `Typed`. +/// +/// # Safety +/// `future` is a live `Typed` that only this call accesses, and each of `outs` and +/// `err` is null or points where the function's result types go. +unsafe fn poll_typed i32>( + future: *mut Future, + waker: Waker, + outs: &[*mut c_void], + err: *mut c_void, + what: &str, +) -> i32 { + // SAFETY: the caller's contract. + let Typed { head, future, lower } = unsafe { &mut *future.cast::>() }; + let Some(inner) = future.as_mut() else { + fail(what, "a future is polled after it finished"); + }; + if !head.waker.as_ref().is_some_and(|(last, _)| last.same(&waker)) { + head.waker = Some((waker, waker.lift())); + } + let (_, rust) = head.waker.as_ref().expect("set above"); + let mut cx = Context::from_waker(rust); + // SAFETY: the allocation never moves, and the future drops in place. + match unsafe { Pin::new_unchecked(inner) }.poll(&mut cx) { + std::task::Poll::Pending => PENDING, + std::task::Poll::Ready(value) => { + *future = None; + head.waker = None; + lower(value, outs, err) + } + } +} + +/// [`Future`]'s drop of a `Typed`. +/// +/// # Safety +/// `future` is a live `Typed`, dropped once. +unsafe fn drop_typed(future: *mut Future) { + // SAFETY: the caller's contract; `new` boxed it. + drop(unsafe { Box::from_raw(future.cast::>()) }); +} + /// Write `value` through `out` if the caller wants it, or drop it. /// /// # Safety diff --git a/ffi/runtime/rt.swift b/ffi/runtime/rt.swift index 42d6fc4d1..ead8aa6e0 100644 --- a/ffi/runtime/rt.swift +++ b/ffi/runtime/rt.swift @@ -675,10 +675,6 @@ extension Native { return { owner.raw.wake?(owner.raw.data) } } - /// The waker of a future's first poll, which wakes nothing: a poll that pends polls - /// again with its signal's. - static var idle: CWaker { CWaker(wake: nil, drop: nil, data: nil) } - /// What a future parks on: it latches a wake that lands between a poll that pends and /// the park, so the next poll sees it. final class Signal: @unchecked Sendable { @@ -733,20 +729,39 @@ extension Native { } } - /// Await the future `raw`: poll it until `poll` returns its output, then free it. A - /// cancelled task frees it unfinished, which cancels it in Rust and loses nothing, and - /// throws `CancellationError`. Its first poll makes no signal, which only a future - /// that pends needs. + /// The call that starts a future and polls it once, whose output, if ready, has the + /// type `T` names. + static func start(_: T.Type, _ call: () throws -> T?) rethrows -> T? { + try call() + } + + /// Await the future `raw` an eager call made, unpolled, as `run` does, but take nothing, + /// even what is ready, if the task is cancelled already, and poll first with no waker. + static func made( + _ raw: OpaquePointer?, _ free: @convention(c) (OpaquePointer?) -> Void, + _ poll: (OpaquePointer?, CWaker) throws -> T? + ) async throws -> T { + do { + try Task.checkCancellation() + if let output = try poll(raw, CWaker(wake: nil, drop: nil, data: nil)) { + free(raw) + return output + } + } catch { + free(raw) + throw error + } + return try await run(raw, free, poll) + } + + /// Await the future `raw`, whose first poll pended: poll it until `poll` returns its + /// output, then free it. A cancelled task frees it unfinished, which cancels it in Rust + /// and loses nothing, and throws `CancellationError`. static func run( _ raw: OpaquePointer?, _ free: @convention(c) (OpaquePointer?) -> Void, _ poll: (OpaquePointer?, CWaker) throws -> T? ) async throws -> T { defer { free(raw) } - // A task cancelled already takes nothing, even what is ready. - try Task.checkCancellation() - if let output = try poll(raw, idle) { - return output - } let signal = Signal() var given = false // The handler's operation returns only a copyable result, so it leaves through this. diff --git a/ffi/src/abi.rs b/ffi/src/abi.rs index 63b22ced2..fbe58b76c 100644 --- a/ffi/src/abi.rs +++ b/ffi/src/abi.rs @@ -112,8 +112,19 @@ pub struct Export { /// The C name its path gives it, which another language names it by, as a clash in C /// renames only C's. pub named: String, - /// For a future: its handle type, its poll, and its free. - pub future: Option<(String, Sig, Sig)>, + /// For a future: its handle type and the functions over it. + pub future: Option, +} + +/// The C functions of a future besides the call that makes it. +pub struct Future { + /// The handle's type. + pub handle: String, + /// The call and its first poll in one, which writes a ready result as the poll does and + /// hands back the handle only if the future pends. + pub start: Sig, + pub poll: Sig, + pub free: Sig, } /// `ty` as a clone of each borrow it holds. @@ -781,7 +792,15 @@ impl<'a> Abi<'a> { ("future".to_string(), Ty::Future(handle.clone())), ("waker".to_string(), Ty::Waker), ]; - poll_params.extend(outs); + poll_params.extend(outs.iter().cloned()); + let mut start_params = params.clone(); + start_params.extend(outs); + start_params.push(("future".to_string(), Ty::Out(Box::new(Ty::Future(handle.clone()))))); + let start = Sig { + name: format!("{name}_start"), + params: start_params, + ret: Ty::Poll, + }; let poll = Sig { name: format!("{name}_poll"), params: poll_params, @@ -798,7 +817,12 @@ impl<'a> Abi<'a> { params, ret: Ty::Future(handle.clone()), }, - future: Some((handle, poll, free)), + future: Some(Future { + handle, + start, + poll, + free, + }), named, }; } @@ -1066,8 +1090,14 @@ impl<'a> Abi<'a> { ); let own = self.own(&item, &name, segments); self.claim(name.clone(), what.clone(), function.span, own); - if let Some((handle, poll, free)) = &export.future { - for made in [handle, &poll.name, &free.name] { + if let Some(Future { + handle, + start, + poll, + free, + }) = &export.future + { + for made in [handle, &start.name, &poll.name, &free.name] { self.claim(made.clone(), what.clone(), function.span, Self::made(&item, &name)); } } diff --git a/ffi/src/c.rs b/ffi/src/c.rs index acdee884d..1a58c464c 100644 --- a/ffi/src/c.rs +++ b/ffi/src/c.rs @@ -491,14 +491,13 @@ impl Header<'_> { )); } self.out.push_str(&comment(docs.trim(), "")); - if let Some((handle, poll, free)) = &export.future { + if let Some(future) = &export.future { + let handle = &future.handle; self.line(format!("typedef struct {handle} {handle};")); - let call = self.prototype(&export.call); - let poll = self.prototype(poll); - let free = self.prototype(free); - self.line(call); - self.line(poll); - self.line(free); + for sig in [&export.call, &future.start, &future.poll, &future.free] { + let prototype = self.prototype(sig); + self.line(prototype); + } return; } let call = self.prototype(&export.call); diff --git a/ffi/src/cpp.rs b/ffi/src/cpp.rs index ce5e1cd04..b98ffca3a 100644 --- a/ffi/src/cpp.rs +++ b/ffi/src/cpp.rs @@ -506,7 +506,7 @@ impl<'a> Header<'a> { let futures = self.functions(); for (function, owner) in &futures { let export = abi.export(function, owner.as_ref()); - if let Some((handle, _, _)) = &export.future { + if let Some(abi::Future { handle, .. }) = &export.future { let name = self.global(&self.name(handle)); self.line(format!( "namespace detail {{\nstruct {name}_op;\n}}\nusing {name} = ::{ns}::future;" @@ -1111,7 +1111,7 @@ impl<'a> Header<'a> { }; let free = !matches!(class, Some((_, Owner::Member))); let ret = match &export.future { - Some((handle, _, _)) => self.q(handle), + Some(future) => self.q(&future.handle), None => self.output(function), }; let mut params: Vec = Vec::new(); @@ -1219,7 +1219,7 @@ impl<'a> Header<'a> { } let call = format!("::{}({})", export.call.name, args.join(", ")); let mut body = prelude; - if let Some((handle, _, _)) = &export.future { + if let Some(abi::Future { handle, .. }) = &export.future { body.push(format!("return {d}::access::wrap<{}>({call});", self.q(handle))); } else { let (declared, ok, err) = self.outs(function, &outs, &ret); @@ -1423,7 +1423,7 @@ impl<'a> Header<'a> { fn future(&mut self, function: &Function, owner: Option<&Ty>) { let abi = self.abi; let export = abi.export(function, owner); - let Some((handle, poll, free)) = &export.future else { + let Some(abi::Future { handle, poll, free, .. }) = &export.future else { return; }; let name = self.name(handle); diff --git a/ffi/src/dart.rs b/ffi/src/dart.rs index 0318d9dc4..a3be6599d 100644 --- a/ffi/src/dart.rs +++ b/ffi/src/dart.rs @@ -2040,7 +2040,7 @@ impl<'a> Gen<'a> { let mut body = Vec::new(); let ret_type; - if let Some((_, poll, free)) = &export.future { + if let Some(abi::Future { start, poll, free, .. }) = &export.future { ret_type = format!("Cancelable<{ok_dt}>"); // The poll, which completes the future. let mut poll_args = vec!["_n.home".to_string(), "f".to_string(), "w".to_string()]; @@ -2057,13 +2057,14 @@ impl<'a> Gen<'a> { _ => self.lift_outs(&function.ok, &ok), }; let mut cases = vec![format!("\t\tcase 1:\n\t\t\tc._ready({ready});\n\t\t\treturn true;")]; + let mut started = vec![format!("case 1:\n\tc_._ready({ready});")]; if let (Some(err), Some(thrown)) = (&err, &thrown) { let lifted = self.lift(err, &self.out_value("err", poll_outs.last().map(|(_, t)| t))); - cases.push(format!( - "\t\tcase 2:\n\t\t\tc._error({});\n\t\t\treturn true;", - thrown_value(thrown, &lifted) - )); + let error = thrown_value(thrown, &lifted); + cases.push(format!("\t\tcase 2:\n\t\t\tc._error({error});\n\t\t\treturn true;")); + started.push(format!("case 2:\n\tc_._error({error});")); } + started.push("default:\n\tc_._f = o_f_.value;".to_string()); let poll_name = format!("_p_{c_name}"); self.helpers.insert( poll_name.clone(), @@ -2074,15 +2075,29 @@ impl<'a> Gen<'a> { cases.iter().map(|c| c.replace("\n\t\t", "\n\t\t\t").replacen("\t\tcase", "\t\t\tcase", 1)).collect::>().join("\n") ), ); - body.push("final ffi.Pointer f_;".to_string()); + // The call and its first poll in one, which completes the call if ready, or else + // leaves it the future to poll. + let start_fn = self.served(start); + let mut start_args = args.clone(); let mut inner = pre.clone(); - inner.push(format!("f_ = {call_fn}({});", args.join(", "))); + for (n, ty) in &poll_outs { + inner.push(self.out_decl(n, ty)); + start_args.push(format!("o_{n}")); + } + inner.push("final o_f_ = a_>();".to_string()); + start_args.push("o_f_".to_string()); + inner.push(format!( + "switch ({start_fn}({})) {{\n{}\n}}", + start_args.join(", "), + indent(&started.join("\n")) + )); inner.extend(after.clone()); + body.push(format!("final c_ = _Call<{ok_dt}>._({free_fn}, {poll_name});")); body.push(format!( "final a_ = _Arena();\ntry {{\n{}\n}} finally {{\n\ta_.end();\n}}", indent(&inner.join("\n")) )); - body.push(format!("return _call<{ok_dt}>(f_, {free_fn}, {poll_name});")); + body.push("return _call(c_);".to_string()); } else { ret_type = ok_dt.clone(); let mut inner = pre.clone(); diff --git a/ffi/src/glue.rs b/ffi/src/glue.rs index 22667f7ac..bd3c55674 100644 --- a/ffi/src/glue.rs +++ b/ffi/src/glue.rs @@ -1615,7 +1615,7 @@ impl Gen<'_> { )); } let body = stmts.join("\n\t"); - if let Some((_, poll, free)) = &export.future { + if let Some(abi::Future { start, poll, free, .. }) = &export.future { // A function that is not `async` runs when called, and what it returns // borrows nothing, so it is called now; an `async fn` runs when polled. let (now, later) = match (function.eager, &function.task) { @@ -1647,10 +1647,15 @@ impl Gen<'_> { .enumerate() .map(|(n, out)| format!("let {out} = outs[{n}];\n\t\t\t")) .collect(); + // The make's body, emitted again inside `_start` so its aborts name the + // function C called. + let make = format!( + "{body}{now}\n\trt::Future::new(async move {{ {later} }}, move |result, outs: &[*mut ::std::ffi::c_void], err: *mut ::std::ffi::c_void| -> i32 {{\n\t\tlet _ = (outs, err);\n\t\t{bind}{}\n\t}})", + self.finish(function, &ok, ("v", "rt::READY", "rt::ERROR")) + ); self.export(format!( - "#[unsafe(no_mangle)]\npub unsafe extern \"C\" fn {name}({}) -> *mut rt::Future {{\n\tlet w = \"{name}\";\n\tlet _ = w;\n\t{body}{now}\n\trt::Future::new(async move {{\n\t\tlet result = {later};\n\t\t::std::boxed::Box::new(move |outs: &[*mut ::std::ffi::c_void], err: *mut ::std::ffi::c_void| -> i32 {{\n\t\t\tlet _ = (outs, err);\n\t\t\t{bind}{}\n\t\t}}) as rt::Done\n\t}})\n}}\n", + "#[unsafe(no_mangle)]\npub unsafe extern \"C\" fn {name}({}) -> *mut rt::Future {{\n\tlet w = \"{name}\";\n\tlet _ = w;\n\t{make}\n}}\n", params.join(", "), - self.finish(function, &ok, ("v", "rt::READY", "rt::ERROR")) )); let poll_params: Vec = poll.params[2..] .iter() @@ -1672,6 +1677,17 @@ impl Gen<'_> { "#[unsafe(no_mangle)]\npub unsafe extern \"C\" fn {}(future: *mut rt::Future) {{\n\tunsafe {{ rt::Future::free(future) }}\n}}\n", free.name )); + // The call and its first poll in one: most futures are ready at once. + let start_params: Vec = (params.iter().cloned()) + .chain(poll_params) + .chain(["future: *mut *mut rt::Future".to_string()]) + .collect(); + self.export(format!( + "#[unsafe(no_mangle)]\npub unsafe extern \"C\" fn {}({}) -> i32 {{\n\tlet w = \"{}\";\n\tlet _ = w;\n\tlet made = {{\n\t{make}\n\t}};\n\tunsafe {{ rt::Future::start(made, &[{outs}], {err}, future, w) }}\n}}\n", + start.name, + start_params.join(", "), + start.name + )); return; } let outs: Vec = export.call.params[params.len()..] diff --git a/ffi/src/go.rs b/ffi/src/go.rs index 8d5143264..a252d3baf 100644 --- a/ffi/src/go.rs +++ b/ffi/src/go.rs @@ -2359,18 +2359,21 @@ impl<'a> Gen<'a> { } } - if let Some((_, poll, free)) = &export.future { - body.push(format!("f_ := C.{c_name}({})", args.join(", "))); - body.push(format!("defer C.{}(f_)", free.name)); + if let Some(abi::Future { + handle, + start, + poll, + free, + }) = &export.future + { let mut decls = Vec::new(); for (k, ty) in oks.iter().enumerate() { let go = self.go(ty); decls.push(format!("var r{k}_ {go}")); } + body.extend(decls); + let zeros: String = (0..oks.len()).map(|k| format!("r{k}_, ")).collect(); let poll_outs: Vec<(String, Ty)> = poll.params[2..].to_vec(); - let mut inner: Vec = poll_outs.iter().map(|(n, ty)| self.out_decl(n, ty)).collect(); - let mut call_args = vec!["f_".to_string(), "w_".to_string()]; - call_args.extend(poll_outs.iter().map(|(n, _)| format!("&c_{n}_"))); let mut ready: Vec = Vec::new(); for (k, ty) in oks.iter().enumerate() { let out = out_name(&function.ok, k); @@ -2382,17 +2385,62 @@ impl<'a> Gen<'a> { let lifted = self.lift_err(err, "c_err_"); cases.push(format!("case {}:\n\treturn true, {lifted}", poll_error(abi))); } - inner.push(format!( - "switch C.{}({}) {{\n{}\n}}\nreturn false, nil", + let cases = cases.join("\n"); + // Lifts what a call or a poll wrote, once either reports the future finished. + let decls: Vec = poll_outs.iter().map(|(n, ty)| self.out_decl(n, ty)).collect(); + let ptrs: Vec = poll_outs.iter().map(|(n, _)| format!("&c_{n}_")).collect(); + let mut call_args = vec!["f_".to_string(), "w_".to_string()]; + call_args.extend(ptrs.iter().cloned()); + let mut polled = decls.clone(); + polled.push(format!( + "switch C.{}({}) {{\n{cases}\n}}\nreturn false, nil", poll.name, - call_args.join(", "), - cases.join("\n") - )); - body.extend(decls); - body.push(format!( - "err_ := await_(ctx, func(w_ cWaker) (bool, error) {{\n{}\n}})", - indent(&inner.join("\n")) + call_args.join(", ") )); + let polled = indent(&polled.join("\n")); + if function.eager { + // An eager call runs Rust, and any callback it makes, before its future + // exists, and a callback may cancel the context: so it is made, then the + // context checked, then polled, never started. + body.push(format!( + "f_ := C.{c_name}({})\ndefer C.{}(f_)", + args.join(", "), + free.name + )); + body.push(format!( + "if err_ := ctx.Err(); err_ != nil {{\n\treturn {zeros}err_\n}}" + )); + body.push(format!("poll_ := func(w_ cWaker) (bool, error) {{\n{polled}\n}}")); + body.push("done_, err_ := poll_(cWaker{})\nif !done_ {\n\terr_ = await_(ctx, poll_)\n}".to_string()); + } else { + // A context done already takes nothing, even what is ready: the future is made + // and dropped unpolled, as Rust drops one it never awaits. Making an `async fn`'s + // future runs nothing, so the check needs no repeat after it. + body.push(format!( + "if err_ := ctx.Err(); err_ != nil {{\n\tC.{}(C.{c_name}({}))\n\treturn {zeros}err_\n}}", + free.name, + args.join(", ") + )); + let mut start_args = args.clone(); + start_args.extend(ptrs.iter().cloned()); + start_args.push("&f_".to_string()); + let mut inner = decls; + inner.push(format!( + "switch C.{}({}) {{\n{cases}\n}}\nreturn false, nil", + start.name, + start_args.join(", ") + )); + body.push(format!("var f_ *C.{handle}")); + body.push(format!( + "done_, err_ := func() (bool, error) {{\n{}\n}}()", + indent(&inner.join("\n")) + )); + body.push(format!( + "if !done_ {{\n\tdefer C.{}(f_)\n\terr_ = await_(ctx, func(w_ cWaker) (bool, error) {{\n{}\n\t}})\n}}", + free.name, + indent(&polled) + )); + } let mut returns: Vec = (0..oks.len()).map(|k| format!("r{k}_")).collect(); returns.push("err_".to_string()); body.push(format!("return {}", returns.join(", "))); diff --git a/ffi/src/kotlin.rs b/ffi/src/kotlin.rs index 4ad239d41..db5733393 100644 --- a/ffi/src/kotlin.rs +++ b/ffi/src/kotlin.rs @@ -1870,7 +1870,7 @@ impl<'a> Gen<'a> { let args: Vec<&str> = args.iter().map(|a| a.as_str()).collect(); format!("crate::{c_name}({})", args.join(", ")) }; - if let Some((_, poll, free)) = &export.future { + if let Some(abi::Future { poll, free, .. }) = &export.future { // The call makes the future and polls it once, returning its result if ready, or // else the future, which a native poll and free take. let poll_bare = self.bare(&poll.name).to_string(); diff --git a/ffi/src/python.rs b/ffi/src/python.rs index 183115cfe..40e8335fd 100644 --- a/ffi/src/python.rs +++ b/ffi/src/python.rs @@ -1585,7 +1585,7 @@ impl<'a> Gen<'a> { let ok = strip_refs(&function.ok); let err = function.err.as_ref().map(strip_refs); let exception = err.as_ref().map(|e| self.exception(e)); - if let Some((_, poll, free)) = &export.future { + if let Some(abi::Future { poll, free, .. }) = &export.future { let poll_outs = &poll.params[2..]; let (decl, ptrs, value) = self.outs( poll_outs, diff --git a/ffi/src/swift.rs b/ffi/src/swift.rs index 2e0b9fe3a..fc5bd3189 100644 --- a/ffi/src/swift.rs +++ b/ffi/src/swift.rs @@ -138,6 +138,7 @@ pub fn generate(abi: &Abi<'_>, packages: &str, contract: u64) -> Result Gen<'a> { }; // Only a future or a `Result` throws, and a future only once it is awaited. let throws = function.future || function.err.is_some(); - let tr = match function.err.is_some() && !function.future { + // A future's error throws from the call that starts it, as from its polls. + let tr = match function.err.is_some() { true => "try ", false => "", }; @@ -1541,12 +1543,14 @@ impl<'a> Gen<'a> { let mut inner = Vec::new(); let tail; - if let Some((_, poll, free)) = &export.future { - inner.push(format!("{c_name}({})", exprs.join(", "))); + // What a cancelled task runs instead of the start: the call alone, whose future + // it drops unpolled, as Rust drops one it never awaits. + let mut cancelled = None; + if let Some(abi::Future { start, poll, free, .. }) = &export.future { + cancelled = Some(format!("{}({c_name}({}))", free.name, exprs.join(", "))); let poll_outs: Vec<(String, Ty)> = poll.params[2..].to_vec(); - let mut body: Vec = poll_outs.iter().map(|(n, ty)| self.out_decl(n, ty)).collect(); - let mut call_args = vec!["f_".to_string(), "w_".to_string()]; - call_args.extend(poll_outs.iter().map(|(n, _)| format!("&c_{n}_"))); + let decls: Vec = poll_outs.iter().map(|(n, ty)| self.out_decl(n, ty)).collect(); + let ptrs: Vec = poll_outs.iter().map(|(n, _)| format!("&c_{n}_")).collect(); let ready = match ok { Type::Unit => "()".to_string(), _ => self.lift_outs(&function.ok, &ok), @@ -1562,11 +1566,23 @@ impl<'a> Gen<'a> { )); } cases.push("default: return nil".to_string()); + let cases = cases.join("\n"); + inner.extend(decls.iter().cloned()); + let start_args: Vec = (exprs.iter().cloned()) + .chain(ptrs.iter().cloned()) + .chain(["&f_".to_string()]) + .collect(); + inner.push(format!( + "switch {}({}) {{\n{cases}\n}}", + start.name, + start_args.join(", ") + )); + let mut body = decls; + let call_args: Vec = ["f_".to_string(), "w_".to_string()].into_iter().chain(ptrs).collect(); body.push(format!( - "switch {}({}) {{\n{}\n}}", + "switch {}({}) {{\n{cases}\n}}", poll.name, - call_args.join(", "), - cases.join("\n") + call_args.join(", ") )); tail = Some(format!( "return try await Native.run(f_, {}) {{ f_, w_ in\n{}\n}}", @@ -1606,27 +1622,64 @@ impl<'a> Gen<'a> { } } } - // Wrap the call in each argument's closure, innermost last. - let mut code = inner.join("\n"); - for arg in args.iter().rev() { - if let Some((open, close)) = &arg.open { - let lines = code.lines().count(); - // A closure of one line returns its value; of more, its own `return` does. - let body = match lines > 1 || code.starts_with("return ") || code.starts_with("throw ") { - true => code.clone(), - false => format!("return {code}"), - }; - code = format!("return {open}\n{}\n{close}", indent(&body)); + // Wrap the call in each argument's closure, innermost last, each opened with `try` + // only where the call throws. + let wrap = |inner: String, throws: bool| { + let mut code = inner; + for arg in args.iter().rev() { + if let Some((open, close)) = &arg.open { + // An opener may hold several `try`s, as a lent closure's two do. + let open = match throws { + true => open.clone(), + false => open.replace("try ", ""), + }; + let lines = code.lines().count(); + // A closure of one line returns its value; of more, its own `return` does. + let body = match lines > 1 || code.starts_with("return ") || code.starts_with("throw ") { + true => code.clone(), + false => format!("return {code}"), + }; + code = format!("return {open}\n{}\n{close}", indent(&body)); + } } - } + code + }; + let code = wrap(inner.join("\n"), true); let mut body = prelude; match tail { + // An eager call runs Rust, and any callback it makes, before its future exists, + // and a callback may cancel the task: so it is made, then `Native.made` checks the + // task before its first poll, never started. + Some(tail) if function.eager => { + let made = wrap(format!("{c_name}({})", exprs.join(", ")), false); + body.push(format!("let f_ = {}", made.strip_prefix("return ").unwrap_or(&made))); + body.push(tail.replacen("Native.run(", "Native.made(", 1)); + } Some(tail) => { - let create = match code.starts_with("return ") { - true => code["return ".len()..].to_string(), - false => code, + let made = wrap(cancelled.expect("a future has its cancel"), false); + let made = made.strip_prefix("return ").unwrap_or(&made); + body.push(format!( + "// A task cancelled already takes nothing, even what is ready.\nif Task.isCancelled {{\n{}\n\tthrow CancellationError()\n}}", + indent(made) + )); + // The call and its first poll in one: the output if ready, else the future. + let output = match ok { + Type::Unit => "Void".to_string(), + _ => self.sw(&ok), + }; + let tr = match err.is_some() { + true => "try ", + false => "", }; - body.push(format!("let f_ = {create}")); + let code = match code.lines().count() > 1 || code.starts_with("return ") { + true => code, + false => format!("return {code}"), + }; + body.push("var f_: OpaquePointer? = nil".to_string()); + body.push(format!( + "if let r_ = {tr}Native.start(({output}).self, {{\n{}\n}}) {{\n\treturn r_\n}}", + indent(&code) + )); body.push(tail); } None => body.push(code), diff --git a/ffi/tests/api.c b/ffi/tests/api.c index 0991381bc..8edebc0a2 100644 --- a/ffi/tests/api.c +++ b/ffi/tests/api.c @@ -126,6 +126,48 @@ static bool foreign_futures(void) { return ok && dropped.freed && !dropped.waker.wake; } +/* Futures started, made and polled once in one call: whether each behaved. */ +static bool started_futures(void) { + api_waker waker = {.wake = wake, .drop = drop, .data = NULL}; + uint32_t out = 0; + + /* Ready at once: the result is written, the future freed, and the handle nulled + * over what the caller left in it. */ + foreign now = {.ready = 1}; + api_after_future *after = (api_after_future *)&out; + bool ok = api_after_start(foreign_future(&now), 4, &out, &after) == API_POLL_READY; + ok = ok && out == 8 && !after && now.polls == 1 && now.freed; + + /* Pending: the handle is returned, and a poll with a waker replaces the first's, + * which wakes nothing. */ + foreign later = {.ready = 3}; + ok = ok && api_after_start(foreign_future(&later), 5, &out, &after) == API_POLL_PENDING && after; + later.waker.wake(later.waker.data); + int woken = wakes; + ok = ok && api_after_poll(after, waker, &out) == API_POLL_PENDING && later.polls == 2; + later.waker.wake(later.waker.data); + ok = ok && wakes == woken + 1; + ok = ok && api_after_poll(after, waker, &out) == API_POLL_READY && out == 10; + api_after_future_free(after); + ok = ok && later.freed; + + /* Pending, then freed unfinished: the future it owns is freed with it. */ + foreign dropped = {.ready = 1000}; + after = NULL; + ok = ok && api_after_start(foreign_future(&dropped), 6, &out, &after) == API_POLL_PENDING; + api_after_future_free(after); + ok = ok && dropped.freed && !dropped.waker.wake; + + /* Failed at once: the error is written, and the handle nulled. */ + foreign never = {.ready = 1000}; + api_within_future *within = (api_within_future *)&out; + api_Error err; + ok = ok && api_within_start(0, foreign_future(&never), &err, &within) == API_POLL_ERROR; + ok = ok && err.tag == API_ERROR_OVERFLOW && !within && never.freed; + api_Error_free(&err); + return ok; +} + /* made's closure: a string it hands Rust, as api_buf_copy made it. */ static api_buf made_times(void *data, uint32_t n) { (void)data; @@ -1598,7 +1640,7 @@ static int script(void) { if (drops != 12) { return 1; } - return closures() && foreign_futures() ? 0 : 1; + return closures() && foreign_futures() && started_futures() ? 0 : 1; } /* Each breaks a contract, which must abort with a message naming the function. */ @@ -1660,6 +1702,15 @@ static int broken(const char *which) { } else if (strcmp(which, "future") == 0) { /* A future with no poll. */ api_after((api_foreign_future){0}, 1); + } else if (strcmp(which, "started") == 0) { + /* A future with no poll, through a start, which the abort names. */ + uint32_t out; + api_after_future *after; + api_after_start((api_foreign_future){0}, 1, &out, &after); + } else if (strcmp(which, "start") == 0) { + /* A start with nowhere to put the handle of a future that pends. */ + uint32_t out; + api_yielded_start(1, &out, NULL); } else if (strcmp(which, "array") == 0) { api_vec_u16 ports = api_ports((api_vec_socket_addr){0}); ports.len = 1; diff --git a/ffi/tests/api.go b/ffi/tests/api.go index d185b44e0..a7f8880e1 100644 --- a/ffi/tests/api.go +++ b/ffi/tests/api.go @@ -451,6 +451,15 @@ func script() { cancel() check(errors.Is(channel.SendLater(cancelled, []byte("eager")), context.Canceled), "a cancelled call returns its context's error") fmt.Printf("recv %s\n", must(channel.Recv(ctx)).Copy()) + // An eager call whose callback cancels its context while it makes its future is never + // polled: only the eager half logs. + { + inner, cancel := context.WithCancel(ctx) + log := &cancelLog{cancel: cancel} + _, err := api.EmitLater(inner, log, []byte{1, 2}) + check(errors.Is(err, context.Canceled), "an eager call cancelled as it makes its future returns its context's error") + check(log.lines.Load() == 2, "an eager call cancelled as it makes its future is never polled") + } channel.Send([]byte("sized")) sized, size, err := channel.RecvSized(ctx) check(err == nil, "recv sized") @@ -1473,6 +1482,18 @@ func broken(which string) { } } +// cancelLog cancels its context at each line, and counts them. +type cancelLog struct { + cancel context.CancelFunc + lines atomic.Int32 +} + +func (l *cancelLog) Log(uint8, string) bool { + l.cancel() + l.lines.Add(1) + return true +} + type panicking struct{} func (panicking) Log(uint8, string) bool { diff --git a/ffi/tests/api.swift b/ffi/tests/api.swift index b218f56c7..1628cf933 100644 --- a/ffi/tests/api.swift +++ b/ffi/tests/api.swift @@ -21,6 +21,22 @@ final class PrintLog: Log { } } +/// A Log that cancels a task at each line, and counts them. +final class CancelLog: Log, @unchecked Sendable { + let task: Cell?> + let lines = Cell(0) + + init(_ task: Cell?>) { + self.task = task + } + + func log(_: UInt8, _: String) -> Bool { + task.value?.cancel() + lines.value += 1 + return true + } +} + func bound(_ b: BoundU64) -> String { switch b { case .included(let n): return "Included(\(n))" @@ -932,6 +948,24 @@ func futures() { check(text(payload) == "ready", "a cancelled task's receive takes nothing") } + // An eager call whose callback cancels its task while it makes its future is never + // polled: only the eager half logs. + do { + let task = Cell?>(nil) + let log = CancelLog(task) + let threw = Cell(false) + task.value = q.launch { + do { + _ = try await emitLater(log, Data([1, 2])) + } catch is CancellationError { + threw.value = true + } + } + q.drain() + check(threw.value, "an eager call cancelled as it makes its future throws") + check(log.lines.value == 2, "an eager call cancelled as it makes its future is never polled") + } + // Cancelling a task that waits for no wake resumes it. do { let threw = Cell(false) @@ -980,16 +1014,19 @@ func futures() { until { q.drain(); return got.value != 0 } check(got.value == 10, "an async closure's end wakes Rust") } + // The closure waits, so it pends whether or not its task has run yet, and the check + // waits for the throw, wherever the task resumes. do { + let (stream, _) = AsyncStream.makeStream() let threw = Cell(false) q.launch { do { - try await within(0) {} + try await within(0) { for await _ in stream { break } } } catch is Api.Error { threw.value = true } } - q.drain() + until { q.drain(); return threw.value } check(threw.value, "Rust gives up on an async closure") } do { diff --git a/ffi/tests/fixtures.rs b/ffi/tests/fixtures.rs index ba18356f2..0f054aab0 100644 --- a/ffi/tests/fixtures.rs +++ b/ffi/tests/fixtures.rs @@ -200,6 +200,8 @@ fn api_c() { ("family", "api_nexts: an IPv6 socket address where IPv4 is taken"), ("array", "api_vec_u16_free: an array is not as Rust returned it"), ("future", "api_after: a future's poll is null"), + ("started", "api_after_start: a future's poll is null"), + ("start", "api_yielded_start: a future's handle pointer is null"), ("lent", "api_Registry_edit: a lent handle was freed"), ( "lent moved", diff --git a/quest/m0/swift-gives-up.md b/quest/m0/swift-gives-up.md deleted file mode 100644 index dd5650f5a..000000000 --- a/quest/m0/swift-gives-up.md +++ /dev/null @@ -1,28 +0,0 @@ -# [XS] api_swift waits for the error an async closure's timeout raises - -## Goal - -`api_swift` in `ffi/tests/fixtures.rs` passes every run: its "Rust gives up on an async -closure" check in `ffi/tests/api.swift` no longer fails intermittently. - -## Plan - -Found while validating [#740](https://github.com/kixel-ai/rust2/pull/740) on 2026-11-13: -`just ffi test --test fixtures` failed `api_swift` there in 2 of 4 runs, and at `main` -`7008d1d3` in 1 of 5, each time with `"futures" failed: Rust gives up on an async closure`. -The test passes alone. - -Two races fit, and the fix covers both: - -- The closure is empty, so it can finish before the budget runs out: `Pollable.poll` in - `ffi/runtime/rt.swift` starts the closure's Task and then loads `done` again, and - `Within::poll` in `ffi/fixtures/api/src/lib.rs` polls the inner future before checking - its budget. When the Task finishes in that window, `within(0) {}` returns `Ok(())` and - nothing throws. The closure must stay pending, such as by awaiting a stream nothing - yields to, as the check above it does. -- The check reads `threw` after a single `q.drain()`, so it also fails if the task's - resumption with the `Overflow` error reaches the queue after `drain` returned. The check - above it waits with `until { q.drain(); return got.value != 0 }`; wait the same way. - -Confirm which one fails before fixing, such as by delaying the closure's Task or the -resumption, and fix the test's timing rather than retrying it. diff --git a/quest/m1/ffi-export/README.md b/quest/m1/ffi-export/README.md index f022ef841..0de723427 100644 --- a/quest/m1/ffi-export/README.md +++ b/quest/m1/ffi-export/README.md @@ -56,8 +56,8 @@ Every generator, the [targets m3 plans](/quest/m3/README.md) included: other call, and fails loud on a mismatch (done in all six); - keeps the C ABI's bulk and zero-copy payloads, with no per-byte loop on either side (done); - wraps the C ABI's futures in the language's native future (done). Those futures already - take the result in the poll that finishes them, so a cancel never loses a frame, cost three - crossings when ready at once, and reuse a repeated waker. The native future must latch a + take the result in the poll that finishes them, so a cancel never loses a frame, cost one + crossing when ready at once, and reuse a repeated waker. The native future must latch a wake that lands between `PENDING` and parking, prove no loss on cancel in a test that forces that window, and pick a semantic for concurrent pollers. Decided with the maintainer on 2026-10-27: async lands with each generator, not as its own quest. @@ -82,7 +82,7 @@ Decided with the maintainer on 2026-11-12, in the quest audit: epic's `Required` holds back every child, so none could start until moq's pin bump, which waits on a rind release the maintainer holds. - ✅ Every change that breaks the C ABI or a generated API is a child here, so it lands - before moq pins: [the fused async start](/quest/m1/ffi-export/async-start.md), + before moq pins: the fused async start, [owned buffers](/quest/m1/ffi-export/owned-bufs.md), and [numeric slices](/quest/m1/ffi-export/numeric-slices.md). Work that changes no generated API is [its own epic](/quest/m1/ffi-speed/README.md), which does not hold the pin. diff --git a/quest/m1/ffi-export/async-start.md b/quest/m1/ffi-export/async-start.md deleted file mode 100644 index d96cd7453..000000000 --- a/quest/m1/ffi-export/async-start.md +++ /dev/null @@ -1,44 +0,0 @@ -# [M] A ready async call in one crossing - -## Goal - -A [rust2-ffi](/ffi/README.md) async call whose future is ready at its first poll costs one -crossing and no future handle: one C function makes the future, polls it once, and writes -the result. A call that suspends allocates once. [The binding benchmark](/bench/ffi/README.md)'s -ready read then costs each language close to its write, against 155 ns more in C, 315 in -Swift, and 571 in Go today. - -## Plan - -Decided with the maintainer on 2026-11-12, in the quest audit: ✅ a child of the exporter -epic, since it changes every future's C signature and so lands before moq pins. - -What a ready call costs today, in `ffi/runtime/rt.rs` and `ffi/src/glue.rs`: - -- Four allocations: `Future::new` boxes the future and then the `Future` around it, each - result is a boxed `Done` closure, and the first poll wraps even the all-null waker in an - `Arc`. -- Three crossings from Go and Swift: the call, the poll, and the free - (`ffi/runtime/ffi_cgo.go`, `ffi/src/go.rs`). Kotlin and Python already fuse the call and - first poll inside their Rust glue. - -Recommended, in this order: - -- A null waker lifts to `Waker::noop()`, with no `Arc`. No contract change. -- One typed allocation per future, with a poll per export that lowers the result - directly, so the `Done` box and the second box go. -- `_start(args, outs, err, &future)` makes the future and polls it once. Ready: it - writes the result and hands back no handle. Pending: it hands back the handle, and - `_poll` and `_future_free` stay as they are. - -Each generator's native future keeps the rules [the epic](/quest/m1/ffi-export/README.md) -sets: the result arrives in the poll that finishes the future, so a cancel loses no frame, -and a wake between `PENDING` and parking is latched. Re-run each language's forced-window -cancel test, and re-measure the bench's ready and woken reads. - -Public API: every async export's C functions and the contract hash (breaking), and each -binding's future wrapper. - -## Related - -- [A cheaper woken read](/quest/m1/ffi-speed/woken-read.md) - the pending path this leaves alone diff --git a/quest/m1/ffi-export/ship.md b/quest/m1/ffi-export/ship.md index fa4c84e41..c8b52197d 100644 --- a/quest/m1/ffi-export/ship.md +++ b/quest/m1/ffi-export/ship.md @@ -25,7 +25,6 @@ then see change. A sibling planned later adds itself to `Required` here. - [Copy types cross by value](/quest/m1/ffi-export/copy-values.md) - how a `Copy` type is held in every language - [Not Send](/quest/m1/ffi-export/not-send.md) - the types and futures that export checked by thread - [Export moq for every target](/quest/m1/ffi-export/moq-targets.md) - one binding across the targets moq ships -- [Fused async start](/quest/m1/ffi-export/async-start.md) - every future's C signature - [Owned buffers](/quest/m1/ffi-export/owned-bufs.md) - what a buffer's owner means on the way in - [Numeric slices](/quest/m1/ffi-export/numeric-slices.md) - the types of numeric slices and vectors - [Lock every language's names](/quest/m1/ffi-export/lock-names.md) - `--baseline` catches a binding's own rename, so moq can drop its api-diff diff --git a/quest/m1/ffi-speed/ready-await.md b/quest/m1/ffi-speed/ready-await.md new file mode 100644 index 000000000..a8b577aa2 --- /dev/null +++ b/quest/m1/ffi-speed/ready-await.md @@ -0,0 +1,22 @@ +# [M] A cheaper ready await in Swift, Go, and Dart + +## Goal + +A [rust2-ffi](/ffi/README.md) Swift, Go, or Dart await of a future ready at its first +poll costs close to C's, once [returned-object ownership](/quest/m1/ffi-speed/object-ownership.md) +is out of the way. Since `_start` made a ready call one crossing and one allocation, +[the binding benchmark](/bench/ffi/README.md)'s ready read costs C 126 ns over its write, +Rust 93, and Swift 289, Go 414, and Dart 522. Less the returned frame's ownership, Swift's +await still adds about 150 ns over C's, Go's 100, and Dart's 180. + +## Plan + +- Profile each language's ready path from the generated call to the result, after + object-ownership lands so its share is gone: Swift's `async` entry and `Native.start`, + Go's escaped out-pointers and error check, and Dart's `_Call` and `_Arena`. +- Fix what dominates in the generator or runtime, with no API change. +- Re-measure the ready read interleaved with the commit before, CPU first. + +## Required + +- [Returned-object ownership](/quest/m1/ffi-speed/object-ownership.md) - removes Go's and Dart's ownership share of the ready read, so the profile shows only the await's