Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions bench/ffi/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -398,6 +398,23 @@ frame path:
Go 414, Dart 522, Python 445, and Kotlin 590 (from 381, 556, 868, 659, and 615). Go's and
Dart's include the returned frame's ownership, about 190 and 215 ns over C's object return;
the rest is each language's own await.

Profiled 2026-11-15 on **nixos** in user-space instructions a read adds to a write, a
million reads alone in each language: C 1,360, Swift 2,248, Go 4,790, and Dart 5,776.
Swift's await spent most of its share in `Task.isCancelled`'s lookup of the current
task, the async frame's allocation, and `swift_task_switch`, the hop a `nonisolated`
async function makes onto the global executor; since kixel-ai/rust2#839 a future is
`nonisolated(nonsending)` and makes no hop (2,104), which on a caller's actor saves two
jobs a read (9.9 us a ready read on the main actor, then 0.32). Dart's start made a call
object and a completer before it knew it was ready; it now returns `Future.syncValue`
(5,298), leaving Dart's own await, which schedules a microtask even for a complete
future. Go's is the returned frame's ownership (`AddCleanup`, `Cleanup.Stop`, and a
close's two crossings), and cgo's check and heap allocation of each out pointer, which
[Go's pointers](../../quest/m1/ffi-speed/go-pointers.md) takes: what remains after both
is one crossing. Off the main thread, glibc's malloc arena for the thread grows and
trims with `mprotect`, which adds about 60 ns to a write and 20 to a read in C on a
pthread (`MALLOC_ARENA_MAX=1` removes it). Swift's ready read runs in a task on such a
thread and its write above on the main one, so its ready row carries it and C's does not.
5. **Kotlin's record codec.** A record return costs Kotlin 113 ns wall against C's 31,
through the byte-array codec; Swift, which reads the C struct, costs 37.
6. **The handle borrow at the C ABI.** The u64 getter costs Rust 0.4 ns and C 10: the
Expand Down
7 changes: 4 additions & 3 deletions ffi/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -841,8 +841,9 @@ and, for `Swift.Duration`, macOS 13 or iOS 16, and builds in either language mod
`SeekFrom::Start` takes a `UInt64`, calls `seek(start:)`, which by default throws. Rust
calls it from any thread, one call at a time, and reads a thrown error's `description`
as its `io::Error`'s message.
- **Futures** are `async throws` functions, which resume on the task's executor, never
inside a wake. Cancelling the task frees the future, losing nothing, and throws
- **Futures** are `async throws` functions that run on their caller's executor
(`nonisolated(nonsending)`), so a call ready at its first poll never leaves the caller's
actor, and one that suspends resumes there, never inside a wake. Cancelling the task frees the future, losing nothing, and throws
`CancellationError`. Each call is its own future, so it has one poller. The call makes
and polls it in one crossing, `_start`, before suspending, and makes the signal it parks
on only if that poll pends; the signal latches a wake that lands before it parks.
Expand Down Expand Up @@ -1091,7 +1092,7 @@ the library's own URI, mapped to the library by a build hook or by `--native-ass
- **Futures** are `Cancelable<T>`, a `Future<T>` completed on the isolate that made it:
an extension type on Dart's own future, which `await` takes its shortest path through.
The call makes and polls it in one crossing, `_start`, returning it complete if that
poll finished, and makes a waker only if it pends. A wake, from any
poll finished, and makes the state that polls it again, and a waker, only if it pends. A wake, from any
thread, posts to the isolate's port, and its event loop polls again, so a wake never
polls inside Rust, and one that lands during a poll is kept. `cancel()` frees the
future unless it finished, which cancels it in Rust and loses nothing: the next call
Expand Down
58 changes: 17 additions & 41 deletions ffi/runtime/rt.dart
Original file line number Diff line number Diff line change
Expand Up @@ -62,40 +62,33 @@ extension type Cancelable<T>._(Future<T> _future) implements Future<T> {
void cancel() => _n.calls[_future]?.cancel();
}

/// The future of `call`, which its start completed if its first poll finished, or else
/// left the future in Rust that pended, polled again here with a waker.
/// The future of `call`, whose start left the future in Rust that pended: polled again
/// here with a waker, which its first poll had no use for.
Cancelable<T> _call<T>(_Call<T> call) {
final first = call._first;
if (first != null) return Cancelable._(first);
final done = call._done = dasync.Completer<T>();
_n.calls[done.future] = call;
// Pending: poll again with a waker, which its first poll had no use for.
final future = call._done.future;
_n.calls[future] = call;
call._token = _n.listen(call._wake);
call._waker = _n.waker(call._token);
call._step(call._waker!);
return Cancelable._(done.future);
return Cancelable._(future);
}

/// A [Cancelable]'s future in Rust, until it finishes.
/// A [Cancelable]'s future in Rust, from when its start pended until it finishes.
final class _Call<T> {
/// The future in Rust, once its start pended, until it finishes.
ffi.Pointer<ffi.Void> _f = ffi.nullptr;
/// The future in Rust, until it finishes.
ffi.Pointer<ffi.Void> _f;
final void Function(ffi.Pointer<ffi.Void>, ffi.Pointer<ffi.Void>) _free;

/// Poll once, completing this if it finished: whether it did.
final bool Function(ffi.Pointer<ffi.Void>, _Waker, _Call<T>) _poll;

/// What its first poll finished with, which no one awaits yet.
Future<T>? _first;

/// What completes it once its first poll pended.
dasync.Completer<T>? _done;
final _done = dasync.Completer<T>();
_Waker? _waker;
int _token = 0;
bool _polling = false;
bool _cancelled = false;

_Call._(this._free, this._poll);
_Call._(this._free, this._poll, this._f);

void _wake() {
if (_f != ffi.nullptr) _step(_waker!);
Expand All @@ -120,33 +113,16 @@ final class _Call<T> {
}
}

/// Complete it with what Rust returned, from its poll: synchronously at the first, as
/// nothing listens yet.
void _ready(T value) {
final done = _done;
if (done == null) {
_first = (dasync.Completer<T>.sync()..complete(value)).future;
} else {
done.complete(value);
}
}
/// Complete it with what Rust returned, from its poll.
void _ready(T value) => _done.complete(value);

/// Complete it with the error Rust returned, from its poll: never synchronously, since a
/// future that fails with no listener reports its error as uncaught.
void _error(Object error, [StackTrace? stack]) {
final done = _done;
if (done == null) {
_first = Future<T>.error(error, stack);
} else {
done.completeError(error, stack);
}
}
/// Complete it with the error Rust returned, from its poll.
void _error(Object error, [StackTrace? stack]) => _done.completeError(error, stack);

void _finish() {
_free(_n.home, _f);
_f = ffi.nullptr;
final done = _done;
if (done != null) _n.calls.remove(done.future);
_n.calls.remove(_done.future);
if (_token != 0) {
_n.forget(_token);
_token = 0;
Expand All @@ -156,8 +132,8 @@ final class _Call<T> {
void _cancel() {
_finish();
// Awaiting it is optional once cancelled.
_done!.future.ignore();
_done!.completeError(const Cancelled());
_done.future.ignore();
_done.completeError(const Cancelled());
}

void cancel() {
Expand Down
6 changes: 3 additions & 3 deletions ffi/runtime/rt.swift
Original file line number Diff line number Diff line change
Expand Up @@ -993,7 +993,7 @@ extension Native {
}

/// Suspend until a wake since `reset`, at once if one already landed.
func park() async {
nonisolated(nonsending) func park() async {
await withUnsafeContinuation { (continuation: UnsafeContinuation<Void, Never>) in
parked = continuation
if swap(state, Signal.idle, Signal.parked) != Signal.idle {
Expand All @@ -1011,7 +1011,7 @@ extension Native {

/// Await the future `raw` an eager call made, unpolled, as `run` does, but take nothing,
/// even what is ready, if the task is cancelled already, and poll first with no waker.
static func made<T: ~Copyable>(
nonisolated(nonsending) static func made<T: ~Copyable>(
_ raw: OpaquePointer?, _ free: @convention(c) (OpaquePointer?) -> Void,
_ poll: (OpaquePointer?, CWaker) throws -> T?
) async throws -> T {
Expand All @@ -1031,7 +1031,7 @@ extension Native {
/// Await the future `raw`, whose first poll pended: poll it until `poll` returns its
/// output, then free it. A cancelled task frees it unfinished, which cancels it in Rust
/// and loses nothing, and throws `CancellationError`.
static func run<T: ~Copyable>(
nonisolated(nonsending) static func run<T: ~Copyable>(
_ raw: OpaquePointer?, _ free: @convention(c) (OpaquePointer?) -> Void,
_ poll: (OpaquePointer?, CWaker) throws -> T?
) async throws -> T {
Expand Down
18 changes: 13 additions & 5 deletions ffi/src/dart.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2253,14 +2253,22 @@ impl<'a> Gen<'a> {
_ => self.lift_outs(&function.ok, &ok),
};
let mut cases = vec![format!("\t\tcase 1:\n\t\t\tc._ready({ready});\n\t\t\treturn true;")];
let mut started = vec![format!("case 1:\n\tc_._ready({ready});")];
// A start that finished returns a future of its own, complete already if ready:
// only one that pended makes a call to poll it again.
let mut started = vec![format!(
"case 1:\n\treturn Cancelable._(Future<{ok_dt}>.syncValue({ready}));"
)];
if let (Some(err), Some(thrown)) = (&err, &thrown) {
let lifted = self.lift(err, &self.out_value("err", poll_outs.last().map(|(_, t)| t)));
let error = thrown_value(thrown, &lifted);
cases.push(format!("\t\tcase 2:\n\t\t\tc._error({error});\n\t\t\treturn true;"));
started.push(format!("case 2:\n\tc_._error({error});"));
// Never complete already: a future that fails with no listener reports its
// error as uncaught.
started.push(format!(
"case 2:\n\treturn Cancelable._(Future<{ok_dt}>.error({error}));"
));
}
started.push("default:\n\tc_._f = o_f_.value;".to_string());
started.push("default:\n\tf_ = o_f_.value;".to_string());
let poll_name = format!("_p_{c_name}");
self.helpers.insert(
poll_name.clone(),
Expand Down Expand Up @@ -2295,12 +2303,12 @@ impl<'a> Gen<'a> {
inner.push(check.clone());
}
inner.push(format!("switch (started_) {{\n{started}\n}}"));
body.push(format!("final c_ = _Call<{ok_dt}>._({free_fn}, {poll_name});"));
body.push("final ffi.Pointer<ffi.Void> f_;".to_string());
body.push(format!(
"final a_ = _Arena();\ntry {{\n{}\n}} finally {{\n\ta_.end();\n}}",
indent(&inner.join("\n"))
));
body.push("return _call(c_);".to_string());
body.push(format!("return _call(_Call<{ok_dt}>._({free_fn}, {poll_name}, f_));"));
} else {
ret_type = ok_dt.clone();
let mut inner = pre.clone();
Expand Down
8 changes: 7 additions & 1 deletion ffi/src/swift.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2059,10 +2059,16 @@ impl<'a> Gen<'a> {
member.clone(),
);

// A future runs on its caller's executor, as Swift 6.2's `nonisolated(nonsending)`
// says, so a call ready at its first poll never hops off an actor and back.
let isolation = match function.future {
true => "nonisolated(nonsending) ",
false => "",
};
let head = match var {
true => format!("public {stat}var {member}{}", ret.replacen(" -> ", ": ", 1)),
false => format!(
"public {stat}{mutation}func {member}({}){effects}{ret}",
"{isolation}public {stat}{mutation}func {member}({}){effects}{ret}",
params.join(", ")
),
};
Expand Down
10 changes: 10 additions & 0 deletions ffi/tests/api.swift
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,13 @@ final class Queue: TaskExecutor, @unchecked Sendable {
}
}

/// An actor awaiting an eager future, whose closure Rust calls before the call returns.
actor Probe {
func count(_ n: UInt32) async throws -> UInt32 {
try await counted(n) { _ in self.preconditionIsolated("a future runs on its caller's actor") }
}
}

/// `body`'s result, waiting on this thread for it to finish on the global executor.
func block<T: Sendable>(_ body: @escaping @Sendable () async throws -> T) -> T {
let done = DispatchSemaphore(value: 0)
Expand Down Expand Up @@ -1156,6 +1163,9 @@ func futures() {
check(block { try await halved(4) } == 2, "a future ready with Some returns its value")
}

// A future runs on its caller's actor, never hopping off it and back.
check(block { try await Probe().count(2) } == 4, "a future on an actor returns its value")

// Cancelled between its wake and its poll, a receive loses nothing: the payload stays
// for the next.
do {
Expand Down
20 changes: 0 additions & 20 deletions quest/m1/ffi-speed/ready-await.md

This file was deleted.