Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 52 additions & 8 deletions bench/ffi/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,14 +218,14 @@ GIL, which dwarfs the handle's own contention.
## Leaf calls

A leaf is a call the export proves runs no code but std's and the glue's, waits on no
lock, and calls no binding: a primitive `pub` field of a `Send` object reached without a
`Deref`, or a `PartialEq` or `Hash` derived down to primitives ([`Abi::leaf`](../../ffi/src/abi.rs)).
lock, and calls no binding: a method or `pub` field of a `Send` object, taking and returning
scalars, whose body and each `Deref`'s on the way its MIR proves so, or a `PartialEq` or
`Hash` derived down to primitives ([`Abi::leaf`](../../ffi/src/abi.rs)).
Python keeps the GIL across one, Dart makes it a leaf call (`isLeaf`) to the C function
itself on the isolate's thread, with no serving thunk, no thread hop, and no `_Arena`, and
Kotlin marks its native method `@FastNative` ([Kotlin on ART](#kotlin-on-art)).
The benchmark's u64 getter, `demand.sequence()`, is a method, which the proof cannot see
into, so the `field` case reads `track::Info::priority` instead; `group.sequence` is a
field too, but reached through `group::Producer`'s `Deref`, which is user code.
The `field` case reads `track::Info::priority`, a field; the u64 getter, `demand.start()`,
a method returning one, is a leaf since the export reads its body ([below](#leaf-bodies)).

Measured 2026-11-19 on **fedora**, an AMD Ryzen 9 9950X (16 cores, 32 threads), through
`quest gate`, against moq `ffcb7cd9`. Before is rust2 `52070b3d` with this change's
Expand Down Expand Up @@ -263,7 +263,32 @@ within the spread of its rounds.
Python's 37 ns is more than the 10 to 15 the per-language audit estimated for the GIL's
round trip, as `rt::call` does more around it: it may look for the event loop running, and
after the call drains what other threads freed and resumes what the call woke, all of which
a leaf skips. A getter that is no leaf, such as `demand.sequence()`, still pays it.
a leaf skips. A call that is no leaf, such as `track.is_closed()`, which takes moq's lock,
still pays it.

### Leaf bodies

A method's body, and a `Deref`'s on the way to a member, is a leaf when its MIR
([`leaf.rs`](../../ffi/src/leaf.rs)) calls only bodies proven the same way and core's
intrinsics that only compute, drops nothing whose drop runs other code, has no loop, no
recursion, no assertion, and no static. So `demand.start()` and `group::Producer`'s
`start`, through its `Deref`, are leaves, as are 91 more of the stand-in's methods and
accessors: Dart makes 348 of its calls leaf calls, against 255.

Measured 2026-11-20 on **fedora**, through `quest gate`, against moq `1aa86d5e`: before is
rust2 `8798a5d7`, after is this change, each program alternating before and after for 5
rounds at a load average of 7 to 15; each cell is the median wall time in ns per call.

| case | before | after |
|---|---|---|
| Python `getter` | 64.5 | 26.7 |
| Python `shared:getter` | 1,790 | 119 |
| Python `own:getter` | 1,779 | 113 |
| Dart `getter` | 26.5 | 15.8 |

The four threads no longer hand the GIL to each other on every call, which was most of
their 1.8 us. Every other case of both languages moved within 4% either way, Python's
`field` included, whose code is the same.

## Kotlin on ART

Expand Down Expand Up @@ -858,8 +883,27 @@ the refusal), 3 rounds each, alternating, at a load average of 24 to 31; medians
The refusal adds about 11 CPU seconds, and its walk of the stand-in's 299,455 reachable
instances takes 6.5 s of them. Wall time ranged 24 to 60 s and 39 to 69 s between rounds
at this load, so its gap is rough; the walk's check runs after moq's crates, with nothing
beside it. Peak memory doubles, steady across rounds. A native export reads no MIR and
pays none of it.
beside it. Peak memory doubles, steady across rounds.

A native export reads MIR only for its [leaf bodies](#leaf-bodies), which every crate
encodes for it with `-Zalways-encode-mir`, as `cargo check` encodes none. Measured
2026-11-20 on **fedora** the same way, the stand-in's native export from an emptied target
before (rust2 `8798a5d7`) and after this change, 10 rounds each, alternating, at a load
average of 9 to 19; medians:

| export | wall s | CPU s | peak RSS |
|---|---|---|---|
| before | 23.0 | 101.8 | 896 MiB |
| after | 25.1 | 110.7 | 937 MiB |

So leaves cost the export about 9 CPU seconds, 9%, and 5% of its peak memory, and the
crates' metadata grows from 135 to 228 MB. The flag alone, measured apart at rust2
`8798a5d7` in 7 rounds at a load of 12 to 25, added 8 CPU seconds and 44 MiB, so nearly
all of it is the flag's: the proof walks only the bodies of methods whose signature a leaf
could have. CPU time ranged 95 to 117 s and 104 to 134 s between rounds, so its gap is
rough. Recording each crate's leaves in the driver as it compiles the crate would cost
less, but needs a store of facts keyed by each crate's build beside cargo's own, which
9 seconds does not repay.

### Lending returned bytes

Expand Down
25 changes: 15 additions & 10 deletions ffi/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -830,12 +830,15 @@ free-threaded build (3.13t, 3.14t), which the module refuses at import with an `
loads the library from beside itself: `lib<prefix>.so`, `lib<prefix>.dylib`, or
`<prefix>.dll`. Its glue converts each Python value to the C value and calls the C ABI
function in-process, so a call is one crossing, which releases the GIL while Rust runs,
so Rust may call Python back from any thread meanwhile. A leaf keeps it: a read or write of
a primitive `pub` field of a `Send` object reached without a `Deref`, or a `PartialEq` or
`Hash` derived, or std's, down to primitives, which the export proves runs no code but
std's and the glue's, waits on no lock, and calls no Python, so it cannot deadlock on the
GIL (`Abi::leaf` in `src/abi.rs` gives the argument). A field read costs 26 ns that way,
against 63 releasing it ([the benchmark](../bench/ffi/README.md#leaf-calls)).
so Rust may call Python back from any thread meanwhile. A leaf keeps it: a `&self` or
`&mut self` method, or a read or write of a `pub` field, of a `Send` object, taking and
returning only scalars, whose body and each `Deref`'s on the way the export proves, from
their MIR, run no code but std's and the glue's in a bounded number of steps, or a
`PartialEq` or `Hash` derived, or std's, down to primitives. A leaf waits on no lock and
calls no Python, so it cannot deadlock on the GIL (`Abi::leaf` in `src/abi.rs` gives the
argument, and `src/leaf.rs` the proof). So `demand.start()`, which returns a field, costs
27 ns, against 65 releasing the GIL, and 0.1 us on four threads at once, against 1.8
([the benchmark](../bench/ffi/README.md#leaf-calls)).
Each function, method, and property but an `async def` is the extension's own, bound in
the module or set on its class as it loads, so a call enters it with no Python frame
between; the Python declares them under `typing.TYPE_CHECKING`, for type checkers. They
Expand Down Expand Up @@ -1448,7 +1451,8 @@ the library's own URI, mapped to the library by a build hook or by `--native-ass
into Dart meanwhile. Any other call runs on the isolate's thread, since nothing it waits
on can call into Dart. A leaf, as Python keeps the GIL across, is a leaf call (`isLeaf`)
to the C function on the isolate's thread, with no hop and no arena: a field read costs
13 ns, against 31, and against 2 us while Rust holds a closure. A call that holds only
13 ns, against 31, and against 2 us while Rust holds a closure, and `demand.start()` 16,
against 27. A call that holds only
its receiver holds it with no arena either. An isolate may move between threads, so a
value that is not `Send` is made on a home, a thread Rust owns, which runs each call on
it.
Expand Down Expand Up @@ -1525,9 +1529,10 @@ packs the api fixture's with npm, installs it into a project of its own, type-ch
function, and each one a crate's `.init_array` static runs when the module loads,
through every call, `dyn` implementation, function pointer, static, and drop it can
reach, std's included, and into C and whatever Rust the C calls back, so a new std needs
no change here: every crate builds with `-Zalways-encode-mir`, std's from the dev
shell's MIR sysroot or else `-Zbuild-std`, which doubles the export's peak memory, as
[measured](../bench/ffi/README.md#the-exports-build-cost); a native export reads no MIR.
no change here: every crate builds with `-Zalways-encode-mir`, as every export's does
for its [leaves](#python), std's from the dev shell's MIR sysroot or else
`-Zbuild-std`, which doubles the export's peak memory, as
[measured](../bench/ffi/README.md#the-exports-build-cost).
A C function or static no linked library defines, which would fail the module's link, is
refused the same way, as is a call into wasm-bindgen, whose imports on `wasm32-wasip1`
compile to stubs that panic. `thread::Builder::spawn` and a socket's `connect` and `bind`
Expand Down
102 changes: 102 additions & 0 deletions ffi/fixtures/api/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2667,6 +2667,13 @@ impl Hooked {
pub fn new(n: u64) -> Self {
Self { n, gated: Gated(n) }
}

/// Its count, by a method that runs the hook first: no leaf call, though it returns
/// what a leaf would.
pub fn count(&self) -> u64 {
run_hook();
self.n
}
}

/// A value whose `PartialEq`, written by hand, claims to be derived, as a procedural
Expand Down Expand Up @@ -2752,6 +2759,76 @@ impl Plain {
pinned: (-1, 'p'),
}
}

/// Its count, by a method whose body only reads it: a leaf call.
pub fn count(&self) -> u64 {
self.n
}

/// Set its count, by a method whose body only writes it: a leaf call.
pub fn recount(&mut self, n: u64) {
self.n = n;
}

/// Count up to its count: a loop as long as the caller makes it, so no leaf call.
pub fn steps(&self) -> u64 {
let mut i = 0u64;
while i < self.n {
i = i.wrapping_add(1);
}
i
}

/// How many locals have dropped, a static's count: no leaf call.
pub fn locals(&self) -> u32 {
LOCALS.load(std::sync::atomic::Ordering::Relaxed)
}
}

/// A [`Plain`] behind a `Deref` and `DerefMut` that only borrow it, so its fields and
/// methods are leaf calls through it too.
pub struct Wrapped(Plain);

impl Wrapped {
pub fn new(n: u64) -> Self {
Self(Plain::new(n, false))
}
}

impl std::ops::Deref for Wrapped {
type Target = Plain;

fn deref(&self) -> &Plain {
&self.0
}
}

impl std::ops::DerefMut for Wrapped {
fn deref_mut(&mut self) -> &mut Plain {
&mut self.0
}
}

/// A [`Plain`] behind a `Deref` that runs the hook, so nothing reached through it is a
/// leaf call.
#[cfg(not(target_arch = "wasm32"))]
pub struct Guarded(Plain);

#[cfg(not(target_arch = "wasm32"))]
impl Guarded {
pub fn new(n: u64) -> Self {
Self(Plain::new(n, false))
}
}

#[cfg(not(target_arch = "wasm32"))]
impl std::ops::Deref for Guarded {
type Target = Plain;

fn deref(&self) -> &Plain {
run_hook();
&self.0
}
}

/// A value whose derived `PartialEq` and `Hash` reach no user code but walk a string as
Expand All @@ -2768,6 +2845,31 @@ impl Labeled {
}
}

/// Bytes as many as the caller asks for.
pub struct Buffer {
bytes: Vec<u8>,
}

impl Buffer {
/// A buffer of `len` ones.
pub fn new(len: u32) -> Self {
Self {
bytes: vec![1; len as usize],
}
}

/// Zero its bytes: a fill as long as the caller makes it, with no loop in its MIR, so no
/// leaf call.
pub fn blank(&mut self) {
Comment thread
kixelated marked this conversation as resolved.
Resolved
self.bytes.fill(0);
}

/// Whether every byte is zero.
pub fn zeroed(&self) -> bool {
self.bytes.iter().all(|&b| b == 0)
}
}

/// A handle to `local`'s count if there is one, else a new local: made where `local`
/// stays, or, with none, wherever a local made from nothing would be.
pub fn local_or_new(local: Option<&Local>) -> Local {
Expand Down
45 changes: 23 additions & 22 deletions ffi/src/abi.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1368,9 +1368,17 @@ impl<'a> Abi<'a> {
/// into no binding. Python then keeps the GIL across it, Dart makes it a leaf call,
/// which runs on the isolate's own thread with no thread hop, and Kotlin marks it
/// `@FastNative`, which ART's collector waits on, so its bound keeps a collection from
/// stalling behind it. It is the read or write of a
/// primitive `pub` field of a `Send` object reached without a `Deref`, which no object
/// that is not `Send` may lend.
/// stalling behind it. It is a `&self` or
/// `&mut self` method, or the read or write of a `pub` field, of a `Send` object, which no
/// object that is not `Send` may lend, taking only scalars and returning one or nothing,
/// whose Rust [`Function::leaf`] proves: each body it runs, the method's and each
/// `Deref`'s on the way, calls only bodies proven the same way and core's intrinsics that
/// only compute, drops nothing whose drop runs other code, loops nowhere, recurses
/// nowhere, asserts nothing, which would run the panic hook on failure, and uses no
/// static. Its MIR proves it, which is the code that runs, not how its source looks or
/// what an attribute claims: the MIR the export's check compiles, with `debug_assertions`
/// on, so code only a build without them compiles, under `cfg(not(debug_assertions))`, is
/// code the proof never sees.
///
/// Why holding the GIL across one cannot deadlock: a deadlock needs the call to wait on
/// another thread that waits on the GIL. A leaf's only shared state is the handle's
Expand All @@ -1380,33 +1388,26 @@ impl<'a> Abi<'a> {
/// other threads queued of the thread's values that are not `Send`, by any `Drop`; a
/// handle lent from another borrows that one too, so no owner up its chain of lenders
/// may be other than `Send` either, and that chain is no longer than the export's objects
/// are many, as no cycle of lending methods reaches the object. A field
/// reached through `Deref`, of an enum, or of anything but a primitive would run user
/// code to read, clone, or drop. So it ends in a bounded number of steps whoever holds
/// the GIL, and since it runs no Python, nothing in it needs the GIL either.
/// are many, as no cycle of lending methods reaches the object. A body with no loop,
/// no recursion, and no call but to such bodies ends in a bounded number of steps whoever
/// holds the GIL, and since it runs no Python, nothing in it needs the GIL either.
pub(crate) fn leaf(&self, function: &Function, owner: Option<&Type>) -> bool {
let Some(Type::Object(i)) = owner else {
return false;
};
let primitive = |ty: &Type| {
matches!(
ty,
Type::Bool | Type::Int(_) | Type::NonZero(_) | Type::F32 | Type::F64 | Type::Char
)
};
let field = match function.access {
let shape = match function.access {
Access::Field { set: false, .. } => {
matches!(&function.ok, Type::Ref(_, ty) if primitive(ty)) && function.params.is_empty()
matches!(&function.ok, Type::Ref(_, ty) if ty.scalar()) && function.params.is_empty()
}
Access::Field { set: true, .. } => matches!(&function.params[..], [param] if param.ty.scalar()),
Access::Call => {
matches!(function.receiver, Some(Receiver::Ref | Receiver::Mut))
&& (function.ok.scalar() || matches!(function.ok, Type::Unit))
&& function.params.iter().all(|p| p.ty.scalar())
}
Access::Field { set: true, .. } => matches!(&function.params[..], [param] if primitive(&param.ty)),
_ => false,
};
field
&& !self.affine[*i]
&& !self.chained[*i]
&& function.derefs == 0
&& function.err.is_none()
&& !function.future
shape && function.leaf && !self.affine[*i] && !self.chained[*i] && function.err.is_none() && !function.future
}

/// Whether `ty`'s `<T>_eq`, or with `hash` its `<T>_hash`, is a leaf, as [`Self::leaf`]
Expand Down
Loading