Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 41 additions & 6 deletions bench/ffi/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -475,9 +475,11 @@ frame path:
Go's object return fell from 254 to 174 and Dart's from 225 to 96, Go's ready read from
833 to 728 and Dart's from 1,020 to 786, and Dart's woken read from 2,930 to 2,274.
What Go has left is mostly its two cgo crossings, the call and the close, and the pool
that finds its P's nursery. Kotlin's
137 ns wall (210 CPU) is a phantom reference linked and unlinked under one global lock.
Every returned frame, group, and future pays it.
that finds its P's nursery. Since kixel-ai/rust2#880, each Kotlin thread holds one too, and
a close no longer links and unlinks a phantom reference under a lock: Kotlin's object return
fell from 124 to 118 ns median wall (from 112 to 94 at best) and from 56 to 38 on 4
threads at once. What it has left is mostly a checked close's two JNI crossings, its drop
and then its free ([TypeScript](#the-nursery) has the web's).
3. **A woken read.** C's 427 ns is Rust's 329 plus a future's crossings. Kotlin's 1.6 us and
Dart's 1.8 us add their event loops; Go's 5.1 us and Swift's 5.7 us wall (16.6 us CPU)
are a goroutine or `Task` and a cross-thread wake per read, and Python's 6.9 us is
Expand Down Expand Up @@ -648,8 +650,37 @@ profile), Dart 3.13.0. C and Dart are from the same run, for scale; wall ns per
| 64 KiB | 10,312 | 6,871 | 12,228 | 11,070 | 7,218 |
| 1 MiB | 8,912 | 8,164 | 16,127 | 16,986 | 12,071 |

An object return pays the handle's `FinalizationRegistry` entry, as Go's and Dart's pay
their collectors'. Firefox runs every row at about 2 to 3 times Node's.
An object return paid the handle's `FinalizationRegistry` entry, as Go's and Dart's paid
their collectors', until [the nursery](#the-nursery). Firefox runs every row at about 2 to
3 times Node's.

### The nursery

Since kixel-ai/rust2#880, the last 64 objects and lent bytes the module returned wait in a
nursery, where disposing one frees it without a `FinalizationRegistry` entry, and only one
pushed out undisposed registers, as Go's, Dart's, and Kotlin's objects do. Measured
2026-11-19 on **fedora**, 5 rounds alternating with main at `52070b3d`, at a load average of
16 to 36; median wall ns, main then this, beside Kotlin's 8 rounds at a load of 17 to 25.
Kotlin lends only from 1 KiB, so its 100 B payload is a copy either way:

| call | Kotlin | Node | Chromium | Firefox |
|---|---|---|---|---|
| `track.info()`, an object return | 124 / 118 | 357 / 181 | 255 / 177 | 908 / 239 |
| `frame.payload`, 100 B | 67 / 66 | 415 / 212 | 244 / 209 | 1,008 / 227 |
| a 100 B frame written and read, ready | 676 / 625 | 1,258 / 998 | 1,099 / 1,129 | 2,595 / 1,534 |
| the same, read after a wake | 1,867 / 1,803 | 2,246 / 1,932 | 2,493 / 2,714 | 4,162 / 3,056 |

100 B throughput rose from 57 to 78 MB/s on Node and from 27 to 56 on Firefox. Chromium's
other rows moved both ways by up to a quarter between rounds at this load, so read only its
object and payload rows. A lent payload now costs 205 to 255 ns at every length on all three
engines (see below).

An object dropped undisposed in a nursery no later return pushes it out of waits for the
nursery's flush: a sentinel in a registry of its own, which a collection finds and then
registers all the nursery holds, as a ring the GC drops does in Kotlin. In the api fixture's
tests, a forced full collection at a time, such an object was freed by the second, on Node
and the JVM alike, in every run, against the first on main; one pushed out by later returns
was freed by the first, as before.

### The export's build cost

Expand Down Expand Up @@ -699,7 +730,11 @@ its crossover is near 4 KiB; JavaScriptCore's, measured in Bun 1.4.4, is between
threshold is one constant for every engine: 64 B costs Firefox about 350 to 400 ns a
payload from 100 B to 1 KiB, and any higher one would cost V8 more at each length below it.
The bytes up to it now pay about 50 ns on Node and Firefox for the `rt.Bytes` around their
copy, against a bare `Uint8Array` before.
copy, against a bare `Uint8Array` before. Since [the nursery](#the-nursery), a lend disposed
skips its `FinalizationRegistry` entry and costs 205 to 255 ns on all three engines (Node 207
at 100 B against 421, Firefox 246 against 913), so SpiderMonkey's crossover falls from near
4 KiB to between 64 and 100 B, and the one threshold suits V8 and SpiderMonkey alike;
JavaScriptCore's was not remeasured.

A lend needs a dispose: the collector cannot see wasm memory, so it never hurries to free
what it lends. In a probe of a million 100 B lends left to the collector on Node, all
Expand Down
29 changes: 21 additions & 8 deletions ffi/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -598,8 +598,13 @@ the rest cross encoded.
`close()` throws. Of a checked type, `close()` drops the value through Rust's own
`_drop`, as Rust's `drop` would, and throws instead while a call borrows the value, as
Rust refuses that drop, leaving the value to a later `close()`. Of any other type,
`close()` frees the handle, and throws while a call holds it. One never closed is freed
once the GC collects it, on a daemon thread a phantom reference queue feeds. A `self`
`close()` frees the handle, and throws while a call holds it. A returned object waits in
a nursery of the last 64 its thread returned, where `close()` frees it without the GC's
registration, a phantom reference linked into a list under a lock. One never closed is
registered once pushed out, by 64 later returns on its thread or with its nursery, which
the thread holds only weakly, so a collection reaching it while it is not returning, as a
full one does, drops it; then
the GC frees it once it collects it, on a daemon thread a phantom reference queue feeds. A `self`
method, or passing the object by value, moves its value out and closes it once the call
returns. `Clone` is `clone()`, `PartialEq` is `equals`, and `Hash` is `hashCode`, Rust's
hash folded as `Long.hashCode` folds it, or 0 without one. A value that is not `Send`
Expand Down Expand Up @@ -1376,7 +1381,8 @@ target there.
caller holds; up to 64 it holds a copy, since an engine keeps a typed array that short
in its own heap, which costs less than a lend. Disposing it, as `using` does, frees Rust's
buffer once no call reading it in place is running, and a read after throws `Bytes is
closed`, whichever it holds. One never disposed waits for the `FinalizationRegistry`,
closed`, whichever it holds. Lent bytes wait in the nursery objects do (below), and
one never disposed then waits for the `FinalizationRegistry`,
which runs only once the collector does for its own heap, which no lent buffer grows: so
past a quarter of memory lent and not yet freed, or once memory has grown by a third
since none were, a call copies instead, so bytes nobody disposes grow memory by no more
Expand All @@ -1388,16 +1394,21 @@ target there.
`Uint8Array`, since only a caller holding them alone disposes them. No view of Rust's buffer reaches a caller, as a `Uint8Array` over memory
would: it detaches once memory grows, writes what Rust's other owners of the bytes read,
and through a `subarray` or its `buffer` outlives the buffer it views. On Node 24, a lent
payload costs 0.4 µs a call at any length, against a copy's 0.5 µs at 100 B, 1.1 µs at
payload costs 0.2 µs a call at any length, against a copy's 0.5 µs at 100 B, 1.1 µs at
1 KiB, and 97 µs at 1 MiB; [the benchmark](../bench/ffi/README.md#lending-returned-bytes)
has each engine's.
- **Objects** are an `rt.Handle`, disposed by `[Symbol.dispose]`, as `using` does. Of a
checked type, disposing it drops the value through Rust's own `_drop`, which fails while
a call borrows it; of any other, it throws while a call holds it. The handle is freed once
disposed and no call holds it, so one disposed while a later argument lowers fails the
call's check rather than reach Rust freed. A call that meets a disposed handle throws
`<Type> is closed`, and a `FinalizationRegistry` frees one never disposed. On Node 24, a returned object disposed
costs 0.5 µs, 0.2 µs of it the registry's, against 0.1 µs for the C calls alone.
`<Type> is closed`. A returned object waits in a nursery of the last 64 objects and lent
bytes returned, where disposing it frees it without a `FinalizationRegistry` entry, which
costs more than the rest of a return and its disposal. One never disposed registers once
pushed out, by 64 later returns or once a collection finds the nursery's sentinel, which
then pushes out all it holds, and the registry frees it once the collector finds it
unreachable after. On Node 24, a returned object disposed costs 0.18 µs, against 0.36
µs with the registry entry and 0.1 µs for the C calls alone.
- **Traits** are interfaces JavaScript implements, with an optional `close()` Rust calls
once it drops an implementation, and **closures** are functions: each is a C function
the module imports from `rust2`, whose data is a token the binding finds the value by, so
Expand Down Expand Up @@ -1444,8 +1455,10 @@ at 17 and 23, builds it under Clang's `-Wconsumed -Wdangling -Werror`, each case
wake and its poll that loses nothing, and a coroutine resumed or destroyed. `tests/api.kt`
runs the script through the Kotlin on the JVM, checks its futures the same way, that a call
after `close()` throws, that `Bytes` and their slices keep Rust's buffer, past the object or
record they came from, until the GC frees it, and that calls racing `close()` or the GC
read live values, and breaks each runtime check, which must throw or abort with its message. `just ffi android` runs
record they came from, until the GC frees it, that calls racing `close()` or the GC
read live values, that the GC frees an object never closed left in an idle nursery or
pushed out of one, and that each object of many threads' returns, closes, and drops is
freed once, and breaks each runtime check, which must throw or abort with its message. `just ffi android` runs
the same on ART: the Kotlin and `tests/api.kt`, shrunk by R8 into an app's dex with only the
generated rules keeping what JNI looks up, against the library built by the NDK for the
device's ABI, which ANDROID_HOME and ANDROID_NDK_HOME locate. Nightly runs it on an
Expand Down
152 changes: 144 additions & 8 deletions ffi/runtime/rt.kt
Original file line number Diff line number Diff line change
Expand Up @@ -472,15 +472,16 @@ abstract class __Cleanup(owner: Any?) : java.lang.ref.PhantomReference<Any>(owne
* once no call does, and a call after [close] throws. [close] of one whose value Rust
* checks, which [drop] drops, drops the value as Rust's own drop does, throwing while a
* call borrows it in Rust; of any other, it throws while a call holds it. An owned one is
* freed once the GC collects [owner], if nothing closed it.
* freed once the GC collects [owner], if nothing closed it: it starts in its thread's
* [__Nursery], and registers with the GC only once it leaves that open.
*/
class __Handle(
owner: Any,
@PublishedApi internal val raw: Long,
private val free: __Drop?,
private val type: String,
private val drop: __Close? = null,
) : __Cleanup(if (free == null) null else owner) {
) {
// The holds, one for the owner until closed and one per call, shifted past a bit set
// once closed, one set while an unchecked [close] decides, and one set once calls
// contended for them, after which they hold it in [SLOTS]: one word, so a call reads
Expand All @@ -492,8 +493,23 @@ class __Handle(
// even while a closed handle is still reachable.
@kotlin.concurrent.Volatile @kotlin.jvm.JvmField internal var key = 0L

// What frees it once the GC collects its owner, from when it leaves its nursery open.
// Plain: a free that misses an eviction's write leaves a cleanup that finds it closed.
private var cleanup: __Collected? = null

init {
if (free != null) track()
if (free != null) __Nursery.push(owner, this)
}

/**
* Register it with the GC as it leaves its nursery, unless closed, when its last hold
* frees it. Racing that close, it may register one closed meanwhile, whose cleanup then
* finds it closed and frees nothing: [CLOSED], claimed once, decides the one free.
*/
internal fun evict(owner: Any) {
// An unchecked close still deciding may leave it open.
if (state and (CLOSED or CLOSING) == CLOSED) return
cleanup = __Collected(owner, this)
}

/** Hold it for a call, until [release] takes what this returns. Throws once closed. */
Expand Down Expand Up @@ -633,14 +649,14 @@ class __Handle(
release(-1)
}

// Free it once no hold is left.
// Free it once no hold is left, which happens once.
private fun freed(n: Long) {
if (n < HELD && free != null && untrack()) free.drop(raw)
if (n < HELD && free != null) {
cleanup?.stop()
free.drop(raw)
}
}

// A call still holding it frees it once it returns.
override fun collected() = abandon()

/** Run [block] with the raw handle, held until it returns. */
inline fun <R> hold(block: (Long) -> R): R {
val slot = hold()
Expand Down Expand Up @@ -681,6 +697,126 @@ class __Handle(
}
}

/** What frees an owned handle once the GC collects [owner]: a call still holding it frees it once it returns. */
private class __Collected(owner: Any, private val handle: __Handle) : __Cleanup(owner) {
init {
track()
}

override fun collected() {
untrack()
handle.abandon()
}

/** Stop it, as the handle is freed first. */
fun stop() {
untrack()
}
}

/**
* The latest owned handles a thread was returned, each beside its owner, which a free
* leaves without the GC's registration, a phantom reference linked under a lock, which
* costs more than the rest of a return and its close together. One that leaves it open,
* pushed out by later returns or with its nursery once the GC drops that, registers then.
*/
private class __Nursery {
// Kept reachable while here, so an eviction can register them. Only the thread
// whose [__Ring] reaches it touches these, until [flush], once nothing reaches that.
@kotlin.jvm.JvmField val owners = arrayOfNulls<Any>(LEN)
@kotlin.jvm.JvmField val handles = arrayOfNulls<__Handle>(LEN)

/**
* Register each handle still open, once the GC collected the ring that reached this,
* and let go of them all, so none waits on the others to be collected. The collector
* stopped the thread that filled them after its last push, which orders its writes
* before this.
*/
fun flush() {
for (i in 0 until LEN) {
handles[i]?.evict(owners[i]!!)
handles[i] = null
owners[i] = null
}
}

companion object {
/**
* How many of the latest objects returned a nursery holds: an object dropped
* unclosed is freed only once this many more were returned on its thread, or the
* GC dropped its nursery, so it bounds what a dropped object keeps of Rust's.
*/
const val LEN = 64

// Each thread's ring, held weakly, which a collection that reaches it while the
// thread is not pushing drops: so an idle thread's nursery keeps what it holds only
// until such a collection and one more, not forever.
private val RINGS = java.lang.ThreadLocal<java.lang.ref.WeakReference<__Ring>>()

/** Put [handle], which [owner] owns, in this thread's nursery, oldest out. */
fun push(owner: Any, handle: __Handle) {
var ring = RINGS.get()?.get()
if (ring == null) {
ring = __Ring(__Nursery())
__Flush(ring)
RINGS.set(java.lang.ref.WeakReference(ring))
}
ring.push(owner, handle)
}
}
}

/** What a thread holds of its nursery, only weakly, so the GC drops an idle one. */
private class __Ring(@kotlin.jvm.JvmField val nursery: __Nursery) {
// The oldest slot once full.
private var next = 0

fun push(owner: Any, handle: __Handle) {
val n = nursery
val i = next
n.handles[i]?.evict(n.owners[i]!!)
n.owners[i] = owner
n.handles[i] = handle
next = (i + 1) and (__Nursery.LEN - 1)
// Its flush must not run while this pushes, and no plain store keeps it reachable.
__Keep.fence(this)
}
}

/** Keeps an object reachable up to here, by the strongest means the platform has. */
private object __Keep {
// Decided once, so the compiler folds the branch away.
@kotlin.jvm.JvmField val REFERENCE = try {
java.lang.ref.Reference::class.java.getMethod("reachabilityFence", Any::class.java)
true
} catch (e: NoSuchMethodException) {
false
}

/**
* `Reference.reachabilityFence` where the platform has it, free on HotSpot, which
* Android adds at API 28. Elsewhere a monitor held on [o], which keeps [o] a root while
* held, and which no earlier write is ordered after.
*/
fun fence(o: Any) {
if (REFERENCE) java.lang.ref.Reference.reachabilityFence(o) else synchronized(o) {}
}
}

/** What flushes a ring's nursery once the GC collects the ring. */
private class __Flush(ring: __Ring) : __Cleanup(ring) {
private val nursery = ring.nursery

init {
track()
}

override fun collected() {
untrack()
nursery.flush()
}
}

/** A waker Rust returned: [wake] wakes it through Rust until [close], or the GC, drops it. */
class __Returned(raw: Long) : Waker, AutoCloseable {
private val handle = __Handle(this, raw, __Drop { __waker_drop(it) }, "Waker")
Expand Down
Loading