diff --git a/bench/ffi/README.md b/bench/ffi/README.md index 360f6d6e5..ce1050e59 100644 --- a/bench/ffi/README.md +++ b/bench/ffi/README.md @@ -185,8 +185,8 @@ What they show: code in 16-byte steps moves the getter between 4.0 and 4.5 with a 64-byte period, and shifting the Rust export alone, by up to 128 bytes, moves it by no more than 0.1. The earlier count-only borrow, rebuilt on the same tree, costs 4.15. A Swift wrapper marked - `@inlinable`, so the loop calls the export directly, costs 3.5 at every shift, which - [a quest](/quest/m1/ffi-speed/shared-borrow.md) takes up. + `@inlinable`, so the loop calls the export directly, costs 3.5 at every shift, as + every generated wrapper now is: [Swift's inlined calls](#swifts-inlined-calls). The load limits these numbers: 4 threads beside others' leave the scheduler free to place them on one die or across both, whose line transfer is slower. Read the shared rows as @@ -194,6 +194,50 @@ their order of magnitude, against a handle each, at the same load. Rust's and Py columns are not remeasured: Rust borrows natively, and Python's threads serialize on the GIL, which dwarfs the handle's own contention. +## Swift's inlined calls + +Measured 2026-11-16 on **nixos**, as above, through `quest gate`, against moq `ffcb7cd9`. +Before is rust2 `8ae7404b`, whose wrappers a consumer calls into the Swift module; after, +every generated body is `@inlinable`, so the bench program's loop calls the C export +itself. Each cell is the median of 3 rounds, before and after alternating at each +placement, at a load average of 2.3 to 6.2: a `.text` pad of 0, 16, 32, or 48 bytes, +linked ahead of the Swift objects, shifts all code through the 64-byte period the getter's +cost followed. Nanoseconds per call, CPU first, then wall; C's getter costs 3.4 and its +handle call 6.4. + +| call | pad 0 | pad 16 | pad 32 | pad 48 | +|---|---|---|---|---| +| u64 getter, before | 4.84 / 4.87 | 4.46 / 4.48 | 4.38 / 4.40 | 4.71 / 4.73 | +| u64 getter, after | 3.39 / 3.40 | 3.39 / 3.42 | 3.38 / 3.39 | 3.33 / 3.34 | +| handle call, before | 10.08 / 10.12 | 10.11 / 10.16 | 9.95 / 10.02 | 9.97 / 10.02 | +| handle call, after | 9.87 / 9.92 | 9.85 / 9.91 | 9.90 / 9.95 | 9.79 / 9.84 | +| getter, 4 threads, a handle each, before | 9.84 / 10.49 | 8.75 / 9.99 | 8.21 / 9.11 | 9.32 / 10.09 | +| getter, 4 threads, a handle each, after | 7.24 / 7.88 | 6.99 / 7.71 | 6.71 / 7.19 | 6.92 / 7.93 | +| object, before | 26.52 / 26.61 | 26.55 / 26.68 | 25.68 / 25.77 | 25.91 / 25.99 | +| object, after | 27.62 / 27.74 | 30.22 / 30.34 | 27.72 / 27.84 | 26.68 / 26.78 | +| ready read, before | 458 / 460 | 461 / 463 | 452 / 454 | 449 / 451 | +| ready read, after | 443 / 445 | 445 / 447 | 436 / 438 | 440 / 442 | + +1. **The getter costs C's at every placement.** Its loop calls `moq_group_Demand_sequence` + directly, with no merged thunk between, so where the linker puts code no longer moves it. +2. **The handle call inlines too, but keeps its hold.** `TrackProducer` is a class, whose + call holds its handle with a compare-and-swap and lets go with an add of its own, about + 3.4 ns over C's. Its body first inlined only once a failed check's error, a handle's + free, and the release of a hold in the table moved out of line, and the handle's + constants became computed properties, which another module folds instead of reading + through an accessor. +3. **A call the optimizer declines to inline still goes through the thunk.** The program's + `track.info()`, whose object it then drops, calls the module's copy before and after, so + its cost still follows placement: 26.7 to 30.2 against 25.7 to 26.6. Most other rows + fell a little: the record by up to 2 ns at three placements, a write by 1.5 to 6, a + stamped write by 2.6 to 5.5, and the ready read, an `async` call, by 9 to 16. + Throughput is bimodal on either side, about 2,200 or 8,600 MB/s for 16 KiB frames in a + run, as the global executor's threads interleave. +4. **The cost is in the Swift module's build.** moq's module, 90,000 lines of Swift, takes + 81 to 89 s to build with `-O` against 66, its `.swiftmodule` 19.1 MB against 3.6, and its + library 11.6 to 11.8 MB against 11.5; the bench program's own build is unchanged, 1.2 to + 2.1 s. + ## Long-lived reader measurement Measured 2026-11-13 on **nixos**, an AMD Ryzen 9 5900XT (16 cores, 32 threads), after a @@ -442,7 +486,7 @@ frame path: (5,298), leaving Dart's own await, which schedules a microtask even for a complete future. Go's is the returned frame's ownership (`AddCleanup`, `Cleanup.Stop`, and a close's two crossings), and cgo's check and heap allocation of each out pointer, which - [Go's pointers](../../quest/m1/ffi-speed/go-pointers.md) takes: what remains after both + [kixel-ai/rust2#837](https://github.com/kixel-ai/rust2/pull/837) took: what remains after both is one crossing. Off the main thread, glibc's malloc arena for the thread grows and trims with `mprotect`, which adds about 60 ns to a write and 20 to a read in C on a pthread (`MALLOC_ARENA_MAX=1` removes it). Swift's ready read runs in a task on such a diff --git a/ffi/README.md b/ffi/README.md index 943264ceb..1e751dd2a 100644 --- a/ffi/README.md +++ b/ffi/README.md @@ -956,6 +956,16 @@ value of a Rust type crosses as one Swift type that holds it exactly, and back u returns wakes through Rust and drops once Swift releases it. - **The contract** is checked once, at the first call into the library: a Swift file and a library from different builds trap. +- **Every body may inline into its caller.** Each function, initializer, subscript, and + computed property is `@inlinable`, and what they name `@usableFromInline`, so a + consumer's optimizer in another module can inline a call and reach the C function + directly, with no flag of its own. Where it declines, the call goes to the module's + copy, which Swift may merge with like wrappers into one that calls the C function + through a pointer, whose cost follows where the linker places it. Their cold paths, a + failed check's error and a handle's free, stay out of line, so a call more often looks + worth inlining. The module's `.swiftmodule` holds the bodies, about 5 times its size + without them. + ## Go `go/` holds a Go package per crate whose items the facade exports, over the C header diff --git a/ffi/runtime/rt.swift b/ffi/runtime/rt.swift index b3ad11c3b..8acd61724 100644 --- a/ffi/runtime/rt.swift +++ b/ffi/runtime/rt.swift @@ -358,6 +358,11 @@ public struct CheckError: Swift.Error, CustomStringConvertible, Sendable { /// What broke. public let message: Swift.String + init(function: Swift.String, message: Swift.String) { + self.function = function + self.message = message + } + public var description: Swift.String { "\(function): \(message)" } } @@ -396,6 +401,8 @@ extension SeekableReader { extension Native { /// A seek past `Int64.max` of a reader with no `seek(start:)` of its own. struct SeekPast: Swift.Error, CustomStringConvertible { + init() {} + var description: Swift.String { "a seek past Int64.max of a reader with no seek(start:)" } } } @@ -814,17 +821,17 @@ extension Native { let free: (@convention(c) (OpaquePointer?) -> Void)? let type: StaticString - static let closed = 1 - static let bias = 2 - static let held = 4 + static var closed: Int { 1 } + static var bias: Int { 2 } + static var held: Int { 4 } /// Set in a slot whose hold a close counted. - static let revoked = 1 + static var revoked: Int { 1 } /// A cache line of rows to a column, which the thread's stack picks, and a row per /// handle, with a line between columns: two threads in one column, or two handles in /// one row on one thread, use the count. - static let rowBits = 3 - static let columnBits = 8 - static let stride = 2 << rowBits + static var rowBits: Int { 3 } + static var columnBits: Int { 8 } + static var stride: Int { 2 << rowBits } /// Only ever touched atomically. nonisolated(unsafe) static let slots: UnsafeMutablePointer = { let count = stride << columnBits @@ -853,11 +860,24 @@ extension Native { /// Hold it for a call, until `release` takes what this returns. Traps once closed. func hold() -> Int { guard let slot = held() else { - fatalError("\(type) is closed") + closed() } return slot } + /// Trap for a call of a closed handle: out of line, so a caller's inlined call stays + /// small enough to inline. + @inline(never) + func closed() -> Never { + fatalError("\(type) is closed") + } + + /// The error a call of `what` throws once it is closed, out of line as `closed` is. + @inline(never) + func closed(_ what: StaticString) -> CheckError { + CheckError(function: "\(what)", message: "\(type) is closed") + } + /// Hold it for a call, until `release` takes what this returns, unless it is closed. @inline(__always) func held() -> Int? { @@ -927,14 +947,27 @@ extension Native { @inline(__always) func release(_ slot: Int) { - if slot >= 0 && exchange(Handle.slots + slot, 0) == key { - return + if slot >= 0 { + release(listed: slot) + } else if add(state, -Handle.held) < Handle.held { + freed() } - if add(state, -Handle.held) < Handle.held { - free?(raw) + } + + /// `release` of a hold in `slots`, out of line, as `closed` is. + @inline(never) + func release(listed slot: Int) { + if exchange(Handle.slots + slot, 0) != key && add(state, -Handle.held) < Handle.held { + freed() } } + /// Free it once its last hold lets go: out of line, as `closed` is. + @inline(never) + func freed() { + free?(raw) + } + /// Run `body` with the raw handle, held until it returns. @inline(__always) func with(_ body: (OpaquePointer?) throws -> R) rethrows -> R { @@ -948,7 +981,7 @@ extension Native { @inline(__always) func with(_ what: StaticString, _ body: (OpaquePointer?) throws -> R) throws -> R { guard let slot = held() else { - throw CheckError(function: "\(what)", message: "\(type) is closed") + throw closed(what) } defer { release(slot) } return try body(raw) @@ -1104,7 +1137,7 @@ extension Native { /// a throw for the call if it is closed. func held(_ handle: Handle) throws -> OpaquePointer? { guard let slot = handle.held() else { - throw CheckError(function: "\(what)", message: "\(handle.type) is closed") + throw handle.closed(what) } held.append((handle, slot)) return handle.raw @@ -1307,9 +1340,9 @@ extension Native { /// What a future parks on: it latches a wake that lands between a poll that pends and /// the park, so the next poll sees it. final class Signal: @unchecked Sendable { - static let idle = 0 - static let woken = 1 - static let parked = 2 + static var idle: Int { 0 } + static var woken: Int { 1 } + static var parked: Int { 2 } /// Which of those it is, changed only atomically. private let state = UnsafeMutablePointer.allocate(capacity: 1) diff --git a/ffi/src/swift.rs b/ffi/src/swift.rs index bdcccaf28..fe5936c82 100644 --- a/ffi/src/swift.rs +++ b/ffi/src/swift.rs @@ -102,7 +102,10 @@ pub fn generate(abi: &Abi<'_>, packages: &str, contract: u64) -> Result> target) & 1 == 0 {{\n\t\t\tfatalError(\"\\(what): it is not on this library's target\")\n\t\t}}\n\t}}\n\n\ \t/// Throws what `status`, written by a call of `what`, says it failed, if anything.\n\ - \t@inline(__always)\n\tstatic func status(_ status: Swift.Int32, _ what: Swift.StaticString) throws {{\n\t\tif status != 0 {{\n\t\t\tthrow CheckError(function: \"\\(what)\", message: Swift.String(cString: {p}_status_message(status)))\n\t\t}}\n\t}}\n\n\ + \t@inline(__always)\n\tstatic func status(_ status: Swift.Int32, _ what: Swift.StaticString) throws {{\n\t\tif status != 0 {{\n\t\t\tthrow failure(status, what)\n\t\t}}\n\t}}\n\n\ + \t/// The error `status` says a call of `what` failed with: out of line, so each call that\n\ + \t/// checks one stays small enough for its caller to inline.\n\ + \t@inline(never)\n\tstatic func failure(_ status: Swift.Int32, _ what: Swift.StaticString) -> CheckError {{\n\t\tCheckError(function: \"\\(what)\", message: Swift.String(cString: {p}_status_message(status)))\n\t}}\n\n\ \tstatic func free(_ buf: Buf) {{\n\t\t{p}_buf_free(buf)\n\t}}\n\n\ \t/// A buffer Rust owns, holding a copy of what `view` lends.\n\ \tstatic func copy(_ view: Buf) -> Buf {{\n\t\t{p}_buf_copy(view)\n\t}}\n\n\ @@ -130,6 +133,7 @@ pub fn generate(abi: &Abi<'_>, packages: &str, contract: u64) -> Result Gen<'a> { let mut body = vec![match unique { // A lend ends with `_end`, which leaves Rust the handle it lent to free. true => format!( - "\tlet handle: OpaquePointer?\n\n\tinit(raw: OpaquePointer?) {{\n\t\thandle = raw\n\t}}\n\n\tdeinit {{\n\t\t{c}_free(handle)\n\t}}\n\n\tconsuming func _end() {{\n\t\tdiscard self\n\t}}\n" + "\tlet handle: OpaquePointer?\n\n\tinit(raw: OpaquePointer?) {{\n\t\thandle = raw\n\t}}\n\n\tdeinit {{\n\t\t{c}_free(handle)\n\t}}\n\n\t@usableFromInline consuming func _end() {{\n\t\tdiscard self\n\t}}\n" ), false => format!( "\tlet handle: Native.Handle\n\n\tinit(raw: OpaquePointer?) {{\n\t\thandle = Native.Handle(raw, {c}_free, \"{name}\")\n\t}}\n\n\tinit(lent raw: OpaquePointer?) {{\n\t\thandle = Native.Handle(raw, nil, \"{name}\")\n\t}}\n\n\tdeinit {{\n\t\thandle.drop()\n\t}}\n" @@ -2497,6 +2501,114 @@ fn kw(name: &str) -> String { } } +/// `swift` with every body another module may inline: each function, initializer, +/// subscript, and computed property of a type or the file `@inlinable`, and each other +/// declaration they could name `@usableFromInline` unless public, `private` ones made +/// internal. So a consumer's call of a wrapper compiles to a direct call of its C function, +/// not a call into this module, whose cost would follow where the linker places its code. +/// +/// It reads the generated file a line at a time, as the generator writes it: one +/// declaration a line, each brace that opens a body ending its line. +fn inlinable(swift: &str) -> String { + /// What a brace opened: a type's members, or code. + #[derive(PartialEq)] + enum Scope { + Members, + Protocol, + Code, + } + const ACCESS: &[&str] = &["public ", "private ", "fileprivate "]; + const MODIFIERS: &[&str] = &[ + "static ", + "mutating ", + "consuming ", + "final ", + "nonisolated ", + "nonisolated(unsafe) ", + "nonisolated(nonsending) ", + ]; + let mut scopes: Vec = Vec::new(); + let mut out = String::with_capacity(swift.len() * 11 / 10); + for line in swift.lines() { + let code = line.trim_start_matches('\t'); + let tabs = &line[..line.len() - code.len()]; + let mut opened = Scope::Code; + let mut written = None; + if !code.starts_with("//") && scopes.iter().all(|s| *s == Scope::Members) { + // Attributes on the line, then its access and modifiers, then its keyword. + let mut rest = code; + let mut attributes = String::new(); + while rest.starts_with('@') { + let end = rest.find(' ').unwrap_or(rest.len()); + attributes.push_str(&rest[..end]); + attributes.push(' '); + rest = rest[end..].trim_start(); + } + // Access and modifiers come in either order, as `nonisolated(nonsending) public`, + // and keep it, but for a `private` or `fileprivate`, which an inlined body could + // not name. + let mut public = false; + let mut modifiers = String::new(); + loop { + if let Some(a) = ACCESS.iter().find(|a| rest.starts_with(**a)) { + if *a == "public " { + public = true; + modifiers.push_str(a); + } + rest = &rest[a.len()..]; + } else if let Some(m) = MODIFIERS.iter().find(|m| rest.starts_with(**m)) { + modifiers.push_str(m); + rest = &rest[m.len()..]; + } else { + break; + } + } + let keyword = rest.split([' ', '(', '<', '?']).next().unwrap_or(""); + // A property is computed when its line opens a body before any `=`. + let computed = match (rest.find('{'), rest.find('=')) { + (Some(brace), Some(assign)) => brace < assign, + (brace, _) => brace.is_some(), + }; + let attribute = match keyword { + "func" | "init" | "subscript" => Some("@inlinable "), + "var" | "let" if computed => Some("@inlinable "), + "var" | "let" | "struct" | "class" | "enum" | "typealias" if !public => Some("@usableFromInline "), + _ => None, + }; + opened = match keyword { + "struct" | "class" | "enum" | "extension" => Scope::Members, + "protocol" => Scope::Protocol, + _ => Scope::Code, + }; + // A declaration the generator marks itself keeps its mark. + let marked = attributes.contains("@inlinable") || attributes.contains("@usableFromInline"); + if let Some(attribute) = attribute.filter(|_| !marked) { + written = Some(format!("{tabs}{attributes}{attribute}{modifiers}{rest}")); + } + } + let opens = line.matches('{').count(); + let closes = line.matches('}').count(); + if !code.starts_with("//") { + match opens.cmp(&closes) { + std::cmp::Ordering::Greater => { + scopes.push(opened); + scopes.extend((1..opens - closes).map(|_| Scope::Code)); + } + std::cmp::Ordering::Less => { + for _ in 0..closes - opens { + scopes.pop().expect("the generated Swift's braces balance"); + } + } + std::cmp::Ordering::Equal => {} + } + } + out.push_str(written.as_deref().unwrap_or(line)); + out.push('\n'); + } + assert!(scopes.is_empty(), "the generated Swift's braces balance"); + out +} + /// Swift doc comments holding `docs`, each line indented by `indent`, or nothing for none. fn docs(docs: &str, indent: &str) -> String { docs.lines() @@ -2532,7 +2644,105 @@ fn conform(protocols: &[&str]) -> String { #[cfg(test)] mod tests { - use super::{RUNTIME, TAKEN}; + use super::{RUNTIME, TAKEN, inlinable}; + + /// Each member a body may name crosses: functions and computed properties inline, + /// internal storage and types are usable from them, and code, protocols, public + /// storage, and what the generator marks itself stay as written. + #[test] + fn inlinable_marks_members() { + let swift = "\ +public struct A: ~Copyable { + let handle: OpaquePointer? + public let name: Swift.String + private var count = 0 + var size: Swift.Int { 1 } + public var bytes: Swift.Int { + let n = size + return n + } + + init(raw: OpaquePointer?) { + handle = raw + } + + deinit { + free(handle) + } + + @usableFromInline consuming func _end() { + discard self + } + + /// A call {in braces. + nonisolated(nonsending) public func get() async throws -> Swift.Int { + let x = { () -> Int in 1 }() + return x + } +} + +public protocol P: Sendable { + func call() -> Swift.Int +} + +enum Native { + struct Handle { + static let slots: UnsafeMutablePointer = { + let count = 1 + return .allocate(capacity: count) + }() + } +} + +public func top() {} +"; + let want = "\ +public struct A: ~Copyable { + @usableFromInline let handle: OpaquePointer? + public let name: Swift.String + @usableFromInline var count = 0 + @inlinable var size: Swift.Int { 1 } + @inlinable public var bytes: Swift.Int { + let n = size + return n + } + + @inlinable init(raw: OpaquePointer?) { + handle = raw + } + + deinit { + free(handle) + } + + @usableFromInline consuming func _end() { + discard self + } + + /// A call {in braces. + @inlinable nonisolated(nonsending) public func get() async throws -> Swift.Int { + let x = { () -> Int in 1 }() + return x + } +} + +public protocol P: Sendable { + func call() -> Swift.Int +} + +@usableFromInline enum Native { + @usableFromInline struct Handle { + @usableFromInline static let slots: UnsafeMutablePointer = { + let count = 1 + return .allocate(capacity: count) + }() + } +} + +@inlinable public func top() {} +"; + assert_eq!(inlinable(swift), want); + } /// Each type the runtime declares at its top level, and each protocol one conforms to, /// is a name no export may take, as either would clash with or shadow it. diff --git a/quest/m1/README.md b/quest/m1/README.md index 1aeccd499..bad03c3d0 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -19,5 +19,5 @@ since moq never builds on it and its only consumers here were the binding benchm baseline and nightly jobs. Rejected: keeping it until moq switches; keeping it as is. Decided with the maintainer on 2026-11-12, in the quest audit: ✅ exporter work that -changes no generated API is [its own epic](/quest/m1/ffi-speed/README.md), so it does not -hold moq's pin open. Rejected: keeping it in the exporter's epic. +changes no generated API is its own epic, so it does not +hold moq's pin open; the epic retired on 2026-11-16 once its children landed. Rejected: keeping it in the exporter's epic. diff --git a/quest/m1/ffi-export/README.md b/quest/m1/ffi-export/README.md index 216ffd500..8f5bdfa4c 100644 --- a/quest/m1/ffi-export/README.md +++ b/quest/m1/ffi-export/README.md @@ -85,7 +85,8 @@ Decided with the maintainer on 2026-11-12, in the quest audit: before moq pins: the fused async start, owned buffers ([kixel-ai/rust2#744](https://github.com/kixel-ai/rust2/pull/744)), and numeric slices ([kixel-ai/rust2#743](https://github.com/kixel-ai/rust2/pull/743)). Work that changes no generated API is - [its own epic](/quest/m1/ffi-speed/README.md), which does not hold the pin. + its own epic, which did not hold the pin and was retired on 2026-11-16 once its children + landed. - Not planned: one walker emitting lift and lower instructions every generator implements, as wit-bindgen's `Bindgen` does. A new rule touches 19 to 38 files today, but it would collide with the four `[L]` children that each rewrite every generator. diff --git a/quest/m1/ffi-speed/README.md b/quest/m1/ffi-speed/README.md deleted file mode 100644 index 3f5d269ce..000000000 --- a/quest/m1/ffi-speed/README.md +++ /dev/null @@ -1,28 +0,0 @@ -# Exporter speed and proof, with no API change - -## Goal - -[rust2-ffi](/ffi/README.md)'s bindings cost less and are proven harder, in ways that change -no generated API: each child cuts a cost [the binding benchmark](/bench/ffi/README.md) or -a measurement in its own Goal shows, measures a cost no benchmark shows yet, or adds a proof -the hand-written fixtures cannot give. -Nothing here holds [moq's pin](/quest/m1/ffi-export/ship.md) open. - -## Plan - -Decided with the maintainer on 2026-11-12, in the quest audit: ✅ this epic, beside -[the exporter epic](/quest/m1/ffi-export/README.md), which keeps only what breaks the C -ABI or a generated API. Kotlin records, -returned-object ownership, and the woken read -moved here from it. Rejected: keeping them there, where three quests that change no API held -the pin open. - -Each child's PR shows its change on the benchmark where it has a row through CodSpeed's -comment on the PR, as [the benchmark's README](/bench/ffi/README.md#codspeed) says, and -copies numbers by hand only for what CodSpeed does not count: wall and CPU time, the -contention cases, a language that left CodSpeed, and a measurement of its own. - -Decided with the maintainer on 2026-11-12, in `/quest-plan`: ✅ the benchmark's own -measurement work joins this epic, as CodSpeed, -the export's cost (since done), and concurrent calls, since they serve its PRs. -Rejected: standalone m1 quests; holding the last two in m3. diff --git a/quest/m1/ffi-speed/shared-borrow.md b/quest/m1/ffi-speed/shared-borrow.md deleted file mode 100644 index b27e9d6b0..000000000 --- a/quest/m1/ffi-speed/shared-borrow.md +++ /dev/null @@ -1,32 +0,0 @@ -# [M] Swift's calls reach the C export directly - -## Goal - -A Swift caller of a generated method calls its C export directly, not through a thunk, so -Swift's u64 getter on one thread costs what C's does (3.4 ns in -[the binding benchmark](/bench/ffi/README.md#concurrent-calls), against 4.0 to 4.6 now) -wherever the linker places the code. The generated API stays the same. - -## Plan - -- Found on 2026-11-15, chasing the getter's rise from 3.8 to 4.6 ns after the - reader-biased borrow: the borrow is at most 0.2 ns of it. Each wrapper calls one merged thunk - (`$s3Moq4PathV3lenSuyKFTm`, from Swift's function merging) that takes the C function as - a pointer and calls it indirectly, and the getter's cost follows that code's 64-byte - placement: 4.0 to 4.5 ns as all code shifts 16 bytes at a time, flat as the Rust export - alone shifts. The earlier count-only borrow costs 4.15 on the same tree, and packing the - bias into the count's word, or threading the ticket out of the fast path, moved nothing. -- Hand-marking `GroupDemand.sequence()` `@inlinable` in the generated Swift, with - `@usableFromInline` on `handle`, `Native`, and `Native.status`, made the bench loop call - `moq_group_Demand_sequence` directly: 3.5 ns at every 16-byte shift of 0 to 48. -- Recommended: generate every wrapper `@inlinable`, and `@usableFromInline` on each - internal declaration an inlined body names: handles, `Native` and its helpers, the - runtime's hold counts in `ffi/runtime/rt.swift`. A consumer already imports the C module - the wrappers call, so an inlined body compiles in its module. Measure the handle call's - row too (10 ns against C's 6.4), whose class wrapper goes through the same thunk, and - report the Swift module's size and compile time beside the rows. -- Rejected: aligning Swift's functions, which takes a compiler flag from each consumer, and - a build setting that turns off function merging, for the same reason. -- Prove it with the single-thread rows, CPU first, at the load beside them, at each - placement in the 64-byte period: a `.text` pad of 0, 16, 32, and 48 bytes linked ahead - of the Swift objects.