diff --git a/ffi/README.md b/ffi/README.md index c951b9a45..22f751707 100644 --- a/ffi/README.md +++ b/ffi/README.md @@ -299,7 +299,9 @@ where it was while its file is unchanged. fails with `_STATUS_NOT_SEND` before any Rust code runs, so such a future's poll takes a trailing `status` too. A `_free` or `_drop` elsewhere drops nothing there: it queues the drop to the value's thread, which runs it at its next check of one of its - values or as it exits. One freed once its thread exited leaks, printing why, since + values or as it exits. A thread running a loop that may call nothing for a while + passes `_thread_wake` a waker, which such a free wakes so the loop schedules + `_thread_drain` there. One freed once its thread exited leaks, printing why, since nothing may drop it. A function that makes such a value, taking none or only in an `Option`, has a twin `_homed` of the same signature, as do its future's start, `_start_homed`, and `Default`'s, `_default_homed`: with no such value passed, @@ -471,7 +473,9 @@ packages place it in. - **Checks**: a call passes no `status`, so a borrow or move Rust's rules refuse aborts with its message, naming the function, as C's does with a NULL one: so does a use or poll of a value that is not `Send` on another thread than the one that made it, and - one destroyed on another drops on its own thread at its next call or as it exits. + one destroyed on another drops on its own thread: a job posted to the executor of the + thread's latest `then` or `via` drops it, while that task lives and if the job runs on + the thread, else its next call or its exit does. - **Sharing** stands in for `clone` where no call can tell the two apart but by counting an `Arc`'s or `Rc`'s clones, which the exporter decides at compile time. A type's copies share when it is `Clone`; `Sync`, since one that is not is borrowed on one thread at a time, which copies on two threads @@ -730,7 +734,9 @@ caller imports it with no ignores. borrows from, as for a closed object, and `RuntimeError` for any other, as for a value that is not `Send` used on another thread than the one that made it. Freed on another, as the collector or the last reference there frees it, such a value drops on its own - thread at its next call or as it exits. + thread: on the next turn of the event loop that ran there when a call made a value that + is not `Send`, though the loop calls nothing into Rust, else at its next call or as it + exits. - **Records** are slotted dataclasses, compared by their fields, a float by IEEE `==` even against itself, as Rust does, and **enums** an `Enum` of upper-case members, or for one with data a class of a dataclass per variant, diff --git a/ffi/runtime/native.py b/ffi/runtime/native.py index d20787fc0..cb46b0a99 100644 --- a/ffi/runtime/native.py +++ b/ffi/runtime/native.py @@ -28,6 +28,8 @@ def fileno(self) -> _int: ... def take(self, read: _bool, /) -> _list[_int]: ... + def drain(self) -> None: ... + class Node: def __init__(self, queue: _n.Queue, /) -> None: ... diff --git a/ffi/runtime/python.rs b/ffi/runtime/python.rs index edbe578f3..06a8a9239 100644 --- a/ffi/runtime/python.rs +++ b/ffi/runtime/python.rs @@ -243,12 +243,68 @@ pub fn call(f: impl FnOnce() -> T) -> T { // SAFETY: released just above. unsafe { PyEval_RestoreThread(state) }; RUNNING.set(outer); + // First, as a drain it queues at once may be one to resume. + if rt::Thread::made() { + drains(); + } if !RESUMES.with_borrow(Vec::is_empty) { resume(); } result } +/// The token of a drain, which no future or `Waker` takes. +const DRAIN: i64 = 0; + +/// A call made a value that is not `Send`: give this thread a wake that queues a drain on +/// the event loop running here, if one is, so a free on another thread reaches an idle +/// loop, which its `Queue.drain` then runs. +#[cold] +fn drains() { + // SAFETY: the GIL is held. An exception the caller has set is kept aside while Python + // runs, and put back after. + unsafe { + let (mut kind, mut value, mut trace) = (null_mut(), null_mut(), null_mut()); + PyErr_Fetch(&mut kind, &mut value, &mut trace); + // With no loop running here, the thread drains at its next call instead. + let running = running_loop(); + let woken = DRAINS.with_borrow(|drains| { + drains.upgrade().is_some_and(|queue| queue.event_loop == running && !queue.closed.load(Ordering::Relaxed)) + }); + if running != 0 && !woken { + match Obj::new(PyObject_CallNoArgs(ty(T_DRAINS))).and_then(|queue| queue_of(queue.ptr())) { + Ok(queue) => drain_on(&queue), + // The thread drains at its next call instead, and an interrupt its Python ran + // into is raised again at the thread's next chance. + Err(Raised) if PyErr_ExceptionMatches(PyExc_KeyboardInterrupt) != 0 => { + PyErr_Clear(); + PyErr_SetInterrupt(); + } + Err(Raised) => PyErr_WriteUnraisable(ty(T_DRAINS)), + } + } + PyErr_Restore(kind, value, trace); + } +} + +/// Wake this thread through `queue` for each drain. +fn drain_on(queue: &Arc) { + let weak = Arc::downgrade(queue); + DRAINS.set(weak.clone()); + rt::Thread::wake_here(Arc::new(move || { + if let Some(queue) = weak.upgrade() { + queue.push(DRAIN); + } + })); +} + +/// `drain()`, on the thread running its loop: drop what other threads freed of the +/// thread's values that are not `Send`. +unsafe extern "C" fn queue_drain(_: *mut PyObject, _: *mut PyObject) -> *mut PyObject { + call(rt::Thread::drain_here); + wrap(none) +} + thread_local! { /// The count of what waits on the event loop that last polled or took on this thread, /// until a call finds none running: a call looks for the loop running only while @@ -258,6 +314,8 @@ thread_local! { static RUNNING: Cell = const { Cell::new(0) }; /// The queues woken on their loop's thread during the calls it is in. static RESUMES: RefCell>> = const { RefCell::new(Vec::new()) }; + /// The queue this thread's wake queues a drain on. + static DRAINS: RefCell> = const { RefCell::new(std::sync::Weak::new()) }; } static GET_RUNNING_LOOP: std::sync::OnceLock = std::sync::OnceLock::new(); @@ -479,7 +537,8 @@ static MEMBERS: Gil>>> = Gil::new(Vec::new()); /// The runtime's entries, at the start of the registry: the awaitable a pending future /// is, the runtime's `Duration`, `_resume`, which schedules the running loop's dispatch, -/// `Waker`, `_Pollable`, which runs an awaitable Rust polls, and `ipaddress`'s two classes. +/// `Waker`, `_Pollable`, which runs an awaitable Rust polls, `ipaddress`'s two classes, and +/// `_drains`, which returns the running loop's queue. pub const T_FUTURE: usize = 0; const T_DURATION: usize = 1; const T_RESUME: usize = 2; @@ -487,6 +546,7 @@ const T_WAKER: usize = 3; const T_POLLABLE: usize = 4; const T_IPV4: usize = 5; const T_IPV6: usize = 6; +const T_DRAINS: usize = 7; /// The class registered at `index`. pub fn ty(index: usize) -> *mut PyObject { @@ -2035,6 +2095,9 @@ pub struct Queue { woken: AtomicBool, /// The address of its event loop, which [`RUNNING`] matches while it runs a call. event_loop: usize, + /// Whether its `Queue` was deallocated, as it is once its loop goes, after which + /// another loop may take the address. + closed: AtomicBool, /// How many of its futures are parked and its `Waker`s alive, which a wake may queue. waits: Arc, tx: Socket, @@ -2239,6 +2302,7 @@ unsafe extern "C" fn queue_new(class: *mut PyTypeObject, args: *mut PyObject, _: signaled: AtomicBool::new(false), woken: AtomicBool::new(false), event_loop, + closed: AtomicBool::new(false), waits: Arc::new(AtomicUsize::new(0)), tx, rx, @@ -2252,7 +2316,9 @@ unsafe extern "C" fn queue_new(class: *mut PyTypeObject, args: *mut PyObject, _: unsafe extern "C" fn queue_dealloc(obj: *mut PyObject) { // SAFETY: Python deallocates a `Queue` once. unsafe { - drop((*obj.cast::()).queue.take()); + if let Some(queue) = (*obj.cast::()).queue.take() { + queue.closed.store(true, Ordering::Relaxed); + } dealloc(obj); } } @@ -2807,10 +2873,11 @@ static FUTURE_METHODS: Methods<4> = Methods(UnsafeCell::new([ END, ])); -static QUEUE_METHODS: Methods<4> = Methods(UnsafeCell::new([ +static QUEUE_METHODS: Methods<5> = Methods(UnsafeCell::new([ noargs(c"fileno", queue_fileno), onearg(c"take", queue_take), noargs(c"signaled", queue_signaled), + noargs(c"drain", queue_drain), END, ])); diff --git a/ffi/runtime/rt.hpp b/ffi/runtime/rt.hpp index aeb5574ff..029533ad3 100644 --- a/ffi/runtime/rt.hpp +++ b/ffi/runtime/rt.hpp @@ -713,6 +713,33 @@ namespace detail { // A future driven on an executor, which a `task` cancels. struct task_node : node { virtual void cancel() noexcept = 0; + // Post a drain of what other threads freed of this thread's values that are not + // `Send` to the executor. + virtual void drain() noexcept = 0; +}; + +// What a free elsewhere of a value of its thread that is not `Send` wakes: the thread's +// latest task, which posts a drain to its executor, until the task is gone. Rust holds it +// rather than the task, so a gone task's callback and executor are freed with it. +class drainer final : public node { + ::std::mutex mutex_; + task_node *task_; + +public: + explicit drainer(task_node *task) noexcept : task_(task) {} + + void wake() noexcept override { + ::std::lock_guard<::std::mutex> lock(mutex_); + if (task_ != nullptr) { + task_->drain(); + } + } + + // Called as the task goes, after which it is never woken again. + void detach() noexcept { + ::std::lock_guard<::std::mutex> lock(mutex_); + task_ = nullptr; + } }; template @@ -1038,6 +1065,8 @@ class driver final : public task_node { // Held across each call to the executor, so `cancel` returns only once none is under // way, and none starts after. ::std::mutex posting_; + // What its thread's frees elsewhere wake while it is the thread's latest task, if any. + drainer *drainer_ = nullptr; // Null once the future finished or was cancelled. typename Op::raw *raw_; Exec exec_; @@ -1118,6 +1147,11 @@ class driver final : public task_node { } void cancel() noexcept override { + // First, so no drain is under way or starts once this returns. + if (drainer_ != nullptr) { + drainer_->detach(); + ::std::exchange(drainer_, nullptr)->unref(); + } // From its own callback, the future is already gone. if (on_.load(::std::memory_order_relaxed) != ::std::this_thread::get_id()) { ::std::lock_guard<::std::mutex> lock(mutex_); @@ -1131,6 +1165,19 @@ class driver final : public task_node { } unref(); } + + void drain() noexcept override { + ::std::lock_guard<::std::mutex> lock(posting_); + // It drains the thread it runs on, which is this one's only if the executor runs it + // here; a job elsewhere leaves the drops to this thread's next call. + exec_([]() { c_thread_drain(); }); + } + + // Be its thread's latest task, which its frees elsewhere wake. + void drains() { + drainer_ = new drainer(this); + c_thread_wake(give(drainer_)); + } }; template @@ -1138,6 +1185,11 @@ void spawn(task &slot, typename Op::raw *raw, Exec exec, F f) { auto *d = new driver(raw, ::std::move(exec), ::std::move(f)); slot = task(); slot.node_ = d; + // The latest executor of this thread drains what other threads free of its values + // that are not `Send`, though it calls nothing into Rust, until its task is gone. + if constexpr (drains) { + d->drains(); + } d->wake(); } diff --git a/ffi/runtime/rt.py b/ffi/runtime/rt.py index b051716b4..a3ce4f64f 100644 --- a/ffi/runtime/rt.py +++ b/ffi/runtime/rt.py @@ -59,6 +59,10 @@ async def _pump(self, loop: _asyncio.AbstractEventLoop, sock: _socket.socket) -> def dispatch(self, read: _bool) -> None: loop: _asyncio.AbstractEventLoop | None = None for token in self.queue.take(read): + if token == 0: + # Other threads freed values of this thread that are not Send. + self.queue.drain() + continue if token < 0: self.wakers.pop(-token, None) continue @@ -87,6 +91,12 @@ def _resume() -> _n.Queue | None: return w.queue +def _drains() -> _n.Queue: + """The queue of the loop running here, which a free on another thread of a value this + thread made that is not Send queues a drain on.""" + return _wakeup(_get_running_loop()).queue + + def _wakeup(loop: _asyncio.AbstractEventLoop) -> _Wakeup: w = _wakeups.get(loop) if w is None: diff --git a/ffi/runtime/rt/thread.rs b/ffi/runtime/rt/thread.rs index da3c0e602..20eba4fba 100644 --- a/ffi/runtime/rt/thread.rs +++ b/ffi/runtime/rt/thread.rs @@ -2,13 +2,15 @@ //! records the thread that made it, which alone borrows, polls, moves, or drops it: a use //! on another thread fails its check, and a free there queues the drop to its thread, //! which runs it at its next check, when it exits, or at once if it waits for work. One -//! freed once its thread exited leaks, saying so, as nothing may drop it. +//! freed once its thread exited leaks, saying so, as nothing may drop it. A binding whose +//! thread runs a loop, as a Python event loop or a C++ executor, gives the thread a +//! [`Wake`], which schedules a drain on that loop so a free reaches an idle thread. //! //! A home is a thread Rust owns, for a binding with no thread of its own that stays put, //! as a goroutine or a Dart isolate moves between threads: a call it makes on a value a //! home owns runs there while the caller waits, so the value never leaves its thread. -use std::cell::OnceCell; +use std::cell::{Cell, OnceCell}; use std::sync::OnceLock; use std::mem::{ManuallyDrop, MaybeUninit}; use std::panic::AssertUnwindSafe; @@ -31,8 +33,14 @@ struct Jobs { queue: Vec, /// Whether the thread exited, so that nothing queued runs. gone: bool, + /// Called, from the thread that queues, as the queue goes from empty to not. + wake: Option, } +/// What schedules a drain of a thread's queue on the loop it runs, from any thread: +/// [`Thread::drain_here`] there does it. +pub type Wake = Arc; + enum Job { Drop(Dropped), /// A call another thread waits on, on a home. @@ -119,8 +127,9 @@ impl Job { } } -/// The record of the thread it is a field of, until the thread exits, then empty. -struct Here(OnceCell>); +/// The record of the thread it is a field of, until the thread exits, then empty, and +/// whether a value made here since [`Thread::made`] last looked asked for it. +struct Here(OnceCell>, Cell); impl Drop for Here { fn drop(&mut self) { @@ -130,7 +139,7 @@ impl Drop for Here { } } -thread_local!(static HERE: Here = const { Here(OnceCell::new()) }); +thread_local!(static HERE: Here = const { Here(OnceCell::new(), Cell::new(false)) }); /// The homes no value lives on, idle until a call takes one. static IDLE: Mutex>> = Mutex::new(Vec::new()); @@ -142,6 +151,7 @@ impl Thread { jobs: Mutex::new(Jobs { queue: Vec::new(), gone, + wake: None, }), parked: OnceLock::new(), home, @@ -151,10 +161,51 @@ impl Thread { /// This thread's record, which a value it makes keeps. A thread past its thread locals' /// teardown has one that is gone, so a value it makes is used nowhere and leaks. pub fn here() -> Arc { + let _ = HERE.try_with(|here| here.1.set(true)); + Self::record() + } + + /// This thread's record, as [`Self::here`] without noting a value made. + fn record() -> Arc { HERE.try_with(|here| here.0.get_or_init(|| Arc::new(Self::new(false, false))).clone()) .unwrap_or_else(|_| Arc::new(Self::new(false, true))) } + /// Whether this thread made a value that is not `Send` since this last asked: a binding + /// whose thread runs a loop then gives it a [`Wake`] for the loop running. + #[inline] + pub fn made() -> bool { + HERE.try_with(|here| here.1.replace(false)).unwrap_or(false) + } + + /// Have each free elsewhere of a value of this thread call `wake` as it queues the + /// first drop since the last drain, in place of any earlier one: the thread's loop + /// then drains, though it calls nothing. A drop already queued wakes it at once, as + /// no earlier wake may have scheduled its drain. + pub fn wake_here(wake: Wake) { + let thread = Self::record(); + let mut jobs = thread.lock(); + let now = thread.pending.load(Ordering::Relaxed).then(|| wake.clone()); + let old = jobs.wake.replace(wake); + drop(jobs); + // Outside the lock, as a foreign drop or wake may take its own. + drop(old); + if let Some(wake) = now { + wake(); + } + } + + /// Run what other threads queued for this thread, if any: what a [`Wake`] schedules. + pub fn drain_here() { + let _ = HERE.try_with(|here| { + if let Some(thread) = here.0.get() + && thread.pending.load(Ordering::Acquire) + { + thread.drain(); + } + }); + } + /// Whether this is its thread. pub fn is_here(&self) -> bool { HERE.try_with(|here| here.0.get().is_some_and(|thread| std::ptr::eq(&**thread, self))) @@ -190,11 +241,16 @@ impl Thread { return Err(job); } jobs.queue.push(job); - self.pending.store(true, Ordering::Release); + let first = !self.pending.swap(true, Ordering::Release); + let wake = jobs.wake.clone().filter(|_| first); drop(jobs); if let Some(thread) = self.parked.get() { thread.unpark(); } + // Outside the lock, as it calls into the binding. + if let Some(wake) = wake { + wake(); + } Ok(()) } @@ -217,6 +273,9 @@ impl Thread { let mut jobs = self.lock(); if jobs.queue.is_empty() { jobs.gone = true; + let wake = jobs.wake.take(); + drop(jobs); + drop(wake); return; } std::mem::take(&mut jobs.queue) diff --git a/ffi/src/abi.rs b/ffi/src/abi.rs index caccb2f5f..98f03eff7 100644 --- a/ffi/src/abi.rs +++ b/ffi/src/abi.rs @@ -1325,6 +1325,10 @@ impl<'a> Abi<'a> { "foreign_reader", "buf_copy", "buf_clone", + "thread_wake", + "thread_drain", + // Python's extension module function, beside each native function by this name. + "_register", "poll", "view", "items", diff --git a/ffi/src/c.rs b/ffi/src/c.rs index cf19adc7a..7e75a09d0 100644 --- a/ffi/src/c.rs +++ b/ffi/src/c.rs @@ -99,6 +99,11 @@ impl Header<'_> { self.line(format!( "/* Wakes a future's poller: wake may be called from any thread, any number of\n * times, until drop releases data. A poll passing the same wake and data again\n * reuses the waker. Either function may be null. One Rust returns is the caller's,\n * to drop once or pass back into Rust. */\ntypedef struct {p}_waker {{\n\tvoid (*wake)(void *data);\n\tvoid (*drop)(void *data);\n\tvoid *data;\n}} {p}_waker;\n" )); + if api.unsent() { + self.line(format!( + "/* A value that is not Send, freed on another thread than the one that made it,\n * drops on that thread at its next call on such a value, or when it exits. A thread\n * running a loop that may call nothing for a while passes {p}_thread_wake a waker,\n * replacing any earlier one, which a free elsewhere wakes from its own thread as it\n * queues a drop: the waker then schedules {p}_thread_drain on the thread, which drops\n * each value queued for the thread it runs on. */\nvoid {p}_thread_wake({p}_waker wake);\nvoid {p}_thread_drain(void);\n" + )); + } self.line(format!( "/* A future C implements, of nothing: Rust polls it from any thread with a waker of\n * its own each time, which C wakes once the future can make progress and drops once,\n * until poll returns true, and calls free with data once it drops the future, done or\n * not. free may be null. */\ntypedef struct {p}_foreign_future {{\n\tbool (*poll)(void *data, {p}_waker waker);\n\tvoid (*free)(void *data);\n\tvoid *data;\n}} {p}_foreign_future;\n" )); diff --git a/ffi/src/cpp.rs b/ffi/src/cpp.rs index 2bb00b114..96eccef9d 100644 --- a/ffi/src/cpp.rs +++ b/ffi/src/cpp.rs @@ -558,8 +558,15 @@ impl<'a> Header<'a> { "#if defined(__cpp_impl_coroutine) && __cpp_impl_coroutine >= 201902L && __has_include()\n#include \n#endif\n", ); self.line(format!("namespace {ns} {{\n")); + // Whether a call makes a value that is not `Send`, whose thread an executor drains. + let drains = match self.abi.api.unsent() { + true => format!( + "inline constexpr bool drains = true;\n\ninline void c_thread_wake(c_waker wake) noexcept {{\n\t::{p}_thread_wake(wake);\n}}\n\ninline void c_thread_drain() noexcept {{\n\t::{p}_thread_drain();\n}}\n\n" + ), + false => "inline constexpr bool drains = false;\n\ninline void c_thread_wake(c_waker) noexcept {}\n\ninline void c_thread_drain() noexcept {}\n\n".to_string(), + }; self.line(format!( - "namespace detail {{\n\nusing c_buf = ::{p}_buf;\nusing c_waker = ::{p}_waker;\nusing c_foreign_future = ::{p}_foreign_future;\nusing c_foreign_reader = ::{p}_foreign_reader;\n\ninline void c_buf_free(c_buf buf) noexcept {{\n\t::{p}_buf_free(buf);\n}}\n\ninline c_buf c_buf_copy(c_buf view) noexcept {{\n\treturn ::{p}_buf_copy(view);\n}}\n\ninline c_buf c_buf_clone(c_buf buf) noexcept {{\n\treturn ::{p}_buf_clone(buf);\n}}\n\n}} // namespace detail\n" + "namespace detail {{\n\nusing c_buf = ::{p}_buf;\nusing c_waker = ::{p}_waker;\nusing c_foreign_future = ::{p}_foreign_future;\nusing c_foreign_reader = ::{p}_foreign_reader;\n\ninline void c_buf_free(c_buf buf) noexcept {{\n\t::{p}_buf_free(buf);\n}}\n\ninline c_buf c_buf_copy(c_buf view) noexcept {{\n\treturn ::{p}_buf_copy(view);\n}}\n\ninline c_buf c_buf_clone(c_buf buf) noexcept {{\n\treturn ::{p}_buf_clone(buf);\n}}\n\n{drains}}} // namespace detail\n" )); self.out.push_str(RUNTIME); self.line(format!( diff --git a/ffi/src/glue.rs b/ffi/src/glue.rs index 87a18930a..2184e09a9 100644 --- a/ffi/src/glue.rs +++ b/ffi/src/glue.rs @@ -1295,6 +1295,15 @@ impl Gen<'_> { "#[unsafe(no_mangle)]\npub extern \"C\" fn {p}_status_message(status: i32) -> *const ::std::ffi::c_char {{\n\trt::Status::text(status).as_ptr()\n}}\n" )); + if api.unsent() { + self.line(format!( + "#[unsafe(no_mangle)]\npub extern \"C\" fn {p}_thread_drain() {{\n\trt::Thread::drain_here()\n}}\n" + )); + self.line(format!( + "#[unsafe(no_mangle)]\npub unsafe extern \"C\" fn {p}_thread_wake(wake: rt::Waker) {{\n\tlet wake = wake.lift();\n\trt::Thread::wake_here(::std::sync::Arc::new(move || wake.wake_by_ref()))\n}}\n" + )); + } + for (name, ty) in &self.abi.ranges { let (kind, int) = match ty { Type::Range(int) => ("Range", int), diff --git a/ffi/src/ir.rs b/ffi/src/ir.rs index 1c2c87501..ef78aeadf 100644 --- a/ffi/src/ir.rs +++ b/ffi/src/ir.rs @@ -96,6 +96,20 @@ impl Api { api } + /// Whether a call may make a value that is not `Send`, an object or a future, whose + /// thread a binding running a loop gives a wake to drain its frees elsewhere. + pub fn unsent(&self) -> bool { + let records = self.records.iter().flat_map(|r| &r.methods); + let enums = self.enums.iter().flat_map(|e| &e.methods); + let objects = self.objects.iter().flat_map(|o| &o.methods); + self.objects.iter().any(|o| !o.send) + || records + .chain(enums) + .chain(objects) + .chain(&self.functions) + .any(|f| f.future && !f.send) + } + /// Whether an item on `on` is on every target the API serves. pub fn everywhere(&self, on: On) -> bool { on.0 & self.all().0 == self.all().0 diff --git a/ffi/src/python.rs b/ffi/src/python.rs index 82687250a..e9ef9d258 100644 --- a/ffi/src/python.rs +++ b/ffi/src/python.rs @@ -55,6 +55,7 @@ const TAKEN: &[&str] = &[ "_wakeups", "_wakeup", "_resume", + "_drains", "_Const", "_PENDING", "_consts", @@ -109,7 +110,7 @@ const V4: &str = "_tuple[_str, _int]"; const V6: &str = "_tuple[_str, _int, _int, _int]"; /// The registry's first entries, the runtime's: the glue's `T_FUTURE`, `T_DURATION`, -/// `T_RESUME`, `T_WAKER`, `T_POLLABLE`, `T_IPV4`, and `T_IPV6`. +/// `T_RESUME`, `T_WAKER`, `T_POLLABLE`, `T_IPV4`, `T_IPV6`, and `T_DRAINS`. const RUNTIME_TYPES: &[&str] = &[ "_Future", "Duration", @@ -118,6 +119,7 @@ const RUNTIME_TYPES: &[&str] = &[ "_Pollable", "_ipaddress.IPv4Address", "_ipaddress.IPv6Address", + "_drains", ]; /// The Python for `abi`, whose header's contract is `contract`, or the names two items diff --git a/ffi/tests/api.cpp b/ffi/tests/api.cpp index 24e813740..b56d5f776 100644 --- a/ffi/tests/api.cpp +++ b/ffi/tests/api.cpp @@ -1703,7 +1703,8 @@ int futures() { } // A value that is not Send stays on the thread that made it, whose executor polls its - // futures; destroyed on another thread, it drops on its own at its next call. + // futures; destroyed on another thread, it drops on its own, posted to the executor of + // its thread's latest task, or else at its next call. { api::Local local = api::Local::new_(); api::Local shared = local.share(); @@ -1719,7 +1720,37 @@ int futures() { std::uint32_t before = api::locals_dropped(); std::thread([&]() { api::Local gone = std::move(local); }).join(); check(api::locals_dropped() == before, "a local destroyed elsewhere waits for its thread"); - check(shared.bump() == 4 && api::locals_dropped() == before + 1, "a local destroyed elsewhere drops on its thread"); + q.drain(); + check(api::locals_dropped() == before + 1, "a local destroyed elsewhere drops on its idle executor"); + + // An executor running the drain on another thread leaves it to the next call. + queue away; + auto kept = std::make_shared(0); + api::task there = api::yielded(7).then(away.executor(), [&got, kept](std::uint32_t value) { got = value; }); + api::Local other = shared.share(); + std::thread([&]() { api::Local gone = std::move(other); }).join(); + std::thread([&]() { away.drain(); }).join(); + check(got == 7 && api::locals_dropped() == before + 1, "a drain on another thread drops nothing of this one"); + check(shared.bump() == 4 && api::locals_dropped() == before + 2, "a local destroyed elsewhere drops on its thread"); + + // Once the latest task is gone, its executor is never called again. + there = api::task(); + check(kept.use_count() == 1, "a gone task's callback is freed, though its thread's wake outlives it"); + api::Local last = shared.share(); + std::thread([&]() { api::Local gone = std::move(last); }).join(); + check(away.jobs.empty(), "a drain is never posted to a gone task's executor"); + check(shared.bump() == 5 && api::locals_dropped() == before + 3, "a local destroyed elsewhere drops at its next call"); + + // A drop queued before its thread's first task drains through that task's executor. + std::thread([]() { + queue mine; + api::Local early = api::Local::new_(); + std::uint32_t was = api::locals_dropped(); + std::thread([&]() { api::Local gone = std::move(early); }).join(); + api::task first = api::yielded(1).then(mine.executor(), [](std::uint32_t) {}); + mine.drain(); + check(api::locals_dropped() == was + 1, "a drop queued before its thread's first task drains through it"); + }).join(); } // Cancelled between its wake and its poll, a receive loses nothing: the payload stays diff --git a/ffi/tests/api.py b/ffi/tests/api.py index fc62a9947..4dcdd26f6 100644 --- a/ffi/tests/api.py +++ b/ffi/tests/api.py @@ -963,6 +963,38 @@ def elsewhere(): check(api.locals_dropped() == before, "a local freed elsewhere waits for its thread") check(shared.bump() == 4 and api.locals_dropped() == before + 1, "a local freed elsewhere drops on its thread") + # Made while a loop runs, one freed elsewhere drops on the loop's next turn, though the + # loop calls nothing into Rust meanwhile. + # One made with no loop running and freed elsewhere waits, until a value made while one + # runs gives the thread its wake. + early = [api.Local.new()] + before = api.locals_dropped() + thread = threading.Thread(target=early.clear) + thread.start() + thread.join() + check(api.locals_dropped() == before, "a local freed elsewhere with no wake waits") + + async def idle() -> None: + before = api.locals_dropped() + held = [api.Local.new()] + await asyncio.sleep(0) + await asyncio.sleep(0) + check(api.locals_dropped() == before + 1, "a drop queued before its thread's wake drains once it is set") + before += 1 + freed = loop.create_future() + + def free() -> None: + held.clear() + loop.call_soon_threadsafe(freed.set_result, None) + + thread = threading.Thread(target=free) + thread.start() + await freed + thread.join() + check(api.locals_dropped() == before + 1, "a local freed elsewhere drops on its idle loop") + + loop.run_until_complete(idle()) + # Cancelled between its wake and its poll, a receive loses nothing: the payload stays # for the next. called = False diff --git a/quest/m1/ffi-speed/loop-drain.md b/quest/m1/ffi-speed/loop-drain.md deleted file mode 100644 index 93189680e..000000000 --- a/quest/m1/ffi-speed/loop-drain.md +++ /dev/null @@ -1,34 +0,0 @@ -# [M] A free elsewhere drains on an idle Python loop or C++ executor - -## Goal - -A [rust2-ffi](/ffi/README.md) value that is not `Send`, freed on another thread, drops -promptly on its owner thread even while that thread's Python event loop or C++ executor -calls nothing into Rust. Today the drop waits in the owner's queue for its next call or -its exit ([kixel-ai/rust2#844](https://github.com/kixel-ai/rust2/pull/844)), so an idle -loop holds what was freed, such as a closed session's sockets, for as long as it stays -idle. The Goal's measurement is that delay: unbounded today, one loop turn after. - -## Plan - -Decided with the maintainer on 2026-11-16, picking -[kixel-ai/rust2#844](https://github.com/kixel-ai/rust2/pull/844) to merge: ✅ schedule a -drain on Python's wake queue and on a C++ executor when a free elsewhere queues one. -Rejected: waiting for the owner's next call or exit, as #844 does. - -- `rt/thread.rs`'s owner record gains an optional wake the binding sets, which the queue - calls once when it goes from empty to non-empty. Go and Dart already send every such - value to a home, which drains on each wake, so they set none. Kotlin and Swift home only - a future made from none; a value they make stays on the caller's thread, which has no - loop Rust can post to, so it keeps draining at its next call or exit, outside this quest. -- Python: the owner's wake queues a drain token on the wake queue its futures use, with - the same byte to the loop's socket, and the loop's dispatch drains on the loop thread. - The binding sets the wake when a value that is not `Send` is first made on a thread - with a running loop. -- C++: an executor may run its job on any thread, so a drain posted to one runs only if - it lands on the owner. Recommended: post the drain job to the executor of the owner's - latest `then` or `via`, and drain if the job runs on the owner thread, else leave the - queue for the next call. Rejected: a generated `api::drain()` for the caller to - schedule, which is new API for a case the runtime can mostly cover. -- Prove it in `api.py` and `api.cpp`: free a `Local` on another thread while the owner's - loop is idle, and observe its drop with no further call into Rust.