Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 16 additions & 7 deletions ffi/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -789,7 +789,8 @@ caller imports it with no ignores.
- **Objects** are classes owning their handle, freed once their last reference goes,
which the caller's reference keeps through a call, so there is no `close()`. A `self`
method, or passing the object by value, moves its value out, after which a call raises
`ValueError`. `Clone` is `clone()`, and `PartialEq` is `==`, unhashable unless the type implements `Hash`, which `hash()` hashes it by. A field is a
`ValueError`. `Clone` is `clone()`, and `PartialEq` is `==`, unhashable unless the type implements `Hash`, which `hash()` hashes it by. An object, as
an opaque value and a record, takes weak references (`weakref.ref`). A field is a
property, `first_get`, and its setter a method, `first_set(value)`, named as the C ones,
since setting an object moves it in and closes it, which an assignment would hide.
- **Checks**: a borrow or move Rust's rules refuse raises, naming the function and the rule,
Expand All @@ -800,10 +801,13 @@ caller imports it with no ignores.
thread: on the next turn of the event loop that ran there when a call made a value that
is not `Send`, though the loop calls nothing into Rust, else at its next call or as it
exits.
- **Records** are slotted dataclasses, compared by their fields, a float by IEEE `==` even
against itself, as Rust does, and **enums** an `Enum`
of upper-case members, or for one with data a class of a dataclass per variant,
`Event.Moved(p)`. `Display` is `str()`, `Debug` `repr()`, and `Default` a static
- **Records** are slotted classes, written out as `@dataclass(slots=True,
weakref_slot=True)` would make them, so no `dataclass` runs at import: an `__init__`
taking each field, `__match_args__`, a dataclass's `repr`, and `__eq__` comparing the
fields, a float by IEEE `==` even against itself, as Rust does. `dataclasses.fields`,
`replace`, and `asdict` take one, as its `__dataclass_fields__` is made on its first read,
and so does `copy.replace`. **Enums** are an `Enum` of upper-case members, or for one
with data a class of such a record per variant, `Event.Moved(p)`. `Display` is `str()`, `Debug` `repr()`, and `Default` a static
`default()`. Constants are module or class attributes, made anew on each read unless a
literal. An **opaque value** is a class of the runtime's `Value`, which holds its bytes
in place, so only Rust makes or reads one: making one raises `TypeError`. It is shared
Expand All @@ -819,7 +823,11 @@ caller imports it with no ignores.
raises `OverflowError` or `ValueError` passed in out of their range; its `from_timedelta`
and `to_timedelta` raise rather than lose anything, as a `timedelta` holds only whole
microseconds up to 999,999,999 days. A zero passed for a `NonZero` raises `ValueError`. A `Range` is a `range`, and an `Err` a raised `<E>Exception`
holding it in `error`, whose `str` is its `Display`. A `[u8; N]` is a generated
holding it in `error`, whose `str` is its `Display`. An enum's, or an object's whose
type is an enum, raises a subclass per variant, `ErrorException.Timeout`, which
`ErrorException(error)` makes too, so `except ErrorException.Timeout` catches that
variant alone; one of a variant a `#[non_exhaustive]` enum's crate adds after the export
raises `ErrorException` itself. A `[u8; N]` is a generated
`ArrayU8x<N>`, a `bytes` that raises `ValueError` unless it holds exactly `N`; a
parameter takes any `bytes` of `N`.
- A **path** is a `str`, or any path passed in, which crosses as `os.fsencode` encodes it, so
Expand Down Expand Up @@ -861,7 +869,8 @@ caller imports it with no ignores.
to a socket the loop reads, without the GIL, or during a call on the loop's own thread,
writes nothing and schedules the loop's dispatch once the call returns. Cancelling the task frees the future and
loses nothing. The call polls it once before suspending, and a future is awaited once,
as a coroutine is.
as a coroutine is. Futures and wakers run on asyncio's event loop alone: an await or a
`Waker` where none runs, as under trio, raises `RuntimeError` saying so.
- An **awaitable Python implements** is an `Awaitable[None]`, such as a coroutine, passed
on a running event loop, where Rust's first poll runs it as a task. Rust dropping it
cancels the task, or closes a coroutine it never started. One that raises, or that
Expand Down
36 changes: 36 additions & 0 deletions ffi/fixtures/api/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4454,6 +4454,42 @@ pub fn refuse(reason: &str) -> Result<u32, Box<Refusal>> {
}))
}

/// Why [`jam`] failed: an enum whose own `Drop` makes it an object, whose variants a
/// binding still tells apart, as Python raises each as its own subclass.
#[derive(Debug)]
pub enum Fault {
/// Nothing is free.
Busy,
/// What was sent is lost.
Lost(String),
/// Too late, by this many.
Late { by: u32 },
}

impl Drop for Fault {
fn drop(&mut self) {}
}

impl fmt::Display for Fault {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Fault::Busy => write!(f, "busy"),
Fault::Lost(what) => write!(f, "lost {what}"),
Fault::Late { by } => write!(f, "late by {by}"),
}
}
}

/// Fails with the fault `n` picks: busy, lost, or late, and for any other `n`, returns it.
pub fn jam(n: u32) -> Result<u32, Fault> {
match n {
0 => Err(Fault::Busy),
1 => Err(Fault::Lost("frame".to_string())),
2 => Err(Fault::Late { by: 7 }),
n => Ok(n),
}
}

/// The kind of target the library was built for. One binding serves the host and wasm32,
/// so it has every target's variants, and a target refuses one it lacks. As wasm32's comes
/// first, `Native`'s Rust discriminant differs between the two, which its C tag does not.
Expand Down
59 changes: 57 additions & 2 deletions ffi/runtime/python.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1112,6 +1112,8 @@ pub fn array_out(items: &[u8], index: usize) -> R<Obj> {
#[repr(C)]
struct ValueHead {
ob_base: PyVarObject,
/// The weak references to it, which `weakref` keeps.
weaklist: *mut PyObject,
/// The class registered at this index made it: Python can reassign `__class__`, to
/// any class of `Value`'s layout, whose bytes may be fewer or another type's.
index: usize,
Expand Down Expand Up @@ -1142,6 +1144,7 @@ pub fn value_out<T: Copy>(index: usize, value: T) -> R<Obj> {
unsafe {
let alloc: allocfunc = std::mem::transmute(PyType_GetSlot(class.cast(), Py_tp_alloc));
let obj = Obj::new(alloc(class.cast(), size_of::<T>() as Py_ssize_t))?;
(*obj.ptr().cast::<ValueHead>()).weaklist = null_mut();
(*obj.ptr().cast::<ValueHead>()).index = index;
obj.ptr().cast::<u8>().add(size_of::<ValueHead>()).cast::<T>().write_unaligned(value);
Ok(obj)
Expand Down Expand Up @@ -1610,13 +1613,14 @@ pub fn member_in(obj: *mut PyObject, index: usize) -> R<i32> {
}
}

/// Raise the exception class registered at `index`, holding `error`.
/// Raise the exception class registered at `index`, holding `error`: the subclass of its
/// variant, for an enum's, which the class makes.
pub fn raise_error<T>(index: usize, error: R<Obj>) -> R<T> {
let args = tuple(vec![error])?;
// SAFETY: the GIL is held, and the class an exception's.
unsafe {
let exc = Obj::new(PyObject_CallObject(ty(index), args.ptr()))?;
PyErr_SetObject(ty(index), exc.ptr());
PyErr_SetObject(Py_TYPE(exc.ptr()).cast(), exc.ptr());
}
Err(Raised)
}
Expand Down Expand Up @@ -1846,6 +1850,8 @@ pub unsafe extern "C" fn free_callback(data: *mut c_void) {
#[repr(C)]
pub struct Handle {
ob_base: PyObject,
/// The weak references to it, which `weakref` keeps.
weaklist: *mut PyObject,
/// The class registered at this index made it: Python can reassign `__class__`.
index: usize,
raw: *const c_void,
Expand Down Expand Up @@ -1890,6 +1896,7 @@ fn handle(raw: *const c_void, index: usize, free: Option<unsafe extern "C" fn(*c
return Err(Raised);
};
let handle = obj.ptr().cast::<Handle>();
(*handle).weaklist = null_mut();
(*handle).index = index;
(*handle).raw = raw;
(*handle).free = free;
Expand Down Expand Up @@ -1969,11 +1976,56 @@ unsafe fn free_now(handle: *mut Handle) {
unsafe extern "C" fn handle_dealloc(obj: *mut PyObject) {
// SAFETY: Python deallocates an `Object` once, holding the GIL; nothing holds it.
unsafe {
clear_weak(obj, (*obj.cast::<Handle>()).weaklist);
free_now(obj.cast());
dealloc(obj);
}
}

unsafe extern "C" fn value_dealloc(obj: *mut PyObject) {
// SAFETY: Python deallocates a `Value` once, holding the GIL.
unsafe {
clear_weak(obj, (*obj.cast::<ValueHead>()).weaklist);
dealloc(obj);
}
}

/// Clear the weak references to `obj`, as its dealloc must: Python's `subtype_dealloc`
/// leaves them to it, for a class of `Object` or `Value` is not tracked by the collector.
///
/// # Safety
/// `obj` is being deallocated, holding the GIL, and `weaklist` is its list.
unsafe fn clear_weak(obj: *mut PyObject, weaklist: *mut PyObject) {
if !weaklist.is_null() {
// SAFETY: the caller's contract.
unsafe { PyObject_ClearWeakRefs(obj) };
}
}

/// The members of a type whose instances hold a list of weak references at `offset`:
/// `__weaklistoffset__` alone, which `PyType_FromSpec` reads to let `weakref` take them.
const fn weak_members(offset: usize) -> [PyMemberDef; 2] {
[
PyMemberDef {
name: c"__weaklistoffset__".as_ptr(),
type_code: Py_T_PYSSIZET,
offset: offset as Py_ssize_t,
flags: Py_READONLY,
doc: null(),
},
PyMemberDef {
name: null(),
type_code: 0,
offset: 0,
flags: 0,
doc: null(),
},
]
}

static OBJECT_MEMBERS: Gil<[PyMemberDef; 2]> = Gil::new(weak_members(std::mem::offset_of!(Handle, weaklist)));
static VALUE_MEMBERS: Gil<[PyMemberDef; 2]> = Gil::new(weak_members(std::mem::offset_of!(ValueHead, weaklist)));

/// Free an instance of a heap type, and the reference to its type it held.
///
/// # Safety
Expand Down Expand Up @@ -2977,6 +3029,7 @@ unsafe fn runtime(module: *mut PyObject) -> R<()> {
let base = (Py_TPFLAGS_BASETYPE | Py_TPFLAGS_DISALLOW_INSTANTIATION) as u64;
add_type(module, &OBJECT, c"Object", size_of::<Handle>(), base, &mut [
slot(Py_tp_dealloc, handle_dealloc as *mut c_void),
slot(Py_tp_members, OBJECT_MEMBERS.get().as_mut_ptr().cast()),
slot(
Py_tp_doc,
c"An object owning a Rust value's handle, freed once the last reference to it goes.".as_ptr().cast_mut().cast(),
Expand All @@ -2985,6 +3038,8 @@ unsafe fn runtime(module: *mut PyObject) -> R<()> {
])?;
// Its items are a value's bytes, past the header.
add_sized(module, &VALUE_BASE, c"Value", size_of::<ValueHead>(), 1, base, &mut [
slot(Py_tp_dealloc, value_dealloc as *mut c_void),
slot(Py_tp_members, VALUE_MEMBERS.get().as_mut_ptr().cast()),
slot(
Py_tp_doc,
c"A Rust `Copy` value's bytes, which only Rust makes or reads.".as_ptr().cast_mut().cast(),
Expand Down
60 changes: 58 additions & 2 deletions ffi/runtime/rt.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# The runtime every generated Python module includes: the awaitable a future that pends
# is, the queue each event loop reads its wakes from, `Waker`, what runs an awaitable Rust
# polls, `Mut`, `ForeignReader`, and `Duration`. Its names start with `_` but `Waker`,
# `Mut`, `ForeignReader`, and `Duration`, so no exported name clashes with them.
# polls, `Mut`, `ForeignReader`, `Duration`, and what each record shares. Its names start
# with `_` but `Waker`, `Mut`, `ForeignReader`, and `Duration`, so no exported name clashes
# with them.


class _Future(_n.Future, _typing.Generic[_T]):
Expand Down Expand Up @@ -231,3 +232,58 @@ def __init__(self, get: _typing.Callable[[], _object]) -> None:

def __get__(self, obj: _object, cls: _object = None) -> _object:
return self.get()


class _Dataclass:
"""A record class's `__dataclass_fields__` or `__dataclass_params__`, which `dataclasses`
makes on the first read of either, so `dataclasses` takes a record, or a subclass of
one, as the dataclass it was, while no record pays for `dataclasses` at import."""

__slots__ = ("name",)

def __init__(self, name: _str) -> None:
self.name = name

def __get__(self, obj: _object, cls: _type[_Record]) -> _object:
import dataclasses

# The record that declares the fields, which a subclass of it inherits. It is the first
# this module made, as a subclass made elsewhere may set its own `__match_args__`.
owner: _typing.Any = _Record
for c in cls.__mro__:
if c.__module__ == _Record.__module__ and "__match_args__" in c.__dict__:
owner = c
break
hints = owner.__annotations__
shape: _typing.Any = _type(owner.__name__, (), {"__annotations__": {name: hints[name] for name in owner.__match_args__}})
shape = dataclasses.dataclass(shape, slots=True, weakref_slot=True)
made = {name: _getattr(shape, name) for name in ("__dataclass_fields__", "__dataclass_params__")}
# `_Record`'s own stay, as each record reads them.
if owner is not _Record:
for name, value in made.items():
_type.__setattr__(owner, name, value)
return made[self.name]


class _Record:
"""What each record shares besides the `__init__` and `__eq__` it writes out as a
dataclass would: a dataclass's `repr`, `__dataclass_fields__`, `__dataclass_params__`,
and `__replace__`."""

__slots__ = ()

__match_args__: _typing.ClassVar[_tuple[_str, ...]] = ()
if _typing.TYPE_CHECKING:
__dataclass_fields__: _typing.ClassVar[_dict[_str, _dataclasses.Field[_typing.Any]]]
else:
__dataclass_fields__ = _Dataclass("__dataclass_fields__")
__dataclass_params__ = _Dataclass("__dataclass_params__")

def __repr__(self) -> _str:
fields = ", ".join(f"{name}={_getattr(self, name)!r}" for name in self.__match_args__)
return f"{_type(self).__qualname__}({fields})"

def __replace__(self, /, **changes: _object) -> _typing.Self:
import dataclasses

return dataclasses.replace(self, **changes)
14 changes: 14 additions & 0 deletions ffi/src/collect.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1732,6 +1732,7 @@ impl<'tcx> Collector<'tcx> {
lifetimes,
methods: Vec::new(),
traits: Traits::default(),
variants: Vec::new(),
foreign: false,
opaque: None,
});
Expand Down Expand Up @@ -1881,6 +1882,7 @@ impl<'tcx> Collector<'tcx> {
clone: !owned,
..Traits::default()
},
variants: Vec::new(),
foreign: true,
opaque: Some(match owned {
true => Held::Value,
Expand Down Expand Up @@ -2016,6 +2018,7 @@ impl<'tcx> Collector<'tcx> {
lifetimes: 0,
methods,
traits,
variants: Vec::new(),
foreign: true,
opaque: None,
}
Expand Down Expand Up @@ -2434,6 +2437,17 @@ impl<'tcx> Collector<'tcx> {
object.methods = methods;
object.traits = traits;
object.shared = shared && object.sync;
if kind == DefKind::Enum && object.opaque.is_none() {
object.variants = (tcx.adt_def(def_id).variants().iter())
.map(|v| Variant {
name: v.name.to_string(),
discriminant: 0,
fields: Vec::new(),
tuple: false,
on: On::ALL,
})
.collect();
}
}
_ => {}
}
Expand Down
34 changes: 34 additions & 0 deletions ffi/src/glue.rs
Original file line number Diff line number Diff line change
Expand Up @@ -430,6 +430,7 @@ impl Crate<'_> {
checking: Cell::new(false),
reserved: Cell::new(false),
web: web.is_some(),
variants: String::new(),
};
generator.lib();
output.file(&out.join("Cargo.toml"), &manifest("0.0.0"));
Expand Down Expand Up @@ -637,6 +638,8 @@ struct Gen<'a> {
reserved: Cell<bool>,
/// Whether the API serves the web, whose glue the crate's `web` feature builds.
web: bool,
/// The Python glue's functions telling which variant an object that is an enum holds.
variants: String,
}

impl Gen<'_> {
Expand Down Expand Up @@ -1498,6 +1501,12 @@ impl Gen<'_> {
for function in &api.functions {
self.function(function, None);
}
if !self.variants.is_empty() {
let variants = crate::kotlin::indent_by(&std::mem::take(&mut self.variants), 1);
self.line(format!(
"/// Which variant each object that is an enum holds, for the Python glue, which raises an\n/// error's own subclass by it.\n#[cfg(feature = \"python\")]\npub(crate) mod variant {{\n\tuse super::*;\n\n{variants}}}\n"
));
}
}

/// The functions object `i`'s std traits export, borrowing its handles `this`, `a`,
Expand Down Expand Up @@ -1575,6 +1584,31 @@ impl Gen<'_> {
"#[unsafe(no_mangle)]\npub unsafe extern \"C\" fn {name}_to_string(this: {handle}{param}) -> rt::Buf {{\n\tlet w = \"{name}_to_string\";\n\t{away}{body}\n}}\n"
));
}
let variants = &self.abi.api.objects[i].variants;
if !variants.is_empty() {
// Each variant by its index plus one, or `0` for one the export knows no name of,
// as a `#[non_exhaustive]` enum's own crate may add.
let arms: String = (variants.iter().enumerate())
.map(|(n, v)| format!("{}\n\t\tT::{} {{ .. }} => {},", self.arm(v.on), ident(&v.name), n + 1))
.collect();
let body = wrap(
status,
"u32",
&format!("let this = {this};\n\tmatch {deref}this {{{arms}\n\t\t_ => 0,\n\t}}"),
);
let away = away(&name, &["this"]);
let function = format!(
"#[allow(unreachable_patterns, reason = \"an exhaustive enum's wildcard\")]\npub(crate) unsafe fn {name}(this: {handle}{param}) -> u32 {{\n\ttype T = {path};\n\tlet w = \"{name}_variant\";\n\t{away}{body}\n}}\n"
);
let function = match self.cfg(self.on) {
None => function,
Some(cfg) => format!(
"#[cfg({cfg})]\n{function}#[cfg(not({cfg}))]\n#[allow(unused_variables, reason = \"a call fails before reading any\")]\npub(crate) unsafe fn {name}(this: {handle}{param}) -> u32 {{\n\trt::fail(\"{name}_variant\", \"it is not on this library's target\")\n}}\n"
),
};
self.variants.push_str(&function);
self.variants.push('\n');
}
if traits.debug {
let body = wrap(
status,
Expand Down
Loading