Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,10 @@ jobs:
- name: Export
run: nix develop --command just ffi moq

# Every sampled type shape round-tripped in every language; the gate runs a subset.
- name: Shapes
run: nix develop --command just ffi shapes

# What exports, and every refusal, ranked.
- name: Census
run: nix develop --command just ffi census
Expand Down
12 changes: 12 additions & 0 deletions ffi/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -950,6 +950,7 @@ just ffi moq # export moq's crates at the pinned revision, build them, and
just ffi moq-cpp # the same export, running only moq/moq.cpp
just ffi moq-swift # the same export, running only moq/moq.swift
just ffi census # census each into target/census
just ffi shapes # round-trip every sampled type shape in every language but KNOWN's, as the nightly does
nix develop .#android --command just ffi android # the Kotlin test, shrunk by R8, on the Android device adb reaches
nix develop .#bench --command just ffi bench # the binding benchmark: bench/ffi/README.md's cases from Rust, and through the export from C, C++, Kotlin, Python, Swift, Go, and Dart
nix develop .#sanitize --command just ffi sanitize # api's C and C++ scripts under ASan and TSan, and its C ABI under Miri
Expand Down Expand Up @@ -994,6 +995,17 @@ each runtime check, which must panic with its message; a runtime package from an
build must fail its contract check. The `target` test exports a package for the host, for wasm32, and for both into one
binding, which builds for each, and staged with `--api` and `--with`; a build for a target
not exported fails, and items that differ between the two are refused, naming both.
The shape tests (`tests/fixtures/shapes.rs`) sample value types from the IR's `Type`, each
leaf alone and in each wrapper, then nested to depth 3 from a fixed seed, and emit a crate
of `make_N(seed)`, `echo_N(v)`, and `digest_N(&v)` per shape, with the shape in a closure's
parameter and return and in a future besides. Its census decides what exports, so a shape
refused by design is left out rather than failed. Each language's driver is a generated
flat list of `digest(echo(make(seed)))` calls whose printed digests must match the crate's
own, run natively. The gate runs a subset; `just ffi shapes` runs every sampled shape, as
the nightly does. `KNOWN` lists what a binding still gets wrong, each with the quest that
fixes it; the driver skips those shapes, so the fix deletes its entry, and the nightly fails
on an entry that matches no exported shape. A new IR `Type` variant fails `ir_covered`
until the sampler covers it.
The `reject` fixture's expected errors, each with its notes, are `fixtures/reject/expected.txt`.
The `lock` test exports the `lock` fixture for wasm32, staged, and the host, then again
against that export's `api.json`: its `added` feature only adds and must pass, and its `next`
Expand Down
8 changes: 8 additions & 0 deletions ffi/fixtures/api/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2264,6 +2264,14 @@ pub fn prefixed(payload: &[u8], prefix: Option<&[u8]>) -> Vec<u8> {
[prefix.unwrap_or_default(), payload].concat()
}

/// The sum of `a`, `b`, and `c`, if any: a list owned, then lent as a slice and as itself.
pub fn sum_lists(a: Vec<i32>, b: Option<&[i32]>, c: Option<&Vec<i32>>) -> i32 {
a.iter()
.chain(b.unwrap_or_default())
.chain(c.into_iter().flatten())
.sum()
}

/// `point`'s coordinates summed, or 0 for none: an optional value, lent shared.
pub fn coordinates(point: Option<&Point>) -> i32 {
point.map_or(0, |point| point.x + point.y)
Expand Down
4 changes: 4 additions & 0 deletions ffi/justfile
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,10 @@ check:
test *args:
cargo test --locked {{ args }}

# Round-trip every sampled type shape in every language, as the nightly does; the gate runs a subset.
shapes:
cargo test --locked --test fixtures shapes::nightly -- --ignored

# Shrink the api fixture's Kotlin with R8 and its generated rules, and run its test on the Android device adb reaches, in the Android shell (nix develop .#android) with ANDROID_HOME and ANDROID_NDK_HOME set.
android:
cargo test --locked --test fixtures -- --ignored --exact api_android
Expand Down
15 changes: 8 additions & 7 deletions ffi/runtime/python.rs
Original file line number Diff line number Diff line change
Expand Up @@ -828,15 +828,16 @@ pub fn lent_bytes(buf: rt::Buf) -> R<Obj> {
}

/// A path's bytes, from a `str`, `bytes`, or `os.PathLike`, as `os.fsencode` encodes it,
/// so a name a Unix path holds that is not UTF-8 crosses as it came.
/// so a name a Unix path holds that is not UTF-8 crosses as it came, and so does a NUL,
/// which a Rust path holds and `PyUnicode_FSConverter` refuses.
pub fn path_in(obj: *mut PyObject, s: &mut Scope) -> R<rt::Buf> {
let mut encoded: *mut PyObject = null_mut();
// SAFETY: the GIL is held, and `obj` is live; the converter writes a new `bytes`.
// SAFETY: the GIL is held, and `obj` is live; each call returns a new reference.
unsafe {
if PyUnicode_FSConverter(obj, (&raw mut encoded).cast()) == 0 {
return Err(Raised);
}
let encoded = s.keep(Obj::new(encoded)?);
let path = s.keep(Obj::new(PyOS_FSPath(obj))?);
let encoded = match PyBytes_Check(path) != 0 {
true => path,
false => s.keep(Obj::new(PyUnicode_EncodeFSDefault(path))?),
};
let mut ptr: *mut c_char = null_mut();
let mut len = 0;
if PyBytes_AsStringAndSize(encoded, &mut ptr, &mut len) != 0 {
Expand Down
19 changes: 12 additions & 7 deletions ffi/runtime/rt.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -778,17 +778,22 @@ class arena {
// `items` as the C array `C`, each lowered by `lower`: never null, so an empty one is
// told apart from a missing one. Each item is mutable where `items` is, so a handle
// moved in can be owned alone.
// The array is no `std::vector`, whose `bool` form holds no array, and `items` may be
// one, whose items are proxies rather than references.
template <class C, class R, class F>
C list(R &items, F lower) {
using E = ::std::remove_const_t<::std::remove_pointer_t<decltype(C{}.ptr)>>;
auto array = ::std::make_shared<::std::vector<E>>();
array->reserve(items.size());
for (auto &item : items) {
array->push_back(lower(item));
}
static const E empty{};
const E *ptr = array->empty() ? &empty : array->data();
::std::size_t len = array->size();
::std::size_t len = items.size();
if (len == 0) {
return C{&empty, 0, nullptr};
}
::std::shared_ptr<E[]> array(new E[len]);
::std::size_t i = 0;
for (auto &&item : items) {
array[i++] = lower(item);
}
const E *ptr = array.get();
held_.push_back(::std::move(array));
return C{ptr, len, nullptr};
}
Expand Down
34 changes: 26 additions & 8 deletions ffi/src/dart.rs
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ pub fn generate(abi: &Abi<'_>, packages: &str, contract: u64) -> Result<Output,
structs: BTreeMap::new(),
natives: BTreeMap::new(),
helpers: BTreeMap::new(),
writers: HashMap::new(),
types: String::new(),
top: String::new(),
glue: String::new(),
Expand Down Expand Up @@ -375,6 +376,9 @@ struct Gen<'a> {
natives: BTreeMap<String, String>,
/// The conversions, polls, and trampolines, by name.
helpers: BTreeMap<String, String>,
/// Each writer's name, by the type it writes: types of one C struct, such as a `Vec` of
/// `SocketAddrV4` and one of `SocketAddr`, check what they write apart.
writers: HashMap<Type, String>,
/// The types, then the top-level functions and constants.
types: String,
top: String,
Expand Down Expand Up @@ -895,7 +899,7 @@ impl<'a> Gen<'a> {
Type::Slice | Type::Vec | Type::Bytes => format!("a_.bytes({dst}, {e});"),
Type::List(..) => {
let c = self.c_name(inner);
let w = self.writer(&c, inner);
let w = self.writer(&c, inner.base());
format!("{w}({e}, {dst}, a_);")
}
_ => {
Expand Down Expand Up @@ -929,10 +933,17 @@ impl<'a> Gen<'a> {

/// The function writing a Dart value of type `ty` into the C struct `c`.
fn writer(&mut self, c: &str, ty: &Type) -> String {
let name = format!("_w_{c}");
if self.helpers.contains_key(&name) {
return name;
if let Some(name) = self.writers.get(ty) {
return name.clone();
}
let mut name = format!("_w_{c}");
for k in 2.. {
if !self.helpers.contains_key(&name) {
break;
}
name = format!("_w_{c}_{k}");
}
self.writers.insert(ty.clone(), name.clone());
self.helpers.insert(name.clone(), String::new());
let class = self.struct_class(&Ty::Struct(c.to_string()));
let dt = self.dt(ty);
Expand Down Expand Up @@ -1848,10 +1859,17 @@ impl<'a> Gen<'a> {
/// The Dart statements closing each object the value `e` of type `ty` holds, once a
/// call moved them out.
fn close(&mut self, ty: &Type, e: &str) -> String {
// A local named apart from `e`, which may be a parameter named `v`, and from the
// local an outer level declared, which Dart would read before it is declared.
let v: String = e
.chars()
.map(|c| if c.is_ascii_alphanumeric() { c } else { '_' })
.collect::<String>()
+ "_";
match ty {
Type::Option(inner) => {
let close = self.close(inner, "v");
format!("{{ final v = {e}; if (v != null) {{ {close} }} }}")
let close = self.close(inner, &v);
format!("{{ final {v} = {e}; if ({v} != null) {{ {close} }} }}")
}
Type::Box(inner) => self.close(inner, e),
Type::Tuple(elements) => {
Expand All @@ -1862,8 +1880,8 @@ impl<'a> Gen<'a> {
closes.join(" ")
}
Type::List(_, inner) => {
let close = self.close(inner, "v");
format!("for (final v in {e}) {{ {close} }}")
let close = self.close(inner, &v);
format!("for (final {v} in {e}) {{ {close} }}")
}
_ => format!("{e}.close();"),
}
Expand Down
19 changes: 18 additions & 1 deletion ffi/src/glue.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1015,7 +1015,12 @@ impl Gen<'_> {
.fields
.iter()
.zip(&bindings)
.map(|(f, b)| format!("{}: {b}", f.name))
// A field named as its binding is, `f0`, takes the shorthand, which
// the lint would ask for.
.map(|(f, b)| match f.name == *b {
true => b.clone(),
false => format!("{}: {b}", f.name),
})
.collect();
format!("{path} {{ {} }}", fields.join(", "))
}
Expand Down Expand Up @@ -1367,6 +1372,18 @@ impl Gen<'_> {
format!("let {local} = {};", self.lift(ty, c)),
format!("{local}.as_deref()"),
),
// A lifted `Vec` lends itself or, by coercion to the parameter's type, its slice,
// since a borrowed `Vec` and a slice share their type.
(Type::List(Collection::Vec, _), _) => {
let (binding, borrow) = match m {
Mutability::Mut => ("let mut", "&mut"),
Mutability::Not => ("let", "&"),
};
(
format!("{binding} {local} = {};", self.lift(ty, c)),
format!("match {borrow} {local} {{ Some(v) => Some(v), None => None }}"),
)
}
_ => (
format!("let {local} = {};", self.lift(ty, c)),
format!("{local}.as_ref()"),
Expand Down
Loading