diff --git a/.github/workflows/nightly.yml b/.github/workflows/nightly.yml index 9452c2810..835fd02a1 100644 --- a/.github/workflows/nightly.yml +++ b/.github/workflows/nightly.yml @@ -189,10 +189,6 @@ jobs: - name: Export run: nix develop --command just ffi moq - # Every sampled type shape round-tripped in every language; the gate runs a subset. - - name: Shapes - run: nix develop --command just ffi shapes - # Longer random protocol scripts through the C ABI and each binding, against the # model of what each step returns; the gate runs fewer. - name: Protocol @@ -216,6 +212,25 @@ jobs: - name: Clang run: nix develop .#clang --command bash -c 'export CC=gcc; just ffi test cpp && just ffi moq-cpp' + # Every sampled type shape round-tripped in every language: the fixed set, then shapes + # drawn from the day's seed, which a failure names to reproduce it. The gate runs a + # subset. A job of its own, since it takes as long as the export's. + ffi-shapes: + name: Shapes in every language + runs-on: ubuntu-24.04-arm + timeout-minutes: 60 + + steps: + - name: Checkout + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + - uses: ./.github/actions/nix + + - name: Shapes + run: nix develop --command just ffi shapes + # The pinned corpus of popular crates, each censused, exported through a facade, and # bound in every language: a panic, an error, or a binding that does not build fails it. # Their build scripts run here, on a runner of its own holding no secret. @@ -489,7 +504,7 @@ jobs: # schedule, which runs main, gets the write token: a dispatch can run any branch. report: name: Report - needs: [bindings, gate, ts, bench, ffi-api, ffi, ffi-corpus, ffi-web, ffi-sanitize, ffi-cpp, ffi-android, ffi-bench, census] + needs: [bindings, gate, ts, bench, ffi-api, ffi, ffi-shapes, ffi-corpus, ffi-web, ffi-sanitize, ffi-cpp, ffi-android, ffi-bench, census] if: ${{ !cancelled() && github.event_name == 'schedule' }} runs-on: ubuntu-24.04 timeout-minutes: 5 diff --git a/ffi/README.md b/ffi/README.md index 25205cd5e..ad5aa9321 100644 --- a/ffi/README.md +++ b/ffi/README.md @@ -263,7 +263,10 @@ item, method, trait impl, or a variant of a `#[non_exhaustive]` enum. A constant an item's docs may change. The check runs once, on the API every language is generated from; it compares C's names and the C++, Kotlin, Python, Swift, Go, and Dart generators' own public names. A clash that renames a kept class, function, or member reports its language -and old and new names at the item's span. Staged `--api` files have no binding names; a +and old and new names at the item's span. An error type's exception class is known by its +error type and the type the language holds it as, since Kotlin's `UShortArray` and +`List` may share one error type: one added beside another adds a class, and a +native type respelled or gone breaks nothing while a class still bears its old name. Staged `--api` files have no binding names; a baseline must be a complete export. An older baseline without name tables still has its structure checked, reporting each break before the error that it lacks names, so its re-export with this version accepts no break that check would have caught. A binding @@ -1348,7 +1351,8 @@ runtime package and each crate's include [`runtime/rt.go`](runtime/rt.go) and ## Census `--census dir` exports nothing. It writes `.md` for the facade, which counts what exports and ranks every refusal by construct and by the type it names, -lists each refused item at its span, then the `exclude` list that exports the rest, and +lists each refused item at its span, a binding's own refusals such as Swift's included, +then the `exclude` list that exports the rest, and what else it left out, such as what C can never hold and the traits C cannot implement, and why. That list is checked by collecting again with it: it holds what Swift alone refuses, and not an instance only refused items named, which nothing names once they are left out. @@ -1566,15 +1570,20 @@ binding, which builds for each, and staged with `--api` and `--with`; a build fo not exported fails, and items that differ between the two are refused, naming both. The shape tests (`tests/fixtures/shapes.rs`) sample value types from the IR's `Type`, each leaf alone and in each wrapper, then nested to depth 3 from a fixed seed, and emit a crate -of `make_N(seed)`, `echo_N(v)`, and `digest_N(&v)` per shape, with the shape in a closure's -parameter and return and in a future besides. Its census decides what exports, so a shape -refused by design is left out rather than failed. Each language's driver is a generated -flat list of `digest(echo(make(seed)))` calls whose printed digests must match the crate's -own, run natively. The gate runs a subset; `just ffi shapes` runs every sampled shape, as -the nightly does. `KNOWN` lists what a binding still gets wrong, each with the quest that -fixes it; the driver skips those shapes, so the fix deletes its entry, and the nightly fails -on an entry that matches no exported shape. A new IR `Type` variant fails `ir_covered` -until the sampler covers it. +of `make_N(seed)`, `echo_N(v)`, and `digest_N(&v)` per shape, with the shape in each +position a binding maps apart: a closure's parameter and return, a future, a `Result`'s +value and error, a `&mut` written back, beside other parameters, an `Option<&T>`, an +object's methods, and a trait a caller implements. A digest hashes floats by their bits, +so a NaN's payload counts, and the sampler draws signaling and quiet NaNs. Its census +decides what exports, so a shape refused by design is left out rather than failed. Each +language's driver, the web's on Node included, is a generated list of these calls whose +printed digests must match the crate's own, run natively, or as wasm under Node's WASI +for the web, where `usize` is 32 bits. The gate runs a subset; `just ffi shapes` runs +every sampled shape and, as the nightly does, more drawn from the day's seed, which it +prints and `RUST2_SHAPES_SEED` replays. `KNOWN` lists what a binding still gets wrong, +each with the quest that fixes it; the driver skips those calls, so the fix deletes its +entry, and the nightly fails on an entry that matches no call of the fixed set. A new IR +`Type` variant fails `ir_covered` until the sampler covers it. The corpus (`tests/fixtures/corpus.rs`) exports stock crates as a consumer would: each crate of `corpus/crates.txt`, the top crates on crates.io by downloads that are libraries building as they are, pinned by `corpus/Cargo.lock`, through a facade of its own, `pub use diff --git a/ffi/fixtures/api/ffi.toml b/ffi/fixtures/api/ffi.toml index 27026dcd6..8550166ab 100644 --- a/ffi/fixtures/api/ffi.toml +++ b/ffi/fixtures/api/ffi.toml @@ -4,7 +4,8 @@ # converts into them, and # `field@` leaves out a field alone where a method takes its path, as `Retry`'s and # `Slot`'s do, and `Retry::jitter` and `field@Retry::jitter` leave out a refused method -# and its field. `Slot` has no such method, so its path alone names its field. A +# and its field, as `Retry::spread` and `field@Retry::spread` do a method Swift alone +# refuses and its field. `Slot` has no such method, so its path alone names its field. A # function instance is left out by its path, as `depth_past` at `polar::Point` is, and a # method a `Deref` forwards by the object's, as `Pad::lines` is. A forwarded field # sharing its path with a method is `field@`, as `Pad`'s `title` is, so `Marker::x` @@ -25,9 +26,11 @@ exclude = [ "Scoped", "Slot::value", "Retry::jitter", + "Retry::spread", "field@Pad::title", "field@Retry::delay", "field@Retry::jitter", + "field@Retry::spread", "field@Slot::value", "Stack::depth_past", "Stored", diff --git a/ffi/fixtures/api/src/lib.rs b/ffi/fixtures/api/src/lib.rs index b9f325e25..6fbc5931f 100644 --- a/ffi/fixtures/api/src/lib.rs +++ b/ffi/fixtures/api/src/lib.rs @@ -2843,6 +2843,9 @@ pub struct Retry { /// Its field exports, but the method of its name is refused, so the config excludes /// both, and the census lists only `field@Retry::jitter` as excluding what exports. pub jitter: u32, + /// Its field exports, but Swift refuses the method of its name, so the config excludes + /// both, and the census lists only `field@Retry::spread` as excluding what exports. + pub spread: u64, } impl Retry { @@ -2851,6 +2854,7 @@ impl Retry { limit, delay: None, jitter: 0, + spread: 0, } } @@ -2866,6 +2870,11 @@ impl Retry { self.limit = Some(limit); } + /// The spread plus three optional borrowed counters, past what Swift takes. + pub fn spread(&self, first: Option<&Counter>, second: Option<&Counter>, third: Option<&Counter>) -> u64 { + self.spread + among(first, second, third) + } + /// The jitter capped by `cap`, if any: a nested `Option`, which the export refuses. pub fn jitter(&self, cap: Option>>) -> u32 { cap.and_then(|cap| *cap).map_or(self.jitter, |cap| self.jitter.min(cap)) diff --git a/ffi/fixtures/lock/expected.txt b/ffi/fixtures/lock/expected.txt index b356f8fc0..e84446f72 100644 --- a/ffi/fixtures/lock/expected.txt +++ b/ffi/fixtures/lock/expected.txt @@ -9,9 +9,9 @@ fixtures/lock/src/lib.rs:202:1: rust2-ffi: the API breaks the baseline at struct fixtures/lock/src/lib.rs:21:2: rust2-ffi: the API breaks the baseline at const `lock::limits_max::RATE`: its C name `LOCK_LIMITS_MAX_RATE` is now `LOCK_LIMITS_MAX_RATE_`, and `LOCK_LIMITS_MAX_RATE` names const `lock::limits::MAX_RATE` fixtures/lock/src/lib.rs:231:1: rust2-ffi: the API breaks the baseline at struct `lock::Beat`: how it is held: a word compared by its bits, and now bytes compared by Rust fixtures/lock/src/lib.rs:30:1: rust2-ffi: the API breaks the baseline at fn `lock::removed`: it is removed -fixtures/lock/src/lib.rs:356:1: rust2-ffi: the API breaks the baseline at enum `lock::Speed`: variant `Fast` is removed; variant `Quick` is added to an exhaustive enum fixtures/lock/src/lib.rs:36:1: rust2-ffi: the API breaks the baseline at fn `lock::portable`: it is no longer on wasm32-unknown-unknown -fixtures/lock/src/lib.rs:379:1: rust2-ffi: the API breaks the baseline at type `lock::Gauge`: its handle is now checked at run time, so each call taking it gains a status, and throws in Swift +fixtures/lock/src/lib.rs:373:1: rust2-ffi: the API breaks the baseline at enum `lock::Speed`: variant `Fast` is removed; variant `Quick` is added to an exhaustive enum +fixtures/lock/src/lib.rs:396:1: rust2-ffi: the API breaks the baseline at type `lock::Gauge`: its handle is now checked at run time, so each call taking it gains a status, and throws in Swift fixtures/lock/src/lib.rs:46:1: rust2-ffi: the API breaks the baseline at fn `lock::scale`: its parameter `x`'s type; its return type fixtures/lock/src/lib.rs:66:2: rust2-ffi: the API breaks the baseline at fn `lock::Meter::add`: its parameter `n`'s type fixtures/lock/src/lib.rs:7:2: rust2-ffi: the API breaks the baseline at fn `lock::audio_encoder::finish`: its C name `lock_audio_encoder_finish` is now `lock_audio_encoder_finish_`, and `lock_audio_encoder_finish` names fn `lock::audio::encoder_finish` diff --git a/ffi/fixtures/lock/src/lib.rs b/ffi/fixtures/lock/src/lib.rs index bd083a004..7d1225202 100644 --- a/ffi/fixtures/lock/src/lib.rs +++ b/ffi/fixtures/lock/src/lib.rs @@ -289,6 +289,23 @@ pub fn failing_borrowed() -> Result { Err("failure") } +/// An error whose key, `vec_u16`, `added` gives a `BTreeSet` too, which Kotlin, +/// Python, and Dart hold as another type and so throw as an exception of its own: an +/// addition. A `Vec>` shares its C type, which Dart holds apart. +pub fn samples() -> Result> { + Err(vec![1]) +} + +#[cfg(feature = "added")] +pub fn samples_set() -> Result> { + Err([1].into()) +} + +#[cfg(feature = "added")] +pub fn samples_nonzero() -> Result>> { + Err(vec![std::num::NonZero::::MIN]) +} + /// Error wrappers and variants occupy their own language namespaces. #[cfg(not(feature = "closures"))] pub fn failing() -> Result { diff --git a/ffi/src/dart.rs b/ffi/src/dart.rs index 23b1522ed..2b2f992a8 100644 --- a/ffi/src/dart.rs +++ b/ffi/src/dart.rs @@ -2713,18 +2713,24 @@ impl<'a> Gen<'a> { Thrown::Itself } _ => { + // One exception for each C type and Dart type: a `Vec>` shares + // `Vec`'s C type, but Dart holds it as a `List`, not a `Uint16List`. let key = self.abi.key(err); - let seen = self.thrown.iter().find(|(_, seen)| self.abi.key(seen) == key); + let dt = self.dt(err); + let thrown: Vec<(String, Type)> = (self.thrown.iter()) + .filter(|(_, seen)| self.abi.key(seen) == key) + .map(|(name, seen)| (name.clone(), seen.clone())) + .collect(); + let seen = thrown.into_iter().find(|(_, seen)| self.dt(seen) == dt); let name = match seen { - Some((name, _)) => name.clone(), + Some((name, _)) => name, None => { - let dt = self.dt(err); let name = self.claim(&format!("{}Exception", pascal(&key)), format!("the error type `{dt}`")); self.thrown.insert(name.clone(), err.clone()); name } }; - self.locked.insert(crate::lock::error_key(self.abi, err), name.clone()); + crate::lock::exception(&mut self.locked, self.abi, err, &dt, &name); Thrown::Wrapped(name) } } diff --git a/ffi/src/driver.rs b/ffi/src/driver.rs index fa284181f..8ee0dec4d 100644 --- a/ffi/src/driver.rs +++ b/ffi/src/driver.rs @@ -191,22 +191,18 @@ impl Callbacks for Export { Output::Dir(_) | Output::Api(_) => &[], }, }; - let collected = collect::collect(tcx, &facade); - let report = &collected.report; + let mut collected = collect::collect(tcx, &facade); if let Output::Census(dir) = &self.output { let abi = Abi::new(&collected.api, &self.facade.lib); - // What Swift refuses needs its exclude, as the export says, so it is not exporting. - // Swift exports the API without what takes a JNI value, as the export does. - let rest = collected.api.without_jni(); - let swift: FxHashSet<_> = crate::swift::refusals(&abi.without_jni(&rest)) - .into_iter() - .map(|(f, _)| f.shown.clone()) - .collect(); - let exporting = exporting(&collected.api, &report.audited, &swift); - let config = census_config(tcx, &facade, &self.facade.lib, report, &swift); - let text = census::render(&collected.api, report, &config, &exporting, &abi.clashes, &|span| { - at(tcx, span) - }); + // What Swift refuses needs its exclude, as the export says, so it is a refusal and + // not exporting. + let (api, swift) = swift_refused(&collected.api, &self.facade.lib); + let refused: FxHashSet<_> = swift.iter().map(|r| r.path.clone()).collect(); + let report = &mut collected.report; + report.refusals.extend(swift.into_iter().map(|r| r.refusal)); + let exporting = exporting(&collected.api, &report.audited, &refused); + let config = census_config(tcx, &facade, &self.facade.lib, report); + let text = census::render(&api, report, &config, &exporting, &abi.clashes, &|span| at(tcx, span)); // A census reports what a crate uses, but a missed instance is the exporter's bug. for refusal in &report.refusals { if refusal.why.iter().any(|why| why.construct == collect::NOT_INFERRED) { @@ -225,6 +221,7 @@ impl Callbacks for Export { } return stop_on_errors(tcx); } + let report = &collected.report; for refusal in &report.refusals { refuse(tcx, refusal, "exclude it in the config to leave it out"); } @@ -261,23 +258,76 @@ impl Callbacks for Export { } } -/// The `exclude` that exports what maps: each item collection or Swift refuses, checked by -/// collecting again with it. One naming only an instance a refused item alone named names -/// nothing once that item is left out, and the export would fail on it, so it is dropped; -/// what that collection refuses besides is added, until the config holds. -fn census_config( - tcx: TyCtxt<'_>, - facade: &collect::Facade, - lib: &str, - report: &collect::Report, - swift: &FxHashSet, -) -> Vec { - let refused = |report: &collect::Report, swift: &FxHashSet| -> Vec { - (report.refusals.iter().map(|r| r.path.clone())) - .chain(swift.iter().map(|path| path[1..].join("::"))) - .collect() - }; - let mut config = refused(report, swift); +/// A function or method Swift refuses, by the path the config names it by, as a census +/// refusal. +struct SwiftRefused { + path: ir::Path, + refusal: collect::Refusal, +} + +/// `api` without what Swift refuses, and each it refuses, which the census adds to its +/// config: to a fixed point, since leaving out a function that throws an object bare can +/// make that object noncopyable, refusing another that throws it inside a value. Swift +/// exports the API without what takes a JNI value, as the export does. +fn swift_refused(api: &Api, lib: &str) -> (Api, Vec) { + let mut api = api.clone(); + let mut refused = Vec::new(); + loop { + let abi = Abi::new(&api, lib); + let rest = api.without_jni(); + // A function refused for each of several reasons is one refusal of them all, its + // reasons in a row. + let mut found: Vec = Vec::new(); + for r in crate::swift::refusals(&abi.without_jni(&rest)) { + let why = collect::Why { + construct: r.construct, + detail: String::new(), + note: None, + }; + let path = configured(r.function); + match found.last_mut() { + Some(last) if last.path == path => last.refusal.why.push(why), + _ => found.push(SwiftRefused { + refusal: collect::Refusal { + path: path[1..].join("::"), + kind: r.kind, + span: r.function.span, + why: vec![why], + }, + path, + }), + } + } + if found.is_empty() { + return (api, refused); + } + let gone: FxHashSet<&ir::Path> = found.iter().map(|r| &r.path).collect(); + let keep = |methods: &mut Vec| methods.retain(|f| !gone.contains(&configured(f))); + api.records.iter_mut().for_each(|r| keep(&mut r.methods)); + api.enums.iter_mut().for_each(|e| keep(&mut e.methods)); + api.objects.iter_mut().for_each(|o| keep(&mut o.methods)); + keep(&mut api.functions); + refused.extend(found); + } +} + +/// The path the config names `f` by: a field's getter and setter alike, by `field@` where +/// a method or constant of its type shares its path, so refusing that one keeps the other. +fn configured(f: &ir::Function) -> ir::Path { + let mut path = f.shown.clone(); + if let ir::Access::Field { shared: true, .. } = f.access { + path[1].insert_str(0, "field@"); + } + path +} + +/// The `exclude` that exports what maps: each item `report` refuses, Swift's included, +/// checked by collecting again with it. One naming only an instance a refused item alone +/// named names nothing once that item is left out, and the export would fail on it, so it +/// is dropped; what that collection or Swift refuses besides is added, until the config +/// holds. +fn census_config(tcx: TyCtxt<'_>, facade: &collect::Facade, lib: &str, report: &collect::Report) -> Vec { + let mut config: Vec = report.refusals.iter().map(|r| r.path.clone()).collect(); let mut seen = Vec::new(); loop { config.sort(); @@ -310,13 +360,10 @@ fn census_config( audit: &[], }, ); - let abi = Abi::new(&checked.api, lib); - let rest = checked.api.without_jni(); - let swift: FxHashSet<_> = crate::swift::refusals(&abi.without_jni(&rest)) - .into_iter() - .map(|(f, _)| f.shown.clone()) - .collect(); - let more = refused(&checked.report, &swift); + let (_, swift) = swift_refused(&checked.api, lib); + let more = (checked.report.refusals.iter().map(|r| r.path.clone())) + .chain(swift.into_iter().map(|r| r.refusal.path)) + .collect::>(); let dead: Vec<&String> = (checked.report.excluded.iter()) .filter(|(path, count)| *count == 0 && !configured_out(tcx, path)) .map(|(path, _)| path) @@ -328,7 +375,7 @@ fn census_config( /// Each exclude whose item would export without it, and that item's paths: what an /// exclude checked, from [`collect::Report::audited`], that the census's API holds by the -/// same path, except a function `refused` names by its shown path, since a generic's +/// same path, except a function `refused` names by its configured path, since a generic's /// instances share a span but Swift may refuse only some. fn exporting( api: &Api, @@ -341,7 +388,7 @@ fn exporting( paths.insert(path[1..].join("::")); }; let methods = |methods: &[ir::Function], add: &mut dyn FnMut(&[String])| { - for f in methods.iter().filter(|f| !refused.contains(&f.shown)) { + for f in methods.iter().filter(|f| !refused.contains(&configured(f))) { match f.access { // A field is checked by `field@` and, unless a method or constant takes its // name, by its path, which then names that method or constant instead. @@ -552,10 +599,15 @@ impl Export { } // Swift exports the API without what takes a JNI value, which only Kotlin passes. let rest = api.without_jni(); - for (function, why) in crate::swift::refusals(&abi.without_jni(&rest)) { + for refusal in crate::swift::refusals(&abi.without_jni(&rest)) { tcx.dcx().span_err( - function.span, - format!("rust2-ffi: {why}; exclude it in the config to leave it out"), + refusal.function.span, + format!( + "rust2-ffi: cannot export {} `{}`: {}; exclude it in the config to leave it out", + refusal.kind, + refusal.function.shown[1..].join("::"), + refusal.why + ), ); } let can_generate = tcx.dcx().has_errors().is_none(); diff --git a/ffi/src/kotlin.rs b/ffi/src/kotlin.rs index dd0aab07d..87a43ce83 100644 --- a/ffi/src/kotlin.rs +++ b/ffi/src/kotlin.rs @@ -2289,14 +2289,14 @@ impl<'a> Gen<'a> { fn exception(&mut self, e: &Type) -> String { let kt = self.kt(e); if let Some(name) = self.thrown.get(&kt) { - self.locked.insert(crate::lock::error_key(self.abi, e), name.clone()); + crate::lock::exception(&mut self.locked, self.abi, e, &kt, name); return name.clone(); } // Named by the type as it reads unqualified, so qualifying it renames no exception. let bare = self.unqualified(&kt); let name = format!("{}Exception", bare.replace(['?', '<', '>', ',', ' ', '.'], "")); let name = self.claim(&name, format!("the exception of `{kt}`")); - self.locked.insert(crate::lock::error_key(self.abi, e), name.clone()); + crate::lock::exception(&mut self.locked, self.abi, e, &kt, &name); self.exceptions.insert(name.clone(), e.clone()); self.thrown.insert(kt, name.clone()); name diff --git a/ffi/src/lock.rs b/ffi/src/lock.rs index 5c4dca59b..c64ffb511 100644 --- a/ffi/src/lock.rs +++ b/ffi/src/lock.rs @@ -79,6 +79,19 @@ pub(crate) fn error_key(abi: &Abi<'_>, ty: &Type) -> String { format!("error {}", abi.spell(&owned(ty))) } +/// Record `name`, the exception class an error of type `err` throws as, which the language +/// holds as `native`. One error key may name several classes, as Kotlin's `UShortArray` and +/// `List` both spell `vec_u16`, so each is known by its native type too: a type +/// added beside it adds a class rather than renaming one, and a class that loses its name to +/// it is a rename. The key alone names the last, as a baseline from before knew it. A +/// generator's spelling of `native` may change, so [`name_breaks`] holds the key alone, or a +/// native type gone from it, kept while any class of the key still bears its name. +pub(crate) fn exception(names: &mut BTreeMap, abi: &Abi<'_>, err: &Type, native: &str, name: &str) { + let key = error_key(abi, err); + names.insert(format!("{key} / as {native}"), name.to_string()); + names.insert(key, name.to_string()); +} + /// Compare the names the generators actually emitted, retaining the current item's span. /// /// An item whose C name `base_api`, the baseline's API, gave it changed is already a structural @@ -173,6 +186,18 @@ pub(crate) fn name_breaks(base: &Names, now: &Names, base_api: &Api, abi: &Abi<' continue; } let new = now.get(language).and_then(|names| names.get(key)); + // An exception known by its key alone names one class of several the key may have, + // as a baseline from before they were known apart wrote it, and one known by a + // native type no class has now was respelled or dropped: kept while one has it. + if key.starts_with("error ") + && (key == owner || new.is_none()) + && now.get(language).is_some_and(|names| { + (names.range(format!("{owner} / as ")..)) + .take_while(|(k, _)| k.starts_with(&format!("{owner} / as "))) + .any(|(_, name)| name == old) + }) { + continue; + } // Missing parts need a name diagnostic unless the owner's structural change // already reports the break, such as a removed variant or changed signature. if new.is_none() diff --git a/ffi/src/python.rs b/ffi/src/python.rs index 57c6b6053..0cf7e3f45 100644 --- a/ffi/src/python.rs +++ b/ffi/src/python.rs @@ -1545,7 +1545,7 @@ impl<'a> Gen<'a> { thrown } }; - self.locked.insert(crate::lock::error_key(self.abi, e), thrown.clone()); + crate::lock::exception(&mut self.locked, self.abi, e, &py, &thrown); thrown } diff --git a/ffi/src/swift.rs b/ffi/src/swift.rs index 0a3087c92..952f7a2fb 100644 --- a/ffi/src/swift.rs +++ b/ffi/src/swift.rs @@ -787,7 +787,7 @@ impl<'a> Gen<'a> { }; let _ = write!( self.types, - "/// A `{}` Rust returned as an error, or a closure throws for Rust to return.\npublic struct {name}: Swift.Error{unchecked}, CustomStringConvertible {{\n\tpublic let error: {sw}\n\n\tpublic init(error: {sw}) {{\n\t\tself.error = error\n\t}}\n\n\tpublic var description: Swift.String {{\n\t\t\"\\(error)\"\n\t}}\n}}\n\n", + "/// A `{}` Rust returned as an error, or a closure throws for Rust to return.\npublic struct {name}: Swift.Error{unchecked}, CustomStringConvertible {{\n\tpublic let error: {sw}\n\n\tpublic init(error: {sw}) {{\n\t\tself.error = error\n\t}}\n\n\tpublic var description: Swift.String {{\n\t\tSwift.String(describing: error)\n\t}}\n}}\n\n", sw.trim_start_matches("Swift.") ); } @@ -2259,11 +2259,21 @@ fn thrown_value(thrown: &Thrown, lifted: &str) -> String { /// overloads. const MAX_OPTIONAL: usize = 2; -/// Each function or method Swift cannot export, and why, which the driver -/// refuses before any language is generated: one taking more optional borrowed structs -/// than [`MAX_OPTIONAL`], or returning an error that holds one, which `Swift.Error` -/// requires be copyable. -pub(crate) fn refusals<'a>(abi: &Abi<'a>) -> Vec<(&'a Function, String)> { +/// A function or method Swift cannot export. +pub(crate) struct Refusal<'a> { + pub(crate) function: &'a Function, + /// `fn` or `method`. + pub(crate) kind: &'static str, + /// What it uses that Swift cannot export, as the census ranks it. + pub(crate) construct: &'static str, + /// Why, in full. + pub(crate) why: String, +} + +/// Each function or method Swift cannot export, which the driver refuses before any +/// language is generated: one taking more optional borrowed structs than [`MAX_OPTIONAL`], +/// or returning an error that holds one, which `Swift.Error` requires be copyable. +pub(crate) fn refusals<'a>(abi: &Abi<'a>) -> Vec> { let structs = structs(abi); let api = abi.api; let methods = (api.records.iter().flat_map(|r| &r.methods)) @@ -2271,23 +2281,26 @@ pub(crate) fn refusals<'a>(abi: &Abi<'a>) -> Vec<(&'a Function, String)> { .chain(api.objects.iter().flat_map(|o| &o.methods)); let mut refusals = Vec::new(); for (kind, function) in (api.functions.iter().map(|f| ("fn", f))).chain(methods.map(|f| ("method", f))) { - let what = format!("cannot export {kind} `{}`", function.shown[1..].join("::")); let optional = (function.params.iter()) .filter(|p| lent_option(&structs, &p.ty)) .count(); if optional > MAX_OPTIONAL { - refusals.push(( + refusals.push(Refusal { function, - format!( - "{what}: {optional} optional borrowed objects, past the {MAX_OPTIONAL} Swift takes, which overloads each with and without it since its optional holds an object only by consuming it" + kind, + construct: "optional borrowed objects past what Swift overloads", + why: format!( + "{optional} optional borrowed objects, past the {MAX_OPTIONAL} Swift takes, which overloads each with and without it since its optional holds an object only by consuming it" ), - )); + }); } if function.err.as_ref().is_some_and(|err| unique(&structs, err)) { - refusals.push(( + refusals.push(Refusal { function, - format!("{what}: error holding an object Swift cannot copy, as `Swift.Error` requires"), - )); + kind, + construct: "error holding an object Swift cannot copy", + why: "error holding an object Swift cannot copy, as `Swift.Error` requires".into(), + }); } } refusals diff --git a/ffi/src/web.rs b/ffi/src/web.rs index c71d222a9..b4e70a535 100644 --- a/ffi/src/web.rs +++ b/ffi/src/web.rs @@ -1325,7 +1325,9 @@ impl<'a> Gen<'a> { Type::List(_, elem) => { let l = self.layout(&abi.ty(elem)); let stride = round(l.size, l.align); - let item = self.put(elem, "v[i]", &format!("ptr + i * {stride}"), false); + // The element bound, since TypeScript narrows no `v[i]` past an `undefined` check. + let item = self.put(elem, "x", &format!("ptr + i * {stride}"), false); + let item = format!("const x = v[i];\n{item}"); format!( "if (v === undefined) {{\n\tconst d = rt.dv();\n\td.setUint32(p, 0, true);\n\td.setUint32(p + 4, 0, true);\n\td.setUint32(p + 8, 0, true);\n\treturn;\n}}\nconst ptr = rt.alloc(Math.max(v.length * {stride}, 1), {});\nfor (let i = 0; i < v.length; i++) {{\n{}\n}}\nconst d = rt.dv();\nd.setUint32(p, ptr, true);\nd.setUint32(p + 4, v.length, true);\nd.setUint32(p + 8, 0, true);", l.align, diff --git a/ffi/tests/fixtures.rs b/ffi/tests/fixtures.rs index c27968b00..2a1c883ab 100644 --- a/ffi/tests/fixtures.rs +++ b/ffi/tests/fixtures.rs @@ -1842,6 +1842,13 @@ fn swift_protocol(scripts: &[protocol::Script]) { protocol::check(&dir.join("scripts"), scripts, |_| Command::new(&exe)); } +/// Node's script running the WASI program its first argument names. +const WASI_MAIN: &str = "import { WASI } from 'node:wasi';\n\ + import { readFileSync } from 'node:fs';\n\ + const wasi = new WASI({ version: 'preview1', args: [], env: {}, returnOnExit: false });\n\ + const module = await WebAssembly.compile(readFileSync(process.argv[1]));\n\ + wasi.start(await WebAssembly.instantiate(module, wasi.getImportObject()));\n"; + /// The fixture's script compiled for wasm32-wasip1 and run under Node's WASI: native Rust on /// the target the web binding serves, whose `usize` is 32 bits. fn wasm_transcript() -> String { @@ -1870,13 +1877,8 @@ fn wasm_transcript() -> String { .join("debug") .join("examples") .join("transcript.wasm"); - let script = "import { WASI } from 'node:wasi';\n\ - import { readFileSync } from 'node:fs';\n\ - const wasi = new WASI({ version: 'preview1', args: [], env: {}, returnOnExit: false });\n\ - const module = await WebAssembly.compile(readFileSync(process.argv[1]));\n\ - wasi.start(await WebAssembly.instantiate(module, wasi.getImportObject()));\n"; let transcript = run(Command::new("node") - .args(["--no-warnings", "--input-type=module", "-e", script]) + .args(["--no-warnings", "--input-type=module", "-e", WASI_MAIN]) .arg(wasm)); String::from_utf8_lossy(&transcript.stdout).into_owned() } @@ -3325,16 +3327,21 @@ fn relative_target() { errors.iter().all(|error| error.ends_with(", which exports")), "the census failed otherwise:\n{stderr}" ); - assert!( - errors.contains( - &"error: rust2-ffi: exclude `field@Retry::jitter` in the config leaves out `field@Retry::jitter`, which exports" - ), - "the census does not fail on `field@Retry::jitter`:\n{stderr}" - ); - assert!( - !errors.iter().any(|error| error.contains("exclude `Retry::jitter`")), - "the census fails on `Retry::jitter`:\n{stderr}" - ); + // Nor on a method Swift alone refuses, though not its field, which Swift keeps. + for exclude in ["jitter", "spread"] { + assert!( + errors.contains(&format!( + "error: rust2-ffi: exclude `field@Retry::{exclude}` in the config leaves out `field@Retry::{exclude}`, which exports" + ).as_str()), + "the census does not fail on `field@Retry::{exclude}`:\n{stderr}" + ); + assert!( + !errors + .iter() + .any(|error| error.contains(&format!("exclude `Retry::{exclude}`"))), + "the census fails on `Retry::{exclude}`:\n{stderr}" + ); + } assert!( !errors.iter().any(|error| error.contains("exclude `among`")), "the census fails on `among`, which Swift refuses:\n{stderr}" @@ -3356,6 +3363,7 @@ fn relative_target() { "- `Pair::swap`: `Pair::swap`", "- `Slot::value`: `Slot::value`", "- `field@Retry::jitter`: `field@Retry::jitter`", + "- `field@Retry::spread`: `field@Retry::spread`", "- `Stack::depth_past`: `Stack::depth_past`", ] { assert!(text.contains(line), "the census lists no `{line}`:\n{text}"); @@ -3372,6 +3380,7 @@ fn relative_target() { "- `Queued`", "- `Layer`", "- `Retry::jitter`", + "- `Retry::spread`", ] { assert!( !exporting.contains(refused), @@ -3709,6 +3718,58 @@ fn lock() { "an addition broke the baseline:\n{}", String::from_utf8_lossy(&added.stderr) ); + // A baseline from before an exception was known by its native type too, whose key alone + // names one class of those an addition gives it. + let original: serde_json::Value = serde_json::from_slice(&before).unwrap(); + let mut legacy = original.clone(); + for names in legacy["names"].as_object_mut().unwrap().values_mut() { + names.as_object_mut().unwrap().retain(|key, _| !key.contains(" / as ")); + } + assert_ne!(legacy, original, "the baseline holds no native exception key to drop"); + let path = dir.join("legacy.json"); + std::fs::write(&path, serde_json::to_vec(&legacy).unwrap()).unwrap(); + let args = ["--baseline".as_ref(), path.as_os_str()]; + let output = exporter("lock", &dir.join("legacy"), &targets, &args, &["--features", "added"]) + .output() + .unwrap(); + assert!( + output.status.success(), + "an addition broke a baseline without native exception keys:\n{}", + String::from_utf8_lossy(&output.stderr) + ); + // A generator that respells a native type, such as an alias it renames, renames no class. + let mut respelled = original.clone(); + for names in respelled["names"].as_object_mut().unwrap().values_mut() { + let names = names.as_object_mut().unwrap(); + *names = std::mem::take(names) + .into_iter() + .map(|(key, name)| match key.contains(" / as ") { + true => (format!("{key}_respelled"), name), + false => (key, name), + }) + .collect(); + } + assert_ne!( + respelled, original, + "the baseline holds no native exception key to respell" + ); + let path = dir.join("respelled.json"); + std::fs::write(&path, serde_json::to_vec(&respelled).unwrap()).unwrap(); + let args = ["--baseline".as_ref(), path.as_os_str()]; + let output = exporter( + "lock", + &dir.join("respelled"), + &targets, + &args, + &["--features", "added"], + ) + .output() + .unwrap(); + assert!( + output.status.success(), + "respelling a native exception type broke the baseline:\n{}", + String::from_utf8_lossy(&output.stderr) + ); let names = locked("names"); assert!(!names.status.success(), "binding renames passed the lock"); let stderr = String::from_utf8_lossy(&names.stderr); diff --git a/ffi/tests/fixtures/shapes.rs b/ffi/tests/fixtures/shapes.rs index 1a2afb65e..9ecc33f18 100644 --- a/ffi/tests/fixtures/shapes.rs +++ b/ffi/tests/fixtures/shapes.rs @@ -206,12 +206,11 @@ fn is_ord(shape: &Shape) -> bool { ) } -/// The shapes a run covers: each leaf alone, each leaf in each wrapper, then `extra` +/// The fixed set [`KNOWN`] checks: each leaf alone, each leaf in each wrapper, then `extra` /// sampled to depth 3 from a fixed seed. fn shapes(extra: usize) -> Vec { let leaves = leaves(); let mut shapes = leaves.clone(); - let mut rng = Rng(0x5eed); for leaf in &leaves { let inner = || Box::new(leaf.clone()); shapes.extend([ @@ -228,10 +227,56 @@ fn shapes(extra: usize) -> Vec { shapes.push(Shape::Map(inner(), Box::new(Shape::String))); } } - shapes.extend((0..extra).map(|_| sample(&mut rng, &leaves, 3))); + shapes.extend(drawn(0x5eed, extra)); shapes } +/// `n` shapes sampled to depth 3 from `seed`. +fn drawn(seed: u64, n: usize) -> Vec { + let leaves = leaves(); + let mut rng = Rng(seed); + (0..n).map(|_| sample(&mut rng, &leaves, 3)).collect() +} + +/// The seed the nightly draws its new shapes from: `RUST2_SHAPES_SEED`, to reproduce a run, +/// else the day's UTC date as the digits `YYYYMMDD`, so each night explores new shapes. +fn day_seed() -> u64 { + if let Some(seed) = std::env::var_os("RUST2_SHAPES_SEED") { + let seed = seed.to_str().and_then(|s| s.parse().ok()); + return seed.expect("RUST2_SHAPES_SEED is a u64"); + } + let secs = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .expect("the clock is past 1970") + .as_secs(); + let (year, month, day) = civil((secs / 86_400) as i64); + (year * 10_000 + month * 100 + day) as u64 +} + +/// The proleptic Gregorian date of `days` since 1970-01-01, after Howard Hinnant's +/// `civil_from_days`. +fn civil(days: i64) -> (i64, i64, i64) { + let z = days + 719_468; + let era = z.div_euclid(146_097); + let doe = z.rem_euclid(146_097); + let yoe = (doe - doe / 1460 + doe / 36_524 - doe / 146_096) / 365; + let doy = doe - (365 * yoe + yoe / 4 - yoe / 100); + let mp = (5 * doy + 2) / 153; + let day = doy - (153 * mp + 2) / 5 + 1; + let month = if mp < 10 { mp + 3 } else { mp - 9 }; + (yoe + era * 400 + i64::from(month <= 2), month, day) +} + +#[test] +fn civil_dates() { + assert_eq!(civil(0), (1970, 1, 1)); + assert_eq!(civil(-1), (1969, 12, 31)); + // A leap day, and the day after one a century skips. + assert_eq!(civil(11_016), (2000, 2, 29)); + assert_eq!(civil(47_541), (2100, 3, 1)); + assert_eq!(civil(20_776), (2026, 11, 19)); +} + /// The gate's subset: each leaf alone, and each wrapper around a buffer, a /// record field, and an object. fn gate() -> Vec { @@ -344,6 +389,11 @@ impl Emit { let _ = writeln!(self.items, "\tpub f{i}: {ty},"); } let _ = writeln!(self.items, "}}\n"); + let fed: String = (0..types.len()).map(|i| format!(" self.f{i}.feed(h);")).collect(); + let _ = writeln!( + self.items, + "impl Feed for {name} {{\n\tfn feed(&self, h: &mut Fnv) {{\n\t\th.put(\"{name}{{\");{fed}\n\t\th.put(\"}}\");\n\t}}\n}}\n" + ); let fields: Vec = makes.iter().enumerate().map(|(i, m)| format!("f{i}: {m}")).collect(); (name.clone(), format!("{name} {{ {} }}", fields.join(", "))) } @@ -352,7 +402,11 @@ impl Emit { let name = self.name("Enum"); let _ = writeln!( self.items, - "#[derive(Debug)]\npub enum {name} {{\n\tUnit,\n\tTuple({ty}),\n\tStruct {{ value: {ty} }},\n}}\n" + "#[derive(Debug)]\npub enum {name} {{\n\tUnit,\n\tTuple({ty}),\n\tStruct {{ value: {ty} }},\n}}\n\n\ + impl Feed for {name} {{\n\tfn feed(&self, h: &mut Fnv) {{\n\t\tmatch self {{\n\t\t\tSelf::Unit => h.put(\"Unit;\"),\n\ + \t\t\tSelf::Tuple(v) => {{\n\t\t\t\th.put(\"Tuple(\");\n\t\t\t\tv.feed(h);\n\t\t\t\th.put(\")\");\n\t\t\t}}\n\ + \t\t\tSelf::Struct {{ value }} => {{\n\t\t\t\th.put(\"Struct{{\");\n\t\t\t\tvalue.feed(h);\n\t\t\t\th.put(\"}}\");\n\t\t\t}}\n\ + \t\t}}\n\t}}\n}}\n" ); ( name.clone(), @@ -364,7 +418,11 @@ impl Emit { Shape::Object(inner) => { let (ty, make) = self.shape(inner); let name = self.name("Object"); - let _ = writeln!(self.items, "#[derive(Debug)]\npub struct {name}({ty});\n"); + let _ = writeln!( + self.items, + "#[derive(Debug)]\npub struct {name}({ty});\n\n\ + impl Feed for {name} {{\n\tfn feed(&self, h: &mut Fnv) {{\n\t\th.put(\"{name}(\");\n\t\tself.0.feed(h);\n\t\th.put(\")\");\n\t}}\n}}\n" + ); (name.clone(), format!("{name}({make})")) } } @@ -421,14 +479,25 @@ impl Rng { } } + /// Any bits, a NaN's payload and sign among them: a signaling NaN, which a load that + /// quiets it would change, and a quiet one. The gate's seed 1 draws the first, and the + /// other seed `swap`, `pair`, and `held` make of 0xdead_beef the second. fn f32(&mut self) -> f32 { - let f = f32::from_bits(self.int() as u32); - if f.is_nan() { -0.0 } else { f } + let (class, bits) = (self.below(4), self.next() as u32); + match class { + 1 => f32::from_bits(bits & 1 << 31 | 0x7f80_0000 | (1 + bits % 0x3f_ffff)), + 2 => f32::from_bits(bits | 0x7fc0_0000), + _ => f32::from_bits(self.int() as u32), + } } fn f64(&mut self) -> f64 { - let f = f64::from_bits(self.int()); - if f.is_nan() { f64::INFINITY } else { f } + let (class, bits) = (self.below(4), self.next()); + match class { + 1 => f64::from_bits(bits & 1 << 63 | 0x7ff0_0000_0000_0000 | (1 + bits % 0x7_ffff_ffff_ffff)), + 2 => f64::from_bits(bits | 0x7ff8_0000_0000_0000), + _ => f64::from_bits(self.int()), + } } fn char(&mut self) -> char { @@ -514,13 +583,170 @@ impl Rng { } } -/// FNV-1a of the value's `Debug`, which prints every bit of each shape's value. -fn digest(value: &dyn std::fmt::Debug) -> u64 { - format!("{value:?}").bytes().fold(0xcbf2_9ce4_8422_2325, |h, b| (h ^ b as u64).wrapping_mul(0x100_0000_01b3)) +/// FNV-1a of what a value feeds it. +struct Fnv(u64); + +impl Fnv { + fn put(&mut self, text: &str) { + for b in text.bytes() { + self.0 = (self.0 ^ b as u64).wrapping_mul(0x100_0000_01b3); + } + } +} + +impl std::fmt::Write for Fnv { + fn write_str(&mut self, text: &str) -> std::fmt::Result { + self.put(text); + Ok(()) + } +} + +/// Every bit of a value: `Debug`'s, but a float's bits, since `Debug` prints any NaN alike. +trait Feed { + fn feed(&self, h: &mut Fnv); +} + +macro_rules! debug { + ($($ty:ty),*) => {$( + impl Feed for $ty { + fn feed(&self, h: &mut Fnv) { + use std::fmt::Write; + let _ = write!(h, "{self:?};"); + } + } + )*}; +} + +debug!( + bool, u8, u16, u32, u64, usize, i8, i16, i32, i64, isize, u128, char, str, String, + std::num::NonZero, std::num::NonZero, std::num::NonZero, std::num::NonZero, + std::num::NonZero, std::num::NonZero, std::num::NonZero, std::num::NonZero, + std::num::NonZero, std::num::NonZero, bytes::Bytes, std::time::Duration, std::path::Path, + std::path::PathBuf, std::net::SocketAddr, std::net::SocketAddrV4, std::net::SocketAddrV6, std::net::IpAddr, + std::net::Ipv4Addr, std::net::Ipv6Addr, std::ops::Range, std::ops::Range, std::ops::Bound, + std::ops::Bound, serde_json::Value, serde_json::Map, url::Url +); + +impl Feed for f32 { + fn feed(&self, h: &mut Fnv) { + use std::fmt::Write; + let _ = write!(h, "f32 {:#x};", self.to_bits()); + // The transcript marks a call that digests a NaN other than the one a float's + // arithmetic makes: signaling, its quiet bit clear, or quiet with a payload or sign. + #[cfg(feature = "transcript")] + if self.is_nan() && self.to_bits() != f32::NAN.to_bits() { + let class = if self.to_bits() & 1 << 22 == 0 { "signaling" } else { "quiet" }; + println!("nan {class} f32"); + } + } +} + +impl Feed for f64 { + fn feed(&self, h: &mut Fnv) { + use std::fmt::Write; + let _ = write!(h, "f64 {:#x};", self.to_bits()); + #[cfg(feature = "transcript")] + if self.is_nan() && self.to_bits() != f64::NAN.to_bits() { + let class = if self.to_bits() & 1 << 51 == 0 { "signaling" } else { "quiet" }; + println!("nan {class} f64"); + } + } +} + +impl Feed for &T { + fn feed(&self, h: &mut Fnv) { + (**self).feed(h); + } +} + +impl Feed for Box { + fn feed(&self, h: &mut Fnv) { + (**self).feed(h); + } +} + +impl Feed for Option { + fn feed(&self, h: &mut Fnv) { + match self { + None => h.put("None;"), + Some(v) => { + h.put("Some("); + v.feed(h); + h.put(")"); + } + } + } +} + +/// The items of a list, between `open` and `]`. +fn feed_all<'a, T: Feed + 'a>(h: &mut Fnv, open: &str, items: impl IntoIterator) { + h.put(open); + for item in items { + item.feed(h); + } + h.put("]"); +} + +impl Feed for [T] { + fn feed(&self, h: &mut Fnv) { + feed_all(h, "[", self); + } +} + +impl Feed for Vec { + fn feed(&self, h: &mut Fnv) { + feed_all(h, "[", self); + } +} + +impl Feed for [T; N] { + fn feed(&self, h: &mut Fnv) { + feed_all(h, "[", self); + } +} + +impl Feed for std::collections::BTreeSet { + fn feed(&self, h: &mut Fnv) { + feed_all(h, "{", self); + } +} + +impl Feed for std::collections::BTreeMap { + fn feed(&self, h: &mut Fnv) { + h.put("{"); + for (k, v) in self { + k.feed(h); + v.feed(h); + } + h.put("}"); + } +} + +macro_rules! tuple { + ($($t:ident $i:tt),*) => { + impl<$($t: Feed),*> Feed for ($($t,)*) { + fn feed(&self, h: &mut Fnv) { + h.put("("); + $(self.$i.feed(h);)* + h.put(")"); + } + } + }; +} + +tuple!(A 0); +tuple!(A 0, B 1); +tuple!(A 0, B 1, C 2); + +/// FNV-1a of every bit of `value`. +fn digest(value: &T) -> u64 { + let mut h = Fnv(0xcbf2_9ce4_8422_2325); + value.feed(&mut h); + h.0 } "#; -/// The functions each shape `N` exports, which the census may refuse one by one. +/// What each shape `N` exports, which the census may refuse one by one. #[derive(Clone, Copy, PartialEq, Eq, Debug)] enum Item { Make, @@ -534,63 +760,207 @@ enum Item { Made, /// `later_N(v)`: a future of `v`. Later, + /// `try_N(seed)`: the made value, `Ok` for an even seed and `Err` for an odd one. + Try, + /// `swap_N(&mut v, seed)`: the digest of `v`, which it replaces with the seed's value. + Swap, + /// `pair_N(a, seed, &b)`: a digest of all three. + Pair, + /// `maybe_N(v)`: the digest of an `Option` of a borrow. + Maybe, + /// `HolderN`: an object of the value, and its methods by each receiver. + Holder, + /// `sink_N(seed, s)`: the digest of what the foreign `SinkN::echo` returns of the + /// made value. + Sink, } impl Item { + /// The paths the census names it by, any of which it may refuse. + fn paths(self, n: usize) -> Vec { + let function = |name: &str| vec![format!("{name}_{n}")]; + match self { + Item::Make => function("make"), + Item::Echo => function("echo"), + Item::Digest => function("digest"), + Item::Lent => function("lent"), + Item::Peek => function("peek"), + Item::Made => function("made"), + Item::Later => function("later"), + Item::Try => function("try"), + Item::Swap => function("swap"), + Item::Pair => function("pair"), + Item::Maybe => function("maybe"), + Item::Holder => ["", "::new", "::digest", "::swap", "::into_inner"] + .map(|method| format!("Holder{n}{method}")) + .into(), + Item::Sink => vec![format!("Sink{n}"), format!("sink_{n}")], + } + } + + /// The function's name, or the type's. fn name(self, n: usize) -> String { - let item = match self { - Item::Make => "make", - Item::Echo => "echo", - Item::Digest => "digest", - Item::Lent => "lent", - Item::Peek => "peek", - Item::Made => "made", - Item::Later => "later", - }; - format!("{item}_{n}") + self.paths(n).swap_remove(0) + } + + /// The Rust source of its items for shape `n` of type `ty`, which `digest_N` borrows + /// as `lent`. + fn source(self, n: usize, ty: &str, lent: &str, make: &str) -> String { + match self { + Item::Make => { + format!("pub fn make_{n}(seed: u64) -> {ty} {{\n\tlet r = &mut Rng::new(seed);\n\t{make}\n}}\n") + } + Item::Echo => format!("pub fn echo_{n}(v: {ty}) -> {ty} {{\n\tv\n}}\n"), + Item::Digest => format!("pub fn digest_{n}(v: {lent}) -> u64 {{\n\tdigest(&v)\n}}\n"), + Item::Lent => { + format!("pub fn lent_{n}(seed: u64, f: impl FnOnce({ty}) -> u64) -> u64 {{\n\tf(make_{n}(seed))\n}}\n") + } + Item::Peek => format!( + "pub fn peek_{n}(seed: u64, f: impl FnOnce({lent}) -> u64) -> u64 {{\n\tf(&make_{n}(seed))\n}}\n" + ), + Item::Made => { + format!("pub fn made_{n}(seed: u64, f: impl FnOnce(u64) -> {ty}) -> u64 {{\n\tdigest(&f(seed))\n}}\n") + } + Item::Later => format!("pub async fn later_{n}(v: {ty}) -> {ty} {{\n\tv\n}}\n"), + Item::Try => format!( + "pub fn try_{n}(seed: u64) -> Result<{ty}, {ty}> {{\n\tlet v = make_{n}(seed);\n\tif seed % 2 == 0 {{ Ok(v) }} else {{ Err(v) }}\n}}\n" + ), + Item::Swap => format!( + "pub fn swap_{n}(v: &mut {ty}, seed: u64) -> u64 {{\n\tlet d = digest(&*v);\n\t*v = make_{n}(seed);\n\td\n}}\n" + ), + Item::Pair => format!( + "pub fn pair_{n}(a: {ty}, seed: u64, b: {lent}) -> u64 {{\n\tdigest(&(digest(&a), seed, digest(&b)))\n}}\n" + ), + Item::Maybe => format!("pub fn maybe_{n}(v: Option<{lent}>) -> u64 {{\n\tdigest(&v)\n}}\n"), + Item::Holder => format!( + "pub struct Holder{n}({ty});\n\n\ + impl Holder{n} {{\n\ + \tpub fn new(v: {ty}) -> Self {{\n\t\tSelf(v)\n\t}}\n\n\ + \tpub fn digest(&self) -> u64 {{\n\t\tdigest(&self.0)\n\t}}\n\n\ + \tpub fn swap(&mut self, v: {ty}) -> {ty} {{\n\t\tstd::mem::replace(&mut self.0, v)\n\t}}\n\n\ + \tpub fn into_inner(self) -> {ty} {{\n\t\tself.0\n\t}}\n\ + }}\n" + ), + Item::Sink => format!( + "pub trait Sink{n}: Send + Sync {{\n\tfn echo(&self, v: {ty}) -> {ty};\n}}\n\n\ + pub fn sink_{n}(seed: u64, s: impl Sink{n}) -> u64 {{\n\tdigest(&s.echo(make_{n}(seed)))\n}}\n" + ), + } } } -/// A line of the script each driver runs, printing `label value`. -#[derive(Clone, Copy, Debug)] -enum Call { +/// What a line of the script calls, each the item it is named after, of its shape and seed. +#[derive(Clone, Copy, PartialEq, Eq, Debug)] +enum Kind { /// `digest_N(echo_N(make_N(seed)))` - Echo(usize, u64), + Echo, /// `lent_N(seed, digest_N)` - Lent(usize, u64), + Lent, /// `peek_N(seed, digest_N)` - Peek(usize, u64), + Peek, /// `made_N(seed, make_N)` - Made(usize, u64), + Made, /// `digest_N(await later_N(make_N(seed)))` - Later(usize, u64), + Later, + /// `try_N(seed)`, printing `ok` or `err` and the digest of its value. + Try, + /// `swap_N(&mut v, seed ^ 1)` of `v = make_N(seed)`, printing what it returns and + /// `digest_N(v)` after. + Swap, + /// `pair_N(make_N(seed), seed, make_N(seed ^ 1))` + Pair, + /// `maybe_N(make_N(seed))` for an odd seed, else `maybe_N(None)`. + Maybe, + /// `HolderN::new(make_N(seed))`, `swap(make_N(seed ^ 1))`, printing the digests of + /// what that returns, of `digest()`, and of `into_inner()`. + Held, + /// `sink_N(seed, s)` of an `s` whose `echo` returns what it takes, printing `echoed` in a + /// foreign driver so its line shows the callback ran. + Sink, } -impl Call { - /// The `N` of its shape. - fn shape(self) -> usize { - let (Call::Echo(n, _) | Call::Lent(n, _) | Call::Peek(n, _) | Call::Made(n, _) | Call::Later(n, _)) = self; - n +impl Kind { + /// What the shape must export for the line, besides `make_N` and `digest_N`. + fn item(self) -> Item { + match self { + Kind::Echo => Item::Echo, + Kind::Lent => Item::Lent, + Kind::Peek => Item::Peek, + Kind::Made => Item::Made, + Kind::Later => Item::Later, + Kind::Try => Item::Try, + Kind::Swap => Item::Swap, + Kind::Pair => Item::Pair, + Kind::Maybe => Item::Maybe, + Kind::Held => Item::Holder, + Kind::Sink => Item::Sink, + } } +} + +/// Every [`Kind`], in the order a shape's lines run. +const KINDS: [Kind; 11] = [ + Kind::Echo, + Kind::Lent, + Kind::Peek, + Kind::Made, + Kind::Later, + Kind::Try, + Kind::Swap, + Kind::Pair, + Kind::Maybe, + Kind::Held, + Kind::Sink, +]; + +/// Every [`Item`], in the order the crate declares them. +const ITEMS: [Item; 13] = [ + Item::Make, + Item::Echo, + Item::Digest, + Item::Lent, + Item::Peek, + Item::Made, + Item::Later, + Item::Try, + Item::Swap, + Item::Pair, + Item::Maybe, + Item::Holder, + Item::Sink, +]; +/// A line of the script each driver runs, printing `kind n seed value`. +#[derive(Clone, Copy, Debug)] +struct Call { + kind: Kind, + /// Its shape's `N`. + n: usize, + seed: u64, +} + +impl Call { fn label(self) -> String { - match self { - Call::Echo(n, seed) => format!("echo {n} {seed}"), - Call::Lent(n, seed) => format!("lent {n} {seed}"), - Call::Peek(n, seed) => format!("peek {n} {seed}"), - Call::Made(n, seed) => format!("made {n} {seed}"), - Call::Later(n, seed) => format!("later {n} {seed}"), - } + let kind = format!("{:?}", self.kind).to_lowercase(); + format!("{kind} {} {}", self.n, self.seed) + } + + /// The other seed, which `swap`, `pair`, and `held` make a second value from. + fn other(self) -> u64 { + self.seed ^ 1 } } -/// The seeds each shape is made from. +/// The seeds each shape is made from: even, which `try_N` returns `Ok` for, and odd. const SEEDS: [u64; 3] = [0, 1, 0xdead_beef]; /// The shapes' crate, as the census left it: which shapes export, and the script. struct Crate { dir: PathBuf, shapes: Vec, + /// How many of `shapes` are the fixed set, which [`KNOWN`] checks, and the seed the + /// rest are drawn from, if any. + drawn: Option<(usize, u64)>, /// Each shape's Rust type, by `N`. types: Vec, calls: Vec, @@ -600,7 +970,7 @@ struct Crate { /// Write the crate of `shapes` named `name` in the scratch directory, ask the exporter what /// exports, and run its script natively. -fn emit(name: &str, shapes: &[Shape]) -> Crate { +fn emit(name: &str, shapes: &[Shape], drawn: Option<(usize, u64)>) -> Crate { let dir = scratch().join(name); let mut emit = Emit { items: String::new(), @@ -608,29 +978,25 @@ fn emit(name: &str, shapes: &[Shape]) -> Crate { }; let mut functions = String::new(); let mut types = Vec::new(); + let mut lents = Vec::new(); for (n, shape) in shapes.iter().enumerate() { let (ty, make) = emit.shape(shape); let lent = borrowed(shape, &ty); - let _ = write!( - functions, - "pub fn make_{n}(seed: u64) -> {ty} {{\n\tlet r = &mut Rng::new(seed);\n\t{make}\n}}\n\n\ - pub fn echo_{n}(v: {ty}) -> {ty} {{\n\tv\n}}\n\n\ - pub fn digest_{n}(v: {lent}) -> u64 {{\n\tdigest(&v)\n}}\n\n\ - pub fn lent_{n}(seed: u64, f: impl FnOnce({ty}) -> u64) -> u64 {{\n\tf(make_{n}(seed))\n}}\n\n\ - pub fn peek_{n}(seed: u64, f: impl FnOnce({lent}) -> u64) -> u64 {{\n\tf(&make_{n}(seed))\n}}\n\n\ - pub fn made_{n}(seed: u64, f: impl FnOnce(u64) -> {ty}) -> u64 {{\n\tdigest(&f(seed))\n}}\n\n\ - pub async fn later_{n}(v: {ty}) -> {ty} {{\n\tv\n}}\n\n" - ); + for item in ITEMS { + let _ = writeln!(functions, "{}", item.source(n, &ty, &lent, &make)); + } types.push(ty); + lents.push(lent); } let lib = format!( - "//! Generated by `tests/fixtures/shapes.rs`: each shape's `make`, `echo`, and `digest`.\n\ + "//! Generated by `tests/fixtures/shapes.rs`: each shape's items.\n\ #![allow(clippy::all, dead_code, reason = \"generated\")]\n{RUNTIME}\n{}{functions}", emit.items ); let manifest = format!( "[package]\nname = \"{name}\"\nversion = \"0.0.0\"\nedition = \"2024\"\npublish = false\n\n\ - [dependencies]\nbytes = \"1\"\nserde_json = \"1\"\nurl = \"2\"\n\n[workspace]\n" + [dependencies]\nbytes = \"1\"\nserde_json = \"1\"\nurl = \"2\"\n\n\ + # Marks each call of the native script that digests a NaN of its own bits.\n[features]\ntranscript = []\n\n[workspace]\n" ); super::write_all( &dir, @@ -640,8 +1006,8 @@ fn emit(name: &str, shapes: &[Shape]) -> Crate { super::step::copy(&manifest_dir().join("Cargo.lock"), &dir.join("Cargo.lock")); let refused = refused(&dir); - let exported = |item: Item, n: usize| !refused.iter().any(|path| *path == item.name(n)); - let mut excludes: Vec<&String> = refused.iter().collect(); + let exported = |item: Item, n: usize| item.paths(n).iter().all(|path| !refused.absent.contains(path)); + let mut excludes: Vec<&String> = refused.exclude.iter().collect(); excludes.sort(); let config = excludes .iter() @@ -655,45 +1021,73 @@ fn emit(name: &str, shapes: &[Shape]) -> Crate { continue; } for seed in SEEDS { - if exported(Item::Echo, n) { - calls.push(Call::Echo(n, seed)); - } - if exported(Item::Lent, n) { - calls.push(Call::Lent(n, seed)); - } - if exported(Item::Peek, n) { - calls.push(Call::Peek(n, seed)); - } - if exported(Item::Made, n) { - calls.push(Call::Made(n, seed)); - } - if exported(Item::Later, n) { - calls.push(Call::Later(n, seed)); + for kind in KINDS { + if exported(kind.item(), n) { + calls.push(Call { kind, n, seed }); + } } } } - let mut script = format!("//! The shapes' script, run natively.\n\nuse {name}::*;\n{NATIVE}"); - for call in &calls { - let value = match *call { - Call::Echo(n, seed) => format!("digest_{n}(&echo_{n}(make_{n}({seed})))"), - Call::Lent(n, seed) => format!("lent_{n}({seed}, |v| digest_{n}(&v))"), - Call::Peek(n, seed) => format!("peek_{n}({seed}, digest_{n})"), - Call::Made(n, seed) => format!("made_{n}({seed}, make_{n})"), - Call::Later(n, seed) => format!("digest_{n}(&later(later_{n}(make_{n}({seed}))))"), + let mut sinks = String::new(); + let lines = by_shape(&calls, |call| { + let Call { n, seed, .. } = call; + let other = call.other(); + let value = match call.kind { + Kind::Echo => format!("digest_{n}(&echo_{n}(make_{n}({seed})))"), + Kind::Lent => format!("lent_{n}({seed}, |v| digest_{n}(&v))"), + Kind::Peek => format!("peek_{n}({seed}, digest_{n})"), + Kind::Made => format!("made_{n}({seed}, make_{n})"), + Kind::Later => format!("digest_{n}(&later(later_{n}(make_{n}({seed}))))"), + Kind::Try => format!( + "match try_{n}({seed}) {{ Ok(v) => format!(\"ok {{}}\", digest_{n}(&v)), Err(e) => format!(\"err {{}}\", digest_{n}(&e)) }}" + ), + Kind::Swap => format!( + "{{ let mut v = make_{n}({seed}); let d = swap_{n}(&mut v, {other}); format!(\"{{d}} {{}}\", digest_{n}(&v)) }}" + ), + Kind::Pair => format!("pair_{n}(make_{n}({seed}), {seed}, &make_{n}({other}))"), + Kind::Maybe => match seed % 2 { + 1 => format!("maybe_{n}(Some::<{}>(&make_{n}({seed})))", lents[n]), + _ => format!("maybe_{n}(None)"), + }, + Kind::Held => format!( + "{{ let mut h = Holder{n}::new(make_{n}({seed})); let old = h.swap(make_{n}({other})); \ + format!(\"{{}} {{}} {{}}\", digest_{n}(&old), h.digest(), digest_{n}(&h.into_inner())) }}" + ), + Kind::Sink => { + if seed == SEEDS[0] { + let _ = writeln!( + sinks, + "struct Echo{n};\n\nimpl Sink{n} for Echo{n} {{\n\tfn echo(&self, v: {ty}) -> {ty} {{\n\t\tv\n\t}}\n}}\n", + ty = types[n] + ); + } + format!("sink_{n}({seed}, Echo{n})") + } }; - let _ = writeln!(script, "\tprintln!(\"{} {{}}\", {value});", call.label()); + format!("\tprintln!(\"{} {{}}\", {value});", call.label()) + }); + // Each shape's calls in a function of its own, past whose locals wasm's limit no + // function grows. + let mut script = format!("//! The shapes' script, run natively.\n\nuse {name}::*;\n{NATIVE}"); + for (n, _) in &lines { + let _ = writeln!(script, "\tshape_{n}();"); + } + script.push_str("}\n\n"); + for (n, lines) in &lines { + let _ = writeln!(script, "fn shape_{n}() {{\n{lines}}}\n"); } - script.push_str("}\n"); + script.push_str(&sinks); super::write_all(&dir, &[("examples/transcript.rs", &script)]); // Apart from the exporter's build, which cleans the crate each time. let transcript = run(cargo() - .args(["run", "--quiet", "--example", "transcript"]) + .args(["run", "--quiet", "--example", "transcript", "--features", "transcript"]) .env("CARGO_TARGET_DIR", dir.join("native")) .current_dir(&dir)); Crate { dir, shapes: shapes.to_vec(), + drawn, types, calls, transcript: String::from_utf8_lossy(&transcript.stdout).into_owned(), @@ -712,8 +1106,9 @@ fn later(f: impl Future) -> T { fn main() { "; -/// The paths the census of the crate in `dir` refuses: its config that exports what maps. -fn refused(dir: &Path) -> Vec { +/// What the census of the crate in `dir` says does not export: the config that exports +/// what maps, and what leaves itself out. +fn refused(dir: &Path) -> Refused { let census = dir.join("census"); run(Command::new(env!("CARGO_BIN_EXE_rust2-ffi")) .arg("--config") @@ -723,16 +1118,40 @@ fn refused(dir: &Path) -> Vec { .env("CARGO_TARGET_DIR", dir.join("build"))); let name = dir.file_name().unwrap().to_string_lossy(); let report = std::fs::read_to_string(census.join(format!("{name}.md"))).unwrap(); - let Some((_, config)) = report.split_once("## The config that exports what maps") else { - return Vec::new(); + let exclude = match report.split_once("## The config that exports what maps") { + Some((_, config)) => config + .lines() + .skip_while(|line| *line != "exclude = [") + .skip(1) + .take_while(|line| *line != "]") + .map(|line| line.trim().trim_end_matches(',').trim_matches('"').to_string()) + .collect(), + None => Vec::new(), }; - config - .lines() - .skip_while(|line| *line != "exclude = [") - .skip(1) - .take_while(|line| *line != "]") - .map(|line| line.trim().trim_end_matches(',').trim_matches('"').to_string()) - .collect() + // The traits C cannot implement, and what C can never hold, leave themselves out. + let mut absent = exclude.clone(); + for section in [ + "## Traits left out, since C cannot implement them", + "## What C can never hold, left out", + ] { + let Some((_, list)) = report.split_once(section) else { + continue; + }; + absent.extend( + (list.lines().skip(1)) + .take_while(|line| !line.starts_with("## ")) + .filter_map(|line| Some(line.strip_prefix("- `")?.split_once('`')?.0.to_string())), + ); + } + Refused { exclude, absent } +} + +/// What the census of the shapes' crate says does not export. +struct Refused { + /// The paths its config leaves out. + exclude: Vec, + /// Those, and what leaves itself out. + absent: Vec, } /// Each variant of the IR's `Type`, from its source. @@ -768,17 +1187,23 @@ struct Built { lib: PathBuf, /// What a program linking the static library links besides, as rustc prints it. native: Vec, + /// The web binding's module and native transcript, built for the web test alone. + web: std::sync::OnceLock<(PathBuf, String)>, } -/// Emit, census, export, and build the crate of `shapes` named `name`. -fn build(name: &str, shapes: &[Shape]) -> Built { - let krate = emit(name, shapes); +/// Emit, census, export, and build the crate of `shapes` named `name`, of which `drawn` +/// says how many are the fixed set and the seed of the rest, if any. +fn build(name: &str, shapes: &[Shape], drawn: Option<(usize, u64)>) -> Built { + let krate = emit(name, shapes, drawn); let out = krate.dir.join("out"); run(Command::new(env!("CARGO_BIN_EXE_rust2-ffi")) .arg("--config") .arg(krate.dir.join("ffi.toml")) .arg("--out") .arg(&out) + // One binding for the host and the target the web binding serves, as the api + // fixture's. + .args(["--target", super::host(), "--target", super::WEB]) .env("CARGO_TARGET_DIR", krate.dir.join("build"))); let target = krate.dir.join("glue"); let build = |target: &Path, feature: &str| { @@ -809,6 +1234,7 @@ fn build(name: &str, shapes: &[Shape]) -> Built { out, lib: target.join("debug"), native, + web: std::sync::OnceLock::new(), } } @@ -817,6 +1243,50 @@ impl Built { self.krate.dir.file_name().unwrap().to_string_lossy().into_owned() } + /// The module the export's `web` feature builds for wasm32-wasip1, and what the native + /// script prints there, run under Node's WASI: its `usize` is 32 bits, and a path there + /// is text. + fn web(&self) -> &(PathBuf, String) { + self.web.get_or_init(|| { + let name = self.name(); + let glue = self.krate.dir.join("glue-web"); + run(cargo() + .args(["build", "--quiet", "--locked", "--manifest-path"]) + .arg(self.out.join("Cargo.toml")) + .arg("--target-dir") + .arg(&glue) + .args(["--target", super::WEB, "--features", "web"]) + .env("RUSTFLAGS", "-D warnings")); + let native = self.krate.dir.join("native-web"); + run(cargo() + .args([ + "build", + "--quiet", + "--example", + "transcript", + "--features", + "transcript", + ]) + .args(["--target", super::WEB]) + .env("CARGO_TARGET_DIR", &native) + .current_dir(&self.krate.dir)); + let wasm = native + .join(super::WEB) + .join("debug") + .join("examples") + .join("transcript.wasm"); + let transcript = run(Command::new("node") + .args(["--no-warnings", "--input-type=module", "-e", super::WASI_MAIN]) + .arg(wasm)); + let module = glue.join(super::WEB).join("debug").join(format!("{name}.wasm")); + (module, String::from_utf8_lossy(&transcript.stdout).into_owned()) + }) + } + + fn wasm_transcript(&self) -> &str { + &self.web().1 + } + fn staticlib(&self) -> PathBuf { let name = self.name(); self.lib.join(match cfg!(target_env = "msvc") { @@ -830,44 +1300,76 @@ impl Built { self.krate.dir.join("drivers").join(language) } - /// The exported shapes `language` gets wrong today, by [`KNOWN`], each entry of which must - /// match one. - fn known(&self, language: &str) -> Vec { - let mut known = Vec::new(); - for Known { what, skips, quest, .. } in KNOWN.iter().filter(|k| k.language == language) { - let matched: Vec = (0..self.krate.shapes.len()) - .filter(|&n| skips(&self.krate.shapes[n]) && self.krate.calls.iter().any(|call| call.shape() == n)) - .collect(); - // The gate's subset may hold none of an entry's shapes; the nightly's holds each. - assert!( - !matched.is_empty() || self.name() != "sampled", - "KNOWN's {language} entry for {what} ({quest}) matches no shape: remove it" - ); - known.extend(matched); + /// The calls `language`'s driver runs, each with the line native Rust printed for it in + /// `transcript`: every call but those [`KNOWN`] says it gets wrong today, each entry of + /// which must match one of the fixed set's. + fn expected<'a>(&self, language: &str, transcript: &'a str) -> Vec<(Call, &'a str)> { + // Each call's line, after the NaNs it digested, as `signaling f32` or `quiet f64`. + let mut lines: Vec<(&str, Vec<&str>)> = Vec::new(); + let mut nans = Vec::new(); + for line in transcript.lines() { + match line.strip_prefix("nan ") { + Some(nan) => nans.push(nan), + None => lines.push((line, std::mem::take(&mut nans))), + } } - known + assert_eq!( + lines.len(), + self.krate.calls.len(), + "the native script printed a line per call" + ); + let entries: Vec<&Known> = KNOWN.iter().filter(|k| k.language == language).collect(); + let calls: Vec<(Call, &str, &[&str])> = (self.krate.calls.iter().zip(&lines)) + .map(|(call, (line, nans))| (*call, *line, nans.as_slice())) + .collect(); + let skipped = + |(call, _, nans): &(Call, &str, &[&str]), k: &Known| (k.skips)(&self.krate.shapes[call.n], call.kind, nans); + if let Some((fixed, _)) = self.krate.drawn { + for k in &entries { + // The gate's subset may hold none of an entry's calls; the nightly's fixed set + // holds each. + assert!( + calls.iter().any(|c| c.0.n < fixed && skipped(c, k)), + "KNOWN's {language} entry for {} ({}) matches no call: remove it", + k.what, + k.quest + ); + } + } + (calls.iter()) + .filter(|c| !entries.iter().any(|k| skipped(c, k))) + .map(|&(call, line, _)| (call, line)) + .collect() } - /// Asserts `language`'s driver printed the native transcript but for the shapes it gets - /// wrong today, naming each line that differs by its shape's Rust type. - fn check(&self, language: &str, known: &[usize], output: &std::process::Output) { + /// Asserts `language`'s driver printed what native Rust did for each of `expected`, + /// naming each line that differs by its shape's Rust type. + fn check(&self, language: &str, expected: &[(Call, &str)], output: &std::process::Output) { let got = String::from_utf8_lossy(&output.stdout); let mut wrong = String::new(); - let expected: Vec<&str> = (self.krate.transcript.lines().zip(&self.krate.calls)) - .filter(|(_, call)| !known.contains(&call.shape())) - .map(|(line, _)| line) - .collect(); let lines: Vec<&str> = got.lines().collect(); - for (i, want) in expected.iter().enumerate() { + // A sink's foreign `echo` prints `echoed` once, before its call's line. + let expected: Vec<(Call, &str)> = (expected.iter()) + .flat_map(|&(call, want)| { + (call.kind == Kind::Sink) + .then_some((call, "echoed")) + .into_iter() + .chain([(call, want)]) + }) + .collect(); + for (i, (call, want)) in expected.iter().enumerate() { let line = lines.get(i).copied().unwrap_or(""); if line != *want { - let n: usize = want.split(' ').nth(1).unwrap().parse().unwrap(); - let _ = writeln!(wrong, " {want} ({}), got {line}", self.krate.types[n]); + let _ = writeln!(wrong, " {want} ({}), got {line}", self.krate.types[call.n]); } } + let drawn = match self.krate.drawn { + Some((_, seed)) => format!("; shapes past the fixed set drawn from RUST2_SHAPES_SEED={seed}"), + None => String::new(), + }; assert!( wrong.is_empty() && lines.len() == expected.len(), - "{language}'s round trips differ from native Rust's in {} of {} lines, {} printed:\n{wrong}{}", + "{language}'s round trips differ from native Rust's in {} of {} lines, {} printed{drawn}:\n{wrong}{}", wrong.lines().count(), expected.len(), lines.len(), @@ -880,17 +1382,49 @@ impl Built { struct Known { language: &'static str, what: &'static str, - /// Whether a shape holds what it gets wrong, which the language's driver skips. - skips: fn(&Shape) -> bool, + /// Whether a call of a shape, which digested these NaNs natively, as `signaling f32` or + /// `quiet f64`, does what it gets wrong, which the language's driver skips. + skips: fn(&Shape, Kind, &[&str]) -> bool, /// The quest that fixes it. quest: &'static str, } /// The round trips a binding gets wrong today, each the quest's to fix: the language, what -/// it gets wrong, the shapes it skips for it, and the quest. The nightly fails on an entry -/// matching no exported shape, but its shapes stay skipped after a fix, so the quest that -/// fixes one deletes its entry. -const KNOWN: &[Known] = &[]; +/// it gets wrong, the calls it skips for it, and the quest. The nightly fails on an entry +/// matching no call of its fixed set, but its calls stay skipped after a fix, so the quest +/// that fixes one deletes its entry. +const KNOWN: &[Known] = &[ + Known { + language: "python", + what: "a signaling NaN of an f32, which a float holds quieted", + skips: |shape, _, nans| nans.contains(&"signaling f32") && crosses(shape, &|s| matches!(s, Shape::F32)), + quest: "quest/m1/ffi-export/signaling-nan.md", + }, + Known { + language: "dart", + what: "a signaling NaN of an f32, which a double holds quieted", + skips: |shape, _, nans| nans.contains(&"signaling f32") && crosses(shape, &|s| matches!(s, Shape::F32)), + quest: "quest/m1/ffi-export/signaling-nan.md", + }, + Known { + language: "web", + what: "a NaN's bits, which a number loses: quieted, and canonicalized once V8 optimizes", + skips: |shape, _, nans| !nans.is_empty() && crosses(shape, &|s| matches!(s, Shape::F32 | Shape::F64)), + quest: "quest/m1/ffi-export/signaling-nan.md", + }, +]; + +/// Whether a value of `shape` holds a `leaf` the binding converts, which one inside an +/// object does not: the object crosses as a handle, and only Rust reads its field. +fn crosses(shape: &Shape, leaf: &dyn Fn(&Shape) -> bool) -> bool { + match shape { + Shape::Object(_) => false, + Shape::Option(s) | Shape::Vec(s) | Shape::Set(s) | Shape::Box(s) | Shape::Enum(s) => crosses(s, leaf), + Shape::Map(k, v) => crosses(k, leaf) || crosses(v, leaf), + Shape::Tuple(s) | Shape::Record(s) => s.iter().any(|s| crosses(s, leaf)), + s => leaf(s), + } +} /// A function the C header declares: each parameter's type and name. struct Proto { @@ -952,30 +1486,44 @@ impl Proto { /// `ty`. A poll's call must return `ready`. fn call(&self, function: &str, args: &[String], ty: &str, var: &str, ready: Option<&str>) -> String { let args = args.join(", "); - let end = self.params.len() - usize::from(self.status()); - let outs = &self.params[end - self.outs()..end]; - if outs.is_empty() { + let Some(Outs { decls, pointers, value }) = self.gather(ty, var) else { return format!("\t\t{ty} {var} = {function}({args}{});\n", self.tail()); + }; + let mut out = decls; + let call = format!("{function}({args}, {pointers}{})", self.tail()); + let _ = match ready { + Some(ready) => writeln!(out, "\t\tif ({call} != {ready}) abort();"), + None => writeln!(out, "\t\t{call};"), + }; + let _ = writeln!(out, "\t\t{ty} {var} = {value};"); + out + } + + /// The out pointers it writes its output through, `out`, or `out0` on for a returned + /// tuple's elements, declared as `var_K`, and their value as `ty`. + fn gather(&self, ty: &str, var: &str) -> Option { + let is_out = |name: &str| name == "out" || name.strip_prefix("out").is_some_and(|k| k.parse::().is_ok()); + let outs: Vec<&(String, String)> = self.params.iter().filter(|(_, name)| is_out(name)).collect(); + if outs.is_empty() { + return None; } - let mut out = String::new(); + let mut decls = String::new(); let mut pointers = Vec::new(); let mut values = Vec::new(); for (k, (pointer, _)) in outs.iter().enumerate() { - let _ = writeln!(out, "\t\t{} {var}_{k};", pointer.strip_suffix('*').unwrap().trim()); + let _ = writeln!(decls, "\t\t{} {var}_{k};", pointer.strip_suffix('*').unwrap().trim()); pointers.push(format!("&{var}_{k}")); values.push(format!("{var}_{k}")); } - let call = format!("{function}({args}, {}{})", pointers.join(", "), self.tail()); - let _ = match ready { - Some(ready) => writeln!(out, "\t\tif ({call} != {ready}) abort();"), - None => writeln!(out, "\t\t{call};"), - }; - // `out` is the value; `out0` on are a returned tuple's elements, of one or more. - let _ = match outs[0].1 == "out" { - true => writeln!(out, "\t\t{ty} {var} = {var}_0;"), - false => writeln!(out, "\t\t{ty} {var} = ({ty}){{{}}};", values.join(", ")), + let value = match outs[0].1 == "out" { + true => format!("{var}_0"), + false => format!("({ty}){{{}}}", values.join(", ")), }; - out + Some(Outs { + decls, + pointers: pointers.join(", "), + value, + }) } /// Whether it takes a trailing status, for a check one of its handles can fail. @@ -991,12 +1539,6 @@ impl Proto { } } - /// How many out pointers it writes its output through: `out`, or `out0` on for a tuple. - fn outs(&self) -> usize { - let out = |name: &str| name == "out" || name.strip_prefix("out").is_some_and(|k| k.parse::().is_ok()); - self.params.iter().filter(|(_, name)| out(name)).count() - } - /// What a function pointer parameter `(*name)` returns and takes after its data. fn pointer(&self, name: &str) -> (String, String) { let ty = self.param(name); @@ -1006,6 +1548,23 @@ impl Proto { } } +/// The type of the value an option struct `ty` holds, if `ty` is one. +fn option_value(header: &str, ty: &str) -> Option { + let body = header + .split(&format!("typedef struct {ty} {{\n\tbool some;\n\t")) + .nth(1)?; + let field = body.lines().next()?; + Some(field.strip_suffix(" value;")?.to_string()) +} + +/// What a C function writes its output through: the declarations of its out pointers, the +/// pointers to pass, and the value they gather. +struct Outs { + decls: String, + pointers: String, + value: String, +} + /// The C driver: each call's value in a block of its own, typed by the header's prototypes. fn c(built: &Built, calls: &[Call]) -> String { let name = built.name(); @@ -1016,18 +1575,168 @@ fn c(built: &Built, calls: &[Call]) -> String { let mut defined = std::collections::BTreeSet::new(); for call in calls { let label = call.label(); - let (Call::Echo(n, seed) - | Call::Lent(n, seed) - | Call::Peek(n, seed) - | Call::Made(n, seed) - | Call::Later(n, seed)) = *call; + let Call { n, seed, .. } = *call; let make = proto(Item::Make, n); let digest = proto(Item::Digest, n); let digested = digest.param("v").to_string(); let fname = |item: Item| format!("{name}_{}", item.name(n)); + // The digest of the value `v` holds. + let digested_of = |v: &str| format!("{}({v}{})", fname(Item::Digest), digest.tail()); let mut block = String::new(); - match *call { - Call::Echo(..) => { + match call.kind { + Kind::Try => { + let tried = proto(Item::Try, n); + let err = tried.param("err").strip_suffix('*').unwrap().trim().to_string(); + let Outs { decls, pointers, value } = tried.gather(&digested, "a").expect("try_N writes its value out"); + block += &decls; + let _ = writeln!( + block, + "\t\t{err} e;\n\t\tif ({}({seed}u, {pointers}, &e)) {{\n\t\t\t{digested} v = {value};\n\ + \t\t\tprintf(\"{label} ok %\" PRIu64 \"\\n\", {});\n\t\t}} else {{\n\ + \t\t\tprintf(\"{label} err %\" PRIu64 \"\\n\", {});\n\t\t}}", + fname(Item::Try), + digested_of("v"), + digested_of("e"), + ); + } + Kind::Swap => { + let swap = proto(Item::Swap, n); + block += &make.call(&fname(Item::Make), &[format!("{seed}u")], swap.param("v"), "a", None); + let other = call.other(); + // A value is written back through `v_out`; an object's handle is borrowed. + let after = match swap.params.iter().any(|(_, name)| name == "v_out") { + true => { + let ty = swap.param("v_out").strip_suffix('*').unwrap().trim().to_string(); + let _ = writeln!( + block, + "\t\t{ty} b;\n\t\tuint64_t d = {}(a, {other}u, &b{});", + fname(Item::Swap), + swap.tail() + ); + "b" + } + false => { + let _ = writeln!( + block, + "\t\tuint64_t d = {}(a, {other}u{});", + fname(Item::Swap), + swap.tail() + ); + "a" + } + }; + let _ = writeln!( + block, + "\t\tprintf(\"{label} %\" PRIu64 \" %\" PRIu64 \"\\n\", d, {});", + digested_of(after) + ); + } + Kind::Pair => { + let pair = proto(Item::Pair, n); + block += &make.call(&fname(Item::Make), &[format!("{seed}u")], pair.param("a"), "a", None); + block += &make.call( + &fname(Item::Make), + &[format!("{}u", call.other())], + pair.param("b"), + "b", + None, + ); + let _ = writeln!( + block, + "\t\tprintf(\"{label} %\" PRIu64 \"\\n\", {}(a, {seed}u, b{}));", + fname(Item::Pair), + pair.tail() + ); + } + Kind::Maybe => { + let maybe = proto(Item::Maybe, n); + let ty = maybe.param("v").to_string(); + // An option of a value is a struct of it; of a buffer or handle, null is None. + let option = option_value(&header, &ty); + let v = match (seed % 2, &option) { + (0, Some(_)) => format!("({ty}){{0}}"), + (0, None) if ty.ends_with('*') => "NULL".into(), + (0, None) => format!("({ty}){{0}}"), + (_, Some(value)) => { + block += &make.call(&fname(Item::Make), &[format!("{seed}u")], value, "a", None); + format!("({ty}){{true, a}}") + } + (_, None) => { + block += &make.call(&fname(Item::Make), &[format!("{seed}u")], &ty, "a", None); + "a".into() + } + }; + let _ = writeln!( + block, + "\t\tprintf(\"{label} %\" PRIu64 \"\\n\", {}({v}{}));", + fname(Item::Maybe), + maybe.tail() + ); + } + Kind::Held => { + let holder = format!("{name}_Holder{n}"); + let new = prototype(&header, &format!("{holder}_new")); + let swap = prototype(&header, &format!("{holder}_swap")); + let held = prototype(&header, &format!("{holder}_digest")); + let inner = prototype(&header, &format!("{holder}_into_inner")); + block += &make.call(&fname(Item::Make), &[format!("{seed}u")], new.param("v"), "a", None); + block += &new.call( + &format!("{holder}_new"), + &["a".into()], + &format!("{holder} *"), + "h", + None, + ); + block += &make.call( + &fname(Item::Make), + &[format!("{}u", call.other())], + swap.param("v"), + "b", + None, + ); + block += &swap.call( + &format!("{holder}_swap"), + &["h".into(), "b".into()], + &digested, + "old", + None, + ); + let _ = writeln!(block, "\t\tuint64_t d = {holder}_digest(h{});", held.tail()); + block += &inner.call(&format!("{holder}_into_inner"), &["h".into()], &digested, "c", None); + let _ = writeln!( + block, + "\t\t{holder}_free(h);\n\t\tprintf(\"{label} %\" PRIu64 \" %\" PRIu64 \" %\" PRIu64 \"\\n\", {}, d, {});", + digested_of("old"), + digested_of("c") + ); + } + Kind::Sink => { + let sink = format!("{name}_Sink{n}"); + if defined.insert(format!("Sink{n}")) { + let vtable = header + .split(&format!("typedef struct {sink}_vtable {{")) + .nth(1) + .expect("the header declares the sink's vtable"); + let echo = vtable.lines().find(|line| line.contains("(*echo)")).unwrap(); + let echo = Proto { + params: vec![(echo.trim().trim_end_matches(';').to_string(), "echo".into())], + }; + let (ret, param) = echo.pointer("echo"); + // Its parameter, named in the vtable. + let param = ¶m[..param.rfind([' ', '*']).unwrap() + 1]; + let _ = writeln!( + closures, + "static {ret} sink{n}_echo(void *data, {param}v) {{\n\t(void)data;\n\tprintf(\"echoed\\n\");\n\treturn v;\n}}\n\n\ + static void sink{n}_free(void *data) {{\n\t(void)data;\n}}\n\n\ + static const {sink}_vtable sink{n} = {{.free = sink{n}_free, .echo = sink{n}_echo}};\n" + ); + } + let _ = writeln!( + block, + "\t\tprintf(\"{label} %\" PRIu64 \"\\n\", {name}_sink_{n}({seed}u, ({sink}){{&sink{n}, NULL}}));" + ); + } + Kind::Echo => { let echo = proto(Item::Echo, n); block += &make.call(&fname(Item::Make), &[format!("{seed}u")], echo.param("v"), "a", None); block += &echo.call(&fname(Item::Echo), &["a".into()], &digested, "b", None); @@ -1038,8 +1747,8 @@ fn c(built: &Built, calls: &[Call]) -> String { digest.tail() ); } - Call::Lent(..) | Call::Peek(..) => { - let item = if matches!(call, Call::Lent(..)) { + Kind::Lent | Kind::Peek => { + let item = if call.kind == Kind::Lent { Item::Lent } else { Item::Peek @@ -1061,7 +1770,7 @@ fn c(built: &Built, calls: &[Call]) -> String { item.name(n) ); } - Call::Made(..) => { + Kind::Made => { if defined.insert(Item::Made.name(n)) { let (ret, _) = proto(Item::Made, n).pointer("f"); let made = make.call(&fname(Item::Make), &["seed".into()], &ret, "v", None); @@ -1083,7 +1792,7 @@ fn c(built: &Built, calls: &[Call]) -> String { Item::Made.name(n) ); } - Call::Later(..) => { + Kind::Later => { let later = proto(Item::Later, n); let future = format!("{}_future", fname(Item::Later)); let poll = prototype(&header, &format!("{}_poll", fname(Item::Later))); @@ -1115,37 +1824,116 @@ fn c(built: &Built, calls: &[Call]) -> String { ) } -/// The C++ driver, over the header's own types and futures. +/// The C++ driver, over the header's own types and futures: each call a statement of its +/// own, since some declare what they print. fn cpp(built: &Built, calls: &[Call]) -> String { let name = built.name(); + let header = std::fs::read_to_string(built.out.join("include").join(&name).join(format!("{name}.hpp"))).unwrap(); + let mut sinks = String::new(); + // Each sink's class, defined by its first call the driver runs. + let mut echoes = std::collections::BTreeSet::new(); let mut body = String::new(); for call in calls { - let value = match *call { - Call::Echo(n, seed) => format!("{name}::digest_{n}({name}::echo_{n}({name}::make_{n}({seed}u)))"), - Call::Lent(n, seed) => { - format!("{name}::lent_{n}({seed}u, [](auto v) {{ return {name}::digest_{n}(std::move(v)); }})") - } - Call::Peek(n, seed) => { - format!("{name}::peek_{n}({seed}u, [](auto &&v) {{ return {name}::digest_{n}(v); }})") + let Call { n, seed, .. } = *call; + let other = call.other(); + let label = call.label(); + let make = |seed: u64| format!("{name}::make_{n}({seed}u)"); + let digest = |v: &str| format!("{name}::digest_{n}({v})"); + let print = |value: String| format!("\tstd::cout << \"{label} \" << {value} << '\\n';"); + let line = match call.kind { + Kind::Echo => print(digest(&format!("{name}::echo_{n}({})", make(seed)))), + Kind::Lent => print(format!( + "{name}::lent_{n}({seed}u, [](auto v) {{ return {}; }})", + digest("std::move(v)") + )), + Kind::Peek => print(format!( + "{name}::peek_{n}({seed}u, [](auto &&v) {{ return {}; }})", + digest("v") + )), + Kind::Made => print(format!( + "{name}::made_{n}({seed}u, [](std::uint64_t s) {{ return {name}::make_{n}(s); }})" + )), + Kind::Later => print(digest(&format!("{name}::later_{n}({}).get()", make(seed)))), + Kind::Try => format!( + "\t{{\n\t\tauto r = {name}::try_{n}({seed}u);\n\ + \t\tif (r) std::cout << \"{label} ok \" << {} << '\\n';\n\ + \t\telse std::cout << \"{label} err \" << {} << '\\n';\n\t}}", + digest("std::move(r.value())"), + digest("std::move(r.error())") + ), + Kind::Swap => format!( + "\t{{\n\t\tauto v = {};\n\t\tauto d = {name}::swap_{n}(v, {other}u);\n\ + \t\tstd::cout << \"{label} \" << d << ' ' << {} << '\\n';\n\t}}", + make(seed), + digest("std::move(v)") + ), + Kind::Pair => print(format!("{name}::pair_{n}({}, {seed}u, {})", make(seed), make(other))), + Kind::Maybe => { + // An object's option is a pointer to it; anything else's an optional. + let pointer = declaration(&header, &format!("maybe_{n}(")).contains(" *v)"); + match (seed % 2, pointer) { + (0, true) => print(format!("{name}::maybe_{n}(nullptr)")), + (0, false) => print(format!("{name}::maybe_{n}(std::nullopt)")), + (_, true) => format!( + "\t{{\n\t\tauto v = {};\n\t\tstd::cout << \"{label} \" << {name}::maybe_{n}(&v) << '\\n';\n\t}}", + make(seed) + ), + (_, false) => print(format!("{name}::maybe_{n}({})", make(seed))), + } } - Call::Made(n, seed) => { - format!("{name}::made_{n}({seed}u, [](std::uint64_t s) {{ return {name}::make_{n}(s); }})") + Kind::Held => format!( + "\t{{\n\t\tauto h = {name}::Holder{n}::new_({});\n\t\tauto old = h.swap({});\n\ + \t\tauto d = h.digest();\n\t\tauto c = std::move(h).into_inner();\n\ + \t\tstd::cout << \"{label} \" << {} << ' ' << d << ' ' << {} << '\\n';\n\t}}", + make(seed), + make(other), + digest("std::move(old)"), + digest("std::move(c)") + ), + Kind::Sink => { + if echoes.insert(n) { + let class = header + .split(&format!("\nclass Sink{n} {{")) + .nth(1) + .expect("the header declares the sink"); + let echo = class.lines().find(|l| l.contains(" echo(")).unwrap().trim(); + let echo = echo.strip_prefix("virtual ").unwrap().strip_suffix(" = 0;").unwrap(); + let _ = writeln!( + sinks, + "struct Echo{n} final : {name}::Sink{n} {{\n\t{echo} override {{\n\t\tstd::cout << \"echoed\\n\";\n\t\treturn v;\n\t}}\n}};\n" + ); + } + // Its result first, since C++ streams the label before evaluating the next operand. + format!( + "\t{{\n\t\tauto d = {name}::sink_{n}({seed}u, std::make_unique());\n\t\tstd::cout << \"{label} \" << d << '\\n';\n\t}}" + ) } - Call::Later(n, seed) => format!("{name}::digest_{n}({name}::later_{n}({name}::make_{n}({seed}u)).get())"), }; - let _ = writeln!(body, "\tstd::cout << \"{} \" << {value} << '\\n';", call.label()); + let _ = writeln!(body, "{line}"); } format!( - "#include \n#include \n#include \n\n#include \"{name}.hpp\"\n\nint main() {{\n{body}}}\n" + "#include \n#include \n#include \n#include \n#include \n\n\ + #include \"{name}.hpp\"\n\n{sinks}int main() {{\n{body}}}\n" ) } +/// The line of `source` that declares `function`, a prefix of its name and `(`. +fn declaration<'a>(source: &'a str, function: &str) -> &'a str { + let at = source + .match_indices(function) + .map(|(at, _)| at) + .find(|&at| at == 0 || source[..at].ends_with(['\n', '\t', ' ', ':'])) + .unwrap_or_else(|| panic!("the source declares {function}")); + let start = source[..at].rfind('\n').map_or(0, |i| i + 1); + source[start..].lines().next().unwrap() +} + /// Each shape's calls in a function of its own, so no function grows past a compiler's /// limit: `(n, lines)`. -fn by_shape(calls: &[Call], line: impl Fn(Call) -> String) -> Vec<(usize, String)> { +fn by_shape(calls: &[Call], mut line: impl FnMut(Call) -> String) -> Vec<(usize, String)> { let mut shapes: Vec<(usize, String)> = Vec::new(); for call in calls { - let (Call::Echo(n, _) | Call::Lent(n, _) | Call::Peek(n, _) | Call::Made(n, _) | Call::Later(n, _)) = *call; + let n = call.n; if shapes.last().is_none_or(|(last, _)| *last != n) { shapes.push((n, String::new())); } @@ -1156,13 +1944,59 @@ fn by_shape(calls: &[Call], line: impl Fn(Call) -> String) -> Vec<(usize, String /// The Kotlin driver, on the JVM with kotlinx-coroutines. fn kotlin(built: &Built, calls: &[Call]) -> String { + let name = built.name(); + let source = std::fs::read_to_string(built.out.join("kotlin").join(format!("{name}.kt"))).unwrap(); + // Each exception a `Result`'s error is thrown as, by the type of its `error`. + let thrown: std::collections::HashMap<&str, &str> = (source.lines()) + .filter_map(|line| { + let rest = line.strip_prefix("class ")?; + let (class, rest) = rest.split_once("(val error: ")?; + Some((rest.strip_suffix(") : Exception() {")?, class)) + }) + .collect(); let shapes = by_shape(calls, |call| { - let value = match call { - Call::Echo(n, seed) => format!("digest{n}(echo{n}(make{n}({seed}uL)))"), - Call::Lent(n, seed) => format!("lent{n}({seed}uL) {{ digest{n}(it) }}"), - Call::Peek(n, seed) => format!("peek{n}({seed}uL) {{ digest{n}(it) }}"), - Call::Made(n, seed) => format!("made{n}({seed}uL) {{ make{n}(it) }}"), - Call::Later(n, seed) => format!("digest{n}(later{n}(make{n}({seed}uL)))"), + let Call { n, seed, .. } = call; + let other = call.other(); + let value = match call.kind { + Kind::Echo => format!("digest{n}(echo{n}(make{n}({seed}uL)))"), + Kind::Lent => format!("lent{n}({seed}uL) {{ digest{n}(it) }}"), + Kind::Peek => format!("peek{n}({seed}uL) {{ digest{n}(it) }}"), + Kind::Made => format!("made{n}({seed}uL) {{ make{n}(it) }}"), + Kind::Later => format!("digest{n}(later{n}(make{n}({seed}uL)))"), + Kind::Try => { + let declared = declaration(&source, &format!("fun try{n}(")); + let ty = declared.split_once("): ").unwrap().1.split_once(" = ").unwrap().0; + let class = thrown.get(ty).unwrap_or_else(|| panic!("no exception holds a {ty}")); + format!( + "try {{ \"ok ${{digest{n}(try{n}({seed}uL))}}\" }} catch (e: {class}) {{ \"err ${{digest{n}(e.error)}}\" }}" + ) + } + // A value is passed in a `Mut`, which Rust writes back; an object's handle is borrowed. + Kind::Swap => match declaration(&source, &format!("fun swap{n}(")).contains("v: _Mut<") { + true => format!( + "run {{ val v = Mut(make{n}({seed}uL)); val d = swap{n}(v, {other}uL); \"$d ${{digest{n}(v.value)}}\" }}" + ), + false => format!( + "run {{ val v = make{n}({seed}uL); val d = swap{n}(v, {other}uL); \"$d ${{digest{n}(v)}}\" }}" + ), + }, + Kind::Pair => format!("pair{n}(make{n}({seed}uL), {seed}uL, make{n}({other}uL))"), + Kind::Maybe => match seed % 2 { + 0 => format!("maybe{n}(null)"), + _ => format!("maybe{n}(make{n}({seed}uL))"), + }, + Kind::Held => format!( + "Holder{n}.new(make{n}({seed}uL)).use {{ h -> val old = h.swap(make{n}({other}uL)); val d = h.digest(); \ + \"${{digest{n}(old)}} $d ${{digest{n}(h.intoInner())}}\" }}" + ), + Kind::Sink => { + let interface = source + .split(&format!("\ninterface Sink{n} {{\n")) + .nth(1) + .expect("the Kotlin declares the sink"); + let echo = interface.lines().next().unwrap().trim(); + format!("sink{n}({seed}uL, object : Sink{n} {{ override {echo} {{ println(\"echoed\"); return v }} }})") + } }; format!("\tprintln(\"{} ${{{value}}}\")", call.label()) }); @@ -1185,20 +2019,77 @@ fn kotlin(built: &Built, calls: &[Call]) -> String { /// The Python driver, awaiting futures on asyncio. fn python(built: &Built, calls: &[Call]) -> String { let name = built.name(); + let source = std::fs::read_to_string(built.out.join("python").join(format!("{name}.py"))).unwrap(); + let mut sinks = String::new(); + // Each sink's class, defined by its first call the driver runs. + let mut echoes = std::collections::BTreeSet::new(); let mut body = String::new(); for call in calls { - let value = match *call { - Call::Echo(n, seed) => format!("s.digest_{n}(s.echo_{n}(s.make_{n}({seed})))"), - Call::Lent(n, seed) => format!("s.lent_{n}({seed}, s.digest_{n})"), - Call::Peek(n, seed) => format!("s.peek_{n}({seed}, s.digest_{n})"), - Call::Made(n, seed) => format!("s.made_{n}({seed}, s.make_{n})"), - Call::Later(n, seed) => format!("s.digest_{n}(await s.later_{n}(s.make_{n}({seed})))"), + let Call { n, seed, .. } = *call; + let other = call.other(); + let value = match call.kind { + Kind::Echo => format!("s.digest_{n}(s.echo_{n}(s.make_{n}({seed})))"), + Kind::Lent => format!("s.lent_{n}({seed}, s.digest_{n})"), + Kind::Peek => format!("s.peek_{n}({seed}, s.digest_{n})"), + Kind::Made => format!("s.made_{n}({seed}, s.make_{n})"), + Kind::Later => format!("s.digest_{n}(await s.later_{n}(s.make_{n}({seed})))"), + Kind::Try => format!("tried(lambda: s.try_{n}({seed}), s.digest_{n})"), + // A value is passed in a `Mut`, which Rust writes back; an object is borrowed. + Kind::Swap => { + let boxed = declaration(&source, &format!("def swap_{n}(")).contains("v: Mut["); + let boxed = if boxed { "True" } else { "False" }; + format!("swapped(s.make_{n}, s.swap_{n}, s.digest_{n}, {seed}, {other}, {boxed})") + } + Kind::Pair => format!("s.pair_{n}(s.make_{n}({seed}), {seed}, s.make_{n}({other}))"), + Kind::Maybe => match seed % 2 { + 0 => format!("s.maybe_{n}(None)"), + _ => format!("s.maybe_{n}(s.make_{n}({seed}))"), + }, + Kind::Held => format!("held(s.Holder{n}, s.make_{n}, s.digest_{n}, {seed}, {other})"), + Kind::Sink => { + if echoes.insert(n) { + let _ = writeln!( + sinks, + "class Echo{n}(s.Sink{n}):\n\tdef echo(self, v: typing.Any) -> typing.Any:\n\t\tprint(\"echoed\")\n\t\treturn v\n\n" + ); + } + format!("s.sink_{n}({seed}, Echo{n}())") + } }; let _ = writeln!(body, "\tprint(f\"{} {{{value}}}\")", call.label()); } - format!("import asyncio\n\nimport {name} as s\n\n\nasync def main() -> None:\n{body}\n\nasyncio.run(main())\n") + format!( + "import asyncio\nimport typing\n\nimport {name} as s\n\n\n{PYTHON_HELPERS}{sinks}async def main() -> None:\n{body}\n\nasyncio.run(main())\n" + ) } +/// What the Python driver's lines call: `tried` digests a `Result`'s value or its error, +/// which only the binding's own exceptions hold; `swapped` a value before and after `&mut`; +/// `held` an object's values through each of its methods. +const PYTHON_HELPERS: &str = "def tried(f: typing.Any, digest: typing.Any) -> str: + try: + return f\"ok {digest(f())}\" + except Exception as e: + if type(e).__module__ != s.__name__: + raise + return f\"err {digest(typing.cast(typing.Any, e).error)}\" + + +def swapped(make: typing.Any, swap: typing.Any, digest: typing.Any, seed: int, other: int, boxed: bool) -> str: + v = s.Mut(make(seed)) if boxed else make(seed) + d = swap(v, other) + return f\"{d} {digest(v.value if boxed else v)}\" + + +def held(holder: typing.Any, make: typing.Any, digest: typing.Any, seed: int, other: int) -> str: + h = holder.new(make(seed)) + old = h.swap(make(other)) + d = h.digest() + return f\"{digest(old)} {d} {digest(h.into_inner())}\" + + +"; + /// The Swift driver, in the generated module's `main.swift`: each shape's calls in a /// function of its own, since type-checking them as one body of top-level code takes /// tens of gigabytes. @@ -1219,27 +2110,118 @@ fn swift(calls: &[Call], source: &str) -> String { true => "try! ", false => "", }; + let mut sinks = String::new(); + // Each sink's class, defined by its first call the driver runs. + let mut echoes = std::collections::BTreeSet::new(); let shapes = by_shape(calls, |call| { - let value = match call { - Call::Echo(n, seed) => format!( + let Call { n, seed, .. } = call; + let other = call.other(); + let label = call.label(); + let make = |seed: u64| format!("{}make{n}({seed})", tr(&[format!("make{n}")])); + let digest = |v: &str| format!("{}digest{n}({v})", tr(&[format!("digest{n}")])); + let value = match call.kind { + Kind::Try => { + // What it throws: an error holding the value, or the value, which is one. + let at = source.find(&format!("public func try{n}(")).unwrap(); + let body = &source[at..at + source[at..].find("\n}\n").unwrap()]; + let thrown = body.split_once("\n\tthrow ").expect("try_N throws").1; + let (thrown, error) = match thrown.split_once("(error: ") { + Some((thrown, _)) if !thrown.contains([' ', '(']) => (thrown, "e.error"), + _ => { + let declared = body.lines().next().unwrap(); + (declared.split_once("throws -> ").unwrap().1.trim_end_matches(" {"), "e") + } + }; + return format!( + "\tdo {{\n\t\tlet v = try try{n}({seed})\n\t\tprint(\"{label} ok \\({})\")\n\ + \t}} catch let e as {thrown} {{\n\t\tprint(\"{label} err \\({})\")\n\t}}", + digest("v"), + digest(error) + ); + } + Kind::Swap => { + // A value is `inout`; an object's handle is borrowed. + let at = source.find(&format!("public func swap{n}(")).unwrap(); + let (var, arg) = match source[at..].lines().next().unwrap().contains("inout") { + true => ("var", "&v"), + false => ("let", "v"), + }; + return format!( + "\tdo {{\n\t\t{var} v = {}\n\t\tlet d = {}swap{n}({arg}, {other})\n\t\tprint(\"{label} \\(d) \\({})\")\n\t}}", + make(seed), + tr(&[format!("swap{n}")]), + digest("v") + ); + } + Kind::Pair => format!( + "{}pair{n}({}, {seed}, {})", + tr(&[format!("pair{n}")]), + make(seed), + make(other) + ), + Kind::Maybe => match seed % 2 { + 0 => format!("{}maybe{n}(nil)", tr(&[format!("maybe{n}")])), + _ => format!("{}maybe{n}({})", tr(&[format!("maybe{n}")]), make(seed)), + }, + Kind::Held => { + // Its methods, which throw where they take its checked handle; a struct's + // `swap` is `mutating`. + let at = source + .find(&format!(" Holder{n}: ")) + .unwrap_or_else(|| panic!("the Swift declares Holder{n}")); + let block = &source[at..at + source[at..].find("\n}\n").unwrap()]; + let method = |name: &str| { + let line = block.lines().find(|l| l.contains(&format!(" func {name}("))).unwrap(); + (line.contains(" throws "), line.contains(" mutating ")) + }; + let t = |name: &str| if method(name).0 { "try " } else { "" }; + let var = if method("swap").1 { "var" } else { "let" }; + return format!( + "\tdo {{\n\t\t{var} h = {}Holder{n}.new({})\n\t\tlet old = {}h.swap({})\n\t\tlet d = {}h.digest()\n\ + \t\tlet c = {}h.intoInner()\n\t\tprint(\"{label} \\({}) \\(d) \\({})\")\n\t}}", + t("new"), + make(seed), + t("swap"), + make(other), + t("digest"), + t("intoInner"), + digest("old"), + digest("c") + ); + } + Kind::Sink => { + if echoes.insert(n) { + let protocol = source + .split(&format!("public protocol Sink{n}: Sendable {{\n")) + .nth(1) + .expect("the Swift declares the sink"); + let echo = protocol.lines().next().unwrap().trim(); + let _ = writeln!( + sinks, + "struct Echo{n}: Sink{n} {{\n\t{echo} {{\n\t\tprint(\"echoed\")\n\t\treturn v\n\t}}\n}}\n" + ); + } + format!("{}sink{n}({seed}, Echo{n}())", tr(&[format!("sink{n}")])) + } + Kind::Echo => format!( "{}digest{n}(echo{n}(make{n}({seed})))", tr(&[format!("digest{n}"), format!("echo{n}"), format!("make{n}")]) ), - Call::Lent(n, seed) => { + Kind::Lent => { format!( "{}lent{n}({seed}) {{ {}digest{n}($0) }}", tr(&[format!("lent{n}")]), closed(format!("digest{n}")) ) } - Call::Peek(n, seed) => { + Kind::Peek => { format!( "{}peek{n}({seed}) {{ {}digest{n}($0) }}", tr(&[format!("peek{n}")]), closed(format!("digest{n}")) ) } - Call::Made(n, seed) => { + Kind::Made => { format!( "{}made{n}({seed}) {{ {}make{n}($0) }}", tr(&[format!("made{n}")]), @@ -1247,11 +2229,11 @@ fn swift(calls: &[Call], source: &str) -> String { ) } // The future throws, so `try` always marks the call. - Call::Later(n, seed) => format!("try await digest{n}(later{n}(make{n}({seed})))"), + Kind::Later => format!("try await digest{n}(later{n}(make{n}({seed})))"), }; format!("\tprint(\"{} \\({value})\")", call.label()) }); - let mut out = String::new(); + let mut out = sinks; for (n, lines) in &shapes { let _ = writeln!(out, "func shape{n}() async throws {{\n{lines}}}\n"); } @@ -1267,7 +2249,8 @@ fn swift(calls: &[Call], source: &str) -> String { /// `rt.Bytes` to the `[]byte` a callback is lent. fn go(built: &Built, calls: &[Call], source: &str, rt: &str) -> String { let mut body = String::new(); - // How the generated `func` returns: its values, and whether an error follows them. + // How the generated `func` returns: its values, and whether an error follows them. A + // method is named after its receiver, as `(self *T) M`. let results = |function: &str| -> (usize, bool) { let at = source .find(&format!("\nfunc {function}(")) @@ -1275,7 +2258,7 @@ fn go(built: &Built, calls: &[Call], source: &str, rt: &str) -> String { let line = source[at + 1..].lines().next().unwrap(); let mut depth = 0; let mut close = 0; - for (i, c) in line.char_indices().skip(line.find('(').unwrap()) { + for (i, c) in line.char_indices().skip(5 + function.len()) { match c { '(' | '[' => depth += 1, ')' | ']' => depth -= 1, @@ -1332,16 +2315,107 @@ fn go(built: &Built, calls: &[Call], source: &str, rt: &str) -> String { values = values.join(", "), ); } + // The parameter types of a `func`, after its name. + let params = |function: &str| -> String { + let at = source + .find(&format!("\nfunc {function}(")) + .unwrap_or_else(|| panic!("the Go declares {function}")); + let line = &source[at + 7 + function.len()..]; + line[..line.find(") ").unwrap()].to_string() + }; + let mut sinks = String::new(); + // Each sink's class, defined by its first call the driver runs. + let mut echoes = std::collections::BTreeSet::new(); for call in calls { - let n = call.shape(); - let printed = match *call { - Call::Echo(n, seed) => { + let Call { n, seed, .. } = *call; + let other = call.other(); + let label = call.label(); + let made = |seed: u64| value(&format!("Make{n}"), format!("s.Make{n}({seed})")); + let digest = |v: &str| format!("one(s.Digest{n}({v}))"); + // The parameter `v` of `function` takes a value where `Digest` takes it as is: an + // object's handle, which it borrows. + let handle = |function: &str| { + let digested = params(&format!("Digest{n}")); + params(function) + .split(", ") + .any(|p| p.starts_with("v ") && p == digested) + }; + let printed = match call.kind { + Kind::Try => { + let (k, _) = results(&format!("Try{n}")); + let (values, v) = match k { + 1 => ("v".to_string(), "v".to_string()), + k => { + let names: Vec = (0..k).map(|i| format!("v{i}")).collect(); + (names.join(", "), format!("tuple{k}({})", names.join(", "))) + } + }; + let lift = if results(&format!("Digest{n}")).1 { + "liftErr" + } else { + "lift" + }; + let _ = writeln!( + body, + "\t{{\n\t\t{values}, err := s.Try{n}({seed})\n\t\tif err == nil {{\n\ + \t\t\tfmt.Println(\"{label} ok\", {})\n\t\t}} else {{\n\ + \t\t\tfmt.Println(\"{label} err\", failed({lift}(s.Digest{n}), err))\n\t\t}}\n\t}}", + digest(&v) + ); + continue; + } + Kind::Swap => { + let arg = if handle(&format!("Swap{n}")) { "v" } else { "&v" }; + let _ = writeln!( + body, + "\t{{\n\t\tv := {}\n\t\td := one(s.Swap{n}({arg}, {other}))\n\t\tfmt.Println(\"{label}\", d, {})\n\t}}", + made(seed), + digest("v") + ); + continue; + } + Kind::Pair => format!("one(s.Pair{n}({}, {seed}, {}))", made(seed), made(other)), + Kind::Maybe => match (seed % 2, handle(&format!("Maybe{n}"))) { + (0, _) => format!("one(s.Maybe{n}(nil))"), + (_, true) => format!("one(s.Maybe{n}({}))", made(seed)), + (_, false) => format!("one(s.Maybe{n}(ptr({})))", made(seed)), + }, + Kind::Held => { + let holder = format!("(self *Holder{n})"); + let swapped = value(&format!("{holder} Swap"), format!("h.Swap({})", made(other))); + let inner = value(&format!("{holder} IntoInner"), "h.IntoInner()".into()); + let _ = writeln!( + body, + "\t{{\n\t\th := one(s.NewHolder{n}({}))\n\t\told := {swapped}\n\t\td := one(h.Digest())\n\ + \t\tc := {inner}\n\t\tif err := h.Close(); err != nil {{\n\t\t\tpanic(err)\n\t\t}}\n\ + \t\tfmt.Println(\"{label}\", {}, d, {})\n\t}}", + made(seed), + digest("old"), + digest("c") + ); + continue; + } + Kind::Sink => { + if echoes.insert(n) { + let interface = source + .split(&format!("\ntype Sink{n} interface {{\n")) + .nth(1) + .expect("the Go declares the sink"); + let echo = interface.lines().next().unwrap().trim().replace("p_rt.", "rt."); + let _ = writeln!( + sinks, + "type echo{n} struct{{}}\n\nfunc (echo{n}) {echo} {{\n\tfmt.Println(\"echoed\")\n\treturn v\n}}\n" + ); + } + format!("one(s.Sink{n}_({seed}, echo{n}{{}}))") + } + Kind::Echo => { let made = value(&format!("Make{n}"), format!("s.Make{n}({seed})")); let echoed = value(&format!("Echo{n}"), format!("s.Echo{n}({made})")); format!("one(s.Digest{n}({echoed}))") } - Call::Lent(_, seed) | Call::Peek(_, seed) => { - let item = if matches!(call, Call::Lent(..)) { "Lent" } else { "Peek" }; + Kind::Lent | Kind::Peek => { + let item = if call.kind == Kind::Lent { "Lent" } else { "Peek" }; let lift = if results(&format!("Digest{n}")).1 { "liftErr" } else { @@ -1358,7 +2432,7 @@ fn go(built: &Built, calls: &[Call], source: &str, rt: &str) -> String { }; format!("s.{item}{n}({seed}, {adapt}({lift}(s.Digest{n})))") } - Call::Made(_, seed) => { + Kind::Made => { let lift = if results(&format!("Make{n}")).1 { "liftErr" } else { @@ -1366,16 +2440,17 @@ fn go(built: &Built, calls: &[Call], source: &str, rt: &str) -> String { }; format!("s.Made{n}({seed}, {lift}(s.Make{n}))") } - Call::Later(_, seed) => { + Kind::Later => { let made = value(&format!("Make{n}"), format!("s.Make{n}({seed})")); let later = value(&format!("Later{n}"), format!("s.Later{n}(ctx, {made})")); format!("one(s.Digest{n}({later}))") } }; - let _ = writeln!(body, "\tfmt.Println(\"{}\", {printed})", call.label()); + let _ = writeln!(body, "\tfmt.Println(\"{label}\", {printed})"); } format!( - "package main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\n\ts \"{name}/{name}\"\n\t\"{name}/rt\"\n)\n\n\ + "package main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"reflect\"\n\t\"strings\"\n\n\ts \"{name}/{name}\"\n\t\"{name}/rt\"\n)\n\n\ + {sinks}{GO_HELPERS}\ func one[T any](v T, err ...error) T {{\n\tif len(err) > 0 && err[0] != nil {{\n\t\tpanic(err[0])\n\t}}\n\treturn v\n}}\n\n\ {tuples}func lift[A, B any](f func(A) B) func(A) B {{\n\treturn f\n}}\n\n\ func liftErr[A, B any](f func(A) (B, error)) func(A) B {{\n\treturn func(a A) B {{ return one(f(a)) }}\n}}\n\n\ @@ -1386,34 +2461,270 @@ fn go(built: &Built, calls: &[Call], source: &str, rt: &str) -> String { ) } -/// The Dart driver, which prints each digest, a `U64`, unsigned. +/// What the Go driver's lines call besides: `ptr` points at a value, as an `Option<&T>` +/// takes it; `failed` is what `f` returns of the value a `Result`'s error holds, an object +/// that is the error or the `Value` of the runtime's error of it. +const GO_HELPERS: &str = "func ptr[T any](v T) *T { + return &v +} + +func failed[A, B any](f func(A) B, err error) B { + if v, ok := err.(A); ok { + return f(v) + } + e := reflect.ValueOf(err) + if e.Kind() != reflect.Struct || !strings.HasSuffix(e.Type().Name(), \"Error\") { + panic(err) + } + return f(e.FieldByName(\"Value\").Interface().(A)) +} + +"; + +/// The Dart driver, which prints each digest, a `U64`, unsigned. Each shape's calls are +/// a function of their own: in one `main` of the nightly set's 37,000 lines, the VM let +/// a `try350` exception its `on` clause names go uncaught, which alone it catches. fn dart(built: &Built, calls: &[Call]) -> String { let name = built.name(); + let source = std::fs::read_to_string(built.out.join("dart").join(format!("{name}.dart"))).unwrap(); + let mut sinks = String::new(); + // Each sink's class, defined by its first call the driver runs. + let mut echoes = std::collections::BTreeSet::new(); let mut body = String::new(); + let mut shapes = Vec::new(); for call in calls { + if shapes.last() != Some(&call.n) { + if !shapes.is_empty() { + body.push_str("}\n\n"); + } + let _ = writeln!(body, "Future shape{}() async {{", call.n); + shapes.push(call.n); + } // A seed as the `int` of its bits. let u = |seed: u64| format!("s.U64.fromBits({})", seed as i64); - let value = match *call { - Call::Echo(n, seed) => format!("s.digest{n}(s.echo{n}(s.make{n}({})))", u(seed)), - Call::Lent(n, seed) => format!("s.lent{n}({}, s.digest{n})", u(seed)), - Call::Peek(n, seed) => format!("s.peek{n}({}, s.digest{n})", u(seed)), - Call::Made(n, seed) => format!("s.made{n}({}, s.make{n})", u(seed)), - Call::Later(n, seed) => format!("s.digest{n}(await s.later{n}(s.make{n}({})))", u(seed)), + let Call { n, seed, .. } = *call; + let other = call.other(); + let label = call.label(); + let make = |seed: u64| format!("s.make{n}({})", u(seed)); + let digest = |v: &str| format!("${{s.digest{n}({v}).toRadixString(10)}}"); + let value = match call.kind { + Kind::Echo => format!("s.digest{n}(s.echo{n}({}))", make(seed)), + Kind::Lent => format!("s.lent{n}({}, s.digest{n})", u(seed)), + Kind::Peek => format!("s.peek{n}({}, s.digest{n})", u(seed)), + Kind::Made => format!("s.made{n}({}, s.make{n})", u(seed)), + Kind::Later => format!("s.digest{n}(await s.later{n}({}))", make(seed)), + Kind::Try => { + // What it throws: an exception holding the value, or the value, which is one. + let declared = declaration(&source, &format!("try{n}(")); + let ty = declared.split_once(" try").unwrap().0; + let at = source.find(declared).unwrap(); + let body_ = &source[at..at + source[at..].find("\n}\n").unwrap()]; + let thrown = body_.split_once("throw ").expect("try_N throws").1; + let thrown = &thrown[..thrown.find(['(', '.']).unwrap()]; + let wraps = source.contains(&format!("\nfinal class {thrown} implements Exception {{\n\tfinal ")); + let (thrown, error) = match wraps { + true => (thrown, "e.error"), + false => (ty, "e"), + }; + let _ = writeln!( + body, + " try {{\n final v = s.try{n}({});\n print('{label} ok {}');\n }} on s.{thrown} catch (e) {{\n print('{label} err {}');\n }}", + u(seed), + digest("v"), + digest(error) + ); + continue; + } + Kind::Swap => { + // A value is passed in a `Mut`, which Rust writes back; an object is borrowed. + let boxed = declaration(&source, &format!("swap{n}(")).contains("(Mut<"); + let (v, after) = match boxed { + true => (format!("s.Mut({})", make(seed)), "v.value"), + false => (make(seed), "v"), + }; + let _ = writeln!( + body, + " {{\n final v = {v};\n final d = s.swap{n}(v, {});\n print('{label} ${{d.toRadixString(10)}} {}');\n }}", + u(other), + digest(after) + ); + continue; + } + Kind::Pair => format!("s.pair{n}({}, {}, {})", make(seed), u(seed), make(other)), + Kind::Maybe => match seed % 2 { + 0 => format!("s.maybe{n}(null)"), + _ => format!("s.maybe{n}({})", make(seed)), + }, + Kind::Held => { + let _ = writeln!( + body, + " {{\n final h = s.Holder{n}({});\n final old = h.swap({});\n final d = h.digest();\n\ + \x20 final c = h.intoInner();\n h.close();\n print('{label} {} ${{d.toRadixString(10)}} {}');\n }}", + make(seed), + make(other), + digest("old"), + digest("c") + ); + continue; + } + Kind::Sink => { + if echoes.insert(n) { + let class = source + .split(&format!("\nabstract class Sink{n} {{\n")) + .nth(1) + .expect("the Dart declares the sink"); + // Its types, the binding's own prefixed. + let echo = class.lines().next().unwrap().trim().trim_end_matches(';'); + let mut prefixed = String::new(); + for (i, c) in echo.char_indices() { + let starts = !echo[..i].ends_with(|p: char| p.is_alphanumeric() || p == '_' || p == '.'); + if starts && c.is_uppercase() && !echo[i..].starts_with("String") { + prefixed.push_str("s."); + } + prefixed.push(c); + } + let echo = prefixed; + let _ = writeln!( + sinks, + "final class Echo{n} extends s.Sink{n} {{\n @override\n {echo} {{\n print('echoed');\n return v;\n }}\n}}\n" + ); + } + format!("s.sink{n}({}, Echo{n}())", u(seed)) + } + }; + let _ = writeln!(body, " print('{label} ${{({value}).toRadixString(10)}}');"); + } + if !shapes.is_empty() { + body.push_str("}\n\n"); + } + let main: String = shapes.iter().map(|n| format!(" await shape{n}();\n")).collect(); + format!("import 'package:{name}/{name}.dart' as s;\n\n{sinks}{body}Future main() async {{\n{main}}}\n") +} + +/// The web driver, run by Node over the module the export's `web` feature builds, which it +/// loads from `RUST2_WASM`: each digest, a `u64`, in decimal. +fn web(built: &Built, calls: &[Call]) -> String { + let source = std::fs::read_to_string(built.out.join("web").join("index.ts")).unwrap(); + let mut holders = std::collections::BTreeSet::new(); + let mut body = String::new(); + for call in calls { + let Call { n, seed, .. } = *call; + let other = call.other(); + let label = call.label(); + let u = |seed: u64| format!("rt.u64FromBig({seed}n)"); + let make = |seed: u64| format!("s.make{n}({})", u(seed)); + let digest = |v: &str| format!("${{dec(s.digest{n}({v}))}}"); + let value = match call.kind { + Kind::Echo => format!("s.digest{n}(s.echo{n}({}))", make(seed)), + Kind::Lent => format!("s.lent{n}({}, (v) => s.digest{n}(v))", u(seed)), + Kind::Peek => format!("s.peek{n}({}, (v) => s.digest{n}(v))", u(seed)), + Kind::Made => format!("s.made{n}({}, (seed) => s.make{n}(seed))", u(seed)), + Kind::Later => format!("s.digest{n}(await s.later{n}({}))", make(seed)), + Kind::Try => { + let _ = writeln!( + body, + "{{\n\tconst r = s.try{n}({});\n\tprint(r.ok ? `{label} ok {}` : `{label} err {}`);\n}}", + u(seed), + digest("r.value"), + digest("r.error") + ); + continue; + } + Kind::Swap => { + // A value is passed in a place, which Rust writes back; an object is borrowed. + let declared = declaration(&source, &format!("swap{n}(")); + let (v, after) = match declared.contains("v: rt.Place<") { + true => (format!("{{ v: plain({}) }}", make(seed)), "v.v"), + false => (make(seed), "v"), + }; + let _ = writeln!( + body, + "{{\n\tconst v = {v};\n\tconst d = s.swap{n}(v, {});\n\tprint(`{label} ${{dec(d)}} {}`);\n}}", + u(other), + digest(after) + ); + continue; + } + Kind::Pair => format!("s.pair{n}({}, {}, {})", make(seed), u(seed), make(other)), + Kind::Maybe => match seed % 2 { + 0 => format!("s.maybe{n}(undefined)"), + _ => format!("s.maybe{n}({})", make(seed)), + }, + Kind::Held => { + holders.insert(n); + let _ = writeln!( + body, + "{{\n\tconst h = Holder{n}.new({});\n\tconst old = Holder{n}.swap(h, {});\n\tconst d = Holder{n}.digest(h);\n\ + \tconst c = Holder{n}.intoInner(h);\n\th[Symbol.dispose]();\n\tprint(`{label} {} ${{dec(d)}} {}`);\n}}", + make(seed), + make(other), + digest("old"), + digest("c") + ); + continue; + } + Kind::Sink => format!( + "s.sink{n}({}, {{ echo: (v) => {{ print(\"echoed\"); return v; }} }})", + u(seed) + ), }; - let _ = writeln!(body, " print('{} ${{({value}).toRadixString(10)}}');", call.label()); + let _ = writeln!(body, "print(`{label} ${{dec({value})}}`);"); } - format!("import 'package:{name}/{name}.dart' as s;\n\nFuture main() async {{\n{body}}}\n") + let imports: String = holders + .iter() + .map(|n| format!("import * as Holder{n} from \"./web/Holder{n}.ts\";\n")) + .collect(); + format!( + "{WEB_HEAD}{imports}\nawait s.load(process.getBuiltinModule(\"node:fs\").readFileSync(process.env.RUST2_WASM!));\n\n{body}" + ) } +/// The web driver's head: what it imports, and Node's globals, which the binding's types +/// leave out. +const WEB_HEAD: &str = "import * as rt from \"./web/rt.ts\"; +import * as s from \"./web/index.ts\"; + +declare const process: { + readonly env: Record; + getBuiltinModule(id: \"node:fs\"): { readFileSync(path: string): Uint8Array }; +}; + +const print = (line: string) => console.log(line); + +/** A `u64` in decimal. */ +const dec = (v: rt.u64) => (typeof v === \"number\" ? `${v}` : `${rt.u64ToBig(v)}`); + +/** `v` as a place holds it: the bytes Rust returned copied into arrays. */ +type Plain = T extends rt.Bytes + ? Uint8Array + : T extends readonly unknown[] + ? { -readonly [K in keyof T]: Plain } + : T; + +function plain(v: T): Plain { + if (v instanceof rt.Bytes) return v.slice() as Plain; + if (Array.isArray(v)) return v.map(plain) as Plain; + return v as Plain; +} + +"; + /// Build and run `language`'s driver of `built`, checking it against the native transcript. fn round_trip(built: &Built, language: &str) { + // The test that drew the shapes may have passed, its message captured: each failing one + // names the seed again. + if let Some((_, seed)) = built.krate.drawn { + eprintln!("shapes past the fixed set drawn from RUST2_SHAPES_SEED={seed}"); + } let name = built.name(); let dir = built.dir(language); let exe = dir.join(format!("shapes{}", std::env::consts::EXE_SUFFIX)); - let known = built.known(language); - let calls: Vec = (built.krate.calls.iter().copied()) - .filter(|call| !known.contains(&call.shape())) - .collect(); + let transcript = match language { + "web" => built.wasm_transcript(), + _ => &built.krate.transcript, + }; + let expected = built.expected(language, transcript); + let calls: Vec = expected.iter().map(|(call, _)| *call).collect(); let calls = &calls[..]; let output = match language { "c" => { @@ -1463,8 +2774,9 @@ fn round_trip(built: &Built, language: &str) { writes: &[&classes], }; step.run( + // The nightly's binding is past what kotlinc's default heap of 512 MB compiles. Command::new("kotlinc") - .args(["-Werror", "-cp"]) + .args(["-J-Xmx4g", "-Werror", "-cp"]) .arg(&jar) .arg(&generated) .arg(&main) @@ -1559,24 +2871,53 @@ fn round_trip(built: &Built, language: &str) { let dill = super::dart_compile(&name, &dir, &built.out.join("dart"), &lib, &main, &[]); run(&mut super::dart_run(&dill, &[])) } + "web" => { + let (module, _) = built.web(); + let main = dir.join("main.ts"); + step::mirror(&built.out.join("web"), &dir.join("web")); + step::write(&main, web(built, calls)); + // Type-checked as a caller's project would be, then run by Node. + let config = dir.join("tsconfig.json"); + step::write(&config, super::TSCONFIG.replace("\"api.ts\"", "\"main.ts\"")); + let checked = dir.join("checked"); + let web = dir.join("web"); + let step = Step { + stamp: &step::stamp(&checked), + reads: &[&main, &web, &config], + writes: &[], + }; + step.run(Command::new("tsc").arg("-p").arg(&dir)); + run(Command::new("node") + .arg("main.ts") + .current_dir(&dir) + .env("RUST2_WASM", module)) + } _ => unreachable!("no driver for {language}"), }; - built.check(language, &known, &output); + built.check(language, &expected, &output); } /// The gate's subset of shapes, built once. fn gated() -> &'static Built { static BUILT: std::sync::OnceLock = std::sync::OnceLock::new(); - BUILT.get_or_init(|| build("shapes", &gate())) + BUILT.get_or_init(|| build("shapes", &gate(), None)) } -/// Every sampled shape, built once: the nightly's. +/// The nightly's shapes, built once: the fixed set, then as many again drawn from the +/// day's seed. fn sampled() -> &'static Built { static BUILT: std::sync::OnceLock = std::sync::OnceLock::new(); - BUILT.get_or_init(|| build("sampled", &shapes(SAMPLES))) + BUILT.get_or_init(|| { + let mut all = shapes(SAMPLES); + let fixed = all.len(); + let seed = day_seed(); + eprintln!("the nightly's new shapes are drawn from seed {seed}: RUST2_SHAPES_SEED={seed} reproduces them"); + all.extend(drawn(seed, SAMPLES)); + build("sampled", &all, Some((fixed, seed))) + }) } -/// How many shapes the nightly samples beyond each leaf in each wrapper. +/// How many shapes the nightly samples from each seed. const SAMPLES: usize = 64; macro_rules! languages { @@ -1602,4 +2943,4 @@ macro_rules! languages { }; } -languages!(c, cpp, kotlin, python, swift, go, dart); +languages!(c, cpp, kotlin, python, swift, go, dart, web); diff --git a/quest/m1/ffi-export/signaling-nan.md b/quest/m1/ffi-export/signaling-nan.md new file mode 100644 index 000000000..80cca2842 --- /dev/null +++ b/quest/m1/ffi-export/signaling-nan.md @@ -0,0 +1,40 @@ +# [M] NaN bits cross Python, Dart, and the web exactly + +## Goal + +Every `f32` and `f64` crosses each binding and returns with its exact bits, a NaN's +payload, sign, and quiet bit included. Today Python and Dart quiet an `f32`'s signaling +NaN, and the web binding loses any NaN's bits: it quiets a signaling NaN, and once V8 +optimizes the code it canonicalizes a quiet NaN's payload and sign too. The shape tests' +`KNOWN` entries for them are deleted. + +## Plan + +Decided with the maintainer on 2026-11-19, in the pick of [#879](https://github.com/kixel-ai/rust2/pull/879): ✅ keep it, and (a) the web holds floats as runtime `rt.f32`/`rt.f64` types that keep NaN bits, with checked conversion to and from `number`, as the lossless-types rule prescribes (rejected: (b) documenting that the web keeps no NaN's bits; dropping the quest). + +Found by the shape tests (`ffi/tests/fixtures/shapes.rs`), whose floats draw signaling and +quiet NaNs, and whose native transcript marks each call that digests a NaN of its own bits +(`nan signaling f32`, `nan quiet f64`), which `KNOWN` skips. AGENTS.md's lossless rule +allows a native type only where it is exact. + +Python's `float` and Dart's `double` are 64-bit, and widening an `f32` to them with the +hardware's conversion sets the quiet bit (`cvtss2sd` does), as narrowing back does. +Recommended: widen and narrow by bits instead, keeping the native type: an `f32` NaN's +payload shifted into an `f64` NaN's, its quiet bit kept, so the round trip is exact. + +On the web the engine is where the bits go, so no glue fix keeps them in a number. Run in a +loop under Node 24, `Holder246.swap(h, v)` with `v[0]` the quiet NaN `0x7ffde633` hands Rust +those bits for the first 8 iterations, then `0x7fc00000` once V8 tiers the code up; the +nightly for `RUST2_SHAPES_SEED=20261119` failed the same way on `held 246` and `held 253` +(`f64` `0xfffff56f76bde633`). `tests/api.ts` already keeps NaN bits out of numbers for +Deno's V8. + +## Waiting + +- Maintainer: how the web binding holds a float, since a number cannot keep a NaN's bits. + Options: (a) a runtime float, `rt.f32` and `rt.f64`, a number but for a NaN of its own + bits, which crosses as its bits, as `rt.u64` is a number but past a safe integer, + breaking for every web caller of a float; (b) floats stay numbers, and the README says + the web keeps no NaN's bits, which breaks the lossless rule for one value class. An agent + cannot pick, since it is a public API break against a rule only the maintainer may relax. + Recommended: (a), since V8 is where the bits are lost. diff --git a/quest/m1/ffi-proofs/README.md b/quest/m1/ffi-proofs/README.md index 4c66dd9da..2a1b2cd43 100644 --- a/quest/m1/ffi-proofs/README.md +++ b/quest/m1/ffi-proofs/README.md @@ -16,5 +16,4 @@ Go's race detector, Miri, and loom on `rt/borrow.rs` and the hold counts. This epic changes no generated API, so it does not hold moq's pin ([the exporter epic](/quest/m1/ffi-export/README.md#plan)). Considered and dropped by the audit: Kani, since the runtime's `unsafe` is FFI and concurrency, where Miri and loom fit -better; and mutation testing of the generators until [the shapes](/quest/m1/ffi-proofs/shapes.md) -land, since every mutant reruns eight languages. +better; and mutation testing of the generators, since every mutant reruns eight languages. diff --git a/quest/m1/ffi-proofs/shapes.md b/quest/m1/ffi-proofs/shapes.md deleted file mode 100644 index 51a678d8a..000000000 --- a/quest/m1/ffi-proofs/shapes.md +++ /dev/null @@ -1,26 +0,0 @@ -# [M] Sample more shapes, positions, and seeds - -## Goal - -The shape tests (`ffi/tests/fixtures/shapes.rs`) cover: - -- the web binding, which `languages!` leaves out today; -- a `Result` with each sampled error, a `&mut` in-out parameter, several parameters, - `Option<&T>`, an object's methods, and a callback, beside today's echo, closure, and - future positions; -- floats compared by `to_bits`, NaN payloads included, since `Debug` cannot show them; -- besides the fixed `0x5eed` set the `KNOWN` list checks, a set drawn nightly from a dated - seed the run prints, so the nightly explores new shapes and a failure reproduces. - -## Plan - -Found by the 2026-11-18 language audit: `languages!(c, cpp, kotlin, python, swift, go, -dart)` at `shapes.rs:1623` predates the web target ([#847](https://github.com/kixel-ai/rust2/pull/847)), -and JavaScript is where lossless `u64` and `f32` are hardest; floats exclude NaN today, -so "every value crosses exactly" is untested for them; a `Result` whose error is `()`, -which crashed once_cell's export, is refused since the corpus found it. - -A shape refused by design stays left out by the census, as today. Each new mismatch found -goes into `KNOWN` with a fix quest, or is fixed here. - -Public API: none.