Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,11 @@ jobs:
- name: Build
run: nix build .#ts

# The fixtures against std checked in their own target directories, as outside Nix,
# whose snapshots hold the flake's sysroot to the same output.
- name: Fixtures without the sysroot
run: nix develop --command env RUST2_TS_SYSROOT= just ts test --test fixtures

uniffi-fixtures:
name: UniFFI fixtures
# x86_64: nixpkgs marks Swift 6.3 broken on aarch64-linux, where it fails to link with
Expand Down
27 changes: 27 additions & 0 deletions flake.nix
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,31 @@
# the repo toolchain. ts/justfile puts it first for its recipes.
tsToolchain = pkgs.rust-bin.fromRustupToolchainFile ./ts/rust-toolchain.toml;
quest-cli = quest.packages.${system}.default;
# std checked once per toolchain for the TypeScript driver's target, with the
# profile translation reads its MIR at, as the sysroot rust2-ts takes instead of
# checking std with -Zbuild-std in every target directory. Offline, from the
# crates rust-src vendors.
rust2-ts-std = pkgs.stdenv.mkDerivation {
name = "rust2-ts-std";
dontUnpack = true;
nativeBuildInputs = [ tsToolchain ];
buildPhase = ''
export CARGO_HOME=$TMPDIR/cargo CARGO_UNSTABLE_PROFILE_RUSTFLAGS=true
mkdir -p sysroot/src && cd sysroot
printf '[package]\nname = "sysroot"\nversion = "0.0.0"\nedition = "2024"\n' > Cargo.toml
touch src/lib.rs
# The crates ts/src/cli.rs's STD names.
cargo check --offline --target wasm32-unknown-unknown -Zbuild-std=core,alloc,std,panic_abort \
--config "source.crates-io.replace-with = 'vendored'" \
--config "source.vendored.directory = '${tsToolchain}/lib/rustlib/src/rust/library/vendor'" \
--config ${./ts/src/profile.toml}
'';
installPhase = ''
lib=$out/lib/rustlib/wasm32-unknown-unknown/lib
mkdir -p $lib
find target/wasm32-unknown-unknown -name 'lib*.rmeta' ! -name 'libsysroot-*' -exec cp -t $lib {} +
'';
};
# The driver links its nightly's rustc libraries, and runs cargo from it too.
rust2-ts =
(pkgs.makeRustPlatform {
Expand Down Expand Up @@ -101,6 +126,7 @@
# tsc checks bindings, from the caller's PATH first.
postInstall = ''
wrapProgram $out/bin/rust2-ts --set-default CARGO ${tsToolchain}/bin/cargo \
--set-default RUST2_TS_SYSROOT ${rust2-ts-std} \
--suffix PATH : ${pkgs.typescript}/bin
'';
# The tests need node, tsc, and wasm builds: `just ts test` runs them.
Expand Down Expand Up @@ -275,6 +301,7 @@
kotlin
];
RUST2_TS_TOOLCHAIN = tsToolchain;
RUST2_TS_SYSROOT = rust2-ts-std;
KOTLINX_COROUTINES = "${kotlinx-coroutines}";
};
# `just ts bench` adds Chrome and Firefox as engines, and wasm-bindgen for moq-wasm's
Expand Down
1 change: 0 additions & 1 deletion quest/m2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,5 @@ The fork's feature quests folded into the [exporter](/quest/m1/ffi-export/README
- [Async in the TypeScript translator](/quest/m2/ts-async.md) - moq-net's and kio's async code translates with Rust's lazy futures
- [Shared ownership](/quest/m2/ts-shared.md) - `Arc`/`Rc`/`Weak` keep exact counts, and locks and atomics become plain values
- [Translate moq-net's model to TypeScript](/quest/m2/ts-model/README.md) - the model translates with kio, moq-clock, and moq-pattern, for moq to compare against WASM
- [Translate against the prebuilt std](/quest/m2/ts-std-build.md) - measure whether `-Zbuild-std` is still needed now that panics abort, and drop it if not
- [Waiting: Swift 6.4 in nixpkgs-unstable](/quest/m2/uniffi-swift-arm.md) - the nightly UniFFI fixtures move back to arm
- [Waiting: ubrn reads UniFFI 0.32](/quest/m2/ubrn-release.md) - the binding benchmark's browser runner pins a ubrn release, not an open PR
21 changes: 0 additions & 21 deletions quest/m2/ts-std-build.md

This file was deleted.

4 changes: 3 additions & 1 deletion ts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,9 @@ rust2-ts --config ts.toml (--out dir | --census file.md) [--manifest-path Cargo.
```

It runs `cargo check` on the package with itself as `RUSTC_WRAPPER`, for
`wasm32-unknown-unknown`, with std built from source for it with every function's MIR, which needs the toolchain's `rust-src`. Every crate compiles as rustc
`wasm32-unknown-unknown`, against std checked from source with every function's MIR. The flake's
package and dev shell check it once per toolchain and pass it as `RUST2_TS_SYSROOT`; without
that, each target directory checks it with `-Zbuild-std`, which needs the toolchain's `rust-src`. Every crate compiles as rustc
would; the configured library is also translated, after type and borrow checking, starting from its roots and following
what they reach. Every crate it depends on but std's translates with it as one program,
as far as the roots reach: each compiles with every function's MIR, and its types and
Expand Down
85 changes: 48 additions & 37 deletions ts/src/cli.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
//! The command line: run cargo on the configured package with this binary as rustc.

use std::path::Path;
use std::process::ExitCode;

use rust2_driver::cargo::{Args, Check, Output};
Expand All @@ -8,7 +9,7 @@ use crate::config::{self, Config};

/// Translation type-checks for this target: its 32-bit `usize` is what a JS number
/// holds exactly, and its `cfg`s are the browser's.
const TARGET: &str = "wasm32-unknown-unknown";
pub const TARGET: &str = "wasm32-unknown-unknown";

const USAGE: &str = "usage: rust2-ts --config <ts.toml> (--out <dir> | --census <file>) [--manifest-path <Cargo.toml>] [-- <cargo args>]

Expand Down Expand Up @@ -38,17 +39,18 @@ fn try_run(args: &[String]) -> Result<ExitCode, String> {
Output::Dir(out) => (config::env::OUT, out.as_os_str()),
Output::Census(census) => (config::env::CENSUS, census.as_os_str()),
};
// `cargo clean` reads the profiles too.
let sysroot = sysroot()?;
// `cargo clean` reads the profiles too. The sysroot is absolute for the wrapper, which
// cargo runs elsewhere, and empty for none.
let env = [
(config::env::CONFIG, args.config.as_os_str()),
output,
("CARGO_UNSTABLE_PROFILE_RUSTFLAGS", "true".as_ref()),
(config::env::SYSROOT, sysroot.as_deref().unwrap_or("").as_ref()),
];
let mut cargo = std_from_source();
cargo.push("--config".into());
cargo.push(format!("profile.dev.rustflags={}", toml_list(FLAGS)));
// The profile those flags and std's are set on, which cargo refuses to combine with
// one the cargo args pick.
let mut cargo = profile_args(sysroot.is_none());
// The profile `PROFILE` sets, which cargo refuses to combine with one the cargo args
// pick.
cargo.extend(["--profile".into(), "dev".into()]);
cargo.extend(args.cargo);
Check {
Expand All @@ -62,39 +64,48 @@ fn try_run(args: &[String]) -> Result<ExitCode, String> {
.run()
}

/// What every crate compiles with, through the profile, which cargo's fingerprint sees,
/// for a crate cached without them to compile again: the package and every crate it
/// depends on translate as one program.
///
/// Every function's MIR, which a `cargo check` encodes for no dependency's functions
/// otherwise. Without the checks of undefined behavior that debug builds add around raw
/// pointers, such as a box's: JS has no pointers to check. With every call where the
/// source has it, as a release build's inlining would not keep it, since a call is what a
/// binding or a std function's translation stands in for. And with `rust2_ts` set, so a
/// crate can give translation a safe twin of its `unsafe`. std's crates have their own
/// below, and the rest of std's take these to no effect, since none translates.
const FLAGS: &[&str] = &[
"-Zalways-encode-mir",
"-Zub-checks=no",
"-Zinline-mir=no",
"--cfg=rust2_ts",
];
/// The cargo config of the profile every crate compiles with, one `--config` per line.
const PROFILE: &str = include_str!("profile.toml");

fn toml_list(items: &[&str]) -> String {
let items: Vec<String> = items.iter().map(|item| format!("{item:?}")).collect();
format!("[{}]", items.join(", "))
/// The std crates `-Zbuild-std` checks, which flake.nix's sysroot holds too.
const STD: &str = "-Zbuild-std=core,alloc,std,panic_abort";

/// Cargo args that set [`PROFILE`], and check std from source if `std`.
fn profile_args(std: bool) -> Vec<String> {
let mut args = Vec::new();
if std {
args.push(STD.to_string());
}
for line in PROFILE
.lines()
.filter(|line| !line.is_empty() && !line.starts_with('#'))
{
args.extend(["--config".to_string(), line.to_string()]);
}
args
}

/// Cargo args that build std from source with every function's MIR, so the bodies of
/// std's combinators lower as the crate's do: the target's prebuilt std encodes the MIR
/// of few, optimized. Set through the profile, which cargo's fingerprint sees.
fn std_from_source() -> Vec<String> {
let mut args = vec!["-Zbuild-std=core,alloc,std,panic_abort".to_string()];
for krate in ["core", "alloc", "std"] {
args.push("--config".into());
args.push(format!(
"profile.dev.package.{krate}.rustflags=[\"-Zalways-encode-mir\"]"
/// The sysroot of std checked with [`PROFILE`] for [`TARGET`], which rustc takes for the
/// target's crates instead of a `-Zbuild-std` in every target directory, if set:
/// absolute.
pub fn sysroot() -> Result<Option<String>, String> {
let dir = match std::env::var(config::env::SYSROOT) {
Ok(dir) if !dir.is_empty() => dir,
Err(std::env::VarError::NotUnicode(_)) => return Err(format!("{} is not UTF-8", config::env::SYSROOT)),
_ => return Ok(None),
};
let dir = std::path::absolute(&dir)
.map_err(|err| format!("resolving {}: {err}", config::env::SYSROOT))?
.into_os_string()
.into_string()
.expect("a UTF-8 path resolves to a UTF-8 path");
let lib = Path::new(&dir).join("lib/rustlib").join(TARGET).join("lib");
if !lib.is_dir() {
return Err(format!(
"{} is {dir}, which holds no std for {TARGET} in {}",
config::env::SYSROOT,
lib.display()
));
}
args
Ok(Some(dir))
}
2 changes: 2 additions & 0 deletions ts/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -80,4 +80,6 @@ pub mod env {
pub const OUT: &str = "RUST2_TS_OUT";
/// Where to write a census instead of translating, absolute.
pub const CENSUS: &str = "RUST2_TS_CENSUS";
/// A sysroot of std checked for translation, which flake.nix builds once per toolchain.
pub const SYSROOT: &str = "RUST2_TS_SYSROOT";
}
8 changes: 7 additions & 1 deletion ts/src/driver.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,17 @@ use rustc_interface::interface::Compiler;
use rustc_middle::ty::TyCtxt;

use crate::config::{self, Config};
use crate::{lower, program};
use crate::{cli, lower, program};

/// Run as rustc, with `args` as cargo passed them: the wrapped rustc's path first.
pub fn run(args: Vec<String>) -> ExitCode {
rust2_driver::rustc::run("rust2-ts", args, |args| {
// The target's crates only: build scripts and proc macros run on the host, with its std.
if args.windows(2).any(|w| w[0] == "--target" && w[1] == cli::TARGET)
&& let Some(sysroot) = cli::sysroot()?
{
args.extend(["--sysroot".into(), sysroot]);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include the selected sysroot in Cargo's cache identity

When the sysroot changes without a toolchain or shared-rustflags change, Cargo can reuse dependencies checked against the previous std because this flag is added inside the wrapper, after Cargo computes its fingerprints. For example, changing only profile.dev.package.core.rustflags in profile.toml rebuilds the Nix sysroot but leaves cached target dependencies intact. Check::run cleans only the configured package, so its subsequent compilation fails with E0460 against those dependencies until the target directory is manually cleaned. Include the resolved sysroot in Cargo-visible fingerprint inputs or isolate target artifacts by sysroot identity so updating the dev shell does not break an existing translation directory.

Useful? React with 👍 / 👎.

}
Ok(Translate::for_args(args)?.map(|job| Box::new(job) as rust2_driver::rustc::Job))
})
}
Expand Down
17 changes: 17 additions & 0 deletions ts/src/profile.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# The profile every crate compiles with, one `--config` per line, which cargo's
# fingerprint sees, for a crate cached without it to compile again: the package and every
# crate it depends on translate as one program. flake.nix checks std with it too.
#
# Every function's MIR, which a `cargo check` encodes for no dependency's functions
# otherwise. Without the checks of undefined behavior that debug builds add around raw
# pointers, such as a box's: JS has no pointers to check. With every call where the source
# has it, as a release build's inlining would not keep it, since a call is what a binding
# or a std function's translation stands in for. And with `rust2_ts` set, so a crate can
# give translation a safe twin of its `unsafe`.
profile.dev.rustflags = ["-Zalways-encode-mir", "-Zub-checks=no", "-Zinline-mir=no", "--cfg=rust2_ts"]
# std's own crates, with every function's MIR, so the bodies of std's combinators lower as
# the crate's do: the target's prebuilt std encodes the MIR of few, optimized. The rest of
# std's take the flags above to no effect, since none translates.
profile.dev.package.core.rustflags = ["-Zalways-encode-mir"]
profile.dev.package.alloc.rustflags = ["-Zalways-encode-mir"]
profile.dev.package.std.rustflags = ["-Zalways-encode-mir"]