Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
8ade968
chore(quest): claim quest/m1/ffi-packaging/reproducible.md
kixelated Nov 19, 2026
b3fa890
Make binding releases reproducible and attested
kixelated Nov 19, 2026
815fbff
tmp: diagnose Apple rebuild differences
kixelated Nov 19, 2026
de9f503
tmp: api fixture only
kixelated Nov 19, 2026
448ec1e
tmp: upload the differing dylibs
kixelated Nov 19, 2026
57d6cf8
Name the dylib by rpath, and stamp no time in xcodebuild's libtool
kixelated Nov 19, 2026
50d316a
Sort the XCFramework's libraries, which xcodebuild lists at random
kixelated Nov 19, 2026
8248097
Merge remote-tracking branch 'origin/main' into quest/m1/ffi-packagin…
kixelated Nov 19, 2026
1c28382
Keep cargo's RUSTFLAGS tokenization and each dependency source apart …
kixelated Nov 19, 2026
6612a32
Settle the review's findings on the reproducible build
kixelated Nov 19, 2026
d5a963b
Merge origin/main into the reproducible quest, and route Go's librari…
kixelated Nov 19, 2026
cf06390
Attest the Go module, and map C++'s paths as C's
kixelated Nov 19, 2026
7f2fc79
Name the members of a static library a rebuild made differently
kixelated Nov 19, 2026
ee30610
Show where a rebuilt static library's member first differs
kixelated Nov 19, 2026
d6f8f36
Rebuild from a new export at the same path, as dlltool names Windows …
kixelated Nov 19, 2026
ea08271
Zero the random directory in a Windows GNU library's import symbols
kixelated Nov 19, 2026
72c8ea1
Zero the random import directory only in the import library's members…
kixelated Nov 20, 2026
32b85f1
Merge origin/main into the reproducible quest, and route C archives a…
kixelated Nov 20, 2026
90ffa30
Attest the npm tarball with the web export's own SBOM
kixelated Nov 20, 2026
88480ad
Name the C archive's dylib by its versioned soname under the reproduc…
kixelated Nov 20, 2026
57d437c
Stamp no time in the C archive's prelinked static library on Apple, a…
kixelated Nov 20, 2026
ad7c3b7
Make npm and JSR wait for attest, and say what it attests
kixelated Nov 20, 2026
5435001
quest: record the maintainer's follow-up answers
kixelated Nov 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/bindings-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ jobs:
features: doubled
example: transcript
python-check: ffi/tests/api.py
# The small crate, so the second build costs little: moq's stand-in leaves it to its release.
reproduce: true
# api.cpp rather than api.c, whose broken cases name items only wasm32 has.
c-check: ffi/tests/api.cpp
web-manifest-path: ffi/fixtures/api/Cargo.toml
Expand Down
31 changes: 30 additions & 1 deletion .github/workflows/bindings.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,10 @@ on:
description: The main.swift of an executable run on the installed Swift package
default: ''
type: string
reproduce:
description: Build every host's packages a second time, from a new export, and fail unless the bytes match
default: false
type: boolean
c-check:
description: A C or C++ program built by clean CMake and pkg-config consumers of each C archive
default: ''
Expand Down Expand Up @@ -327,6 +331,20 @@ jobs:
run: |
read -ra targets <<< "$TARGETS"
python -I rust2/ffi/release/package.py build --manifest-path "source/$MANIFEST" --features "$FEATURES" --export export --out dist --example "$EXAMPLE" "${targets[@]}"
# The build again, as a rerun makes it: the same packages byte for byte, from a new
# export at the same path, since dlltool names the Windows GNU library's import symbols
# after the target directory. It doubles the job's time.
- name: Rebuild
if: inputs.reproduce
env:
TARGETS: ${{ matrix.targets }}
GO_MODULE: ${{ inputs.go-module }}
run: |
read -ra targets <<< "$TARGETS"
mv export first-export
python3 -I rust2/ffi/release/package.py export --manifest-path "source/$MANIFEST" --features "$FEATURES" --api api --go-module "$GO_MODULE" --out export "${targets[@]}"
python -I rust2/ffi/release/package.py build --manifest-path "source/$MANIFEST" --features "$FEATURES" --export export --out again/dist "${targets[@]}"
python3 -I rust2/ffi/release/package.py same dist again/dist
- name: Checks
env:
PYTHON_CHECK: ${{ inputs.python-check }}
Expand All @@ -347,7 +365,8 @@ jobs:
if-no-files-found: error

# The hosts' exports must be one binding: a module and a library from different builds
# refuse to load, but two wheels of different bindings would each load.
# refuse to load, but two wheels of different bindings would each load. Their SBOMs, of
# the one dependency graph, must match as well.
agree:
needs: [facts, build]
runs-on: ubuntu-24.04
Expand All @@ -364,6 +383,10 @@ jobs:
echo '::error::The build hosts exported different bindings.'
exit 1
fi
if [[ $(sha256sum builds/*/sbom.cdx.json | cut -d' ' -f1 | sort -u | wc -l) != 1 ]]; then
echo '::error::The build hosts listed different dependencies.'
exit 1
fi

# The release's API, which the next release is checked against, and its changelog from
# the previous release's: what the caller's release publishes beside the packages.
Expand Down Expand Up @@ -627,6 +650,7 @@ jobs:
WEB_FEATURES: ${{ inputs.web-features }}
WEB_EXAMPLE: ${{ inputs.web-example }}
WEB_CHECK: ${{ inputs.web-check }}
REPRODUCE: ${{ inputs.reproduce }}
run: |
if [[ "$(realpath -m "source/$WEB_MANIFEST")" == "$(realpath -m "source/$MANIFEST")" ]]; then
if [[ -n "$WEB_FEATURES$WEB_EXAMPLE" ]]; then
Expand All @@ -639,6 +663,11 @@ jobs:
fi
python3 -I rust2/ffi/release/package.py web --manifest-path "source/$WEB_MANIFEST" --features "$features" --scope "$SCOPE" --version "$VERSION" --example "$example" --out web
if [[ -n "$WEB_CHECK" ]]; then cp "source/$WEB_CHECK" web/check.ts; fi
# The build again, from a new export, which must pack the same tarball.
if [[ "$REPRODUCE" == true ]]; then
python3 -I rust2/ffi/release/package.py web --manifest-path "source/$WEB_MANIFEST" --features "$features" --scope "$SCOPE" --version "$VERSION" --example "" --out web-again
python3 -I rust2/ffi/release/package.py same web web-again
fi
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: ${{ needs.facts.outputs.name }}-web
Expand Down
47 changes: 42 additions & 5 deletions ffi/release/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,16 +48,15 @@ Releasing is pushing `<crate>-v<version>`, the crate's version, which the releas
Called without a tag, such as from a nightly, it runs everything up to publication.
A release that failed while publishing is recovered with **Re-run failed jobs**, which keeps
the run's successful build and its artifacts: each publishing job then finishes what the
first attempt left, uploading only what is missing. A rebuild makes other bytes, as wheels
and the XCFramework archive carry build times, so **Re-run all jobs** fails wherever a
registry already holds the first attempt's files.
first attempt left, uploading only what is missing. A rebuild makes the same bytes
([Reproducible builds](#reproducible-builds)), so **Re-run all jobs** works as well.
A Cargo prerelease, such as `1.0.0-rc.1`, is a GitHub prerelease too.

The builder takes the crate's `Cargo.toml`, the `go-module` path, which without it is the
export's own `<prefix>` that no proxy serves, and optional inputs the caller's tests own:
`features` to ship, an `example` whose stdout is native Rust's transcript, and the
`python-check`, `swift-check`, `c-check`, and `go-check` programs that must print it through
the installed packages. Without a check, a consumer only imports the module, or links a
the installed packages, and `reproduce`, which builds everything twice. Without a check, a consumer only imports the module, or links a
function of each C and C++ header. A `c-check` is C or C++ by its extension. The web package
takes `web-manifest-path`, the package exported for `wasm32-wasip1`, often the crate's own,
which no Cargo fact says builds for the web, and `scope`; and an optional `web-check`, a
Expand Down Expand Up @@ -91,7 +90,7 @@ crate apart from it takes `web-features` and `web-example` instead.
`win_amd64`.
6. Swift: static libraries for macOS arm64 at macOS 13, and iOS arm64 and its simulator
at iOS 16, the floors of the generated Swift's `Swift.Duration`, each checked against
every object's declared minimum OS; one XCFramework of the three, zipped with `ditto`,
every object's declared minimum OS; one XCFramework of the three, zipped by the builder itself,
its SHA-256 checked against `swift package compute-checksum`; and a `Package.swift`
naming it at `https://github.com/<repo>/releases/download/<crate>-v<version>/<Module>FFI.xcframework.zip`.
The package depends on swift-collections by the release tag at the exporter's pinned
Expand Down Expand Up @@ -175,6 +174,44 @@ wheels, C++ consumers, and web package must print its native transcript, and on
stand-in facade at the exporter's moq pin, through `moq/moq.py`, `moq/moq.swift`,
`moq/moq.cpp`, and `moq/moq.go`, with its wasm32 stand-in as the web package.

## Reproducible builds

A rebuild of a commit makes the same bytes, so **Re-run all jobs** finds each registry
holding exactly the files it would upload, and anyone with the commit can check them.

- **Time.** Every file takes `SOURCE_DATE_EPOCH`, or else the commit's time, whatever its
archive format: the builder writes each ZIP itself, with sorted entries, fixed modes, and
no host, umask, or time of its own, and rewrites each repaired wheel the same way, its
RECORD rebuilt. Zlib's own output is the one thing a host could still change.
- **Paths.** The crate's workspace, the exported crate, and `CARGO_HOME` are remapped to
fixed names for rustc and the C compilers, so panic messages and debug info name no
checkout. This takes `RUSTFLAGS` over a `.cargo/config.toml`'s `rustflags`, so a config
that sets any fails the build: export them as `RUSTFLAGS`, which the builder keeps. Apple
builds also stamp no archive time (`ZERO_AR_DATE`), and Windows links with `/Brepro`.
- **Checked.** The Swift job builds the XCFramework twice and fails unless both ZIPs match.
With `reproduce: true`, every build host builds a second time, from a new export at the same
path, and `package.py same` fails unless each wheel, static library, and C archive, the Go
module's included, matches byte for byte, and the web job packs its npm tarball twice; rust2 turns it on
for its api fixture. It doubles the build, so a large crate may leave it to a nightly.
- **Not yet.** The exporter writes the crate's absolute path into the generated crate's
version, which cargo hashes into every symbol, so a rebuild must keep the crate at the
same path, as a rerun does
([planned](/quest/m1/ffi-packaging/relocatable.md)). So must the export directory of a
Windows GNU library, which is the Go module's: dlltool names the import library's symbols
after the path of the target directory, which no `--remap-path-prefix` reaches. rustc's
random temporary directory in that name is replaced by zeros.

**Attestations and SBOM.** `package.py build` writes `sbom.cdx.json`, a CycloneDX SBOM of
the crates the binding links (normal dependencies, platform-neutral, with each registry
crate's checksum from `Cargo.lock`), which every wheel also carries in
`.dist-info/sboms/` (PEP 770) and the build hosts must agree on. The caller's `attest` job
runs on a tag, before anything is published, and gives every wheel, the XCFramework, the Go
module's `.zip` and `.mod`, each C archive, and the npm tarball a build provenance
attestation and an SBOM attestation (the tarball's from its own `sbom.cdx.json`), which
`gh attestation verify <file> --repo <owner>/<repo>` checks. Only that job holds `id-token` and `attestations`
write; the builder stays read-only. PyPI's trusted publishing uploads its own PEP 740
attestations for the wheels as well.

## Other registries

The same boundary holds for each destination: the builder makes and checks the artifact
Expand Down
Loading