Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 63 additions & 1 deletion .github/workflows/bindings.yml
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,8 @@ jobs:
web-fixture: ${{ steps.mode.outputs.web-fixture }}
rust2: ${{ steps.identity.outputs.workflow_repository }}
rust2-sha: ${{ steps.identity.outputs.workflow_sha }}
previous: ${{ steps.previous.outputs.previous }}
baseline: ${{ steps.previous.outputs.baseline }}
steps:
# Read new job identity fields through JSON until actionlint knows their types.
- id: identity
Expand Down Expand Up @@ -139,6 +141,19 @@ jobs:
- id: facts
working-directory: source
run: python3 -I ../rust2/ffi/release/package.py facts --manifest-path "$MANIFEST" >> "$GITHUB_OUTPUT"
# The APIs of the release the changelog follows and of the newest one Cargo treats as
# compatible with this version, which the export must not break.
- id: previous
working-directory: source
env:
GH_TOKEN: ${{ github.token }}
run: python3 -I ../rust2/ffi/release/package.py previous --manifest-path "$MANIFEST" --repo "$GITHUB_REPOSITORY" --out ../previous >> "$GITHUB_OUTPUT"
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
if: steps.previous.outputs.previous != ''
with:
name: ${{ steps.facts.outputs.name }}-previous
path: previous
if-no-files-found: error

# Each target's API, checked on the host whose C toolchain its build scripts need, for
# every build host's export to merge into one binding.
Expand Down Expand Up @@ -285,13 +300,22 @@ jobs:
pattern: ${{ needs.facts.outputs.name }}-api-*
path: api
merge-multiple: true
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
if: needs.facts.outputs.baseline != ''
with:
name: ${{ needs.facts.outputs.name }}-previous
path: previous
# A version must not break the API of the newest release Cargo treats as compatible.
- name: Export
env:
TARGETS: ${{ matrix.targets }}
BASELINE: ${{ needs.facts.outputs.baseline }}
GO_MODULE: ${{ inputs.go-module }}
run: |
read -ra targets <<< "$TARGETS"
python3 -I rust2/ffi/release/package.py export --manifest-path "source/$MANIFEST" --features "$FEATURES" --api api --go-module "$GO_MODULE" --out export "${targets[@]}"
baseline=()
if [[ -n "$BASELINE" ]]; then baseline=(--baseline previous/baseline/api.json); fi
python3 -I rust2/ffi/release/package.py export --manifest-path "source/$MANIFEST" --features "$FEATURES" --api api "${baseline[@]}" --go-module "$GO_MODULE" --out export "${targets[@]}"
- name: Build
env:
TARGETS: ${{ matrix.targets }}
Expand Down Expand Up @@ -334,6 +358,44 @@ jobs:
exit 1
fi

# The release's API, which the next release is checked against, and its changelog from
# the previous release's: what the caller's release publishes beside the packages.
changes:
needs: [facts, agree]
runs-on: ubuntu-24.04
timeout-minutes: 10
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: ${{ needs.facts.outputs.rust2 }}
ref: ${{ needs.facts.outputs.rust2-sha }}
path: rust2
sparse-checkout: ffi/release
persist-credentials: false
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
name: ${{ needs.facts.outputs.name }}-build-linux-x64
path: build
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
if: needs.facts.outputs.previous != ''
with:
name: ${{ needs.facts.outputs.name }}-previous
path: previous
- env:
PREVIOUS: ${{ needs.facts.outputs.previous }}
run: |
mkdir release
cp build/export/api.json release/api.json
old=()
if [[ -n "$PREVIOUS" ]]; then old=(--old previous/previous/api.json --since "$PREVIOUS"); fi
python3 -I rust2/ffi/release/package.py changes "${old[@]}" --new release/api.json > release/changes.md
cat release/changes.md >> "$GITHUB_STEP_SUMMARY"
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: ${{ needs.facts.outputs.name }}-release
path: release
if-no-files-found: error

swift:
needs: [facts, build]
runs-on: macos-26
Expand Down
11 changes: 11 additions & 0 deletions ffi/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -275,6 +275,17 @@ break, export without `--baseline` and commit the `api.json` it writes. Its span
to the workspace and cargo's home, as a staged API's are, so a removed item's error points
where it was while its file is unchanged.

The lock compares the API and the names each generator gives it, not the signatures each
language renders, so a generator change alone, such as a Kotlin type for a Rust one, passes
it. rust2 locks those too: [`surface/`](surface) holds each binding's API as its language
renders it, for the api fixture and moq's stand-in, and `just ci` fails where a generator
renders another. `just ffi surface` rewrites them; a PR changing any binding's API commits
the diff. Each is the language's own reading where one needs no build: Python's `ast` for
the `.pyi` a type checker sees, `go doc -all` for Go, `tsc`'s declarations for the web
binding. C, C++, Kotlin, Swift, and Dart are an outline of the generated source: each
public declaration up to its body, without comments or private members. The files are x86_64
Linux's exports, as the merge queue runs on, since a header names its targets.

## The C ABI

- **Handles** are `Arc`s, and a call borrows the value as Rust's receiver or parameter
Expand Down
21 changes: 21 additions & 0 deletions ffi/justfile
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,27 @@ fix:
run *args:
cargo run --locked --quiet -- {{ args }}

# Write surface/, each binding's API as its language renders it, from the api fixture's export and moq's stand-in's at the pinned moq revision: --check fails where a committed file differs instead. The files are x86_64 Linux's exports, whose targets they name.
surface *flags: (moq-write "") (standin "" "web")
#!/usr/bin/env bash
set -euo pipefail
if [ "{{ arch() }}-{{ os() }}" != x86_64-linux ]; then
echo "the surface files are x86_64 Linux's exports: run this there" >&2
exit 1
fi
# moq-write's export, of the host alone.
if [ "$(jq -c .api.targets target/export/moq/api.json)" != '["x86_64-unknown-linux-gnu"]' ]; then
echo "target/export/moq merged the APIs staged in target/moq-api: delete them for the host's export" >&2
exit 1
fi
# The api fixture as its tests export it, with wasm32 for the web binding.
cargo run --locked --quiet -- --config fixtures/api/ffi.toml --out target/surface/api --target x86_64-unknown-linux-gnu --target wasm32-wasip1 -- --features doubled
cargo run --locked --quiet -- --config moq/web.toml --manifest-path target/moq-ffi-web/Cargo.toml --target wasm32-wasip1 --out target/surface/moq-web
python3 -I surface/surface.py {{ flags }} target/surface/api surface/api
python3 -I surface/surface.py {{ flags }} target/export/moq surface/moq
# Its other bindings are moq's on wasm32, which no one ships.
python3 -I surface/surface.py {{ flags }} target/surface/moq-web surface/moq-web web.d.ts

# Census the stand-in facade moq/ffi.toml exports into target/census/moq-ffi.md, from the pinned moq revision or a checkout, failing when moq/ffi.toml excludes an item that exports.
census checkout="": (standin checkout)
cargo run --locked --quiet -- --config moq/ffi.toml --manifest-path target/moq-ffi/Cargo.toml --census target/census
Expand Down
24 changes: 17 additions & 7 deletions ffi/release/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,21 +67,28 @@ crate apart from it takes `web-features` and `web-example` instead.

## What the builder does

1. Stages each target's API on the host whose C toolchain its build scripts need.
2. On each build host, exports the binding of every target, merging the others' staged
1. Finds the earlier releases among the `<crate>-v<version>` GitHub releases, and their
`api.json` assets: the newest below the crate's version, which the changelog follows,
and the newest below it Cargo treats as compatible (the same leftmost nonzero part,
neither a prerelease), which each export passes as `--baseline`, so a release that
breaks its API without a breaking version fails
([The API lock](/ffi/README.md#the-api-lock)). A release without the asset fails the
build.
2. Stages each target's API on the host whose C toolchain its build scripts need.
3. On each build host, exports the binding of every target, merging the others' staged
APIs, so an item only some targets have fails there at run time instead of being
missing ([Targets](/ffi/README.md#targets)). A job checks that every host generated the
same binding.
3. Builds each target with the crate's own toolchain, as rustup resolves it from the
4. Builds each target with the crate's own toolchain, as rustup resolves it from the
crate's directory; only the export runs on rust2's nightly.
4. Python: a `cp311-abi3` wheel per target, the oldest CPython the generated extension's
5. Python: a `cp311-abi3` wheel per target, the oldest CPython the generated extension's
stable ABI loads in, holding the module as `<lib>/__init__.py` beside its library, with
PEP 639 license metadata and a complete hashed RECORD. Each is repaired by its platform's
standard tool, which claims the portable tag or fails: auditwheel for
`manylinux_2_28` on x86_64 and aarch64 (built by cargo-zigbuild against glibc 2.28, as
[crabx](/crabx/README.md) builds), delocate for `macosx_11_0_arm64`, and delvewheel for
`win_amd64`.
5. Swift: static libraries for macOS arm64 at macOS 13, and iOS arm64 and its simulator
6. Swift: static libraries for macOS arm64 at macOS 13, and iOS arm64 and its simulator
at iOS 16, the floors of the generated Swift's `Swift.Duration`, each checked against
every object's declared minimum OS; one XCFramework of the three, zipped with `ditto`,
its SHA-256 checked against `swift package compute-checksum`; and a `Package.swift`
Expand All @@ -90,7 +97,7 @@ crate apart from it takes `web-features` and `web-example` instead.
revision, since a package resolved by version cannot depend on a revision. Rust's
native link libraries become linker settings, per platform where they differ; any
other flag fails the build.
6. Go: one module for every target, at the module path and `v<version>`, the major
7. Go: one module for every target, at the module path and `v<version>`, the major
version's `/vN` suffix checked: the export's `go/`, whose runtime's `#cgo` lines link
`rt/lib<prefix>_<goos>_<goarch>.a` for `linux/amd64` and `linux/arm64` (built against
glibc 2.28), `darwin/arm64` and `ios/arm64` (the Swift package's macOS and iOS device
Expand All @@ -105,7 +112,10 @@ crate apart from it takes `web-features` and `web-example` instead.
limit, each platform's library moves to a module of its own, whose one package's `#cgo`
line links it and which the runtime imports under that platform's build constraint, so
`go get` downloads only the platform it builds for.
7. Consumers, with no checkout, Rust, or source path: each wheel installed with
8. The release's `api.json`, and `changes.md`, the items removed, changed, and added since
the previous release's, by kind and Rust path; the caller's release publishes the first
as an asset and the second as its notes.
9. Consumers, with no checkout, Rust, or source path: each wheel installed with
`--no-index --no-deps` into a fresh environment on its oldest CPython, run with `-I`
from an empty directory; and the Swift package built by its own scheme for macOS, iOS,
and the iOS Simulator, then consumed by a new executable package that runs the check;
Expand Down
Loading