Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions .github/ffi.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Shared FFI releases

`release-shared-ffi.yml` owns the existing `moq-ffi-v*` tag trigger for Go,
Kotlin and Swift. Nightly calls the same coordinator without publisher secrets.
It builds nine distinct default-feature targets, replacing fourteen native
builds across those packages. Dart retains its eleven minimal-feature targets;
Python retains its wheel builds and Python-specific environment.

C++ retains its host CMake builds and independent `cpp-v*` release tags. All
four sampled C++ static libraries have different hashes from the candidates
shared here. In particular, ARM Linux builds natively on Ubuntu 24.04 rather
than on the shared cross-compilation host. Sharing those packages needs
separate artifact/ABI proof and a settled prebuilt-library CMake interface.

The shared build uploads `default-lib-<target>`. Binding jobs read the built
Linux library with `just rs package-ffi --bindings-only --library PATH`, so
binding generation does not compile moq-ffi again. Package workflows select
only their own targets, retain language binding names and package outputs,
and keep their existing publisher steps and credentials. Manual calls remain
standalone. Go's PR dry-run also remains standalone.

A target failure must not suppress a package that does not consume that target.
The build matrix and binding matrix do not fail fast; packaging still runs
after a failed shared job and refuses missing artifacts. The wrapper workflows
accept only the existing trusted owner/push/tag inputs. Their read-only
preflight checks the completed coordinator's own language publisher and jobs,
so a failed Android build does not prevent a successful Go wrapper release.
The [jobs API](https://docs.github.com/en/rest/actions/workflow-jobs#list-jobs-for-a-workflow-run)
query includes all attempts and uses each job's latest execution. A publisher
counts only if it ran in the triggering attempt: a partial rerun copies earlier
successes with their original start time, and their wrappers already ran when
that attempt completed. Rerunning only Kotlin on an old tag therefore cannot
re-cut the Go wrapper from old sources after a newer tag published, while a
rerun publisher still releases its wrapper. Missing publisher names or an API
error fail loudly. No publication is polled.
The legacy per-language workflow names remain accepted for in-flight runs.

## Evidence and validation

[The sampled Nightly](https://github.com/moq-dev/moq/actions/runs/38564590314)
contains fourteen default-feature native builds. Downloaded library payloads
from all nine uploads for duplicated targets confirm:

| Target | Consumers | Comparison |
| --- | --- | --- |
| x86\_64-unknown-linux-gnu | Go, Kotlin | static and shared libraries have identical SHA-256 hashes |
| aarch64-unknown-linux-gnu | Go, Kotlin | static and shared libraries have identical SHA-256 hashes |
| aarch64-apple-darwin | Go, Kotlin, Swift | static and shared libraries have identical SHA-256 hashes |
| x86\_64-pc-windows-msvc | Go, Kotlin | import library identical; DLL and static library differ only in timestamps and DLL debug GUID |

The Windows DLL has 23 differing bytes, all in PE/COFF timestamps, debug-directory
timestamps and the CodeView GUID. The static archive has 85 differing bytes,
all in COFF timestamps. Zeroing those metadata fields makes both payloads
byte-identical; no executable code or binding metadata differs. The source
build command is the same target/default-feature Cargo invocation for every
consumer. Sharing preserves the same native code and library files.

`just gh test` runs local fixtures for artifact subsets,
build-free binding generation, independent language outcomes, failed API
queries and workflow guards. `just gh check` validates workflow syntax and
alert coverage. The first Nightly after landing measures actual elapsed time;
the sampled five duplicate native builds cost about seventy runner-minutes.
7 changes: 7 additions & 0 deletions .github/justfile
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ check:
test:
python3 sh/gh/install-formula.test.py
sh/gh/release.test.sh
python3 sh/gh/ffi.test.py
bun test sh/gh/ffi.test.ts
python3 sh/rs/package-rename.test.py
sh/go/publish-wrapper.test.sh
bash .claude/hooks/direnv.test.sh
Expand Down Expand Up @@ -38,3 +40,8 @@ install-formula *args:
[positional-arguments]
trigger-repo-publish *args:
sh/gh/trigger-repo-publish.sh "$@"

# Select shared native artifacts or check an upstream FFI publisher.
[positional-arguments]
ffi *args:
python3 sh/gh/ffi.py "$@"
1 change: 1 addition & 0 deletions .github/workflows/alert.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ on:
- release-brew
- Release Dart
- Release Dart FFI
- Release FFI
- Release Go
- Release Go FFI
- Release JS
Expand Down
18 changes: 3 additions & 15 deletions .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -280,24 +280,12 @@ jobs:
contents: read
id-token: write

release-kt-ffi:
name: Release Kotlin FFI
uses: ./.github/workflows/release-kt-ffi.yml
permissions:
contents: read

release-swift-ffi:
name: Release Swift FFI
uses: ./.github/workflows/release-swift-ffi.yml
release-shared-ffi:
name: Release FFI
uses: ./.github/workflows/release-shared-ffi.yml
permissions:
contents: write

release-go-ffi:
name: Release Go FFI
uses: ./.github/workflows/release-go-ffi.yml
permissions:
contents: read

release-dart-ffi:
name: Release Dart FFI
uses: ./.github/workflows/release-dart-ffi.yml
Expand Down
64 changes: 45 additions & 19 deletions .github/workflows/release-ffi.yml
Original file line number Diff line number Diff line change
@@ -1,14 +1,8 @@
name: release-ffi

# Builds rs/moq-ffi for each requested target and generates its UniFFI
# bindings. The release-{dart,go,kt,swift}-ffi.yml callers package and publish;
# they keep the moq-ffi-v* tag trigger and the workflow name that alert.yml and
# the workflow_run chains match on.
#
# Uploads, all prefixed by `language` so callers sharing one nightly run never
# collide:
# <language>-lib-<target> the target's native libraries (the lib/ directory)
# <language>-bindings dist/bindings/<language>/, when `bindings` is set
# Builds each requested target and generates bindings. The shared coordinator
# uploads default-lib-<target>; standalone callers retain their language prefix.
# Binding artifacts retain their language names in either mode.

on:
workflow_call:
Expand All @@ -29,7 +23,18 @@ on:
description: "Build moq-ffi with its default features (audio and video)"
type: boolean
default: true
reuse-native:
description: "Consume the coordinator's native libraries and bindings"
type: boolean
default: false
binding-languages:
description: "JSON array of binding languages; empty means language"
type: string
default: ""
outputs:
targets:
description: "Requested targets, for selecting and staging native artifacts"
value: ${{ jobs.parse-version.outputs.targets }}
version:
description: "moq-ffi version, from the tag or (off a tag) rs/moq-ffi/Cargo.toml"
value: ${{ jobs.parse-version.outputs.version }}
Expand All @@ -47,6 +52,7 @@ jobs:
name: Parse version
runs-on: ubuntu-latest
outputs:
targets: ${{ inputs.targets }}
version: ${{ steps.parse.outputs.version }}

steps:
Expand All @@ -63,6 +69,7 @@ jobs:
build:
name: Build moq-ffi (${{ matrix.target }})
needs: [parse-version]
if: ${{ !inputs.reuse-native }}
runs-on: ${{ contains(matrix.target, '-apple-') && 'macos-latest' || contains(matrix.target, '-windows-') && 'windows-latest' || 'ubuntu-latest' }}

strategy:
Expand Down Expand Up @@ -120,10 +127,16 @@ jobs:
path: dist/moq-ffi-${{ needs.parse-version.outputs.version }}-${{ matrix.target }}/lib/

bindings:
name: Generate bindings
needs: [parse-version]
if: inputs.bindings
name: Generate bindings (${{ matrix.language }})
needs: [parse-version, build]
# An unrelated target failure must not suppress bindings for successful
# consumers. A missing Linux artifact fails its download below.
if: ${{ always() && !cancelled() && inputs.bindings && !inputs.reuse-native }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
language: ${{ fromJSON(inputs.binding-languages || format('["{0}"]', inputs.language)) }}

steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
Expand All @@ -140,7 +153,7 @@ jobs:

# Go uses a third-party generator; build.sh skips Go when it is absent.
- name: Install uniffi-bindgen-go
if: inputs.language == 'go'
if: matrix.language == 'go'
env:
# A fork until NordSecurity ships a uniffi 0.32 generator; see flake.nix.
# The revision is v0.10.0-kixelated.1+v0.32.0.
Expand All @@ -151,19 +164,32 @@ jobs:
--git "$UNIFFI_BINDGEN_GO_REPO" \
--rev "$UNIFFI_BINDGEN_GO_REV"

- name: Download binding metadata
if: contains(fromJSON(inputs.targets), 'x86_64-unknown-linux-gnu')
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
name: ${{ inputs.language }}-lib-x86_64-unknown-linux-gnu
path: binding-library

- name: Generate bindings
env:
BUILD_VERSION: ${{ needs.parse-version.outputs.version }}
REUSE_LIBRARY: ${{ contains(fromJSON(inputs.targets), 'x86_64-unknown-linux-gnu') }}
DEFAULT_FEATURES: ${{ inputs.default-features }}
run: |
just rs package-ffi \
--bindings-only \
--version "$BUILD_VERSION" \
--output dist
args=(--bindings-only --version "$BUILD_VERSION" --output dist)
if [[ "$REUSE_LIBRARY" == true ]]; then
args+=(--library binding-library/libmoq_ffi.so)
fi
if [[ "$DEFAULT_FEATURES" == false ]]; then
args+=(--no-default-features)
fi
just rs package-ffi "${args[@]}"

- name: Upload bindings
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: ${{ inputs.language }}-bindings
path: dist/bindings/${{ inputs.language }}/
name: ${{ matrix.language }}-bindings
path: dist/bindings/${{ matrix.language }}/
# A language build.sh does not generate would otherwise upload nothing.
if-no-files-found: error
34 changes: 21 additions & 13 deletions .github/workflows/release-go-ffi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,20 @@ name: Release Go FFI
# (release-go.yml), which also auto-bumps its require to each new ffi here.

on:
push:
tags:
- "moq-ffi-v*"
# Dry-run: build everything, publish nothing. Nightly calls every release
# workflow this way so a broken build surfaces before the tag is cut.
# The tag coordinator supplies shared default-feature artifacts. Manual
# calls and Go PR dry-runs still build their own target subset.
workflow_call:
inputs:
reuse-native:
type: boolean
default: false
workflow_dispatch:
pull_request:
paths:
- ".github/workflows/release-go-ffi.yml"
- ".github/workflows/release-ffi.yml"
- ".github/workflows/release-shared-ffi.yml"
- "sh/gh/ffi*"
- "sh/gh/release.sh"
- "go/ffi/**"
- "sh/go/package-ffi.sh"
Expand All @@ -40,6 +43,7 @@ jobs:
uses: ./.github/workflows/release-ffi.yml
with:
language: go
reuse-native: ${{ inputs.reuse-native || false }}
targets: >-
[
"x86_64-unknown-linux-gnu",
Expand All @@ -60,20 +64,24 @@ jobs:

- uses: ./.github/actions/just

- name: Select native artifacts
id: native
env:
LIB_PREFIX: ${{ inputs.reuse-native && 'default' || 'go' }}
BUILD_TARGETS: ${{ needs.ffi.outputs.targets }}
run: just gh ffi pattern

- name: Download per-target libs
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
path: libs-raw
pattern: go-lib-*
pattern: ${{ steps.native.outputs.pattern }}

- name: Flatten lib layout
run: |
mkdir -p libs
for dir in libs-raw/go-lib-*; do
target="${dir#libs-raw/go-lib-}"
mv "$dir" "libs/$target"
done
ls -la libs
env:
LIB_PREFIX: ${{ inputs.reuse-native && 'default' || 'go' }}
BUILD_TARGETS: ${{ needs.ffi.outputs.targets }}
run: just gh ffi stage

- name: Download bindings
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
Expand Down
Loading
Loading