Tracking: Build reproducible shared native libraries for all language bindings #1997

Summary

Build each payjoin-ffi native library once, reproducibly, and ship the same bytes through
every language binding. Get Dart consumers who need reproducible builds (Bull Bitcoin,
who currently depend on a fork) onto upstream right away, without work that a later
redesign throws away.

The plan has three phases:

• Phase 0: a locked, path-remapped Dart source build, shipped in a shared
  payjoin-ffi 0.25.1 and gated by a reproducibility check in CI. Lock and remap Dart native builds for reproducibility #1996.
• Phase 1: one native build contract that every binding follows. All 12 targets, iOS
  included, build on Linux.
• Phase 2: CI builds each target once and checks it with a second build. Python, C#
  and Dart ship the same files. Dart is prebuilt by default, with source builds on request.

Related: #1815, #1996, #1307, #1308, #1885, #1952.

Why

• Dart consumers resolve their dependency graph fresh on every build. The pub.dev
  archive carries no Cargo.lock. Builds drift as crates publish, and broke once already
  (Resolve every binding build against the lockfile #1885). Bull Bitcoin needs reproducible APKs, so they ship from a fork
  (SatoshiPortal/payjoin-rust) that carries Make Dart native builds reproducible #1815.
• Each binding builds its native library differently.
  • Feature sets differ: Python "", C# csharp, Dart dart through a wrapper crate.
    Features change Cargo's -C metadata hash, so they change the bytes even with no
    cfg-gated code.
  • Rust is 1.85.0 everywhere except Dart's 1.85.1.
  • The Linux glibc floor is 2.17 for C# and unpinned for Python.
  • There are two separate zig installs, and clang-cl is unpinned.
  • No binding remaps build paths, and Dart has no attestation.
• Today's builds can't be reproduced or cross-checked, so there's no single set of
  hashes anyone can verify.

What we already verified

All of this was tested locally with Rust 1.85.1 against the #1996 staged lock.

• The staged lock builds on the consumer toolchain. A lock staged by cargo 1.99.1
  builds with cargo 1.85.1 --locked. The compiled graph (154 to 156 crates per target)
  matches Cargo-recent.lock exactly.
• Linux x86_64 reproduces. Two builds from different package roots, output dirs (one
  with a space) and CARGO_HOMEs gave byte-identical .so and .a files with no build
  paths, at the default 16 codegen units.
• iOS cross-builds reproducibly on Linux, with no Apple SDK.
  • Tooling: zig 0.16.0 as linker and C compiler, zig's Darwin headers, and generated
    libSystem and libiconv .tbd stubs.
  • Result: two aarch64-apple-ios builds from different paths were byte-identical.
    Each is a valid iOS Mach-O (min OS 13.0, install name @rpath/…, 170 public imports).
  • Required flags: -install_name @rpath/<lib>, -headerpad_max_install_names
    (Flutter runs install_name_tool and codesign on these dylibs), and -Wl,-S (zig
    otherwise writes object paths and mtimes into stabs).
• The shipped lock carries 84 stale entries that are never compiled. They come from
  payjoin-test-utils = "0.0.1", which resolves to the 2025 crates.io release (payjoin 0.24, ring 0.16, rustls 0.21) once the workspace [patch] no longer applies. cargo audit would flag them.
• payjoin core comes from crates.io in staged builds. payjoin-ffi requires the
  exact core version, so staging fails between a core bump and its crates.io publish.
• Flutter 3.44.9 bundles hooks_runner 1.1.1. It strips all CARGO_*, RUSTUP_*,
  CFLAGS and SOURCE_DATE_EPOCH variables before running build hooks, so consumers can't
  pass build flags through the environment. It does support user_defines, read from the
  root or workspace pubspec.

Phase 0: locked Dart source builds in 0.25.1

Before tagging, we also build a bullbitcoin-mobile branch against the staged
package twice and compare the APKs with their verify_build.sh.

Then we release. A shared 0.25.1 with no API changes, after which Bull can switch to
the hosted package.

Phase 1: one native build contract

Work, in order:

1. iOS on Linux first, because it gates Phase 2: the stub generator, a linker
   wrapper, and double builds of all three iOS targets. A macOS runner verifies but never
   produces artifacts:
   • it checks each dylib's imports against the real Xcode libSystem.tbd, read in place
   • it runs the simulator builds through Flutter's embed and codesign path
   • it runs the Dart tests in the simulator
2. contrib/native/build.sh <triple> and a reusable native.yml. The workflow
   double-builds 12 targets plus a universal2-apple-darwin file derived with llvm-lipo.
3. C# and Python move onto the contract. Python's Linux wheel becomes
   manylinux_2_17.
4. The Dart source build gets C prefix maps, plus 16 KB link args for NDK r27.
5. CI toolchain cleanup.

Phase 2: build once, ship everywhere

• One native build per release run. Python and C# package those files instead of
  building their own.
• Every native file is attested, attached to the GitHub release and listed in
  SHA256SUMS. The release job asserts that the files inside wheels and nupkgs are
  byte-identical to them. Dart gets attested too.
• Dart goes prebuilt by default.
  • The package carries a sha256 per target, and the hook downloads from the GitHub
    release, verifies and caches.
  • hooks: user_defines: payjoin: build: selects prebuilt (the default), source or
    workspace, with an optional mirror URL.
  • A failed download is a hard error that points to build: source, never a silent
    fallback.
  • workspace replaces the marker file as the dev signal, so git-ref consumers stop
    getting dev builds by accident.
• publish-dart runs after github-release, so the download URLs exist before
  anyone resolves the package.
• A VERIFYING.md explains how to rebuild any target from the devshell and compare
  against SHA256SUMS.

Disclosure: this issue was drafted with AI assistance (Claude Code) from an investigation
I directed, including the local reproducibility experiments described above. I reviewed
it and wanted to post it here as a high level tracker while i work on implementing this.