diff --git a/.github/workflows/release-ffi.yml b/.github/workflows/release-ffi.yml index 6f994e3c940..0d95494bf89 100644 --- a/.github/workflows/release-ffi.yml +++ b/.github/workflows/release-ffi.yml @@ -66,7 +66,8 @@ jobs: name: Build moq-ffi (${{ matrix.target }}) needs: [parse-version] if: ${{ !inputs.reuse-native }} - runs-on: ${{ contains(matrix.target, '-apple-') && 'macos-latest' || contains(matrix.target, '-windows-') && 'windows-latest' || 'ubuntu-latest' }} + # Linux ARM64 builds natively, so no cross toolchain is fetched at build time. + runs-on: ${{ contains(matrix.target, '-apple-') && 'macos-latest' || contains(matrix.target, '-windows-') && 'windows-latest' || matrix.target == 'aarch64-unknown-linux-gnu' && 'ubuntu-24.04-arm' || 'ubuntu-latest' }} strategy: fail-fast: false @@ -88,14 +89,6 @@ jobs: # No Rust cache: shipped binaries compile from source rather than from # whatever the shared R2 cache holds. - - name: Install cross-compilation tools (Linux ARM64) - if: matrix.target == 'aarch64-unknown-linux-gnu' - run: | - sudo apt-get update - # g++ as well as gcc: moq-video's openh264 fallback is C++, and - # cc-rs looks for aarch64-linux-gnu-g++ by name. - sudo apt-get install -y gcc-aarch64-linux-gnu g++-aarch64-linux-gnu - - name: Install cargo-ndk (Android) if: contains(matrix.target, 'android') run: cargo install --locked "cargo-ndk@$CARGO_NDK_VERSION" diff --git a/quest/m0/nightly/gst-image.md b/quest/m0/nightly/gst-image.md new file mode 100644 index 00000000000..cd360affc0a --- /dev/null +++ b/quest/m0/nightly/gst-image.md @@ -0,0 +1,52 @@ +# [M] Publish a GStreamer CI image to ghcr.io every week + +## Goal + +A weekly job on `main` publishes a multi-arch (amd64, arm64) image to ghcr.io +with Ubuntu 24.04's GStreamer packages preinstalled, and proposes bumping the +digest pin whenever the image changes. moq-gst's Linux jobs can then pull it +instead of fetching from Ubuntu's apt mirrors, as +[moq-gst's apt outage](/quest/m0/nightly/release-builds.md) plans. + +## Plan + +Decided 2026-11-13 (maintainer, /quest-complete): ✅ a weekly-published +ghcr.io image with the packages preinstalled (rejected: accepting a mirror +outage as a real failure). This quest publishes the image; the consumer +switch is its own quest, since a pin needs a digest that only a `main` run +can publish. + +- The Dockerfile lives in `.github/docker/gstreamer/`. Pin its `ubuntu:24.04` + base by digest, as the root Dockerfile does, and add the directory to + Dependabot's `docker` ecosystem. Install what both consumers need: + `gstreamer1.0-tools`, `gstreamer1.0-plugins-base` and + `gstreamer1.0-plugins-good` for the smoke test, plus + `libgstreamer1.0-dev`, `libgstreamer-plugins-base1.0-dev`, `pkgconf` and + `gcc`/`libc6-dev` for the package build. Label it + `org.opencontainers.image.source=https://github.com/moq-dev/moq` so the + package links to the repository and this repo's Actions can pull it with + `GITHUB_TOKEN`. +- Follow `docker.yml`'s shape: build linux/amd64 on `ubuntu-latest` and + linux/arm64 on `ubuntu-24.04-arm` natively, push by digest, and assemble + the manifest list in a merge job. Name it `ghcr.io/moq-dev/gstreamer-ci`. + Triggers: a weekly `schedule`, `workflow_dispatch`, and `push` to `main` + touching the Dockerfile. Only jobs in the `publish` environment (which + deploys from `main`, `release` and release tags only) get + `packages: write`, so a quest branch's run can't push. +- Dependabot can't bump a `container:` or `docker run` reference in a + workflow + ([dependabot/dependabot-core#5819](https://github.com/dependabot/dependabot-core/issues/5819)), + so the publisher proposes the bump itself through moq-bot, the way + `smoke-toolchain.yml` does. Keep the pin in a plain file, + `.github/docker/gstreamer/image` (one `ghcr.io/...@sha256:...` line), that + the consumers read, not in a workflow file: an app token that edits + workflows needs `workflows: write`. +- A scheduled run skips the push and the PR when the built image's + `dpkg-query -W` package list matches the pinned image's, so a week with no + Ubuntu updates opens no PR. A push or dispatch always publishes, so a + Dockerfile change that installs the same packages still ships. With no pin + file yet, the first run publishes and proposes the file. +- Validate the image locally with podman before pushing: the smoke test + loads a moq-gst tarball, and `cargo zigbuild` of `moq-gst` links in it. + After merge, confirm the push-triggered run published both architectures + and opened the pin PR. diff --git a/quest/m0/nightly/release-builds.md b/quest/m0/nightly/release-builds.md index 522537d2893..cfdfb10d44e 100644 --- a/quest/m0/nightly/release-builds.md +++ b/quest/m0/nightly/release-builds.md @@ -1,30 +1,45 @@ -# [S] Nightly's release builds pass +# [S] Nightly's moq-gst builds survive an apt mirror outage ## Goal -The release builds Nightly runs pass every night, so the jobs that need them, -such as `Shipping sizes`, run instead of being skipped. +`moq-gst`'s Nightly builds stop failing when Ubuntu's apt mirrors can't be +reached, so `Shipping sizes` still runs on those nights. ## Plan -Found by the triage of Nightly run -[43447849289](https://github.com/moq-dev/moq/actions/runs/43447849289) -(2026-11-13): `Shipping sizes` was skipped 11-08 to 11-12 because release -builds it depends on failed, so six days of size growth surfaced at once. -Failures seen, each with its own cause: - -- 11-12, `moq-cli / release / Build (armv7-unknown-linux-gnueabihf)`: - `error[E0463]: can't find crate for core`, so the toolchain lacks that - target's standard library. -- 11-11, `Release FFI / Shared moq-ffi / Build moq-ffi (aarch64-unknown-linux-gnu)`: - apt failed with `Temporary failure resolving 'security.ubuntu.com'`, and - `Release C`/`Release Cpp` aarch64 then failed on a missing - `cpp-native-aarch64-unknown-linux-gnu` artifact. -- 11-11/11-12: the macOS `moq-cli` tarball, the Go FFI bindings and package, - the Kotlin FFI package, and the aarch64 `moq-gst` builds, not yet triaged. - -Fix each at its root. A network fetch at build time that can fail on a DNS -blip is a defect to remove, for example by taking the cross toolchain from -Nix, not to retry. - -Public API: none. Wire: none. +On 11-11 an apt DNS failure (`Temporary failure resolving +'security.ubuntu.com'`) failed both `moq-gst / Build tarball` jobs and both +`Package .deb/.rpm` jobs at their apt step, and so skipped `Shipping sizes`. +That apt use is deliberate. The tarball job smoke-tests the plugin against +the distro's own GStreamer, and the package job links against the distro's +GStreamer headers. Nix can't stand in for either. + +Decided 2026-11-13 (maintainer, /quest-complete): ✅ the Linux jobs take the +distro GStreamer from a weekly-published ghcr.io image with the packages +preinstalled (rejected: accepting a mirror outage as a real failure). +[The GStreamer CI image](/quest/m0/nightly/gst-image.md) publishes it and the +digest pin, `.github/docker/gstreamer/image`. + +- Tarball job: on Linux, run `just rs gst-smoke` over the extracted tarball + inside the pinned image with `docker run`, instead of apt-installing + GStreamer on the runner. macOS keeps `brew install gstreamer`. +- Package job: run only the `cargo zigbuild` step inside the image, with the + checkout, `$HOME` (rustup's toolchain), and `/nix` (the packaging tools) + mounted, so pkg-config finds the image's GStreamer. Don't make it a + `container:` job: Nix's installer and the composite actions would then run + inside the image too. +- Read the pin from the file, log in to ghcr.io with `GITHUB_TOKEN`, and + grant `packages: read` in `moq-gst.yml` and in Nightly's `release-moq-gst` + caller. +- Drop the apt steps. Validate with a `workflow_dispatch` of `moq-gst.yml` + on the quest branch, which publishes nothing off a tag. + +The other release-build failures from 11-11 to 11-12 are fixed: armv7 by +[#8509](https://github.com/moq-dev/moq/pull/8509), the Go bindings by +[#8585](https://github.com/moq-dev/moq/pull/8585), and the FFI aarch64 +cross toolchain by +[moq-dev/moq#8668](https://github.com/moq-dev/moq/pull/8668). + +## Required + +- [GStreamer CI image](/quest/m0/nightly/gst-image.md) - publishes the image and the pin file this reads