ci: assemble an OpenVMM source archive and gate distribution builds

[Ben Hillis (benhillis)]

Purpose

Add the source-archive and distribution-build portion of #4150, now that the build identity contract from #4162 has landed.

Linux distributions build OpenVMM from a source tarball rather than from a git checkout, using distribution-provided native dependencies. Nothing in the repo produced such a tarball, and nothing proved OpenVMM could be built from one. This adds both, and gates the result in CI so the distribution build path cannot silently regress.

Assembling the archive

• assemble openvmm-<VERSION>.tar.gz from the tracked tree at HEAD, unpacking into openvmm-<VERSION>/
• make repeated assembly deterministic with a fixed archive mode mask and gzip -n
• produce SHA256SUMS
• reject checkouts with tracked modifications, and identity/version mismatches
• transfer archive identity through typed Flowey artifact metadata

The archive name and its root directory both match %{name}-%{version}, so RPM's %autosetup and Fedora's forge macros need no override.

Gating the distribution build

• extract outside the checkout and build OpenVMM using distribution-provided native dependencies
• run the distribution build in normal PR and CI gates

The build uses system protoc, OpenSSL, compiler/linker, Linux headers, and pkg-config; it deliberately does not restore .packages/ or openvmm-deps.

Documentation

Add a Linux packaging guide covering archive identity, checksum verification, build requirements and their distribution package names, offline vendoring, RPM/Debian integration requirements, package versioning, and expected runtime dependencies.

Out of scope

Publication, GitHub releases, attestations, release preparation, and maintainer release procedures remain separate follow-up work. Nothing currently uploads the assembled archive as a release asset; the node is shared so that the future release pipeline ships the exact bytes CI already builds.

OpenVMM does not yet publish an OpenPGP signature alongside the archive, so uscan signature verification is unavailable to Debian packagers. That needs a signing-key decision and is left to the publication work.

Validation

• cargo xtask fmt --fix, including flowey pipeline regeneration
• targeted flowey_lib_hvlite tests
• targeted clippy and cargo doc
• Guide test and build
• assembled the archive from a scratch repo and confirmed byte-identical output across runs

[Copilot]

Pull request overview

This PR adds a new CI/checkin gate that assembles a deterministic OpenVMM source archive from the tracked HEAD tree, carries a typed “source identity” across Flowey artifacts, and validates that OpenVMM can be built from the extracted archive using only distribution-provided native dependencies (no .packages/ / openvmm-deps). It also documents the Linux packaging expectations and wires the new gate into both Flowey pipelines and generated CI YAML.

Changes:

• Add a Flowey node to assemble a deterministic openvmm-<VERSION>-source.tar.gz plus SHA256SUMS, and persist identity metadata in the artifact directory.
• Add a new “distribution config” build job that extracts the assembled archive outside the checkout and builds openvmm with system dependencies (Ubuntu apt packages + Rust toolchain).
• Add a Linux packaging guide page and link it into the Guide navigation.

Reviewed changes

Copilot reviewed 14 out of 15 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
Guide/src/toc.yml	Adds (generated) Guide TOC entry including the new packaging page.
Guide/src/SUMMARY.md	Adds the new “Packaging OpenVMM for Linux” page to mdBook navigation.
Guide/src/dev_guide/contrib/openvmm_packaging.md	New documentation describing the source archive contract and distro build requirements/config.
flowey/flowey_lib_hvlite/src/lib.rs	Exposes the new source-archive assembly module.
flowey/flowey_lib_hvlite/src/assemble_openvmm_source_release.rs	Implements deterministic source archive assembly + identity transfer helpers and tests.
flowey/flowey_lib_hvlite/src/_jobs/mod.rs	Registers new distro-build job nodes.
flowey/flowey_lib_hvlite/src/_jobs/check_distro_build.rs	Implements the distro-style build from extracted source archive using dist packages.
flowey/flowey_lib_hvlite/src/_jobs/check_distro_build_from_checkout.rs	Wires together identity resolution, source assembly, and the distro build job.
flowey/flowey_lib_hvlite/Cargo.toml	Adds toml_edit dependency for reading [workspace.package] version.
flowey/flowey_hvlite/src/pipelines/checkin_gates.rs	Adds the distro-build job to checkin gates.
ci-flowey/openvmm-pr.yaml	Updates generated ADO pipeline to include the new distro-build job.
Cargo.lock	Updates lockfile for the new dependency.
.github/workflows/openvmm-pr.yaml	Updates generated GitHub PR workflow to include the new distro-build job.
.github/workflows/openvmm-pr-release.yaml	Updates generated PR-release workflow to include the new distro-build job (when label-gated).
.github/workflows/openvmm-ci.yaml	Updates generated CI workflow wiring (job reshuffle + new distro-build job).

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

2 Petri tests failed (0 unstable)

[Copilot]

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated no new comments.

Suppressed comments (5)

flowey/flowey_lib_hvlite/src/assemble_openvmm_source_release.rs:79

• The PR description says the assembled archive is named openvmm-<VERSION>-source.tar.gz, but archive_name() currently produces openvmm-<VERSION>.tar.gz. Please align the PR description and implementation (and any downstream consumers) so packagers/gates aren’t relying on the wrong filename.

    pub fn archive_name(&self) -> String {
        format!("{}.tar.gz", self.source_root())
    }

flowey/flowey_lib_hvlite/src/assemble_openvmm_source_release.rs:188

• This dirty-check uses git status --porcelain --untracked-files=no, so untracked files are ignored. The error message currently says “dirty working tree”, which can be misleading; consider clarifying that this is about tracked modifications not present in git archive.

                    anyhow::bail!(
                        "refusing to assemble a source archive from a dirty working tree; \
                         the archive would not match HEAD.\nmodified:\n{dirty}"
                    );

flowey/flowey_lib_hvlite/src/assemble_openvmm_source_release.rs:207

• gzip output is currently captured into memory via .output() and then written to disk. For a large source tree this can unnecessarily spike memory usage (and potentially OOM CI runners). Stream gzip stdout directly to the destination file instead.

                let source_archive = output_dir.join(identity.archive_name());
                let compressed =
                    flowey::shell_cmd!(rt, "gzip -n --best --stdout {source_tar}").output()?;
                fs_err::write(&source_archive, compressed.stdout)?;
                fs_err::remove_file(source_tar)?;

flowey/flowey_lib_hvlite/src/_jobs/check_distro_build.rs:60

• On non-Ubuntu platforms this job silently skips installing the required distro packages, which can lead to confusing failures later (e.g., missing protoc, OpenSSL headers, etc.). Consider failing fast when not running on Ubuntu (or explicitly supporting other distros).

        if matches!(
            ctx.platform(),
            FlowPlatform::Linux(FlowPlatformLinuxDistro::Ubuntu)
        ) {

Guide/src/dev_guide/contrib/openvmm_packaging.md:11

• Several lines in this new guide page exceed the 80-character wrap guideline for Guide/ content (e.g., the introductory paragraphs in the Source archive section). Please reflow the prose to ~80 columns for consistency/readability and to match the documented Guide style rules.

The Flowey source-archive node exports the tracked repository tree at `HEAD` under an `openvmm-<VERSION>/` prefix. `<VERSION>` is the canonical `[workspace.package] version` in the root `Cargo.toml`.

The archive is named `openvmm-<VERSION>.tar.gz` and unpacks into `openvmm-<VERSION>/`. The filename and the root directory deliberately match `%{name}-%{version}`, so RPM's `%autosetup` and Fedora's forge macros work without a `-n` override or a renamed source.

Archive assembly uses `git archive` with a fixed mode mask and `gzip -n`, so repeated assembly at the same commit produces the same `openvmm-<VERSION>.tar.gz` bytes. The assembly also generates `SHA256SUMS`.

[Copilot]

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated no new comments.

Suppressed comments (3)

flowey/flowey_lib_hvlite/src/assemble_openvmm_source_release.rs:212

• gzip will fail if the output .tar.gz already exists (it prompts/refuses to overwrite unless -f), which makes “repeated assembly” in the same output directory unreliable. Force overwrite (or delete the existing archive) before compressing so re-running the node is deterministic and idempotent.

                flowey::shell_cmd!(rt, "gzip -n --best {source_tar}").run()?;

flowey/flowey_lib_hvlite/src/_jobs/check_distro_build.rs:88

• Using the external which command adds an unnecessary host dependency; minimal build environments may not have /usr/bin/which even when protoc is installed. Prefer the POSIX shell builtin command -v (matches the packaging guide) to locate protoc.

                let protoc = flowey::shell_cmd!(rt, "which protoc").read()?;
                let protoc = protoc.trim();

Guide/src/dev_guide/contrib/openvmm_packaging.md:15

• The PR description states the archive name is openvmm-<VERSION>-source.tar.gz, but both the docs and implementation here describe/produce openvmm-<VERSION>.tar.gz. Please align either the PR description or the documented/implemented naming so downstream packagers know what to expect.

The archive is named `openvmm-<VERSION>.tar.gz` and unpacks into
`openvmm-<VERSION>/`. The filename and the root directory deliberately
match `%{name}-%{version}`, so RPM's `%autosetup` and Fedora's forge
macros work without a `-n` override or a renamed source.

[Copilot]

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated no new comments.

Suppressed comments (1)

flowey/flowey_lib_hvlite/src/_jobs/check_distro_build.rs:92

• Relying on the external which utility makes this gate more fragile than necessary; minimal runner images/containers may not have which installed even when protoc is. Since this crate already depends on the which Rust crate, prefer which::which("protoc") and pass the resulting absolute path to PROTOC.

                // `which` rather than the `command -v` shown in the packaging
                // guide: flowey execs directly instead of through a shell, so
                // a shell builtin is not callable here.
                let protoc = flowey::shell_cmd!(rt, "which protoc").read()?;
                let protoc = protoc.trim();

[Copilot]

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated no new comments.

Suppressed comments (2)

flowey/flowey_lib_hvlite/src/_jobs/check_distro_build.rs:92

• Avoid invoking the external which binary to find protoc. Minimal distro images (or non-Ubuntu runners) may not have which installed, and this crate already depends on the which Rust crate. Using which::which("protoc") keeps the gate focused on the protoc requirement and removes an extra implicit dependency.

                let protoc = flowey::shell_cmd!(rt, "which protoc").read()?;
                let protoc = protoc.trim();

Guide/src/dev_guide/contrib/openvmm_packaging.md:69

• The requirements list includes "glibc development headers", but the package mapping table that follows omits them even though the surrounding text says the table reflects what the CI distribution-build gate installs. Add a row (or clarify they are pulled in transitively) to avoid confusing packagers.

- the Rust toolchain required by the workspace;
- a C compiler and linker;
- glibc development headers;
- Linux UAPI headers;
- OpenSSL development headers;

[Copilot]

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated no new comments.

Suppressed comments (3)

Guide/src/dev_guide/contrib/openvmm_packaging.md:36

• The PR explicitly leaves publication for follow-up work, and there is no pipeline that uploads these files yet. Saying the checksum is already published directs packagers to an asset that does not exist; describe the generated assembly output until publication lands.

`SHA256SUMS` is published alongside the archive and covers it by its
published name:

flowey/flowey_lib_hvlite/src/assemble_openvmm_source_release.rs:159

• This directory is reused by the caller, but only the current archive name is overwritten. After a version change, an older openvmm-<VERSION>.tar.gz remains in the typed artifact alongside the new assets, so a later publisher that uploads the artifact directory can ship stale releases. Clear this dedicated output directory before assembling it (or explicitly reject/remove unexpected entries).

                fs_err::create_dir_all(&output_dir)?;

Guide/src/dev_guide/contrib/openvmm_packaging.md:127

• Publication is explicitly out of scope for this PR, so this present-tense claim is not true after these changes. Make this section conditional on the future publication work rather than documenting a currently available upstream asset.

OpenVMM publishes an upstream source archive and its checksum. Mapping
those onto a distribution's own conventions is the packager's job, but
the points below are the ones OpenVMM's layout affects directly.

[Copilot]

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated no new comments.

Suppressed comments (2)

flowey/flowey_lib_hvlite/src/assemble_openvmm_source_release.rs:220

• The PR's reproducibility and checksum contracts are not exercised automatically: these unit tests only cover naming/version parsing, while the CI gate assembles once and never checks SHA256SUMS. A regression that drops -n, changes archive modes, or writes a bad checksum could therefore pass. Add a small temporary-Git-repository test that runs the real assembly twice, compares archive bytes, verifies the checksum, and checks the root prefix.

                flowey::shell_cmd!(rt, "gzip -n --best -f {source_tar}").run()?;

Guide/src/dev_guide/contrib/openvmm_packaging.md:71

• The requirements list includes glibc development headers, but this table omits their package mapping. Add the direct package names so Debian and Fedora packagers can declare this requirement explicitly rather than relying on it being pulled in transitively.

| C compiler and linker | `build-essential` | `gcc`, `binutils` |

[github-actions]

33 Petri tests failed (4 unstable)

[Copilot]

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Guide/src/dev_guide/contrib/openvmm_packaging.md:84

• The Fedora mapping omits the required glibc development headers. Installing gcc does not require glibc-devel, so a packager following this table can still be missing standard C headers. Add an explicit row for this requirement (the Debian equivalent is libc6-dev, even though build-essential normally pulls it in).

| C compiler and linker | `build-essential` | `gcc`, `binutils` |

flowey/flowey_lib_hvlite/src/assemble_openvmm_source_release.rs:210

• The current tests never execute archive assembly or verify its reproducibility. Because the distribution gate extracts and builds only one archive, removing gzip -n or otherwise introducing nondeterministic bytes would leave CI green despite determinism being a core release contract. Add an automated test using a temporary Git repository that assembles the same commit twice and compares the archive bytes (and validates SHA256SUMS).

                flowey::shell_cmd!(
                    rt,
                    "git -c tar.umask=0002 archive --format=tar --output {source_tar} --prefix={prefix} HEAD"
                )

[github-actions]

1 Petri tests failed (0 unstable)

[Copilot]

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated no new comments.

Suppressed comments (2)

flowey/flowey_lib_hvlite/src/assemble_openvmm_source_release.rs:220

• The new gate assembles each commit only once, so it cannot detect regressions in the core reproducibility guarantee—for example, dropping -n would still let every CI build pass while making release bytes vary by assembly time. Please add an automated scratch-repository test that assembles twice and compares the archive bytes and SHA256SUMS; the manual validation described in the PR will not protect future changes.

                flowey::shell_cmd!(rt, "gzip -n --best -f {source_tar}").run()?;

Guide/src/dev_guide/contrib/openvmm_packaging.md:185

• SHA256SUMS is not a substitute for signature verification when the checksum is distributed beside the archive: replacing the archive and checksum together still passes this command. Please make clear that the digest must be pinned or obtained through a trusted channel and that it verifies integrity, not archive authenticity.

OpenVMM does not currently publish an OpenPGP signature alongside the
archive, so `uscan` signature verification cannot be enabled. Verify the
archive against `SHA256SUMS` instead.

[Copilot]

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated no new comments.

Suppressed comments (3)

Guide/src/dev_guide/contrib/openvmm_packaging.md:185

• An unsigned SHA256SUMS file is not a substitute for OpenPGP authentication; if it is obtained alongside a modified archive, both can be replaced. Describe this as an integrity check rather than signature verification so the packaging guidance does not overstate what it proves.

OpenVMM does not currently publish an OpenPGP signature alongside the
archive, so `uscan` signature verification cannot be enabled. Verify the
archive against `SHA256SUMS` instead.

flowey/flowey_lib_hvlite/src/assemble_openvmm_source_release.rs:220

• The core reproducibility guarantee is not covered by the new tests: the CI gate assembles only once, while the unit tests cover naming and manifest parsing. Removing -n, changing the mode mask, or otherwise making the archive nondeterministic would therefore leave CI green. Add an automated scratch-repository test that assembles the same commit twice and compares the archive bytes (and ideally the checksum/root layout).

                flowey::shell_cmd!(rt, "gzip -n --best -f {source_tar}").run()?;

Guide/src/dev_guide/contrib/openvmm_packaging.md:65

• This implies that packagers can currently use an attestation to establish provenance, but the PR explicitly leaves attestations and publication to follow-up work. Clarify the present guarantee so readers do not mistake the generated checksum for authenticated provenance.

This issue also appears on line 183 of the same file.

The checksum and the attestation are what establish provenance.

[Copilot]

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated no new comments.

Suppressed comments (1)

Guide/src/dev_guide/contrib/openvmm_packaging.md:129

• The offline example omits the environment overrides required by the source tree. In a fresh packaging shell, the archived .cargo/config.toml sets PROTOC to the nonexistent .packages/Google.Protobuf.Tools/... path, so this command fails before using the distribution-provided compiler. Repeat both overrides here (the OpenSSL override also preserves the documented system-library build).

cargo build --release --locked --offline -p openvmm \
    --target x86_64-unknown-linux-gnu

[github-actions]

12 Petri tests failed (0 unstable)