ci: derive OpenVMM source archive identity from the workspace - #4139
Closed
Ben Hillis (benhillis) wants to merge 4 commits into
Closed
ci: derive OpenVMM source archive identity from the workspace#4139Ben Hillis (benhillis) wants to merge 4 commits into
Ben Hillis (benhillis) wants to merge 4 commits into
Conversation
`openvmm --version` did not exist, and there was nothing to report if it had: the workspace set no `version`, so every crate in it was `0.0.0`. OpenVMM ships as a source archive that packagers build themselves, which rules out the usual approach of having CI inject a version at build time -- the packager builds long after our pipeline has exited, from a tree with no git history to recover a version from. The version therefore has to be committed to the tree. QEMU, systemd, the kernel, cloud-hypervisor and rustc all do this, and each falls back gracefully when `.git` is absent rather than deriving the release version from a tag. So: set `version` in `[workspace.package]`, inherit it in `openvmm` and `openvmm_entry`, and wire up clap's `version`. `OPENVMM_PKGVERSION` lets a packager append their own build identity, as QEMU's `-Dpkgversion` and cloud-hypervisor's `CH_EXTRA_VERSION` do; an empty value is ignored, since build systems routinely pass an undefined variable through as "". Two things fall out of that: - The Windows VERSIONINFO resource is a second version surface, stamped from `OPENVMM_MAJOR`/`MINOR`/`PATCH`/`REVISION` and defaulting to `0.0.0.0`. No in-repo caller sets those, so leaving it alone would have left one binary reporting `0.1.0-dev` from `--version` and `0.0.0.0` from its file properties. It now defaults to the crate version, with the env vars still overriding per-component so a pipeline can stamp a build number in. - `cargo xtask fmt` deliberately strips `version` from `[package]`, partly because doing so also makes a crate unpublishable. The three crates on its exception list now set `publish = false` explicitly, restoring the property the lint was relying on. Verified by building `openvmm` from an extracted archive with no `.git` anywhere above it, which is the case that actually matters.
The workspace version advances only in the reviewed pull request that selects the next release. Development-build identity is added separately from the committed product version. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0ba1c0df-faf0-4a0e-b39d-c3eb0e363fce
OpenVMM is meant to be built and packaged by Linux distributions from a source archive, which makes that configuration a shipping interface. It is also the one configuration nothing in CI covers: every other build provisions native dependencies through `.packages/`, which a packager cannot use. A change that only resolves through `.packages/` therefore breaks downstream packagers silently. Add a job that does what a packager does, in order: 1. assemble a source archive from the commit under test, plus a `SHA256SUMS` covering it; 2. `sha256sum --check --strict`; 3. unpack outside the repository, so nothing can reach back into the checkout; 4. assert there is no `.git` inside; 5. assert the archive's embedded `.openvmm-release.json` matches what it was assembled as; 6. `cargo build --release --locked -p openvmm` with `PROTOC` and `OPENSSL_NO_VENDOR=1`, matching what a spec file does; 7. assert the binary's own `NEEDED` entries name the system OpenSSL. Building the checkout rather than an archive would let this pass on a tree a packager cannot reproduce, since a packager has no `.git` directory and no untracked files. The archive also has to carry its identity in a file, for the same reason. Step 7 is what keeps the build honest: if something acquired a vendored or static OpenSSL the build would still succeed, but the packaged binary would no longer be one the distribution can service. It reads `readelf -d` rather than `ldd` because `ldd` reports the whole transitive closure, and so would be satisfied by an unrelated library pulling OpenSSL in. Assembly is reproducible -- `git archive` is deterministic for a commit, `gzip -n` omits the timestamp, and `tar.umask` is pinned rather than inherited from the machine. Measured at 5m35s on a GitHub-hosted runner, against an existing 11m x64-linux baseline. Dropping debug info from the release profile and disabling incremental compilation is what keeps it there. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0ba1c0df-faf0-4a0e-b39d-c3eb0e363fce
Remove stamped release metadata from the assembled source. The archive already contains the canonical workspace version, so the distribution-build gate now builds the extracted tree and requires the resulting binary to report that exact version. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0ba1c0df-faf0-4a0e-b39d-c3eb0e363fce
Contributor
There was a problem hiding this comment.
Pull request overview
Updates OpenVMM’s CI/source-release machinery to treat the workspace version ([workspace.package].version) as the single source of truth for source-archive identity, and adds a CI gate that builds OpenVMM exactly as a Linux distribution would (from the assembled source archive, without .packages/).
Changes:
- Derive source archive naming/tagging identity from
[workspace.package].versionand assemble deterministically viagit archive+gzip -n. - Add a new distribution-configuration build gate that verifies the assembled archive is buildable and that
openvmm --versionmatches the archive version. - Propagate workspace version/publish rules into
openvmm,openvmm_entry, andvmgstool, and wire CLI--versionoutput to the intended product name/version.
Reviewed changes
Copilot reviewed 17 out of 18 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| xtask/src/tasks/fmt/lints/package_info.rs | Expands/clarifies version exceptions for house-rules linting. |
| vm/vmgs/vmgstool/Cargo.toml | Adds publish = false for a versioned internal tool crate. |
| openvmm/openvmm/Cargo.toml | Inherits workspace version and explicitly sets publish = false. |
| openvmm/openvmm/build.rs | Defaults Windows VERSIONINFO stamping to crate version components. |
| openvmm/openvmm_entry/src/cli_args.rs | Defines version string for --version and pins clap command name/version. |
| openvmm/openvmm_entry/Cargo.toml | Inherits workspace version and explicitly sets publish = false. |
| flowey/flowey_lib_hvlite/src/lib.rs | Exposes new source-release assembly node module. |
| flowey/flowey_lib_hvlite/src/assemble_openvmm_source_release.rs | Implements deterministic source archive assembly + identity derivation from workspace version. |
| flowey/flowey_lib_hvlite/src/_jobs/mod.rs | Registers the new distro-build CI job module. |
| flowey/flowey_lib_hvlite/src/_jobs/check_distro_build.rs | Adds CI job to verify distro-style build from assembled source archive and validate linkage/versioning. |
| flowey/flowey_lib_hvlite/Cargo.toml | Adds toml_edit dependency needed for reading workspace manifest version. |
| flowey/flowey_hvlite/src/pipelines/checkin_gates.rs | Wires the new distro-build job into checkin gates pipeline generation. |
| ci-flowey/openvmm-pr.yaml | Generated pipeline update adding the new distro-build job (ADO). |
| Cargo.toml | Defines [workspace.package].version as the canonical OpenVMM version. |
| Cargo.lock | Updates crate versions and dependency set to reflect workspace versioning and new deps. |
| .github/workflows/openvmm-pr.yaml | Generated workflow update adding the new distro-build job (GitHub). |
| .github/workflows/openvmm-pr-release.yaml | Generated workflow update adding distro-build job under release-ci label conditions. |
| .github/workflows/openvmm-ci.yaml | Generated workflow update (job ordering/bootstrap artifact uid updates, and job additions). |
Comment on lines
+67
to
+74
| /// Resolve the identity of the OpenVMM checkout in the current working | ||
| /// directory. | ||
| pub fn resolve_identity(rt: &mut RustRuntimeServices<'_>) -> anyhow::Result<SourceIdentity> { | ||
| let revision = flowey::shell_cmd!(rt, "git rev-parse HEAD").read()?; | ||
| let version = workspace_version(&std::env::current_dir()?.join("Cargo.toml"))?; | ||
|
|
||
| Ok(SourceIdentity { version, revision }) | ||
| } |
Comment on lines
+21
to
26
| // Default to the crate version so that the version Windows reports in | ||
| // the file properties and the one `openvmm --version` prints cannot | ||
| // disagree. The `OPENVMM_*` vars still win, which is how a build | ||
| // pipeline stamps its own build number in. There is no crate | ||
| // equivalent of the fourth component, so it stays 0 unless set. | ||
| let parse_u16 = |s: String| s.parse::<u16>().unwrap_or(0); |
Member
Author
|
Superseded by RFC #4150 and the integrated prototype in benhillis#12. The accepted behavior will be submitted as smaller follow-up PRs. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends on #4075 and #4132. After those merge, this diff reduces to the manifest-derived archive identity.
The distribution gate currently needs a second piece of release metadata because OpenVMM had no product version in its source tree. With the workspace version established, that duplication is unnecessary.
This change:
[workspace.package] versiondirectly from the selected tree;openvmm-<VERSION>-source.tar.gz, its root directory, and the eventualopenvmm-v<VERSION>tag from that value;git archiveplusgzip -n;openvmm --versionoutput to match the archive version exactly.This PR does not add a release trigger or publish anything.