Repository navigation
feat(ci): Publish Maven artifacts and release tarballs on release - #43
Merged
Merged
Conversation
giggsoff
force-pushed
the
feature/NGSOK-1746
branch
from
September 2, 2026 12:58
af0a0d5 to
e00674a
Compare
giggsoff
changed the base branch from
develop/4.3.0/4.2.0.1
to
develop/4.4.0/4.2.0.2
September 2, 2026 12:58
Deploy the reactor to the Arenadata GitHub Packages Maven registry and attach spark-<version>-bin-hadoop3.tgz, spark-<version>-bin-hadoop3-cloud.tgz and the PySpark sdist to the GitHub release. A verify job gates both on the tag sitting on a release/** or develop/** branch, CI having passed for that commit, and the tag matching the pom version. The deploy target now shares the id of the `arenadata` <repositories> entry, so one settings.xml server authenticates both the reads and the deploy.
giggsoff
force-pushed
the
feature/NGSOK-1746
branch
from
September 2, 2026 13:19
e00674a to
1298bd2
Compare
werzerbb
approved these changes
Sep 2, 2026
This was referenced Sep 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The 4.2.x half of the pair; #61 is the 3.5.x counterpart. Publish Spark to the Arenadata GitHub Packages Maven registry and attach the binary distributions to the GitHub release when a release is created.
Guards — a
verifyjob runs first and is the only thing that can reject the release, so a bad tag fails in a minute rather than two hours into a reactor build. Ported from kyuubi's release workflow: the tag must sit on arelease/**ordevelop/**branch, and CI must have passed for that commit. Added to those, the tag must match the pom version. Unlike the kyuubi original, an unfinished CI run is reported as still running rather than as missing —conclusionis empty in both cases.Maven —
mvn deploywithdeployAtEnd, so the whole reactor builds before any upload and a mid-reactor failure cannot leave a partial release that GitHub Packages then refuses to re-publish. Manual dispatch can resume the reactor via-rf. Javadoc/scaladoc jars are skipped (attach-scaladocsis bound toverify, sodeploywould build docs for the whole reactor); source jars are still published.Credentials — the reactor reads the Arenadata Hive artifacts (
hive.version) from thearenadataGitHub Packages repository, and GitHub Packages requires authentication even for public repos. Sinceactions/setup-javaconfigures only one server, the deploy target id is renamedgithub→arenadatato share the id of the<repositories>entry; one<server>then covers both the reads and the deploy. Nothing else referenced the old id.Assets — builds
spark-<version>-bin-hadoop3.tgz,spark-<version>-bin-hadoop3-cloud.tgzand the PySpark sdist and attaches them to the release, matching the asset set ofv4.2.0.1-4.3.0-2. The profile list is the one recorded in that release'sRELEASEfile;-Phadoop-cloudis added only for the-clouddistribution and the Maven deploy, since it pulls the AWS SDK bundle into the assembly and doubles the tarball. Asset paths are globbed and asserted to match exactly one file rather than derived, because the PySpark version is normalised by PEP 440 and is not a substring of the pom version. Reference sizes to check the first run against: 525 / 1134 / 1040 MB.Note that a
releaseevent resolves the workflow from the tag's tree, so this takes effect for tags cut after it merges; it cannot retroactively attach assets to an existing release. Therelease_tagdispatch input is the re-run path for a failed upload.