ci: cache Maven distributions and retry bootstrap downloads - #5422
Open
sunchao wants to merge 1 commit into
Open
ci: cache Maven distributions and retry bootstrap downloads#5422sunchao wants to merge 1 commit into
sunchao wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why are the changes needed?
A Java CI job can fail before it builds or tests Comet even when its Maven dependency cache is a hit. The cache contains
.m2/repository, but the Maven wrapper installs Maven itself under.m2/wrapper/dists. Fresh runners therefore still download the Maven distribution from Maven Central.For example, the Spark 3.4 / JDK 11 shuffle job on #5403 successfully restored its dependency cache, then received HTTP 429 while downloading
apache-maven-3.9.6-bin.zip. No shuffle tests ran. The subsequent diagnostic step also failed becausetargetdid not exist, adding a second error that obscured the original setup failure.This makes a temporary download failure require a manual CI rerun without providing any information about the patch being tested. We should reuse the Maven installation and tolerate brief bootstrap failures while preserving real build and test failures.
Which issue does this PR close? None. This is a CI reliability follow-up motivated by #5403, not a change to that PR's expression implementation.
What changes were proposed in this PR?
The shared Java-test action now treats Maven bootstrap as a separate setup phase. Linux jobs restore a small distribution cache keyed by the wrapper configuration, independently of the dependency cache. The action then runs
./mvnw -B --versionwith up to four attempts and increasing, jittered delays. Once bootstrap succeeds, it saves the distribution immediately, so a later test failure does not discard a usable Maven installation.Compilation and tests still use their existing commands exactly once. Maven remains at 3.9.6, suite selection is unchanged, and the existing macOS caching workaround remains in place; macOS receives the bootstrap retry without re-enabling caching. This change is limited to callers of the shared Java-test action, not other Maven callers such as RAT or Spark's own test builders.
Failure reporting also distinguishes missing build output from a failed diagnostic command. The debug listing tolerates an absent root
targetdirectory, and artifact uploads tolerate files that were never produced. Existing test reports can now be uploaded after a test failure, while cancellation still skips that upload.How was this PR tested?
installinvocation with their exit code preserved, diagnostics with and withouttarget, unchanged existing test/dependency-cache steps, and restore/bootstrap/save ordering.actionlint1.7.12 on the repository workflows and a temporary workflow containing the composite action's actual steps, andgit diff --check.The local cache test copies a real wrapper distribution; it does not emulate the GitHub cache service. Hosted cache integration remains for this PR's CI. Comet's Rust/JVM suites were not rerun locally because the patch changes only the CI action.