Skip to content

TEZ-4719: Move to Junit6#502

Merged
abstractdog merged 4 commits into
apache:masterfrom
Aggarwal-Raghav:TEZ-4719
Jul 2, 2026
Merged

TEZ-4719: Move to Junit6#502
abstractdog merged 4 commits into
apache:masterfrom
Aggarwal-Raghav:TEZ-4719

Conversation

@Aggarwal-Raghav

@Aggarwal-Raghav Aggarwal-Raghav commented May 31, 2026

Copy link
Copy Markdown
Contributor
  1. Update Annotations:
@Test (Change import from org.junit.Test to org.junit.jupiter.api.Test)
@Before → @BeforeEach
@After → @AfterEach
@BeforeClass → @BeforeAll (Method must be static)
@AfterClass → @AfterAll (Method must be static)
@Ignore → @Disabled
  1. Update Assertions:
    org.junit.Assert.* → org.junit.jupiter.api.Assertions.*

  2. Removed the transitive dependency on Hamcrest matchers.

  3. Migrated all @Test(timeout = ...) parameters to the @Timeout annotation while maintianing the Millisecond timeunit

  4. Replaced @Rule TemporaryFolder with JUnit Jupiter's @TempDir

  5. Note: In JUnit 4, the custom assertion message was the first parameter. In JUnit 6, it is the last parameter.
    In JUnit 4: assertEquals("Values should match", expected, actual);
    In JUnit 6:assertEquals(expected, actual, "Values should match");

@tez-yetus

This comment was marked as outdated.

@Aggarwal-Raghav Aggarwal-Raghav changed the title [WIP] TEZ-4719: Move to Junit6 TEZ-4719: Move to Junit6 Jun 1, 2026

import org.junit.Assert;
import org.junit.Test;
import org.junit.jupiter.api.*;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which is the preferred way?

  1. Using single line separte import
  2. * imports

@tez-yetus

This comment was marked as outdated.

@Aggarwal-Raghav

Copy link
Copy Markdown
Contributor Author

UT has passed, which is good thing, few indendations and java formatting is left that will be handled once final rebase with #505 is done.

@Aggarwal-Raghav Aggarwal-Raghav force-pushed the TEZ-4719 branch 3 times, most recently from 5fb400c to 5f401be Compare June 23, 2026 18:42
@tez-yetus

This comment was marked as outdated.

@tez-yetus

This comment was marked as outdated.

@tez-yetus

Copy link
Copy Markdown

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 1m 58s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 2s No case conflicting files found.
+0 🆗 detsecrets 0m 2s detect-secrets was not available.
+0 🆗 xmllint 0m 0s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 247 new or modified test files.
_ master Compile Tests _
+0 🆗 mvndep 0m 39s Maven dependency ordering for branch
+1 💚 mvninstall 2m 51s master passed
+1 💚 compile 2m 31s master passed
+1 💚 checkstyle 6m 23s master passed
+1 💚 javadoc 6m 33s master passed
+0 🆗 spotbugs 0m 30s hadoop-shim in master has 3 extant spotbugs warnings.
+0 🆗 spotbugs 0m 46s tez-api in master has 92 extant spotbugs warnings.
+0 🆗 spotbugs 0m 30s tez-common in master has 13 extant spotbugs warnings.
+0 🆗 spotbugs 0m 35s tez-runtime-internals in master has 92 extant spotbugs warnings.
+0 🆗 spotbugs 0m 44s tez-runtime-library in master has 143 extant spotbugs warnings.
+0 🆗 spotbugs 0m 36s tez-mapreduce in master has 97 extant spotbugs warnings.
+0 🆗 spotbugs 0m 27s tez-examples in master has 2 extant spotbugs warnings.
+0 🆗 spotbugs 0m 56s tez-dag in master has 537 extant spotbugs warnings.
+0 🆗 spotbugs 0m 29s tez-tests in master has 6 extant spotbugs warnings.
+0 🆗 spotbugs 0m 30s tez-plugins/tez-protobuf-history-plugin in master has 7 extant spotbugs warnings.
+0 🆗 spotbugs 0m 28s tez-plugins/tez-yarn-timeline-history in master has 2 extant spotbugs warnings.
+0 🆗 spotbugs 0m 29s tez-plugins/tez-yarn-timeline-history-with-acls in master has 2 extant spotbugs warnings.
+0 🆗 spotbugs 0m 23s tez-plugins/tez-yarn-timeline-cache-plugin in master has 1 extant spotbugs warnings.
+0 🆗 spotbugs 0m 28s tez-plugins/tez-yarn-timeline-history-with-fs in master has 5 extant spotbugs warnings.
+0 🆗 spotbugs 0m 30s tez-plugins/tez-history-parser in master has 21 extant spotbugs warnings.
+0 🆗 spotbugs 0m 32s tez-plugins/tez-aux-services in master has 5 extant spotbugs warnings.
+0 🆗 spotbugs 3m 41s root in master has 1054 extant spotbugs warnings.
+0 🆗 spotbugs 0m 31s tez-tools/analyzers/job-analyzer in master has 20 extant spotbugs warnings.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 10s Maven dependency ordering for patch
+1 💚 mvninstall 2m 15s the patch passed
+1 💚 codespell 0m 49s No new issues.
+1 💚 compile 2m 34s the patch passed
-1 ❌ javac 2m 34s /results-compile-javac-root.txt root generated 22 new + 684 unchanged - 23 fixed = 706 total (was 707)
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 21s /results-checkstyle-tez-api.txt tez-api: The patch generated 5 new + 149 unchanged - 8 fixed = 154 total (was 157)
-0 ⚠️ checkstyle 0m 25s /results-checkstyle-tez-runtime-library.txt tez-runtime-library: The patch generated 234 new + 606 unchanged - 53 fixed = 840 total (was 659)
-0 ⚠️ checkstyle 0m 20s /results-checkstyle-tez-mapreduce.txt tez-mapreduce: The patch generated 1 new + 119 unchanged - 3 fixed = 120 total (was 122)
-0 ⚠️ checkstyle 0m 27s /results-checkstyle-tez-dag.txt tez-dag: The patch generated 63 new + 936 unchanged - 40 fixed = 999 total (was 976)
-0 ⚠️ checkstyle 0m 20s /results-checkstyle-tez-tests.txt tez-tests: The patch generated 31 new + 182 unchanged - 22 fixed = 213 total (was 204)
-0 ⚠️ checkstyle 0m 17s /results-checkstyle-tez-plugins_tez-protobuf-history-plugin.txt tez-plugins/tez-protobuf-history-plugin: The patch generated 8 new + 17 unchanged - 6 fixed = 25 total (was 23)
-0 ⚠️ checkstyle 0m 16s /results-checkstyle-tez-plugins_tez-aux-services.txt tez-plugins/tez-aux-services: The patch generated 4 new + 49 unchanged - 6 fixed = 53 total (was 55)
-0 ⚠️ checkstyle 0m 48s /results-checkstyle-root.txt root: The patch generated 346 new + 2307 unchanged - 138 fixed = 2653 total (was 2445)
+1 💚 javadoc 6m 32s the patch passed
+1 💚 spotbugs 16m 25s the patch passed
_ Other Tests _
+1 💚 unit 62m 11s root in the patch passed.
+1 💚 asflicense 6m 5s The patch does not generate ASF License warnings.
145m 1s
Subsystem Report/Notes
Docker ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-502/7/artifact/out/Dockerfile
Optional Tests dupname compile unit asflicense codespell detsecrets javac javadoc xmllint spotbugs checkstyle
uname Linux 1c89ba3c7658 5.15.0-179-generic #189-Ubuntu SMP Tue May 5 18:20:56 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality tez-personality.sh
git revision master / 17d1549
Default Java Eclipse Adoptium-21.0.11+10-LTS
Test Results https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-502/7/testReport/
Max. process+thread count 1546 (vs. ulimit of 5500)
modules C: hadoop-shim tez-api tez-common tez-runtime-internals tez-runtime-library tez-mapreduce tez-examples tez-dag tez-tests tez-ext-service-tests tez-plugins/tez-protobuf-history-plugin tez-plugins/tez-yarn-timeline-history tez-plugins/tez-yarn-timeline-history-with-acls tez-plugins/tez-yarn-timeline-cache-plugin tez-plugins/tez-yarn-timeline-history-with-fs tez-plugins/tez-history-parser tez-plugins/tez-aux-services tez-tools/analyzers/job-analyzer . U: .
Console output https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-502/7/console
versions git=2.43.0 maven=3.9.15 spotbugs=4.9.3 codespell=2.4.1
Powered by Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

@Aggarwal-Raghav

Copy link
Copy Markdown
Contributor Author

@abstractdog , this is ready for review.

@abstractdog abstractdog left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is huge so far, thanks @Aggarwal-Raghav ! left some comments

Comment on lines +128 to +132
Environment.PWD.$()
+ File.pathSeparator
+ "USER_PATH"
+ File.pathSeparator
+ "DEFAULT_PATH",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's a matter of taste, but I don't think indenting a string concatenation this way makes the code better, a single line is more compact and still pretty much readable

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i was using google java formatter in intelliJ, now I have moved to tez project checkstyle present at tez-build-tools/src/main/resources/checkstyle/checkstyle.xml and updated the whole PR manually file by file. Fixed line length >120 in this PR. I hope it should help in checkstyle/spotbugs as well.

@abstractdog abstractdog Jul 2, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for clarifying!
I believe we can start soon a community discussion about which formatter to accept, and then we can enforce it by spotless: that's going to be another huge patch, not a problem :)

Comment thread tez-common/src/test/java/org/apache/tez/common/MockDNSToSwitchMapping.java Outdated
Comment thread pom.xml Outdated
Comment thread pom.xml
Comment thread tez-tests/src/test/java/org/apache/tez/test/TestDriver.java Outdated
Comment thread tez-tests/src/test/java/org/apache/tez/test/TestSecureShuffle.java Outdated
Comment thread tez-tests/src/test/java/org/apache/tez/test/TestSecureShuffle.java Outdated
Comment thread tez-dag/src/test/java/org/apache/tez/dag/app/dag/impl/TestDAGImpl.java Outdated
@Aggarwal-Raghav Aggarwal-Raghav force-pushed the TEZ-4719 branch 2 times, most recently from dd8e0a0 to 6374968 Compare June 30, 2026 19:25
@tez-yetus

This comment was marked as outdated.

@Aggarwal-Raghav

Copy link
Copy Markdown
Contributor Author

CI has failed because of some artifactory issue, have re-triggered it

@tez-yetus

This comment was marked as outdated.

@Aggarwal-Raghav

Aggarwal-Raghav commented Jul 1, 2026

Copy link
Copy Markdown
Contributor Author

Giving 1 more try, there is some infra issue i guess, puppycrawler is going OOM, and different issue in checkstyle , sporbugs. then will play aroung Jenkinsfile. I know there are few checkstyle issues still because of this PR, will handle once CI start passing.

@Aggarwal-Raghav

Copy link
Copy Markdown
Contributor Author

Failing with OOM.

13:05:14  [INFO] Running org.apache.tez.common.TestTezCommonUtils
13:05:24  [ERROR] java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached

@Aggarwal-Raghav

Copy link
Copy Markdown
Contributor Author

@abstractdog , is it because of some jenkins node problem?
https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-502/09, UT never ran — Some other failure
https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-502/10, UT passed — Ran on hadoop14 node — Still other error are present
https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-502/11, UT OOM — Ran on hadoop52 node — UT failed

@Aggarwal-Raghav

Copy link
Copy Markdown
Contributor Author

rebasing as there are merge conflict. The older run is stuck, neither failing nor proceeding.

@abstractdog

Copy link
Copy Markdown
Contributor

rebasing as there are merge conflict. The older run is stuck, neither failing nor proceeding.

yeah, please be aware of a recent unit test, which might need to be migrated: 8fde353

@Aggarwal-Raghav

Copy link
Copy Markdown
Contributor Author

rebasing as there are merge conflict. The older run is stuck, neither failing nor proceeding.

yeah, please be aware of a recent unit test, which might need to be migrated: 8fde353

yes, doing that only :-)

@Aggarwal-Raghav

Copy link
Copy Markdown
Contributor Author

For TestTezCommonUtils where OOM happened in last run, testMkDirForAM, i optimized that particular test as MiniDFSCluster was already created in BeforeAll setup

@tez-yetus

Copy link
Copy Markdown

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 12s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 3s No case conflicting files found.
+0 🆗 detsecrets 0m 3s detect-secrets was not available.
+0 🆗 xmllint 0m 0s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 224 new or modified test files.
_ master Compile Tests _
+0 🆗 mvndep 0m 46s Maven dependency ordering for branch
+1 💚 mvninstall 4m 31s master passed
+1 💚 compile 4m 15s master passed
+1 💚 checkstyle 9m 25s master passed
+1 💚 javadoc 9m 23s master passed
+0 🆗 spotbugs 0m 39s hadoop-shim in master has 3 extant spotbugs warnings.
+0 🆗 spotbugs 1m 18s tez-api in master has 92 extant spotbugs warnings.
+0 🆗 spotbugs 0m 44s tez-common in master has 13 extant spotbugs warnings.
+0 🆗 spotbugs 0m 51s tez-runtime-internals in master has 92 extant spotbugs warnings.
+0 🆗 spotbugs 1m 11s tez-runtime-library in master has 143 extant spotbugs warnings.
+0 🆗 spotbugs 0m 53s tez-mapreduce in master has 97 extant spotbugs warnings.
+0 🆗 spotbugs 0m 43s tez-examples in master has 2 extant spotbugs warnings.
+0 🆗 spotbugs 1m 37s tez-dag in master has 537 extant spotbugs warnings.
+0 🆗 spotbugs 0m 49s tez-tests in master has 6 extant spotbugs warnings.
+0 🆗 spotbugs 0m 47s tez-plugins/tez-protobuf-history-plugin in master has 7 extant spotbugs warnings.
+0 🆗 spotbugs 0m 44s tez-plugins/tez-yarn-timeline-history in master has 2 extant spotbugs warnings.
+0 🆗 spotbugs 0m 40s tez-plugins/tez-yarn-timeline-history-with-acls in master has 2 extant spotbugs warnings.
+0 🆗 spotbugs 0m 43s tez-plugins/tez-yarn-timeline-cache-plugin in master has 1 extant spotbugs warnings.
+0 🆗 spotbugs 0m 45s tez-plugins/tez-yarn-timeline-history-with-fs in master has 5 extant spotbugs warnings.
+0 🆗 spotbugs 0m 46s tez-plugins/tez-history-parser in master has 21 extant spotbugs warnings.
+0 🆗 spotbugs 0m 46s tez-plugins/tez-aux-services in master has 5 extant spotbugs warnings.
+0 🆗 spotbugs 6m 26s root in master has 1054 extant spotbugs warnings.
+0 🆗 spotbugs 0m 46s tez-tools/analyzers/job-analyzer in master has 20 extant spotbugs warnings.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 16s Maven dependency ordering for patch
+1 💚 mvninstall 4m 1s the patch passed
+1 💚 codespell 1m 42s No new issues.
+1 💚 compile 4m 11s the patch passed
-1 ❌ javac 4m 11s /results-compile-javac-root.txt root generated 24 new + 682 unchanged - 25 fixed = 706 total (was 707)
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 22s The patch passed checkstyle in hadoop-shim
-0 ⚠️ checkstyle 0m 27s /results-checkstyle-tez-api.txt tez-api: The patch generated 5 new + 144 unchanged - 13 fixed = 149 total (was 157)
+1 💚 checkstyle 0m 22s The patch passed checkstyle in tez-common
+1 💚 checkstyle 0m 24s The patch passed checkstyle in tez-runtime-internals
-0 ⚠️ checkstyle 0m 35s /results-checkstyle-tez-runtime-library.txt tez-runtime-library: The patch generated 165 new + 596 unchanged - 57 fixed = 761 total (was 653)
-0 ⚠️ checkstyle 0m 26s /results-checkstyle-tez-mapreduce.txt tez-mapreduce: The patch generated 1 new + 117 unchanged - 4 fixed = 118 total (was 121)
+1 💚 checkstyle 0m 24s The patch passed checkstyle in tez-examples
-0 ⚠️ checkstyle 0m 43s /results-checkstyle-tez-dag.txt tez-dag: The patch generated 42 new + 919 unchanged - 42 fixed = 961 total (was 961)
-0 ⚠️ checkstyle 0m 28s /results-checkstyle-tez-tests.txt tez-tests: The patch generated 9 new + 139 unchanged - 24 fixed = 148 total (was 163)
+1 💚 checkstyle 0m 25s The patch passed checkstyle in tez-ext-service-tests
-0 ⚠️ checkstyle 0m 24s /results-checkstyle-tez-plugins_tez-protobuf-history-plugin.txt tez-plugins/tez-protobuf-history-plugin: The patch generated 8 new + 17 unchanged - 6 fixed = 25 total (was 23)
+1 💚 checkstyle 0m 24s The patch passed checkstyle in tez-yarn-timeline-history
+1 💚 checkstyle 0m 24s The patch passed checkstyle in tez-yarn-timeline-history-with-acls
+1 💚 checkstyle 0m 23s The patch passed checkstyle in tez-yarn-timeline-cache-plugin
+1 💚 checkstyle 0m 25s The patch passed checkstyle in tez-yarn-timeline-history-with-fs
+1 💚 checkstyle 0m 26s The patch passed checkstyle in tez-history-parser
+1 💚 checkstyle 0m 23s tez-plugins/tez-aux-services: The patch generated 0 new + 49 unchanged - 6 fixed = 49 total (was 55)
+1 💚 checkstyle 0m 25s The patch passed checkstyle in job-analyzer
-0 ⚠️ checkstyle 1m 14s /results-checkstyle-root.txt root: The patch generated 230 new + 2220 unchanged - 152 fixed = 2450 total (was 2372)
+1 💚 javadoc 9m 16s the patch passed
+1 💚 spotbugs 26m 32s the patch passed
_ Other Tests _
+1 💚 unit 76m 33s root in the patch passed.
+1 💚 asflicense 7m 43s The patch does not generate ASF License warnings.
200m 19s
Subsystem Report/Notes
Docker ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-502/12/artifact/out/Dockerfile
Optional Tests dupname compile unit asflicense codespell detsecrets javac javadoc xmllint spotbugs checkstyle
uname Linux c32fbec5b6b3 5.15.0-177-generic #187-Ubuntu SMP Sat Apr 11 22:54:33 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality tez-personality.sh
git revision master / 8fde353
Default Java Eclipse Adoptium-21.0.11+10-LTS
Test Results https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-502/12/testReport/
Max. process+thread count 2113 (vs. ulimit of 5500)
modules C: hadoop-shim tez-api tez-common tez-runtime-internals tez-runtime-library tez-mapreduce tez-examples tez-dag tez-tests tez-ext-service-tests tez-plugins/tez-protobuf-history-plugin tez-plugins/tez-yarn-timeline-history tez-plugins/tez-yarn-timeline-history-with-acls tez-plugins/tez-yarn-timeline-cache-plugin tez-plugins/tez-yarn-timeline-history-with-fs tez-plugins/tez-history-parser tez-plugins/tez-aux-services tez-tools/analyzers/job-analyzer . U: .
Console output https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-502/12/console
versions git=2.43.0 maven=3.9.15 spotbugs=4.9.3 codespell=2.4.1
Powered by Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

@Aggarwal-Raghav

Copy link
Copy Markdown
Contributor Author

CI has passed @abstractdog , I think the checkstyle especially in javac can be handled separately?

@abstractdog

Copy link
Copy Markdown
Contributor

CI has passed @abstractdog , I think the checkstyle especially in javac can be handled separately?

agree! those problems were not introduced by this patch, and this is already huge, it's worth moving on before further conflicts :)

@abstractdog abstractdog self-requested a review July 2, 2026 07:10

@abstractdog abstractdog left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@abstractdog abstractdog merged commit d53e67d into apache:master Jul 2, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants