DAOS-18626 test: Add per-sanitizer suite selection and JUnit reporting to run_utest.py#18697
Draft
knard38 wants to merge 1 commit into
Draft
DAOS-18626 test: Add per-sanitizer suite selection and JUnit reporting to run_utest.py#18697knard38 wants to merge 1 commit into
knard38 wants to merge 1 commit into
Conversation
…g to run_utest.py Extend utils/run_utest.py so the unit-test suite can be run selectively under ASan, UBSan, or TSan, and so results are reported in a form CI can consume: - New --asan/--ubsan/--tsan flags select suites via new 'asan'/'tsan' Suite config keys (utest.yaml); suites unsuitable under a given sanitizer (e.g. memcheck-only suites, control Go tests) opt out. - New --list flag prints the suites/tests that would run, honoring all selection/filter options, without running them. - Per-test sanitizer log capture: new-since-setup() sanitizer log files (found via the ASAN_OPTIONS/UBSAN_OPTIONS/TSAN_OPTIONS log_path= directory) are copied into each test's own log dir, tagged with a companion '<logfile>.testname' file for attribution. - cmocka XML results are now written into CMOCKA_XML_FILE's directory (the Docker-mounted test-results/ volume) instead of a source-relative test_results/ directory, and only newly-appeared XML files are relabeled per test/suite. - In GHA context, one JUnit TestCase per test is now written per suite (write_xml_result()) so result counts are accurate. - utils/utest.yaml: tag suites with 'tsan: True' (opt-in, TSan suites are a small subset) and 'asan: False'/'gha: False' (opt-out) as appropriate for each suite. None of these flags or config keys are referenced by any workflow yet; this is a standalone, backward-compatible change to the test runner. Part of the DAOS-18626 ASan CI split (see ckochhof/dev/master/daos-18626/patch-001 for the original, unsplit commit). Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>
|
Ticket title is 'Add ASan/UBSan/TSan sanitizer checks to the GitHub Actions unit-test CI' |
Collaborator
|
Test stage Functional Cluster Box Medium MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18697/1/execution/node/1755/log |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is PR 2 of 10 in the split of DAOS-18626 (see
ckochhof/dev/master/daos-18626/patch-001for the original, unsplit commit). It extendsutils/run_utest.pyso the unit-test suite can be run selectively under ASan, UBSan, or TSan, and so results are reported in a form CI can consume. Nothing calls these new flags yet in this PR — that comes later in this stack.What/why
--asan/--ubsan/--tsanflags select suites via newasan/tsanSuiteconfig keys (utest.yaml); suites unsuitable under a given sanitizer (e.g. memcheck-only suites, control Go tests) opt out viaasan: False/gha: False.--listflag prints the suites/tests that would run, honoring all selection/filter options, without running them.setup()sanitizer log files (found via theASAN_OPTIONS/UBSAN_OPTIONS/TSAN_OPTIONSlog_path=directory) are copied into each test's own log dir, tagged with a companion<logfile>.testnamefile for attribution.CMOCKA_XML_FILE's directory (the Docker-mountedtest-results/volume) instead of a source-relativetest_results/directory, and only newly-appeared XML files are relabeled per test/suite.TestCaseis now written per test per suite (write_xml_result()) so result counts are accurate.utils/utest.yaml: tag suites withtsan: True(opt-in, TSan suites are a small subset) andasan: False/gha: False(opt-out) as appropriate for each suite.Testing
python3 utils/run_utest.py --asan --list,--ubsan --list,--tsan --listprint the expected suite/test subsets (manually verified againstutest.yaml's new tags).patch-008) and activated in CI by PR 9 (patch-010) of this stack.Steps for the author:
After all prior steps are complete: