Repository navigation
BUG: Seven ImageProcessing tests never run under ctest because their names contain a semicolon #1898
Description
Problem
Seven ImageProcessing test cases never run under ctest, and ctest reports them as passed. Their names contain a
semicolon, and test discovery splits each name at the semicolon into two ctest entries. Neither entry matches a test
case, so Catch2 runs nothing and exits successfully.
- Expected: every visible test case is registered with ctest under its full name and runs.
- Actual: each of these cases becomes two entries such as
ImageProcessing::BinaryMorphologicalClosingImageFilter: Computed-expected (safeBorder=falseand
both algorithm paths). Both printNo test cases matchedand pass in about 0.02 s.
Occurs on develop at f58acc1. The affected test cases:
BinaryMorphologicalClosingImageFilterTest.cppL38 and L104ContourEngineTest.cppL648GrayscaleMorphologicalOpeningImageFilterTest.cppL18GrayscaleMorphologicalClosingImageFilterTest.cppL18WhiteTopHatImageFilterTest.cppL18BlackTopHatImageFilterTest.cppL18
Steps to Reproduce
- Configure a build with
SIMPLNX_BUILD_TESTS=ONand buildImageProcessingUnitTest. - List the registered tests:
ctest --test-dir "<build-dir>" -N -R "safeBorder=false$|both algorithm paths\)$|labels differ\)$".
The list shows the name fragments, not the full names. - Ask ctest for one full name:
ctest --test-dir "<build-dir>" -R "^ImageProcessing::BinaryMorphologicalClosingImageFilter: Computed-expected \(safeBorder=false; both algorithm paths\)$" --no-tests=error.
ctest finds no test. - Run one fragment with
-V: the command passes" both algorithm paths)"to the executable, Catch2 prints
No test cases matched, and ctest reportsPassed.
Cause
cmake/Plugin.cmake L410 registers the plugin tests with
catch_discover_tests(${UNIT_TEST_TARGET}). In Catch2 2.13.6, the discovery script CatchAddTests.cmake (in the
vcpkg Catch2 package) turns the output of --list-test-names-only into a CMake list and registers one test per
element:
string(REPLACE "\n" ";" output "${output}")
...
foreach(line ${output})A semicolon is the CMake list separator, so a test name that contains ; becomes two list elements. The script
escapes ,, [ and ] in the test spec it passes to the executable, but nothing protects ;.
Verification Unit Test
The existing test cases are the verification; no new test is needed. For example, from
src/Plugins/ImageProcessing/test/BinaryMorphologicalClosingImageFilterTest.cpp:
TEST_CASE("ImageProcessing::BinaryMorphologicalClosingImageFilter: Computed-expected (safeBorder=false; both algorithm paths)",
"[ImageProcessing][BinaryMorphologicalClosingImageFilter]")Command that should run it (in-core Release build of develop, macOS arm64):
ctest --test-dir "<build-dir>" -R "^ImageProcessing::BinaryMorphologicalClosingImageFilter: Computed-expected \(safeBorder=false; both algorithm paths\)$" -j 1 --output-on-failure --timeout 300 --no-tests=error
Observed:
No tests were found!!!
Errors while running CTest
And the fragment that ctest does run:
203: Test command: .../Bin/ImageProcessingUnitTest " both algorithm paths)"
203: No test cases matched 'both algorithm paths)'
1/1 Test #203: both algorithm paths) ........... Passed 0.02 sec
Proposed Fix
- Replace the
;in the seven test names with,(for example
Computed-expected (safeBorder=false, both algorithm paths)). Run the seven tests afterwards: they have not run
under ctest, so they may fail. - Keep it from coming back: add a rule to
docs/Code_Style_Guide.md(Testing) that a test case name contains no
;, and optionally a CMake check after discovery that fails when a registered name does not match a test case.
Notes
- Found while checking the test registration for DOC: Add coding rules, a knowledge base and a shared AGENTS.md #1889; that pull request does not change
this behavior (its split discovery registers the same names). - One SimplnxOoc test case has the same problem.
- Test names with commas, brackets or parentheses register correctly.
Activity
imikejackson commentedon Oct 30, 2026
@joeykleingers Run a sweep over all unit tests to make sure NONE of them have a semi-colon. Then update the PR that has the Code-Rules to update that document also.