Skip to content

BUG: Seven ImageProcessing tests never run under ctest because their names contain a semicolon #1898

Open

Description

@joeykleingers

Problem

Seven ImageProcessing test cases never run under ctest, and ctest reports them as passed. Their names contain a
semicolon, and test discovery splits each name at the semicolon into two ctest entries. Neither entry matches a test
case, so Catch2 runs nothing and exits successfully.

  • Expected: every visible test case is registered with ctest under its full name and runs.
  • Actual: each of these cases becomes two entries such as
    ImageProcessing::BinaryMorphologicalClosingImageFilter: Computed-expected (safeBorder=false and
    both algorithm paths). Both print No test cases matched and pass in about 0.02 s.

Occurs on develop at f58acc1. The affected test cases:

Steps to Reproduce

  1. Configure a build with SIMPLNX_BUILD_TESTS=ON and build ImageProcessingUnitTest.
  2. List the registered tests: ctest --test-dir "<build-dir>" -N -R "safeBorder=false$|both algorithm paths\)$|labels differ\)$".
    The list shows the name fragments, not the full names.
  3. Ask ctest for one full name:
    ctest --test-dir "<build-dir>" -R "^ImageProcessing::BinaryMorphologicalClosingImageFilter: Computed-expected \(safeBorder=false; both algorithm paths\)$" --no-tests=error.
    ctest finds no test.
  4. Run one fragment with -V: the command passes " both algorithm paths)" to the executable, Catch2 prints
    No test cases matched, and ctest reports Passed.

Cause

cmake/Plugin.cmake L410 registers the plugin tests with
catch_discover_tests(${UNIT_TEST_TARGET}). In Catch2 2.13.6, the discovery script CatchAddTests.cmake (in the
vcpkg Catch2 package) turns the output of --list-test-names-only into a CMake list and registers one test per
element:

string(REPLACE "\n" ";" output "${output}")
...
foreach(line ${output})

A semicolon is the CMake list separator, so a test name that contains ; becomes two list elements. The script
escapes ,, [ and ] in the test spec it passes to the executable, but nothing protects ;.

Verification Unit Test

The existing test cases are the verification; no new test is needed. For example, from
src/Plugins/ImageProcessing/test/BinaryMorphologicalClosingImageFilterTest.cpp:

TEST_CASE("ImageProcessing::BinaryMorphologicalClosingImageFilter: Computed-expected (safeBorder=false; both algorithm paths)",
          "[ImageProcessing][BinaryMorphologicalClosingImageFilter]")

Command that should run it (in-core Release build of develop, macOS arm64):

ctest --test-dir "<build-dir>" -R "^ImageProcessing::BinaryMorphologicalClosingImageFilter: Computed-expected \(safeBorder=false; both algorithm paths\)$" -j 1 --output-on-failure --timeout 300 --no-tests=error

Observed:

No tests were found!!!
Errors while running CTest

And the fragment that ctest does run:

203: Test command: .../Bin/ImageProcessingUnitTest " both algorithm paths)"
203: No test cases matched 'both algorithm paths)'
1/1 Test #203:  both algorithm paths) ...........   Passed    0.02 sec

Proposed Fix

  1. Replace the ; in the seven test names with , (for example
    Computed-expected (safeBorder=false, both algorithm paths)). Run the seven tests afterwards: they have not run
    under ctest, so they may fail.
  2. Keep it from coming back: add a rule to docs/Code_Style_Guide.md (Testing) that a test case name contains no
    ;, and optionally a CMake check after discovery that fails when a registered name does not match a test case.

Notes

Activity

  1. added
    bugSomething isn't working
    TestingDealing with bug hunting in software
    on Oct 30, 2026
  2. imikejackson commented on Oct 30, 2026

    @imikejackson
    Contributor

    @joeykleingers Run a sweep over all unit tests to make sure NONE of them have a semi-colon. Then update the PR that has the Code-Rules to update that document also.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    TestingDealing with bug hunting in softwarebugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions