diff --git a/src/Plugins/SimplnxCore/docs/ErodeDilateBadDataFilter.md b/src/Plugins/SimplnxCore/docs/ErodeDilateBadDataFilter.md index 581a02a135..94ed551e41 100644 --- a/src/Plugins/SimplnxCore/docs/ErodeDilateBadDataFilter.md +++ b/src/Plugins/SimplnxCore/docs/ErodeDilateBadDataFilter.md @@ -8,6 +8,8 @@ Processing (Cleanup) This **Filter** grows or shrinks regions of *bad data* (cells with *Feature Id = 0*) by one cell-layer per iteration, using standard image-morphology operations. It is a building block for cleaning up isolated noise or for compensating for boundaries that were systematically under- or over-measured by the experiment. +**Cell Feature Ids** and every array in **Attribute Arrays to Ignore** must be in the selected geometry's Cell Attribute Matrix. + ![Fig. 1: Dilate grows the bad-data (Feature Id 0) region outward by one cell layer per iteration; Erode shrinks it, reassigning each bad cell to its majority neighbor. The Number of Iterations sets the layer count.](Images/ErodeDilateBadData_IterationSequence.png) ### Dilation @@ -47,8 +49,11 @@ slice. ### Preflight Errors -The Filter refuses to run in two cases: +The Filter refuses to run in four cases: +- **-14605**: *Cell Feature Ids* is not in the selected **Image Geometry**'s Cell Attribute Matrix, or the geometry has no + Cell Attribute Matrix. +- **-14604**: an array in *Attribute Arrays to Ignore* is not in the selected **Image Geometry**'s Cell Attribute Matrix. - **-14601**: all three of *X Direction*, *Y Direction*, and *Z Direction* are disabled. At least one direction is required, otherwise there are no neighbors to erode or dilate across. - **-14602**: the selected **Image Geometry** has a dimension of *0* **Cells**. All three dimensions must be non-zero. diff --git a/src/Plugins/SimplnxCore/docs/ErodeDilateCoordinationNumberFilter.md b/src/Plugins/SimplnxCore/docs/ErodeDilateCoordinationNumberFilter.md index d6ff9c42f5..bd769bed73 100644 --- a/src/Plugins/SimplnxCore/docs/ErodeDilateCoordinationNumberFilter.md +++ b/src/Plugins/SimplnxCore/docs/ErodeDilateCoordinationNumberFilter.md @@ -8,6 +8,8 @@ Processing (Cleanup) This **Filter** smooths the interface between *good* and *bad* cells (or, more generally, between any two adjacent feature regions) by removing cells whose neighborhood is mostly the opposite class. It targets isolated voxels and small protrusions that are surrounded by cells of a different feature, then fills them in with their neighbors' data via isotropic coarsening. +**Cell Feature Ids** and every array in **Attribute Arrays to Ignore** must be in the selected geometry's Cell Attribute Matrix. + The *coordination number* is the count of a cell's face-sharing neighbors that belong to a different class. With 6 face neighbors, the coordination number ranges from **0 to 6**: - **CN = 0** -- all 6 neighbors agree with the cell. Solidly inside a region. diff --git a/src/Plugins/SimplnxCore/docs/FillBadDataFilter.md b/src/Plugins/SimplnxCore/docs/FillBadDataFilter.md index 6a10682d15..1b45eaaab8 100644 --- a/src/Plugins/SimplnxCore/docs/FillBadDataFilter.md +++ b/src/Plugins/SimplnxCore/docs/FillBadDataFilter.md @@ -8,6 +8,8 @@ Processing (Cleanup) This **Filter** distinguishes small *bad-data noise* from large *real defects* in a segmented dataset and fills only the noise. **Cells** with *Feature Id = 0* are considered *bad*; the filter groups them into connected components, classifies each component by its size, and fills components below the user threshold by copying the most common neighbor's data into them. Components at or above the threshold are left intact, since they likely represent real features (pores, cracks, voids). +**Cell Feature Ids** and every array in **Attribute Arrays to Ignore** must be in the selected geometry's Cell Attribute Matrix. **Cell Phases** must also be in that Attribute Matrix when **Store Defects as New Phase** is enabled. + | Small IN100 Before | Small IN100 After | |--------------------------------------|-------------------------------------| | ![](Images/fill_bad_data_before.png) | ![](Images/fill_bad_data_after.png) | diff --git a/src/Plugins/SimplnxCore/docs/KeepRemoveRankedFeaturesFilter.md b/src/Plugins/SimplnxCore/docs/KeepRemoveRankedFeaturesFilter.md index 51174855d0..4f2592b399 100644 --- a/src/Plugins/SimplnxCore/docs/KeepRemoveRankedFeaturesFilter.md +++ b/src/Plugins/SimplnxCore/docs/KeepRemoveRankedFeaturesFilter.md @@ -6,6 +6,8 @@ This **Filter** keeps or removes a count or percentage of **Features** based on a chosen scalar **Feature** level array. For example, "keep the 10 largest grains by *NumElements*", or "remove the smallest 10% of grains". +Cell Feature Ids must be in the selected Image Geometry's Cell Attribute Matrix; when Fill-in Removed Features is on, Attribute Arrays to Ignore must also be in that matrix. The parent of Feature Ranking Array must be a different Attribute Matrix directly under the selected Image Geometry. + This is a **rank** threshold, not a **value** threshold. It answers *"which are the 10 biggest?"* rather than *"which are bigger than 100 voxels?"*. The cut therefore depends on the whole population rather than on each **Feature** independently. For a value threshold, use **Remove Minimum Size diff --git a/src/Plugins/SimplnxCore/docs/QuickSurfaceMeshFilter.md b/src/Plugins/SimplnxCore/docs/QuickSurfaceMeshFilter.md index 52eb72952c..0475eb7694 100644 --- a/src/Plugins/SimplnxCore/docs/QuickSurfaceMeshFilter.md +++ b/src/Plugins/SimplnxCore/docs/QuickSurfaceMeshFilter.md @@ -12,6 +12,8 @@ Surface Meshing (Generation) This **Filter** generates a **Triangle Geometry** surface mesh from a grid **Geometry** (Image or Rectilinear Grid). For every cell face shared between two different **Features** (i.e., a Feature boundary), the filter emits two triangles, producing the *stair-stepped* surface that defines the boundary of each Feature. +**Cell Feature Ids** must have one tuple per cell of the selected Image or Rectilinear Grid Geometry. When **Attempt to Fix Problem Voxels** is on, **Cell Feature Ids** must also be in that geometry's Cell Attribute Matrix because the filter modifies this array in place. When this option is off, **Cell Feature Ids** may be in any Attribute Matrix. + The resulting mesh is "blocky" because each triangle aligns with a voxel face. To smooth it, apply [Laplacian Smoothing](LaplacianSmoothingFilter.md) afterward -- or, preferably, use [Surface Nets](SurfaceNetsFilter.md), which combines meshing and smoothing in one step. ![Example Quick Mesh Output](Images/QuickSurface_Output.png) diff --git a/src/Plugins/SimplnxCore/docs/RemoveFlaggedFeaturesFilter.md b/src/Plugins/SimplnxCore/docs/RemoveFlaggedFeaturesFilter.md index aae6dd62df..b044d5b5f8 100644 --- a/src/Plugins/SimplnxCore/docs/RemoveFlaggedFeaturesFilter.md +++ b/src/Plugins/SimplnxCore/docs/RemoveFlaggedFeaturesFilter.md @@ -8,6 +8,8 @@ Processing (Cleanup) This **Filter** removes or extracts **Features** that an earlier filter in the pipeline has flagged. The user supplies a boolean (or unsigned 8-bit) array at the **Feature** level. Features with a value of *true* are removed or extracted. This filter discards features by any criterion that a flag can express: biased features, surface features, minority phases, or user-selected features. For the specific case of removing features smaller than a size threshold, see [Remove Minimum Size Features](RequireMinimumSizeFeaturesFilter.md). For a rank-based selection ("remove the 10 smallest"), see [Keep/Remove Ranked Features](KeepRemoveRankedFeaturesFilter.md). +For all operations, Cell Feature Ids must be in the selected Image Geometry's Cell Attribute Matrix; when removing with Fill-in Removed Features on, Attribute Arrays to Ignore must also be in that matrix. The parent of Flagged Features must be a different Attribute Matrix directly under the selected Image Geometry. + ### Selected Operation The *Selected Operation* parameter provides the following choices: diff --git a/src/Plugins/SimplnxCore/docs/RequireMinNumNeighborsFilter.md b/src/Plugins/SimplnxCore/docs/RequireMinNumNeighborsFilter.md index 313653ca68..6cfc99a8b4 100644 --- a/src/Plugins/SimplnxCore/docs/RequireMinNumNeighborsFilter.md +++ b/src/Plugins/SimplnxCore/docs/RequireMinNumNeighborsFilter.md @@ -8,6 +8,8 @@ Processing (Cleanup) This **Filter** removes **Features** that have fewer contiguous neighboring **Features** than the selected minimum. It uses a precomputed *Number of Neighbors* array, which is normally created by the [Compute Feature Neighbors](./ComputeFeatureNeighborsFilter.md) **Filter**. **Feature** tuple 0 is retained as the background tuple. +Cell Feature Ids and Cell Arrays to Ignore must be in the selected Image Geometry's Cell Attribute Matrix. The parent of Number of Neighbors must be a different Attribute Matrix directly under the selected Image Geometry. + The most common use case is cleaning up isolated single-feature islands left over from segmentation -- features whose surroundings turned out to be too sparsely connected to support a meaningful grain. After flagged features are removed, the remaining features grow outward via [isotropic coarsening](RequireMinimumSizeFeaturesFilter.md) until every cell is reassigned. The threshold is a **count of contiguous neighbors** (a non-negative integer). Setting the threshold to 0 removes nothing. Setting it larger than the maximum number of neighbors any feature has produces an error (since all features would be removed). Inspect the *Number of Neighbors* output of [Compute Feature Neighbors](ComputeFeatureNeighborsFilter.md) before choosing a threshold. @@ -44,7 +46,10 @@ This **Filter** operates in place and does not create a new geometry or output a ## Validation and Failure Conditions +- Preflight fails with `-55575` when *Cell Feature Ids* is not in the selected **Image Geometry**'s Cell Attribute Matrix. - Preflight fails with `-55571` when *Cell Feature Ids* does not contain exactly one tuple per **Image Geometry** cell. +- Preflight fails with `-55573` when an array in *Cell Arrays to Ignore* is not in that Cell Attribute Matrix. +- Preflight fails with `-55574` when the parent of *Number of Neighbors* is not an Attribute Matrix directly under the selected **Image Geometry**, or is its Cell Attribute Matrix. - Preflight fails with `-252` when *Number of Neighbors* and the enabled *Feature Phases* array do not have matching tuple counts. - Execution fails with `-5555` when the selected *Phase Index* is not present in *Feature Phases*. - Execution fails with `-55569` when the selected minimum would reject every eligible non-background **Feature**. diff --git a/src/Plugins/SimplnxCore/docs/RequireMinimumSizeFeaturesFilter.md b/src/Plugins/SimplnxCore/docs/RequireMinimumSizeFeaturesFilter.md index c56751733a..d459e998d3 100644 --- a/src/Plugins/SimplnxCore/docs/RequireMinimumSizeFeaturesFilter.md +++ b/src/Plugins/SimplnxCore/docs/RequireMinimumSizeFeaturesFilter.md @@ -8,6 +8,8 @@ Processing (Cleanup) This **Filter** removes small **Features** from a segmented dataset and fills in the space they leave behind by expanding their neighbors. Features whose cell count is below the user-specified *Minimum Allowed Features Size* are erased; the remaining features grow outward into the erased cells until every cell is reassigned to a surviving feature. +Cell Feature Ids must be in the selected Image Geometry's Cell Attribute Matrix. The parent of Feature Num. Cells must be a different Attribute Matrix directly under the selected Image Geometry. + This is typically used immediately after segmentation ([Segment Features (Misorientation)](../OrientationAnalysis/EBSDSegmentFeaturesFilter.md) or similar) to discard spurious single-cell or few-cell "grains" that are almost always noise rather than real features. ### How This Filter Works diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/ErodeDilateBadDataFilter.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/ErodeDilateBadDataFilter.cpp index 7e58266462..3d0dc6f8a5 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/ErodeDilateBadDataFilter.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/ErodeDilateBadDataFilter.cpp @@ -11,6 +11,7 @@ #include "simplnx/Parameters/DataGroupSelectionParameter.hpp" #include "simplnx/Parameters/GeometrySelectionParameter.hpp" #include "simplnx/Parameters/MultiArraySelectionParameter.hpp" +#include "simplnx/Utilities/DataArrayUtilities.hpp" #include "simplnx/Utilities/SIMPLConversion.hpp" @@ -100,6 +101,22 @@ IFilter::PreflightResult ErodeDilateBadDataFilter::preflightImpl(const DataStruc auto zDirOn = filterArgs.value(k_ZDirOn_Key); auto imageGeometryPath = filterArgs.value(k_SelectedImageGeometryPath_Key); + auto featureIdsResult = ValidateArrayInGeometryCellData(dataStructure, imageGeometryPath, pFeatureIdsArrayPathValue, -14605, "Cell Feature Ids"); + if(featureIdsResult.invalid()) + { + return {ConvertResultTo(std::move(featureIdsResult), {})}; + } + + auto ignoredArraysResult = ValidateArraysInGeometryCellData(dataStructure, imageGeometryPath, pIgnoredDataArrayPathsValue, -14604, "Attribute Arrays to Ignore"); + if(ignoredArraysResult.invalid()) + { + return {ConvertResultTo(std::move(ignoredArraysResult), {})}; + } + + // The Cell Attribute Matrix can have more than one DataPath. Use the geometry's Cell Attribute Matrix path so that the ignored paths match the modified paths. + const auto cellDataPath = dataStructure.getDataRefAs(imageGeometryPath).getCellDataPath(); + pIgnoredDataArrayPathsValue = MapToGeometryCellDataPath(cellDataPath, pIgnoredDataArrayPathsValue); + PreflightResult preflightResult; nx::core::Result resultOutputActions; @@ -125,7 +142,7 @@ IFilter::PreflightResult ErodeDilateBadDataFilter::preflightImpl(const DataStruc // Report arrays that the filter modifies in place. // Cell Data arrays are modified. - nx::core::AppendDataObjectModifications(dataStructure, resultOutputActions.value().modifiedActions, pFeatureIdsArrayPathValue.getParent(), pIgnoredDataArrayPathsValue); + nx::core::AppendDataObjectModifications(dataStructure, resultOutputActions.value().modifiedActions, cellDataPath, pIgnoredDataArrayPathsValue); // Return actions and updated preflight values. return {std::move(resultOutputActions), std::move(preflightUpdatedValues)}; @@ -145,6 +162,11 @@ Result<> ErodeDilateBadDataFilter::executeImpl(DataStructure& dataStructure, con inputValues.IgnoredDataArrayPaths = filterArgs.value(k_IgnoredDataArrayPaths_Key); inputValues.InputImageGeometry = filterArgs.value(k_SelectedImageGeometryPath_Key); + // The algorithm compares DataPaths and finds the Cell arrays through the parent of Feature Ids. Use the geometry's Cell Attribute Matrix path for Feature Ids and each ignored array. + const auto cellDataPath = dataStructure.getDataRefAs(inputValues.InputImageGeometry).getCellDataPath(); + inputValues.FeatureIdsArrayPath = MapToGeometryCellDataPath(cellDataPath, inputValues.FeatureIdsArrayPath); + inputValues.IgnoredDataArrayPaths = MapToGeometryCellDataPath(cellDataPath, inputValues.IgnoredDataArrayPaths); + return ErodeDilateBadData(dataStructure, messageHandler, shouldCancel, &inputValues)(); } diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/ErodeDilateCoordinationNumberFilter.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/ErodeDilateCoordinationNumberFilter.cpp index 77c1d50dae..0f77175934 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/ErodeDilateCoordinationNumberFilter.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/ErodeDilateCoordinationNumberFilter.cpp @@ -3,6 +3,7 @@ #include "SimplnxCore/Filters/Algorithms/ErodeDilateCoordinationNumber.hpp" #include "simplnx/DataStructure/DataPath.hpp" +#include "simplnx/DataStructure/Geometry/ImageGeom.hpp" #include "simplnx/Filter/Actions/EmptyAction.hpp" #include "simplnx/Parameters/ArraySelectionParameter.hpp" #include "simplnx/Parameters/BoolParameter.hpp" @@ -10,6 +11,7 @@ #include "simplnx/Parameters/GeometrySelectionParameter.hpp" #include "simplnx/Parameters/MultiArraySelectionParameter.hpp" #include "simplnx/Parameters/NumberParameter.hpp" +#include "simplnx/Utilities/DataArrayUtilities.hpp" #include "simplnx/Utilities/FilterUtilities.hpp" #include "simplnx/Utilities/SIMPLConversion.hpp" @@ -89,6 +91,23 @@ IFilter::PreflightResult ErodeDilateCoordinationNumberFilter::preflightImpl(cons auto pCoordinationNumberValue = filterArgs.value(k_CoordinationNumber_Key); auto pFeatureIdsArrayPathValue = filterArgs.value(k_CellFeatureIdsArrayPath_Key); auto pIgnoredDataArrayPathsValue = filterArgs.value(k_IgnoredDataArrayPaths_Key); + auto imageGeometryPath = filterArgs.value(k_SelectedImageGeometryPath_Key); + + auto featureIdsResult = ValidateArrayInGeometryCellData(dataStructure, imageGeometryPath, pFeatureIdsArrayPathValue, -16801, "Cell Feature Ids"); + if(featureIdsResult.invalid()) + { + return {ConvertResultTo(std::move(featureIdsResult), {})}; + } + + auto ignoredArraysResult = ValidateArraysInGeometryCellData(dataStructure, imageGeometryPath, pIgnoredDataArrayPathsValue, -16802, "Attribute Arrays to Ignore"); + if(ignoredArraysResult.invalid()) + { + return {ConvertResultTo(std::move(ignoredArraysResult), {})}; + } + + // The Cell Attribute Matrix can have more than one DataPath. Use the geometry's Cell Attribute Matrix path so that the ignored paths match the modified paths. + const auto cellDataPath = dataStructure.getDataRefAs(imageGeometryPath).getCellDataPath(); + pIgnoredDataArrayPathsValue = MapToGeometryCellDataPath(cellDataPath, pIgnoredDataArrayPathsValue); if(pCoordinationNumberValue < 0 || pCoordinationNumberValue > 6) { @@ -100,7 +119,7 @@ IFilter::PreflightResult ErodeDilateCoordinationNumberFilter::preflightImpl(cons // Inform users that the following arrays are going to be modified in place // Cell Data is going to be modified - nx::core::AppendDataObjectModifications(dataStructure, resultOutputActions.value().modifiedActions, pFeatureIdsArrayPathValue.getParent(), pIgnoredDataArrayPathsValue); + nx::core::AppendDataObjectModifications(dataStructure, resultOutputActions.value().modifiedActions, cellDataPath, pIgnoredDataArrayPathsValue); // Return both the resultOutputActions and the preflightUpdatedValues via std::move() return {std::move(resultOutputActions), std::move(preflightUpdatedValues)}; @@ -118,6 +137,11 @@ Result<> ErodeDilateCoordinationNumberFilter::executeImpl(DataStructure& dataStr inputValues.IgnoredDataArrayPaths = filterArgs.value(k_IgnoredDataArrayPaths_Key); inputValues.InputImageGeometry = filterArgs.value(k_SelectedImageGeometryPath_Key); + // The algorithm compares DataPaths and finds the Cell arrays through the parent of Feature Ids. Use the geometry's Cell Attribute Matrix path for Feature Ids and each ignored array. + const auto cellDataPath = dataStructure.getDataRefAs(inputValues.InputImageGeometry).getCellDataPath(); + inputValues.FeatureIdsArrayPath = MapToGeometryCellDataPath(cellDataPath, inputValues.FeatureIdsArrayPath); + inputValues.IgnoredDataArrayPaths = MapToGeometryCellDataPath(cellDataPath, inputValues.IgnoredDataArrayPaths); + return ErodeDilateCoordinationNumber(dataStructure, messageHandler, shouldCancel, &inputValues)(); } diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/FillBadDataFilter.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/FillBadDataFilter.cpp index 99cd1f927b..4ab5dce948 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/FillBadDataFilter.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/FillBadDataFilter.cpp @@ -3,6 +3,7 @@ #include "SimplnxCore/Filters/Algorithms/FillBadData.hpp" #include "simplnx/DataStructure/DataPath.hpp" +#include "simplnx/DataStructure/Geometry/ImageGeom.hpp" #include "simplnx/Parameters/ArraySelectionParameter.hpp" #include "simplnx/Parameters/AttributeMatrixSelectionParameter.hpp" #include "simplnx/Parameters/BoolParameter.hpp" @@ -10,6 +11,7 @@ #include "simplnx/Parameters/GeometrySelectionParameter.hpp" #include "simplnx/Parameters/MultiArraySelectionParameter.hpp" #include "simplnx/Parameters/NumberParameter.hpp" +#include "simplnx/Utilities/DataArrayUtilities.hpp" #include "simplnx/Utilities/FilterUtilities.hpp" #include "simplnx/Utilities/SIMPLConversion.hpp" @@ -82,6 +84,36 @@ IFilter::PreflightResult FillBadDataFilter::preflightImpl(const DataStructure& d const ExecutionContext& executionContext) const { auto pMinAllowedDefectSizeValue = filterArgs.value(k_MinAllowedDefectSize_Key); + auto imageGeometryPath = filterArgs.value(k_SelectedImageGeometryPath_Key); + auto featureIdsPath = filterArgs.value(k_CellFeatureIdsArrayPath_Key); + auto ignoredDataArrayPaths = filterArgs.value(k_IgnoredDataArrayPaths_Key); + auto storeAsNewPhase = filterArgs.value(k_StoreAsNewPhase_Key); + auto cellPhasesPath = filterArgs.value(k_CellPhasesArrayPath_Key); + + auto featureIdsResult = ValidateArrayInGeometryCellData(dataStructure, imageGeometryPath, featureIdsPath, -16501, "Cell Feature Ids"); + if(featureIdsResult.invalid()) + { + return {ConvertResultTo(std::move(featureIdsResult), {})}; + } + + if(storeAsNewPhase) + { + auto phasesResult = ValidateArrayInGeometryCellData(dataStructure, imageGeometryPath, cellPhasesPath, -16502, "Cell Phases"); + if(phasesResult.invalid()) + { + return {ConvertResultTo(std::move(phasesResult), {})}; + } + } + + auto ignoredArraysResult = ValidateArraysInGeometryCellData(dataStructure, imageGeometryPath, ignoredDataArrayPaths, -16503, "Attribute Arrays to Ignore"); + if(ignoredArraysResult.invalid()) + { + return {ConvertResultTo(std::move(ignoredArraysResult), {})}; + } + + // The Cell Attribute Matrix can have more than one DataPath. Use the geometry's Cell Attribute Matrix path so that the ignored paths match the modified paths. + const auto cellDataPath = dataStructure.getDataRefAs(imageGeometryPath).getCellDataPath(); + ignoredDataArrayPaths = MapToGeometryCellDataPath(cellDataPath, ignoredDataArrayPaths); if(pMinAllowedDefectSizeValue < 1) { @@ -99,9 +131,7 @@ IFilter::PreflightResult FillBadDataFilter::preflightImpl(const DataStructure& d // Report arrays that the filter modifies in place. // Cell Data arrays are modified. - auto featureIdsPath = filterArgs.value(k_CellFeatureIdsArrayPath_Key); - auto ignoredDataArrayPaths = filterArgs.value(k_IgnoredDataArrayPaths_Key); - AppendDataObjectModifications(dataStructure, resultOutputActions.value().modifiedActions, featureIdsPath.getParent(), ignoredDataArrayPaths); + AppendDataObjectModifications(dataStructure, resultOutputActions.value().modifiedActions, cellDataPath, ignoredDataArrayPaths); return {std::move(resultOutputActions), std::move(preflightUpdatedValues)}; } @@ -118,6 +148,12 @@ Result<> FillBadDataFilter::executeImpl(DataStructure& dataStructure, const Argu inputValues.ignoredDataArrayPaths = filterArgs.value(k_IgnoredDataArrayPaths_Key); inputValues.inputImageGeometry = filterArgs.value(k_SelectedImageGeometryPath_Key); + // Both algorithms discover Cell arrays through the geometry's Cell Attribute Matrix path. + // Normalize Feature Ids and ignored paths so their comparisons match those discovered paths. + const auto cellDataPath = dataStructure.getDataRefAs(inputValues.inputImageGeometry).getCellDataPath(); + inputValues.featureIdsArrayPath = MapToGeometryCellDataPath(cellDataPath, inputValues.featureIdsArrayPath); + inputValues.ignoredDataArrayPaths = MapToGeometryCellDataPath(cellDataPath, inputValues.ignoredDataArrayPaths); + return FillBadData(dataStructure, messageHandler, shouldCancel, &inputValues)(); } diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/KeepRemoveRankedFeaturesFilter.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/KeepRemoveRankedFeaturesFilter.cpp index 280992b99f..c5a7cf5a5a 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/KeepRemoveRankedFeaturesFilter.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/KeepRemoveRankedFeaturesFilter.cpp @@ -7,12 +7,14 @@ #include "simplnx/DataStructure/AttributeMatrix.hpp" #include "simplnx/DataStructure/DataArray.hpp" #include "simplnx/DataStructure/DataPath.hpp" +#include "simplnx/DataStructure/Geometry/ImageGeom.hpp" #include "simplnx/Parameters/ArraySelectionParameter.hpp" #include "simplnx/Parameters/BoolParameter.hpp" #include "simplnx/Parameters/ChoicesParameter.hpp" #include "simplnx/Parameters/GeometrySelectionParameter.hpp" #include "simplnx/Parameters/MultiArraySelectionParameter.hpp" #include "simplnx/Parameters/NumberParameter.hpp" +#include "simplnx/Utilities/DataArrayUtilities.hpp" #include "simplnx/Utilities/DataGroupUtilities.hpp" #include "simplnx/Utilities/FilterUtilities.hpp" @@ -24,7 +26,6 @@ using namespace nx::core; namespace { -constexpr int32 k_ParentNotAttributeMatrix = -78100; constexpr int32 k_NotEnoughFeatures = -78101; constexpr int32 k_PercentOutOfRange = -78102; constexpr int32 k_ZeroNumFeatures = -78103; @@ -129,18 +130,40 @@ IFilter::PreflightResult KeepRemoveRankedFeaturesFilter::preflightImpl(const Dat auto pCriterionValue = filterArgs.value(k_Criterion_Key); auto pNumFeaturesValue = filterArgs.value(k_NumFeatures_Key); auto pPercentValue = filterArgs.value(k_Percent_Key); + auto imageGeometryPath = filterArgs.value(k_SelectedImageGeometryPath_Key); + auto fillRemovedFeatures = filterArgs.value(k_FillRemovedFeatures_Key); + auto ignoredDataArrayPaths = filterArgs.value(k_IgnoredDataArrayPaths_Key); - nx::core::Result resultOutputActions; + auto featureIdsResult = ValidateArrayInGeometryCellData(dataStructure, imageGeometryPath, pFeatureIdsArrayPathValue, -78125, "Cell Feature Ids"); + if(featureIdsResult.invalid()) + { + return {ConvertResultTo(std::move(featureIdsResult), {})}; + } - const DataPath featureAttributeMatrixPath = pRankingArrayPathValue.getParent(); - const auto* featureAmPtr = dataStructure.getDataAs(featureAttributeMatrixPath); - if(featureAmPtr == nullptr) + if(fillRemovedFeatures) + { + auto ignoredArraysResult = ValidateArraysInGeometryCellData(dataStructure, imageGeometryPath, ignoredDataArrayPaths, -78126, "Attribute Arrays to Ignore"); + if(ignoredArraysResult.invalid()) + { + return {ConvertResultTo(std::move(ignoredArraysResult), {})}; + } + } + + auto featureAttributeMatrixResult = + ValidateFeatureAttributeMatrixInGeometry(dataStructure, imageGeometryPath, pRankingArrayPathValue.getParent(), -78127, "Feature Ranking Array", pRankingArrayPathValue); + if(featureAttributeMatrixResult.invalid()) { - return {MakeErrorResult(k_ParentNotAttributeMatrix, - fmt::format("The parent of the Feature Ranking Array at path '{}' must be an Attribute Matrix, but no Attribute Matrix was found at '{}'.", - pRankingArrayPathValue.toString(), featureAttributeMatrixPath.toString()))}; + return {ConvertResultTo(std::move(featureAttributeMatrixResult), {})}; } + // The Cell Attribute Matrix can have more than one DataPath. Use the geometry's Cell Attribute Matrix path for Feature Ids. + const auto cellDataPath = dataStructure.getDataRefAs(imageGeometryPath).getCellDataPath(); + pFeatureIdsArrayPathValue = MapToGeometryCellDataPath(cellDataPath, pFeatureIdsArrayPathValue); + + nx::core::Result resultOutputActions; + + const DataPath featureAttributeMatrixPath = pRankingArrayPathValue.getParent(); + const auto* featureAmPtr = dataStructure.getDataAs(featureAttributeMatrixPath); const auto& rankingArray = dataStructure.getDataRefAs(pRankingArrayPathValue); const usize numTuples = rankingArray.getNumberOfTuples(); if(numTuples < 2) @@ -221,7 +244,7 @@ IFilter::PreflightResult KeepRemoveRankedFeaturesFilter::preflightImpl(const Dat } } - nx::core::AppendDataObjectModifications(dataStructure, resultOutputActions.value().modifiedActions, pFeatureIdsArrayPathValue.getParent(), {}); + nx::core::AppendDataObjectModifications(dataStructure, resultOutputActions.value().modifiedActions, cellDataPath, {}); nx::core::AppendDataObjectModifications(dataStructure, resultOutputActions.value().modifiedActions, pRankingArrayPathValue.getParent(), {}); std::vector preflightUpdatedValues; @@ -256,6 +279,11 @@ Result<> KeepRemoveRankedFeaturesFilter::executeImpl(DataStructure& dataStructur inputValues.RankingArrayPath = filterArgs.value(k_RankingArrayPath_Key); inputValues.IgnoredDataArrayPaths = filterArgs.value(k_IgnoredDataArrayPaths_Key); + // The algorithm compares DataPaths and finds the Cell arrays through the parent of Feature Ids. Use the geometry's Cell Attribute Matrix path for Feature Ids and each ignored array. + const auto cellDataPath = dataStructure.getDataRefAs(inputValues.ImageGeometryPath).getCellDataPath(); + inputValues.FeatureIdsArrayPath = MapToGeometryCellDataPath(cellDataPath, inputValues.FeatureIdsArrayPath); + inputValues.IgnoredDataArrayPaths = MapToGeometryCellDataPath(cellDataPath, inputValues.IgnoredDataArrayPaths); + return KeepRemoveRankedFeatures(dataStructure, messageHandler, shouldCancel, &inputValues)(); } } // namespace nx::core diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/QuickSurfaceMeshFilter.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/QuickSurfaceMeshFilter.cpp index 063891fa3b..0aa6ad230e 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/QuickSurfaceMeshFilter.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/QuickSurfaceMeshFilter.cpp @@ -16,6 +16,7 @@ #include "simplnx/Parameters/DataObjectNameParameter.hpp" #include "simplnx/Parameters/GeometrySelectionParameter.hpp" #include "simplnx/Parameters/MultiArraySelectionParameter.hpp" +#include "simplnx/Utilities/DataArrayUtilities.hpp" #include "simplnx/Utilities/Meshing/TriangleUtilities.hpp" #include "simplnx/Utilities/SIMPLConversion.hpp" @@ -141,6 +142,24 @@ IFilter::PreflightResult QuickSurfaceMeshFilter::preflightImpl(const DataStructu auto pFaceLabelsName = filterArgs.value(k_FaceLabelsArrayName_Key); auto pFeatureDataPaths = filterArgs.value>(k_SelectedFeatureDataArrayPaths_Key); + auto fixProblemVoxels = filterArgs.value(k_FixProblemVoxels_Key); + + const auto& gridGeom = dataStructure.getDataRefAs(pGridGeomDataPath); + const auto& featureIdsRef = dataStructure.getDataRefAs(pFeatureIdsArrayPathValue); + if(featureIdsRef.getNumberOfTuples() != gridGeom.getNumberOfCells()) + { + return MakePreflightErrorResult(-76533, + fmt::format("**Cell Feature Ids** DataArray '{}' has {} tuples, but the selected grid geometry '{}' has {} cells. Select a DataArray with one tuple per cell.", + pFeatureIdsArrayPathValue.toString(), featureIdsRef.getNumberOfTuples(), pGridGeomDataPath.toString(), gridGeom.getNumberOfCells())); + } + if(fixProblemVoxels) + { + auto featureIdsResult = ValidateArrayInGeometryCellData(dataStructure, pGridGeomDataPath, pFeatureIdsArrayPathValue, -76532, "Cell Feature Ids"); + if(featureIdsResult.invalid()) + { + return {ConvertResultTo(std::move(featureIdsResult), {})}; + } + } DataPath pVertexGroupDataPath = pTriangleGeometryPath.createChildPath(pVertexGroupDataName); DataPath pFaceGroupDataPath = pTriangleGeometryPath.createChildPath(pFaceGroupDataName); @@ -148,7 +167,11 @@ IFilter::PreflightResult QuickSurfaceMeshFilter::preflightImpl(const DataStructu nx::core::Result resultOutputActions; std::vector preflightUpdatedValues; - const auto& gridGeom = dataStructure.getDataRefAs(pGridGeomDataPath); + if(fixProblemVoxels) + { + const auto featureIdsPath = MapToGeometryCellDataPath(gridGeom.getCellDataPath(), pFeatureIdsArrayPathValue); + resultOutputActions.value().appendDataObjectModificationNotification(featureIdsPath, DataObjectModification::ModifiedType::Modified); + } const usize elementTupleCount = gridGeom.getCellData()->getNumberOfTuples(); constexpr usize numElements = 0; diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/RemoveFlaggedFeaturesFilter.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/RemoveFlaggedFeaturesFilter.cpp index bcee99e486..f4994c16cb 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/RemoveFlaggedFeaturesFilter.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/RemoveFlaggedFeaturesFilter.cpp @@ -5,6 +5,7 @@ #include "simplnx/DataStructure/AttributeMatrix.hpp" #include "simplnx/DataStructure/DataArray.hpp" #include "simplnx/DataStructure/DataPath.hpp" +#include "simplnx/DataStructure/Geometry/ImageGeom.hpp" #include "simplnx/Parameters/ArraySelectionParameter.hpp" #include "simplnx/Parameters/BoolParameter.hpp" #include "simplnx/Parameters/ChoicesParameter.hpp" @@ -12,6 +13,7 @@ #include "simplnx/Parameters/GeometrySelectionParameter.hpp" #include "simplnx/Parameters/MultiArraySelectionParameter.hpp" #include "simplnx/Parameters/StringParameter.hpp" +#include "simplnx/Utilities/DataArrayUtilities.hpp" #include "simplnx/Utilities/FilterUtilities.hpp" #include "simplnx/Utilities/SIMPLConversion.hpp" @@ -117,30 +119,50 @@ IFilter::PreflightResult RemoveFlaggedFeaturesFilter::preflightImpl(const DataSt auto pFeatureIdsArrayPathValue = filterArgs.value(k_CellFeatureIdsArrayPath_Key); auto pFlaggedFeaturesArrayPathValue = filterArgs.value(k_FlaggedFeaturesArrayPath_Key); auto operationType = filterArgs.value(k_Functionality_Key); + auto imageGeometryPath = filterArgs.value(k_SelectedImageGeometryPath_Key); + auto fillRemovedFeatures = filterArgs.value(k_FillRemovedFeatures_Key); + auto ignoredDataArrayPaths = filterArgs.value(k_IgnoredDataArrayPaths_Key); - nx::core::Result resultOutputActions; + auto featureIdsResult = ValidateArrayInGeometryCellData(dataStructure, imageGeometryPath, pFeatureIdsArrayPathValue, -9893, "Cell Feature Ids"); + if(featureIdsResult.invalid()) + { + return {ConvertResultTo(std::move(featureIdsResult), {})}; + } - const auto& featureIdsArray = dataStructure.getDataRefAs(pFeatureIdsArrayPathValue); + if((operationType == to_underlying(Functionality::Remove) || operationType == to_underlying(Functionality::ExtractThenRemove)) && fillRemovedFeatures) + { + auto ignoredArraysResult = ValidateArraysInGeometryCellData(dataStructure, imageGeometryPath, ignoredDataArrayPaths, -9894, "Attribute Arrays to Ignore"); + if(ignoredArraysResult.invalid()) + { + return {ConvertResultTo(std::move(ignoredArraysResult), {})}; + } + } - DataPath const cellFeatureAttributeMatrixPath = pFlaggedFeaturesArrayPathValue.getParent(); - const auto* cellFeatureAmPtr = dataStructure.getDataAs(cellFeatureAttributeMatrixPath); - if(cellFeatureAmPtr == nullptr) + auto featureAttributeMatrixResult = + ValidateFeatureAttributeMatrixInGeometry(dataStructure, imageGeometryPath, pFlaggedFeaturesArrayPathValue.getParent(), -9895, "Flagged Features", pFlaggedFeaturesArrayPathValue); + if(featureAttributeMatrixResult.invalid()) { - return {MakeErrorResult( - -9892, fmt::format("Could not find the parent Attribute Matrix for the selected Flagged Features Data Array at path '{}'", pFlaggedFeaturesArrayPathValue.toString()))}; + return {ConvertResultTo(std::move(featureAttributeMatrixResult), {})}; } + // The Cell Attribute Matrix can have more than one DataPath. Use the geometry's Cell Attribute Matrix path for Feature Ids. + const auto cellDataPath = dataStructure.getDataRefAs(imageGeometryPath).getCellDataPath(); + pFeatureIdsArrayPathValue = MapToGeometryCellDataPath(cellDataPath, pFeatureIdsArrayPathValue); + + nx::core::Result resultOutputActions; + + DataPath const cellFeatureAttributeMatrixPath = pFlaggedFeaturesArrayPathValue.getParent(); std::vector preflightUpdatedValues; // If we are in any way removing features, inform the user - if(operationType == 0 || operationType == 2) + if(operationType == to_underlying(Functionality::Remove) || operationType == to_underlying(Functionality::ExtractThenRemove)) { // Warn about stale IDataArrays in the Feature AM; NeighborListRemovalPreflightCode handles NeighborList deletion for Remove/ExtractThenRemove below nx::core::AppendRenumberedFeatureAMWarnings(dataStructure, cellFeatureAttributeMatrixPath, pFeatureIdsArrayPathValue, preflightUpdatedValues, false); // Inform users that the following arrays are going to be modified in place // Cell Data is going to be modified - nx::core::AppendDataObjectModifications(dataStructure, resultOutputActions.value().modifiedActions, pFeatureIdsArrayPathValue.getParent(), {}); + nx::core::AppendDataObjectModifications(dataStructure, resultOutputActions.value().modifiedActions, cellDataPath, {}); // Feature Data is going to be modified nx::core::AppendDataObjectModifications(dataStructure, resultOutputActions.value().modifiedActions, pFlaggedFeaturesArrayPathValue.getParent(), {}); @@ -170,6 +192,11 @@ Result<> RemoveFlaggedFeaturesFilter::executeImpl(DataStructure& dataStructure, inputValues.CreatedImageGeometryPrefix = filterArgs.value(k_CreatedImageGeometryPrefix_Key); inputValues.TempBoundsPath = inputValues.FlaggedFeaturesArrayPath.replaceName(k_boundsName); + // The algorithm compares DataPaths and finds the Cell arrays through the parent of Feature Ids. Use the geometry's Cell Attribute Matrix path for Feature Ids and each ignored array. + const auto cellDataPath = dataStructure.getDataRefAs(inputValues.ImageGeometryPath).getCellDataPath(); + inputValues.FeatureIdsArrayPath = MapToGeometryCellDataPath(cellDataPath, inputValues.FeatureIdsArrayPath); + inputValues.IgnoredDataArrayPaths = MapToGeometryCellDataPath(cellDataPath, inputValues.IgnoredDataArrayPaths); + return RemoveFlaggedFeatures(dataStructure, messageHandler, shouldCancel, &inputValues)(); } diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/RequireMinNumNeighborsFilter.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/RequireMinNumNeighborsFilter.cpp index e61996a2ca..0d24b593e9 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/RequireMinNumNeighborsFilter.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/RequireMinNumNeighborsFilter.cpp @@ -11,6 +11,7 @@ #include "simplnx/Parameters/GeometrySelectionParameter.hpp" #include "simplnx/Parameters/MultiArraySelectionParameter.hpp" #include "simplnx/Parameters/NumberParameter.hpp" +#include "simplnx/Utilities/DataArrayUtilities.hpp" #include "simplnx/Utilities/FilterUtilities.hpp" #include "simplnx/Utilities/SIMPLConversion.hpp" @@ -101,15 +102,20 @@ IFilter::PreflightResult RequireMinNumNeighborsFilter::preflightImpl(const DataS auto featurePhasesPath = filterArgs.value(k_FeaturePhasesPath_Key); auto numNeighborsPath = filterArgs.value(k_NumNeighborsPath_Key); auto minNumNeighbors = filterArgs.value(k_MinNumNeighbors_Key); + auto ignoredDataArrayPaths = filterArgs.value(k_IgnoredVoxelArrays_Key); - nx::core::Result resultOutputActions; - - std::vector preflightUpdatedValues; + auto featureIdsResult = ValidateArrayInGeometryCellData(dataStructure, imageGeomPath, featureIdsPath, -55575, "Cell Feature Ids"); + if(featureIdsResult.invalid()) + { + return {ConvertResultTo(std::move(featureIdsResult), {})}; + } - std::vector dataArrayPaths; + // The Cell Attribute Matrix can have more than one DataPath. Use the geometry's Cell Attribute Matrix path for Feature Ids. + const auto cellDataPath = dataStructure.getDataRefAs(imageGeomPath).getCellDataPath(); + featureIdsPath = MapToGeometryCellDataPath(cellDataPath, featureIdsPath); - auto& featureIds = dataStructure.getDataRefAs(featureIdsPath); - auto& imageGeom = dataStructure.getDataRefAs(imageGeomPath); + const auto& featureIds = dataStructure.getDataRefAs(featureIdsPath); + const auto& imageGeom = dataStructure.getDataRefAs(imageGeomPath); if(featureIds.getNumberOfTuples() != imageGeom.getNumberOfCells()) { return MakePreflightErrorResult(k_FeatureIdsTupleCountMismatch, @@ -117,6 +123,24 @@ IFilter::PreflightResult RequireMinNumNeighborsFilter::preflightImpl(const DataS featureIdsPath.toString(), featureIds.getNumberOfTuples(), imageGeomPath.toString(), imageGeom.getNumberOfCells())); } + auto ignoredArraysResult = ValidateArraysInGeometryCellData(dataStructure, imageGeomPath, ignoredDataArrayPaths, -55573, "Cell Arrays to Ignore"); + if(ignoredArraysResult.invalid()) + { + return {ConvertResultTo(std::move(ignoredArraysResult), {})}; + } + + auto featureAttributeMatrixResult = ValidateFeatureAttributeMatrixInGeometry(dataStructure, imageGeomPath, numNeighborsPath.getParent(), -55574, "Number of Neighbors", numNeighborsPath); + if(featureAttributeMatrixResult.invalid()) + { + return {ConvertResultTo(std::move(featureAttributeMatrixResult), {})}; + } + + nx::core::Result resultOutputActions; + + std::vector preflightUpdatedValues; + + std::vector dataArrayPaths; + dataArrayPaths.push_back(numNeighborsPath); if(applyToSinglePhase) @@ -132,7 +156,7 @@ IFilter::PreflightResult RequireMinNumNeighborsFilter::preflightImpl(const DataS // Report arrays that the filter modifies in place. // Cell Data arrays are modified. - nx::core::AppendDataObjectModifications(dataStructure, resultOutputActions.value().modifiedActions, featureIdsPath.getParent(), {}); + nx::core::AppendDataObjectModifications(dataStructure, resultOutputActions.value().modifiedActions, cellDataPath, {}); // Feature Data arrays are modified. nx::core::AppendDataObjectModifications(dataStructure, resultOutputActions.value().modifiedActions, numNeighborsPath.getParent(), {}); @@ -164,6 +188,11 @@ Result<> RequireMinNumNeighborsFilter::executeImpl(DataStructure& dataStructure, inputValues.NumNeighborsPath = filterArgs.value(k_NumNeighborsPath_Key); inputValues.IgnoredVoxelArrayPaths = filterArgs.value(k_IgnoredVoxelArrays_Key); + // The algorithm compares DataPaths and finds the Cell arrays through the parent of Feature Ids. Use the geometry's Cell Attribute Matrix path for Feature Ids and each ignored array. + const auto cellDataPath = dataStructure.getDataRefAs(inputValues.ImageGeomPath).getCellDataPath(); + inputValues.FeatureIdsPath = MapToGeometryCellDataPath(cellDataPath, inputValues.FeatureIdsPath); + inputValues.IgnoredVoxelArrayPaths = MapToGeometryCellDataPath(cellDataPath, inputValues.IgnoredVoxelArrayPaths); + return RequireMinNumNeighbors(dataStructure, messageHandler, shouldCancel, &inputValues)(); } diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/RequireMinimumSizeFeaturesFilter.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/RequireMinimumSizeFeaturesFilter.cpp index aed21a8e1f..7b3ac38284 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/RequireMinimumSizeFeaturesFilter.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/RequireMinimumSizeFeaturesFilter.cpp @@ -10,6 +10,7 @@ #include "simplnx/Parameters/DataGroupSelectionParameter.hpp" #include "simplnx/Parameters/GeometrySelectionParameter.hpp" #include "simplnx/Parameters/NumberParameter.hpp" +#include "simplnx/Utilities/DataArrayUtilities.hpp" #include "simplnx/Utilities/FilterUtilities.hpp" #include "simplnx/Utilities/SIMPLConversion.hpp" @@ -26,7 +27,6 @@ using PhasesArrayType = Int32Array; namespace { constexpr int32 k_BadMinAllowedFeatureSize = -5555; -constexpr int32 k_ParentlessPathError = -5557; } // namespace @@ -105,6 +105,22 @@ IFilter::PreflightResult RequireMinimumSizeFeaturesFilter::preflightImpl(const D auto applyToSinglePhase = filterArgs.value(k_ApplySinglePhase_Key); auto minAllowedFeatureSize = filterArgs.value(k_MinAllowedFeaturesSize_Key); + auto featureIdsResult = ValidateArrayInGeometryCellData(dataStructure, imageGeomPath, featureIdsPath, -5562, "Cell Feature Ids"); + if(featureIdsResult.invalid()) + { + return {ConvertResultTo(std::move(featureIdsResult), {})}; + } + + auto featureAttributeMatrixResult = ValidateFeatureAttributeMatrixInGeometry(dataStructure, imageGeomPath, featureNumCellsPath.getParent(), -5563, "Feature Num. Cells", featureNumCellsPath); + if(featureAttributeMatrixResult.invalid()) + { + return {ConvertResultTo(std::move(featureAttributeMatrixResult), {})}; + } + + // The Cell Attribute Matrix can have more than one DataPath. Use the geometry's Cell Attribute Matrix path for Feature Ids. + const auto cellDataPath = dataStructure.getDataRefAs(imageGeomPath).getCellDataPath(); + featureIdsPath = MapToGeometryCellDataPath(cellDataPath, featureIdsPath); + std::vector dataArrayPaths; if(minAllowedFeatureSize < 0) @@ -130,13 +146,6 @@ IFilter::PreflightResult RequireMinimumSizeFeaturesFilter::preflightImpl(const D return MakePreflightErrorResult(-2071, fmt::format("The following DataArrays all must have equal number of tuples but this was not satisfied.\n{}", tupleValidityCheck.error())); } - DataPath featureGroupDataPath = featureNumCellsPath.getParent(); - const auto* featureDataGroup = dataStructure.getDataAs(featureGroupDataPath); - if(nullptr == featureDataGroup) - { - return {MakeErrorResult(k_ParentlessPathError, "The provided NumCells DataPath does not have a parent.")}; - } - // Create the preflightResult object nx::core::Result resultOutputActions; std::vector preflightUpdatedValues; @@ -153,7 +162,7 @@ IFilter::PreflightResult RequireMinimumSizeFeaturesFilter::preflightImpl(const D // Inform users that the following arrays are going to be modified in place // Cell Data is going to be modified - nx::core::AppendDataObjectModifications(dataStructure, resultOutputActions.value().modifiedActions, featureIdsPath.getParent(), {}); + nx::core::AppendDataObjectModifications(dataStructure, resultOutputActions.value().modifiedActions, cellDataPath, {}); // Return both the resultOutputActions and the preflightUpdatedValues via std::move() return {std::move(resultOutputActions), std::move(preflightUpdatedValues)}; @@ -173,6 +182,10 @@ Result<> RequireMinimumSizeFeaturesFilter::executeImpl(DataStructure& dataStruct inputValues.ApplySinglePhase = filterArgs.value(k_ApplySinglePhase_Key); inputValues.PhaseNumber = filterArgs.value(k_SinglePhaseNumber_Key); + // The algorithm finds the Cell arrays through the parent of Feature Ids. Use the geometry's Cell Attribute Matrix path for Feature Ids. + const auto cellDataPath = dataStructure.getDataRefAs(inputValues.InputImageGeometryPath).getCellDataPath(); + inputValues.FeatureIdsPath = MapToGeometryCellDataPath(cellDataPath, inputValues.FeatureIdsPath); + return RequireMinimumSizeFeatures(dataStructure, messageHandler, shouldCancel, &inputValues)(); } diff --git a/src/Plugins/SimplnxCore/test/ErodeDilateBadDataTest.cpp b/src/Plugins/SimplnxCore/test/ErodeDilateBadDataTest.cpp index f401d5ea54..63aa6cf7d9 100644 --- a/src/Plugins/SimplnxCore/test/ErodeDilateBadDataTest.cpp +++ b/src/Plugins/SimplnxCore/test/ErodeDilateBadDataTest.cpp @@ -1,10 +1,14 @@ -#include "SimplnxCore/SimplnxCore_test_dirs.hpp" -#include - #include "SimplnxCore/Filters/Algorithms/ErodeDilateBadData.hpp" #include "SimplnxCore/Filters/ErodeDilateBadDataFilter.hpp" +#include "SimplnxCore/SimplnxCore_test_dirs.hpp" + +#include #include "simplnx/Core/Application.hpp" +#include "simplnx/DataStructure/AttributeMatrix.hpp" +#include "simplnx/DataStructure/DataGroup.hpp" +#include "simplnx/DataStructure/DataStore.hpp" +#include "simplnx/DataStructure/Geometry/ImageGeom.hpp" #include "simplnx/Parameters/BoolParameter.hpp" #include "simplnx/Parameters/ChoicesParameter.hpp" #include "simplnx/Parameters/MultiArraySelectionParameter.hpp" @@ -193,6 +197,7 @@ DataStructure CreateTestData() geom->setDimensions(k_GeometryDimensions); auto* cellData = AttributeMatrix::Create(dataStructure, ::k_CellData, k_TupleShape, geom->getId()); + geom->setCellData(*cellData); // Feature IDs auto featureIdsPtr = std::make_shared(k_TupleShape, ShapeType{1}, 0); @@ -584,6 +589,117 @@ TEST_CASE("SimplnxCore::ErodeDilateBadDataFilter No Dimensions", "[SimplnxCore][ UnitTest::CheckArraysInheritTupleDims(dataStructure); } +TEST_CASE("SimplnxCore::ErodeDilateBadDataFilter: Strict Feature Ids", "[SimplnxCore][ErodeDilateBadDataFilter][StrictFeatureIds]") +{ + UnitTest::LoadPlugins(); + DataStructure dataStructure; + const DataPath geometryPath({"Strict Geometry"}); + const DataPath cellPath = geometryPath.createChildPath("Cell Data"); + const DataPath featureIdsPath = cellPath.createChildPath("Feature Ids"); + const DataPath otherPath = geometryPath.createChildPath("Other Data").createChildPath("Other Values"); + const ShapeType shape{1, 2, 2}; + auto* imageGeomPtr = ImageGeom::Create(dataStructure, geometryPath.getTargetName()); + REQUIRE(imageGeomPtr != nullptr); + imageGeomPtr->setDimensions({2, 2, 1}); + auto* cellsPtr = AttributeMatrix::Create(dataStructure, "Cell Data", shape, imageGeomPtr->getId()); + REQUIRE(cellsPtr != nullptr); + imageGeomPtr->setCellData(*cellsPtr); + REQUIRE(AttributeMatrix::Create(dataStructure, "Feature Data", ShapeType{3}, imageGeomPtr->getId()) != nullptr); + auto* otherPtr = AttributeMatrix::Create(dataStructure, "Other Data", shape, imageGeomPtr->getId()); + REQUIRE(otherPtr != nullptr); + REQUIRE(Int32Array::Create(dataStructure, "Feature Ids", std::make_shared>(shape, ShapeType{1}, 1), cellsPtr->getId()) != nullptr); + REQUIRE(Int32Array::Create(dataStructure, "Other Values", std::make_shared>(shape, ShapeType{1}, 1), otherPtr->getId()) != nullptr); + + ErodeDilateBadDataFilter filter; + auto args = filter.getDefaultArguments(); + args.insertOrAssign(ErodeDilateBadDataFilter::k_SelectedImageGeometryPath_Key, std::make_any(geometryPath)); + args.insertOrAssign(ErodeDilateBadDataFilter::k_CellFeatureIdsArrayPath_Key, std::make_any(featureIdsPath)); + + int32 expectedCode = 0; + SECTION("Correct Cell Attribute Matrix") + { + args.insertOrAssign(ErodeDilateBadDataFilter::k_IgnoredDataArrayPaths_Key, std::make_any>(std::vector{featureIdsPath})); + } + SECTION("Feature Ids in another Attribute Matrix with the same tuple shape") + { + args.insertOrAssign(ErodeDilateBadDataFilter::k_CellFeatureIdsArrayPath_Key, std::make_any(otherPath)); + expectedCode = -14605; + } + SECTION("Ignored array in another Attribute Matrix") + { + args.insertOrAssign(ErodeDilateBadDataFilter::k_IgnoredDataArrayPaths_Key, std::make_any>(std::vector{otherPath})); + expectedCode = -14604; + } + const auto result = filter.preflight(dataStructure, args); + if(expectedCode == 0) + { + REQUIRE(result.outputActions.valid()); + } + else + { + REQUIRE(result.outputActions.invalid()); + REQUIRE(result.outputActions.errors().size() == 1); + CHECK(result.outputActions.errors().front().code == expectedCode); + CHECK(result.outputActions.errors().front().message.find(otherPath.toString()) != std::string::npos); + CHECK(result.outputActions.errors().front().message.find(cellPath.toString()) != std::string::npos); + CHECK(result.outputActions.errors().front().message.find(geometryPath.toString()) != std::string::npos); + } + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + +TEST_CASE("SimplnxCore::ErodeDilateBadDataFilter: Ignored Array Cell AM Alias", "[SimplnxCore][ErodeDilateBadDataFilter][StrictFeatureIds]") +{ + UnitTest::LoadPlugins(); + DataStructure dataStructure; + const DataPath geometryPath({"Alias Test Geometry"}); + const DataPath cellPath = geometryPath.createChildPath("Cells"); + const DataPath ignoredPath = cellPath.createChildPath("Ignored"); + const DataPath aliasCellPath({"Alternate Parent", "Cells"}); + const ShapeType shape{1, 1, 4}; + auto* imageGeomPtr = ImageGeom::Create(dataStructure, geometryPath.getTargetName()); + REQUIRE(imageGeomPtr != nullptr); + imageGeomPtr->setDimensions({4, 1, 1}); + auto* cellsPtr = AttributeMatrix::Create(dataStructure, "Cells", shape, imageGeomPtr->getId()); + REQUIRE(cellsPtr != nullptr); + imageGeomPtr->setCellData(*cellsPtr); + auto* alternateParentPtr = DataGroup::Create(dataStructure, "Alternate Parent"); + REQUIRE(alternateParentPtr != nullptr); + REQUIRE(dataStructure.setAdditionalParent(cellsPtr->getId(), alternateParentPtr->getId())); + REQUIRE(imageGeomPtr->getCellDataPath() == cellPath); + auto* idsPtr = Int32Array::Create(dataStructure, "Ids", std::make_shared>(shape, ShapeType{1}, 0), cellsPtr->getId()); + auto* ignoredPtr = Int32Array::Create(dataStructure, "Ignored", std::make_shared>(shape, ShapeType{1}, 0), cellsPtr->getId()); + REQUIRE(idsPtr != nullptr); + REQUIRE(ignoredPtr != nullptr); + const std::vector originalIds{1, 0, 0, 1}; + for(usize index = 0; index < 4; index++) + { + (*idsPtr)[index] = originalIds[index]; + (*ignoredPtr)[index] = static_cast(10 * (index + 1)); + } + ErodeDilateBadDataFilter filter; + auto args = filter.getDefaultArguments(); + args.insertOrAssign(ErodeDilateBadDataFilter::k_SelectedImageGeometryPath_Key, std::make_any(geometryPath)); + args.insertOrAssign(ErodeDilateBadDataFilter::k_CellFeatureIdsArrayPath_Key, std::make_any(aliasCellPath.createChildPath("Ids"))); + args.insertOrAssign(ErodeDilateBadDataFilter::k_IgnoredDataArrayPaths_Key, std::make_any>(std::vector{aliasCellPath.createChildPath("Ignored")})); + args.insertOrAssign(ErodeDilateBadDataFilter::k_NumIterations_Key, std::make_any(1)); + const auto preflight = filter.preflight(dataStructure, args); + REQUIRE(preflight.outputActions.valid()); + for(const auto& modification : preflight.outputActions.value().modifiedActions) + { + CHECK(modification.modifiedPath != ignoredPath); + } + const auto execute = filter.execute(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(execute.result); + bool idsChanged = false; + for(usize index = 0; index < 4; index++) + { + CHECK((*ignoredPtr)[index] == static_cast(10 * (index + 1))); + idsChanged = idsChanged || (*idsPtr)[index] != originalIds[index]; + } + CHECK(idsChanged); + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + TEST_CASE("SimplnxCore::ErodeDilateBadDataFilter: SIMPL Backwards Compatibility", "[SimplnxCore][ErodeDilateBadDataFilter][BackwardsCompatibility]") { auto app = Application::GetOrCreateInstance(); diff --git a/src/Plugins/SimplnxCore/test/ErodeDilateCoordinationNumberTest.cpp b/src/Plugins/SimplnxCore/test/ErodeDilateCoordinationNumberTest.cpp index 9a6e0cbb78..e862bed20b 100644 --- a/src/Plugins/SimplnxCore/test/ErodeDilateCoordinationNumberTest.cpp +++ b/src/Plugins/SimplnxCore/test/ErodeDilateCoordinationNumberTest.cpp @@ -1,14 +1,14 @@ -#include +#include "SimplnxCore/Filters/ErodeDilateCoordinationNumberFilter.hpp" +#include "SimplnxCore/SimplnxCore_test_dirs.hpp" +#include #include #include -#include "SimplnxCore/Filters/ErodeDilateCoordinationNumberFilter.hpp" -#include "SimplnxCore/SimplnxCore_test_dirs.hpp" - #include "simplnx/Core/Application.hpp" #include "simplnx/DataStructure/AttributeMatrix.hpp" #include "simplnx/DataStructure/DataArray.hpp" +#include "simplnx/DataStructure/DataGroup.hpp" #include "simplnx/DataStructure/DataStore.hpp" #include "simplnx/DataStructure/Geometry/ImageGeom.hpp" #include "simplnx/Parameters/ArraySelectionParameter.hpp" @@ -765,6 +765,118 @@ TEST_CASE("SimplnxCore::ErodeDilateCoordinationNumberFilter: Bulk Storage Failur } } +TEST_CASE("SimplnxCore::ErodeDilateCoordinationNumberFilter: Strict Feature Ids", "[SimplnxCore][ErodeDilateCoordinationNumberFilter][StrictFeatureIds]") +{ + UnitTest::LoadPlugins(); + DataStructure dataStructure; + const DataPath geometryPath({"Strict Geometry"}); + const DataPath cellPath = geometryPath.createChildPath("Cell Data"); + const DataPath featureIdsPath = cellPath.createChildPath("Feature Ids"); + const DataPath otherPath = geometryPath.createChildPath("Other Data").createChildPath("Other Values"); + const ShapeType shape{1, 2, 2}; + auto* imageGeomPtr = ImageGeom::Create(dataStructure, geometryPath.getTargetName()); + REQUIRE(imageGeomPtr != nullptr); + imageGeomPtr->setDimensions({2, 2, 1}); + auto* cellsPtr = AttributeMatrix::Create(dataStructure, "Cell Data", shape, imageGeomPtr->getId()); + REQUIRE(cellsPtr != nullptr); + imageGeomPtr->setCellData(*cellsPtr); + REQUIRE(AttributeMatrix::Create(dataStructure, "Feature Data", ShapeType{3}, imageGeomPtr->getId()) != nullptr); + auto* otherPtr = AttributeMatrix::Create(dataStructure, "Other Data", shape, imageGeomPtr->getId()); + REQUIRE(otherPtr != nullptr); + REQUIRE(Int32Array::Create(dataStructure, "Feature Ids", std::make_shared>(shape, ShapeType{1}, 1), cellsPtr->getId()) != nullptr); + REQUIRE(Int32Array::Create(dataStructure, "Other Values", std::make_shared>(shape, ShapeType{1}, 1), otherPtr->getId()) != nullptr); + + ErodeDilateCoordinationNumberFilter filter; + auto args = filter.getDefaultArguments(); + args.insertOrAssign(ErodeDilateCoordinationNumberFilter::k_SelectedImageGeometryPath_Key, std::make_any(geometryPath)); + args.insertOrAssign(ErodeDilateCoordinationNumberFilter::k_CellFeatureIdsArrayPath_Key, std::make_any(featureIdsPath)); + + int32 expectedCode = 0; + SECTION("Correct Cell Attribute Matrix") + { + args.insertOrAssign(ErodeDilateCoordinationNumberFilter::k_IgnoredDataArrayPaths_Key, std::make_any>(std::vector{featureIdsPath})); + } + SECTION("Feature Ids in another Attribute Matrix with the same tuple shape") + { + args.insertOrAssign(ErodeDilateCoordinationNumberFilter::k_CellFeatureIdsArrayPath_Key, std::make_any(otherPath)); + expectedCode = -16801; + } + SECTION("Ignored array in another Attribute Matrix") + { + args.insertOrAssign(ErodeDilateCoordinationNumberFilter::k_IgnoredDataArrayPaths_Key, std::make_any>(std::vector{otherPath})); + expectedCode = -16802; + } + const auto result = filter.preflight(dataStructure, args); + if(expectedCode == 0) + { + REQUIRE(result.outputActions.valid()); + } + else + { + REQUIRE(result.outputActions.invalid()); + REQUIRE(result.outputActions.errors().size() == 1); + CHECK(result.outputActions.errors().front().code == expectedCode); + CHECK(result.outputActions.errors().front().message.find(otherPath.toString()) != std::string::npos); + CHECK(result.outputActions.errors().front().message.find(cellPath.toString()) != std::string::npos); + CHECK(result.outputActions.errors().front().message.find(geometryPath.toString()) != std::string::npos); + } + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + +TEST_CASE("SimplnxCore::ErodeDilateCoordinationNumberFilter: Ignored Array Cell AM Alias", "[SimplnxCore][ErodeDilateCoordinationNumberFilter][StrictFeatureIds]") +{ + UnitTest::LoadPlugins(); + DataStructure dataStructure; + const DataPath geometryPath({"Alias Test Geometry"}); + const DataPath cellPath = geometryPath.createChildPath("Cells"); + const DataPath ignoredPath = cellPath.createChildPath("Ignored"); + const DataPath aliasCellPath({"Alternate Parent", "Cells"}); + const ShapeType shape{1, 1, 4}; + auto* imageGeomPtr = ImageGeom::Create(dataStructure, geometryPath.getTargetName()); + REQUIRE(imageGeomPtr != nullptr); + imageGeomPtr->setDimensions({4, 1, 1}); + auto* cellsPtr = AttributeMatrix::Create(dataStructure, "Cells", shape, imageGeomPtr->getId()); + REQUIRE(cellsPtr != nullptr); + imageGeomPtr->setCellData(*cellsPtr); + auto* alternateParentPtr = DataGroup::Create(dataStructure, "Alternate Parent"); + REQUIRE(alternateParentPtr != nullptr); + REQUIRE(dataStructure.setAdditionalParent(cellsPtr->getId(), alternateParentPtr->getId())); + REQUIRE(imageGeomPtr->getCellDataPath() == cellPath); + auto* idsPtr = Int32Array::Create(dataStructure, "Ids", std::make_shared>(shape, ShapeType{1}, 0), cellsPtr->getId()); + auto* ignoredPtr = Int32Array::Create(dataStructure, "Ignored", std::make_shared>(shape, ShapeType{1}, 0), cellsPtr->getId()); + REQUIRE(idsPtr != nullptr); + REQUIRE(ignoredPtr != nullptr); + const std::vector originalIds{1, 0, 0, 1}; + for(usize index = 0; index < 4; index++) + { + (*idsPtr)[index] = originalIds[index]; + (*ignoredPtr)[index] = static_cast(10 * (index + 1)); + } + ErodeDilateCoordinationNumberFilter filter; + auto args = filter.getDefaultArguments(); + args.insertOrAssign(ErodeDilateCoordinationNumberFilter::k_SelectedImageGeometryPath_Key, std::make_any(geometryPath)); + args.insertOrAssign(ErodeDilateCoordinationNumberFilter::k_CellFeatureIdsArrayPath_Key, std::make_any(aliasCellPath.createChildPath("Ids"))); + args.insertOrAssign(ErodeDilateCoordinationNumberFilter::k_IgnoredDataArrayPaths_Key, std::make_any>(std::vector{aliasCellPath.createChildPath("Ignored")})); + args.insertOrAssign(ErodeDilateCoordinationNumberFilter::k_CoordinationNumber_Key, std::make_any(1)); + args.insertOrAssign(ErodeDilateCoordinationNumberFilter::k_Loop_Key, std::make_any(false)); + const auto preflight = filter.preflight(dataStructure, args); + REQUIRE(preflight.outputActions.valid()); + for(const auto& modification : preflight.outputActions.value().modifiedActions) + { + CHECK(modification.modifiedPath != ignoredPath); + } + const auto execute = filter.execute(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(execute.result); + bool idsChanged = false; + for(usize index = 0; index < 4; index++) + { + CHECK((*ignoredPtr)[index] == static_cast(10 * (index + 1))); + idsChanged = idsChanged || (*idsPtr)[index] != originalIds[index]; + } + CHECK(idsChanged); + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + TEST_CASE("SimplnxCore::ErodeDilateCoordinationNumberFilter: SIMPL Backwards Compatibility", "[SimplnxCore][ErodeDilateCoordinationNumberFilter][BackwardsCompatibility]") { auto app = Application::GetOrCreateInstance(); diff --git a/src/Plugins/SimplnxCore/test/FillBadDataTest.cpp b/src/Plugins/SimplnxCore/test/FillBadDataTest.cpp index adbcb03f7b..488550083b 100644 --- a/src/Plugins/SimplnxCore/test/FillBadDataTest.cpp +++ b/src/Plugins/SimplnxCore/test/FillBadDataTest.cpp @@ -1,9 +1,14 @@ +#include "SimplnxCore/Filters/FillBadDataFilter.hpp" +#include "SimplnxCore/SimplnxCore_test_dirs.hpp" + #include #include #include "simplnx/Core/Application.hpp" #include "simplnx/DataStructure/AttributeMatrix.hpp" #include "simplnx/DataStructure/DataArray.hpp" +#include "simplnx/DataStructure/DataGroup.hpp" +#include "simplnx/DataStructure/DataStore.hpp" #include "simplnx/DataStructure/Geometry/ImageGeom.hpp" #include "simplnx/Parameters/MultiArraySelectionParameter.hpp" #include "simplnx/Parameters/NumberParameter.hpp" @@ -14,11 +19,9 @@ #include "simplnx/Utilities/AlgorithmDispatch.hpp" #include "simplnx/Utilities/DataStoreUtilities.hpp" -#include "SimplnxCore/Filters/FillBadDataFilter.hpp" -#include "SimplnxCore/SimplnxCore_test_dirs.hpp" - #include #include + namespace fs = std::filesystem; using namespace nx::core; @@ -546,6 +549,7 @@ TEST_CASE("SimplnxCore::FillBadDataFilter:: Invalid Preflight Min Defect Size", auto* imageGeomPtr = ImageGeom::Create(dataStructure, "DataContainer"); imageGeomPtr->setDimensions({1, 1, 1}); auto* cellDataPtr = AttributeMatrix::Create(dataStructure, "CellData", {1, 1, 1}, imageGeomPtr->getId()); + imageGeomPtr->setCellData(*cellDataPtr); UnitTest::CreateTestDataArray(dataStructure, "FeatureIds", {1, 1, 1}, {1}, cellDataPtr->getId()); FillBadDataFilter filter; @@ -882,6 +886,139 @@ TEST_CASE("SimplnxCore::FillBadData: 200x200x200 Ignored Arrays", "[Core][FillBa UnitTest::CheckArraysInheritTupleDims(dataStructure); } +TEST_CASE("SimplnxCore::FillBadDataFilter: Strict Feature Ids", "[SimplnxCore][FillBadDataFilter][StrictFeatureIds]") +{ + UnitTest::LoadPlugins(); + DataStructure dataStructure; + const DataPath geometryPath({"Strict Geometry"}); + const DataPath cellPath = geometryPath.createChildPath("Cell Data"); + const DataPath featureIdsPath = cellPath.createChildPath("Feature Ids"); + const DataPath otherPath = geometryPath.createChildPath("Other Data").createChildPath("Other Values"); + const ShapeType shape{1, 2, 2}; + auto* imageGeomPtr = ImageGeom::Create(dataStructure, geometryPath.getTargetName()); + REQUIRE(imageGeomPtr != nullptr); + imageGeomPtr->setDimensions({2, 2, 1}); + auto* cellsPtr = AttributeMatrix::Create(dataStructure, "Cell Data", shape, imageGeomPtr->getId()); + REQUIRE(cellsPtr != nullptr); + imageGeomPtr->setCellData(*cellsPtr); + REQUIRE(AttributeMatrix::Create(dataStructure, "Feature Data", ShapeType{3}, imageGeomPtr->getId()) != nullptr); + auto* otherPtr = AttributeMatrix::Create(dataStructure, "Other Data", shape, imageGeomPtr->getId()); + REQUIRE(otherPtr != nullptr); + REQUIRE(Int32Array::Create(dataStructure, "Feature Ids", std::make_shared>(shape, ShapeType{1}, 1), cellsPtr->getId()) != nullptr); + REQUIRE(Int32Array::Create(dataStructure, "Other Values", std::make_shared>(shape, ShapeType{1}, 1), otherPtr->getId()) != nullptr); + + FillBadDataFilter filter; + auto args = filter.getDefaultArguments(); + args.insertOrAssign(FillBadDataFilter::k_SelectedImageGeometryPath_Key, std::make_any(geometryPath)); + args.insertOrAssign(FillBadDataFilter::k_CellFeatureIdsArrayPath_Key, std::make_any(featureIdsPath)); + args.insertOrAssign(FillBadDataFilter::k_StoreAsNewPhase_Key, std::make_any(false)); + + int32 expectedCode = 0; + SECTION("Correct Cell Attribute Matrix") + { + args.insertOrAssign(FillBadDataFilter::k_IgnoredDataArrayPaths_Key, std::make_any>(std::vector{featureIdsPath})); + } + SECTION("Feature Ids in another Attribute Matrix with the same tuple shape") + { + args.insertOrAssign(FillBadDataFilter::k_CellFeatureIdsArrayPath_Key, std::make_any(otherPath)); + expectedCode = -16501; + } + SECTION("Ignored array in another Attribute Matrix") + { + args.insertOrAssign(FillBadDataFilter::k_IgnoredDataArrayPaths_Key, std::make_any>(std::vector{otherPath})); + expectedCode = -16503; + } + SECTION("Modified Cell Phases in another Attribute Matrix") + { + args.insertOrAssign(FillBadDataFilter::k_StoreAsNewPhase_Key, std::make_any(true)); + args.insertOrAssign(FillBadDataFilter::k_CellPhasesArrayPath_Key, std::make_any(otherPath)); + expectedCode = -16502; + } + SECTION("Unused Cell Phases in another Attribute Matrix") + { + args.insertOrAssign(FillBadDataFilter::k_CellPhasesArrayPath_Key, std::make_any(otherPath)); + } + SECTION("Modified Cell Phases in the Cell Attribute Matrix") + { + const auto phasesPath = cellPath.createChildPath("Phases"); + REQUIRE(Int32Array::Create(dataStructure, "Phases", std::make_shared>(shape, ShapeType{1}, 1), cellsPtr->getId()) != nullptr); + args.insertOrAssign(FillBadDataFilter::k_StoreAsNewPhase_Key, std::make_any(true)); + args.insertOrAssign(FillBadDataFilter::k_CellPhasesArrayPath_Key, std::make_any(phasesPath)); + } + const auto result = filter.preflight(dataStructure, args); + if(expectedCode == 0) + { + REQUIRE(result.outputActions.valid()); + } + else + { + REQUIRE(result.outputActions.invalid()); + REQUIRE(result.outputActions.errors().size() == 1); + CHECK(result.outputActions.errors().front().code == expectedCode); + CHECK(result.outputActions.errors().front().message.find(otherPath.toString()) != std::string::npos); + CHECK(result.outputActions.errors().front().message.find(cellPath.toString()) != std::string::npos); + CHECK(result.outputActions.errors().front().message.find(geometryPath.toString()) != std::string::npos); + } + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + +TEST_CASE("SimplnxCore::FillBadDataFilter: Ignored Array Cell AM Alias", "[SimplnxCore][FillBadDataFilter][StrictFeatureIds]") +{ + UnitTest::LoadPlugins(); + const auto scenario = GENERATE(from_range(UnitTest::SelectAlgorithmTestScenariosForInMemoryStores())); + CAPTURE(scenario); + UnitTest::AlgorithmTestScope scope(scenario); + DataStructure dataStructure; + const DataPath geometryPath({"Alias Test Geometry"}); + const DataPath cellPath = geometryPath.createChildPath("Cells"); + const DataPath ignoredPath = cellPath.createChildPath("Ignored"); + const DataPath aliasCellPath({"Alternate Parent", "Cells"}); + const ShapeType shape{1, 1, 4}; + auto* imageGeomPtr = ImageGeom::Create(dataStructure, geometryPath.getTargetName()); + REQUIRE(imageGeomPtr != nullptr); + imageGeomPtr->setDimensions({4, 1, 1}); + auto* cellsPtr = AttributeMatrix::Create(dataStructure, "Cells", shape, imageGeomPtr->getId()); + REQUIRE(cellsPtr != nullptr); + imageGeomPtr->setCellData(*cellsPtr); + auto* alternateParentPtr = DataGroup::Create(dataStructure, "Alternate Parent"); + REQUIRE(alternateParentPtr != nullptr); + REQUIRE(dataStructure.setAdditionalParent(cellsPtr->getId(), alternateParentPtr->getId())); + REQUIRE(imageGeomPtr->getCellDataPath() == cellPath); + auto* idsPtr = Int32Array::Create(dataStructure, "Ids", std::make_shared>(shape, ShapeType{1}, 0), cellsPtr->getId()); + auto* ignoredPtr = Int32Array::Create(dataStructure, "Ignored", std::make_shared>(shape, ShapeType{1}, 0), cellsPtr->getId()); + REQUIRE(idsPtr != nullptr); + REQUIRE(ignoredPtr != nullptr); + const std::vector originalIds{1, 0, 0, 1}; + for(usize index = 0; index < 4; index++) + { + (*idsPtr)[index] = originalIds[index]; + (*ignoredPtr)[index] = static_cast(10 * (index + 1)); + } + FillBadDataFilter filter; + auto args = filter.getDefaultArguments(); + args.insertOrAssign(FillBadDataFilter::k_SelectedImageGeometryPath_Key, std::make_any(geometryPath)); + args.insertOrAssign(FillBadDataFilter::k_CellFeatureIdsArrayPath_Key, std::make_any(aliasCellPath.createChildPath("Ids"))); + args.insertOrAssign(FillBadDataFilter::k_IgnoredDataArrayPaths_Key, std::make_any>(std::vector{aliasCellPath.createChildPath("Ignored")})); + args.insertOrAssign(FillBadDataFilter::k_StoreAsNewPhase_Key, std::make_any(false)); + args.insertOrAssign(FillBadDataFilter::k_MinAllowedDefectSize_Key, std::make_any(10)); + const auto preflight = filter.preflight(dataStructure, args); + REQUIRE(preflight.outputActions.valid()); + for(const auto& modification : preflight.outputActions.value().modifiedActions) + { + CHECK(modification.modifiedPath != ignoredPath); + } + const auto execute = scope.executeFilter(filter, dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(execute.result); + bool idsChanged = false; + for(usize index = 0; index < 4; index++) + { + CHECK((*ignoredPtr)[index] == static_cast(10 * (index + 1))); + idsChanged = idsChanged || (*idsPtr)[index] != originalIds[index]; + } + CHECK(idsChanged); + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + TEST_CASE("SimplnxCore::FillBadData: Generate Test Data", "[Core][FillBadDataFilter][.GenerateTestData]") { UnitTest::LoadPlugins(); diff --git a/src/Plugins/SimplnxCore/test/KeepRemoveRankedFeaturesTest.cpp b/src/Plugins/SimplnxCore/test/KeepRemoveRankedFeaturesTest.cpp index 63addabab6..b5eb1b6097 100644 --- a/src/Plugins/SimplnxCore/test/KeepRemoveRankedFeaturesTest.cpp +++ b/src/Plugins/SimplnxCore/test/KeepRemoveRankedFeaturesTest.cpp @@ -1,6 +1,9 @@ #include "SimplnxCore/Filters/KeepRemoveRankedFeaturesFilter.hpp" #include "SimplnxCore/SimplnxCore_test_dirs.hpp" +#include + +#include "simplnx/DataStructure/AttributeMatrix.hpp" #include "simplnx/DataStructure/DataGroup.hpp" #include "simplnx/DataStructure/Geometry/ImageGeom.hpp" #include "simplnx/DataStructure/NeighborList.hpp" @@ -11,8 +14,6 @@ #include "simplnx/Parameters/NumberParameter.hpp" #include "simplnx/UnitTest/UnitTestCommon.hpp" -#include - #include #include #include @@ -613,6 +614,208 @@ TEST_CASE("SimplnxCore::KeepRemoveRankedFeaturesFilter: Unsigned ranking array", UnitTest::CheckArraysInheritTupleDims(dataStructure); } +TEST_CASE("SimplnxCore::KeepRemoveRankedFeaturesFilter: Strict Feature Ids", "[SimplnxCore][KeepRemoveRankedFeaturesFilter][StrictFeatureIds]") +{ + UnitTest::LoadPlugins(); + DataStructure dataStructure; + const DataPath geometryPath({"Strict Geometry"}); + const DataPath cellPath = geometryPath.createChildPath("Cells"); + const DataPath featurePath = geometryPath.createChildPath("Features"); + const DataPath idsPath = cellPath.createChildPath("Ids"); + const DataPath otherPath = geometryPath.createChildPath("Other Cells").createChildPath("Other Values"); + const ShapeType shape{1, 1, 4}; + auto* imageGeomPtr = ImageGeom::Create(dataStructure, geometryPath.getTargetName()); + REQUIRE(imageGeomPtr != nullptr); + imageGeomPtr->setDimensions({4, 1, 1}); + auto* cellsPtr = AttributeMatrix::Create(dataStructure, "Cells", shape, imageGeomPtr->getId()); + REQUIRE(cellsPtr != nullptr); + imageGeomPtr->setCellData(*cellsPtr); + auto* featuresPtr = AttributeMatrix::Create(dataStructure, "Features", {4}, imageGeomPtr->getId()); + REQUIRE(featuresPtr != nullptr); + auto* otherPtr = AttributeMatrix::Create(dataStructure, "Other Cells", shape, imageGeomPtr->getId()); + REQUIRE(otherPtr != nullptr); + REQUIRE(UnitTest::CreateTestDataArray(dataStructure, "Ids", shape, {1}, cellsPtr->getId()) != nullptr); + REQUIRE(UnitTest::CreateTestDataArray(dataStructure, "Other Values", shape, {1}, otherPtr->getId()) != nullptr); + REQUIRE(UnitTest::CreateTestDataArray(dataStructure, "Feature Values", {4}, {1}, featuresPtr->getId()) != nullptr); + REQUIRE(UnitTest::CreateTestDataArray(dataStructure, "Cell Feature Values", shape, {1}, cellsPtr->getId()) != nullptr); + auto* elsewherePtr = DataGroup::Create(dataStructure, "Elsewhere"); + REQUIRE(elsewherePtr != nullptr); + auto* externalFeaturesPtr = AttributeMatrix::Create(dataStructure, "External Features", {4}, elsewherePtr->getId()); + REQUIRE(externalFeaturesPtr != nullptr); + REQUIRE(UnitTest::CreateTestDataArray(dataStructure, "Feature Values", {4}, {1}, externalFeaturesPtr->getId()) != nullptr); + + KeepRemoveRankedFeaturesFilter filter; + auto args = filter.getDefaultArguments(); + args.insertOrAssign(KeepRemoveRankedFeaturesFilter::k_SelectedImageGeometryPath_Key, std::make_any(geometryPath)); + args.insertOrAssign(KeepRemoveRankedFeaturesFilter::k_CellFeatureIdsArrayPath_Key, std::make_any(idsPath)); + args.insertOrAssign(KeepRemoveRankedFeaturesFilter::k_RankingArrayPath_Key, std::make_any(featurePath.createChildPath("Feature Values"))); + args.insertOrAssign(KeepRemoveRankedFeaturesFilter::k_FillRemovedFeatures_Key, std::make_any(true)); + + int32 expectedCode = 0; + DataPath offendingPath; + SECTION("Correct Cell and Feature Attribute Matrices") + { + } + SECTION("Feature Ids in a second Attribute Matrix with the same tuple shape") + { + offendingPath = otherPath; + args.insertOrAssign(KeepRemoveRankedFeaturesFilter::k_CellFeatureIdsArrayPath_Key, std::make_any(offendingPath)); + expectedCode = -78125; + } + SECTION("Feature array outside the selected geometry") + { + offendingPath = DataPath({"Elsewhere", "External Features", "Feature Values"}); + args.insertOrAssign(KeepRemoveRankedFeaturesFilter::k_RankingArrayPath_Key, std::make_any(offendingPath)); + expectedCode = -78127; + } + SECTION("Feature array under a plain DataGroup") + { + REQUIRE(UnitTest::CreateTestDataArray(dataStructure, "Ungrouped Features", {4}, {1}, elsewherePtr->getId()) != nullptr); + offendingPath = DataPath({"Elsewhere", "Ungrouped Features"}); + args.insertOrAssign(KeepRemoveRankedFeaturesFilter::k_RankingArrayPath_Key, std::make_any(offendingPath)); + expectedCode = -78127; + } + SECTION("Feature array in the Cell Attribute Matrix") + { + offendingPath = cellPath.createChildPath("Cell Feature Values"); + args.insertOrAssign(KeepRemoveRankedFeaturesFilter::k_RankingArrayPath_Key, std::make_any(offendingPath)); + expectedCode = -78127; + } + SECTION("Feature Ids through a shared Cell Attribute Matrix") + { + REQUIRE(dataStructure.setAdditionalParent(cellsPtr->getId(), elsewherePtr->getId())); + args.insertOrAssign(KeepRemoveRankedFeaturesFilter::k_CellFeatureIdsArrayPath_Key, std::make_any(DataPath({"Elsewhere", "Cells", "Ids"}))); + } + SECTION("Ignored array in a second Attribute Matrix") + { + offendingPath = otherPath; + args.insertOrAssign(KeepRemoveRankedFeaturesFilter::k_IgnoredDataArrayPaths_Key, std::make_any>(std::vector{otherPath})); + expectedCode = -78126; + } + SECTION("Unused ignore list with fill disabled") + { + args.insertOrAssign(KeepRemoveRankedFeaturesFilter::k_FillRemovedFeatures_Key, std::make_any(false)); + args.insertOrAssign(KeepRemoveRankedFeaturesFilter::k_IgnoredDataArrayPaths_Key, std::make_any>(std::vector{otherPath})); + } + const auto result = filter.preflight(dataStructure, args); + if(expectedCode == 0) + { + REQUIRE(result.outputActions.valid()); + for(const auto& modification : result.outputActions.value().modifiedActions) + { + CHECK(modification.modifiedPath.getParent() != DataPath({"Elsewhere", "Cells"})); + } + } + else + { + REQUIRE(result.outputActions.invalid()); + REQUIRE(result.outputActions.errors().size() == 1); + CHECK(result.outputActions.errors().front().code == expectedCode); + CHECK(result.outputActions.errors().front().message.find(offendingPath.toString()) != std::string::npos); + CHECK(result.outputActions.errors().front().message.find(cellPath.toString()) != std::string::npos); + CHECK(result.outputActions.errors().front().message.find(geometryPath.toString()) != std::string::npos); + } + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + +TEST_CASE("SimplnxCore::KeepRemoveRankedFeaturesFilter: Shared Cell Paths and Selected Feature Matrix", "[SimplnxCore][KeepRemoveRankedFeaturesFilter][StrictFeatureIds]") +{ + UnitTest::LoadPlugins(); + const bool useAliasForIds = GENERATE(false, true); + CAPTURE(useAliasForIds); + DataStructure dataStructure; + const DataPath geometryPath({"Alias Geometry"}); + const DataPath cellPath = geometryPath.createChildPath("Cells"); + const DataPath aliasCellPath({"Earlier Parent", "Cells"}); + const DataPath featurePath = geometryPath.createChildPath("Features"); + const ShapeType shape{1, 1, 4}; + auto* earlierParentPtr = DataGroup::Create(dataStructure, "Earlier Parent"); + REQUIRE(earlierParentPtr != nullptr); + auto* imageGeomPtr = ImageGeom::Create(dataStructure, geometryPath.getTargetName()); + REQUIRE(imageGeomPtr != nullptr); + imageGeomPtr->setDimensions({4, 1, 1}); + auto* cellsPtr = AttributeMatrix::Create(dataStructure, "Cells", shape, imageGeomPtr->getId()); + REQUIRE(cellsPtr != nullptr); + imageGeomPtr->setCellData(*cellsPtr); + REQUIRE(dataStructure.setAdditionalParent(cellsPtr->getId(), earlierParentPtr->getId())); + REQUIRE(imageGeomPtr->getCellDataPath() == cellPath); + auto* idsPtr = UnitTest::CreateTestDataArray(dataStructure, "Ids", shape, {1}, cellsPtr->getId()); + auto* valuesPtr = UnitTest::CreateTestDataArray(dataStructure, "Values", shape, {1}, cellsPtr->getId()); + auto* ignoredPtr = UnitTest::CreateTestDataArray(dataStructure, "Ignored", shape, {1}, cellsPtr->getId()); + REQUIRE(idsPtr != nullptr); + REQUIRE(valuesPtr != nullptr); + REQUIRE(ignoredPtr != nullptr); + const std::vector originalIds{1, 2, 1, 1}; + for(usize index = 0; index < 4; index++) + { + (*idsPtr)[index] = originalIds[index]; + (*valuesPtr)[index] = static_cast(10 + index); + (*ignoredPtr)[index] = static_cast(100 + index); + } + + // The selected array has another parent; only the selected Feature Attribute Matrix is compacted. + auto* featureValuesPtr = UnitTest::CreateTestDataArray(dataStructure, "Feature Values", {3}, {1}, earlierParentPtr->getId()); + auto* externalMarkerPtr = UnitTest::CreateTestDataArray(dataStructure, "External Marker", {3}, {1}, earlierParentPtr->getId()); + REQUIRE(featureValuesPtr != nullptr); + REQUIRE(externalMarkerPtr != nullptr); + auto* featuresPtr = AttributeMatrix::Create(dataStructure, "Features", {3}, imageGeomPtr->getId()); + REQUIRE(featuresPtr != nullptr); + REQUIRE(dataStructure.setAdditionalParent(featureValuesPtr->getId(), featuresPtr->getId())); + auto* markerPtr = UnitTest::CreateTestDataArray(dataStructure, "Marker", {3}, {1}, featuresPtr->getId()); + REQUIRE(markerPtr != nullptr); + for(usize index = 0; index < 3; index++) + { + (*markerPtr)[index] = static_cast(11 * index); + (*externalMarkerPtr)[index] = static_cast(90 + index); + } + (*featureValuesPtr)[0] = 0; + (*featureValuesPtr)[1] = 3; + (*featureValuesPtr)[2] = 1; + + KeepRemoveRankedFeaturesFilter filter; + auto args = filter.getDefaultArguments(); + args.insertOrAssign(KeepRemoveRankedFeaturesFilter::k_SelectedImageGeometryPath_Key, std::make_any(geometryPath)); + args.insertOrAssign(KeepRemoveRankedFeaturesFilter::k_CellFeatureIdsArrayPath_Key, std::make_any((useAliasForIds ? aliasCellPath : cellPath).createChildPath("Ids"))); + args.insertOrAssign(KeepRemoveRankedFeaturesFilter::k_RankingArrayPath_Key, std::make_any(featurePath.createChildPath("Feature Values"))); + args.insertOrAssign(KeepRemoveRankedFeaturesFilter::k_Operation_Key, std::make_any(0)); + args.insertOrAssign(KeepRemoveRankedFeaturesFilter::k_NumFeatures_Key, std::make_any(1)); + args.insertOrAssign(KeepRemoveRankedFeaturesFilter::k_FillRemovedFeatures_Key, std::make_any(true)); + const auto ignoreParent = useAliasForIds ? cellPath : aliasCellPath; + args.insertOrAssign(KeepRemoveRankedFeaturesFilter::k_IgnoredDataArrayPaths_Key, + std::make_any>(std::vector{ignoreParent.createChildPath("Ignored"), ignoreParent.createChildPath("Ids")})); + const auto preflight = filter.preflight(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflight.outputActions); + bool reportsValues = false; + for(const auto& modification : preflight.outputActions.value().modifiedActions) + { + CHECK(modification.modifiedPath.getParent() != aliasCellPath); + reportsValues = reportsValues || modification.modifiedPath == cellPath.createChildPath("Values"); + } + CHECK(reportsValues); + const auto execute = filter.execute(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(execute.result); + for(usize index = 0; index < 4; index++) + { + CHECK((*idsPtr)[index] == 1); + CHECK((*ignoredPtr)[index] == static_cast(100 + index)); + if(index != 1) + { + CHECK((*valuesPtr)[index] == static_cast(10 + index)); + } + } + CHECK(((*valuesPtr)[1] == 10 || (*valuesPtr)[1] == 12)); + CHECK(featuresPtr->getNumberOfTuples() == 2); + REQUIRE(markerPtr->getNumberOfTuples() == 2); + CHECK((*markerPtr)[0] == 0); + CHECK((*markerPtr)[1] == 11); + REQUIRE(externalMarkerPtr->getNumberOfTuples() == 3); + for(usize index = 0; index < 3; index++) + { + CHECK((*externalMarkerPtr)[index] == static_cast(90 + index)); + } + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + TEST_CASE("SimplnxCore::KeepRemoveRankedFeaturesFilter: Preflight validation", "[SimplnxCore][KeepRemoveRankedFeatures]") { UnitTest::LoadPlugins(); @@ -703,6 +906,7 @@ TEST_CASE("SimplnxCore::KeepRemoveRankedFeaturesFilter: Preflight validation", " auto preflightResult = filter.preflight(dataStructure, args); SIMPLNX_RESULT_REQUIRE_INVALID(preflightResult.outputActions); + REQUIRE(preflightResult.outputActions.errors().front().code == -78127); } SECTION("A NeighborList in the Feature Attribute Matrix produces a warning") diff --git a/src/Plugins/SimplnxCore/test/QuickSurfaceMeshFilterTest.cpp b/src/Plugins/SimplnxCore/test/QuickSurfaceMeshFilterTest.cpp index 627f9aff2b..8eb21f6256 100644 --- a/src/Plugins/SimplnxCore/test/QuickSurfaceMeshFilterTest.cpp +++ b/src/Plugins/SimplnxCore/test/QuickSurfaceMeshFilterTest.cpp @@ -7,6 +7,7 @@ #include "simplnx/DataStructure/Geometry/INodeGeometry0D.hpp" #include "simplnx/DataStructure/Geometry/INodeGeometry2D.hpp" #include "simplnx/DataStructure/Geometry/ImageGeom.hpp" +#include "simplnx/DataStructure/Geometry/RectGridGeom.hpp" #include "simplnx/DataStructure/Geometry/TriangleGeom.hpp" #include "simplnx/Parameters/ArrayCreationParameter.hpp" #include "simplnx/Parameters/BoolParameter.hpp" @@ -503,6 +504,104 @@ TEST_CASE("SimplnxCore::QuickSurfaceMeshFilter: Winding and Problem Voxels", "[S UnitTest::CheckArraysInheritTupleDims(dataStructure); } +TEST_CASE("SimplnxCore::QuickSurfaceMeshFilter: Strict Feature Ids", "[SimplnxCore][QuickSurfaceMeshFilter][StrictFeatureIds]") +{ + UnitTest::LoadPlugins(); + const bool rectilinear = GENERATE(false, true); + const bool fixProblemVoxels = GENERATE(false, true); + CAPTURE(rectilinear, fixProblemVoxels); + DataStructure dataStructure; + const DataPath geometryPath({"Strict Geometry"}); + const DataPath cellPath = geometryPath.createChildPath("Cell Data"); + const DataPath featureIdsPath = cellPath.createChildPath("Feature Ids"); + const ShapeType shape{1, 2, 2}; + IGridGeometry* gridGeomPtr = nullptr; + if(rectilinear) + { + gridGeomPtr = RectGridGeom::Create(dataStructure, geometryPath.getTargetName()); + } + else + { + gridGeomPtr = ImageGeom::Create(dataStructure, geometryPath.getTargetName()); + } + REQUIRE(gridGeomPtr != nullptr); + gridGeomPtr->setDimensions({2, 2, 1}); + auto* cellsPtr = AttributeMatrix::Create(dataStructure, "Cell Data", shape, gridGeomPtr->getId()); + REQUIRE(cellsPtr != nullptr); + gridGeomPtr->setCellData(*cellsPtr); + REQUIRE(AttributeMatrix::Create(dataStructure, "Feature Data", ShapeType{3}, gridGeomPtr->getId()) != nullptr); + auto* otherPtr = AttributeMatrix::Create(dataStructure, "Other Data", shape, gridGeomPtr->getId()); + REQUIRE(otherPtr != nullptr); + REQUIRE(Int32Array::Create(dataStructure, "Feature Ids", std::make_shared>(shape, ShapeType{1}, 1), cellsPtr->getId()) != nullptr); + REQUIRE(Int32Array::Create(dataStructure, "Feature Ids", std::make_shared>(shape, ShapeType{1}, 1), otherPtr->getId()) != nullptr); + + QuickSurfaceMeshFilter filter; + auto args = filter.getDefaultArguments(); + args.insertOrAssign(QuickSurfaceMeshFilter::k_GridGeometryDataPath_Key, std::make_any(geometryPath)); + args.insertOrAssign(QuickSurfaceMeshFilter::k_FixProblemVoxels_Key, std::make_any(fixProblemVoxels)); + DataPath selectedPath = featureIdsPath; + int32 expectedCode = 0; + SECTION("Correct Cell Attribute Matrix") + { + } + SECTION("Feature Ids in another Attribute Matrix with the same tuple shape") + { + selectedPath = geometryPath.createChildPath("Other Data").createChildPath("Feature Ids"); + expectedCode = fixProblemVoxels ? -76532 : 0; + } + SECTION("Feature Ids through an alias of the Cell Attribute Matrix") + { + auto* alternateParentPtr = DataGroup::Create(dataStructure, "Alternate Parent"); + REQUIRE(alternateParentPtr != nullptr); + REQUIRE(dataStructure.setAdditionalParent(cellsPtr->getId(), alternateParentPtr->getId())); + selectedPath = DataPath({"Alternate Parent", "Cell Data", "Feature Ids"}); + REQUIRE(gridGeomPtr->getCellDataPath() == cellPath); + } + SECTION("Feature Ids with fewer tuples than grid cells") + { + selectedPath = DataPath({"Short Feature Ids"}); + REQUIRE(Int32Array::Create(dataStructure, "Short Feature Ids", std::make_shared>(ShapeType{3}, ShapeType{1}, 1)) != nullptr); + expectedCode = -76533; + } + SECTION("Feature Ids with more tuples than grid cells") + { + selectedPath = DataPath({"Long Feature Ids"}); + REQUIRE(Int32Array::Create(dataStructure, "Long Feature Ids", std::make_shared>(ShapeType{5}, ShapeType{1}, 1)) != nullptr); + expectedCode = -76533; + } + args.insertOrAssign(QuickSurfaceMeshFilter::k_CellFeatureIdsArrayPath_Key, std::make_any(selectedPath)); + const auto result = filter.preflight(dataStructure, args); + if(expectedCode == 0) + { + SIMPLNX_RESULT_REQUIRE_VALID(result.outputActions); + const auto& modificationsRef = result.outputActions.value().modifiedActions; + if(fixProblemVoxels) + { + REQUIRE(modificationsRef.size() == 1); + CHECK(modificationsRef.front().modifiedPath == featureIdsPath); + CHECK(modificationsRef.front().modifiedType == DataObjectModification::ModifiedType::Modified); + } + else + { + CHECK(modificationsRef.empty()); + } + } + else + { + REQUIRE(result.outputActions.invalid()); + REQUIRE(result.outputActions.errors().size() == 1); + const auto& errorRef = result.outputActions.errors().front(); + CHECK(errorRef.code == expectedCode); + CHECK(errorRef.message.find(selectedPath.toString()) != std::string::npos); + CHECK(errorRef.message.find(geometryPath.toString()) != std::string::npos); + if(expectedCode == -76532) + { + CHECK(errorRef.message.find(cellPath.toString()) != std::string::npos); + } + } + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + TEST_CASE("SimplnxCore::QuickSurfaceMeshFilter: SIMPL Backwards Compatibility", "[SimplnxCore][QuickSurfaceMeshFilter][BackwardsCompatibility]") { auto app = Application::GetOrCreateInstance(); diff --git a/src/Plugins/SimplnxCore/test/RemoveFlaggedFeaturesTest.cpp b/src/Plugins/SimplnxCore/test/RemoveFlaggedFeaturesTest.cpp index 60b4ea12ea..1ee687fc75 100644 --- a/src/Plugins/SimplnxCore/test/RemoveFlaggedFeaturesTest.cpp +++ b/src/Plugins/SimplnxCore/test/RemoveFlaggedFeaturesTest.cpp @@ -1,7 +1,10 @@ #include "SimplnxCore/Filters/RemoveFlaggedFeaturesFilter.hpp" #include "SimplnxCore/SimplnxCore_test_dirs.hpp" +#include + #include "simplnx/Core/Application.hpp" +#include "simplnx/DataStructure/AttributeMatrix.hpp" #include "simplnx/DataStructure/DataGroup.hpp" #include "simplnx/DataStructure/Geometry/ImageGeom.hpp" #include "simplnx/DataStructure/NeighborList.hpp" @@ -15,8 +18,6 @@ #include "simplnx/Utilities/AlgorithmDispatch.hpp" #include "simplnx/Utilities/DataStoreUtilities.hpp" -#include - #include #include #include @@ -79,7 +80,7 @@ constexpr int32 k_FeatureIdOutOfRangeError = -45435; constexpr int32 k_NoFillProgressError = -45436; constexpr int32 k_TupleCountMismatchError = -45437; constexpr int32 k_FeatureIdsCannotBeIgnoredWarning = -45438; -constexpr int32 k_ParentNotAttributeMatrixError = -9892; +constexpr int32 k_ParentNotAttributeMatrixError = -9895; constexpr int32 k_NeighborListRemovalWarning = -5558; constexpr int32 k_EmptyFeatureSkippedWarning = -53905; @@ -667,17 +668,15 @@ TEST_CASE("SimplnxCore::RemoveFlaggedFeaturesFilter: Execute Error - Feature Ids DataStructure dataStructure; BuildFixture(dataStructure, FixtureA()); - auto* otherGroup = DataGroup::Create(dataStructure, "OtherData"); - const DataPath shortIdsPath({"OtherData", "ShortIds"}); - auto shortIdsStore = DataStoreUtilities::CreateDataStore(dataStructure, shortIdsPath, {4}, {1}); - auto* shortIdsArray = Int32Array::Create(dataStructure, "ShortIds", shortIdsStore, otherGroup->getId()); - REQUIRE(shortIdsArray != nullptr); - shortIdsArray->fill(1); + // Cell membership does not validate the geometry dimensions against the Cell Attribute Matrix shape. + REQUIRE_NOTHROW(dataStructure.getDataRefAs(k_ImageGeomPath)); + auto& imageGeomRef = dataStructure.getDataRefAs(k_ImageGeomPath); + auto dimensions = imageGeomRef.getDimensions(); + dimensions[0]++; + imageGeomRef.setDimensions(dimensions); - Arguments args = MakeArgs(k_Remove, false); - args.insertOrAssign(RemoveFlaggedFeaturesFilter::k_CellFeatureIdsArrayPath_Key, std::make_any(shortIdsPath)); - REQUIRE(RunExecuteError(dataStructure, args, scope) == k_TupleCountMismatchError); - REQUIRE(ReadInt32(dataStructure, shortIdsPath) == std::vector{1, 1, 1, 1}); + REQUIRE(RunExecuteError(dataStructure, MakeArgs(k_Remove, false), scope) == k_TupleCountMismatchError); + REQUIRE(ReadInt32(dataStructure, k_FeatureIdsPath) == FixtureA().featureIds); } TEST_CASE("SimplnxCore::RemoveFlaggedFeaturesFilter: Execute Error - no fill progress (-45436)", "[SimplnxCore][RemoveFlaggedFeaturesFilter]") @@ -944,7 +943,7 @@ TEST_CASE("SimplnxCore::RemoveFlaggedFeaturesFilter: Preflight Stale Arrays Only UnitTest::CheckArraysInheritTupleDims(dataStructure); } -TEST_CASE("SimplnxCore::RemoveFlaggedFeaturesFilter: Preflight Error - flag array parent is not an Attribute Matrix (-9892)", "[SimplnxCore][RemoveFlaggedFeaturesFilter]") +TEST_CASE("SimplnxCore::RemoveFlaggedFeaturesFilter: Preflight Error - flag array parent is not an Attribute Matrix (-9895)", "[SimplnxCore][RemoveFlaggedFeaturesFilter]") { UnitTest::LoadPlugins(); DataStructure dataStructure; @@ -1474,6 +1473,212 @@ TEST_CASE("SimplnxCore::RemoveFlaggedFeatures: Test Extract then Remove Algorith UnitTest::CheckArraysInheritTupleDims(dataStructure); } +TEST_CASE("SimplnxCore::RemoveFlaggedFeaturesFilter: Strict Feature Ids", "[SimplnxCore][RemoveFlaggedFeaturesFilter][StrictFeatureIds]") +{ + UnitTest::LoadPlugins(); + DataStructure dataStructure; + const DataPath geometryPath({"Strict Geometry"}); + const DataPath cellPath = geometryPath.createChildPath("Cells"); + const DataPath featurePath = geometryPath.createChildPath("Features"); + const DataPath idsPath = cellPath.createChildPath("Ids"); + const DataPath otherPath = geometryPath.createChildPath("Other Cells").createChildPath("Other Values"); + const ShapeType shape{1, 1, 4}; + auto* imageGeomPtr = ImageGeom::Create(dataStructure, geometryPath.getTargetName()); + REQUIRE(imageGeomPtr != nullptr); + imageGeomPtr->setDimensions({4, 1, 1}); + auto* cellsPtr = AttributeMatrix::Create(dataStructure, "Cells", shape, imageGeomPtr->getId()); + REQUIRE(cellsPtr != nullptr); + imageGeomPtr->setCellData(*cellsPtr); + auto* featuresPtr = AttributeMatrix::Create(dataStructure, "Features", {4}, imageGeomPtr->getId()); + REQUIRE(featuresPtr != nullptr); + auto* otherPtr = AttributeMatrix::Create(dataStructure, "Other Cells", shape, imageGeomPtr->getId()); + REQUIRE(otherPtr != nullptr); + REQUIRE(UnitTest::CreateTestDataArray(dataStructure, "Ids", shape, {1}, cellsPtr->getId()) != nullptr); + REQUIRE(UnitTest::CreateTestDataArray(dataStructure, "Other Values", shape, {1}, otherPtr->getId()) != nullptr); + REQUIRE(UnitTest::CreateTestDataArray(dataStructure, "Feature Values", {4}, {1}, featuresPtr->getId()) != nullptr); + REQUIRE(UnitTest::CreateTestDataArray(dataStructure, "Cell Feature Values", shape, {1}, cellsPtr->getId()) != nullptr); + auto* elsewherePtr = DataGroup::Create(dataStructure, "Elsewhere"); + REQUIRE(elsewherePtr != nullptr); + auto* externalFeaturesPtr = AttributeMatrix::Create(dataStructure, "External Features", {4}, elsewherePtr->getId()); + REQUIRE(externalFeaturesPtr != nullptr); + REQUIRE(UnitTest::CreateTestDataArray(dataStructure, "Feature Values", {4}, {1}, externalFeaturesPtr->getId()) != nullptr); + + RemoveFlaggedFeaturesFilter filter; + auto args = filter.getDefaultArguments(); + args.insertOrAssign(RemoveFlaggedFeaturesFilter::k_SelectedImageGeometryPath_Key, std::make_any(geometryPath)); + args.insertOrAssign(RemoveFlaggedFeaturesFilter::k_CellFeatureIdsArrayPath_Key, std::make_any(idsPath)); + args.insertOrAssign(RemoveFlaggedFeaturesFilter::k_FlaggedFeaturesArrayPath_Key, std::make_any(featurePath.createChildPath("Feature Values"))); + const auto operation = GENERATE(uint64{0}, uint64{1}, uint64{2}); + args.insertOrAssign(RemoveFlaggedFeaturesFilter::k_Functionality_Key, std::make_any(operation)); + args.insertOrAssign(RemoveFlaggedFeaturesFilter::k_FillRemovedFeatures_Key, std::make_any(true)); + + int32 expectedCode = 0; + DataPath offendingPath; + SECTION("Correct Cell and Feature Attribute Matrices") + { + } + SECTION("Feature Ids in a second Attribute Matrix with the same tuple shape") + { + offendingPath = otherPath; + args.insertOrAssign(RemoveFlaggedFeaturesFilter::k_CellFeatureIdsArrayPath_Key, std::make_any(offendingPath)); + expectedCode = -9893; + } + SECTION("Feature array outside the selected geometry") + { + offendingPath = DataPath({"Elsewhere", "External Features", "Feature Values"}); + args.insertOrAssign(RemoveFlaggedFeaturesFilter::k_FlaggedFeaturesArrayPath_Key, std::make_any(offendingPath)); + expectedCode = -9895; + } + SECTION("Feature array under a plain DataGroup") + { + REQUIRE(UnitTest::CreateTestDataArray(dataStructure, "Ungrouped Features", {4}, {1}, elsewherePtr->getId()) != nullptr); + offendingPath = DataPath({"Elsewhere", "Ungrouped Features"}); + args.insertOrAssign(RemoveFlaggedFeaturesFilter::k_FlaggedFeaturesArrayPath_Key, std::make_any(offendingPath)); + expectedCode = -9895; + } + SECTION("Feature array in the Cell Attribute Matrix") + { + offendingPath = cellPath.createChildPath("Cell Feature Values"); + args.insertOrAssign(RemoveFlaggedFeaturesFilter::k_FlaggedFeaturesArrayPath_Key, std::make_any(offendingPath)); + expectedCode = -9895; + } + SECTION("Feature Ids through a shared Cell Attribute Matrix") + { + REQUIRE(dataStructure.setAdditionalParent(cellsPtr->getId(), elsewherePtr->getId())); + args.insertOrAssign(RemoveFlaggedFeaturesFilter::k_CellFeatureIdsArrayPath_Key, std::make_any(DataPath({"Elsewhere", "Cells", "Ids"}))); + } + SECTION("Ignored array in a second Attribute Matrix") + { + offendingPath = otherPath; + args.insertOrAssign(RemoveFlaggedFeaturesFilter::k_IgnoredDataArrayPaths_Key, std::make_any>(std::vector{otherPath})); + expectedCode = operation == 1 ? 0 : -9894; + } + SECTION("Unused ignore list with fill disabled") + { + args.insertOrAssign(RemoveFlaggedFeaturesFilter::k_FillRemovedFeatures_Key, std::make_any(false)); + args.insertOrAssign(RemoveFlaggedFeaturesFilter::k_IgnoredDataArrayPaths_Key, std::make_any>(std::vector{otherPath})); + } + const auto result = filter.preflight(dataStructure, args); + if(expectedCode == 0) + { + REQUIRE(result.outputActions.valid()); + for(const auto& modification : result.outputActions.value().modifiedActions) + { + CHECK(modification.modifiedPath.getParent() != DataPath({"Elsewhere", "Cells"})); + } + } + else + { + REQUIRE(result.outputActions.invalid()); + REQUIRE(result.outputActions.errors().size() == 1); + CHECK(result.outputActions.errors().front().code == expectedCode); + CHECK(result.outputActions.errors().front().message.find(offendingPath.toString()) != std::string::npos); + CHECK(result.outputActions.errors().front().message.find(cellPath.toString()) != std::string::npos); + CHECK(result.outputActions.errors().front().message.find(geometryPath.toString()) != std::string::npos); + } + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + +TEST_CASE("SimplnxCore::RemoveFlaggedFeaturesFilter: Shared Cell Paths and Selected Feature Matrix", "[SimplnxCore][RemoveFlaggedFeaturesFilter][StrictFeatureIds]") +{ + UnitTest::LoadPlugins(); + const auto scenario = GENERATE(from_range(UnitTest::SelectAlgorithmTestScenariosForInMemoryStores())); + CAPTURE(scenario); + UnitTest::AlgorithmTestScope algorithmTestScope(scenario); + const bool useAliasForIds = GENERATE(false, true); + CAPTURE(useAliasForIds); + DataStructure dataStructure; + const DataPath geometryPath({"Alias Geometry"}); + const DataPath cellPath = geometryPath.createChildPath("Cells"); + const DataPath aliasCellPath({"Earlier Parent", "Cells"}); + const DataPath featurePath = geometryPath.createChildPath("Features"); + const ShapeType shape{1, 1, 4}; + auto* earlierParentPtr = DataGroup::Create(dataStructure, "Earlier Parent"); + REQUIRE(earlierParentPtr != nullptr); + auto* imageGeomPtr = ImageGeom::Create(dataStructure, geometryPath.getTargetName()); + REQUIRE(imageGeomPtr != nullptr); + imageGeomPtr->setDimensions({4, 1, 1}); + auto* cellsPtr = AttributeMatrix::Create(dataStructure, "Cells", shape, imageGeomPtr->getId()); + REQUIRE(cellsPtr != nullptr); + imageGeomPtr->setCellData(*cellsPtr); + REQUIRE(dataStructure.setAdditionalParent(cellsPtr->getId(), earlierParentPtr->getId())); + REQUIRE(imageGeomPtr->getCellDataPath() == cellPath); + auto* idsPtr = UnitTest::CreateTestDataArray(dataStructure, "Ids", shape, {1}, cellsPtr->getId()); + auto* valuesPtr = UnitTest::CreateTestDataArray(dataStructure, "Values", shape, {1}, cellsPtr->getId()); + auto* ignoredPtr = UnitTest::CreateTestDataArray(dataStructure, "Ignored", shape, {1}, cellsPtr->getId()); + REQUIRE(idsPtr != nullptr); + REQUIRE(valuesPtr != nullptr); + REQUIRE(ignoredPtr != nullptr); + const std::vector originalIds{1, 2, 1, 1}; + for(usize index = 0; index < 4; index++) + { + (*idsPtr)[index] = originalIds[index]; + (*valuesPtr)[index] = static_cast(10 + index); + (*ignoredPtr)[index] = static_cast(100 + index); + } + + // The selected array has another parent; only the selected Feature Attribute Matrix is compacted. + auto* featureValuesPtr = UnitTest::CreateTestDataArray(dataStructure, "Feature Values", {3}, {1}, earlierParentPtr->getId()); + auto* externalMarkerPtr = UnitTest::CreateTestDataArray(dataStructure, "External Marker", {3}, {1}, earlierParentPtr->getId()); + REQUIRE(featureValuesPtr != nullptr); + REQUIRE(externalMarkerPtr != nullptr); + auto* featuresPtr = AttributeMatrix::Create(dataStructure, "Features", {3}, imageGeomPtr->getId()); + REQUIRE(featuresPtr != nullptr); + REQUIRE(dataStructure.setAdditionalParent(featureValuesPtr->getId(), featuresPtr->getId())); + auto* markerPtr = UnitTest::CreateTestDataArray(dataStructure, "Marker", {3}, {1}, featuresPtr->getId()); + REQUIRE(markerPtr != nullptr); + for(usize index = 0; index < 3; index++) + { + (*markerPtr)[index] = static_cast(11 * index); + (*externalMarkerPtr)[index] = static_cast(90 + index); + } + (*featureValuesPtr)[0] = false; + (*featureValuesPtr)[1] = false; + (*featureValuesPtr)[2] = true; + + RemoveFlaggedFeaturesFilter filter; + auto args = filter.getDefaultArguments(); + args.insertOrAssign(RemoveFlaggedFeaturesFilter::k_SelectedImageGeometryPath_Key, std::make_any(geometryPath)); + args.insertOrAssign(RemoveFlaggedFeaturesFilter::k_CellFeatureIdsArrayPath_Key, std::make_any((useAliasForIds ? aliasCellPath : cellPath).createChildPath("Ids"))); + args.insertOrAssign(RemoveFlaggedFeaturesFilter::k_FlaggedFeaturesArrayPath_Key, std::make_any(featurePath.createChildPath("Feature Values"))); + args.insertOrAssign(RemoveFlaggedFeaturesFilter::k_Functionality_Key, std::make_any(0)); + args.insertOrAssign(RemoveFlaggedFeaturesFilter::k_FillRemovedFeatures_Key, std::make_any(true)); + const auto ignoreParent = useAliasForIds ? cellPath : aliasCellPath; + args.insertOrAssign(RemoveFlaggedFeaturesFilter::k_IgnoredDataArrayPaths_Key, + std::make_any>(std::vector{ignoreParent.createChildPath("Ignored"), ignoreParent.createChildPath("Ids")})); + const auto preflight = filter.preflight(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflight.outputActions); + bool reportsValues = false; + for(const auto& modification : preflight.outputActions.value().modifiedActions) + { + CHECK(modification.modifiedPath.getParent() != aliasCellPath); + reportsValues = reportsValues || modification.modifiedPath == cellPath.createChildPath("Values"); + } + CHECK(reportsValues); + const auto execute = algorithmTestScope.executeFilter(filter, dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(execute.result); + for(usize index = 0; index < 4; index++) + { + CHECK((*idsPtr)[index] == 1); + CHECK((*ignoredPtr)[index] == static_cast(100 + index)); + if(index != 1) + { + CHECK((*valuesPtr)[index] == static_cast(10 + index)); + } + } + CHECK(((*valuesPtr)[1] == 10 || (*valuesPtr)[1] == 12)); + CHECK(featuresPtr->getNumberOfTuples() == 2); + REQUIRE(markerPtr->getNumberOfTuples() == 2); + CHECK((*markerPtr)[0] == 0); + CHECK((*markerPtr)[1] == 11); + REQUIRE(externalMarkerPtr->getNumberOfTuples() == 3); + for(usize index = 0; index < 3; index++) + { + CHECK((*externalMarkerPtr)[index] == static_cast(90 + index)); + } + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + TEST_CASE("SimplnxCore::RemoveFlaggedFeaturesFilter: SIMPL Backwards Compatibility", "[SimplnxCore][RemoveFlaggedFeaturesFilter][BackwardsCompatibility]") { auto app = Application::GetOrCreateInstance(); diff --git a/src/Plugins/SimplnxCore/test/RequireMinNumNeighborsTest.cpp b/src/Plugins/SimplnxCore/test/RequireMinNumNeighborsTest.cpp index a8b701a046..d0351c7388 100644 --- a/src/Plugins/SimplnxCore/test/RequireMinNumNeighborsTest.cpp +++ b/src/Plugins/SimplnxCore/test/RequireMinNumNeighborsTest.cpp @@ -3,9 +3,12 @@ #include "SimplnxCore/Filters/RequireMinNumNeighborsFilter.hpp" #include "SimplnxCore/SimplnxCore_test_dirs.hpp" +#include + #include "simplnx/Core/Application.hpp" #include "simplnx/DataStructure/AttributeMatrix.hpp" #include "simplnx/DataStructure/DataArray.hpp" +#include "simplnx/DataStructure/DataGroup.hpp" #include "simplnx/DataStructure/Geometry/ImageGeom.hpp" #include "simplnx/Parameters/MultiArraySelectionParameter.hpp" #include "simplnx/Pipeline/Pipeline.hpp" @@ -14,8 +17,6 @@ #include "simplnx/Utilities/DataStoreUtilities.hpp" #include "simplnx/Utilities/Parsing/HDF5/IO/FileIO.hpp" -#include - #include #include #include @@ -610,8 +611,9 @@ TEST_CASE("SimplnxCore::RequireMinNumNeighborsFilter: Preflight Error - tuple co imageGeom->setDimensions({4, 1, 1}); // FeatureIds must exist for preflight, but it is not part of the feature-level tuple-count check. - auto* cellAM = AttributeMatrix::Create(dataStructure, "CellData", {4}, imageGeom->getId()); - UnitTest::CreateTestDataArray(dataStructure, "FeatureIds", {4}, {1}, cellAM->getId()); + auto* cellAM = AttributeMatrix::Create(dataStructure, "CellData", {1, 1, 4}, imageGeom->getId()); + imageGeom->setCellData(*cellAM); + UnitTest::CreateTestDataArray(dataStructure, "FeatureIds", {1, 1, 4}, {1}, cellAM->getId()); // NumNeighbors has five feature tuples. auto* featureAM = AttributeMatrix::Create(dataStructure, "FeatureData", {5}, imageGeom->getId()); @@ -665,6 +667,35 @@ TEST_CASE("SimplnxCore::RequireMinNumNeighborsFilter: Preflight Error - FeatureI REQUIRE(preflightResult.outputActions.errors()[0].code == -55571); } +TEST_CASE("SimplnxCore::RequireMinNumNeighborsFilter: Preflight Error - FeatureIds outside Cell Attribute Matrix (-55575)", "[SimplnxCore][RequireMinNumNeighborsFilter][preflight]") +{ + UnitTest::LoadPlugins(); + + DataStructure dataStructure; + auto* imageGeomPtr = ImageGeom::Create(dataStructure, "ImageGeometry"); + imageGeomPtr->setDimensions({4, 1, 1}); + auto* cellAMPtr = AttributeMatrix::Create(dataStructure, "CellData", {1, 1, 4}, imageGeomPtr->getId()); + imageGeomPtr->setCellData(*cellAMPtr); + auto* otherAMPtr = AttributeMatrix::Create(dataStructure, "OtherData", {1, 1, 4}, imageGeomPtr->getId()); + auto* featureAMPtr = AttributeMatrix::Create(dataStructure, "FeatureData", {4}, imageGeomPtr->getId()); + UnitTest::CreateTestDataArray(dataStructure, "FeatureIds", {1, 1, 4}, {1}, otherAMPtr->getId()); + UnitTest::CreateTestDataArray(dataStructure, "NumNeighbors", {4}, {1}, featureAMPtr->getId()); + + RequireMinNumNeighborsFilter filter; + Arguments args; + args.insertOrAssign(RequireMinNumNeighborsFilter::k_MinNumNeighbors_Key, std::make_any(0)); + args.insertOrAssign(RequireMinNumNeighborsFilter::k_ApplyToSinglePhase_Key, std::make_any(false)); + args.insertOrAssign(RequireMinNumNeighborsFilter::k_PhaseNumber_Key, std::make_any(0)); + args.insertOrAssign(RequireMinNumNeighborsFilter::k_SelectedImageGeometryPath_Key, std::make_any(DataPath({"ImageGeometry"}))); + args.insertOrAssign(RequireMinNumNeighborsFilter::k_FeatureIdsPath_Key, std::make_any(DataPath({"ImageGeometry", "OtherData", "FeatureIds"}))); + args.insertOrAssign(RequireMinNumNeighborsFilter::k_NumNeighborsPath_Key, std::make_any(DataPath({"ImageGeometry", "FeatureData", "NumNeighbors"}))); + args.insertOrAssign(RequireMinNumNeighborsFilter::k_IgnoredVoxelArrays_Key, std::make_any>(std::vector{})); + + const auto preflightResult = filter.preflight(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_INVALID(preflightResult.outputActions); + REQUIRE(preflightResult.outputActions.errors()[0].code == -55575); +} + TEST_CASE("SimplnxCore::RequireMinNumNeighborsFilter: Execute Error - feature ID out of range (-55567)", "[SimplnxCore][RequireMinNumNeighborsFilter][execute]") { UnitTest::LoadPlugins(); @@ -898,6 +929,204 @@ TEST_CASE("SimplnxCore::RequireMinNumNeighborsFilter: Execute Error - no coarsen UnitTest::CheckArraysInheritTupleDims(dataStructure); } +TEST_CASE("SimplnxCore::RequireMinNumNeighborsFilter: Strict Feature Ids", "[SimplnxCore][RequireMinNumNeighborsFilter][StrictFeatureIds]") +{ + UnitTest::LoadPlugins(); + DataStructure dataStructure; + const DataPath geometryPath({"Strict Geometry"}); + const DataPath cellPath = geometryPath.createChildPath("Cells"); + const DataPath featurePath = geometryPath.createChildPath("Features"); + const DataPath idsPath = cellPath.createChildPath("Ids"); + const DataPath otherPath = geometryPath.createChildPath("Other Cells").createChildPath("Other Values"); + const ShapeType shape{1, 1, 4}; + auto* imageGeomPtr = ImageGeom::Create(dataStructure, geometryPath.getTargetName()); + REQUIRE(imageGeomPtr != nullptr); + imageGeomPtr->setDimensions({4, 1, 1}); + auto* cellsPtr = AttributeMatrix::Create(dataStructure, "Cells", shape, imageGeomPtr->getId()); + REQUIRE(cellsPtr != nullptr); + imageGeomPtr->setCellData(*cellsPtr); + auto* featuresPtr = AttributeMatrix::Create(dataStructure, "Features", {4}, imageGeomPtr->getId()); + REQUIRE(featuresPtr != nullptr); + auto* otherPtr = AttributeMatrix::Create(dataStructure, "Other Cells", shape, imageGeomPtr->getId()); + REQUIRE(otherPtr != nullptr); + REQUIRE(UnitTest::CreateTestDataArray(dataStructure, "Ids", shape, {1}, cellsPtr->getId()) != nullptr); + REQUIRE(UnitTest::CreateTestDataArray(dataStructure, "Other Values", shape, {1}, otherPtr->getId()) != nullptr); + REQUIRE(UnitTest::CreateTestDataArray(dataStructure, "Feature Values", {4}, {1}, featuresPtr->getId()) != nullptr); + REQUIRE(UnitTest::CreateTestDataArray(dataStructure, "Cell Feature Values", shape, {1}, cellsPtr->getId()) != nullptr); + auto* elsewherePtr = DataGroup::Create(dataStructure, "Elsewhere"); + REQUIRE(elsewherePtr != nullptr); + auto* externalFeaturesPtr = AttributeMatrix::Create(dataStructure, "External Features", {4}, elsewherePtr->getId()); + REQUIRE(externalFeaturesPtr != nullptr); + REQUIRE(UnitTest::CreateTestDataArray(dataStructure, "Feature Values", {4}, {1}, externalFeaturesPtr->getId()) != nullptr); + + RequireMinNumNeighborsFilter filter; + auto args = filter.getDefaultArguments(); + args.insertOrAssign(RequireMinNumNeighborsFilter::k_SelectedImageGeometryPath_Key, std::make_any(geometryPath)); + args.insertOrAssign(RequireMinNumNeighborsFilter::k_FeatureIdsPath_Key, std::make_any(idsPath)); + args.insertOrAssign(RequireMinNumNeighborsFilter::k_NumNeighborsPath_Key, std::make_any(featurePath.createChildPath("Feature Values"))); + + int32 expectedCode = 0; + DataPath offendingPath; + SECTION("Correct Cell and Feature Attribute Matrices") + { + } + SECTION("Feature Ids in a second Attribute Matrix with the same tuple shape") + { + offendingPath = otherPath; + args.insertOrAssign(RequireMinNumNeighborsFilter::k_FeatureIdsPath_Key, std::make_any(offendingPath)); + expectedCode = -55575; + } + SECTION("Feature array outside the selected geometry") + { + offendingPath = DataPath({"Elsewhere", "External Features", "Feature Values"}); + args.insertOrAssign(RequireMinNumNeighborsFilter::k_NumNeighborsPath_Key, std::make_any(offendingPath)); + expectedCode = -55574; + } + SECTION("Feature array under a plain DataGroup") + { + REQUIRE(UnitTest::CreateTestDataArray(dataStructure, "Ungrouped Features", {4}, {1}, elsewherePtr->getId()) != nullptr); + offendingPath = DataPath({"Elsewhere", "Ungrouped Features"}); + args.insertOrAssign(RequireMinNumNeighborsFilter::k_NumNeighborsPath_Key, std::make_any(offendingPath)); + expectedCode = -55574; + } + SECTION("Feature array in the Cell Attribute Matrix") + { + offendingPath = cellPath.createChildPath("Cell Feature Values"); + args.insertOrAssign(RequireMinNumNeighborsFilter::k_NumNeighborsPath_Key, std::make_any(offendingPath)); + expectedCode = -55574; + } + SECTION("Feature Ids through a shared Cell Attribute Matrix") + { + REQUIRE(dataStructure.setAdditionalParent(cellsPtr->getId(), elsewherePtr->getId())); + args.insertOrAssign(RequireMinNumNeighborsFilter::k_FeatureIdsPath_Key, std::make_any(DataPath({"Elsewhere", "Cells", "Ids"}))); + } + SECTION("Ignored array in a second Attribute Matrix") + { + offendingPath = otherPath; + args.insertOrAssign(RequireMinNumNeighborsFilter::k_IgnoredVoxelArrays_Key, std::make_any>(std::vector{otherPath})); + expectedCode = -55573; + } + SECTION("Read-only Feature Phases in another Attribute Matrix") + { + args.insertOrAssign(RequireMinNumNeighborsFilter::k_ApplyToSinglePhase_Key, std::make_any(true)); + args.insertOrAssign(RequireMinNumNeighborsFilter::k_FeaturePhasesPath_Key, std::make_any(otherPath)); + } + const auto result = filter.preflight(dataStructure, args); + if(expectedCode == 0) + { + REQUIRE(result.outputActions.valid()); + for(const auto& modification : result.outputActions.value().modifiedActions) + { + CHECK(modification.modifiedPath.getParent() != DataPath({"Elsewhere", "Cells"})); + } + } + else + { + REQUIRE(result.outputActions.invalid()); + REQUIRE(result.outputActions.errors().size() == 1); + CHECK(result.outputActions.errors().front().code == expectedCode); + CHECK(result.outputActions.errors().front().message.find(offendingPath.toString()) != std::string::npos); + CHECK(result.outputActions.errors().front().message.find(cellPath.toString()) != std::string::npos); + CHECK(result.outputActions.errors().front().message.find(geometryPath.toString()) != std::string::npos); + } + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + +TEST_CASE("SimplnxCore::RequireMinNumNeighborsFilter: Shared Cell Paths and Selected Feature Matrix", "[SimplnxCore][RequireMinNumNeighborsFilter][StrictFeatureIds]") +{ + UnitTest::LoadPlugins(); + const bool useAliasForIds = GENERATE(false, true); + CAPTURE(useAliasForIds); + DataStructure dataStructure; + const DataPath geometryPath({"Alias Geometry"}); + const DataPath cellPath = geometryPath.createChildPath("Cells"); + const DataPath aliasCellPath({"Earlier Parent", "Cells"}); + const DataPath featurePath = geometryPath.createChildPath("Features"); + const ShapeType shape{1, 1, 4}; + auto* earlierParentPtr = DataGroup::Create(dataStructure, "Earlier Parent"); + REQUIRE(earlierParentPtr != nullptr); + auto* imageGeomPtr = ImageGeom::Create(dataStructure, geometryPath.getTargetName()); + REQUIRE(imageGeomPtr != nullptr); + imageGeomPtr->setDimensions({4, 1, 1}); + auto* cellsPtr = AttributeMatrix::Create(dataStructure, "Cells", shape, imageGeomPtr->getId()); + REQUIRE(cellsPtr != nullptr); + imageGeomPtr->setCellData(*cellsPtr); + REQUIRE(dataStructure.setAdditionalParent(cellsPtr->getId(), earlierParentPtr->getId())); + REQUIRE(imageGeomPtr->getCellDataPath() == cellPath); + auto* idsPtr = UnitTest::CreateTestDataArray(dataStructure, "Ids", shape, {1}, cellsPtr->getId()); + auto* valuesPtr = UnitTest::CreateTestDataArray(dataStructure, "Values", shape, {1}, cellsPtr->getId()); + auto* ignoredPtr = UnitTest::CreateTestDataArray(dataStructure, "Ignored", shape, {1}, cellsPtr->getId()); + REQUIRE(idsPtr != nullptr); + REQUIRE(valuesPtr != nullptr); + REQUIRE(ignoredPtr != nullptr); + const std::vector originalIds{1, 2, 1, 1}; + for(usize index = 0; index < 4; index++) + { + (*idsPtr)[index] = originalIds[index]; + (*valuesPtr)[index] = static_cast(10 + index); + (*ignoredPtr)[index] = static_cast(100 + index); + } + + // The selected array has another parent; only the selected Feature Attribute Matrix is compacted. + auto* featureValuesPtr = UnitTest::CreateTestDataArray(dataStructure, "Feature Values", {3}, {1}, earlierParentPtr->getId()); + auto* externalMarkerPtr = UnitTest::CreateTestDataArray(dataStructure, "External Marker", {3}, {1}, earlierParentPtr->getId()); + REQUIRE(featureValuesPtr != nullptr); + REQUIRE(externalMarkerPtr != nullptr); + auto* featuresPtr = AttributeMatrix::Create(dataStructure, "Features", {3}, imageGeomPtr->getId()); + REQUIRE(featuresPtr != nullptr); + REQUIRE(dataStructure.setAdditionalParent(featureValuesPtr->getId(), featuresPtr->getId())); + auto* markerPtr = UnitTest::CreateTestDataArray(dataStructure, "Marker", {3}, {1}, featuresPtr->getId()); + REQUIRE(markerPtr != nullptr); + for(usize index = 0; index < 3; index++) + { + (*markerPtr)[index] = static_cast(11 * index); + (*externalMarkerPtr)[index] = static_cast(90 + index); + } + (*featureValuesPtr)[0] = 0; + (*featureValuesPtr)[1] = 3; + (*featureValuesPtr)[2] = 1; + + RequireMinNumNeighborsFilter filter; + auto args = filter.getDefaultArguments(); + args.insertOrAssign(RequireMinNumNeighborsFilter::k_SelectedImageGeometryPath_Key, std::make_any(geometryPath)); + args.insertOrAssign(RequireMinNumNeighborsFilter::k_FeatureIdsPath_Key, std::make_any((useAliasForIds ? aliasCellPath : cellPath).createChildPath("Ids"))); + args.insertOrAssign(RequireMinNumNeighborsFilter::k_NumNeighborsPath_Key, std::make_any(featurePath.createChildPath("Feature Values"))); + args.insertOrAssign(RequireMinNumNeighborsFilter::k_MinNumNeighbors_Key, std::make_any(2)); + const auto ignoreParent = useAliasForIds ? cellPath : aliasCellPath; + args.insertOrAssign(RequireMinNumNeighborsFilter::k_IgnoredVoxelArrays_Key, std::make_any>(std::vector{ignoreParent.createChildPath("Ignored")})); + const auto preflight = filter.preflight(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflight.outputActions); + bool reportsValues = false; + for(const auto& modification : preflight.outputActions.value().modifiedActions) + { + CHECK(modification.modifiedPath.getParent() != aliasCellPath); + reportsValues = reportsValues || modification.modifiedPath == cellPath.createChildPath("Values"); + } + CHECK(reportsValues); + const auto execute = filter.execute(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(execute.result); + for(usize index = 0; index < 4; index++) + { + CHECK((*idsPtr)[index] == 1); + CHECK((*ignoredPtr)[index] == static_cast(100 + index)); + if(index != 1) + { + CHECK((*valuesPtr)[index] == static_cast(10 + index)); + } + } + CHECK(((*valuesPtr)[1] == 10 || (*valuesPtr)[1] == 12)); + CHECK(featuresPtr->getNumberOfTuples() == 2); + REQUIRE(markerPtr->getNumberOfTuples() == 2); + CHECK((*markerPtr)[0] == 0); + CHECK((*markerPtr)[1] == 11); + REQUIRE(externalMarkerPtr->getNumberOfTuples() == 3); + for(usize index = 0; index < 3; index++) + { + CHECK((*externalMarkerPtr)[index] == static_cast(90 + index)); + } + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + TEST_CASE("SimplnxCore::RequireMinNumNeighborsFilter: SIMPL Backwards Compatibility", "[SimplnxCore][RequireMinNumNeighborsFilter][BackwardsCompatibility]") { auto app = Application::GetOrCreateInstance(); diff --git a/src/Plugins/SimplnxCore/test/RequireMinimumSizeFeaturesTest.cpp b/src/Plugins/SimplnxCore/test/RequireMinimumSizeFeaturesTest.cpp index 0d3f4784b8..1494581421 100644 --- a/src/Plugins/SimplnxCore/test/RequireMinimumSizeFeaturesTest.cpp +++ b/src/Plugins/SimplnxCore/test/RequireMinimumSizeFeaturesTest.cpp @@ -2,7 +2,12 @@ #include "SimplnxCore/Filters/RequireMinimumSizeFeaturesFilter.hpp" #include "SimplnxCore/SimplnxCore_test_dirs.hpp" +#include + #include "simplnx/Core/Application.hpp" +#include "simplnx/DataStructure/AttributeMatrix.hpp" +#include "simplnx/DataStructure/DataGroup.hpp" +#include "simplnx/DataStructure/Geometry/ImageGeom.hpp" #include "simplnx/Parameters/ArraySelectionParameter.hpp" #include "simplnx/Parameters/BoolParameter.hpp" #include "simplnx/Parameters/Dream3dImportParameter.hpp" @@ -12,8 +17,6 @@ #include "simplnx/UnitTest/UnitTestCommon.hpp" #include "simplnx/Utilities/Parsing/HDF5/IO/FileIO.hpp" -#include - #include #include #include @@ -168,6 +171,195 @@ TEST_CASE("SimplnxCore::RequireMinimumSizeFeatures: Small IN100 Pipeline", "[Sim UnitTest::CheckArraysInheritTupleDims(dataStructure); } +TEST_CASE("SimplnxCore::RequireMinimumSizeFeatures: Strict Feature Ids", "[SimplnxCore][RequireMinimumSizeFeatures][StrictFeatureIds]") +{ + UnitTest::LoadPlugins(); + DataStructure dataStructure; + const DataPath geometryPath({"Strict Geometry"}); + const DataPath cellPath = geometryPath.createChildPath("Cells"); + const DataPath featurePath = geometryPath.createChildPath("Features"); + const DataPath idsPath = cellPath.createChildPath("Ids"); + const DataPath otherPath = geometryPath.createChildPath("Other Cells").createChildPath("Other Values"); + const ShapeType shape{1, 1, 4}; + auto* imageGeomPtr = ImageGeom::Create(dataStructure, geometryPath.getTargetName()); + REQUIRE(imageGeomPtr != nullptr); + imageGeomPtr->setDimensions({4, 1, 1}); + auto* cellsPtr = AttributeMatrix::Create(dataStructure, "Cells", shape, imageGeomPtr->getId()); + REQUIRE(cellsPtr != nullptr); + imageGeomPtr->setCellData(*cellsPtr); + auto* featuresPtr = AttributeMatrix::Create(dataStructure, "Features", {4}, imageGeomPtr->getId()); + REQUIRE(featuresPtr != nullptr); + auto* otherPtr = AttributeMatrix::Create(dataStructure, "Other Cells", shape, imageGeomPtr->getId()); + REQUIRE(otherPtr != nullptr); + REQUIRE(UnitTest::CreateTestDataArray(dataStructure, "Ids", shape, {1}, cellsPtr->getId()) != nullptr); + REQUIRE(UnitTest::CreateTestDataArray(dataStructure, "Other Values", shape, {1}, otherPtr->getId()) != nullptr); + REQUIRE(UnitTest::CreateTestDataArray(dataStructure, "Feature Values", {4}, {1}, featuresPtr->getId()) != nullptr); + REQUIRE(UnitTest::CreateTestDataArray(dataStructure, "Cell Feature Values", shape, {1}, cellsPtr->getId()) != nullptr); + auto* elsewherePtr = DataGroup::Create(dataStructure, "Elsewhere"); + REQUIRE(elsewherePtr != nullptr); + auto* externalFeaturesPtr = AttributeMatrix::Create(dataStructure, "External Features", {4}, elsewherePtr->getId()); + REQUIRE(externalFeaturesPtr != nullptr); + REQUIRE(UnitTest::CreateTestDataArray(dataStructure, "Feature Values", {4}, {1}, externalFeaturesPtr->getId()) != nullptr); + + RequireMinimumSizeFeaturesFilter filter; + auto args = filter.getDefaultArguments(); + args.insertOrAssign(RequireMinimumSizeFeaturesFilter::k_ImageGeomPath_Key, std::make_any(geometryPath)); + args.insertOrAssign(RequireMinimumSizeFeaturesFilter::k_FeatureIdsPath_Key, std::make_any(idsPath)); + args.insertOrAssign(RequireMinimumSizeFeaturesFilter::k_FeatureNumCellsPath_Key, std::make_any(featurePath.createChildPath("Feature Values"))); + + int32 expectedCode = 0; + DataPath offendingPath; + SECTION("Correct Cell and Feature Attribute Matrices") + { + } + SECTION("Feature Ids in a second Attribute Matrix with the same tuple shape") + { + offendingPath = otherPath; + args.insertOrAssign(RequireMinimumSizeFeaturesFilter::k_FeatureIdsPath_Key, std::make_any(offendingPath)); + expectedCode = -5562; + } + SECTION("Feature array outside the selected geometry") + { + offendingPath = DataPath({"Elsewhere", "External Features", "Feature Values"}); + args.insertOrAssign(RequireMinimumSizeFeaturesFilter::k_FeatureNumCellsPath_Key, std::make_any(offendingPath)); + expectedCode = -5563; + } + SECTION("Feature array under a plain DataGroup") + { + REQUIRE(UnitTest::CreateTestDataArray(dataStructure, "Ungrouped Features", {4}, {1}, elsewherePtr->getId()) != nullptr); + offendingPath = DataPath({"Elsewhere", "Ungrouped Features"}); + args.insertOrAssign(RequireMinimumSizeFeaturesFilter::k_FeatureNumCellsPath_Key, std::make_any(offendingPath)); + expectedCode = -5563; + } + SECTION("Feature array in the Cell Attribute Matrix") + { + offendingPath = cellPath.createChildPath("Cell Feature Values"); + args.insertOrAssign(RequireMinimumSizeFeaturesFilter::k_FeatureNumCellsPath_Key, std::make_any(offendingPath)); + expectedCode = -5563; + } + SECTION("Feature Ids through a shared Cell Attribute Matrix") + { + REQUIRE(dataStructure.setAdditionalParent(cellsPtr->getId(), elsewherePtr->getId())); + args.insertOrAssign(RequireMinimumSizeFeaturesFilter::k_FeatureIdsPath_Key, std::make_any(DataPath({"Elsewhere", "Cells", "Ids"}))); + } + SECTION("Read-only Feature Phases in another Attribute Matrix") + { + args.insertOrAssign(RequireMinimumSizeFeaturesFilter::k_ApplySinglePhase_Key, std::make_any(true)); + args.insertOrAssign(RequireMinimumSizeFeaturesFilter::k_FeaturePhasesPath_Key, std::make_any(otherPath)); + } + const auto result = filter.preflight(dataStructure, args); + if(expectedCode == 0) + { + REQUIRE(result.outputActions.valid()); + for(const auto& modification : result.outputActions.value().modifiedActions) + { + CHECK(modification.modifiedPath.getParent() != DataPath({"Elsewhere", "Cells"})); + } + } + else + { + REQUIRE(result.outputActions.invalid()); + REQUIRE(result.outputActions.errors().size() == 1); + CHECK(result.outputActions.errors().front().code == expectedCode); + CHECK(result.outputActions.errors().front().message.find(offendingPath.toString()) != std::string::npos); + CHECK(result.outputActions.errors().front().message.find(cellPath.toString()) != std::string::npos); + CHECK(result.outputActions.errors().front().message.find(geometryPath.toString()) != std::string::npos); + } + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + +TEST_CASE("SimplnxCore::RequireMinimumSizeFeatures: Shared Cell Paths and Selected Feature Matrix", "[SimplnxCore][RequireMinimumSizeFeatures][StrictFeatureIds]") +{ + UnitTest::LoadPlugins(); + const bool useAliasForIds = GENERATE(false, true); + CAPTURE(useAliasForIds); + DataStructure dataStructure; + const DataPath geometryPath({"Alias Geometry"}); + const DataPath cellPath = geometryPath.createChildPath("Cells"); + const DataPath aliasCellPath({"Earlier Parent", "Cells"}); + const DataPath featurePath = geometryPath.createChildPath("Features"); + const ShapeType shape{1, 1, 4}; + auto* earlierParentPtr = DataGroup::Create(dataStructure, "Earlier Parent"); + REQUIRE(earlierParentPtr != nullptr); + auto* imageGeomPtr = ImageGeom::Create(dataStructure, geometryPath.getTargetName()); + REQUIRE(imageGeomPtr != nullptr); + imageGeomPtr->setDimensions({4, 1, 1}); + auto* cellsPtr = AttributeMatrix::Create(dataStructure, "Cells", shape, imageGeomPtr->getId()); + REQUIRE(cellsPtr != nullptr); + imageGeomPtr->setCellData(*cellsPtr); + REQUIRE(dataStructure.setAdditionalParent(cellsPtr->getId(), earlierParentPtr->getId())); + REQUIRE(imageGeomPtr->getCellDataPath() == cellPath); + auto* idsPtr = UnitTest::CreateTestDataArray(dataStructure, "Ids", shape, {1}, cellsPtr->getId()); + auto* valuesPtr = UnitTest::CreateTestDataArray(dataStructure, "Values", shape, {1}, cellsPtr->getId()); + auto* ignoredPtr = UnitTest::CreateTestDataArray(dataStructure, "Ignored", shape, {1}, cellsPtr->getId()); + REQUIRE(idsPtr != nullptr); + REQUIRE(valuesPtr != nullptr); + REQUIRE(ignoredPtr != nullptr); + const std::vector originalIds{1, 2, 1, 1}; + for(usize index = 0; index < 4; index++) + { + (*idsPtr)[index] = originalIds[index]; + (*valuesPtr)[index] = static_cast(10 + index); + (*ignoredPtr)[index] = static_cast(100 + index); + } + + // The selected array has another parent; only the selected Feature Attribute Matrix is compacted. + auto* featureValuesPtr = UnitTest::CreateTestDataArray(dataStructure, "Feature Values", {3}, {1}, earlierParentPtr->getId()); + auto* externalMarkerPtr = UnitTest::CreateTestDataArray(dataStructure, "External Marker", {3}, {1}, earlierParentPtr->getId()); + REQUIRE(featureValuesPtr != nullptr); + REQUIRE(externalMarkerPtr != nullptr); + auto* featuresPtr = AttributeMatrix::Create(dataStructure, "Features", {3}, imageGeomPtr->getId()); + REQUIRE(featuresPtr != nullptr); + REQUIRE(dataStructure.setAdditionalParent(featureValuesPtr->getId(), featuresPtr->getId())); + auto* markerPtr = UnitTest::CreateTestDataArray(dataStructure, "Marker", {3}, {1}, featuresPtr->getId()); + REQUIRE(markerPtr != nullptr); + for(usize index = 0; index < 3; index++) + { + (*markerPtr)[index] = static_cast(11 * index); + (*externalMarkerPtr)[index] = static_cast(90 + index); + } + (*featureValuesPtr)[0] = 0; + (*featureValuesPtr)[1] = 3; + (*featureValuesPtr)[2] = 1; + + RequireMinimumSizeFeaturesFilter filter; + auto args = filter.getDefaultArguments(); + args.insertOrAssign(RequireMinimumSizeFeaturesFilter::k_ImageGeomPath_Key, std::make_any(geometryPath)); + args.insertOrAssign(RequireMinimumSizeFeaturesFilter::k_FeatureIdsPath_Key, std::make_any((useAliasForIds ? aliasCellPath : cellPath).createChildPath("Ids"))); + args.insertOrAssign(RequireMinimumSizeFeaturesFilter::k_FeatureNumCellsPath_Key, std::make_any(featurePath.createChildPath("Feature Values"))); + args.insertOrAssign(RequireMinimumSizeFeaturesFilter::k_MinAllowedFeaturesSize_Key, std::make_any(2)); + const auto preflight = filter.preflight(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflight.outputActions); + bool reportsValues = false; + for(const auto& modification : preflight.outputActions.value().modifiedActions) + { + CHECK(modification.modifiedPath.getParent() != aliasCellPath); + reportsValues = reportsValues || modification.modifiedPath == cellPath.createChildPath("Values"); + } + CHECK(reportsValues); + const auto execute = filter.execute(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(execute.result); + for(usize index = 0; index < 4; index++) + { + CHECK((*idsPtr)[index] == 1); + if(index != 1) + { + CHECK((*valuesPtr)[index] == static_cast(10 + index)); + } + } + CHECK(((*valuesPtr)[1] == 10 || (*valuesPtr)[1] == 12)); + CHECK(featuresPtr->getNumberOfTuples() == 2); + REQUIRE(markerPtr->getNumberOfTuples() == 2); + CHECK((*markerPtr)[0] == 0); + CHECK((*markerPtr)[1] == 11); + REQUIRE(externalMarkerPtr->getNumberOfTuples() == 3); + for(usize index = 0; index < 3; index++) + { + CHECK((*externalMarkerPtr)[index] == static_cast(90 + index)); + } + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + TEST_CASE("SimplnxCore::RequireMinimumSizeFeaturesFilter: SIMPL Backwards Compatibility", "[SimplnxCore][RequireMinimumSizeFeaturesFilter][BackwardsCompatibility]") { auto app = Application::GetOrCreateInstance(); diff --git a/src/Plugins/SimplnxCore/vv/ErodeDilateBadDataFilter.md b/src/Plugins/SimplnxCore/vv/ErodeDilateBadDataFilter.md index d9e1e0967e..541e025c1d 100644 --- a/src/Plugins/SimplnxCore/vv/ErodeDilateBadDataFilter.md +++ b/src/Plugins/SimplnxCore/vv/ErodeDilateBadDataFilter.md @@ -16,8 +16,8 @@ |------------------------|--------------------------| | Algorithm Relationship | **Port** — legacy `ErodeDilateBadData.{h,cpp}` was diffed line-by-line against `Algorithms/ErodeDilateBadData.cpp` this pass. Neighbor offsets, boundary-validity checks, vote/tie-break order, and transfer conditions are structurally identical. One divergence, since resolved — see Resolved Defects. | | Oracle (confirmed) | **Class 2 (Reference implementation).** The 28 expected `FeatureIds`/`Misc` arrays (7 direction combinations × 2 operations × 2 iteration counts) are genuine DREAM3D 6.5.171 output, matched element-wise against SIMPLNX and compiled into `ErodeDilateBadDataTest.cpp` as constants so the comparison re-runs in CI without the legacy binary. Confirmed — `(Erode) Expanded` and `(Dilate) Expanded` pass, 28/28 combinations. | -| Code paths enumerated | **10 of 11 exercised.** All 6 face directions (-Z/-Y/-X/+X/+Y/+Z) confirmed hit by instrumentation. Only gap: the `m_ShouldCancel` early exits (Path 11) — no test injects a cancel signal. | -| Tests today | **7 TEST_CASEs, all pass in both in-core and OOC builds** (2283 assertions, identical in each): 1 production-scale exemplar-archive comparison + 2 `GENERATE` parameter sweeps (14 valid runs each over direction × iteration count) + 1 ignored-path test + 2 preflight-error tests + 1 SIMPL backwards-compat. | +| Code paths enumerated | **13 of 14 exercised.** All 6 face directions (-Z/-Y/-X/+X/+Y/+Z) confirmed hit by instrumentation. Only gap: the `m_ShouldCancel` early exits (Path 11) — no test injects a cancel signal. | +| Tests today | **10 TEST_CASEs:** 1 production-scale exemplar-archive comparison + 2 `GENERATE` parameter sweeps (14 valid runs each over direction × iteration count) + 1 ignored-path test + 3 preflight tests (1 warning, 2 errors) + 1 strict-placement test + 1 Cell Attribute Matrix alias test + 1 SIMPL backwards-compat. The 7 tests of the earlier V&V cycle passed in both in-core and OOC builds (2283 assertions, identical in each). | | Exemplar archive | `6_6_erode_dilate_test.tar.gz` — provides `Input Data` plus legacy-generated `Exemplar Bad Data Erode` / `Exemplar Bad Data Dilate` containers on a 189×201×20 Small IN100 slice. Consumed by the `(Erode)` test here and shared with `ErodeDilateMaskTest` and `ErodeDilateCoordinationNumberTest`. SHA512 verified against `test/CMakeLists.txt`. | | Legacy comparison | **Run.** Two independent comparisons: (1) all 28 parameter combinations run through DREAM3D 6.5.171 `PipelineRunner` against an HDF5 twin of the inline fixture and diffed element-wise — **28/28 exact matches** on both `FeatureIds` and `Misc`; (2) the `(Erode)` test compares SIMPLNX against a legacy-generated exemplar at production scale (759,780 cells, 6 cell arrays). | | Bug flags | `ErodeDilateBadDataFilter-D1` (X/Y/Z direction parameters had no effect) — **confirmed and resolved.** One additional hypothesis (Dilate tie-break order) was investigated, found to be a false lead, and reverted — see deviations doc. | @@ -25,7 +25,7 @@ ## Summary -`ErodeDilateBadDataFilter` erodes or dilates voxels with `FeatureId == 0` ("bad data") in an `ImageGeometry`, optionally restricted to any non-empty combination of X, Y, and Z face directions. Verification is **Class 2**: the 28 expected output arrays compiled into `(Erode) Expanded` / `(Dilate) Expanded` are genuine DREAM3D 6.5.171 output for the same fixture, matched element-wise across every operation × direction × iteration combination, and the `(Erode)` test additionally compares against a legacy-generated exemplar archive at production scale. One SIMPLNX-side bug was found and resolved (`ErodeDilateBadDataFilter-D1` — the direction parameters had no effect at all); all 7 tests pass in both in-core and OOC builds with 2283 assertions. +`ErodeDilateBadDataFilter` erodes or dilates voxels with `FeatureId == 0` ("bad data") in an `ImageGeometry`, optionally restricted to any non-empty combination of X, Y, and Z face directions. Verification is **Class 2**: the 28 expected output arrays compiled into `(Erode) Expanded` / `(Dilate) Expanded` are genuine DREAM3D 6.5.171 output for the same fixture, matched element-wise across every operation × direction × iteration combination, and the `(Erode)` test additionally compares against a legacy-generated exemplar archive at production scale. One SIMPLNX-side bug was found and resolved (`ErodeDilateBadDataFilter-D1` — the direction parameters had no effect at all). The 7 tests of the earlier V&V cycle passed in both in-core and OOC builds with 2283 assertions; the suite now has 10 TEST_CASEs. ## Algorithm Relationship @@ -83,7 +83,7 @@ A plausible-looking bug hypothesis (last-bad-neighbor-wins vs. first-bad-neighbo ## Code path coverage -10 of 11 paths exercised. The algorithm has three logical phases: (a) preflight validation, (b) a per-voxel vote/mark scan over face neighbors, and (c) a per-array transfer pass. +13 of 14 paths exercised. The algorithm has three logical phases: (a) preflight validation, (b) a per-voxel vote/mark scan over face neighbors, and (c) a per-array transfer pass. Source: `src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ErodeDilateBadData.cpp` (231 lines). @@ -100,6 +100,9 @@ Source: `src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ErodeDilateB | 9 | (a) Preflight | `!xDirOn && !yDirOn && !zDirOn` → error `-14601` (`k_NoDirectionsError`) | `No Direction` — asserts `invalid()` and `errors()[0].code == -14601` for both operations | | 10 | (a) Preflight | `dims[0] == 0 \|\| dims[1] == 0 \|\| dims[2] == 0` → error `-14602` (`k_NoGeometryDimensionsError`) | `No Dimensions` — directions all **on** so the `-14601` check does not mask this path (the prior revision zeroed them and never reached here); asserts error code is exactly `-14602` | | 11 | (b) Cancel | `m_ShouldCancel` read once per Z-slice in the read-only Feature ID scan, and once before the first morphology pass → early return with the input unchanged. Not read during the passes, which modify the input in place | *Not directly tested. Requires cancel-signal injection; no test sets `m_ShouldCancel` and asserts early termination.* Legacy has no cancel check at all, so SIMPLNX is ahead of legacy here — not a deviation. | +| 12 | (a) Preflight | Cell Feature Ids is not in the selected geometry's Cell Attribute Matrix → error `-14605` | `SimplnxCore::ErodeDilateBadDataFilter: Strict Feature Ids` — Feature Ids in another Attribute Matrix; asserts invalid preflight and exact error code `-14605`. | +| 13 | (a) Preflight | An Attribute Arrays to Ignore entry is not in the selected geometry's Cell Attribute Matrix → error `-14604` | `SimplnxCore::ErodeDilateBadDataFilter: Strict Feature Ids` — ignored array in another Attribute Matrix; asserts invalid preflight and exact error code `-14604`. | +| 14 | (c) Transfer | An ignored array selected through an alias of the Cell Attribute Matrix is excluded from modification | `SimplnxCore::ErodeDilateBadDataFilter: Ignored Array Cell AM Alias` — Cell Attribute Matrix reachable by two paths; asserts the ignored array remains unchanged. | **Per-direction coverage, confirmed by instrumentation:** `Algorithms/ErodeDilateBadData.cpp` was temporarily instrumented with per-face-direction hit counters at (a) the point immediately after the `isValidFaceNeighbor` gate in the vote/mark loop, (b) the point where the Dilate mark / Erode vote condition (`feature > 0`) fires, and (c) the equivalent point in the Erode cleanup loop. Running the full `(Erode) Expanded` + `(Dilate) Expanded` sweep produced non-zero counts for **every one of the 6 directions at all 3 measurement points** — vote/mark loop reached counts `-Z=38 -Y=111 -X=108 +X=106 +Y=64 +Z=97`; Dilate marking fired at `-Z=9 -Y=46 -X=37 +X=35 +Y=16 +Z=44`; Erode voting at `-Z=8 -Y=25 -X=24 +X=24 +Y=8 +Z=32`. The instrumentation was removed afterward; this records the empirical result, not a standing code artifact. @@ -117,11 +120,14 @@ Confirmed correct and deliberately not counted as deviations: | `SimplnxCore::ErodeDilateBadDataFilter(Erode) Expanded` | new-for-V&V | Class 2 small-scale oracle. `GENERATE` over 7 valid direction combinations × 2 iteration counts (14 runs); all-off reported via `SUCCEED`. Both `FeatureIds` and `Misc` asserted against the `k_Exemplars` table; 1039 assertions. Modified this pass: expected arrays regenerated from legacy output (previously byte-identical across direction combinations, per D1), the `Misc` assertion enabled for the first time, and the two duplicated dispatch helpers collapsed into one table lookup. | | `SimplnxCore::ErodeDilateBadDataFilter(Dilate) Expanded` | new-for-V&V | Same sweep and same table, Dilate operation; 1039 assertions. Same modifications as the Erode sweep. | | `SimplnxCore::ErodeDilateBadDataFilter Ignored Path` | new-for-V&V | Confirms an array listed in `IgnoredDataArrayPaths` (`Misc`) is left untouched, for both operations via `GENERATE(k_Dilate, k_Erode)`; 91 assertions. Modified this pass: the prior revision only preflighted, so comparing the DataStructure against a fresh fixture could not fail. It now executes the filter and additionally asserts that `FeatureIds` *did* change, which is what keeps the ignored-path check from passing vacuously. Verified by mutation — emptying the ignore list, or dropping the `execute()` call, each now fails the test. | +| `SimplnxCore::ErodeDilateBadDataFilter: Preflight Feature Data Modification Warning` | kept | Preflight test on the 32-voxel fixture: Dilate, 1 iteration, all directions on, no ignored arrays. Asserts a valid preflight, warning `-14600`, a `Feature Data Modification Warning` preflight value, and no `Stale Arrays` preflight value. Not an oracle test. | | `SimplnxCore::ErodeDilateBadDataFilter No Direction` | new-for-V&V | Preflight-error test: all directions off, geometry otherwise valid, both operations × 2 iteration counts. Asserts `invalid()` and `errors()[0].code == -14601`; 25 assertions. Covers Path 9. | | `SimplnxCore::ErodeDilateBadDataFilter No Dimensions` | new-for-V&V | Preflight-error test: `ImageGeom` dimensions forced to `{0,0,0}`, directions all on, Dilate. Asserts `invalid()` and `errors()[0].code == -14602`; 7 assertions. Covers Path 10. Modified this pass: previously it also zeroed all direction flags, which tripped `-14601` first and left Path 10 unreached. | | `SimplnxCore::ErodeDilateBadDataFilter: SIMPL Backwards Compatibility` | kept | `DYNAMIC_SECTION` over `simpl_conversion/6_5/ErodeDilateBadDataFilter.json` (matched by `Filter_Uuid`) and `simpl_conversion/6_4/ErodeDilateBadDataFilter.json` (matched by `Filter_Name`; that fixture has no UUID field). Loads each legacy pipeline via `Pipeline::FromSIMPLFile`, confirms a single `PipelineFilter` with the right UUID, and checks the converted arguments: `Operation == k_Dilate`, `NumIterations == 5`, `XDirOn/YDirOn/ZDirOn == true`, geometry path `DataPath({"DataContainer"})`, feature-ids path `DataPath({"DataContainer","CellData","TestArray"})`; 27 assertions. `IgnoredDataArrayPaths` verified only by successful pipeline load, matching the pattern in `FillBadDataTest.cpp`. Not an oracle test. | +| `SimplnxCore::ErodeDilateBadDataFilter: Strict Feature Ids` | new-for-V&V | Three SECTIONs: correct Cell Attribute Matrix passes; Feature Ids in another Attribute Matrix returns `-14605`; an ignored array in another Attribute Matrix returns `-14604`. Checks exact error codes and diagnostic paths. Covers Paths 12–13. | +| `SimplnxCore::ErodeDilateBadDataFilter: Ignored Array Cell AM Alias` | new-for-V&V | Cell Attribute Matrix reachable by two paths. Selects the ignored array through the alias path, executes the filter, and checks that Feature Ids change while the ignored array remains unchanged. Covers Path 14. | -All 7 tests pass in both the in-core build (`NX-Com-Qt69-Vtk96-Rel`) and the out-of-core build (`simplnx-ooc-Rel`), with identical assertion counts in each — 2283 total (55 + 1039 + 1039 + 91 + 25 + 7 + 27). +The 7 tests of the earlier V&V cycle passed in both the in-core build (`NX-Com-Qt69-Vtk96-Rel`) and the out-of-core build (`simplnx-ooc-Rel`), with identical assertion counts in each — 2283 total (55 + 1039 + 1039 + 91 + 25 + 7 + 27). ## Exemplar archive diff --git a/src/Plugins/SimplnxCore/vv/FillBadDataFilter.md b/src/Plugins/SimplnxCore/vv/FillBadDataFilter.md index fac8da8093..3f3808e340 100644 --- a/src/Plugins/SimplnxCore/vv/FillBadDataFilter.md +++ b/src/Plugins/SimplnxCore/vv/FillBadDataFilter.md @@ -16,8 +16,8 @@ |------------------------|--------------------------| | Algorithm Relationship | **Rewrite** — 4-phase chunk-sequential CCL+Union-Find (OOC support); legacy used a simpler in-memory approach. Functional behavior preserved. | | Oracle (confirmed) | **Class 2** for `FillBadData_SmallIN100` (`6_5_exemplar.dream3d` from legacy 6.5 pipeline). **Class 1** for Tests 01–13 (hand-authored expected values serialized by a format-conversion script that never runs `FillBadDataFilter`). Circular-oracle concern resolved. | -| Code paths enumerated | 16 of 17 exercised; cancellation remains outside the scope. | -| Tests today | 15 registered cases plus one hidden real-HDF5 slice-boundary oracle with the new-phase option off/on. | +| Code paths enumerated | 20 of 21 exercised; cancellation remains outside the scope. | +| Tests today | 18 registered cases plus two hidden ones: the real-HDF5 slice-boundary oracle with the new-phase option off/on, and the test-data generator, which is a fixture-generation utility, not an oracle. | | Exemplar archive | `6_5_fill_bad_data.tar.gz` — `6_5_input/exemplar.dream3d` (Class 2) + `test_NN_input/expected.dream3d` pairs for Tests 01–07, 11, 13 (Class 1). *(Tests 08–10 and 12 have no fixtures — the numbering is non-contiguous.)* | | Legacy comparison | SmallIN100 in-test (Class 2) — SIMPLNX matches the 6.5.x exemplar element-wise. A separate Test 08 three-way binary A/B was cited by an earlier revision but its working files are unrecoverable, so that claim is **withdrawn** (see deviations). Per-code-path correctness is pinned by the Class 1 analytical fixtures. | | Bug flags | `FillBadDataFilter-B1` (preflight dead-return for `minAllowedDefectSize < 1`) resolved. Resolved: an all-bad-data / enclosed-bad-pocket input previously looped forever in Phase 4 (no fillable neighbor → `count` never reached 0); a no-progress guard now stops with a warning. | @@ -64,7 +64,7 @@ No new defect was found during this OOC recertification. Existing fixes and devi ## Code path coverage -16 of 17 paths exercised. Cancellation is not directly tested. +20 of 21 paths exercised. Cancellation is not directly tested. Source: `Algorithms/FillBadData.cpp` (49 lines), `FillBadDataBFS.cpp` (378 lines), and `FillBadDataCCL.cpp` (903 lines). @@ -87,6 +87,10 @@ Source: `Algorithms/FillBadData.cpp` (49 lines), `FillBadDataBFS.cpp` (378 lines | 15 | Preflight | `minAllowedDefectSize < 1` → `MakePreflightErrorResult(-16500, …)` | `"SimplnxCore::FillBadDataFilter:: Invalid Preflight Min Defect Size"` — inline DataStructure, `minAllowedDefectSize=0`; asserts error code `-16500`. | | 16 | Phase 4 | no fillable neighbor for any remaining bad voxel → no-progress break (warning) | `AllBadData_TerminatesWithoutHang` — all-bad-data slab; asserts the filter returns instead of looping forever. | | 17 | CCL storage | A threshold-sized defect spans rolling slices, and a smaller defect copies a multi-component tuple | `real HDF5 cross-slice defect oracle` — exact full-volume FeatureIds, Phases, and payload | +| 18 | Preflight | Cell Feature Ids is not in the selected geometry's Cell Attribute Matrix → error `-16501` | `SimplnxCore::FillBadDataFilter: Strict Feature Ids` — asserts invalid preflight and exact code `-16501`. | +| 19 | Preflight | Store Defects as New Phase is on and Cell Phases is not in the selected geometry's Cell Attribute Matrix → error `-16502` | `SimplnxCore::FillBadDataFilter: Strict Feature Ids` — asserts invalid preflight and exact code `-16502`; also checks valid Cell Phases and unused external Cell Phases. | +| 20 | Preflight | An Attribute Arrays to Ignore entry is not in the selected geometry's Cell Attribute Matrix → error `-16503` | `SimplnxCore::FillBadDataFilter: Strict Feature Ids` — asserts invalid preflight and exact code `-16503`. | +| 21 | Phase 4 — Fill | An ignored array selected through an alias of the Cell Attribute Matrix is excluded from modification | `SimplnxCore::FillBadDataFilter: Ignored Array Cell AM Alias` — checks unchanged ignored values for each in-memory algorithm scenario. | ## Test inventory @@ -102,12 +106,16 @@ Source: `Algorithms/FillBadData.cpp` (49 lines), `FillBadDataBFS.cpp` (378 lines | `SimplnxCore::FillBadData::Test07_DefectsAtBoundaries` | kept | Resident-store algorithm-equivalence test. threshold=20. Regions at image boundary — exercises Phase 1 CCL boundary handling. Class 1. | | `SimplnxCore::FillBadData::Test11_NeighborTieBreaking` | kept | Resident-store algorithm-equivalence test. threshold=10. Tie-break via scan order. Class 1. | | `SimplnxCore::FillBadData::Test13_StoreAsNewPhase` | kept | Resident-store algorithm-equivalence test. threshold=20, storeAsNewPhase=true. Only test for `cellPhasesPtr ≠ nullptr` path (Path 8). Class 1. | +| `SimplnxCore::FillBadDataFilter: Preflight Feature Data Modification Warning` | kept | Inline 2×2×2 DataStructure, threshold=1, storeAsNewPhase=false, no ignored arrays. Asserts a valid preflight, warning `-14600`, a `Feature Data Modification Warning` preflight value, and no `Stale Arrays` preflight value. Not an oracle test. | | `SimplnxCore::FillBadDataFilter:: Invalid Preflight Min Defect Size` | kept | Inline DataStructure (no file load). `minAllowedDefectSize=0` → asserts invalid **and** error code `-16500`. Covers Path 15. | | `SimplnxCore::FillBadData::AllBadData_TerminatesWithoutHang` | kept | Inline 3×3×1 all-bad-data slab (no good neighbor). Asserts the filter returns rather than looping forever. Covers Path 16 (no-progress guard). | | `SimplnxCore::FillBadDataFilter: SIMPL Backwards Compatibility` | kept | SIMPL 6.4 + 6.5 via `DYNAMIC_SECTION`. UUID + arg-key + value assertions only. Not an oracle test. | | `real HDF5 cross-slice defect oracle` | new-for-V&V | Independent HDF5 boundary outputs; 1053 assertions across the configured options. | | `200x200x200 Correctness` | kept | Existing large generated-data correctness test; algorithm scopes keep its stores resident. | | `200x200x200 Ignored Arrays` | kept | Existing ignored-array preservation regression on resident stores. | +| `SimplnxCore::FillBadDataFilter: Strict Feature Ids` | new-for-V&V | Six SECTIONs: correct Cell Attribute Matrix; Feature Ids in another Attribute Matrix (`-16501`); ignored array in another Attribute Matrix (`-16503`); modified Cell Phases in another Attribute Matrix (`-16502`); unused Cell Phases in another Attribute Matrix; modified Cell Phases in the Cell Attribute Matrix. Checks preflight validity, exact error codes, and diagnostic paths. Covers Paths 18–20. | +| `SimplnxCore::FillBadDataFilter: Ignored Array Cell AM Alias` | new-for-V&V | Cell Attribute Matrix reachable by two paths. Selects the ignored array through the alias path and runs each in-memory algorithm scenario. Checks that Feature Ids change while the ignored array remains unchanged. Covers Path 21. | +| `Generate Test Data` | kept | Hidden (`[.GenerateTestData]`). Creates a 200×200×200 fixture with a full-plane defect and writes `generated_test_data/fill_bad_data/input.dream3d` under the binary test-output directory. Checks that buffer writes and file output succeed. Does not execute FillBadData or compare algorithm output. | OOC recertification (2026-09-21): 15/15 target CTest entries pass in both DREAM3DNX builds. The hidden `real HDF5 cross-slice defect oracle` passes 1053 assertions in the OOC binary. A three-cell defect spans three Z slices and equals the retention threshold. A disconnected one-cell defect must be filled. Exact FeatureIds, Phases, and a two-component payload are checked for every voxel, with StoreAsNewPhase off and on. HDF5 input stores and target-only OOC dispatch counters verify the CCL path. The expected preflight warning -14600 remains present. Existing upstream/develop oracles and tolerances are preserved. No production algorithm or historical sign-off is changed, and no fresh legacy binary run is claimed. diff --git a/src/Plugins/SimplnxCore/vv/RemoveFlaggedFeaturesFilter.md b/src/Plugins/SimplnxCore/vv/RemoveFlaggedFeaturesFilter.md index addc38a613..7e699c3c63 100644 --- a/src/Plugins/SimplnxCore/vv/RemoveFlaggedFeaturesFilter.md +++ b/src/Plugins/SimplnxCore/vv/RemoveFlaggedFeaturesFilter.md @@ -14,9 +14,9 @@ | Aspect | Current state | |------------------------|--------------------------| | Algorithm Relationship | **Minor changes** - SIMPLNX merges legacy `RemoveFlaggedFeatures` (Processing) and `ExtractFlaggedFeatures` (Sampling) into one filter with an operation selector. The removal and fill loop is a port that now lives in the shared `FeatureRemovalUtilities`; the extract path is rebuilt on the `ComputeFeatureRect` and `CropImageGeometry` sub-filters. | -| Oracle (confirmed) | **Class 1 analytical + Class 4 invariants** - three hand-derived fixtures (5x2x1, 4x4x1, 3x3x3) pin every filled cell to the exact neighbor it was copied from, plus a Small IN100 invariant test that fills 8,535 cells. Encoded in `test/RemoveFlaggedFeaturesTest.cpp`; all 18 test cases pass in `simplnx-Rel` and `simplnx-ooc-Rel`. | -| Code paths enumerated | 23 of 31 paths are assertion-covered. The 8 uncovered paths are cancellation and defensive sub-filter failure paths. | -| Tests today | 18 test cases: 8 Class 1 analytical (remove, fill, extract, extract-then-remove), 4 execute-error, 1 preflight-error, 1 preflight-warning, 1 fill-ignore-list warning, 1 empty-feature warning, 1 Class 4 Small IN100 invariant, 1 SIMPL 6.4/6.5 conversion. The 3 original tests are retired. **OOC caveat:** the `simplnx-ooc-Rel` preset registers no OOC backend, so that pass certifies compile and run in that configuration, not out-of-core data-path behavior. | +| Oracle (confirmed) | **Class 1 analytical + Class 4 invariants** - three hand-derived fixtures (5x2x1, 4x4x1, 3x3x3) pin every filled cell to the exact neighbor it was copied from, plus a Small IN100 invariant test that fills 8,535 cells. Encoded in `test/RemoveFlaggedFeaturesTest.cpp`; all 28 test cases pass in `simplnx-Rel` and `simplnx-ooc-Rel`. | +| Code paths enumerated | 25 of 34 paths are assertion-covered. The 9 uncovered paths are cancellation and defensive failure paths. | +| Tests today | 28 test cases: 8 Class 1 analytical (remove, fill, extract, extract-then-remove), 4 execute-error, 1 preflight-error, 1 preflight-warning, 1 fill-ignore-list warning, 1 empty-feature warning, 1 Class 4 Small IN100 invariant, 2 strict Feature Ids (preflight placement, shared Cell paths), 1 SIMPL 6.4/6.5 conversion, 5 kept from #1568, #1748 and #1760, and 3 original algorithm tests. The 3 original algorithm tests are kept next to their Class 1 replacements. **OOC caveat:** the `simplnx-ooc-Rel` preset registers no OOC backend, so that pass certifies compile and run in that configuration, not out-of-core data-path behavior. | | Exemplar archive | No output archive. The inline oracle needs none. The Class 4 test reads the shared input archive `6_5_test_data_1_v2.tar.gz` (SHA512 `585b51ba...3027d6c`, sidecar exists). | | Legacy comparison | **Run (2026-09-02)** - DREAM3D 6.5.171 and SIMPLNX were run on the 5x2x1, 4x4x1 and 3x3x3 fixtures with fill on and off, and on the 4x4x1 extract. Every removal array (25 array comparisons) matched the oracle and each other exactly. The extract geometry, origin and cell arrays matched. Seven deviations are documented. Six are bugs, all present in SIMPLNX before this branch and three (D3, D4, D6) shared with 6.5.171. D7 is an algorithmic choice. | | Bug flags | `RemoveFlaggedFeaturesFilter-D1` through `-D6`. D1 and D2 are hangs that only SIMPLNX had; D3 and D4 are shared with 6.5.171; D5 is SIMPLNX only; D6 affects both differently. All are fixed in SIMPLNX on this branch (D1 was fixed in v7.4.2). | @@ -95,7 +95,7 @@ Five review passes were run on the change set only (adversarial, nit-picky senio ## Code path coverage -23 of 31 paths are assertion-covered. The uncovered paths are cancellation and defensive failure paths of the sub-filters and the compaction helper. +25 of 34 paths are assertion-covered. The uncovered paths are cancellation paths and defensive failure paths of the sub-filters, the marking step and the compaction helper. Source: `src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/RemoveFlaggedFeatures.cpp` and `src/Plugins/SimplnxCore/src/SimplnxCore/utils/FeatureRemovalUtilities.cpp`, plus preflight in `Filters/RemoveFlaggedFeaturesFilter.cpp`. @@ -103,7 +103,7 @@ Phases: (a) preflight, (b) extract (bounding boxes, then one crop per flagged fe | # | Phase | Path | Test case | |---|---|---|---| -| 1 | (a) Preflight | Parent of the flag array is not an Attribute Matrix -> error `-9892` | `Preflight Error - flag array parent is not an Attribute Matrix (-9892)` | +| 1 | (a) Preflight | Parent of the flag array is not an Attribute Matrix directly under the Image Geometry -> error `-9895` | `Preflight Error - flag array parent is not an Attribute Matrix (-9895)`; `Strict Feature Ids` (flag array outside the geometry, under a plain `DataGroup`, and in the Cell Attribute Matrix) | | 2 | (b) Extract | Temporary `tempBounds` array is created by the `ComputeFeatureRect` sub-filter and deleted by the algorithm before the first crop, so no extracted geometry carries it | `Class 1 Oracle - Extract crops the bounding box of each flagged feature` (asserts `tempBounds` is absent from the source and from the extracted geometry) | | 3 | (a) Preflight | *Remove* or *Extract then Remove* -> cell and feature groups marked modified, NeighborLists scheduled for deletion, one warning `-5558` | `Preflight Warning - NeighborLists are removed (-5558)` (`Functionality = 0`, `Functionality = 2`) | | 4 | (a) Preflight | *Extract* -> no NeighborList warning, NeighborLists kept | `Preflight Warning - NeighborLists are removed (-5558)` (`Extract alone does not remove NeighborLists`) | @@ -134,6 +134,9 @@ Phases: (a) preflight, (b) extract (bounding boxes, then one crop per flagged fe | 26 | (d) Fill | Cancel during a pass -> return | *Not directly tested. Requires cancel-signal injection.* | | 27 | (e) Compaction | Surviving features renumbered contiguously from 1 in input order; feature arrays compacted in place; NeighborLists deleted | Every remove test (`CheckFeatureArraysCompacted`), `Class 4 Invariants - Small IN100 remove small features with fill` | | 28 | (e) Compaction | `RemoveInactiveObjects` fails -> error `-45434` | *Not directly tested. Requires a feature group whose arrays do not match the flag count, which the preflight tuple check rejects; defensive.* | +| 29 | (a) Preflight | Cell Feature Ids is not in the selected geometry's Cell Attribute Matrix (every operation) -> error `-9893` | `Strict Feature Ids` (`Feature Ids in a second Attribute Matrix with the same tuple shape`, for all three operations). Asserts invalid preflight, the exact code and the paths in the message. The `Feature Ids through a shared Cell Attribute Matrix` section checks that a second path to the same Cell Attribute Matrix is accepted. | +| 30 | (a) Preflight | *Remove* or *Extract then Remove* with fill on, and an Attribute Arrays to Ignore entry is not in the selected geometry's Cell Attribute Matrix -> error `-9894` | `Strict Feature Ids` (`Ignored array in a second Attribute Matrix`: `-9894` for *Remove* and *Extract then Remove*, valid for *Extract*; `Unused ignore list with fill disabled`: valid) | +| 31 | (d) Fill, (e) Compaction | Feature Ids or ignored arrays selected through a second path to the Cell Attribute Matrix -> modifications are reported on the geometry's Cell path, ignored arrays are untouched, and only the selected Feature Attribute Matrix is compacted | `Shared Cell Paths and Selected Feature Matrix` (Feature Ids selected through each path) | ## Test inventory @@ -146,22 +149,29 @@ Phases: (a) preflight, (b) extract (bounding boxes, then one crop per flagged fe | `SimplnxCore::RemoveFlaggedFeaturesFilter: Class 1 Oracle - Fill majority vote and second iteration (3x3x3)` | new-for-V&V | Fixture E; asserts all 27 `FeatureIds`, the five copy sources including the two-pass chain into cell 0, ignored array and compaction. 19 assertions. | | `SimplnxCore::RemoveFlaggedFeaturesFilter: Execute Error - all features flagged (-45433)` | new-for-V&V | `DYNAMIC_SECTION` for fill off and on; asserts the exact code and that no array was modified. 21 assertions. | | `SimplnxCore::RemoveFlaggedFeaturesFilter: Execute Error - FeatureId out of range (-45435)` | new-for-V&V | Sections for a value equal to the tuple count and for a negative value; asserts the code and that `FeatureIds` is unmodified. Covers D4. 15 assertions. | -| `SimplnxCore::RemoveFlaggedFeaturesFilter: Execute Error - Feature Ids tuple count differs from the geometry (-45437)` | new-for-V&V | A 4-tuple Feature Ids array outside the 10-cell geometry; asserts the code and that the array is unmodified. Covers D4. 9 assertions. | +| `SimplnxCore::RemoveFlaggedFeaturesFilter: Execute Error - Feature Ids tuple count differs from the geometry (-45437)` | new-for-V&V | Geometry X dimension is increased by one so that Feature Ids no longer has one tuple per cell; asserts the code and that the array is unmodified. Covers D4. 9 assertions. | | `SimplnxCore::RemoveFlaggedFeaturesFilter: Execute Error - no fill progress (-45436)` | new-for-V&V | 4x1x1 where the only unflagged feature owns no cell; asserts the code and that all cells are left at -1. Covers D3. 8 assertions. | | `SimplnxCore::RemoveFlaggedFeaturesFilter: Fill cannot ignore the Feature Ids array (-45438 warning)` | new-for-V&V | Fixture A with the Feature Ids array in the ignore list; asserts exactly one warning `-45438` and the same output as the issue #1698 test. Hung before this branch. Covers D3. 18 assertions. | | `SimplnxCore::RemoveFlaggedFeaturesFilter: Class 1 Oracle - Extract crops the bounding box of each flagged feature` | new-for-V&V | Fixture B extract; asserts the original is untouched, the new geometry's dimensions (2,1,1), origin (1,3,0), spacing, all three cell arrays, the carried feature Attribute Matrix, and that `tempBounds` is absent from both the source and the extracted geometry. 35 assertions. | | `SimplnxCore::RemoveFlaggedFeaturesFilter: Class 1 Oracle - Extract zero-pads names and handles several features` | new-for-V&V | 12-feature 1D fixture; asserts two-digit padding (`-03`, `-11`), per-feature dimensions, origin and cell values, and that unflagged features produce no geometry. 37 assertions. | | `SimplnxCore::RemoveFlaggedFeaturesFilter: Extract - flagged feature with no cells is skipped with a warning` | new-for-V&V | Two flagged features own no cell; asserts exactly one warning `-53905` that names both, that feature 3 is still extracted and that features 4 and 5 produce no geometry. Covers D6. 27 assertions. | | `SimplnxCore::RemoveFlaggedFeaturesFilter: Class 1 Oracle - Extract then Remove with fill` | new-for-V&V | Fixture B; asserts the extracted geometry (from the unmodified input) and the filled original in one run. 30 assertions. | -| `SimplnxCore::RemoveFlaggedFeaturesFilter: Preflight Error - flag array parent is not an Attribute Matrix (-9892)` | new-for-V&V | Flag array placed in a plain `DataGroup`; asserts invalid preflight and the code. 5 assertions. | +| `SimplnxCore::RemoveFlaggedFeaturesFilter: Preflight Error - flag array parent is not an Attribute Matrix (-9895)` | new-for-V&V | Flag array placed in a plain `DataGroup`; asserts invalid preflight and the code. 5 assertions. | | `SimplnxCore::RemoveFlaggedFeaturesFilter: Preflight Warning - NeighborLists are removed (-5558)` | new-for-V&V | `DYNAMIC_SECTION` for *Remove* and *Extract then Remove* asserts exactly one warning with code `-5558` and that the NeighborList is gone after execution; a third section asserts *Extract* emits no warning and keeps it. 23 assertions. | | `SimplnxCore::RemoveFlaggedFeaturesFilter: Class 4 Invariants - Small IN100 remove small features with fill` | new-for-V&V | Loads `6_5_test_data_1_v2`, flags the 275 features with fewer than 100 cells, fills, and checks the six invariants listed in the Oracle section over 1,000,000 cells with aggregated counts. 751 assertions. | +| `SimplnxCore::RemoveFlaggedFeaturesFilter: Strict Feature Ids` | new-for-V&V | Inline 4x1x1 geometry with a second Cell-shaped Attribute Matrix and an external Feature Attribute Matrix. `GENERATE` over the three operations, with sections for the correct paths, `-9893`, three `-9895` placements, a shared Cell Attribute Matrix, `-9894`, and an unused ignore list with fill off. Asserts the exact code and that the message names the offending array, the Cell Attribute Matrix and the geometry. Covers paths 1, 29 and 30. | +| `SimplnxCore::RemoveFlaggedFeaturesFilter: Shared Cell Paths and Selected Feature Matrix` | new-for-V&V | Cell Attribute Matrix with a second parent, and a flag array that also lives outside the geometry. Runs *Remove* with fill, with Feature Ids selected through each path. Asserts that no modification is reported on the alias path, that the ignored array is unchanged, that only the selected Feature Attribute Matrix is compacted, and that the external array keeps its 3 tuples. Covers path 31. | +| `SimplnxCore::RemoveFlaggedFeaturesFilter: Preflight Stale Arrays Only When Modifying` | kept | Added by #1748. Asserts that preflight reports `Stale Arrays` for *Remove* and *Extract then Remove*, and not for *Extract*. | +| `SimplnxCore::RemoveFlaggedFeaturesFilter: Extraction progress completes` | kept | Added by #1760. Fixture B extract; asserts that the `Extracting Features:` progress never decreases, reaches 100 before the success message, and that the extracted Feature Ids are correct. | +| `SimplnxCore::RemoveFlaggedFeaturesFilter: Extraction progress does not complete after cancellation` | kept | Added by #1760. Sets the cancel flag on the first extraction progress message; asserts error `-1`, no extracted geometry, progress below 100 and no success message. | +| `SimplnxCore::RemoveFlaggedFeatures: fill direct and scanline parity` | kept | Added by #1568. Runs *Remove* with fill under each in-memory algorithm scenario; asserts the expected Feature Ids, copied multi-component and bool cell arrays, and compacted feature values, and that every scenario gives the same output. | +| `SimplnxCore::RemoveFlaggedFeatures: fill terminates with background zeros present` | kept | Added by #1568. Fill with background cells under a 30-second watchdog; asserts that the fill finishes, that no negative ids remain and that only the two removed cells change. | | `SimplnxCore::RemoveFlaggedFeaturesFilter: SIMPL Backwards Compatibility` | kept | Two `DYNAMIC_SECTION`s (`SIMPL 6.5 (UUID)`, `SIMPL 6.4 (Filter_Name)`); modified this cycle to also assert the two converted ignored-array paths. 23 assertions. | -| `SimplnxCore::RemoveFlaggedFeatures: Test Remove Algorithm` | retired | Ran *Remove* with fill off on the 4x4x1 fixture and checked `FeatureIds` and the compacted feature array. Replaced by `Class 1 Oracle - Remove without fill` (fixture B), which adds the cell-array and ignored-array checks. The fill path was never exercised. | -| `SimplnxCore::RemoveFlaggedFeatures: Test Extract Algorithm` | retired | Checked only two `FeatureIds` values in the extracted geometry. Replaced by `Class 1 Oracle - Extract crops the bounding box of each flagged feature`, which also checks dimensions, origin, spacing, every cell array and the feature Attribute Matrix. | -| `SimplnxCore::RemoveFlaggedFeatures: Test Extract then Remove Algorithm` | retired | Fill off; same two-value extract check. Replaced by `Class 1 Oracle - Extract then Remove with fill`. | +| `SimplnxCore::RemoveFlaggedFeatures: Test Remove Algorithm` | kept (legacy) | Runs *Remove* with fill off on the 4x4x1 fixture and checks `FeatureIds` and the compacted feature array. Superseded by `Class 1 Oracle - Remove without fill` (fixture B), which adds the cell-array and ignored-array checks. It does not run the fill path. | +| `SimplnxCore::RemoveFlaggedFeatures: Test Extract Algorithm` | kept (legacy) | Checks only two `FeatureIds` values in the extracted geometry. Superseded by `Class 1 Oracle - Extract crops the bounding box of each flagged feature`, which also checks dimensions, origin, spacing, every cell array and the feature Attribute Matrix. | +| `SimplnxCore::RemoveFlaggedFeatures: Test Extract then Remove Algorithm` | kept (legacy) | Fill off; same two-value extract check. Superseded by `Class 1 Oracle - Extract then Remove with fill`. | -There are 18 active `TEST_CASE`s: 1 kept (modified) and 17 new for this V&V cycle; 3 retired. `KeepRemoveRankedFeaturesTest.cpp` gained one section (`Fill terminates when the input has background cells`) because that filter shares the fill utility and inherited D2. +There are 28 active `TEST_CASE`s: 19 new for V&V, 1 kept and modified this cycle, 5 kept from other PRs (#1568, #1748, #1760), and 3 legacy algorithm tests kept next to their Class 1 replacements. `KeepRemoveRankedFeaturesTest.cpp` gained one section (`Fill terminates when the input has background cells`) because that filter shares the fill utility and inherited D2. ## Exemplar archive diff --git a/src/Plugins/SimplnxCore/vv/RequireMinNumNeighborsFilter.md b/src/Plugins/SimplnxCore/vv/RequireMinNumNeighborsFilter.md index 4f72f911c8..f806c824ab 100644 --- a/src/Plugins/SimplnxCore/vv/RequireMinNumNeighborsFilter.md +++ b/src/Plugins/SimplnxCore/vv/RequireMinNumNeighborsFilter.md @@ -15,8 +15,8 @@ |------------------------|--------------------------| | Algorithm Relationship | **Minor changes** - the NX port retains the legacy selection, coarsening, and feature-removal while incorporating bug fixes and minor code improvements. | | Oracle (confirmed) | **Class 1 analytical + Class 4 invariants** - two mode-specific 4x1x1 inputs and one common 6x6x6 input check exact cell copying, phase selection, iterative coarsening, and feature compaction. | -| Code paths enumerated | 13 of 18 paths are assertion-covered; cancellation and defensive error paths are not covered. | -| Tests today | 11 active test cases: two analytical oracle tests, four execute-error tests, negative-FeatureId reassignment, two preflight errors, the Small IN100 regression, and SIMPL 6.4/6.5 conversion; two stale disabled tests are retired. | +| Code paths enumerated | 16 of 18 active paths are assertion-covered; cancellation and defensive error paths are not covered. | +| Tests today | 14 active test cases: two analytical oracle tests, four execute-error tests, negative-FeatureId reassignment, three preflight errors, two strict/shared-path tests, the Small IN100 regression, and SIMPL 6.4/6.5 conversion; two stale disabled tests are retired. | | Exemplar archive | The inline oracle needs no output archive; retained input archive `6_5_test_data_1_v2.tar.gz` has SHA512 `585b51ba...3027d6c` and a provenance sidecar. | | Legacy comparison | **Complete (2026-07-31)** - DREAM3D 6.5.171 and SIMPLNX were run on the 4x1x1 and 6x6x6 analytical fixtures in all-phases and single-phase modes; all 22 array comparisons and 2,646 exact values matched the independent expectations. | | Bug flags | `RequireMinNumNeighborsFilter-D1`, `RequireMinNumNeighborsFilter-D2`, `RequireMinNumNeighborsFilter-D3` | @@ -153,18 +153,20 @@ Because feature 5 owns no cells, both modes have identical final cell arrays eve ## Code path coverage -13 of 18 paths are assertion-covered. Cancellation and defensive infrastructure/error paths are documented gaps. +16 of 18 active paths are assertion-covered. Cancellation and defensive infrastructure/error paths are documented gaps. -Source: `src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/RequireMinNumNeighbors.cpp` (362 lines), plus preflight validation in `Filters/RequireMinNumNeighborsFilter.cpp`. +Source: `src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/RequireMinNumNeighbors.cpp`, shared reassignment in `Algorithms/FillBadVoxels.cpp`, and preflight validation in `Filters/RequireMinNumNeighborsFilter.cpp`. + +Rows 4, 15, and 16 are retired and excluded from the active-path count; their former checks are absent from the current implementation. The filter selects features to retain, marks rejected-feature voxels, iteratively copies a dominant face neighbor into each rejected voxel, and removes inactive feature tuples. | # | Phase | Path | Test case | |---|---|---|---| | 1 | Preflight | Feature-level `NumNeighbors` and, when enabled, `FeaturePhases` tuple counts disagree - return `-252`. | `SimplnxCore::RequireMinNumNeighborsFilter: Preflight Error - tuple count mismatch (-252)` | -| 2 | Preflight | FeatureIds tuple count differs from the selected Image Geometry cell count - return `-55571`. | `SimplnxCore::RequireMinNumNeighborsFilter: Preflight Error - FeatureIds tuple count mismatch (-55571)` | +| 2 | Preflight | FeatureIds is not in the selected Image Geometry's Cell Attribute Matrix - return `-55575`. | `SimplnxCore::RequireMinNumNeighborsFilter: Preflight Error - FeatureIds outside Cell Attribute Matrix (-55575)` | | 3 | Preflight | NeighborList-removal preflight warning is produced when feature NeighborLists are present. | `SimplnxCore::RequireMinNumNeighborsFilter` - asserts only one `-5558` warning. | -| 4 | Setup | Cell-data array discovery fails - return `-5556`. | *Not directly tested. Defensive check.* | +| 4 | Retired | The former cell-data discovery error check is absent. `FillBadVoxels` uses `GenerateDataArrayList` without this error return. | Not an active path. | | 5 | Selection | Single-phase mode names a phase absent from `FeaturePhases` - return `-5555`. | `SimplnxCore::RequireMinNumNeighborsFilter: Execute Error - unavailable phase (-5555)` | | 6 | Selection | Every eligible feature is below the neighbor threshold - return `-55569` before mutation. | `SimplnxCore::RequireMinNumNeighborsFilter: Execute Error - all features rejected (-55569)` | | 7 | Selection | All-phase or selected-phase feature meets threshold; nonselected phases remain active, including a zero-cell feature below the threshold. | `SimplnxCore::RequireMinNumNeighborsFilter: Analytical Oracle` (`ApplyToSinglePhase=0/1`) and `SimplnxCore::RequireMinNumNeighborsFilter: Discriminating 6x6x6 Analytical Fixture` | @@ -175,10 +177,13 @@ The filter selects features to retain, marks rejected-feature voxels, iterativel | 12 | Reassignment | Negative Feature IDs skip initial marking and are reassigned from valid face neighbors. | `SimplnxCore::RequireMinNumNeighborsFilter: Execute - negative feature ID is reassigned` | | 13 | Reassignment | Negative Feature IDs remain but an iteration finds no non-negative face neighbor to copy - return `-55572` instead of looping indefinitely. | `SimplnxCore::RequireMinNumNeighborsFilter: Execute Error - no coarsening progress (-55572)` | | 14 | Copy | A rejected cell copies every nonignored cell-data tuple from its selected neighbor while ignored arrays remain unchanged. | Both analytical oracle tests; the 6x6x6 fixture checks scalar `int32`, three-component `float32`, ignored `int32`, and `FeatureIds` tuples for all 216 cells in both modes. | -| 15 | Copy | A destination tuple index is outside the cell array, FeatureIds array, or neighbor map range - return `-55568` before indexing. | *Not directly tested. AttributeMatrix construction and the FeatureIds/Image Geometry tuple-count preflight reject mismatched tuple shapes before execution. Defensive check.* | -| 16 | Copy | A source tuple index is outside the cell array or FeatureIds array range - return `-55573` before indexing. | *Not directly tested. Valid face-neighbor computation constrains source indexes to the Image Geometry cell range. Defensive check.* | +| 15 | Retired | The former destination tuple-index check is absent from the current slice-based copy implementation. | Not an active path. | +| 16 | Retired | No explicit source tuple-index range error exists in the current slice-based copy implementation. Face-neighbor selection constrains source indexes to valid cells. | Not an active path. Boundary copying is covered by `SimplnxCore::RequireMinNumNeighborsFilter: Discriminating 6x6x6 Analytical Fixture`. | | 17 | Finalize | Inactive feature tuples are removed and remaining FeatureIds are remapped; NeighborLists are removed. | `SimplnxCore::RequireMinNumNeighborsFilter`, both analytical oracle tests, and their exact compacted `NumNeighbors`, `Phases`, and `FeatureIds` assertions. | -| 18 | Finalize | Inactive-feature removal fails - return `-55570`. | *Not directly tested. Defensive check.* | +| 18 | Finalize | Inactive-feature removal fails - propagate the error returned by `RemoveInactiveObjects`. | *Not directly tested. Defensive check.* | +| 19 | Preflight | FeatureIds tuple count differs from the selected Image Geometry cell count - return `-55571`. | `SimplnxCore::RequireMinNumNeighborsFilter: Preflight Error - FeatureIds tuple count mismatch (-55571)` | +| 20 | Preflight | A Cell Arrays to Ignore path is not a direct child of the selected geometry's Cell Attribute Matrix - return `-55573`. Membership uses the parent object's ID, so another path to the same matrix is accepted. | `SimplnxCore::RequireMinNumNeighborsFilter: Strict Feature Ids` - `Ignored array in a second Attribute Matrix`; shared-path acceptance is covered by `SimplnxCore::RequireMinNumNeighborsFilter: Shared Cell Paths and Selected Feature Matrix`. | +| 21 | Preflight | The parent of Number of Neighbors is not an Attribute Matrix directly under the selected geometry, or is its Cell Attribute Matrix - return `-55574`. | `SimplnxCore::RequireMinNumNeighborsFilter: Strict Feature Ids` - `Feature array outside the selected geometry`, `Feature array under a plain DataGroup`, and `Feature array in the Cell Attribute Matrix`. | ## Test inventory @@ -190,7 +195,7 @@ The filter selects features to retain, marks rejected-feature voxels, iterativel | `SimplnxCore::RequireMinNumNeighborsFilter: Discriminating 6x6x6 Analytical Fixture` | new-for-V&V | Runs one common input in both phase modes. Checks six arrays with 2,610 exact element comparisons, a 1-vs-1 traversal-order tie, X/Y/Z boundaries, second-pass copying, ignored values, zero-cell feature retention, equal cell output across modes, and tuple dimensions. | | `SimplnxCore::RequireMinNumNeighborsFilter` | kept | Modified: loads `6_5_test_data_1_v2.tar.gz`, recomputes neighbors, derives the expected retained `NumElements` sequence from the pre-filter input and threshold, checks all 791 retained values, asserts the single `-5558` preflight warning, and verifies tuple dimensions. | | `SimplnxCore::RequireMinNumNeighborsFilter: Preflight Error - tuple count mismatch (-252)` | kept | Creates 5-tuple `NumNeighbors` and 4-tuple `Phases` arrays, then asserts invalid preflight and exact code `-252`. | -| `SimplnxCore::RequireMinNumNeighborsFilter: Preflight Error - FeatureIds tuple count mismatch (-55571)` | new-for-V&V | Creates a 5-cell geometry with four `FeatureIds` tuples, then asserts invalid preflight and exact code `-55571`. | +| `SimplnxCore::RequireMinNumNeighborsFilter: Preflight Error - FeatureIds outside Cell Attribute Matrix (-55575)` | new-for-V&V | Places `FeatureIds` in a second Attribute Matrix with the same tuple count as the Cell Attribute Matrix, then asserts invalid preflight and exact code `-55575`. | | `SimplnxCore::RequireMinNumNeighborsFilter: Execute Error - feature ID out of range (-55567)` | new-for-V&V | Supplies `FeatureIds [1,4,2,3]` for four feature tuples, asserts valid preflight, invalid execution, and exact code `-55567`; covers D2. | | `SimplnxCore::RequireMinNumNeighborsFilter: Execute - negative feature ID is reassigned` | new-for-V&V | Supplies `FeatureIds [-1,1,1,2]`, asserts successful execution, and checks all four final IDs equal `[1,1,1,2]`; covers D1. | | `SimplnxCore::RequireMinNumNeighborsFilter: Execute Error - unavailable phase (-5555)` | new-for-V&V | Selects absent phase 5, then asserts valid preflight, invalid execution, and exact code `-5555`. | @@ -201,8 +206,11 @@ The filter selects features to retain, marks rejected-feature voxels, iterativel | `SIMPL 6.4 (Filter_Name)` | kept | `DYNAMIC_SECTION` using `test/simpl_conversion/6_4/RequireMinNumNeighborsFilter.json`; verifies name-based 6.4 pipeline conversion. | | `SimplnxCore::RequireMinNumNeighborsFilter: Bad Phase Number` | retired | Removed from `#if 0`; it used stale parameter keys and could not compile. The active `unavailable phase (-5555)` test replaces its intended error coverage. | | `SimplnxCore::RequireMinNumNeighborsFilter: Phase Array` | retired | Removed from `#if 0`; it used stale parameter keys and could not compile. Current analytical variants cover the all-phase and single-phase selection behavior. | +| `SimplnxCore::RequireMinNumNeighborsFilter: Preflight Error - FeatureIds tuple count mismatch (-55571)` | new-for-V&V | Creates a 5-cell geometry with four `FeatureIds` tuples in its Cell Attribute Matrix, then asserts invalid preflight and exact code `-55571`. | +| `SimplnxCore::RequireMinNumNeighborsFilter: Strict Feature Ids` | new-for-V&V | Checks valid Cell and Feature Attribute Matrices, shared Cell paths, and read-only phases in another matrix. Rejects misplaced Feature Ids (`-55575`), ignored arrays (`-55573`), and invalid Feature Attribute Matrix locations (`-55574`); checks each error code and affected paths. | +| `SimplnxCore::RequireMinNumNeighborsFilter: Shared Cell Paths and Selected Feature Matrix` | new-for-V&V | Alternates geometry and shared-parent paths for Feature Ids and ignored arrays. Checks geometry-based modification paths, reassigned IDs and cell values, unchanged ignored values, selected Feature Attribute Matrix compaction, unchanged external marker values, and inherited tuple dimensions. | -There are 11 active `TEST_CASE`s: 3 kept and 8 new for this V&V cycle. +There are 14 active `TEST_CASE`s: 3 kept and 11 new for this V&V cycle. ## Exemplar archive diff --git a/src/simplnx/Utilities/DataArrayUtilities.cpp b/src/simplnx/Utilities/DataArrayUtilities.cpp index 30f1445b24..f871e73fa4 100644 --- a/src/simplnx/Utilities/DataArrayUtilities.cpp +++ b/src/simplnx/Utilities/DataArrayUtilities.cpp @@ -165,6 +165,80 @@ bool IsChildOfAttributeMatrix(const DataStructure& dataStructure, const DataPath return parentId.has_value() && parentId.value() == attributeMatrix.getId(); } +Result<> ValidateArrayInGeometryCellData(const DataStructure& dataStructure, const DataPath& geometryPath, const DataPath& arrayPath, int32 errorCode, std::string_view parameterName) +{ + const auto* gridGeomPtr = dataStructure.getDataAs(geometryPath); + if(gridGeomPtr == nullptr || gridGeomPtr->getCellData() == nullptr) + { + return MakeErrorResult(errorCode, fmt::format("**{}** DataArray '{}' requires a Cell Attribute Matrix in the selected grid geometry '{}'. Select a geometry with a Cell Attribute Matrix.", + parameterName, arrayPath.toString(), geometryPath.toString())); + } + if(!IsChildOfAttributeMatrix(dataStructure, arrayPath, *gridGeomPtr->getCellData())) + { + return MakeErrorResult(errorCode, fmt::format("**{}** DataArray '{}' must be in the Cell Attribute Matrix '{}' of the selected geometry '{}'. Select the DataArray from that Attribute Matrix.", + parameterName, arrayPath.toString(), gridGeomPtr->getCellDataPath().toString(), geometryPath.toString())); + } + return {}; +} + +Result<> ValidateArraysInGeometryCellData(const DataStructure& dataStructure, const DataPath& geometryPath, const std::vector& arrayPaths, int32 errorCode, std::string_view parameterName) +{ + const auto* gridGeomPtr = dataStructure.getDataAs(geometryPath); + if(gridGeomPtr == nullptr || gridGeomPtr->getCellData() == nullptr) + { + const std::string firstArray = arrayPaths.empty() ? "" : fmt::format(" DataArray '{}'", arrayPaths.front().toString()); + return MakeErrorResult(errorCode, fmt::format("**{}**{} requires a Cell Attribute Matrix in the selected grid geometry '{}'. Select a geometry with a Cell Attribute Matrix.", parameterName, + firstArray, geometryPath.toString())); + } + for(const auto& arrayPath : arrayPaths) + { + if(!IsChildOfAttributeMatrix(dataStructure, arrayPath, *gridGeomPtr->getCellData())) + { + return MakeErrorResult(errorCode, fmt::format("**{}** DataArray '{}' is not in the Cell Attribute Matrix '{}' of the selected geometry '{}'. " + "Only DataArrays in that Attribute Matrix are modified, so only they can be ignored.", + parameterName, arrayPath.toString(), gridGeomPtr->getCellDataPath().toString(), geometryPath.toString())); + } + } + return {}; +} + +DataPath MapToGeometryCellDataPath(const DataPath& cellDataPath, const DataPath& arrayPath) +{ + return cellDataPath.createChildPath(arrayPath.getTargetName()); +} + +std::vector MapToGeometryCellDataPath(const DataPath& cellDataPath, const std::vector& arrayPaths) +{ + std::vector mappedPaths; + mappedPaths.reserve(arrayPaths.size()); + for(const auto& arrayPath : arrayPaths) + { + mappedPaths.push_back(MapToGeometryCellDataPath(cellDataPath, arrayPath)); + } + return mappedPaths; +} + +Result<> ValidateFeatureAttributeMatrixInGeometry(const DataStructure& dataStructure, const DataPath& geometryPath, const DataPath& featureAttributeMatrixPath, int32 errorCode, + std::string_view sourceParameterName, const DataPath& sourceArrayPath) +{ + const auto* gridGeomPtr = dataStructure.getDataAs(geometryPath); + if(gridGeomPtr == nullptr || gridGeomPtr->getCellData() == nullptr) + { + return MakeErrorResult(errorCode, fmt::format("The Feature Attribute Matrix '{}' (the parent of **{}** '{}') requires a Cell Attribute Matrix in the selected grid geometry '{}'. " + "Select a geometry with a Cell Attribute Matrix.", + featureAttributeMatrixPath.toString(), sourceParameterName, sourceArrayPath.toString(), geometryPath.toString())); + } + const auto* featureAttributeMatrixPtr = dataStructure.getDataAs(featureAttributeMatrixPath); + if(featureAttributeMatrixPtr == nullptr || featureAttributeMatrixPtr->getId() == gridGeomPtr->getCellData()->getId() || !gridGeomPtr->contains(featureAttributeMatrixPtr)) + { + return MakeErrorResult(errorCode, + fmt::format("The Feature Attribute Matrix '{}' (the parent of **{}** '{}') must be an Attribute Matrix directly under the selected geometry '{}' " + "and must not be its Cell Attribute Matrix '{}'. Select a feature DataArray from a different Attribute Matrix directly under that geometry.", + featureAttributeMatrixPath.toString(), sourceParameterName, sourceArrayPath.toString(), geometryPath.toString(), gridGeomPtr->getCellDataPath().toString())); + } + return {}; +} + bool CheckArraysAreSameType(const DataStructure& dataStructure, const std::vector& dataArrayPaths) { std::set types; diff --git a/src/simplnx/Utilities/DataArrayUtilities.hpp b/src/simplnx/Utilities/DataArrayUtilities.hpp index 487dad7759..14fe101173 100644 --- a/src/simplnx/Utilities/DataArrayUtilities.hpp +++ b/src/simplnx/Utilities/DataArrayUtilities.hpp @@ -28,6 +28,7 @@ #include #include #include +#include #include #if defined(_MSC_VER) @@ -77,6 +78,68 @@ SIMPLNX_EXPORT Result<> ValidateCellTupleCount(const IGridGeometry& geometry, co */ SIMPLNX_EXPORT bool IsChildOfAttributeMatrix(const DataStructure& dataStructure, const DataPath& arrayPath, const AttributeMatrix& attributeMatrix); +/** + * @brief Validates that an array belongs to a grid geometry's Cell Attribute Matrix. + * @param dataStructure Resolves the selected objects. + * @param geometryPath Identifies the grid geometry. + * @param arrayPath Identifies the DataArray to validate. + * @param errorCode Error code to return for an invalid location or missing Cell Attribute Matrix. + * @param parameterName Parameter display name used in the error. + * @return Success for a direct child of the Cell Attribute Matrix, or an error. + * @note Checks parent membership only. The caller must validate that each selected leaf exists and is a DataArray. + */ +SIMPLNX_EXPORT Result<> ValidateArrayInGeometryCellData(const DataStructure& dataStructure, const DataPath& geometryPath, const DataPath& arrayPath, int32 errorCode, std::string_view parameterName); + +/** + * @brief Validates ignored arrays against a grid geometry's Cell Attribute Matrix. + * @param dataStructure Resolves the selected objects. + * @param geometryPath Identifies the grid geometry. + * @param arrayPaths Identifies the DataArrays to validate, in error-reporting order. + * @param errorCode Error code to return for an invalid location or missing Cell Attribute Matrix. + * @param parameterName Parameter display name used in the error. + * @return Error for the first invalid path or missing Cell Attribute Matrix; otherwise success. + * @note Checks parent membership only. The caller must validate that each selected leaf exists and is a DataArray. + */ +SIMPLNX_EXPORT Result<> ValidateArraysInGeometryCellData(const DataStructure& dataStructure, const DataPath& geometryPath, const std::vector& arrayPaths, int32 errorCode, + std::string_view parameterName); + +/** + * @brief Maps an array path onto the geometry's Cell Attribute Matrix path. + * @param cellDataPath Cell Attribute Matrix path returned by the geometry. + * @param arrayPath Array path whose target name is preserved. + * @return Child path under cellDataPath with the array's target name. + * + * The Cell Attribute Matrix can have more than one DataPath. Algorithms compare DataPaths with == + * and find sibling Cell arrays through FeatureIdsArrayPath.getParent(). Feature Ids and ignored arrays + * must therefore use the geometry's Cell Attribute Matrix path. This function does not validate array membership. + */ +SIMPLNX_EXPORT DataPath MapToGeometryCellDataPath(const DataPath& cellDataPath, const DataPath& arrayPath); + +/** + * @brief Maps array paths onto the geometry's Cell Attribute Matrix path. + * @param cellDataPath Cell Attribute Matrix path returned by the geometry. + * @param arrayPaths Array paths whose target names and order are preserved. + * @return Child paths under cellDataPath with the arrays' target names, in input order. + * + * The Cell Attribute Matrix can have more than one DataPath. Algorithms compare DataPaths with == + * and find sibling Cell arrays through FeatureIdsArrayPath.getParent(). Feature Ids and ignored arrays + * must therefore use the geometry's Cell Attribute Matrix path. This function does not validate array membership. + */ +SIMPLNX_EXPORT std::vector MapToGeometryCellDataPath(const DataPath& cellDataPath, const std::vector& arrayPaths); + +/** + * @brief Validates a Feature Attribute Matrix directly under the selected grid geometry. + * @param dataStructure Resolves the selected objects. + * @param geometryPath Identifies the grid geometry. + * @param featureAttributeMatrixPath Identifies the Feature Attribute Matrix to validate. + * @param errorCode Error code to return for an invalid location or missing Cell Attribute Matrix. + * @param sourceParameterName Display name of the parameter that selects the source array. + * @param sourceArrayPath Identifies the array whose parent supplies the Feature Attribute Matrix. + * @return Success for a direct child Attribute Matrix other than Cell Data, or an error. + */ +SIMPLNX_EXPORT Result<> ValidateFeatureAttributeMatrixInGeometry(const DataStructure& dataStructure, const DataPath& geometryPath, const DataPath& featureAttributeMatrixPath, int32 errorCode, + std::string_view sourceParameterName, const DataPath& sourceArrayPath); + /** * @brief Replaces tuples selected by a mask. * @tparam T Input array value type. diff --git a/test/DataArrayTest.cpp b/test/DataArrayTest.cpp index 1ac964fdda..739fcbd1e3 100644 --- a/test/DataArrayTest.cpp +++ b/test/DataArrayTest.cpp @@ -145,6 +145,90 @@ TEST_CASE("DataArrayUtilities::IsChildOfAttributeMatrix", "[simplnx][DataArrayUt } } +TEST_CASE("DataArrayUtilities::Geometry attribute matrix validation", "[simplnx][DataArrayUtilities][StrictFeatureIds]") +{ + DataStructure dataStructure; + const DataPath geometryPath({"Geometry"}); + IGridGeometry* gridGeomPtr = nullptr; + SECTION("Image Geometry") + { + gridGeomPtr = ImageGeom::Create(dataStructure, "Geometry"); + } + SECTION("Rectilinear Grid Geometry") + { + gridGeomPtr = RectGridGeom::Create(dataStructure, "Geometry"); + } + REQUIRE(gridGeomPtr != nullptr); + const ShapeType shape{1, 2, 2}; + auto* cellsPtr = AttributeMatrix::Create(dataStructure, "Cells", shape, gridGeomPtr->getId()); + auto* otherPtr = AttributeMatrix::Create(dataStructure, "Other", shape, gridGeomPtr->getId()); + REQUIRE(cellsPtr != nullptr); + REQUIRE(otherPtr != nullptr); + gridGeomPtr->setCellData(*cellsPtr); + auto* groupPtr = DataGroup::Create(dataStructure, "Elsewhere"); + REQUIRE(groupPtr != nullptr); + auto* nestedPtr = AttributeMatrix::Create(dataStructure, "Features", shape, groupPtr->getId()); + REQUIRE(nestedPtr != nullptr); + const DataPath cellPath({"Geometry", "Cells"}); + const DataPath arrayPath = cellPath.createChildPath("Ids"); + const DataPath otherPath({"Geometry", "Other", "Ids"}); + REQUIRE(Int32Array::Create(dataStructure, "Ids", std::make_shared>(shape, ShapeType{1}, 0), cellsPtr->getId()) != nullptr); + REQUIRE(Int32Array::Create(dataStructure, "Ids", std::make_shared>(shape, ShapeType{1}, 0), otherPtr->getId()) != nullptr); + + const auto checkError = [](const Result<>& result, const std::vector& messageParts) { + CHECK(result.invalid()); + if(result.valid()) + { + return; + } + REQUIRE(result.errors().size() == 1); + CHECK(result.errors().front().code == -12345); + for(const auto& part : messageParts) + { + CHECK(result.errors().front().message.find(part) != std::string::npos); + } + }; + + CHECK(MapToGeometryCellDataPath(cellPath, DataPath({"Elsewhere", "Cells", "Ids"})) == arrayPath); + CHECK(MapToGeometryCellDataPath(cellPath, otherPath) == arrayPath); + CHECK(MapToGeometryCellDataPath(cellPath, arrayPath) == arrayPath); + const std::vector mappedPaths = MapToGeometryCellDataPath(cellPath, std::vector{DataPath({"Elsewhere", "Cells", "Phases"}), otherPath}); + CHECK(mappedPaths == std::vector{cellPath.createChildPath("Phases"), arrayPath}); + CHECK(MapToGeometryCellDataPath(cellPath, std::vector{}).empty()); + + CHECK(ValidateArrayInGeometryCellData(dataStructure, geometryPath, arrayPath, -12345, "Cell Feature Ids").valid()); + checkError(ValidateArrayInGeometryCellData(dataStructure, geometryPath, otherPath, -12345, "Cell Feature Ids"), + {otherPath.toString(), cellPath.toString(), geometryPath.toString(), "**Cell Feature Ids**"}); + CHECK(ValidateArraysInGeometryCellData(dataStructure, geometryPath, {}, -12345, "Attribute Arrays to Ignore").valid()); + CHECK(ValidateArraysInGeometryCellData(dataStructure, geometryPath, {arrayPath}, -12345, "Attribute Arrays to Ignore").valid()); + checkError(ValidateArraysInGeometryCellData(dataStructure, geometryPath, {arrayPath, otherPath, DataPath({"Missing"})}, -12345, "Attribute Arrays to Ignore"), + {otherPath.toString(), cellPath.toString(), "**Attribute Arrays to Ignore**"}); + + const DataPath featurePath({"Geometry", "Other"}); + CHECK(ValidateFeatureAttributeMatrixInGeometry(dataStructure, geometryPath, featurePath, -12345, "Feature Ranking Array", otherPath).valid()); + checkError(ValidateFeatureAttributeMatrixInGeometry(dataStructure, geometryPath, cellPath, -12345, "Feature Ranking Array", arrayPath), + {cellPath.toString(), arrayPath.toString(), geometryPath.toString(), "**Feature Ranking Array**"}); + const DataPath nestedPath({"Elsewhere", "Features"}); + checkError(ValidateFeatureAttributeMatrixInGeometry(dataStructure, geometryPath, nestedPath, -12345, "Feature Ranking Array", nestedPath.createChildPath("Rank")), + {nestedPath.toString(), nestedPath.createChildPath("Rank").toString(), geometryPath.toString()}); + REQUIRE(DataGroup::Create(dataStructure, "Not an AM", gridGeomPtr->getId()) != nullptr); + const DataPath groupPath({"Geometry", "Not an AM"}); + checkError(ValidateFeatureAttributeMatrixInGeometry(dataStructure, geometryPath, groupPath, -12345, "Feature Ranking Array", groupPath.createChildPath("Rank")), + {groupPath.toString(), geometryPath.toString()}); + + REQUIRE(dataStructure.setAdditionalParent(otherPtr->getId(), groupPtr->getId())); + CHECK(ValidateFeatureAttributeMatrixInGeometry(dataStructure, geometryPath, DataPath({"Elsewhere", "Other"}), -12345, "Feature Ranking Array", DataPath({"Elsewhere", "Other", "Ids"})).valid()); + + auto* noCellsPtr = ImageGeom::Create(dataStructure, "No Cells"); + REQUIRE(noCellsPtr != nullptr); + const DataPath noCellsPath({"No Cells"}); + checkError(ValidateArrayInGeometryCellData(dataStructure, noCellsPath, arrayPath, -12345, "Cell Feature Ids"), {noCellsPath.toString(), arrayPath.toString()}); + checkError(ValidateArraysInGeometryCellData(dataStructure, noCellsPath, {}, -12345, "Attribute Arrays to Ignore"), {noCellsPath.toString(), "**Attribute Arrays to Ignore**"}); + checkError(ValidateFeatureAttributeMatrixInGeometry(dataStructure, noCellsPath, featurePath, -12345, "Feature Ranking Array", otherPath), + {noCellsPath.toString(), featurePath.toString(), otherPath.toString()}); + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + namespace { /**