Repository navigation
BUG: C04: Sibling DataArray tuple counts not compared in 13 filters #1864
Description
Summary
Many filters read two or more input DataArrays with the same index, but preflight does not compare their tuple counts. The selection parameters accept each DataArray from any Attribute Matrix, so the sibling inputs can have different tuple counts. Execute then takes its loop count from one DataArray and reads the others past their end, or it pairs values that do not belong together.
As of develop 419f9db (2026-10-21), there are 13 open findings in 13 filters: 8 CRASH, 4 WRONG_DATA, 1 MINOR. Each finding is a candidate. It was found and re-checked by reading code, not by running it.
Mechanism
A filter reads, for example, an Attribute Array and a Conditional Array (mask) tuple by tuple. It takes the loop count from one of them and assumes that the other has the same number of tuples. If the second DataArray has fewer tuples, the loop reads past its end (CRASH). If it has more tuples, the filter silently pairs each value with an unrelated value (WRONG_DATA). A third variant sizes the outputs from an Attribute Matrix but takes the loop count from an input DataArray, so the filter writes past the end of its outputs.
Example: ConditionalSetValueFilter preflight checks only the DataType of the Conditional Array. The Direct algorithm takes the loop count from the Attribute Array and reads the Conditional Array with the same index:
ConditionalSetValue.cpp#L110-L139
auto& targetDataArray = dynamic_cast<DataArray<TargetType>&>(targetArray);
const auto& conditionalDataArray = dynamic_cast<const DataArray<ConditionalType>&>(conditionalArray);
const usize numTuples = targetDataArray.getNumberOfTuples();
// ...
const ConditionalType* conditionalValues = contiguousConditionalStore->data();
// ...
for(usize tupleIndex = tupleOffset; tupleIndex < tupleEnd; tupleIndex++)
{
if(static_cast<bool>(conditionalValues[tupleIndex]) != invertMask)
{
targetValues[tupleIndex] = replaceValue;
}
}If the Attribute Array has 100 tuples and the Conditional Array has 10 tuples, the loop reads 90 values past the end of the Conditional Array.
The fixed form already exists on develop. SilhouetteFilter (fixed in #1765) compares the sibling tuple counts in preflight:
SilhouetteFilter.cpp#L112-L119
if(pUseMaskValue && !CheckArraysHaveSameTupleCount(dataStructure, {pSelectedArrayPathValue, pMaskArrayPathValue, pFeatureIdsArrayPathValue}))
{
const auto& maskArray = dataStructure.getDataRefAs<IDataArray>(pMaskArrayPathValue);
return MakePreflightErrorResult(-8977, fmt::format("Attribute Array to Silhouette DataArray '{}' ({} tuples), Mask Array DataArray '{}' ({} tuples), and Cluster Ids DataArray '{}' ({} tuples) "
"must have the same number of tuples.",
pSelectedArrayPathValue.toString(), clusterArray->getNumberOfTuples(), pMaskArrayPathValue.toString(), maskArray.getNumberOfTuples(),
pFeatureIdsArrayPathValue.toString(), clusterIds->getNumberOfTuples()));
}Rule
Input DataArrays that a filter reads with the same index must have the same tuple count. Check this in preflight. The DataArrays may be in any Attribute Matrix.
- If the filter sizes its outputs from an Attribute Matrix, each input that sets the loop count must have the same tuple count as that Attribute Matrix.
- If an input is an element DataArray of a geometry (cells, faces, edges, vertices), its tuple count must also equal the geometry's element count (see the companion class "Element DataArray tuple count not checked against the geometry").
- Include optional inputs (for example a mask) in the check only when the option that uses them is on.
How to fix
- In
preflightImpl, collect the paths of all inputs that the algorithm reads with the same index. - Call
CheckArraysHaveSameTupleCount(dataStructure, paths)fromsrc/simplnx/Utilities/DataArrayUtilities.hpp. It returnsbool. Alternatively callDataStructure::validateNumberOfTuples(paths), which returns an error string. - When the outputs are sized from an Attribute Matrix (
ComputeShapesTriangleGeomFilter#1,ComputeMomentInvariants2DFilter#2), compare the input tuple count withAttributeMatrix::getNumberOfTuples(). - When an index parameter selects an ensemble (
WriteGBCDGMTFileFilter#4), also require the index to be less than the tuple count of every ensemble DataArray that is read with it. - For
PartitionGeometryFilter#2, compare the Vertex Mask tuple count with the vertex count of the node geometry. PR BUG: Check that grid-geometry cell inputs have one tuple per cell #1840 has a file-local helpervalidateVertexTupleCountinPartitionGeometryFilter.cppfor this. - Write an error message that names each path and its tuple count, as
SilhouetteFilterdoes. - Error codes: use a new code in the filter's existing range. Filter error codes are more negative than -10000, and each filter uses one range (first code -47500 means all codes -47500..-47599). Check that the code is unique with
git grep -nE -- "-NNNN([^0-9]|$)"(do not use\b) and also in the open PR refs. - Remove any execute-time check that the new preflight check makes unreachable (for example the generic
copyIntoBufferfailure -6030 paths inComputeVectorColorsFilterandComputeTwinBoundariesFilter). Do not keep duplicate checks.
How to test
- Write the test first. It must fail on develop.
- Preflight test: put one sibling input in a different Attribute Matrix with fewer tuples (for example 10 tuples against 100). Assert that preflight fails with the new error code. For the WRONG_DATA findings, add a second case with more tuples.
- Positive test: siblings with equal tuple counts in different Attribute Matrices must pass preflight.
- For an optional input, test that the check runs only when the option is on.
- If the filter has more than one algorithm (for example Direct and Scanline), run the test for each one through the existing algorithm-selection test scope.
- If a fix changes output values, compare them with an independent expected result (oracle), not with the code's own output.
Conventions
- One PR for the whole class. One commit per filter.
- Commit prefix
BUG:. Add aSigned-off-byline. Do not add AI attribution. - Every fix starts with a unit test that fails on develop. If you cannot reproduce a finding, write that with the reason in the PR and on this issue. Do not force a fix.
- Value checks need an independent expected result (oracle), not the code's own output.
- Run ctest serially. Never use
-j; the tests depend on each other. - Format with clang-format 19.1.1 and the repository
.clang-format. - Do NOT add documentation sentences about input requirements to the filter pages. The generated parameter table documents them.
- If a fix touches a filter that has a V&V report (
src/Plugins/<Plugin>/vv/<Filter>.md), update every file of that filter's V&V package (report, deviations, provenance, test tables). V&V documents reference DREAM3D 6.5.171 only. - Rebase on develop. Never merge develop into the branch.
Checklist
- CRASH · ComputeGBCDMetricBasedFilter · ComputeGBCDMetricBasedFilter.cpp:180 — Average Euler Angles with fewer tuples than Phases is read out of bounds.
ComputeGBCDMetricBasedFilter#3 - CRASH · ComputeMomentInvariants2DFilter · ComputeMomentInvariants2D.cpp:445 — Feature Rect with more tuples than the Feature Attribute Matrix writes the outputs out of bounds.
ComputeMomentInvariants2DFilter#2 - CRASH · ComputeShapesTriangleGeomFilter · ComputeShapesTriangleGeom.cpp:534 — Centroids with more tuples than the Face Feature Attribute Matrix makes
setValuewrite the outputs out of bounds.ComputeShapesTriangleGeomFilter#1 - CRASH · ComputeSlipTransmissionMetricsFilter · ComputeSlipTransmissionMetricsFilter.cpp:103 — Neighbor List or Average Quaternions with fewer tuples than Feature Phases is read out of bounds.
ComputeSlipTransmissionMetricsFilter#3 - CRASH · ConditionalSetValueFilter · ConditionalSetValueFilter.cpp:109 — Conditional Array with fewer tuples than Attribute Array is read out of bounds.
ConditionalSetValueFilter#1 - CRASH · PartitionGeometryFilter · PartitionGeometryDirect.cpp:297 — Vertex Mask with fewer tuples than vertices is read out of bounds (PR BUG: Check that grid-geometry cell inputs have one tuple per cell #1840 adds only a type check).
PartitionGeometryFilter#2 - CRASH · WriteGBCDGMTFileFilter · WriteGBCDGMTFileFilter.cpp:112 — Crystal Structures with fewer tuples than GBCD is read out of bounds at index Phase of Interest.
WriteGBCDGMTFileFilter#4 - CRASH · WriteINLFileFilter · WriteINLFile.cpp:209 — Crystal Structures or Number of Features with fewer tuples than Material Names is read out of bounds (not fixed by PR BUG: Check that grid-geometry cell inputs have one tuple per cell #1840).
WriteINLFileFilter#4 - WRONG_DATA · ComputeBoundaryElementFractionsFilter · ComputeBoundaryElementFractionsFilter.cpp:85 — Surface Elements with more tuples than Feature Ids is silently paired with the wrong cells.
ComputeBoundaryElementFractionsFilter#2 - WRONG_DATA · ComputeFeaturePhasesBinaryFilter · ComputeFeaturePhasesBinaryFilter.cpp:86 — Mask with more tuples than Feature Ids is silently paired with the wrong cells (PR BUG: Enforce component shapes and types that filters read #1766 adds only the component shape).
ComputeFeaturePhasesBinaryFilter#3 - WRONG_DATA · ComputeMisorientationsFilter · ComputeMisorientationsFilter.cpp:130 — Phases is not compared with the Euler Angles arrays; a longer Phases is silently paired with the wrong angles.
ComputeMisorientationsFilter#1 - WRONG_DATA · ComputeVectorColorsFilter · ComputeVectorColors.cpp:70 — Mask with more tuples than the vector array gives wrong colors; a shorter one fails with the generic error -6030.
ComputeVectorColorsFilter#2 - MINOR · ComputeTwinBoundariesFilter · ComputeTwinBoundariesFilter.cpp:118 — Short Average Quaternions or Face Normals fail in execute with the generic error -6030, not a clear preflight error.
ComputeTwinBoundariesFilter#2
Already fixed or in progress
These findings of this class are fixed on develop or in an open PR. They are not in the checklist.
- Fixed on develop by BUG: Fix wrong output and invalid input handling in 19 filters #1765 (1986127):
- SilhouetteFilter
SilhouetteFilter#2
- SilhouetteFilter
- Fixed on develop by BUG: Require Feature Ids in the geometry's Cell Attribute Matrix for filters that modify it #1829 (419f9db):
- FillBadDataFilter
FillBadDataFilter#2
- FillBadDataFilter
- Fixed in open PR BUG: Enforce component shapes and types that filters read #1766:
- ComputeKMeansFilter
ComputeKMeansFilter#4 - ComputeKMedoidsFilter
ComputeKMedoidsFilter#1 - DBSCANFilter
DBSCANFilter#2
- ComputeKMeansFilter
Related issues
- BUG: Fix wrong output and invalid input handling in 19 filters #1765 (merged): fixed
SilhouetteFilter; its preflight check is the model for this class. - BUG: Enforce component shapes and types that filters read #1766 (open PR): enforces component shapes and DataTypes that filters read, and fixes the sibling tuple counts in
ComputeKMeansFilter,ComputeKMedoidsFilterandDBSCANFilter. It adds only the component shape forComputeFeaturePhasesBinaryFilter. - BUG: Check that grid-geometry cell inputs have one tuple per cell #1840 (open PR): adds cell tuple-count checks; it does not fix
WriteINLFileFilter#4,ComputeMomentInvariants2DFilter#2orPartitionGeometryFilter#2.
Out of scope
- Findings in filters that have a V&V report are excluded: 4 findings in 3 filters, ComputeFeatureReferenceMisorientationsFilter, GroupMicroTextureRegionsFilter and WritePoleFigureFilter. They are handled on a separate V&V branch.
- Element DataArrays checked against the geometry element count. That is the companion class "Element DataArray tuple count not checked against the geometry", except where a finding here needs both checks.
- Execute-time checks of Feature Id values against the Feature Attribute Matrix. That is a different class.
Activity
[-]BUG: Sibling DataArray tuple counts not compared in 13 filters[/-][+]BUG: C04: Sibling DataArray tuple counts not compared in 13 filters[/+]