Skip to content

DOC: Add coding rules, a knowledge base and a shared AGENTS.md - #1889

Open
joeykleingers wants to merge 13 commits into
BlueQuartzSoftware:developfrom
joeykleingers:topic/code-guidelines
Open

joeykleingers wants to merge 13 commits into
BlueQuartzSoftware:developfrom
joeykleingers:topic/code-guidelines

Conversation

@joeykleingers

@joeykleingers joeykleingers commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds the project rules that a careful C++ developer could still break in simplnx, a knowledge base of lessons from
real defects, and an AGENTS.md that every coding agent reads. The style guide gains sections on filters and
algorithms, concurrency, data stores, error codes, testing and build configuration. The new knowledge base records
25 lessons from pull requests and issues since March 2026. AGENTS.md names the required bluequartz-skills
skills, says where a new rule belongs, and holds the build and test facts of this checkout. The out-of-core test cases
get a ctest label so that AGENTS.md can tell agents how to run them, and stale working files leave docs/.
22 files changed, +1250 / −3594 lines (most of the deletions are the docs/ cleanup).

Companion pull requests: BlueQuartzSoftware/DREAM3DNX#1216 and BlueQuartzSoftware/SimplnxOoc#12 set up the
same documents in those repositories.

1. Code style guide

  • docs/Code_Style_Guide.md: new sections, added after the existing naming and layout rules

  • docs/AlgorithmDispatch.md: the test example uses AlgorithmTestScope instead of a bare force guard; the
    out-of-core section describes storage without private names, shows a copyIntoBuffer slice loop, states the
    ForceInCoreAlgorithm() precedence and corrects the IdentifySample memory descriptions

  • Public code. simplnx holds only public code. Proprietary code stays in the private repository that owns it,
    and simplnx interfaces do not describe a private implementation.

  • Filters and algorithms. An algorithm checks its own inputs, so it runs correctly without its filter. Those
    checks exist once, in a static ValidateInputs() of the algorithm class that preflight also calls. Each check
    sits at the earliest point that can see its data. Preflight validates every parameter value against the domain of
    its algorithm, never throws, and makes the same decisions as execute. Preflight does not allocate or read array
    data: outputs are placeholders, and its memory grows with metadata, not with element data. Further rules cover tuple counts, indices
    read from data, component shapes, DataPath construction, outputs for empty features, saturating casts to the
    output type, and final output paths that collide with inputs. A choice is compared with its named enum value, never a
    raw index. In-core and out-of-core algorithms give the same result. One bulk-transfer implementation replaces
    separate in-core and out-of-core copies only when it is measured to be no slower in core. The feature-id range check
    may run inside the main pass.

  • Feature 0 is not a feature. Code computes and stores nothing for feature 0 (mount material or bad data) unless
    the user asks for it, and a review asks about any code that does.

  • Concurrency. A file-level static header cache is guarded by one mutex. Out-of-core stores are written from one
    thread. Optional working-memory reservations are short and use the granted size. Long loops check for cancel.

  • Data stores and HDF5. Shape products are checked before allocation. A path that uses a raw pointer or span
    into store memory first checks that every store is in memory. A dataset from openDataset() is not valid until it
    is used, so existence is checked with isDataset() and the first read's Result.

  • Errors. Error codes are unique and survive a refactoring; a changed code is listed in the pull request.
    Messages name the right values.

  • Tests. Core tests live in the core target, with a direct test for each new exported helper. Tests pass with
    optional plugins turned off. Filter test names start with their plugin. A test of a known defect is hidden with
    [.][BlockedDefect] and keeps the correct expectation. Geometry-dependent tests use unequal dimensions and
    spacings. Allocation failures are tested through a failure seam, not a real oversized allocation. A test of one of several
    conditions that share an error code also asserts a phrase of that condition's message. The Python
    interpreter for tests comes from the CMake preset.

2. Coding knowledge base

  • docs/Coding_Knowledge_Base.md: new file with an entry template and 25 entries

Each entry states the rule, a pattern to use and one to avoid, the mechanism behind the failure, the pull request or
issue that showed it, and how to recognize it next time. Topics include writer ID maps and HDF5 close errors,
selection-parameter validation, geometry getters on detached objects, periodic centroids, statistics populations,
generated-mesh sizes in preflight, transformations applied twice, mesh clipping and intersection order, legacy SIMPL
keys, binary STL layout, length units of nonphysical geometries, hexagonal lattice corrections, neighborhood radii,
membership checks by Attribute Matrix object ID, legacy SIMPL conversions when a filter moves, file-change tests,
unequal axis spacing, transfer failures after an earlier batch, and Catch2 comparisons of file times. Entries cite code by file and function, not by
line number.

3. Agent instructions

  • AGENTS.md: new, tracked file
  • .gitignore: stops ignoring AGENTS.md
  • CONTRIBUTING.md: links the knowledge base

AGENTS.md lists the required skills (cpp-best-practices, simplnx-best-practices) and requires every task skill
whose description matches the work. An agent without access to the private skills, DREAM3D-NX or SimplnxOoc follows the
style guide and the knowledge base, runs the in-core build and tests, and lists in the pull request what it could not
run. The file routes each new rule to one place (the knowledge base wins when a rule comes from a defect) and gives
observable triggers for recording one. It also says:

  • which build directory to use, and that a normal build already downloads the test data;
  • that SIMPLNX_TEST_ALGORITHM_PATH is a configure-time cache variable;
  • when in-core and out-of-core validation is required (every change to code that runs when a filter executes or a
    store is read or written, with or without AlgorithmDispatch);
  • which tests to run before handing off a change, in a Release build;
  • how to run every out-of-core test: ctest -L "^ooc$" -LE "benchmark" (see section 4).

Coding rules stay in the skills and the two documents above; AGENTS.md does not repeat them.

4. Out-of-core test label

  • cmake/Plugin.cmake, test/CMakeLists.txt: two catch_discover_tests() calls with disjoint filters
    (~[OOC]~[.] and [OOC]~[.]); the second gives the cases tagged [OOC] the ctest label ooc
  • src/Plugins/SimplnxCore/test/CMakeLists.txt, src/Plugins/OrientationAnalysis/test/CMakeLists.txt: the
    OocStoreContracts entries get the label too

Catch2 2.13.6 has no ADD_TAGS_AS_LABELS, so ctest never saw the tag and -R matches only test names. Every visible
test case is still registered once, under the same name; only the label is new. The two discoveries list each test
executable twice after it links, which takes under a second.

5. docs/ cleanup

  • Removes working files that no longer apply (the documentation review tracker and its design note, the
    upstream-filter-sources list, the EBSDSegmentFeatures figure-generation files, Code_Review/vv_work)
  • Moves the old Sphinx and Doxygen configuration (CMakeLists.txt, Doxyfile.in, conf.py) to docs/deprecated/

Test plan

  • Every code example in the style guide and the knowledge base compiles in principle and does not show the
    mistake it warns against.
  • Every class, function, file and error code that an entry names exists on develop.
  • Every index link in the knowledge base resolves to its heading on GitHub.
  • AGENTS.md build and test commands work on a fresh checkout.
  • No file contains code from a private repository.
  • In an out-of-core DREAM3D-NX build, the split discovery registers the same tests under the same names as one
    unfiltered discovery (ImageProcessingUnitTest 1149, OrientationAnalysisUnitTest 373), and the ooc label
    lands only on the [OOC] cases. Hidden cases stay unregistered.

Related Issue

  • This PR has no related issue

@imikejackson
imikejackson force-pushed the topic/code-guidelines branch from 2d863b6 to 8ee01c0 Compare October 30, 2026 14:14

@imikejackson imikejackson left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't be followed as written

  1. "Run the whole [ooc] set" doesn't work with "run tests only through ctest". catch_discover_tests is called without ADD_TAGS_AS_LABELS (cmake/Plugin.cmake:410, test/CMakeLists.txt:143), so Catch2 tags never reach ctest. -R only matches test names, and running the test executable with a tag filter is forbidden. As written, an agent has no way to select the [ooc] set. Either add ADD_TAGS_AS_LABELS and tell agents to use ctest -L OOC, or give a name regex that works.
  2. The [ooc] tag is only part of the out-of-core tests. The tests have [OOC] (14 uses), [.OOC] (1) and [.OocStoreContract] (34). Catch2 tags ignore case, so [ooc] matches [OOC]. The two hidden ([.…]) tags are not part of it, and catch_discover_tests doesn't register hidden tests at all. "The whole [ooc] set" therefore silently leaves out the 34 store-contract tests. The file should say whether those count.
  3. "Record it in the same change" doesn't fit skills. The file says a lesson must be recorded in the same change, but skill changes "go to the bluequartz-skills plugin repository". A change in this repo can't also be a change in that one. It needs to say what to do instead, for example open a pull request or issue there and link it.

Ambiguous

  1. Which build directory to use. "Use the build directory whose cache points at this source tree; a DREAM3D-NX build counts." If both a simplnx build and a DREAM3D-NX build exist, which one wins? And if neither exists, should the agent configure a new one, and with which preset?
  2. When to set SIMPLNX_TEST_ALGORITHM_PATH. It's a CMake cache variable compiled into the test targets as a definition (CMakeLists.txt:62, Plugin.cmake:397). The file doesn't say this, so an agent may try to set it as an environment variable at ctest time, which does nothing.
  3. The Fetch_Remote_Data_Files step reads as manual. The file says to build that target to download test data. It's actually an ALL target, and both simplnx and every unit-test target depend on it, so a normal build already downloads the data. An agent could waste time on it or think the data is missing.
  4. When the dual-build check is required. "Validate in-core and out-of-core together with dual-build-protocol" doesn't sange or only out-of-core changes."When out-of-core behavior chanes that. For example, does a newfilter that uses AlgorithmDispatch count?
  5. Whether the task skills are required. The two required skills are clearly mandatory. "Task skills own the details" doesn't say whether loading filter-implementation and the others is also required, or only advised. No fallback when the skills areublic repository, and an outside contributor's agent won't have e says every agent "first loads"them, but not what to do if it can't, or where to get them.
  6. Where personal rules go assumes Codex. It says personal rules go in ~/.codex/AGENTS.md. Claude Code (and other agents) use ~/.claude/CLAUDE.md. Say "your agent's personal instruction file", or list bot. Knowledge base vs. style guiderom a real defect fits both "a lesson from a real defect" (knntion… no defect history is needed" (style guide). The file should say which one wins, presumably the knowledge base when there's defect evidence.
  7. "Leave scripts/, conda/ and build trees alone unless the task needs them". "Needs" is subjective. Is reconfiguring ae asks for elsewhere) "touching" it? An agent could read these cting.
    Minor
  • The trigger "when work teaches something a later agent or developer needs" is subjective. A concrete test would help, for exorrect it, or it caused a defect".
  • The "public repository, no privae (under "Where new rules go" and"Repository"). The second could st.

@jmarquisbq jmarquisbq left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved

@jmarquisbq jmarquisbq left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I didn't mean to approve yet clicked the wrong button. But I don't have any additional comments for this repo past what Mike wrote above

@joeykleingers

Copy link
Copy Markdown
Contributor Author

Thanks for the review. All points are addressed in e41d951 and 99e8838, and the same fixes are in the AGENTS.md files of BlueQuartzSoftware/DREAM3DNX#1216 and BlueQuartzSoftware/SimplnxOoc#12. Part of points 7 to 12 and of the two minor points came through cut off, so I worked from my reading of them; tell me if I misread one.

Can't be followed as written

  1. Selecting the out-of-core tests. Catch2 here is 2.13.6, which has no ADD_TAGS_AS_LABELS, so each catch_discover_tests() call is now split in two with disjoint filters (~[OOC]~[.] and [OOC]~[.]); the second gives the tagged cases the ctest label ooc. Every visible case is still registered once under the same name. AGENTS.md now says ctest -L "^ooc$" -LE "benchmark".
  2. Hidden store-contract tests. The OocStoreContracts entries now carry the ooc label, so the same command runs them. AGENTS.md says that other hidden cases are not registered with ctest.
  3. Skill changes. A skill change cannot be part of a change here: open a pull request or an issue in the skills repository and link it, or, without access to it, describe the change in the pull request description.

Ambiguous

  1. Build directory. Check CMAKE_HOME_DIRECTORY (simplnx build) or simplnx_SOURCE_DIR (DREAM3D-NX build). For in-core work either kind works, and the simplnx build wins when both exist. Out-of-core tests need a DREAM3D-NX build with DREAM3DNX_USE_OOC=ON. With no build, configure one from a preset that sets SIMPLNX_BUILD_TESTS=ON (the ci-<platform> presets do).
  2. SIMPLNX_TEST_ALGORITHM_PATH. Stated as a cache variable that is set at configure time; the environment does nothing.
  3. Fetch_Remote_Data_Files. A normal build downloads the data: the target is part of the default build and every plugin unit-test target depends on it. (The library depends on it only with SIMPLNX_DOWNLOAD_TEST_FILES_FIRST.)
  4. When dual-build validation is required. For every change to code that runs when a filter executes or a store is read or written, including a new filter with or without AlgorithmDispatch. Not for documentation, comments or build scripts only.
  5. Task skills. Every task skill whose description matches the work is required.
  6. No access to the private skills or repositories. The agent follows the style guide and the knowledge base, runs the in-core build and tests, and lists in the pull request what it could not run; a maintainer completes that validation.
  7. Personal rules. "Your agent's personal instruction file", without naming one tool.
  8. Knowledge base vs. style guide. A rule that comes from a defect goes in the knowledge base, even if it is also a convention.
  9. "Unless the task needs them." Do not edit scripts/ or conda/ unless the user asks. Configure, build and test only in a build directory that you created for the task or that the user named, and never delete or clean one.

Minor

  • Recording trigger. A rule is recorded when a reviewer or the user had to correct something that no document covers, or when a defect came from it.
  • Public-repository rule. It appears once, under "Repository"; "Where new rules go" points to it.

Two further additions: which tests to run before handing off a change (the changed code and its callers, in a Release build), and what to do when vcpkg fails (stop and report the package and its log).

joeykleingers and others added 13 commits October 30, 2026 16:04
* Add rules to docs/Code_Style_Guide.md that a careful C++ developer
  could still break in simplnx. An algorithm checks its own inputs, so
  it runs correctly without its filter. The rules tell where each check
  goes, how to check component shapes, tuple counts and indices that
  come from data, and why preflight must not throw. They also cover
  result propagation, saturating casts, static header caches,
  out-of-core stores, working-memory reservations, error codes, tests
  and the Python interpreter of the tests.
* Add docs/Coding_Knowledge_Base.md with 19 lessons from pull requests
  and issues since March 2026. Each entry gives the rule, the pattern
  to use and to avoid, the cause, the pull request or issue that showed
  it, and how to recognize it next time.
* Link the knowledge base from CONTRIBUTING.md and from the style
  guide.
* Add AGENTS.md for all coding agents. It names the required skills,
  tells where a new rule goes, and gives the repository layout and
  the build and test commands. Remove AGENTS.md from .gitignore so
  that the file is shared.

Signed-off-by: Joey Kleingers <joey.kleingers@bluequartz.net>
* Remove every statement of whether a pull request or issue is open,
  merged, closed or not merged yet from the knowledge base and the
  style guide. Keep the pull request and issue numbers.
* Add the rule to the introduction of the knowledge base: cite pull
  requests and issues by number, but not their status, because the
  status changes and the entry would go stale.

Signed-off-by: Joey Kleingers <joey.kleingers@bluequartz.net>
…kill

Every rule that says how to word an error or warning message now names the bluequartz-skills error-messages skill, and the message examples follow it.
An algorithm class keeps its input checks in one static ValidateInputs() that preflightImpl() and operator()() both call, instead of a second copy in preflight. The rule covers inputs that the filter's own actions move, placeholder counts from earlier filters, dispatch to several algorithm classes, and how to test a rejection that returns before the dispatch.
Preflight runs on every parameter change and on placeholder stores, so its memory may grow with metadata and short previews but not with the element data of an array. The rule allows counts as output shapes and reads past a file header when the format needs them, and names CreateStringArrayAction as a current exception (issue BlueQuartzSoftware#1892).
Name what needs UnitTest::LoadPlugins(): a filter looked up by UUID
through the filter list, a loaded pipeline, or a filter that gets other
filters from the list. A test that constructs its filter directly does
not call it, also when it reads its data from a .dream3d file. Remove
the call from the AlgorithmDispatch example test, which does not need
it.
The guide tests both algorithm paths with AlgorithmTestScope and SelectAlgorithmTestScenariosForInMemoryStores(). Show that pattern in place of a bare ForceOocAlgorithmGuard, and describe ForceInCoreAlgorithm() with ForceOocAlgorithm().
Knowledge base: validate an in-place array by the object ID of its
Attribute Matrix; read a count-only check as IArray; keep the legacy
conversions when a filter moves; set a distinct mtime in a file-change
test; test geometry formulas with unequal spacing; test a transfer
failure after an earlier batch. Entries cite code by file and function,
not by line number.

Style guide: simplnx holds only public code; compare a choice with its
named value; keep error codes through a refactoring; check the stores
before resident pointers or spans; a dataset from openDataset() is not
valid until it is used. Testing: core tests in the core target with a
direct test for each exported helper, tests pass with optional plugins
off, plugin-prefixed test names, hidden known-defect tests, and unequal
dimensions and spacings.
* Entry 25 says to give CHECK and REQUIRE a bool when a test compares
  std::filesystem::file_time_type values or file-clock durations.
* Catch2 converts the operands to text, and libc++ then needs
  std::to_chars from macOS 13.3. The DREAM3D-NX macOS CI builds target
  macOS 11.7, so two tests in PR BlueQuartzSoftware#1764 did not compile there.

Signed-off-by: Joey Kleingers <joey.kleingers@bluequartz.net>
Style guide: feature 0 is not a feature, so code computes and stores
nothing for it unless the user asks; the feature-id range check may run
inside the algorithm's main pass; write one bulk-transfer implementation
instead of in-core and out-of-core copies only when it is measured to
be no slower in core; a test of one of several conditions that share an
error code also asserts a phrase of that condition's message.

AlgorithmDispatch.md: describe out-of-core storage without private
names, show a copyIntoBuffer slice loop, state the ForceInCore
precedence and correct the IdentifySample memory descriptions.

Signed-off-by: Joey Kleingers <joey.kleingers@bluequartz.net>
Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
Catch2 2.13.6 has no ADD_TAGS_AS_LABELS, so ctest never saw the [OOC]
tag. Each discovering test executable now runs two discoveries with
disjoint filters (~[OOC]~[.] and [OOC]~[.]); the second gives the tagged
cases the ctest label "ooc". Every visible case is still registered once
under the same name. The OocStoreContracts entries get the label too, so
`ctest -L "^ooc$"` selects every out-of-core test.

Signed-off-by: Joey Kleingers <joey.kleingers@bluequartz.net>
Answers the review of this pull request:
- select the out-of-core tests with `ctest -L "^ooc$" -LE "benchmark"`,
  including the store-contract entries;
- say which build directory to use, that a normal build downloads the
  test data, and that SIMPLNX_TEST_ALGORITHM_PATH is set at configure
  time;
- state when dual-build validation is required, that matching task
  skills are required, and what an agent without access to the private
  repositories does;
- name the agent's personal instruction file generically, make the
  knowledge base win for defect-derived rules, give concrete triggers
  for recording a rule, and keep one public-repository rule;
- say which tests to run before handing off and what to do when vcpkg
  fails.
The knowledge base introduction now points to the same triggers.

Signed-off-by: Joey Kleingers <joey.kleingers@bluequartz.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants