Repository navigation
Extended testing fixes, simplified np.array reading, citation file - #30
Merged
Merged
Conversation
- explicitly checking comp_population headers for 'story' and 'dir' rather than excluding by index
- High level metrics in compare_runs.evaluate_tolerances now considers path if either relative difference < 4% OR absolute difference < 2 days - test_integration.py ungates ASSERT_POINTWISE - renamed metrics writeup to README.md so that it displays in github in the test folder - Added a section in README.md to describe what is considered passing in Mat-Py testing
- added utility file with functions to batch run folder in testing directory
These values correspond to those set at the Jan 22 2026 standup meeting - MAE lowered to 1% - P95 lowered to 2%
Zaidi et al. (2026) is the citation for the package, but the preferred citation is Cook et al. (2022).
- driver.py now has recursive np conversion to traverse nested arrays without pre-specifying levels. - outputting new damage_consequences as consequences.json (contains red tag info)
switched nested functionality array-to-list conversion to use the recursive method
This reverts commit a51a500. System-level point-wise thresholds reverted to P95 <=4% and MAE <= 2%.
- Removed "matlab" results from reference suite to ensure py-to-py testing - Added default seed in test_reference_comparison to stabilize CI from run to run Expected behavior for CI is now testing seeded single run vs. 20 reference run per test.
dustin-cook
approved these changes
Aug 25, 2026
dustin-cook
left a comment
dustin-cook
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed each commit. Looks great. No comments and clear to merge.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Post Jun 5th meeting updates
RCSW_1storyandhaseltonRCMF_12story. All reference fixtures are generated with ATCPy.np.row_stackcalls tonp.vstackto avoid deprecation in higher Numpy versions.Results
Passed 17/18 integration tests. Failed test had Metric 1 = 2.1 > 2.0 and Metric 2 < 4.0.
[post Aug 4 meeting]
Passed 18/18 integration tests.