Skip to content

[e2e testing] Applies patches to e2e testing - #3274

Merged
ew3361zh merged 2 commits into
field_details_fixfrom
no-deepdiff-e2e-expected-update
Sep 16, 2026
Merged

ew3361zh merged 2 commits into
field_details_fixfrom
no-deepdiff-e2e-expected-update

Conversation

@ew3361zh

@ew3361zh ew3361zh commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

Context

Issue(s) closed by this pull request: closes #3224 closes #3255

What

Applies fixes to the e2e testing updates and tolerance processes:

  1. Removes the deepdiff comparison of results from e2e test simulation runs from old expected results in the step prior to overwriting those results as the new expected e2e results. Implementing on dev branch via [E2E Testing] Fix update e2e bottleneck #3275.
  2. Fixes an issue where the tolerance set for E2E testing results comparison was not being properly implemented. Implemented on dev branch in [E2E] E2E sensitivity patch #3026.

Why

These two fixes will bring e2e back to full functionality and aligning better with where e2e testing is on dev branch.

How

  1. In the update expected results code, there was a deepdiff comparison of the results from the run that just completed. This step has been removed here.
  2. The results comparison mechanism was not traversing deeply enough to catch and compare the differences flagged by DeepDiff as significant. This update allows for deeper traversal to properly apply tolerance mechanism.

Test plan

  1. Windows/Linux users should now be able to update expected results on the test branch.
  2. E2E testing should now be passing more consistently across users with different machines/operating systems.

Input Changes

Output Changes

  • N/A

Filter

@JoeWaddell

Copy link
Copy Markdown
Collaborator

Success! Works on my machine, and it hadn't before. Nice job figuring this out!

@ew3361zh ew3361zh changed the title [e2e testing] Removes deepdiff check from e2e testing expected results update [e2e testing] Applies patches to e2e testing Sep 15, 2026
@ew3361zh
ew3361zh marked this pull request as ready for review September 15, 2026 14:29

@matthew7838 matthew7838 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@JoeWaddell JoeWaddell left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Official approval following my earlier comment that this works to solve the issues I had on my end.

@ew3361zh
ew3361zh merged commit b82b7a2 into field_details_fix Sep 16, 2026
@ew3361zh
ew3361zh deleted the no-deepdiff-e2e-expected-update branch September 16, 2026 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

4 participants