Skip to content

[E2E Testing] Fix update e2e bottleneck - #3275

Merged
ew3361zh merged 16 commits into
devfrom
fix-update-e2e-bottleneck
Sep 16, 2026
Merged

ew3361zh merged 16 commits into
devfrom
fix-update-e2e-bottleneck

Conversation

@ew3361zh

Copy link
Copy Markdown
Collaborator

Context

Issue(s) closed by this pull request: closes #3224

What

Removes the deepdiff comparison of results from e2e test simulation runs from old expected results in the step prior to overwriting those results as the new expected e2e results.

Why

There has been a bottleneck/outright stall out on certain machines (windows/linux primarily) in updating the e2e expected results. Initial indications were that this was occurring at the deepdiff comparison stage. Upon further investigation and reflection, this comparison is not a value add for the process of updating the expected results so it's been removed here to hopefully speed up the process.

How

In the update expected results code, there was a deepdiff comparison of the results from the run that just completed. This step has been removed here.

Test plan

  • Run the update expected e2e results task.
  • Check that the results are passing with e2e testing.

Input Changes

Output Changes

  • N/A

Filter

@github-actions

Copy link
Copy Markdown
Contributor

Current Coverage: 99%

Mypy errors on fix-update-e2e-bottleneck branch: 1164
Mypy errors on dev branch: 1164
No difference in error counts

@github-actions

Copy link
Copy Markdown
Contributor

🚨 Please update the changelog. This PR cannot be merged until changelog_WIP.md is updated.
🚨 Flake8 linting errors were found. Please fix the linting issues.

@github-actions

Copy link
Copy Markdown
Contributor

Current Coverage: 99%

Mypy errors on fix-update-e2e-bottleneck branch: 1164
Mypy errors on dev branch: 1164
No difference in error counts

@matthew7838 matthew7838 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, tested and worked as expected. I agree with the trade-off. The removed warning/log/diff files weren't very useful for the e2e update task anyway. Thanks for looking into this.

@allisterakun allisterakun left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@github-actions

Copy link
Copy Markdown
Contributor

Current Coverage: 99%

Mypy errors on fix-update-e2e-bottleneck branch: 1164
Mypy errors on dev branch: 1164
No difference in error counts

@ew3361zh
ew3361zh merged commit 717f472 into dev Sep 16, 2026
1 check passed
@ew3361zh
ew3361zh deleted the fix-update-e2e-bottleneck branch September 16, 2026 15:27
@ew3361zh ew3361zh mentioned this pull request Sep 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Difficulty updating Open-lot scenario E2E results

3 participants