Repository navigation
Add workflows for performance testing - #1297
Merged
Merged
Conversation
Member
Author
|
@svigerske as our resident jenkins expert, is there some functionality you think should be added? I don't mean a review, just what you would expect something like this to provide that it currently isn't from the PR description :) EDIT: I'm adding the functionality to compare PySCIPOpts with different SCIP versions in a bit. |
…els, note SCIP mismatches
Member
Author
|
Well, since it's opt in only, I suppose there's not much harm in merging. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Typing
/benchmarkstarts the runs. Right now, they take around 15 minutes and should be discussed in a separate PR, this one is just for the infrastructure.One can run specific tests and set time limits (here just the tests under
modeland a 10s time limit). A comment is automatically posted and updated every 5 minutes until the run finishes. This comment points to the comment that originated the run and to the run itself.Both base and release are cached (on PR merges and releases) to reduce building time in future runs. Further, for these two, only the timing experiments are run, the deterministic results (instructions from callgrind) are stored.
Significant regressions and improvements are signaled and presented in a new comment once the run finishes. The comparison to the latest release is minimized at the bottom.
You can see how it would work in here. (Pay more attention to the latest runs, since the others correspond to earlier versions).