Repository navigation
Keep the Perf workflow's write access out of the Duel job - #79
Merged
Merged
Conversation
The Duel job checks out and runs the pull request's code, benchmarks included, so it no longer has pull-requests: write, and its checkout keeps no credentials. It uploads its report, and a Comment job of its own, which checks nothing out and runs none of the branch's code, posts the comment. .duel.rb also watches the benchmarks, itself and the Perf workflow, so a pull request that changes only those is compared, instead of passing with nothing to compare.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Duel passed: none of 23 benchmarks slower
23 benchmarks in 28s. A timing is slower when a 99.0% confidence interval for its ratio, the candidate's to the baseline's, is above 1.02. Across the board
23 benchmarks unchanged
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A review of the same Duel workflow in another project turned up a couple of things worth fixing here too, so this brings
perf.ymlin line with whatduel initgenerates now.The Duel job checks out and runs the pull request's code, benchmarks included, and it also had
pull-requests: writeso it could post its report. Now it only reads, and its checkout keeps no credentials. It uploads the report as an artifact, and a separate Comment job downloads it and posts the comment. That job checks nothing out and runs none of the branch's code, so it's the only one that can write. A pull request from a fork still gets no comment, since its token can't write one, but the job's summary has the report.The other fix is in
.duel.rb. Duel only compares a change that toucheslib/orext/, so a pull request that changed only the benchmarks or the config passed with "nothing to compare", and a broken benchmark could slip through. It now watchesduel/,.duel.rband the workflow too. That also means this pull request runs a real comparison of the code with itself, which checks that the new workflow works.