Skip to content

Keep the Perf workflow's write access out of the Duel job - #73

Merged
sferik merged 1 commit into
mainfrom
duel-workflow
Oct 26, 2026
Merged

sferik merged 1 commit into
mainfrom
duel-workflow

Conversation

@sferik

@sferik sferik commented Oct 26, 2026

Copy link
Copy Markdown
Owner

A review of the same Duel workflow in another project turned up a couple of things worth fixing here too, so this brings perf.yml in line with what duel init generates now.

The Duel job checks out and runs the pull request's code, benchmarks included, and it also had pull-requests: write so it could post its report. Now it only reads, and its checkout keeps no credentials. It uploads the report as an artifact, and a separate Comment job downloads it and posts the comment. That job checks nothing out and runs none of the branch's code, so it's the only one that can write. A pull request from a fork still gets no comment, since its token can't write one, but the job's summary has the report.

The other fix is in .duel.rb. Duel only compares a change that touches lib/ or ext/, so a pull request that changed only the benchmarks or the config passed with "nothing to compare", and a broken benchmark could slip through. It now watches duel/, .duel.rb and the workflow too. That also means this pull request runs a real comparison of the code with itself, which checks that the new workflow works.

The Duel job checks out and runs the pull request's code, benchmarks
included, so it no longer has pull-requests: write, and its checkout
keeps no credentials. It uploads its report, and a Comment job of its
own, which checks nothing out and runs none of the branch's code,
posts the comment.

.duel.rb also watches the benchmarks, itself and the Perf workflow, so
a pull request that changes only those is compared, instead of passing
with nothing to compare.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-26T01:32:04.515508Z 8859c6d PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown

Duel passed: none of 28 benchmarks slower

  • Baseline: c2bf295, where HEAD left origin/main
  • Candidate: a3761ae

28 benchmarks in 14s. A timing is slower when a 99.0% confidence interval for its ratio, the candidate's to the baseline's, is above 1.02.

Benchmark Verdict Timings (ratio [interval]) Counts an iteration
adapter: parse in with_adapter unsettled cpu 1.006 [0.990, 1.022] undecided
generate: a large set of records unsettled cpu 1.020 [1.005, 1.165] undecided

Across the board

  • cpu 1.001 [0.998, 1.006] undecided (limit 1.0050, 28 benchmarks)

Unsettled

A timing too close to the limit to settle doesn't fail the run, but one that's unsettled in run after run is noisy: adapter: parse in with_adapter, generate: a large set of records.

26 benchmarks unchanged
Benchmark Timings (ratio [interval])
adapter: resolve one by name cpu 1.006 [0.990, 1.019]
options: a cache miss in a full cache cpu 1.005 [0.993, 1.014]
legacy: MultiJson.load cpu 1.009 [0.998, 1.018]
legacy: MultiJSON.dump cpu 0.991 [0.944, 1.019]
generate: a tiny hash cpu 0.997 [0.972, 1.015]
generate: a small flat object cpu 1.002 [0.992, 1.019]
generate: a small set of records cpu 1.000 [0.995, 1.016]
generate: a large array-heavy document cpu 1.000 [0.990, 1.008]
generate: pretty cpu 0.974 [0.936, 1.012]
generate: with Oj cpu 0.970 [0.603, 1.006]
generate: pretty, with Oj cpu 1.008 [0.995, 1.018]
generate: through fast_jsonparser cpu 1.000 [0.991, 1.016]
parse: a tiny object cpu 0.998 [0.985, 1.019]
parse: a scalar cpu 1.002 [0.995, 1.020]
parse: blank input cpu 1.003 [0.993, 1.017]
parse: a small flat object cpu 1.004 [0.997, 1.013]
parse: a small set of records cpu 0.998 [0.974, 1.005]
parse: a medium set of records cpu 1.001 [0.987, 1.008]
parse: a large set of records cpu 1.002 [0.992, 1.017]
parse: a large array-heavy document cpu 1.000 [0.989, 1.010]
parse: with symbolize_names cpu 1.009 [1.000, 1.017]
parse: from an IO cpu 1.001 [0.981, 1.017]
parse: binary-encoded input cpu 0.998 [0.990, 1.015]
parse: invalid JSON cpu 0.995 [0.979, 1.016]
parse: with Oj cpu 0.985 [0.954, 1.007]
parse: with fast_jsonparser cpu 1.008 [0.979, 1.019]

@sferik
sferik merged commit 5f48d8b into main Oct 26, 2026
40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant