Repository navigation
Conversation
Benchmark parse and generate through the default adapter in the shapes and sizes of benchmark.rb, plus blank, scalar, IO, binary and invalid input, options, the Oj and fast_jsonparser adapters, adapter resolution, the options cache and the deprecated MultiJson names. The perf workflow compares each pull request with main, each push with the commit before it, and the last release on a schedule. Duel needs Ruby 3.4 or later, so the Gemfile and Rakefile skip it on older Rubies.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Duel: nothing to compare |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds Duel, a gem I wrote to catch performance regressions in CI. On every pull request it runs the benchmarks in
duel/againstmainand the branch side by side, and fails the build if the branch is slower or allocates more.The 28 benchmarks reuse the payload shapes from
benchmark.rb. They parse and generate everything from a tiny object to a 25,000-item array through the default adapter, plus blank and scalar input, IO, binary strings, invalid JSON,symbolize_names, pretty printing, the Oj and fast_jsonparser adapters, adapter lookup, the options cache, and the oldMultiJsonnames.Most of the time on big documents is spent in the JSON library, not in MultiJSON, so those benchmarks mostly catch extra allocations. The small ones are what catch overhead in MultiJSON itself, which is where a regression is most likely to come from anyway.
To check that it actually catches things, I added an
options.duptoAdapter.dump, and Duel failed all nine generate benchmarks and none of the parse ones. A bit of extra work inMultiJSON.parsefailed 12 benchmarks. Comparingmainwith itself passed three times in a row.Duel requires Ruby 3.4, so the Gemfile only adds it on 3.4 and newer, and the Rake task is skipped when it isn't installed, like the Steep task already is.
bundle installand the tests still work on 3.2, 3.3, and JRuby.