Expand benchmark coverage with new modules and comprehensive workloads - #35
Merged
Merged
Conversation
…om data Each already-benchmarked structure now exercises every public method it provides (e.g. dsu drives leader/merge/same/size/groups, not just merge/same), and coverage is extended to modules that had no benchmark before: fenwicktree, acl_math, acl_string, prime_fact, two_sat. Dataset sizes are increased substantially and randomly generated throughout, since these benchmarks only care about execution time, not results.
📊 Performance Benchmark Results⏱️ Execution time (mean, lower is better)
💾 Peak memory (lower is better)
Base = target branch, PR = this branch. Negative delta = improvement. ✅ improvement ≥ 1%, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR significantly expands the benchmark suite by adding workloads for five new modules (fenwicktree, acl_math, acl_string, prime_fact, two_sat) and enhancing existing benchmarks to exercise all public methods of their respective modules. The changes ensure more comprehensive performance regression detection across the codebase.
Key Changes
Enhanced module documentation: Updated the workloads module docstring to explain the philosophy of using large data sizes and exercising all public methods to catch regressions in any single method.
New module imports: Added imports for fenwicktree, acl_math, acl_string, prime_fact, and two_sat modules.
Expanded segtree benchmark:
get(),all_prod(),max_right(), andmin_left()in addition to existingset()andprod()Expanded lazysegtree benchmark:
set(),get(),all_prod(),apply_point(),max_right(), andmin_left()methodsExpanded dsu benchmark:
leader(),merge(),same(),size(), andgroups()Expanded maxflow benchmark:
get_edge(),edges(),change_edge(), andmin_cut()methodsExpanded mincostflow benchmark:
slope(),get_edge(), andedges()methodsEnhanced scc benchmark: Increased problem size from 50K to 120K nodes with proportional edge increase
Enhanced fps benchmark: Added coverage for
diff(),integral(), addition, and subtraction operationsNew fenwicktree benchmark: Comprehensive workload exercising
add()andsum()operations on 300K elements with 300K queriesNew acl_math benchmark: Batch queries for
inv_mod(),inv_gcd(),crt(), andfloor_sum()functions with 100K queries eachNew acl_string benchmark: Workload for
suffix_array(),lcp_array(),z_algorithm()and string operations on 100K character stringNew prime_fact benchmark: Comprehensive testing of
is_probable_prime(),prime_fact(),divisors(),totient(), andlcm()functionsNew two_sat benchmark: Large random 2-SAT instance with 200K variables and 400K clauses
Implementation Details
_rng()) for reproducibilityBENCHMARKSlist for integration with the test frameworkhttps://claude.ai/code/session_01CZ5PzQCeMYkhV9xUhfZmxP