Skip to content

Expand benchmark coverage with new modules and comprehensive workloads - #35

Merged
shakayami merged 1 commit into
masterfrom
claude/performance-test-random-cases-ipd9ob
Jul 17, 2026
Merged

Expand benchmark coverage with new modules and comprehensive workloads#35
shakayami merged 1 commit into
masterfrom
claude/performance-test-random-cases-ipd9ob

Conversation

@shakayami

Copy link
Copy Markdown
Owner

Summary

This PR significantly expands the benchmark suite by adding workloads for five new modules (fenwicktree, acl_math, acl_string, prime_fact, two_sat) and enhancing existing benchmarks to exercise all public methods of their respective modules. The changes ensure more comprehensive performance regression detection across the codebase.

Key Changes

  • Enhanced module documentation: Updated the workloads module docstring to explain the philosophy of using large data sizes and exercising all public methods to catch regressions in any single method.

  • New module imports: Added imports for fenwicktree, acl_math, acl_string, prime_fact, and two_sat modules.

  • Expanded segtree benchmark:

    • Increased problem size from 50K to 200K elements
    • Added coverage for all public methods: get(), all_prod(), max_right(), and min_left() in addition to existing set() and prod()
  • Expanded lazysegtree benchmark:

    • Increased problem size from 50K to 100K elements
    • Added coverage for set(), get(), all_prod(), apply_point(), max_right(), and min_left() methods
  • Expanded dsu benchmark:

    • Increased problem size from 100K to 300K elements
    • Restructured to exercise all public methods: leader(), merge(), same(), size(), and groups()
  • Expanded maxflow benchmark:

    • Increased problem size from 2K nodes/8K edges to 50K nodes/200K edges
    • Added coverage for get_edge(), edges(), change_edge(), and min_cut() methods
  • Expanded mincostflow benchmark:

    • Increased problem size from 300 nodes/1.5K edges to 5K nodes/25K edges
    • Added coverage for slope(), get_edge(), and edges() methods
  • Enhanced scc benchmark: Increased problem size from 50K to 120K nodes with proportional edge increase

  • Enhanced fps benchmark: Added coverage for diff(), integral(), addition, and subtraction operations

  • New fenwicktree benchmark: Comprehensive workload exercising add() and sum() operations on 300K elements with 300K queries

  • New acl_math benchmark: Batch queries for inv_mod(), inv_gcd(), crt(), and floor_sum() functions with 100K queries each

  • New acl_string benchmark: Workload for suffix_array(), lcp_array(), z_algorithm() and string operations on 100K character string

  • New prime_fact benchmark: Comprehensive testing of is_probable_prime(), prime_fact(), divisors(), totient(), and lcm() functions

  • New two_sat benchmark: Large random 2-SAT instance with 200K variables and 400K clauses

Implementation Details

  • All new benchmarks follow the established pattern of separate build and run functions
  • Workloads use deterministic seeding (via _rng()) for reproducibility
  • Query distributions are carefully chosen to exercise different code paths (e.g., segtree uses 30% set, 20% get, 25% prod, etc.)
  • All benchmarks are registered in the BENCHMARKS list for integration with the test framework

https://claude.ai/code/session_01CZ5PzQCeMYkhV9xUhfZmxP

…om data

Each already-benchmarked structure now exercises every public method it
provides (e.g. dsu drives leader/merge/same/size/groups, not just
merge/same), and coverage is extended to modules that had no benchmark
before: fenwicktree, acl_math, acl_string, prime_fact, two_sat. Dataset
sizes are increased substantially and randomly generated throughout,
since these benchmarks only care about execution time, not results.
@github-actions

Copy link
Copy Markdown

📊 Performance Benchmark Results

⏱️ Execution time (mean, lower is better)

Benchmark Base PR Delta
acl_math n/a 1.568s n/a
acl_string n/a 212.19ms n/a
convolution 763.62ms 822.63ms +7.7% ⚠️
dsu 81.36ms 613.32ms +653.9% ⚠️
fenwicktree n/a 1.112s n/a
fps 457.81ms 506.19ms +10.6% ⚠️
lazysegtree 1.210s 2.124s +75.5% ⚠️
maxflow 7.60ms 799.27ms +10414.0% ⚠️
mincostflow 3.41ms 39.24ms +1049.7% ⚠️
prime_fact n/a 1.691s n/a
scc 121.63ms 438.33ms +260.4% ⚠️
segtree 176.97ms 606.73ms +242.9% ⚠️
two_sat n/a 1.357s n/a

💾 Peak memory (lower is better)

Benchmark Base PR Delta
acl_math n/a 4.9KiB n/a
acl_string n/a 13.5MiB n/a
convolution 16.1MiB 16.1MiB +0.0%
dsu 849.2KiB 40.4MiB +4773.7% ⚠️
fenwicktree n/a 8.9MiB n/a
fps 3.9MiB 3.9MiB +0.0%
lazysegtree 5.0MiB 9.8MiB +97.2% ⚠️
maxflow 2.3MiB 97.0MiB +4133.3% ⚠️
mincostflow 662.2KiB 16.3MiB +2423.4% ⚠️
prime_fact n/a 8.5KiB n/a
scc 13.8MiB 32.7MiB +137.4% ⚠️
segtree 1.5MiB 6.0MiB +308.0% ⚠️
two_sat n/a 174.0MiB n/a

Base = target branch, PR = this branch. Negative delta = improvement. ✅ improvement ≥ 1%, ⚠️ regression > 5%. "n/a" base entries mean the target branch has no benchmark suite yet.

@shakayami
shakayami merged commit 03c69d4 into master Jul 17, 2026
20 checks passed
@shakayami
shakayami deleted the claude/performance-test-random-cases-ipd9ob branch July 17, 2026 13:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants