Skip to content

scc: speed up iterative DFS by dropping tuple-based stack simulation - #36

Merged
shakayami merged 1 commit into
masterfrom
claude/scc-performance-08bbah
Jul 30, 2026
Merged

scc: speed up iterative DFS by dropping tuple-based stack simulation#36
shakayami merged 1 commit into
masterfrom
claude/scc-performance-08bbah

Conversation

@shakayami

Copy link
Copy Markdown
Owner

Summary

  • The previous scc() simulated recursion by pushing (v, bef, t) tuples onto a stack — up to two tuple allocations per edge, plus duplicate pushes when multiple frontier nodes shared an undiscovered neighbor (detected and discarded later at pop time).
  • Replaced it with the standard "one frame per active vertex + resumable edge-iterator index" iterative DFS, so each vertex is pushed at most once and each edge is scanned exactly once. Also flattened the nested dfs() closure into the main function body to avoid LOAD_DEREF overhead on low/Ord/visited.
  • Output is byte-for-byte identical to the previous implementation (verified below) — no behavior change, no API change.

Why

Reported that stuffing tuples onto the stack looked like the bottleneck in scc(); profiling/benchmarking confirmed it.

Test plan

  • python3 -m unittest discover -s tests — all 146 existing tests pass (including two_sat.py, which depends on scc)
  • Randomized differential test: 3500+ random graphs (up to 60 vertices/200 edges, and smaller dense graphs) comparing old vs. new implementation output — 0 mismatches
  • Benchmarked old vs. new on larger graphs:
    • random n=200000, m=200000: 2.24x faster
    • random n=100000, m=400000: 1.29x faster
    • random n=50000, m=500000: 1.87x faster
    • chain n=200000: 1.38x faster
    • big single cycle n=300000: 2.31x faster
    • fan-in heavy DAG n=4000, m=100000: 1.21x faster
  • Repo's existing benchmarks/workloads.py already includes an scc workload for the perf-comparison CI

Generated by Claude Code

The previous implementation pushed (v, bef, t) tuples onto the stack to
simulate recursion, allocating up to two tuples per edge and, in graphs
where multiple frontier nodes share an undiscovered neighbor, pushing
duplicate entries that had to be detected and discarded later.

Replace it with the standard technique of keeping one stack frame per
active vertex, paired with a resumable edge-iterator index, so each
vertex is pushed at most once and each edge is scanned once. This
avoids per-edge tuple allocation/unpacking and closure variable lookups
from the nested dfs() function.

Verified against the previous implementation with randomized
differential testing (thousands of graphs) for identical output, and
benchmarked 1.2x-2.3x faster across random, chain, cycle, and fan-in
graphs up to 300k vertices.
@github-actions

Copy link
Copy Markdown

📊 Performance Benchmark Results

⏱️ Execution time (mean, lower is better)

Benchmark Base PR Delta
acl_math 2.032s 2.013s -0.9%
acl_string 247.57ms 249.11ms +0.6%
convolution 921.48ms 954.50ms +3.6%
dsu 705.78ms 711.16ms +0.8%
fenwicktree 1.243s 1.211s -2.5% ✅
fps 604.20ms 626.61ms +3.7%
lazysegtree 2.576s 2.512s -2.5% ✅
maxflow 838.78ms 880.46ms +5.0%
mincostflow 47.00ms 48.41ms +3.0%
prime_fact 2.098s 2.078s -1.0%
scc 470.20ms 411.38ms -12.5% ✅
segtree 830.66ms 815.03ms -1.9% ✅
two_sat 1.542s 1.516s -1.7% ✅

💾 Peak memory (lower is better)

Benchmark Base PR Delta
acl_math 4.9KiB 4.9KiB +0.0%
acl_string 13.5MiB 13.5MiB +0.0%
convolution 16.1MiB 16.1MiB +0.0%
dsu 40.4MiB 40.4MiB +0.0%
fenwicktree 8.9MiB 8.9MiB +0.0%
fps 3.9MiB 3.9MiB +0.0%
lazysegtree 9.8MiB 9.8MiB +0.0%
maxflow 97.0MiB 97.0MiB +0.0%
mincostflow 16.3MiB 16.3MiB +0.0%
prime_fact 8.5KiB 8.5KiB +0.0%
scc 32.7MiB 25.1MiB -23.4% ✅
segtree 6.0MiB 6.0MiB +0.0%
two_sat 174.0MiB 174.0MiB +0.0%

Base = target branch, PR = this branch. Negative delta = improvement. ✅ improvement ≥ 1%, ⚠️ regression > 5%. "n/a" base entries mean the target branch has no benchmark suite yet.

@shakayami
shakayami merged commit 795c402 into master Jul 30, 2026
20 checks passed
@shakayami
shakayami deleted the claude/scc-performance-08bbah branch July 30, 2026 14:47
@shakayami shakayami linked an issue Jul 30, 2026 that may be closed by this pull request
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

scc

2 participants