Skip to content

Make OpenCV performance comparisons equivalent#35

Merged
bghira merged 1 commit into
mainfrom
agent/fair-opencv-benchmarks
Jul 17, 2026
Merged

Make OpenCV performance comparisons equivalent#35
bghira merged 1 commit into
mainfrom
agent/fair-opencv-benchmarks

Conversation

@bghira

@bghira bghira commented Jul 17, 2026

Copy link
Copy Markdown
Owner

Summary

  • compare resize against direct cv2.resize without unrelated RGB/BGR conversions
  • compare luminance against direct COLOR_RGB2GRAY
  • apply the same equivalent-work rule to the combined pipeline benchmark
  • tighten resize output validation and report neutral relative timings
  • clarify that TSR hides loops behind one Python call rather than eliminating them

Root cause

The competitive benchmarks charged two color conversions to OpenCV resize and an unnecessary RGB-to-BGR conversion to OpenCV luminance. The output then labeled the result a TSR advantage regardless of which implementation won. This obscured the actual resize bottleneck and overstated current performance.

Observed corrected result

On Apple M3 Max with the current main implementation and mixed image shapes:

  • direct cv2 resize took about 0.31x TSR time
  • OpenCV luminance took about 2.03x TSR time
  • the direct OpenCV combined pipeline took about 0.76x TSR time

These are diagnostic point-in-time results, not assertions baked into the tests.

Validation

  • corrected resize, luminance, and pipeline comparisons pass
  • Python test suite passes
  • repository formatting, lint, and competitive benchmark hooks pass

@bghira
bghira marked this pull request as ready for review July 17, 2026 15:42
@bghira
bghira requested a review from Copilot July 17, 2026 15:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adjusts the competitive performance benchmarks to ensure TSR vs OpenCV comparisons measure equivalent work (removing unrelated color conversions) and to report results neutrally without implying TSR always wins.

Changes:

  • Update resize benchmarking to compare TSR against direct cv2.resize (no RGB↔BGR conversions) and tighten output validation.
  • Update luminance benchmarking to compare TSR against cv2.cvtColor(..., COLOR_RGB2GRAY) directly (no extra RGB→BGR step).
  • Apply the same “equivalent-work” rule and neutral relative-time reporting to the combined pipeline benchmark, plus clarify messaging about loops vs single Python calls.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@bghira
bghira merged commit acf605f into main Jul 17, 2026
11 checks passed
@bghira
bghira deleted the agent/fair-opencv-benchmarks branch July 17, 2026 16:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants