From c4a02e9781c6acca4965f6066e424863774ab729 Mon Sep 17 00:00:00 2001 From: bghira Date: Fri, 17 Jul 2026 10:57:13 -0600 Subject: [PATCH] docs: cleanup/rewrite --- BENCHMARKS.md | 191 +++++++++------------- README.md | 223 ++++++++++--------------- docs/API_COMPAT_CV2.md | 278 +++++++++++++------------------ docs/BUILDING_STATIC_OPENCV.md | 289 ++++++++++++--------------------- 4 files changed, 388 insertions(+), 593 deletions(-) diff --git a/BENCHMARKS.md b/BENCHMARKS.md index 287b28a..02cb55e 100644 --- a/BENCHMARKS.md +++ b/BENCHMARKS.md @@ -1,137 +1,102 @@ -# Performance Benchmarks +# Benchmarks -TrainingSample includes benchmarks for common preprocessing operations: crop, resize, luminance, resize-plus-luminance pipelines, and video frame resizing. The benchmarks are meant to catch regressions and provide workload-specific guidance, not to guarantee universal speedups over OpenCV or NumPy. +## Commands -## Running Benchmarks - -Use the repository virtual environment when available: +Build the current checkout before measuring: ```bash -.venv/bin/python -m pytest tests/test_performance_benchmarks.py -q -s +python -m venv .venv +.venv/bin/python -m pip install -e '.[dev]' +.venv/bin/maturin develop --release ``` -To run every Python test and benchmark marker in the repo: +Run the performance suite: ```bash -.venv/bin/python -m pytest -q +.venv/bin/python -m pytest tests/test_performance_benchmarks.py -q -s ``` -For a fresh source build before measuring: +Run pytest-benchmark cases only: ```bash -env -u OPENCV_LINK_LIBS -u OPENCV_LINK_PATHS -u OPENCV_INCLUDE_PATHS \ - -u LIBCLANG_PATH -u LLVM_CONFIG_PATH \ - .venv/bin/maturin develop --release +.venv/bin/python -m pytest \ + tests/test_performance_benchmarks.py::TestDetailedBenchmarks \ + --benchmark-only ``` -The OpenCV Rust binding needs a discoverable OpenCV and Clang installation. On this development host, stale macOS-style OpenCV and LLVM environment variables had to be unset before the build could probe the system OpenCV installation. - -## Current Local Snapshot - -Last measured command: +Save machine-readable results: ```bash -.venv/bin/python -m pytest tests/test_performance_benchmarks.py -q -s +.venv/bin/python -m pytest \ + tests/test_performance_benchmarks.py::TestDetailedBenchmarks \ + --benchmark-only \ + --benchmark-json benchmark.json ``` -Environment: - -- Linux x86_64 -- CPython 3.13 -- NumPy 2.3.4 -- system OpenCV 4.11 via the Rust `opencv` crate -- release build installed with `maturin develop --release` - -Point-in-time scenario timings from the benchmark output: - -| Scenario | Before optimization | After optimization | Comparison after optimization | -|----------|---------------------|--------------------|-------------------------------| -| Crop batch, 16 images | 22.9 ms | 0.4 ms | NumPy slicing was still faster because it returns views | -| Mixed-shape crop, 8 images | 50.2 ms | 3.3 ms | NumPy slicing loop was near-zero because it returns views | -| Resize, 4 mixed-size images | 4.1 ms | 0.4 ms | OpenCV loop: 2.6 ms | -| Luminance, 4 mixed-size images | 10.4 ms | 0.6 ms | OpenCV loop: 0.9 ms | -| Resize + luminance pipeline, 4 images | 5.9 ms | 0.6 ms | OpenCV loop: 2.1 ms | -| Mixed-shape luminance, 6 images | 78.3 ms | 3.3 ms | NumPy loop: 19.4 ms | - -Pytest-benchmark means from the same focused run: - -| Benchmark | Mean | -|-----------|------| -| Center crop | 55.2 us | -| Resize operations | 353.1 us | -| Luminance calculation | 417.2 us | -| Crop operations | 583.8 us | -| Pipeline | 3.44 ms | -| Video processing | 2.85 ms | - -A full `pytest -q` run also passed and produced similar benchmark ordering, with normal run-to-run variance. - -## What Changed in the Latest Optimization - -- Owned Rust `ndarray` outputs are transferred into NumPy with `from_owned_array_bound`, avoiding an additional copy in Python-facing result conversion. -- Contiguous luminance inputs use a channel-sum fast path. Instead of computing weighted luminance per pixel, it sums R, G, and B separately and applies the weights once at the end. -- Non-contiguous arrays still use the general ndarray path for correctness. - -## Benchmark Categories - -### Image Operations - -- `batch_crop_images` -- `batch_center_crop_images` -- `batch_random_crop_images` -- `batch_resize_images` -- `batch_calculate_luminance` - -### Pipeline Operations - -- resize followed by luminance -- crop followed by resize -- mixed input sizes and output sizes - -### Video Operations - -- `batch_resize_videos` with frame batches shaped `(T, H, W, 3)` - -## Interpreting Results - -Use these benchmarks to answer practical questions: - -- Is a change adding extra Rust-to-NumPy copies? -- Are contiguous arrays staying on the fast path? -- Is resize dominated by OpenCV work or Python binding overhead? -- Does a mixed-shape batch still behave reasonably? -- Is a video processing change accidentally introducing per-frame Python overhead? - -Some comparisons need context: - -- NumPy crop by slicing often returns a view, so it can be much faster than any function that returns owned cropped arrays. -- Very small images can be dominated by Python call overhead. -- Large images can be dominated by memory bandwidth rather than arithmetic. -- OpenCV performance varies by build options, CPU features, and linked libraries. - -## Quality Checks - -The tests validate basic output behavior alongside timing: +## Measured operations + +| Case | TrainingSample call | Reference | +|---|---|---| +| Crop | `batch_crop_images` | NumPy slicing | +| Resize | `batch_resize_images` | direct `cv2.resize(..., INTER_LINEAR)` loop | +| Luminance | `batch_calculate_luminance` | `cv2.cvtColor(..., COLOR_RGB2GRAY)` plus `numpy.mean` | +| Pipeline | resize, then luminance | equivalent OpenCV loop | +| Video resize | `batch_resize_videos` | no external baseline | +| Center crop | `batch_center_crop_images` | no external baseline | + +## Comparison constraints + +| Topic | Constraint | +|---|---| +| Crop ownership | TrainingSample returns owned arrays; plain NumPy slicing returns views | +| Resize color order | Resize is applied directly to the same RGB byte arrays; no RGB/BGR conversion is included | +| Resize interpolation | Both resize paths use linear interpolation | +| Luminance input | Both paths interpret the input as RGB | +| Batch shape | Mixed-shape inputs are processed one image at a time by the OpenCV reference loop | +| Build mode | Measure release builds only | + +NumPy view-returning crop timings do not measure the cost of producing owned, +contiguous output. Use `.copy()` when owned-output cost is the subject of the +comparison. + +## Result metadata + +Record these fields with timing results: + +```text +git commit +operating system and architecture +CPU model +Python version +NumPy version +TrainingSample version +Rust OpenCV version and link mode +Python cv2 version +OpenCV thread count +input shapes and dtypes +batch size +target sizes +warm-up count +sample count +median and dispersion +``` -- Crop outputs have expected shape and match NumPy slicing where ownership differences do not matter. -- Resize outputs have expected shape and are close to OpenCV output for the configured interpolation. -- Luminance stays within a small tolerance of NumPy/OpenCV-style references. -- Non-contiguous arrays are accepted by safe luminance paths and rejected by strict zero-copy crop/resize paths. +## Test behavior -## Regression Signals +The performance file contains both benchmarks and assertions. -Investigate if a change causes: +| Assertion type | Examples | +|---|---| +| Correctness | output count, shape, dtype, pixel equality, luminance tolerance | +| Resource behavior | repeated calls, concurrent calls, memory cleanup | +| Broad timing guard | completion limits and scaling bounds intended for CI | +| Statistical timing | pytest-benchmark cases in `TestDetailedBenchmarks` | -- Public batch crop to return to multi-millisecond timings for small batches. -- Luminance on contiguous RGB arrays to lose the channel-sum fast path. -- Resize benchmarks to add large overhead beyond OpenCV work. -- Video resizing to scale with per-frame Python object churn. -- Memory usage to grow unexpectedly for repeated batch calls. +Single-run `time.perf_counter` output is diagnostic. It is not a stable result +across hosts or OpenCV builds. -## Future Benchmark Work +## Published numbers -- Store historical benchmark results by commit and host. -- Add explicit memory allocation tracking for Python-facing APIs. -- Separate view-returning crop comparisons from owned-output crop comparisons. -- Add more video pipeline benchmarks. -- Document hardware and OpenCV build details in benchmark artifacts. +This repository does not currently store benchmark artifacts keyed by commit, +host, and OpenCV configuration. Fixed timing tables are therefore not included +in this document. diff --git a/README.md b/README.md index da285c0..dcf6446 100644 --- a/README.md +++ b/README.md @@ -2,179 +2,136 @@ [![Crates.io](https://img.shields.io/crates/v/trainingsample.svg)](https://crates.io/crates/trainingsample) [![PyPI](https://img.shields.io/pypi/v/trainingsample.svg)](https://pypi.org/project/trainingsample/) -[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) +[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE) -TrainingSample provides Rust-backed Python bindings for common image and video preprocessing operations used in ML data pipelines. It combines OpenCV-backed resizing with Rust implementations for batching, cropping, luminance calculation, format conversion, and video helpers. +Rust image and video operations exposed through Python and Rust APIs. -The project is designed for workloads where Python-side loops and repeated boundary crossings become visible. It is not a blanket replacement for all of `cv2`, and performance depends on image size, batch shape, CPU, OpenCV build, and memory bandwidth. +| Item | Value | +|---|---| +| Python package | `trainingsample` | +| Rust crate | `trainingsample` | +| Python version | 3.11 or newer | +| Python array type | NumPy `uint8` | +| Image layout | `(height, width, channels)` | +| Video layout | `(frames, height, width, channels)` | +| Size tuple order | `(width, height)` | +| License | MIT | -## install +## Install ```bash -# python -pip install trainingsample - -# rust +python -m pip install trainingsample cargo add trainingsample ``` -## python usage +## Python example ```python import numpy as np import trainingsample as tsr images = [ - np.random.randint(0, 255, (480, 640, 3), dtype=np.uint8) + np.random.randint(0, 256, (480, 640, 3), dtype=np.uint8) for _ in range(8) ] -crop_boxes = [(50, 50, 200, 200)] * len(images) -cropped = tsr.batch_crop_images(images, crop_boxes) - -target_sizes = [(224, 224)] * len(images) -resized = tsr.batch_resize_images(images, target_sizes) - -luminances = tsr.batch_calculate_luminance(resized) +cropped = tsr.batch_crop_images( + images, + [(50, 50, 320, 320)] * len(images), +) +resized = tsr.batch_resize_images( + cropped, + [(224, 224)] * len(cropped), +) +luminance = tsr.batch_calculate_luminance(resized) ``` -OpenCV-compatible helpers are also exported for common operations: +## Python functions -```python -decoded = tsr.imdecode(image_bytes, tsr.IMREAD_COLOR) -gray = tsr.cvt_color(decoded, tsr.COLOR_RGB2GRAY) -edges = tsr.canny(decoded, threshold1=50, threshold2=150) -resized = tsr.resize(decoded, (224, 224), interpolation=tsr.INTER_LINEAR) -``` +| Function | Input | Output | +|---|---|---| +| `load_image_batch(paths)` | file paths | `list[bytes \| None]` | +| `batch_crop_images(images, boxes)` | images; `(x, y, width, height)` per image | owned images | +| `batch_center_crop_images(images, sizes)` | images; target size per image | owned images | +| `batch_random_crop_images(images, sizes)` | images; target size per image | owned images | +| `batch_resize_images(images, sizes)` | RGB images; target size per image | owned RGB images | +| `batch_resize_videos(videos, sizes)` | RGB videos; target size per video | owned RGB videos | +| `batch_calculate_luminance(images)` | images | `list[float]` | +| `rgb_to_rgba_optimized(image, alpha)` | RGB image; `uint8` alpha | RGBA image and elapsed time | +| `rgba_to_rgb_optimized(image)` | RGBA image | RGB image and elapsed time | -## rust usage +Specialized entry points: -```rust -use ndarray::Array3; -use trainingsample::{ - batch_calculate_luminance_arrays, batch_crop_image_arrays, batch_resize_image_arrays, -}; +| Function | Constraint | +|---|---| +| `batch_crop_images_zero_copy` | C-contiguous input | +| `batch_center_crop_images_zero_copy` | C-contiguous input | +| `batch_resize_images_zero_copy` | C-contiguous, three-channel input | +| `batch_resize_images_iterator` | C-contiguous, three-channel input | +| `batch_calculate_luminance_zero_copy` | accepts ndarray views | -let images: Vec> = (0..10) - .map(|_| Array3::zeros((480, 640, 3))) - .collect(); +The compatibility helpers use names such as `imdecode_py`, `cvt_color_py`, and +`resize_py`. See [the exact Python export table](docs/API_COMPAT_CV2.md). -let crop_boxes = vec![(50, 50, 200, 200); 10]; // (x, y, width, height) -let cropped = batch_crop_image_arrays(&images, &crop_boxes); +## Rust example -let target_sizes = vec![(224, 224); 10]; // (width, height) -let resized = batch_resize_image_arrays(&images, &target_sizes); +```rust +use ndarray::Array3; +use trainingsample::crop_image_array; -let luminances = batch_calculate_luminance_arrays(&images); +let image = Array3::::zeros((480, 640, 3)); +let cropped = crop_image_array(&image.view(), 50, 50, 320, 320).unwrap(); +assert_eq!(cropped.dim(), (320, 320, 3)); ``` -## api reference - -### `batch_crop_images(images, crop_boxes)` - -- `images`: list of NumPy arrays shaped `(H, W, C)` with `uint8` data -- `crop_boxes`: list of `(x, y, width, height)` tuples -- returns: list of cropped NumPy arrays -- notes: output arrays are owned by NumPy without an extra copy from the owned Rust array - -### `batch_center_crop_images(images, target_sizes)` - -- `images`: list of NumPy arrays shaped `(H, W, C)` with `uint8` data -- `target_sizes`: list of `(width, height)` tuples -- returns: list of center-cropped NumPy arrays - -### `batch_random_crop_images(images, target_sizes)` - -- `images`: list of NumPy arrays shaped `(H, W, C)` with `uint8` data -- `target_sizes`: list of `(width, height)` tuples -- returns: list of randomly cropped NumPy arrays - -### `batch_resize_images(images, target_sizes)` - -- `images`: list of NumPy arrays shaped `(H, W, 3)` with `uint8` data -- `target_sizes`: list of `(width, height)` tuples -- returns: list of resized NumPy arrays -- implementation: OpenCV-backed resize with Rust/PyO3 conversion handling +## Implementations -### `batch_calculate_luminance(images)` +| Operation | Implementation | +|---|---| +| Decode | `image` crate; JPEG, PNG, and WebP features enabled | +| Crop | Rust row copies for contiguous arrays; ndarray fallback for strided views | +| Luminance | Rust contiguous fast path; ndarray fallback | +| Image resize | OpenCV for `batch_resize_images` and named OpenCV resize helpers | +| Video resize | OpenCV frame resize into an owned four-dimensional output | +| Color conversion | Rust; SIMD feature used by optimized RGB/RGBA functions | +| Canny helper | `imageproc` | -- `images`: list of NumPy arrays shaped `(H, W, C)` with `uint8` data -- returns: list of float luminance values -- notes: contiguous RGB/RGBA-like arrays use a channel-sum fast path; strided arrays fall back to the general ndarray path +The default Cargo feature set is `simd`. Python wheels are built with +`python-bindings`, `opencv`, and `simd`; macOS release wheels also enable +`metal`. -### `batch_resize_videos(videos, target_sizes)` - -- `videos`: list of NumPy arrays shaped `(T, H, W, 3)` with `uint8` data -- `target_sizes`: list of `(width, height)` tuples -- returns: list of resized video NumPy arrays - -## current benchmark snapshot - -These numbers are from the local benchmark run after the latest Python-interface optimizations: +## Source build ```bash -.venv/bin/python -m pytest tests/test_performance_benchmarks.py -q -s +python -m pip install 'maturin>=1,<2' +maturin develop --release ``` -Environment: Linux x86_64, CPython 3.13, NumPy 2.3.4, system OpenCV 4.11 through the Rust `opencv` crate. Treat these as a point-in-time reference, not a cross-machine guarantee. - -| Benchmark | Before | After | Notes | -|-----------|--------|-------|-------| -| Crop batch, 16 images | 22.9 ms | 0.4 ms | Public `batch_crop_images` path | -| Mixed-shape crop, 8 images | 50.2 ms | 3.3 ms | Mixed input and output sizes | -| Luminance batch, 4 mixed images | 10.4 ms | 0.6 ms | Now faster than the OpenCV comparison in this run | -| Mixed-shape luminance, 6 images | 78.3 ms | 3.3 ms | NumPy comparison was 19.4 ms in this run | -| Complete resize + luminance pipeline | 5.9 ms | 0.6 ms | Four mixed-size inputs to 224x224 | - -Pytest-benchmark means from the same suite: - -| Benchmark | Mean after | -|-----------|------------| -| Center crop | 55.2 us | -| Resize operations | 353.1 us | -| Luminance calculation | 417.2 us | -| Crop operations | 583.8 us | -| Pipeline | 3.44 ms | -| Video processing | 2.85 ms | - -## architecture - -TrainingSample uses different implementations for different operation types: - -- Cropping: Rust/ndarray implementation with owned-array transfer into NumPy. -- Luminance: Rust channel-sum fast path for contiguous arrays, with a general ndarray fallback for non-contiguous inputs. -- Resize: OpenCV-backed implementation for image quality and mature interpolation behavior. -- Video resize: OpenCV-backed frame resizing with batched Python binding output. -- Format conversion: Rust SIMD implementation where the `simd` feature is enabled. - -The optimized path generally requires contiguous `uint8` arrays. Views such as `image[:, ::2, :]` remain supported by safe public APIs, but they may use slower fallback paths. - -## features - -- Python bindings through PyO3 and rust-numpy -- Batch APIs for images and videos -- OpenCV-compatible constants and helper functions for common operations -- Optional SIMD feature for format conversion and selected numeric paths -- Error handling for invalid dimensions, unsupported channels, and invalid crop bounds -- Source build support for dynamic or static OpenCV configurations - -## building from source +Build and test commands used by CI: ```bash -pip install maturin -maturin develop --release +cargo fmt --all -- --check +cargo clippy --all-targets --no-default-features \ + --features python-bindings,simd,opencv -- -D warnings +cargo test --no-default-features --features simd,opencv +python -m pytest -q ``` -The OpenCV Rust bindings need to find a working OpenCV and Clang installation. If the environment has stale OpenCV or LLVM variables, unset them before building: +OpenCV and libclang must be discoverable for source builds with the `opencv` +feature. Static wheel configuration is documented in +[docs/BUILDING_STATIC_OPENCV.md](docs/BUILDING_STATIC_OPENCV.md). -```bash -env -u OPENCV_LINK_LIBS -u OPENCV_LINK_PATHS -u OPENCV_INCLUDE_PATHS \ - -u LIBCLANG_PATH -u LLVM_CONFIG_PATH \ - maturin develop --release -``` +## Documentation -See [docs/BUILDING_STATIC_OPENCV.md](docs/BUILDING_STATIC_OPENCV.md) for static OpenCV bundle notes. +- [Benchmark definitions and commands](BENCHMARKS.md) +- [Python compatibility helpers](docs/API_COMPAT_CV2.md) +- [Static OpenCV build](docs/BUILDING_STATIC_OPENCV.md) -## license +## Limits -MIT. See [LICENSE](LICENSE). +- The Python API is not a drop-in replacement for `cv2`. +- Crop and resize functions return owned arrays. +- OpenCV resize paths require three-channel input. +- Strict zero-copy crop and resize entry points reject non-contiguous input. +- Runtime depends on input dimensions, batch size, host CPU, memory bandwidth, + OpenCV build flags, and OpenCV thread settings. diff --git a/docs/API_COMPAT_CV2.md b/docs/API_COMPAT_CV2.md index 04d2037..513bac8 100644 --- a/docs/API_COMPAT_CV2.md +++ b/docs/API_COMPAT_CV2.md @@ -1,216 +1,162 @@ -# OpenCV API Compatibility Guide +# Python compatibility helpers -TrainingSample exposes a subset of OpenCV-style image APIs plus batch-oriented helpers. The goal is to reduce Python loop overhead for common preprocessing workloads, not to implement the full `cv2` surface. +TrainingSample 0.3.0 exposes a small set of OpenCV-style operations. Exported +names and accepted integer codes are listed below. -## Quick Start +The module is not a drop-in `cv2` replacement. It does not export `IMREAD_*`, +`COLOR_*`, `VideoCapture`, `VideoWriter`, or `CascadeClassifier` under those +names. -```python -import cv2 -import numpy as np -import trainingsample as tsr -``` - -Use TrainingSample where a matching helper exists: - -```python -resized = tsr.resize(image, (224, 224), interpolation=tsr.INTER_LINEAR) -gray = tsr.cvt_color(image, tsr.COLOR_RGB2GRAY) -edges = tsr.canny(image, threshold1=50, threshold2=150) -``` - -For batches, prefer the batch APIs instead of a Python loop: - -```python -images = [load_image(path) for path in paths] -sizes = [(224, 224)] * len(images) - -resized = tsr.batch_resize_images(images, sizes) -luminances = tsr.batch_calculate_luminance(resized) -``` - -## Supported OpenCV-Style Operations - -### Image Decoding - -```python -with open("image.jpg", "rb") as f: - img_bytes = f.read() +## Exported functions -img = tsr.imdecode(img_bytes, tsr.IMREAD_COLOR) -img_gray = tsr.imdecode(img_bytes, tsr.IMREAD_GRAYSCALE) -``` +| Function | Signature | +|---|---| +| Decode | `imdecode_py(buf, flags)` | +| Color conversion | `cvt_color_py(src, code)` | +| Canny | `canny_py(image, threshold1, threshold2)` | +| Compatibility resize | `resize_py(src, dsize, interpolation=None)` | +| OpenCV bilinear resize | `resize_bilinear_opencv(image, target_width, target_height)` | +| OpenCV Lanczos resize | `resize_lanczos4_opencv(image, target_width, target_height)` | +| FourCC | `fourcc_py(c1, c2, c3, c4)` | +| OpenCV data path | `get_opencv_data_path_py()` | -### Color Space Conversion +## Decode ```python -gray = tsr.cvt_color(image, tsr.COLOR_RGB2GRAY) -bgr = tsr.cvt_color(image, tsr.COLOR_RGB2BGR) -``` +with open("image.jpg", "rb") as file: + encoded = file.read() -### Edge Detection - -```python -edges = tsr.canny(image, threshold1=50, threshold2=150) +rgb = tsr.imdecode_py(encoded, 1) +gray_rgb = tsr.imdecode_py(encoded, 0) +unchanged = tsr.imdecode_py(encoded, -1) ``` -### Image Resizing +| Flag | Meaning | +|---:|---| +| `-1` | unchanged channel count where supported | +| `0` | grayscale replicated into `(height, width, 3)` | +| `1` | RGB | -```python -resized = tsr.resize(image, (width, height), interpolation=tsr.INTER_LINEAR) -``` +Decode uses the Rust `image` crate. Cargo explicitly enables JPEG, PNG, and +WebP; the crate's default features are not disabled. Python `bytes` input is +retained without cloning while the GIL is released; `bytearray` input is copied +by PyO3 before decode. -Supported interpolation constants: +## Color conversion ```python -tsr.INTER_NEAREST -tsr.INTER_LINEAR -tsr.INTER_CUBIC -tsr.INTER_LANCZOS4 +gray = tsr.cvt_color_py(rgb, 7) +hsv = tsr.cvt_color_py(rgb, 41) ``` -## Batch Operations +| Code | Conversion | +|---:|---| +| `4` | BGR to RGB | +| `5` | RGB to BGR | +| `7` | RGB to grayscale | +| `8` | grayscale to RGB | +| `41` | RGB to HSV | +| `55` | HSV to RGB | -### Cropping +## Resize ```python -crop_boxes = [(x, y, width, height) for image in images] -cropped = tsr.batch_crop_images(images, crop_boxes) +resized = tsr.resize_py(rgb, (224, 224), tsr.INTER_LINEAR) +batch = tsr.batch_resize_images([rgb], [(224, 224)]) ``` -Center and random crop helpers use target sizes: +| Export | Value | +|---|---:| +| `INTER_NEAREST` | `0` | +| `INTER_LINEAR` | `1` | +| `INTER_CUBIC` | `2` | +| `INTER_LANCZOS4` | `4` | -```python -target_sizes = [(224, 224)] * len(images) -center_cropped = tsr.batch_center_crop_images(images, target_sizes) -random_cropped = tsr.batch_random_crop_images(images, target_sizes) -``` +`batch_resize_images`, `resize_bilinear_opencv`, and +`resize_lanczos4_opencv` use OpenCV. `resize_py` uses the Rust compatibility +implementation in `cv_compat`. -### Resizing +## Canny ```python -target_sizes = [(224, 224)] * len(images) -resized = tsr.batch_resize_images(images, target_sizes) +edges = tsr.canny_py(rgb, 50.0, 150.0) ``` -The public resize API returns a list of owned NumPy arrays. Current implementation uses OpenCV-backed resize internally and transfers owned Rust arrays into NumPy without an additional copy. +`canny_py` uses `imageproc` and returns a three-dimensional `uint8` NumPy +array. -### Luminance +## Video classes -```python -luminances = tsr.batch_calculate_luminance(images) -``` - -For contiguous arrays, luminance uses a channel-sum fast path. Non-contiguous arrays are accepted by the safe public API but may run through a slower ndarray fallback. - -### Video Resizing +| Class | Constructor | +|---|---| +| `PyVideoCapture` | `PyVideoCapture(filename)` | +| `PyVideoWriter` | `PyVideoWriter(filename, fourcc_str, fps, frame_size)` | ```python -videos = [video_array] # shape: (frames, height, width, 3) -target_sizes = [(224, 224)] -resized_videos = tsr.batch_resize_videos(videos, target_sizes) -``` - -## Zero-Copy Entry Points - -Some lower-level APIs expose stricter zero-copy behavior: +capture = tsr.PyVideoCapture("input.mp4") +opened = capture.is_opened() +ok, frame = capture.read() +capture.release() -```python -cropped = tsr.batch_crop_images_zero_copy(images, crop_boxes) -luminances = tsr.batch_calculate_luminance_zero_copy(images) -resized = tsr.batch_resize_images_zero_copy(images, target_sizes) -``` - -These functions are intended for contiguous arrays. Unsafe zero-copy crop and resize paths reject non-contiguous views with a `ValueError`. - -## Video Capture and Writing - -```python -cap = tsr.VideoCapture("video.mp4") - -if cap.is_opened(): - ret, frame = cap.read() - if ret: - luminance = tsr.batch_calculate_luminance([frame]) - -cap.release() -``` - -```python -fourcc = tsr.fourcc("M", "J", "P", "G") -writer = tsr.VideoWriter("output.avi", fourcc, 30.0, (width, height)) - -for frame in frames: +fourcc = tsr.fourcc_py("M", "J", "P", "G") +writer = tsr.PyVideoWriter("output.avi", fourcc, 30.0, (640, 480)) +if ok and frame is not None: writer.write(frame) - writer.release() ``` -## Object Detection +`PyVideoCapture.from_bytes(source, suffix=".mp4")` accepts bytes-like or +file-like input and stores it in a temporary file for the lifetime of the +capture object. -```python -classifier = tsr.CascadeClassifier("haarcascade_frontalface_alt.xml") -faces = classifier.detect_multi_scale(image) -``` +Supported `PyVideoCapture.get` property codes: -## Benchmark Snapshot +| Code | Property | +|---:|---| +| `3` | frame width | +| `4` | frame height | +| `5` | frames per second | +| `7` | frame count | -The following numbers came from the local benchmark suite after the latest Python-interface optimization work: +Other property codes return `0.0`. -```bash -.venv/bin/python -m pytest tests/test_performance_benchmarks.py -q -s -``` - -Environment: Linux x86_64, CPython 3.13, NumPy 2.3.4, system OpenCV 4.11 through the Rust `opencv` crate. - -| Scenario | TrainingSample | Comparison in same run | -|----------|----------------|------------------------| -| Batch resize, 4 mixed-size images | 0.4 ms | OpenCV loop: 2.6 ms | -| Batch luminance, 4 mixed-size images | 0.6 ms | OpenCV loop: 0.9 ms | -| Resize + luminance pipeline, 4 mixed-size images | 0.6 ms | OpenCV loop: 2.1 ms | -| Mixed-shape luminance, 6 images | 3.3 ms | NumPy loop: 19.4 ms | -| Mixed-shape crop, 8 images | 3.3 ms | NumPy slicing loop: near-zero because slicing returns views | - -The crop comparison is intentionally caveated: NumPy slicing can be effectively free when it returns a view. TrainingSample returns owned output arrays, which is the right comparison when the next stage needs independent contiguous buffers. - -## Migration Notes - -### Prefer Batch APIs for Repeated Work - -```python -# OpenCV loop -results = [cv2.resize(img, (224, 224)) for img in images] - -# TrainingSample batch call -results = tsr.batch_resize_images(images, [(224, 224)] * len(images)) -``` - -### Keep Inputs Contiguous When Performance Matters +## Cascade class ```python -if not image.flags["C_CONTIGUOUS"]: - image = np.ascontiguousarray(image) +classifier = tsr.PyCascadeClassifier("cascade.xml") +detections = classifier.detect_multi_scale(image) ``` -Public safe APIs accept many strided views, but contiguous arrays are usually faster and are required by strict zero-copy paths. +| Method | Signature | +|---|---| +| `empty` | `empty()` | +| `detect_multi_scale` | `detect_multi_scale(image, scale_factor=None, min_neighbors=None)` | -### Validate Your Own Workload +## Batch and specialized APIs -Image shape, interpolation, batch size, and memory bandwidth can change results. Benchmark the exact pipeline you intend to ship: +| Function | Input requirement | Return | +|---|---|---| +| `batch_crop_images` | ndarray image | list of owned arrays | +| `batch_center_crop_images` | ndarray image | list of owned arrays | +| `batch_random_crop_images` | ndarray image | list of owned arrays | +| `batch_resize_images` | C-contiguous RGB | list of owned arrays | +| `batch_resize_videos` | contiguous RGB frames | list of owned arrays | +| `batch_calculate_luminance` | ndarray image | list of floats | +| `batch_crop_images_zero_copy` | C-contiguous | list of owned arrays | +| `batch_center_crop_images_zero_copy` | C-contiguous | list of owned arrays | +| `batch_resize_images_zero_copy` | C-contiguous RGB | array for single input; list for batch input | +| `batch_resize_images_iterator` | C-contiguous RGB | `ResizeIterator` | -```python -import time +All crop boxes use `(x, y, width, height)`. All target sizes use +`(width, height)`. -start = time.perf_counter() -results = tsr.batch_resize_images(images, sizes) -duration = time.perf_counter() - start +## Exported class names -print(f"{len(images) / duration:.1f} images/sec") +```text +PyBatchProcessor +PyCascadeClassifier +PyTrueBatchProcessor +PyVideoCapture +PyVideoWriter +ResizeIterator ``` - -## Limitations - -- This is not a complete `cv2` replacement. -- Batch APIs allocate owned output arrays. -- Small inputs can be dominated by call overhead. -- Zero-copy functions require contiguous arrays for crop and resize paths. -- System OpenCV and wheel build configuration can affect performance and available codecs. diff --git a/docs/BUILDING_STATIC_OPENCV.md b/docs/BUILDING_STATIC_OPENCV.md index e1a3a3b..36c177d 100644 --- a/docs/BUILDING_STATIC_OPENCV.md +++ b/docs/BUILDING_STATIC_OPENCV.md @@ -1,198 +1,125 @@ -# Building a statically linked OpenCV bundle - -The `opencv` crate expects to find an existing OpenCV toolkit and, by default, it -links against the dynamic libraries that come with a system installation -(`libopencv_core.dylib`, `libopencv_core.so`, etc.). To ship the `trainingsample` -crate without asking end users to install OpenCV themselves, build a static -OpenCV distribution once and point Cargo at it during compilation. - -## 1. Build FFmpeg as static libraries - -1. Download FFmpeg sources (6.1+ recommended) to a temporary location: - - ```bash - curl -sSLo ~/Downloads/ffmpeg-6.1.1.tar.bz2 https://ffmpeg.org/releases/ffmpeg-6.1.1.tar.bz2 - tar -C ~/Downloads -xjf ~/Downloads/ffmpeg-6.1.1.tar.bz2 - ``` - -2. Configure a static build that keeps only the components required for software - decoding. Disable the executables and hardware accelerators so the build - stays self-contained: - - ```bash - pushd ~/Downloads/ffmpeg-6.1.1 - ./configure \ - --prefix=$(pwd)/../../third_party/ffmpeg-static \ - --pkg-config-flags="--static" \ - --extra-cflags="-fPIC" \ - --extra-cxxflags="-fPIC" \ - --extra-ldflags="-fPIC" \ - --enable-static \ - --disable-shared \ - --disable-doc \ - --disable-debug \ - --disable-autodetect \ - --disable-hwaccels \ - --disable-vulkan \ - --disable-cuvid \ - --disable-nvenc \ - --disable-nvdec \ - --disable-vaapi \ - --disable-vdpau \ - --disable-d3d11va \ - --disable-dxva2 \ - --disable-alsa \ - --disable-sdl2 \ - --disable-libxcb \ - --disable-iconv \ - --disable-libdrm \ - --disable-network \ - --disable-avdevice \ - --disable-postproc \ - --disable-programs \ - --enable-swscale \ - --enable-swresample - make -j$(nproc) - make install - popd - ``` - - The install step writes the static archives and headers to - `third_party/ffmpeg-static`. The OpenCV build picks them up via pkg-config. - Copy the resulting libraries into the OpenCV bundle later (`libavcodec.a`, - `libavfilter.a`, `libavformat.a`, `libavutil.a`, `libswresample.a`, - `libswscale.a`). - -## 2. Build OpenCV as static libraries - -1. Download an official OpenCV source archive (4.9+ recommended) and place it - somewhere outside the repository, e.g. `~/Downloads/opencv-4.10.0`. -2. Configure a *Release* build that turns off shared libraries and only enables - the modules that `trainingsample` uses. Ensure the FFmpeg pkg-config files - are visible (e.g. `export PKG_CONFIG_PATH=$(pwd)/third_party/ffmpeg-static/lib/pkgconfig:$PKG_CONFIG_PATH`). - On macOS or Linux: - - ```bash - cmake -S ~/Downloads/opencv-4.10.0 \ - -B ~/Downloads/opencv-build-static \ - -DBUILD_LIST=core,imgproc,imgcodecs,highgui,video,videoio,calib3d,features2d,photo \ - -DBUILD_SHARED_LIBS=OFF \ - -DBUILD_opencv_world=ON \ - -DOPENCV_FORCE_3RDPARTY_BUILD=ON \ - -DBUILD_JPEG=ON -DWITH_JPEG=ON \ - -DBUILD_PNG=ON -DWITH_PNG=ON \ - -DBUILD_TIFF=ON -DWITH_TIFF=ON \ - -DBUILD_WEBP=ON -DWITH_WEBP=ON \ - -DBUILD_ZLIB=ON -DWITH_ZLIB=ON \ - -DBUILD_JASPER=ON -DWITH_JASPER=ON \ - -DWITH_FFMPEG=ON -DOPENCV_FFMPEG_USE_FIND_LIBS=ON \ - -DBUILD_TESTS=OFF -DBUILD_PERF_TESTS=OFF -DBUILD_EXAMPLES=OFF \ - -DWITH_IPP=OFF -DWITH_OPENCL=OFF -DWITH_CUDA=OFF -DWITH_OPENEXR=OFF - cmake --build ~/Downloads/opencv-build-static --config Release --target opencv_world - cmake --install ~/Downloads/opencv-build-static \ - --config Release \ - --prefix $(pwd)/third_party/opencv-static - ``` - - The `opencv_world` target gives you a single `libopencv_world.a` archive; - switch it off and install the individual module archives if you prefer. - The OpenCV build drops the bundled codec libraries (`liblibjpeg-turbo.a`, - `liblibpng.a`, `liblibtiff.a`, `liblibwebp.a`, `libzlib.a`, `liblibjasper.a`) inside the - build tree under `build/3rdparty/lib/`. Copy them into - `third_party/opencv-static/lib/` and add the expected linker aliases, e.g.: - - ```bash - cp ~/Downloads/opencv-build-static/3rdparty/lib/liblibjpeg-turbo.a third_party/opencv-static/lib/ - ln -sf liblibjpeg-turbo.a third_party/opencv-static/lib/libjpeg.a - # Repeat for liblibpng.a -> libpng.a, liblibtiff.a -> libtiff.a, liblibwebp.a -> libwebp.a, libzlib.a -> libz.a, liblibjasper.a -> libjasper.a - ``` - -3. After installation you should have: - - ```text - third_party/opencv-static/include/opencv4/... - third_party/opencv-static/lib/libopencv_world.a - third_party/opencv-static/lib/libz.a (and other third-party static deps) - third_party/opencv-static/lib/libjasper.a - third_party/opencv-static/lib/libavcodec.a - third_party/opencv-static/lib/libavfilter.a - third_party/opencv-static/lib/libavformat.a - third_party/opencv-static/lib/libavutil.a - third_party/opencv-static/lib/libswresample.a - third_party/opencv-static/lib/libswscale.a - ``` - - If your OpenCV build links to optional third-party components (TBB, JPEG, - PNG, WebP, etc.), install their static archives into the same `lib/` folder - so Cargo can link them in one pass. - -> **C++ runtime note**: many CI images do not install `libstdc++.a`. -> On Linux, prefer `dylib=stdc++` unless you explicitly provide a static C++ -> runtime archive. On macOS, use `dylib=c++` (or `framework=Accelerate` when -> required). - -## 3. Point Cargo at the static toolchain - -Add a `.cargo/config.toml` (kept inside the repo) with the environment variables -that the `opencv` build script understands: - -```toml -[env] -# Paths are resolved relative to the workspace root. -OPENCV_INCLUDE_PATHS = { value = "third_party/opencv-static/include", relative = true } -OPENCV_LINK_PATHS = { value = "third_party/opencv-static/lib", relative = true } -# Tell the build script to stop probing the system. -OPENCV_DISABLE_PROBES = "pkg_config,cmake,vcpkg" -# Static link OpenCV and the extra libraries it depends on. -# Adjust the list to match the archives present in third_party/opencv-static/lib. -OPENCV_LINK_LIBS = "static=opencv_world,static=avformat,static=avcodec,static=avfilter,static=swresample,static=swscale,static=avutil,static=png,static=jpeg,static=tiff,static=webp,static=z,static=jasper,dylib=stdc++" +# Static OpenCV build + +The repository build script creates the static libraries used by release +wheels. + +```bash +./scripts/build-opencv-static.sh ``` -The helper script in this repository disables OpenJPEG support -(`-DWITH_OPENJPEG=OFF`), so `opencv_world` does not depend on `openjp2`. If you -enable JPEG2000 support yourself, append the appropriate OpenJPEG archive -(`static=openjp2`) to `OPENCV_LINK_LIBS`. +## Pinned inputs + +| Component | Version | +|---|---:| +| OpenCV | 4.11.0 | +| FFmpeg | 6.1.1 | -To avoid stale caches, the script drops a `build_signature.txt` file alongside -the libraries containing the configuration fingerprint (OpenCV version and the -set of disabled features). If you rebuild OpenCV manually, update or remove that -file so subsequent runs do not skip regeneration. +The versions are defined at the top of +`scripts/build-opencv-static.sh`. -If you elected to install the individual module archives instead of -`opencv_world`, list each one (`static=opencv_core`, `static=opencv_imgproc`, -and so on). Keep the order roughly from high- to low-level modules so the linker can -resolve symbols in one pass. +## Required commands -For cross-compilation add target-specific sections, e.g.: +```text +bash +curl +tar with bzip2 support +make +cmake +pkg-config +C and C++ compilers +``` -```toml -[target.aarch64-apple-darwin.env] -OPENCV_LINK_LIBS = "static=opencv_world,static=avformat,static=avcodec,static=avfilter,static=swresample,static=swscale,static=avutil,static=png,static=jpeg,static=tiff,static=z,static=jasper,dylib=c++" +CI also installs clang, libclang, LLVM, nasm, and yasm before running the +script. + +## Output + +| Path | Contents | +|---|---| +| `third_party/ffmpeg-static/` | FFmpeg headers, archives, pkg-config files, signature | +| `third_party/opencv-static/include/opencv4/` | OpenCV headers | +| `third_party/opencv-static/lib/libopencv_world.a` | OpenCV archive | +| `third_party/opencv-static/lib/` | codec and FFmpeg archives | +| `third_party/opencv-static/build_signature.txt` | OpenCV build signature | + +Required archives checked by the script: + +```text +libjpeg.a +libpng.a +libtiff.a +libwebp.a +libz.a +libjasper.a +libavcodec.a +libavfilter.a +libavformat.a +libavutil.a +libswresample.a +libswscale.a ``` -## 4. Build the crate +## OpenCV configuration + +| Setting | Value | +|---|---| +| Library form | static `opencv_world` | +| Modules | core, imgproc, imgcodecs, highgui, video, videoio, calib3d, features2d, photo | +| Bundled codecs | JPEG, PNG, TIFF, WebP, zlib, Jasper | +| FFmpeg | enabled | +| Tests, examples, apps, docs | disabled | +| IPP, OpenCL, CUDA, OpenJPEG, OpenEXR, ITT, TBB | disabled | +| GStreamer, V4L, GTK, Qt | disabled | +| Carotene | disabled on macOS and ARM hosts | + +## FFmpeg configuration + +| Setting | Value | +|---|---| +| Library form | static | +| Decoders | H.264, HEVC, MPEG-4, VP8, VP9 | +| Demuxers | MOV, Matroska, Ogg, WebM, image2 | +| Protocols | file, data, pipe | +| Hardware acceleration | disabled | +| Network | disabled | +| Programs and documentation | disabled | -With the static bundle in place you can now build the crate without touching the -system OpenCV installation: +## Cargo and maturin environment + +Linux: + +```bash +export OPENCV_INCLUDE_PATHS="$PWD/third_party/opencv-static/include/opencv4" +export OPENCV_LINK_PATHS="$PWD/third_party/opencv-static/lib" +export OPENCV_DISABLE_PROBES="pkg_config,cmake,vcpkg,vcpkg_cmake" +export OPENCV_LINK_LIBS="static=opencv_world,static=avformat,static=avcodec,static=avfilter,static=swresample,static=swscale,static=avutil,static=jpeg,static=png,static=tiff,static=webp,static=z,static=jasper,dylib=stdc++" +``` + +macOS: ```bash -cargo build --features opencv --release +export OPENCV_INCLUDE_PATHS="$PWD/third_party/opencv-static/include/opencv4" +export OPENCV_LINK_PATHS="$PWD/third_party/opencv-static/lib" +export OPENCV_DISABLE_PROBES="pkg_config,cmake,vcpkg,vcpkg_cmake" +export OPENCV_LINK_LIBS="static=opencv_world,static=avformat,static=avcodec,static=avfilter,static=swresample,static=swscale,static=avutil,static=jpeg,static=png,static=tiff,static=webp,static=jasper,framework=Accelerate,dylib=c++,framework=OpenCL,z" ``` -The resulting `libtrainingsample.{so,dylib}` (or the wheels produced by the -Python bindings) now embed the OpenCV symbols directly, so end users do not need -`opencv_core` on their machines. +Build commands: + +```bash +cargo build --release --no-default-features --features opencv,simd +maturin build --release --no-default-features \ + --features python-bindings,simd,opencv +``` -## 5. Regenerating the bundle +macOS release wheels add the `metal` feature. -Whenever you need to update OpenCV: +## Cache behavior -1. Re-run the CMake configure/build/install steps pointing at the new source - tree. -2. Verify that the list in `OPENCV_LINK_LIBS` still matches the archives produced. -3. Commit the regenerated contents of `third_party/opencv-static/` if you keep - it under version control (or upload it to your release pipeline's artifact - store). +The script skips a bundle when `libopencv_world.a` exists and +`build_signature.txt` matches the configured signature. A signature mismatch +removes and rebuilds the existing install directory. Temporary OpenCV and +FFmpeg build directories are removed after a successful build. -That is all Cargo needs. No changes to `Cargo.toml` are required beyond enabling -the `opencv` feature when you want the acceleration path. +Release workflow settings are in `.github/workflows/publish.yml`. Test-wheel +settings are in `.github/workflows/ci.yml`.