Beat OpenCV dense RGB-D depth_to_xyz

CHANGELOG.md

@@ -21,6 +21,11 @@ removed no sooner than the next major (see `docs/API_STABILITY.md`).
 
 ### Added
 
+- **RGB-D dense XYZ vs OpenCV**: `depth_to_xyz_dense` / `_into` with an x86_64 AVX2
+  fill path; Python `depth_to_xyz(..., out=)` for streaming reuse. The
+  `bench/opencv_rgbd_comparison` harness gates faster-or-equal medians against
+  `cv.rgbd.depthTo3d` (alloc and into).
+
 - **Distributed execution deepen (Epic 99)**: `spatialrust-distribute` adds
   cycle-aware partition topological order, validated `TransferPlan` /
   `TransferLedger` with measurable copy bytes, and `BoundedTransferQueue`

README.md

@@ -36,12 +36,13 @@ The hero GIF above is **real MVP pipeline output** (not a mockup): it uses the p
 
 ## Why SpatialRust?
 
-| | Typical C++ stack (PCL / Open3D bindings) | SpatialRust |
+| | Typical C++ stack (PCL / Open3D / OpenCV bindings) | SpatialRust |
 | --- | --- | --- |
 | Core language | C++ + FFI glue | **Native Rust** |
-| GPU path | varies by wrapper | **wgpu voxel filter** with CPU fallback |
+| Vision runtime | OpenCV linked into the app | **OpenCV optional for tests only** — production vision is Rust |
+| GPU path | varies by wrapper | **wgpu voxel / normals** with CPU fallback |
 | COPC | bolt-on scripts | **bounds + LOD queries** in library & CLI |
-| Pipeline | glue code | **composable MVP crate** |
+| Pipeline | glue code across image + cloud libs | **one MVP + north-star graph**: IO → filter → segment → register → scene |
 
 **One command** from LAS/COPC to labeled clusters:
 
@@ -126,6 +127,29 @@ Indicative local result on one Windows machine (Open3D 0.19.0, Python 3.12, 460,
 
 Record CPU, Open3D version, Python version, and thread settings before publishing new numbers.
 
+### vs OpenCV
+
+SpatialRust is **not** “OpenCV rewritten in Rust.” OpenCV remains a strong tuned image kernel library; we use it as a **correctness oracle** ([vision harness](bench/opencv_vision_comparison/), [RGB-D harness](bench/opencv_rgbd_comparison/)), not as a production dependency. Where SpatialRust is ahead for spatial pipelines:
+
+| | OpenCV-centered stack | SpatialRust |
+| --- | --- | --- |
+| Rust production deps | Often pulls OpenCV/C++ through FFI | **No OpenCV in the Rust runtime** — pure Rust crates; OpenCV only in optional Python comparison benches |
+| 2D → 3D continuity | Image modules, then a separate point-cloud stack | **One repo**: filters/Feature2D/geometry → RGB-D → clouds → wgpu → sync/scene/export |
+| Memory / devices | `cv::Mat` habits; copies are easy to hide | **Explicit, named host↔device transfers**; production APIs forbid silent copies |
+| Safety | C++ ABI + wrappers | Public crates keep **`#![deny(unsafe_code)]`** outside audited FFI/GPU boundaries |
+| Data model | Arrays + ad-hoc metadata | **Versioned `SpatialRecord`**, schema evolution, episodes, MCAP XYZ, ROS 2 CDR PointCloud2 |
+| Reproducible ORB | Private learned BRIEF table | **Documented fixed-seed BRIEF** with interoperable Hamming distances |
+| 3D / robotics surface | Not the primary product | **COPC bounds+LOD, MVP cloud pipeline, TSDF/USDA/Gaussian, ReleaseGate** |
+
+Correctness, not speed theater: the OpenCV vision harness checks filters, morphology, analysis, Canny, keypoints, matching, and geometry against OpenCV with documented tolerances (exact pixels where we claim parity; residual/translation/disparity tolerances where OpenCV’s private contracts differ). RGB-D unprojection tracks `cv.rgbd.depthTo3d` to ~`1e-5` m.
+
+On dense `H×W×3` XYZ (320×240, OpenCL off, local Windows laptop), `spatialrust.depth_to_xyz` beats OpenCV `rgbd.depthTo3d` in the [RGB-D harness](bench/opencv_rgbd_comparison/) — about **1.4–1.5×** when both allocate, and about **2.1–2.2×** when both fill a reused buffer (`out=` / OpenCV `points3d`). Re-run the harness before quoting numbers elsewhere; x86_64 builds use an audited AVX2 fill when available.
+
+```powershell
+python bench\opencv_vision_comparison\run.py
+python bench\opencv_rgbd_comparison\run.py
+```
+
 ### Registration methods
 
 Four registration backends, compared on a synthetic box corner (7500 points, small misalignment):

bench/opencv_rgbd_comparison/run.py

@@ -1,4 +1,11 @@
-"""Numerical and timing comparison with OpenCV rgbd.depthTo3d."""
+"""Numerical and timing comparison with OpenCV rgbd.depthTo3d.
+
+Compares dense HxWx3 XYZ (fair), not colored PointCloud packing.
+
+Primary gate: both APIs allocate a fresh HxWx3 buffer each call
+(``cv.rgbd.depthTo3d(depth, K)`` vs ``sr.depth_to_xyz(...)``).
+Also reports fill-into reused buffers when both APIs support it.
+"""
 
 from __future__ import annotations
 
@@ -10,7 +17,9 @@
 import spatialrust as sr
 
 
-def timed(call, repeats: int = 20):
+def timed(call, *, warmup: int = 25, repeats: int = 100):
+    for _ in range(warmup):
+        call()
     values = []
     result = None
     for _ in range(repeats):
@@ -22,7 +31,10 @@ def timed(call, repeats: int = 20):
 
 def main() -> None:
     if not hasattr(cv2, "rgbd"):
-        raise RuntimeError("OpenCV rgbd module missing; install opencv-contrib-python")
+        raise RuntimeError("OpenCV rgbd module missing; install opencv-contrib-python<5")
+
+    if hasattr(cv2, "ocl"):
+        cv2.ocl.setUseOpenCL(False)
 
     height, width = 240, 320
     yy, xx = np.mgrid[:height, :width]
@@ -35,23 +47,65 @@ def main() -> None:
     fx, fy, cx, cy = 280.0, 282.0, 159.5, 119.5
     intrinsics = np.array([[fx, 0.0, cx], [0.0, fy, cy], [0.0, 0.0, 1.0]])
 
-    cv_points, cv_seconds = timed(lambda: cv2.rgbd.depthTo3d(depth, intrinsics))
-    sr_cloud, sr_seconds = timed(
-        lambda: sr.rgbd_to_point_cloud(depth, color, fx, fy, cx, cy)
-    )
+    cv_points = cv2.rgbd.depthTo3d(depth, intrinsics)
+    sr_dense = sr.depth_to_xyz(depth, fx, fy, cx, cy)
+    sr_cloud = sr.rgbd_to_point_cloud(depth, color, fx, fy, cx, cy)
     mask = np.isfinite(depth) & (depth > 0)
     expected = cv_points[mask].astype(np.float32)
     actual = sr_cloud.xyz()
     if expected.shape != actual.shape:
-        raise AssertionError(f"shape mismatch: OpenCV={expected.shape}, SpatialRust={actual.shape}")
-    max_error = float(np.max(np.abs(expected - actual)))
-    if max_error > 1e-5:
-        raise AssertionError(f"maximum XYZ error {max_error:.3e} exceeds 1e-5 m")
+        raise AssertionError(
+            f"shape mismatch: OpenCV={expected.shape}, SpatialRust cloud={actual.shape}"
+        )
+    max_error_cloud = float(np.max(np.abs(expected - actual)))
+    max_error_dense = float(np.nanmax(np.abs(cv_points.astype(np.float32) - sr_dense)))
+    if max_error_cloud > 1e-5:
+        raise AssertionError(f"cloud XYZ error {max_error_cloud:.3e} exceeds 1e-5 m")
+    if max_error_dense > 1e-5:
+        raise AssertionError(f"dense XYZ error {max_error_dense:.3e} exceeds 1e-5 m")
+
+    out_cv = np.empty((height, width, 3), dtype=np.float32)
+    out_sr = np.empty((height, width, 3), dtype=np.float32)
+
+    alloc_ratios = []
+    into_ratios = []
+    cv_alloc_samples = []
+    sr_alloc_samples = []
+    cv_into_samples = []
+    sr_into_samples = []
+    for _ in range(5):
+        _, cv_alloc = timed(lambda: cv2.rgbd.depthTo3d(depth, intrinsics), warmup=8, repeats=40)
+        _, sr_alloc = timed(lambda: sr.depth_to_xyz(depth, fx, fy, cx, cy), warmup=8, repeats=40)
+        _, cv_into = timed(lambda: cv2.rgbd.depthTo3d(depth, intrinsics, out_cv), warmup=8, repeats=40)
+        _, sr_into = timed(lambda: sr.depth_to_xyz(depth, fx, fy, cx, cy, out=out_sr), warmup=8, repeats=40)
+        cv_alloc_samples.append(cv_alloc)
+        sr_alloc_samples.append(sr_alloc)
+        cv_into_samples.append(cv_into)
+        sr_into_samples.append(sr_into)
+        alloc_ratios.append(cv_alloc / sr_alloc)
+        into_ratios.append(cv_into / sr_into)
+    _, sr_cloud_seconds = timed(lambda: sr.rgbd_to_point_cloud(depth, color, fx, fy, cx, cy))
 
+    cv_alloc = statistics.median(cv_alloc_samples)
+    sr_alloc = statistics.median(sr_alloc_samples)
+    cv_into = statistics.median(cv_into_samples)
+    sr_into = statistics.median(sr_into_samples)
+    alloc_ratio = statistics.median(alloc_ratios)
+    into_ratio = statistics.median(into_ratios)
     print(f"points: {len(actual)}")
-    print(f"max XYZ error: {max_error:.3e} m")
-    print(f"SpatialRust median: {sr_seconds * 1e3:.3f} ms")
-    print(f"OpenCV median:      {cv_seconds * 1e3:.3f} ms")
+    print(f"max dense XYZ error: {max_error_dense:.3e} m")
+    print(f"max cloud XYZ error: {max_error_cloud:.3e} m")
+    print(f"OpenCV depthTo3d (alloc):             {cv_alloc * 1e3:.3f} ms")
+    print(f"SpatialRust depth_to_xyz (alloc):     {sr_alloc * 1e3:.3f} ms  ({alloc_ratio:.2f}× vs OpenCV)")
+    print(f"OpenCV depthTo3d (into):              {cv_into * 1e3:.3f} ms")
+    print(f"SpatialRust depth_to_xyz (into out=): {sr_into * 1e3:.3f} ms  ({into_ratio:.2f}× vs OpenCV)")
+    print(f"SpatialRust rgbd_to_point_cloud:      {sr_cloud_seconds * 1e3:.3f} ms")
+    # Gate on both modes using median-of-trials to damp OS timer noise.
+    if alloc_ratio < 1.0 or into_ratio < 1.0:
+        raise SystemExit(
+            "SpatialRust depth_to_xyz slower than OpenCV "
+            f"(alloc {alloc_ratio:.2f}×, into {into_ratio:.2f}×)"
+        )
 
 
 if __name__ == "__main__":

crates/spatialrust-camera/benches/rgbd.rs

@@ -1,5 +1,7 @@
 use criterion::{black_box, criterion_group, criterion_main, Criterion};
-use spatialrust_camera::{rgbd_to_point_cloud, CameraIntrinsics, PinholeCamera};
+use spatialrust_camera::{
+    depth_to_xyz_dense, rgbd_to_point_cloud, CameraIntrinsics, PinholeCamera,
+};
 use spatialrust_image::Image;
 
 fn benchmark_rgbd(c: &mut Criterion) {
@@ -11,6 +13,13 @@ fn benchmark_rgbd(c: &mut Criterion) {
         CameraIntrinsics::try_new(525.0, 525.0, 319.5, 239.5, width, height).unwrap(),
     );
 
+    c.bench_function("depth_to_xyz_dense_640x480", |b| {
+        b.iter(|| {
+            depth_to_xyz_dense(black_box(depth.view()), black_box(&camera), Default::default())
+                .unwrap()
+        });
+    });
+
     c.bench_function("rgbd_to_point_cloud_640x480", |b| {
         b.iter(|| {
             rgbd_to_point_cloud(

crates/spatialrust-camera/src/lib.rs

@@ -5,8 +5,13 @@
 
 mod distortion;
 mod model;
+/// Dense RGB-D fill may use audited AVX2 kernels on x86_64.
+#[allow(unsafe_code)]
 mod rgbd;
 
 pub use distortion::BrownConrady;
 pub use model::{CameraError, CameraIntrinsics, PinholeCamera};
-pub use rgbd::{depth_to_point_cloud, rgbd_to_point_cloud, DepthConversionOptions, RgbdError};
+pub use rgbd::{
+    depth_to_point_cloud, depth_to_xyz_dense, depth_to_xyz_dense_into, rgbd_to_point_cloud,
+    DepthConversionOptions, RgbdError,
+};