[SamGu-NRX]

What changed

This adds experiments/autodetect/, which tests whether the phone can propose the things on a wall (windows, doors and the rest) so the homeowner confirms them instead of drawing boxes and tapping corners. It also tests reading the meter's brand from the close-up.

• Wall objects: nothing passes. Three candidates were scored on 300 Open Images photos and 378 CMP Facade photos, with thresholds chosen on a separate 150-photo set. The pass bar, committed before any scoring, was 80% recall at 60% precision. OWLv2 zero-shot found 23% of windows at 55% precision. A Create ML detector trained on 1,340 Open Images photos (6.8 MB, 50 ms a photo on the Mac) found 7%, and Vision's rectangle detector reached 6.5% precision. Tables: results/proposals.md.
• Lifting a box onto the wall works; the boxes don't. On the ETH3D electro packet, with door edges measured in the laser scan, a true door box lifted from its inner corners is 0.11 ft off at p90. From the midpoints of its sides it is 0.59 ft off. Detected boxes land 1.2 ft off or worse at p90: results/extent.md.
• Meter brand: ready to port. Matching experiments: meter close-up reading, tap-to-confirm and two retake checks #16's Vision lines against a list of 36 meter makers names the brand correctly on 17 of 19 photos where it is printed in plain letters, and on 22 of 42 where it is part of a logo. It named no maker on the 9 photos without a brand: meter_brand/README.md.
• PLAN.md holds the recommendation, the confirm-or-reject flow for the experience design thread, and the HouseScanKit and app files an integration touches. No app code changes here.

Why

The public photos show whole buildings from far away, while the app sees one wall from 1 to 3 m. When only large objects are counted, the student's window AP50 rises from 9% to 38%. So the next useful run trains and scores on the app's own keyframes rather than on a larger model. The plan leaves out the D-FINE fine-tune (about 10 hours on the shared Mac) and the full Grounding DINO pass (about 5 hours) because both would use the same off-view data.

Verification

• uv run pytest in experiments/autodetect: 16 passed, covering the Open Images scoring rules, threshold choice, CMP parsing and a D-FINE patch. The lifting code has no unit test; its check is the true-box control in results/extent.md.
• Each results file names the command that produced it. The brand scorer reproduces experiments: meter close-up reading, tap-to-confirm and two retake checks #16's figure of 64 of 71 numbers in the top three from experiments: meter close-up reading, tap-to-confirm and two retake checks #16's committed per-photo results.
• Timings come from an M4 Pro Mac that other agents were loading at the same time (load averages from 18 to 492), not from an iPhone.
• Two AI readers labelled the brands: a Claude Sonnet reader, and the maker names in experiments: meter close-up reading, tap-to-confirm and two retake checks #16's notes. The first pass shifted rows m18 to m26 by four photos, and the reader redid them before the final score. After the first score, three matching rules changed; meter_brand/README.md lists them.
• The 3D check covers four doors at 3 to 13 m. No window extents were measured, and no captures from our phones were used.
• AC units, gas meters, electrical boxes, garage doors and batteries have no public boxes, so none of them was scored.

Risk

Photos, weights and predictions stay in ~/house-scanning-data/. The repository holds manifests (id, URL, license, author), code and tables. The brand tables hold maker names and OCR words, and a scan found no digit runs of five or more.

Checklist

• The scope is focused; related changes belong together
• I explained what changed and why
• N/A before/after images: nothing in the app changes
• N/A video: no motion or interaction changes

🤖 Generated with Claude Code

[coderabbitai]

Warning

Review limit reached

Next included review available in 59 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 496fca05-afd6-426b-be3b-5f0e2b65d980

📥 Commits

Reviewing files that changed from the base of the PR and between f852207 and 538f60e.

⛔ Files ignored due to path filters (6)

• experiments/autodetect/manifests/cmp_base.csv is excluded by !**/*.csv
• experiments/autodetect/manifests/oi_eval.csv is excluded by !**/*.csv
• experiments/autodetect/manifests/oi_train.csv is excluded by !**/*.csv
• experiments/autodetect/manifests/oi_tune.csv is excluded by !**/*.csv
• experiments/autodetect/meter_brand/results/brand_per_image.csv is excluded by !**/*.csv
• experiments/autodetect/uv.lock is excluded by !**/*.lock

📒 Files selected for processing (37)

• experiments/autodetect/.gitignore
• experiments/autodetect/.python-version
• experiments/autodetect/PLAN.md
• experiments/autodetect/README.md
• experiments/autodetect/autodetect/__init__.py
• experiments/autodetect/autodetect/cmp.py
• experiments/autodetect/autodetect/config.py
• experiments/autodetect/autodetect/dfine.py
• experiments/autodetect/autodetect/electro.py
• experiments/autodetect/autodetect/extent.py
• experiments/autodetect/autodetect/extent_gt.py
• experiments/autodetect/autodetect/gdino.py
• experiments/autodetect/autodetect/nms.py
• experiments/autodetect/autodetect/oieval.py
• experiments/autodetect/autodetect/openimages.py
• experiments/autodetect/autodetect/owl.py
• experiments/autodetect/autodetect/paths.py
• experiments/autodetect/autodetect/run_vision.py
• experiments/autodetect/autodetect/score.py
• experiments/autodetect/autodetect/sets.py
• experiments/autodetect/autodetect/show.py
• experiments/autodetect/autodetect/student.py
• experiments/autodetect/meter_brand/README.md
• experiments/autodetect/meter_brand/brands.py
• experiments/autodetect/meter_brand/results/brand.md
• experiments/autodetect/meter_brand/score.py
• experiments/autodetect/pyproject.toml
• experiments/autodetect/results/extent.md
• experiments/autodetect/results/proposals.md
• experiments/autodetect/results/resources.md
• experiments/autodetect/tests/test_cmp.py
• experiments/autodetect/tests/test_dfine.py
• experiments/autodetect/tests/test_oieval.py
• experiments/autodetect/vision/Package.swift
• experiments/autodetect/vision/Sources/coremldet/main.swift
• experiments/autodetect/vision/Sources/rects/main.swift
• experiments/autodetect/vision/Sources/trainod/main.swift

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[SamGu-NRX]

@greptileai

[greptile-apps]

Confidence Score: 3/5

[Low risk] Adds a new experiment directory for vision model research.

The PR is not yet safe to merge because the previously reported electro rerun and meter-brand scoring failures remain unresolved.

Findings

1.  Electro images are never prepared. ▶
2.  Unrelated text counts as GE. ▶
3.  Elevation output directory is missing ▶
4.  D-FINE log directory is missing ▶
5.  Aliases match inside unrelated words. ▶
6.  CMP inputs lack setup instructions. ▶

Fix with agent prompt

### Issue 1
experiments/autodetect/autodetect/sets.py:17-18
The documented rerun requests predictions for `electro`, but this path requires JPEGs in `electro_1024` and no step creates them. Even with the ETH3D packet installed, the prediction commands cannot regenerate the 3D extent results.

### Issue 2
experiments/autodetect/meter_brand/score.py:undefined-73
For a GE-labeled meter, `names_brand` treats an OCR line reading “VOLTAGE” as naming GE because it checks for the brand anywhere in the word. This can inflate the reported “brand in any line” result and incorrectly mark a reader’s note as agreeing with the label.

### Issue 3
experiments/autodetect/autodetect/extent_gt.py:139-140
**Elevation output directory is missing.** On a fresh setup, the documented `extent_gt render` command tries to save an image under `DATA/preview`, but nothing creates that directory. Rendering fails instead of producing the images used to inspect the door ground truth.

```suggestion
    out = DATA / "preview" / f"elev_{door}.png"
    out.parent.mkdir(parents=True, exist_ok=True)
    im.save(out)
```

### Issue 4
experiments/autodetect/autodetect/dfine.py:120
**D-FINE log directory is missing.** On a fresh setup, the standalone `train` or `probe` command opens its log in `DATA/student` before that directory exists. It fails with `FileNotFoundError` before training starts unless the separate Create ML preparation step has already created the directory.

```suggestion
    (DATA / "student").mkdir(parents=True, exist_ok=True)
    log = open(DATA / "student" / "dfine_small_train.log", "a")
```

### Issue 5
experiments/autodetect/meter_brand/brands.py:undefined-115
The substring check classifies “GENERAL ELECTRICITY” as GE because it contains the `GENERAL ELECTRIC` alias. Unrelated OCR text can therefore be selected as the meter’s maker, making the proposed brand less reliable.

### Issue 6
experiments/autodetect/README.md:undefined-20
The rerun invokes the CMP builder without first obtaining and extracting the archive to the directory it reads. On a fresh setup, the builder processes no XML files, so the documented steps cannot reproduce the reported 378-image evaluation. Document or automate that prerequisite.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

The PR adds a standalone experiment for wall-object detection, 3D box lifting, and meter-brand reading, without changing app code.

• It records detector evaluations and a recommendation to collect app keyframes before shipping wall-object proposals.
• It supplies a meter-brand scorer, results, and an integration plan.
• Two optional experiment commands need their output directories created before they can run from a fresh setup.

_{Reviews (2) · Last reviewed commit: "experiments/autodetect: meter brand resu..."}

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Repo admins can enable using credits for code reviews in their settings.

[codemetrics-ai]

Note

Review skipped

Codemetrics is installed on this repository, but it hasn't been added to your team yet.

To enable AI code reviews for this repo, go to your dashboard and add house-scanning-master to your team's repositories.