From 6e0722a9f3c735b69b226bbb84f5784312f8545e Mon Sep 17 00:00:00 2001 From: SamGu-NRX Date: Sat, 26 Sep 2026 09:02:55 -0500 Subject: [PATCH 1/2] docs: add the 2026-09-26 morning briefing One self-contained HTML page for the team: what the overnight real-data tests found (phone tracking error, learned depth, meter reading), where the app and server really are, the main blocker (no real-iPhone session yet), and eight decisions with recommendations. Also served at https://house-scanning-briefing.vercel.app. No Base-provided content, dataset images or real-home data. Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/briefings/2026-09-26-morning.html | 754 +++++++++++++++++++++++++ 1 file changed, 754 insertions(+) create mode 100644 docs/briefings/2026-09-26-morning.html diff --git a/docs/briefings/2026-09-26-morning.html b/docs/briefings/2026-09-26-morning.html new file mode 100644 index 000000000..29cc69316 --- /dev/null +++ b/docs/briefings/2026-09-26-morning.html @@ -0,0 +1,754 @@ + + + + + + +8 AM briefing: measure with AR taps, and test the phone against a tape first + + + + +
+
+

8 AM briefing: measure with AR taps, and test the phone against a tape first

+

Last night's tests on real data say a current iPhone's own tracking fits the error allowance the server already uses, and that anything a check depends on should be an AR tap, not a depth map. That tracking data came from a LiDAR iPhone, so measuring a current non-LiDAR iPhone with a tape is the first job this morning.

+ +
+
+

What the data said

+

A 2023 iPhone's own tracking stayed inside the server's error allowance: 18.5 in p90 after 30 ft, against 57.6 in allowed. The 2018 phone was 2 to 3 times over. Learned depth gets flat walls to about 5 in p90 at a realistic phone error, and only with perfect feature matching. Edges: 8 in or worse in every test.

+ The three findings +
+
+

What exists now

+

Server solver, web review page, iOS app, verification and evals, all in open PRs. Nothing is merged. The server is live, with the defects a 05:40 review found in it fixed and redeployed. On a real replay the app talked to the server end to end: a 77-photo scene, answered in 545 ms.

+ PR map and how to try it +
+
+

What we decide

+

Take a current iPhone and a tape to a real meter wall first thing, about an hour. Then settle how points get measured (AR taps), how far to walk, loading Base's real rules, who checks what is hidden behind things, what the scan must see versus ask, merge order and lane owners.

+ Go to the decisions +
+
+ + +
+
+ +
+
+
+ Where we really are +

Works end to end in the Simulator; never run on a real iPhone

+
+

Works

The iOS app runs the whole scan (meter, close-up, guided walk with coverage, one more view, upload, result in 3D and AR) in the Simulator on a recorded real walk, and gets a real answer from the placement server, which is live and tested.

+

Not proven

Nothing has run on a physical iPhone at a real wall. Until this morning's fixes land, no scan could finish on the first try (the app under-reported what it saw), and with placeholder rules every answer is "an installer will take a look".

+

The main blocker

A real-phone session: the app and Measure Lab on a current iPhone, at a real meter wall, with a tape. Hunter's research handoff (#18) reaches the same conclusion. Everything else is waiting on that data.

+

Side work to park

The web review page (#15) shows the server's answer in a browser. It helped check the server before the app could, but it isn't the product. Don't spend more on it now.

+
+
+
+ + +
+
+ The product +

Where last night's findings land

+

A homeowner walks the outside wall around their electric meter with an iPhone. The findings below all hit the second step, where the phone measures.

+
    +
  1. + +

    1. Walk the wall

    +

    The homeowner walks along the wall on both sides of the meter.

    +
    2. +
  2. + +

    2. Phone measures and photographs

    +

    ARKit tracks where the phone is. The app takes photos of the wall and a close-up of the meter.

    +
    Finding 1: driftFinding 2: depthFinding 3: meter read
    +
    3. +
  3. + +

    3. Server applies the rules in plain code

    +

    Placement rules live in rules.yaml with citations. No model makes the call.

    +
    PR #11
    +
    4. +
  4. + +

    4. Spot shown in AR

    +

    The app shows the battery spot on the real wall, placed relative to the meter.

    +
    PR #10
    +
    5. +
+

Sam's priorities, in order: (1) the scan is easy and works first try, (2) the 3D is accurate, (3) the model appears live in the app.

+
+
+ + +
+
+ What the data said +

Three findings from real data

+ + +
+

Finding 1. A current iPhone's tracking stays inside the server's allowance

+

Position error is the distance between where the phone thinks it is and where it really is, sideways drift included. We measured it after walks of 3 to 30 ft on two public datasets, with the same code for both (make drift and make modern-arkit in PR #12).

+ +
+ + + p90 position error against feet walked, with the server's allowance + p90 position error in inches at 3, 10, 20 and 30 ft walked. 2018 iPhone 6s: 18.6, 55.7, 94.6, 132.6. 2023 iPhone 14 Pro Max, trusted walks: 4.0, 8.6, 13.4, 18.5. Server's allowance: 5.8, 19.2, 38.4, 57.6. The current iPhone stays under the allowance at every distance; the 2018 phone is 2 to 3 times over it. The vertical axis is broken above 60 in; the band from 80 to 140 in uses a smaller scale. + + + + + + + 020 in40 in60 in + 100 in140 in + + + 3 ft10 ft20 ft30 ft + Distance walked + + Position error, p90 + + + + inside the allowance + + + + + + + + + + + + axis break: the band above uses a smaller scale + + 2018 phone 132.6 in + allowance 57.6 in + 2023 phone 18.5 in + + 3 ft walked: 2018 phone 18.6 in, 2023 phone 4.0 in, allowance 5.8 in + 10 ft walked: 2018 phone 55.7 in, 2023 phone 8.6 in, allowance 19.2 in + 20 ft walked: 2018 phone 94.6 in, 2023 phone 13.4 in, allowance 38.4 in + 30 ft walked: 2018 phone 132.6 in, 2023 phone 18.5 in, allowance 57.6 in + + +
p90 position error in inches: 9 in 10 cases are better. The allowance is the server's 0.16 ft per ft plus its source default. Counting three doubtful MARViN walks, the current iPhone's p90 is 6.9 / 18.7 / 37.5 / 50.0 in, just over the allowance at 3 ft only. The 2018 phone is 2 to 3 times over it, though that dataset's own reference is noisy and inflates the number, and one of its four walks lost tracking entirely. Hover a column for its values.
+
Data table +
+ + + + + + + +
Walked2018 iPhone 6s (in)2023 iPhone, trusted walks (in)2023 iPhone, incl. 3 doubtful walks (in)Server's allowance (in)
3 ft18.64.06.95.8
10 ft55.78.618.719.2
20 ft94.613.437.538.4
30 ft132.618.550.057.6
+
+
+ +
+
+ + A tape measure printed too small + Two rulers. The top one is the real distance walked, marked at 3, 10, 20 and 30 feet. The bottom one is what a phone with a scale error counts: every mark lands a little short, and the gap between the rulers grows the farther you walk. + + + + + How far you really walked + + + + 3 ft10 ft20 ft30 ft + + + + + + + the gap + What the phone counts + +
+
+

Scale error works like a tape printed too small

+

Every reading comes up short, and the longer the reading, the more inches you lose. On the 2018 phone, ARKit read distances 5% to 17% short against GPS. On the current iPhone, its scale matched the dataset's reference within 2% on 30 of 35 outdoor walks.

+

Where the picture breaks. A misprinted tape is off by the same amount every time, while the 2018 phone's shortfall changed from walk to walk. And a tape only measures along the walk; position error also counts sideways drift.

+
+
+ +

Two caveats on the current iPhone. MARViN doesn't say how its reference got its meters, so it may have been scaled to ARKit itself. And that phone has LiDAR, which the phones we target mostly lack.

+

What it means. The server widens every AR and photo measurement's error bar by 0.16 ft for each foot away from the meter, so a far-away measurement comes back "not sure" instead of passing wrongly. That allowance is right for current phones. A tape test on a current, non-LiDAR iPhone is still the only direct measurement.

+
+ + +
+

Finding 2. Learned 3D from photos gets flat walls to about 5 in at best, never edges

+

We tested on ETH3D: real building scans with laser ground truth, points within 6 m of the camera, 1 to 3 m spans. Each model turns a photo into depth. On its own, each one gets the size of the scene wrong.

+ +
+ + Scale error range of each model on its own + MoGe-2 reads distances 4 to 12 percent long. Depth Anything 3 reads them 7 to 11 percent short. + + + reads short + reads long + MoGe-2Depth Anything 3 + + MoGe-2: 4% to 12% long + Depth Anything 3: 7% to 11% short + + + +4%+12% + −11%−7% + + −20%−10%0+10% + Scale error, each model alone + +
Each bar spans the range of scale errors we saw for that model. That leaves a median error of 6 to 8 in, and a p90 of about 20 in.
+
Data table +
+ + +
Model, aloneScale error
MoGe-24% to 12% long
Depth Anything 37% to 11% short
+
+
+ +
+
+

One taped distance fixes the typical case, not the bad ones

+

One photo plus one taped distance:

+
+
1.6 inmedian
+
9.6 inp90 on surfaces
+
15.1 inp90 on walls
+
26.9 inp90 at edges
+
+
+
+

The best recipe: let the phone's poses set the scale

+

A pose is where the phone stood and which way it pointed for a photo. ARKit records one for every photo. We take MoGe-2's depth and rescale it with the phone's own poses over 8 photos.

+

The chart below changes one thing between rows: how wrong the phone is about scale. Each whisker is a 95% interval, from resampling camera positions and 5 draws of pose noise.

+
+
+ +
+ + + p90 error under three pose settings, with 95% intervals + MoGe-2 depth rescaled with the phone's poses over 8 photos, points within 6 m, spans 1 to 3 m, p90 error in inches with 95% intervals. Exact poses: overall 4.0 (3.3 to 4.8), walls 2.8 (2.0 to 4.4). A current phone with 2 percent scale error: overall 5.6 (4.6 to 6.7), walls 5.0 (3.8 to 6.8). The 2018 phone's error: overall 14.6 (10.6 to 18.7), walls 10.8 (8.6 to 13.1). Edges are 8 in or worse in every setting. + + + + + Edges land here: + 8 in or worse, every setting + + + Exact posesA current phoneThe 2018 phone's error + no scale error2% scale errorADVIO iPhone 6s + + Exact poses, overall: 4.0 in (95% interval 3.3 to 4.8) + Exact poses, walls: 2.8 in (95% interval 2.0 to 4.4) + A current phone, overall: 5.6 in (95% interval 4.6 to 6.7) + A current phone, walls: 5.0 in (95% interval 3.8 to 6.8) + The 2018 phone's error, overall: 14.6 in (95% interval 10.6 to 18.7) + The 2018 phone's error, walls: 10.8 in (95% interval 8.6 to 13.1) + + + + 4.0 in2.8 in + 5.6 in5.0 in + 14.6 in10.8 in + + 04 in8 in12 in16 in20 in + p90 error, spans 1-3 m, points within 6 m (lower is better) + +
Only the phone's scale error changes between rows, and the error follows it. With a current phone's 2% error, walls land at 5.0 in p90 (95% interval 3.8 to 6.8). All three rows assume perfect feature matching between photos; without it, walls reach 6.3 in even with exact poses. The earlier "walls at 4 in" figure came from a single noise draw, the lucky end of the spread.
+
Data table +
+ + + + + + +
PosesOverall, p90 (in) [95% interval]Walls, p90 (in) [95% interval]
Exact poses4.0 [3.3, 4.8]2.8 [2.0, 4.4]
A current phone (2% scale error)5.6 [4.6, 6.7]5.0 [3.8, 6.8]
The 2018 phone's error14.6 [10.6, 18.7]10.8 [8.6, 13.1]
+
+
+ +
+

Verdict. At a realistic phone error, learned depth gets flat walls to about 5 in p90, and only with perfect feature matching between photos. Edges (window and door frames, equipment) stay at 8 in or worse in every setting.

+

For the app. Measurements that decide a check should come from AR taps, which carry the phone's own tracking error from Finding 1, not from depth maps. Learned depth is useful for coverage and the 3D picture, not for clearances near a threshold.

+
+
+ + +
+

Finding 3. Meter close-up: reading works, finding the number doesn't yet

+

The question: how often does Apple's on-device text recognition read the meter number and class from real, openly licensed meter photos (Wikimedia Commons)? And which cheap photo checks (blur, glare, size) predict a failed read?

+
+

We read 80 such photos in the accurate mode. The reference labels come from two AI readers, not a person yet.

+
    +
  • Reading works. The full meter number came out right on 71 of 73 photos (97%). The US class label (CL200 and similar) read on 20 of 22. The fast mode reads only 75% of numbers, so the app uses the accurate mode.
    • +
  • Finding the number doesn't, yet. A nameplate carries serials, model codes and seal numbers too. "Take the tallest line of digits" picked the meter number on only 21 of 75 photos, and no text-only rule did better than 29. Barcodes help: one was decoded on 25 of the 80 photos, and it held the meter number on 20 of 24. No rule reached 95% precision on held-out photos, and a tap-to-confirm list of three contained the number on 85% of them, short of the 95% bar. Half the misses are plates with several real identifiers, so Base has to say which one it needs. Recommendation: always show the candidates and let the homeowner tap the right one, with a barcode match listed first.
    • +
  • When to ask for a retake. Two checks work from what the phone can see: blur (a sharpness score) and a number that is too small (its line 34 px tall or less, so "move closer"). Glare, framing and hand shake have no reliable signal yet, so those prompts can only be general advice.
    • +
  • Caveat. These are deliberate, well-lit close-ups. A homeowner's photos will be worse, so 97% is an upper bound, and the thresholds need checking on live camera frames on field day.
    • +
+
+
+
+
+ + +
+
+ What exists now +

Every PR is still open; nothing is merged

+

The cards run in the recommended merge order (decision 5). The app, the web page and the docs PR's rules sentence all depend on #11, so it goes first.

+ +
+
+

Merge first

+
    +
  1. #11Server: placement solver in plain code, POST /v1/placements, schemas + Ready for review. Tests pass; independent lane 40 of 40. The six defects a 05:40 review found are fixed test-first and redeployed +
    #14, #15 and #10 depend on it
    2. +
  2. #14AGENTS.md trimmed; stale docs fixed + Ready
    3. +
  3. #15Web review page + Checks green; Lighthouse mobile 94/100/96
    4. +
  4. #10iOS app: scan, upload, result in 3D and AR + Draft, CI green. The whole flow runs in the Simulator on the real replay; UI tests audit every screen. A 06:45 review found the upload would exceed the hosted server's 4.5 MB limit and a coverage bug when the ground height changes; fixes are queued for the app agent.
    5. +
  5. #13Verification: end-to-end replay to placement + Harness rebuilt; catches the old defects; found 4 new server issues (being fixed)
    6. +
  6. #12Evals: findings 1 and 2, one command each + CI green
    7. +
+
+
+

Then

+
    +
  1. #7Measure Lab: no-LiDAR measurement rig + Ready
    2. +
  2. #4Scoring harness vs a tape survey + Ready
    3. +
  3. #8TestFlight upload workflow + Ready; needs Sam's one-time App Store Connect key setup
    4. +
  4. #6Research notes + Ready; will be trimmed
    5. +
  5. #16Meter close-up eval + 97% read; finding the number: barcodes help, 85% top-3
    6. +
+
+
+

Not in the merge order

+
    +
  1. #9Hunter's capture-contract probe (draft) + Reviewed, no defects
    2. +
+
+
+
+ Ready or passing + Running or being fixed + Needs Sam + Pending or draft +
+ +

Server deploy. Live at house-scanning-server.vercel.app with public rules only; /health shows the rule source. It answers a sample scene in about 1 s, and the web review page's preview talks to it live. Vercel caps uploads at 4.5 MB, so the app sends scene.json alone; photos stay on the phone for now.

+ +
+ Every PR in full +
+ + + + + + + + + + + + + + + + +
PRWhatState
#14AGENTS.md trimmed; stale Swift/Expo and LiDAR facts fixed in docsReady
#11Server: placement solver in plain code, rules.yaml with citations or labeled demo placeholders, private override, FastAPI POST /v1/placements, site-plan SVG, scene and result schemas204 tests incl. 14 public golden cases and property tests (equality is "not sure", left/right mirror); independent lane: 40 of 40 cases pass. A fresh review at 05:40 found one path where unseen ground could still pass a check, two ways a small upload could stall the server, and a cable-length edge case reported as fail instead of not sure. All six are fixed, each with a test that failed first, and the hosted API is redeployed (05:31).
#15Web review page: decision stamp, spot, cable route, site plan, checks needing a look, missing viewsChecks green; Lighthouse mobile 94/100/96
#10iOS app: onboarding, meter anchor and close-up, guided wall walk with live coverage ("fog of war"), feature marking, one gap loop, scene export and upload, result in 3D and AR, replay mode for the Simulator, demo modeDraft, CI green; runs end to end in the Simulator on the real replay. Review fixes queued (upload size, coverage rebuild, error exits)
#13Verification: end-to-end replay to placement, Simulator screenshots per screen, accessibility audit, product description and bug triage, scoreboardRebuilt after a review showed it had missed four server defects; it now catches all four on the old server. At the current server it found four more (a gap request that never settles, ground past a marked wall end, a mislabelled error, slow hosted answers); the server agent is fixing them. The app reaches the server's answer on the real replay.
#12Evals: findings 1 and 2, each from one commandCI green
#16Meter close-up eval97% of meter numbers read; blur and too-small checks are ready; glare has no signal yet. Which line is the number: barcode holds it on 20 of 24; tap-to-confirm top 3 finds it on 85% (draft)
#17Panel label evalStopped at its gate: only 7 distinct openly licensed US panel photos exist, too few to measure. Needs our own photos.
#4Scoring harness: compare capture pipelines against a tape surveyReady
#6Research notesReady; will be trimmed
#7Measure Lab: no-LiDAR measurement rig and session formatReady
#8TestFlight upload workflow (manual, Sam approves, main only)Ready; needs Sam's one-time App Store Connect key setup
#9Hunter's capture-contract probe (draft)Reviewed, no defects
+
+
+
+ + +
+
+ How to try it +

Four ways in, easiest first

+
+
+

Web review page

+

Nothing to install. It loads synthetic samples.

+

Open the PR #15 preview

+
+
+

Server

+

The live one: https://house-scanning-server.vercel.app/health

+

To run it locally, check out PR #11 (the server isn't on main yet) and use the command in its server/README.md:

+
gh pr checkout 11
+cd server && uv run uvicorn api:app --port 8000
+curl http://localhost:8000/health
+
+
+

iOS app in the Simulator

+

Check out PR #10 and build ios/HouseScan.xcodeproj. Set the launch arguments in the scheme; for <url>, the live server works:

+
-replay <folder> -autopilot -serverURL <url>
+

-uiDemo runs the scripted screens. With no server given, the app falls back to -sampleResult, and the result screen labels it a sample.

+
+
+

Replay file

+

Not in git, because of the dataset license. Unzip it; -replay takes the folder inside.

+
~/house-scanning-data/replays/advio-20-0040-0075.zip
+
+
+
+
+ + +
+
+ Decide this morning +

Eight decisions, with the manager's recommendation

+
+
+

1Run the field test first today?

+
+
Recommend
Yes, first thing this morning, about an hour. Take a current iPhone to a real meter wall and run Measure Lab's one-hour tape protocol (PR #7) plus the field checklist in experiments/evals/README.md section 5 (PR #12): two long taped spans, one taped scale reference, a rig tap at every surveyed point. Follow experiments/evals/field/FIELD_SHEET.md (PR #12): it fixes the tap order so Measure Lab's M1-M10 match the pre-named templates, and you only type tape readings. make field SESSION=… TRUTH=… MAP=… then gives one scored table with the phone's scale error; a dry run on synthetic data recovered a planted 1.5% error exactly.
+
Why
This one number, the phone's scale error, decides how accurate the whole product can be.
+
If not
Every accuracy claim keeps resting on public data from a LiDAR iPhone, checked against a reference that may have been scaled to ARKit itself. A tape test on a current, non-LiDAR iPhone is the only direct measurement.
+
+
+
+

2Measure with AR taps, not depth maps?

+
+
Recommend
Every point a check depends on (window, door and wall edges, the meter, obstacles) is an AR tap the homeowner makes or confirms. Learned depth fills the 3D picture and coverage only.
+
Why
Depth edges were 8 in or worse in every test, and even flat walls only reached about 5 in p90 at a realistic phone error.
+
If not
A clearance check near its threshold passes or fails on depth-map error, which at edges is 8 in or worse.
+
+
+
+

3Walk both ends or stop early?

+
+
Recommend
Walk outward from the meter on both sides, and stop once the nearest spot that fits has all its evidence and everything closer has been seen. Reaching both wall ends is not required.
+
Why
The solver ranks spots by shortest cable run, then by distance from the meter, so once everything closer is seen on both sides, walking farther cannot find a better spot. A shorter walk also drifts less (Finding 1).
+
If not
Walking to both ends costs the homeowner time and adds far-away measurements the answer rarely needs. Stopping at the first spot that fits on one side can miss a closer spot on the other. Either way, unseen ground is never clear: the 05:40 review found one path in #11 where it could still pass; that is fixed and tested now.
+
+
+
+

4Load Base's real rules

+
+
Recommend
Put Base's actual values into private/rules.yaml (git-ignored; the server merges it over the public file), and confirm with Base which rules the demo follows and which identifier on the meter plate they need. Plates often carry several.
+
Why
With the public placeholder rules, automatic approval is off, so every real scan ends in "An installer will take a look", even when all checks pass.
+
If not
The demo can never show a real "fits here" answer.
+
+
+
+

5Merge order

+
+
Recommend
#11, #14, #15, #10, #13, #12, then #7, #4, #8, #6, #16.
+
Why
Server first, because the app, the web page and the docs PR's rules sentence depend on it.
+
If not
Whatever lands before #11 describes or calls a server that main doesn't have yet.
+
+
+
+

6Who owns each lane A-D today?

+
+
Recommend
No recommendation. The team names an owner per lane.
+
Why
Agents need to know whose OK they need before touching a lane's files.
+
If not
Agent work on lane files waits for someone to answer.
+
+
+
+

7Who checks what's hidden behind things?

+
+
Recommend
The app owns it. On every phone, show the homeowner the photo of the chosen spot before the result and ask whether anything blocks the wall or ground there. On LiDAR phones, also run a depth test: against true depth it removed the hidden stretch cleanly (0.1 ft left, no good wall lost, when 0.4 m of wall relief is allowed). Requiring views from different angles doesn't work: it removed the hidden stretch only by also throwing away 6 to 8.5 ft of good wall.
+
Why
Run on real photos of a building with a laser scan, the app's coverage map claimed 1.1 ft of a 19.3 ft wall that no photo saw, hidden behind scaffold footings. The server never sees the photos, so it can't catch this, yet it treats everything the app reports as seen.
+
If not
A gas meter or vent hidden behind a bush could sit inside a clearance the server marks as clear.
+
+
+
+

8What must the scan see, and what can we just ask?

+
+
Recommend
The app measures how far out it really saw the ground and asks only as far as the nearest check needs. The path the homeowner walked counts as proof the space in front of the wall is clear, and one "tilt up" view at the chosen spot covers headroom. A corner the homeowner marks ends the wall instead of becoming a request. Rare, far-away hazards (a pool, a driveway) become one yes/no question, labelled as the homeowner's answer.
+
Why
The verification agent found that today no scan can finish on the first try, however well it's walked: the app always reports ground 4 ft out while the checks need about 6 to 13 ft, it never reports the space in front of or above the wall, and each corner becomes a "walk past this" request. On the real replay that left 8 checks unsure and 6 follow-up requests.
+
If not
Every homeowner gets asked for photos they can't take, the opposite of Sam's first priority.
+
+
+
+
+
+ + +
+
+ Asks +

Things only a person can do

+
+
+

Review bots

+

Only CodeRabbit reviews this repo, and the free open-source plan limits it to about one review per hour. PR #14's review was skipped for the limit. Greptile, Macroscope, Aikido and Codex review are not installed. Codemetrics is installed but not connected to the team.

+

Ask: install or connect the ones you want. Until then, agents queue CodeRabbit on the most important PRs first (#11, #10).

+
+
+

TestFlight

+

Sam: one-time App Store Connect key setup for PR #8. The steps are in that PR.

+
+
+

Web previews

+

The free Vercel plan allows 100 deploys a day, and every push to any branch was building a web preview, so at 06:10 four PRs showed a red "rate limited" check. Automatic deploys for the web project are now off; the web page's previews are deployed by hand, and the red checks clear on each PR's next push.

+

Ask (Sam): reconnect it after the hackathon (vercel git connect), or move the project to a paid plan if you want previews on every push.

+
+
+

Check the meter labels (anyone, 5 minutes)

+

The meter results rest on labels two AI readers made. Open ~/house-scanning-data/meter-closeup/review.html on Sam's laptop and mark each photo's number keep or fix.

+
+
+
+
+ + +
+
+ Today +

What the agents keep doing

+
    +
  • Run the app against the live server end to end, re-check the product description against the app, and polish the capture UX.
    • +
  • Field-day kit: the Measure Lab build, the survey sheet and the importer map (evals agent).
    • +
  • Collect and fix bot review findings as PRs leave draft: CodeRabbit now, the others once installed.
    • +
  • More experiments: meter close-up, frame selection, and a current iPhone's scale error once the field data exists.
    • +
+
+
+
+ +
House scanning team briefing. Local only, never committed. One file, works offline.
+ + From 97ed0958c34362c2baa849150a002362880562c1 Mon Sep 17 00:00:00 2001 From: SamGu-NRX Date: Sat, 26 Sep 2026 09:58:57 -0500 Subject: [PATCH 2/2] docs: rebuild the morning briefing around pictures The first version read as a dense report. This one teaches the system before reporting on it: a to-scale wall with the clearance rules, a six-panel strip that adds one part of a scan at a time, one picture per key idea (error growing with distance, PASS/FAIL/UNSURE on a number line, unseen is not clear), a status pipeline with the real-phone blocker marked, and each overnight finding with the product change it caused. Same facts; no Base-provided content, dataset images or local paths. Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/briefings/2026-09-26-morning.html | 1613 +++++++++++++----------- 1 file changed, 894 insertions(+), 719 deletions(-) diff --git a/docs/briefings/2026-09-26-morning.html b/docs/briefings/2026-09-26-morning.html index 29cc69316..9e8cb28da 100644 --- a/docs/briefings/2026-09-26-morning.html +++ b/docs/briefings/2026-09-26-morning.html @@ -4,751 +4,926 @@ -8 AM briefing: measure with AR taps, and test the phone against a tape first +House scanning briefing: what we are building, what works, what is blocked -
-
-

8 AM briefing: measure with AR taps, and test the phone against a tape first

-

Last night's tests on real data say a current iPhone's own tracking fits the error allowance the server already uses, and that anything a check depends on should be an AR tap, not a depth map. That tracking data came from a LiDAR iPhone, so measuring a current non-LiDAR iPhone with a tape is the first job this morning.

- -
-
-

What the data said

-

A 2023 iPhone's own tracking stayed inside the server's error allowance: 18.5 in p90 after 30 ft, against 57.6 in allowed. The 2018 phone was 2 to 3 times over. Learned depth gets flat walls to about 5 in p90 at a realistic phone error, and only with perfect feature matching. Edges: 8 in or worse in every test.

- The three findings -
-
-

What exists now

-

Server solver, web review page, iOS app, verification and evals, all in open PRs. Nothing is merged. The server is live, with the defects a 05:40 review found in it fixed and redeployed. On a real replay the app talked to the server end to end: a 77-photo scene, answered in 545 ms.

- PR map and how to try it -
-
-

What we decide

-

Take a current iPhone and a tape to a real meter wall first thing, about an hour. Then settle how points get measured (AR taps), how far to walk, loading Base's real rules, who checks what is hidden behind things, what the scan must see versus ask, merge order and lane owners.

- Go to the decisions -
-
- - -
-
- -
-
-
- Where we really are -

Works end to end in the Simulator; never run on a real iPhone

-
-

Works

The iOS app runs the whole scan (meter, close-up, guided walk with coverage, one more view, upload, result in 3D and AR) in the Simulator on a recorded real walk, and gets a real answer from the placement server, which is live and tested.

-

Not proven

Nothing has run on a physical iPhone at a real wall. Until this morning's fixes land, no scan could finish on the first try (the app under-reported what it saw), and with placeholder rules every answer is "an installer will take a look".

-

The main blocker

A real-phone session: the app and Measure Lab on a current iPhone, at a real meter wall, with a tape. Hunter's research handoff (#18) reaches the same conclusion. Everything else is waiting on that data.

-

Side work to park

The web review page (#15) shows the server's answer in a browser. It helped check the server before the app could, but it isn't the product. Don't spend more on it now.

-
-
-
- - -
-
- The product -

Where last night's findings land

-

A homeowner walks the outside wall around their electric meter with an iPhone. The findings below all hit the second step, where the phone measures.

-
    -
  1. - -

    1. Walk the wall

    -

    The homeowner walks along the wall on both sides of the meter.

    -
    2. -
  2. - -

    2. Phone measures and photographs

    -

    ARKit tracks where the phone is. The app takes photos of the wall and a close-up of the meter.

    -
    Finding 1: driftFinding 2: depthFinding 3: meter read
    -
    3. -
  3. - -

    3. Server applies the rules in plain code

    -

    Placement rules live in rules.yaml with citations. No model makes the call.

    -
    PR #11
    -
    4. -
  4. - -

    4. Spot shown in AR

    -

    The app shows the battery spot on the real wall, placed relative to the meter.

    -
    PR #10
    -
    5. -
-

Sam's priorities, in order: (1) the scan is easy and works first try, (2) the 3D is accurate, (3) the model appears live in the app.

-
-
- - -
-
- What the data said -

Three findings from real data

- - -
-

Finding 1. A current iPhone's tracking stays inside the server's allowance

-

Position error is the distance between where the phone thinks it is and where it really is, sideways drift included. We measured it after walks of 3 to 30 ft on two public datasets, with the same code for both (make drift and make modern-arkit in PR #12).

- -
- - - p90 position error against feet walked, with the server's allowance - p90 position error in inches at 3, 10, 20 and 30 ft walked. 2018 iPhone 6s: 18.6, 55.7, 94.6, 132.6. 2023 iPhone 14 Pro Max, trusted walks: 4.0, 8.6, 13.4, 18.5. Server's allowance: 5.8, 19.2, 38.4, 57.6. The current iPhone stays under the allowance at every distance; the 2018 phone is 2 to 3 times over it. The vertical axis is broken above 60 in; the band from 80 to 140 in uses a smaller scale. - - - - - - - 020 in40 in60 in - 100 in140 in - - - 3 ft10 ft20 ft30 ft - Distance walked - - Position error, p90 - - - - inside the allowance - - - - - - - - - - - - axis break: the band above uses a smaller scale - - 2018 phone 132.6 in - allowance 57.6 in - 2023 phone 18.5 in - - 3 ft walked: 2018 phone 18.6 in, 2023 phone 4.0 in, allowance 5.8 in - 10 ft walked: 2018 phone 55.7 in, 2023 phone 8.6 in, allowance 19.2 in - 20 ft walked: 2018 phone 94.6 in, 2023 phone 13.4 in, allowance 38.4 in - 30 ft walked: 2018 phone 132.6 in, 2023 phone 18.5 in, allowance 57.6 in - - -
p90 position error in inches: 9 in 10 cases are better. The allowance is the server's 0.16 ft per ft plus its source default. Counting three doubtful MARViN walks, the current iPhone's p90 is 6.9 / 18.7 / 37.5 / 50.0 in, just over the allowance at 3 ft only. The 2018 phone is 2 to 3 times over it, though that dataset's own reference is noisy and inflates the number, and one of its four walks lost tracking entirely. Hover a column for its values.
-
Data table -
- - - - - - - -
Walked2018 iPhone 6s (in)2023 iPhone, trusted walks (in)2023 iPhone, incl. 3 doubtful walks (in)Server's allowance (in)
3 ft18.64.06.95.8
10 ft55.78.618.719.2
20 ft94.613.437.538.4
30 ft132.618.550.057.6
-
-
- -
-
- - A tape measure printed too small - Two rulers. The top one is the real distance walked, marked at 3, 10, 20 and 30 feet. The bottom one is what a phone with a scale error counts: every mark lands a little short, and the gap between the rulers grows the farther you walk. - - - - - How far you really walked - - - - 3 ft10 ft20 ft30 ft - - - - - - - the gap - What the phone counts - -
-
-

Scale error works like a tape printed too small

-

Every reading comes up short, and the longer the reading, the more inches you lose. On the 2018 phone, ARKit read distances 5% to 17% short against GPS. On the current iPhone, its scale matched the dataset's reference within 2% on 30 of 35 outdoor walks.

-

Where the picture breaks. A misprinted tape is off by the same amount every time, while the 2018 phone's shortfall changed from walk to walk. And a tape only measures along the walk; position error also counts sideways drift.

-
-
- -

Two caveats on the current iPhone. MARViN doesn't say how its reference got its meters, so it may have been scaled to ARKit itself. And that phone has LiDAR, which the phones we target mostly lack.

-

What it means. The server widens every AR and photo measurement's error bar by 0.16 ft for each foot away from the meter, so a far-away measurement comes back "not sure" instead of passing wrongly. That allowance is right for current phones. A tape test on a current, non-LiDAR iPhone is still the only direct measurement.

-
- - -
-

Finding 2. Learned 3D from photos gets flat walls to about 5 in at best, never edges

-

We tested on ETH3D: real building scans with laser ground truth, points within 6 m of the camera, 1 to 3 m spans. Each model turns a photo into depth. On its own, each one gets the size of the scene wrong.

- -
- - Scale error range of each model on its own - MoGe-2 reads distances 4 to 12 percent long. Depth Anything 3 reads them 7 to 11 percent short. - - - reads short - reads long - MoGe-2Depth Anything 3 - - MoGe-2: 4% to 12% long - Depth Anything 3: 7% to 11% short - - - +4%+12% - −11%−7% - - −20%−10%0+10% - Scale error, each model alone - -
Each bar spans the range of scale errors we saw for that model. That leaves a median error of 6 to 8 in, and a p90 of about 20 in.
-
Data table -
- - -
Model, aloneScale error
MoGe-24% to 12% long
Depth Anything 37% to 11% short
-
-
- -
-
-

One taped distance fixes the typical case, not the bad ones

-

One photo plus one taped distance:

-
-
1.6 inmedian
-
9.6 inp90 on surfaces
-
15.1 inp90 on walls
-
26.9 inp90 at edges
-
-
-
-

The best recipe: let the phone's poses set the scale

-

A pose is where the phone stood and which way it pointed for a photo. ARKit records one for every photo. We take MoGe-2's depth and rescale it with the phone's own poses over 8 photos.

-

The chart below changes one thing between rows: how wrong the phone is about scale. Each whisker is a 95% interval, from resampling camera positions and 5 draws of pose noise.

-
-
- -
- - - p90 error under three pose settings, with 95% intervals - MoGe-2 depth rescaled with the phone's poses over 8 photos, points within 6 m, spans 1 to 3 m, p90 error in inches with 95% intervals. Exact poses: overall 4.0 (3.3 to 4.8), walls 2.8 (2.0 to 4.4). A current phone with 2 percent scale error: overall 5.6 (4.6 to 6.7), walls 5.0 (3.8 to 6.8). The 2018 phone's error: overall 14.6 (10.6 to 18.7), walls 10.8 (8.6 to 13.1). Edges are 8 in or worse in every setting. - - - - - Edges land here: - 8 in or worse, every setting - - - Exact posesA current phoneThe 2018 phone's error - no scale error2% scale errorADVIO iPhone 6s - - Exact poses, overall: 4.0 in (95% interval 3.3 to 4.8) - Exact poses, walls: 2.8 in (95% interval 2.0 to 4.4) - A current phone, overall: 5.6 in (95% interval 4.6 to 6.7) - A current phone, walls: 5.0 in (95% interval 3.8 to 6.8) - The 2018 phone's error, overall: 14.6 in (95% interval 10.6 to 18.7) - The 2018 phone's error, walls: 10.8 in (95% interval 8.6 to 13.1) - - - - 4.0 in2.8 in - 5.6 in5.0 in - 14.6 in10.8 in - - 04 in8 in12 in16 in20 in - p90 error, spans 1-3 m, points within 6 m (lower is better) - -
Only the phone's scale error changes between rows, and the error follows it. With a current phone's 2% error, walls land at 5.0 in p90 (95% interval 3.8 to 6.8). All three rows assume perfect feature matching between photos; without it, walls reach 6.3 in even with exact poses. The earlier "walls at 4 in" figure came from a single noise draw, the lucky end of the spread.
-
Data table -
- - - - - - -
PosesOverall, p90 (in) [95% interval]Walls, p90 (in) [95% interval]
Exact poses4.0 [3.3, 4.8]2.8 [2.0, 4.4]
A current phone (2% scale error)5.6 [4.6, 6.7]5.0 [3.8, 6.8]
The 2018 phone's error14.6 [10.6, 18.7]10.8 [8.6, 13.1]
-
-
- -
-

Verdict. At a realistic phone error, learned depth gets flat walls to about 5 in p90, and only with perfect feature matching between photos. Edges (window and door frames, equipment) stay at 8 in or worse in every setting.

-

For the app. Measurements that decide a check should come from AR taps, which carry the phone's own tracking error from Finding 1, not from depth maps. Learned depth is useful for coverage and the 3D picture, not for clearances near a threshold.

-
-
- - -
-

Finding 3. Meter close-up: reading works, finding the number doesn't yet

-

The question: how often does Apple's on-device text recognition read the meter number and class from real, openly licensed meter photos (Wikimedia Commons)? And which cheap photo checks (blur, glare, size) predict a failed read?

-
-

We read 80 such photos in the accurate mode. The reference labels come from two AI readers, not a person yet.

-
    -
  • Reading works. The full meter number came out right on 71 of 73 photos (97%). The US class label (CL200 and similar) read on 20 of 22. The fast mode reads only 75% of numbers, so the app uses the accurate mode.
    • -
  • Finding the number doesn't, yet. A nameplate carries serials, model codes and seal numbers too. "Take the tallest line of digits" picked the meter number on only 21 of 75 photos, and no text-only rule did better than 29. Barcodes help: one was decoded on 25 of the 80 photos, and it held the meter number on 20 of 24. No rule reached 95% precision on held-out photos, and a tap-to-confirm list of three contained the number on 85% of them, short of the 95% bar. Half the misses are plates with several real identifiers, so Base has to say which one it needs. Recommendation: always show the candidates and let the homeowner tap the right one, with a barcode match listed first.
    • -
  • When to ask for a retake. Two checks work from what the phone can see: blur (a sharpness score) and a number that is too small (its line 34 px tall or less, so "move closer"). Glare, framing and hand shake have no reliable signal yet, so those prompts can only be general advice.
    • -
  • Caveat. These are deliberate, well-lit close-ups. A homeowner's photos will be worse, so 97% is an upper bound, and the thresholds need checking on live camera frames on field day.
    • -
-
+ + + +
+ +

House scanning: what we are building, what works, and what is blocked

+

Team briefing, 26 September 2026, facts as of 09:10. Readers: Sam, Hunter, Shrey, Aiden. About ten minutes.

+ +

A homeowner walks the outside wall around their electric meter with an iPhone. The phone measures the wall, a server applies the clearance rules in plain code, and the app shows where the battery goes, in AR, on the real wall.

+

Today it runs end to end in the iPhone Simulator on a recorded real walk and gets a real answer from the live server. It has never run on a physical iPhone at a real wall. That is the blocker, and the first job this morning.

+ + + + +

1. The problem: a battery has to fit between the rules

+ +
+ + An outside wall with the battery and its clearance rules, drawn to scale + Front view of a house wall. From left: an AC unit on the ground, a gas meter, the electric meter, the battery just to its right, a window, a door. Red hatched zones show 3 feet of clearance around the gas meter, AC unit, window and door. A bracket beside the battery shows 6.5 feet of headroom. A blue bracket along the ground shows 20 feet of cable reach either side of the electric meter. A tape along the bottom counts feet from the meter. Below, a view from above shows the battery footprint, 31 by 22 inches, and the 3 feet of clear space needed in front of it. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 20 ft of cable, either side of the meter + + + + AC unit + gas meter + electric meter + window + door + 6.5 ft headroom + battery + + + 3 ft + 3 ft + 3 ft + 3 ft + + + + + + + + + + + 151050 + 510152025 + + feet left and right of the electric meter + + + The same wall from above + + + + + + + + + + + + + + + + + + + + + + + + battery footprint, 31 by 22 in + 3 ft clear in front + AC unit + gas + + +

Wider than the screen: scroll sideways for the rest of the picture.

+
Everything is drawn at one scale: 5 ft between tape marks. The red hatched areas are places the battery may not go. The battery just fits between the electric meter and the window's zone. Rules shown: 3 ft from the gas meter, from AC units, fences and other batteries, and from doors and windows that enter the home; 3 ft of clear space in front; 6.5 ft of headroom; within 20 ft of cable run from the meter; a flat, seen stretch of wall behind it.
+
+ +
+

Base Power puts a home battery on the outside wall, close to the electric meter, because a cable runs from the meter to the battery. The battery is 31 in wide, 22 in deep and 39.5 in tall. Whether it fits, and where, depends on the clearance rules in the picture. Today a person has to go to the house and check.

+

Our product does that check with the homeowner's iPhone, in a couple of minutes. The app measures the wall. A server applies the rules in plain code. The app shows where the battery goes, in AR, on the real wall.

+

The demo's rules come from public sources and live in server/rules.yaml, each with its citation. Two numbers there are placeholders because no public value exists (pool 10 ft, driveway 5 ft). Base's own thresholds are private, load at runtime from a file that is not in git, and do not appear on this page.

+
+
+ The rules the demo uses, with sources +
+ + + + + + + + + + + +
RuleValueSource
Distance from the gas meter3 ftBase's public help page; Austin Energy; Texas Gas Service
Distance from AC units, fences, other batteries3 ftBase's help page
Distance from doors and windows that enter the home3 ftIRC R328.4
Clear space in front of the battery3 ftBase's help page
Cable run from the meterwithin 20 ftBase's help page
Headroom6.5 ftNEC 110.26, a demo choice
Wall behind the batteryflat, and seen by the camerademo requirement
Pool, driveway10 ft, 5 ftplaceholders; no public value exists
+
+ + +

2. How a scan becomes a decision

+

Six pictures. Each one is the last one with one thing added. The tape along the bottom is real: the app shows the wall unrolled like a tape measure, with the electric meter at zero.

+ +
    +
  1. + + Step 1: tap the electric meter, which becomes zero + The wall from the problem picture, smaller, with the electric meter in the middle. A blue ring marks a tap on the meter. A dashed line drops from the meter to the zero mark on a tape along the bottom. + + + + + tap the electric meter + it becomes 0 ft; all else is left or right of it + +
    Point at the electric meter and tap. The meter becomes zero. Every position from now on is measured left or right of it, in feet.
    +
    2. + +
  2. + + Step 2: the phone photographs the meter close up, then walks left, then right + Same wall. A dashed frame around the electric meter marks the close-up. A phone stands on the ground below the meter. Blue arrows run left along the wall and then right. + + + close-up, reads the number + + + + walk left + then right + +
    The phone takes a close-up of the meter by itself when the picture is sharp, to read the meter's number. Then you walk along the wall, left and then right. The phone keeps photos automatically.
    +
    3. + +
  3. + + Step 3: a haze covers wall and ground the phone has not seen yet + Same wall. The phone has walked left to about 8 feet and right to about 9 feet. Those parts of the wall and ground are clear. The rest is covered by a grey hatched haze. + + + + + + + + haze: + not seen yet + haze + seen: the haze lifts + +
    A haze covers wall and ground the phone has not seen and lifts as it sees them. The team calls this fog of war. The tape strip grows as you walk.
    +
    4. + +
  4. + + Step 4: mark things as you pass, and answer one request for another view + Same wall, almost all seen. Orange pins mark the gas meter, the window and the AC unit. A small hatched patch of ground about 6 feet right of the meter is still unseen. A message at the top reads: Show the ground about 6 ft right of your meter. + + + + + + gas + window + AC + + + Show the ground about 6 ft right of your meter + +
    Mark things as you pass: gas meter, AC unit, door, window, driveway edge, fence. If something a rule needs is still unseen, the app asks for one more view, in one sentence.
    +
    5. + +
  5. + + Step 5: a small measurement file goes to the server, which tries every spot + Same wall, fully seen. A small file icon labelled scene.json, 17 KB, rises from the phone toward a server box at the top. On the wall, dashed battery outlines slide along in small steps. + + + + + + tries every spot, 2 in apart + + + scene.json, 17 KB + photos stay on the phone + + + server: checks every rule + +
    The phone sends a small file of measurements, about 17 KB for a real walk. The server slides the battery's footprint along the wall in 2-inch steps and checks every rule at every position.
    +
    6. + +
  6. + + Step 6: the answer comes back and the battery is shown on the wall in AR + Same wall. The server box at the top sends an arrow back down to the phone. The battery is drawn on the wall just right of the electric meter with an orange outline, and a cable runs from the meter to it. A label reads: best spot, shortest cable. + + + server answers + + + + + + best spot, + shortest cable + shown in 3D and in AR, placed from the meter + +
    The server answers with the best spot (shortest cable run first), the cable route, and every check's outcome and reason. The phone draws the battery on the wall, placed relative to the meter, in 3D and in AR.
    +
    7. +
+ +
+ + The whole loop: phone to server and back + Two boxes. Left: the phone, which taps, walks, marks and shows. Right: the server, with the rules file and plain code. An arrow from phone to server carries scene.json. An arrow from server to phone carries the spot, the cable route and every check's outcome. + + Phone + tap, walk, mark + show the spot in AR + + Server + rules in a file + plain code, no AI + + scene.json + + spot, cable, checks + +
The recap. AI is used only to recognise things: it reads the meter's text and finds objects. The placement decision is ordinary code with the rules in a file, so every answer traces back to a rule and a measurement.
+
+ + +

3. Three ideas that make the answer honest

+ +

The phone counts its own steps, so error grows with distance

+
+ + Error bars widen the farther you walk from the meter + A tape along the top counts feet from the meter at 0, 5, 10, 15 and 20. Below it, four positions carry horizontal error bars that get wider with distance: plus or minus 0.3 feet at the meter, 1.1 feet at 5 feet, 1.9 feet at 10 feet and 3.5 feet at 20 feet. Footprints lead away from the meter. + + + + + + + 05101520 ft + + + meter + + + + + + + + + + + + + + + ± 0.3 ft at the meter + ± 1.1 ft at 5 ft + ± 1.9 ft at 10 ft + ± 3.5 ft at 20 ft + + error bar = 0.3 ft + 0.16 ft for every foot from the meter + +
AR tracking is dead reckoning: the phone estimates how far it moved from the camera and motion sensors, like counting your own steps from a doorway. Small errors add up, so the farther from the meter, the less sure a position is. The server widens every AR measurement's error bar by 0.3 ft plus 0.16 ft per foot along the wall. Where the analogy breaks: a miscounted step is only a stretch along the wall, while the phone's error also drifts sideways.
+
+ +

The error bar decides: PASS, FAIL or UNSURE

+
+ + Three measurements against the 3 foot gas meter rule + A number line from 0 to 6 feet with a vertical rule at 3 feet. Row one: 4.5 feet plus or minus 0.8, the whole bar sits right of 3, PASS. Row two: 2.0 feet plus or minus 0.8, the whole bar sits left of 3, FAIL. Row three: 3.4 feet plus or minus 0.8, the bar straddles 3, UNSURE. + + + + + 0123456 ft + + + 3 ft rule (gas meter) + too close + clear + + + + + + + 4.5 ft ± 0.8: clear by 1.5, more than 0.8 + + ✓ PASS + + + + + + + 2.0 ft ± 0.8: short by 1.0, more than 0.8 + + ✕ FAIL + + + + + + + 3.4 ft ± 0.8: clear by only 0.4 + + ? UNSURE + +
Every measurement carries an error bar. A check passes only when the margin beats the error, fails only when it is past the rule by more than the error, and is UNSURE otherwise. Equal is UNSURE. UNSURE means a person checks it, or the app asks for a better view. The three examples are the dossier's, for the 3 ft gas meter rule with ± 0.8 ft.
+
+ +

Unseen is not clear

+
+ + A check whose ground was never seen stays UNSURE and names the missing view + A candidate battery spot on the wall. The ground beside it is covered by grey haze. Beside it a check reads: ground near the spot, checks need 6 to 13 ft of seen ground, UNSURE, missing evidence: show the ground about 6 ft right of your meter. + + + + + + meter + + + candidate spot + + never seen + + Ground near the spot + checks need 6 to 13 ft of ground + + ? UNSURE + missing evidence: show the + ground 6 ft right of your meter + +
If the camera never saw part of the area a rule depends on, that check cannot pass. It stays UNSURE, and the server names the view that would settle it. Here the ground beside the spot was never seen, and the checks that use ground need 6 to 13 ft of it. A blank on the map is never treated as empty ground.
+
+ + +

4. Where it stands this morning

+
+ ✓ proven in the Simulator on a recorded real walk + ! fixed this morning + ? not yet proven on a real phone +
+
    +
  1. Tap the meterIt becomes zero.
    ✓Simulator?Real phone
    2. +
  2. Read the meter's numberClose-up; three candidates to tap.
    ✓Simulator?Real phone
    3. +
  3. Walk the wallHaze lifts; tape strip grows.
    ✓Simulator!Now reports ground depth, clear space in front, and a tilt-up view for headroom?Real phone
    4. +
  4. Mark things, one more viewGas, AC, door, window, fence.
    ✓Simulator!Server now publishes exactly what settles each check?Real phone
    5. +
  5. Upload scene.jsonMeasurements only.
    !17 KB, not 16.5 MB of photos; answer in 0.5 s?Real phone
    6. +
  6. Server decidesLive on Vercel.
    ✓252 tests; independent checker 50 of 50
    7. +
  7. Show the spot in ARPlaced from the meter.
    ✓Simulator, 3D and AR?Real phone
    8. +
+
✕
Never run on a physical iPhone at a real wall.Every stage marked "?" above shares this one gap. Everything that decides accuracy is waiting on that data.
+
+
Placeholder rulesWith the public placeholder rules, automatic approval is off, so every real result says "An installer will take a look."
+
Hidden wallThings standing in front of the wall can hide it, and nobody owns checking that yet (decision 7).
+
Parked: web review page (#15)It shows the server's answer in a browser. It helped check the server before the app could upload. It is not the product.
+
+ +
+
Does what we built work, and is it part of the app we'd ship?
In the Simulator on a recorded real walk, yes: meter to result, with a real answer from the live server. The app and the server are the product. The web review page is not, and it is parked.
+
Are the overnight tests doing anything?
Each of the five changed the app or the server (section 6 lists what). Three other pieces of overnight work changed nothing: the panel-label search, finer statistics on the modern-phone data, and the research notes.
+
What is done that achieves the goal?
The server, live with 252 tests. The whole app flow in the Simulator. Uploads fixed to 17 KB. The first-try fixes, merging now while the build is repaired.
+
What is the main blocker?
A real-phone session. See the next section.
+
+ + +

5. The main blocker, and what to do this morning

+
+ + The field test: the phone's reading against a tape on the same wall + A wall with the electric meter. A yellow tape runs along the wall with marks at 5, 10, 15 and 20 feet. A phone stands in front. Blue dots show where the phone thinks the same marks are; the gap between dot and mark grows with distance. A label reads: the gap is the phone's scale error. + + + + + + + + + + + 5101520 ft + + + + + + blue dots: where the phone says the tape marks are + the gap is the scale error + + a current iPhone, no LiDAR + +
Nothing decides accuracy until this runs. Hunter's research handoff (PR #18) reaches the same conclusion.
+
+
+

Run the field test first, about an hour. You need a current non-LiDAR iPhone, the Measure Lab build, a 30 ft tape and a real meter wall. Follow experiments/evals/field/FIELD_SHEET.md: type the tape readings into the prepared templates, and one command turns them into a score and the phone's real scale error. A dry run recovered a planted 1.5% error exactly, so the scoring end is ready.

+

The overnight tracking numbers (finding 1) come from a 2023 iPhone with LiDAR and a dataset whose reference may itself be scaled to the phone's tracking. Most homeowners' phones lack LiDAR. The field test measures our own phone on our own wall, which is the only direct measurement.

+
+ + +

6. What the overnight tests found, and what each changed

+

All five ran on public real-world datasets with laser-measured ground truth. Dataset images are never in git. Each finding below ends with the line that answers "did this test do anything".

+ +
+

Finding 1. How far off is the phone's own tracking?

+
+
+ 2018 iPhone 6s (ADVIO) + 2023 iPhone 14 Pro Max (MARViN) + server's allowance
-
-
- - -
-
- What exists now -

Every PR is still open; nothing is merged

-

The cards run in the recommended merge order (decision 5). The app, the web page and the docs PR's rules sentence all depend on #11, so it goes first.

- -
-
-

Merge first

-
    -
  1. #11Server: placement solver in plain code, POST /v1/placements, schemas - Ready for review. Tests pass; independent lane 40 of 40. The six defects a 05:40 review found are fixed test-first and redeployed -
    #14, #15 and #10 depend on it
    2. -
  2. #14AGENTS.md trimmed; stale docs fixed - Ready
    3. -
  3. #15Web review page - Checks green; Lighthouse mobile 94/100/96
    4. -
  4. #10iOS app: scan, upload, result in 3D and AR - Draft, CI green. The whole flow runs in the Simulator on the real replay; UI tests audit every screen. A 06:45 review found the upload would exceed the hosted server's 4.5 MB limit and a coverage bug when the ground height changes; fixes are queued for the app agent.
    5. -
  5. #13Verification: end-to-end replay to placement - Harness rebuilt; catches the old defects; found 4 new server issues (being fixed)
    6. -
  6. #12Evals: findings 1 and 2, one command each - CI green
    7. -
-
-
-

Then

-
    -
  1. #7Measure Lab: no-LiDAR measurement rig - Ready
    2. -
  2. #4Scoring harness vs a tape survey - Ready
    3. -
  3. #8TestFlight upload workflow - Ready; needs Sam's one-time App Store Connect key setup
    4. -
  4. #6Research notes - Ready; will be trimmed
    5. -
  5. #16Meter close-up eval - 97% read; finding the number: barcodes help, 85% top-3
    6. -
-
-
-

Not in the merge order

-
    -
  1. #9Hunter's capture-contract probe (draft) - Reviewed, no defects
    2. -
-
-
-
- Ready or passing - Running or being fixed - Needs Sam - Pending or draft -
- -

Server deploy. Live at house-scanning-server.vercel.app with public rules only; /health shows the rule source. It answers a sample scene in about 1 s, and the web review page's preview talks to it live. Vercel caps uploads at 4.5 MB, so the app sends scene.json alone; photos stay on the phone for now.

- -
- Every PR in full -
- + + p90 position error in inches against feet walked, two phones and the server's allowance + At 3, 10, 20 and 30 feet walked: 2018 iPhone 6s 18.6, 55.7, 94.6, 132.6 inches. 2023 iPhone 14 Pro Max 4.0, 8.6, 13.4, 18.5 inches. Server's allowance 5.8, 19.2, 38.4, 57.6 inches. The 2023 phone stays under the allowance at every distance. The 2018 phone is over it at every distance. + + + + 040 in80 in120 in + 3 ft10 ft20 ft30 ftdistance walked + + + + + + + + + 2018 phone, 132.6 in + allowance, 57.6 in + 2023 phone, 18.5 in + + 3 ft walked: 2018 phone 18.6 in, 2023 phone 4.0 in, allowance 5.8 in + 10 ft walked: 2018 phone 55.7 in, 2023 phone 8.6 in, allowance 19.2 in + 20 ft walked: 2018 phone 94.6 in, 2023 phone 13.4 in, allowance 38.4 in + 30 ft walked: 2018 phone 132.6 in, 2023 phone 18.5 in, allowance 57.6 in + + +
Position error after walking, p90 (9 in 10 cases are better). A current iPhone's own tracking stays inside the server's allowance at every distance; the 2018 phone is far over it. The allowance line is the 0.16 ft per foot part of the server's error bar. Caveats: the MARViN reference may itself be scaled to the phone's tracking, and that phone has LiDAR, which most homeowners' phones lack.
+
Data table +
PRWhatState
+ - - - - - - - - - - - - - + + + +
Walked2018 iPhone 6s (ADVIO), in2023 iPhone 14 Pro Max (MARViN), inServer's allowance, in
#14AGENTS.md trimmed; stale Swift/Expo and LiDAR facts fixed in docsReady
#11Server: placement solver in plain code, rules.yaml with citations or labeled demo placeholders, private override, FastAPI POST /v1/placements, site-plan SVG, scene and result schemas204 tests incl. 14 public golden cases and property tests (equality is "not sure", left/right mirror); independent lane: 40 of 40 cases pass. A fresh review at 05:40 found one path where unseen ground could still pass a check, two ways a small upload could stall the server, and a cable-length edge case reported as fail instead of not sure. All six are fixed, each with a test that failed first, and the hosted API is redeployed (05:31).
#15Web review page: decision stamp, spot, cable route, site plan, checks needing a look, missing viewsChecks green; Lighthouse mobile 94/100/96
#10iOS app: onboarding, meter anchor and close-up, guided wall walk with live coverage ("fog of war"), feature marking, one gap loop, scene export and upload, result in 3D and AR, replay mode for the Simulator, demo modeDraft, CI green; runs end to end in the Simulator on the real replay. Review fixes queued (upload size, coverage rebuild, error exits)
#13Verification: end-to-end replay to placement, Simulator screenshots per screen, accessibility audit, product description and bug triage, scoreboardRebuilt after a review showed it had missed four server defects; it now catches all four on the old server. At the current server it found four more (a gap request that never settles, ground past a marked wall end, a mislabelled error, slow hosted answers); the server agent is fixing them. The app reaches the server's answer on the real replay.
#12Evals: findings 1 and 2, each from one commandCI green
#16Meter close-up eval97% of meter numbers read; blur and too-small checks are ready; glare has no signal yet. Which line is the number: barcode holds it on 20 of 24; tap-to-confirm top 3 finds it on 85% (draft)
#17Panel label evalStopped at its gate: only 7 distinct openly licensed US panel photos exist, too few to measure. Needs our own photos.
#4Scoring harness: compare capture pipelines against a tape surveyReady
#6Research notesReady; will be trimmed
#7Measure Lab: no-LiDAR measurement rig and session formatReady
#8TestFlight upload workflow (manual, Sam approves, main only)Ready; needs Sam's one-time App Store Connect key setup
#9Hunter's capture-contract probe (draft)Reviewed, no defects
3 ft18.64.05.8
10 ft55.78.619.2
20 ft94.613.438.4
30 ft132.618.557.6
-
-
- - -
-
- How to try it -

Four ways in, easiest first

-
-
-

Web review page

-

Nothing to install. It loads synthetic samples.

-

Open the PR #15 preview

-
-
-

Server

-

The live one: https://house-scanning-server.vercel.app/health

-

To run it locally, check out PR #11 (the server isn't on main yet) and use the command in its server/README.md:

-
gh pr checkout 11
-cd server && uv run uvicorn api:app --port 8000
-curl http://localhost:8000/health
-
-
-

iOS app in the Simulator

-

Check out PR #10 and build ios/HouseScan.xcodeproj. Set the launch arguments in the scheme; for <url>, the live server works:

-
-replay <folder> -autopilot -serverURL <url>
-

-uiDemo runs the scripted screens. With no server given, the app falls back to -sampleResult, and the result screen labels it a sample.

-
-
-

Replay file

-

Not in git, because of the dataset license. Unzip it; -replay takes the folder inside.

-
~/house-scanning-data/replays/advio-20-0040-0075.zip
-
+ +

What it changed: the server's error allowance comes from this. The field test measures our own phone.

+
+ +
+

Finding 2. Can photos alone give 3D measurements?

+
+ + Error of learned depth from photos: flat walls and edges + A horizontal scale from 0 to 10 inches. Flat walls: 5.0 inches at p90, with a 95 percent interval from 3.8 to 6.8. Window and door edges: 8 inches or worse in every setting, drawn as a bar starting at 8 and running off the right edge. + + + + 0246810 inerror, p90 + flat wall + + + + 5.0 in (3.8 to 6.8) + edges + + + 8 in or worse, every setting + +
Learned depth models (MoGe-2, Depth Anything 3), rescaled with the phone's poses. Flat walls come out at about 5 in at best at a realistic phone error. Window and door edges, which are what clearances are measured from, come out at 8 in or worse in every setting.
+
+

What it changed: no photo-to-3D measurement pipeline. Every point a check depends on is an AR tap; learned depth only draws the 3D picture. This saved building the wrong thing.

+
+ +
+

Finding 3. Does the app's coverage map claim wall it never saw?

+
+
+ wall a photo saw + claimed, never seen + good wall thrown away
-
-
- - -
-
- Decide this morning -

Eight decisions, with the manager's recommendation

-
-
-

1Run the field test first today?

-
-
Recommend
Yes, first thing this morning, about an hour. Take a current iPhone to a real meter wall and run Measure Lab's one-hour tape protocol (PR #7) plus the field checklist in experiments/evals/README.md section 5 (PR #12): two long taped spans, one taped scale reference, a rig tap at every surveyed point. Follow experiments/evals/field/FIELD_SHEET.md (PR #12): it fixes the tap order so Measure Lab's M1-M10 match the pre-named templates, and you only type tape readings. make field SESSION=… TRUTH=… MAP=… then gives one scored table with the phone's scale error; a dry run on synthetic data recovered a planted 1.5% error exactly.
-
Why
This one number, the phone's scale error, decides how accurate the whole product can be.
-
If not
Every accuracy claim keeps resting on public data from a LiDAR iPhone, checked against a reference that may have been scaled to ARKit itself. A tape test on a current, non-LiDAR iPhone is the only direct measurement.
-
-
-
-

2Measure with AR taps, not depth maps?

-
-
Recommend
Every point a check depends on (window, door and wall edges, the meter, obstacles) is an AR tap the homeowner makes or confirms. Learned depth fills the 3D picture and coverage only.
-
Why
Depth edges were 8 in or worse in every test, and even flat walls only reached about 5 in p90 at a realistic phone error.
-
If not
A clearance check near its threshold passes or fails on depth-map error, which at edges is 8 in or worse.
-
-
-
-

3Walk both ends or stop early?

-
-
Recommend
Walk outward from the meter on both sides, and stop once the nearest spot that fits has all its evidence and everything closer has been seen. Reaching both wall ends is not required.
-
Why
The solver ranks spots by shortest cable run, then by distance from the meter, so once everything closer is seen on both sides, walking farther cannot find a better spot. A shorter walk also drifts less (Finding 1).
-
If not
Walking to both ends costs the homeowner time and adds far-away measurements the answer rarely needs. Stopping at the first spot that fits on one side can miss a closer spot on the other. Either way, unseen ground is never clear: the 05:40 review found one path in #11 where it could still pass; that is fixed and tested now.
-
-
-
-

4Load Base's real rules

-
-
Recommend
Put Base's actual values into private/rules.yaml (git-ignored; the server merges it over the public file), and confirm with Base which rules the demo follows and which identifier on the meter plate they need. Plates often carry several.
-
Why
With the public placeholder rules, automatic approval is off, so every real scan ends in "An installer will take a look", even when all checks pass.
-
If not
The demo can never show a real "fits here" answer.
-
-
-
-

5Merge order

-
-
Recommend
#11, #14, #15, #10, #13, #12, then #7, #4, #8, #6, #16.
-
Why
Server first, because the app, the web page and the docs PR's rules sentence depend on it.
-
If not
Whatever lands before #11 describes or calls a server that main doesn't have yet.
-
-
-
-

6Who owns each lane A-D today?

-
-
Recommend
No recommendation. The team names an owner per lane.
-
Why
Agents need to know whose OK they need before touching a lane's files.
-
If not
Agent work on lane files waits for someone to answer.
-
-
-
-

7Who checks what's hidden behind things?

-
-
Recommend
The app owns it. On every phone, show the homeowner the photo of the chosen spot before the result and ask whether anything blocks the wall or ground there. On LiDAR phones, also run a depth test: against true depth it removed the hidden stretch cleanly (0.1 ft left, no good wall lost, when 0.4 m of wall relief is allowed). Requiring views from different angles doesn't work: it removed the hidden stretch only by also throwing away 6 to 8.5 ft of good wall.
-
Why
Run on real photos of a building with a laser scan, the app's coverage map claimed 1.1 ft of a 19.3 ft wall that no photo saw, hidden behind scaffold footings. The server never sees the photos, so it can't catch this, yet it treats everything the app reports as seen.
-
If not
A gas meter or vent hidden behind a bush could sit inside a clearance the server marks as clear.
-
-
-
-

8What must the scan see, and what can we just ask?

-
-
Recommend
The app measures how far out it really saw the ground and asks only as far as the nearest check needs. The path the homeowner walked counts as proof the space in front of the wall is clear, and one "tilt up" view at the chosen spot covers headroom. A corner the homeowner marks ends the wall instead of becoming a request. Rare, far-away hazards (a pool, a driveway) become one yes/no question, labelled as the homeowner's answer.
-
Why
The verification agent found that today no scan can finish on the first try, however well it's walked: the app always reports ground 4 ft out while the checks need about 6 to 13 ft, it never reports the space in front of or above the wall, and each corner becomes a "walk past this" request. On the real replay that left 8 checks unsure and 6 follow-up requests.
-
If not
Every homeowner gets asked for photos they can't take, the opposite of Sam's first priority.
-
-
-
-
-
- - -
-
- Asks -

Things only a person can do

-
-
-

Review bots

-

Only CodeRabbit reviews this repo, and the free open-source plan limits it to about one review per hour. PR #14's review was skipped for the limit. Greptile, Macroscope, Aikido and Codex review are not installed. Codemetrics is installed but not connected to the team.

-

Ask: install or connect the ones you want. Until then, agents queue CodeRabbit on the most important PRs first (#11, #10).

-
-
-

TestFlight

-

Sam: one-time App Store Connect key setup for PR #8. The steps are in that PR.

-
-
-

Web previews

-

The free Vercel plan allows 100 deploys a day, and every push to any branch was building a web preview, so at 06:10 four PRs showed a red "rate limited" check. Automatic deploys for the web project are now off; the web page's previews are deployed by hand, and the red checks clear on each PR's next push.

-

Ask (Sam): reconnect it after the hackathon (vercel git connect), or move the project to a paid plan if you want previews on every push.

-
-
-

Check the meter labels (anyone, 5 minutes)

-

The meter results rest on labels two AI readers made. Open ~/house-scanning-data/meter-closeup/review.html on Sam's laptop and mark each photo's number keep or fix.

-
-
-
-
- - -
-
- Today -

What the agents keep doing

-
    -
  • Run the app against the live server end to end, re-check the product description against the app, and polish the capture UX.
    • -
  • Field-day kit: the Measure Lab build, the survey sheet and the importer map (evals agent).
    • -
  • Collect and fix bot review findings as PRs leave draft: CodeRabbit now, the others once installed.
    • -
  • More experiments: meter close-up, frame selection, and a current iPhone's scale error once the field data exists.
    • -
-
-
-
+ + A 19.3 foot wall under three versions of the coverage map + Three bars, each 19.3 feet long. First, as claimed: 1.1 feet marked as seen that no photo saw, hidden behind scaffold footings. Second, after requiring views from different angles: the false 1.1 feet is gone but 6 to 8.5 feet of good wall is also thrown away. Third, after a depth test on LiDAR phones: the false part is removed cleanly and the rest is kept. + + as claimed + + + 1.1 ft never seen + views from different angles + + + + 6 to 8.5 ft of good wall lost too + depth test (LiDAR phones) + + + clean + +
Run on real photos of a laser-scanned building. The map claimed 1.1 ft of a 19.3 ft wall that no photo saw, hidden behind scaffold footings. Requiring views from different angles removed it only by also discarding 6 to 8.5 ft of good wall. A depth test, available on LiDAR phones, removed it cleanly. The hatched span is drawn at 8.5 ft, the larger end of the range.
+ +

What it changed: decision 7 (who checks what is hidden). A wrong claim in the app's code, "the server rechecks from the images", was corrected: the server never sees images.

+ + +
+

Finding 4. Can the phone read the meter?

+
+ + Meter reading success on real meter photos + Four horizontal bars on a 0 to 100 percent scale. Read the full number: 71 of 73, 97 percent. Picked the right line: 21 of 75. Barcode held the number: 20 of 24 photos that had one. A tap list of three candidates contained it: 85 percent of held-out photos. + + + + 0255075100% + + full number readright line pickedbarcode had itin the tap list of 3 + + + + + + 71 of 73, 97% + 21 of 75 + 20 of 24 photos that had one + 85% of held-out photos + + +
Apple's on-device text recognition read the full meter number on 71 of 73 real meter photos, but could not tell which line on the plate was the number (right on 21 of 75). A barcode held the number on 20 of 24 photos that had one. A tap-to-confirm list of three candidates contained it on 85% of held-out photos.
+
+

What it changed: the app shows three candidates to tap, barcode match first, plus two photo checks that trigger a retake (blurry; number too small, "move closer").

+
+ +
+

Finding 5. Can a first-time homeowner finish a scan?

+
+ + Ground depth the app reported against what the checks need + A scale from 0 to 14 feet. The app reported ground seen out to 4 feet. The checks need 6 to 13 feet. The two do not overlap, so no scan could finish. + + + + 04812 ft + app reported + + 4 ft of ground seen + checks need + + 6 to 13 ft + +
The verification agent's friction audit: no, not before this morning's fixes. However well it was walked, the app said it had seen ground only 4 ft out while checks need 6 to 13 ft, never reported the space in front of or above the wall, and turned corners into "walk past this" requests.
+
+

What it changed: the capture contract, and the app fixes in section 4: the server publishes exactly what settles each check, and the app reports real ground depth, the walked path as clear space in front, and a tilt-up view for headroom.

+
+ +

Three pieces of overnight work changed nothing in the product: the panel-label search (too few open photos: 7 US panels against 40 needed), finer statistics on the modern-phone data, and the research notes.

+ + +

7. Decisions for this morning, recommendation first

+
    +

  1. Run the field test first

    Recommend: about an hour, before anything else.

    A current non-LiDAR iPhone, Measure Lab, a 30 ft tape, a real meter wall. Follow experiments/evals/field/FIELD_SHEET.md. One command turns tape readings into a score and the phone's real scale error.

    2. +

  2. How points get measured

    Recommend: AR taps, not depth maps.

    Finding 2: learned depth gets edges to 8 in or worse.

    3. +

  3. How far to walk

    Recommend: outward from the meter on both sides; stop when the nearest spot that fits is fully seen.

    The server prefers the shortest cable, so walking farther cannot find a better spot, and shorter walks drift less.

    4. +

  4. Base's real rules

    Recommend: load them into the private rules file now.

    Confirm with Base which rules the demo follows and which number on the meter plate they need (plates carry several). Otherwise the demo can never show a real "fits here".

    5. +

  5. Merge order

    Recommend: #11 server first, since the others depend on its schemas.

    Then #14 docs, #10 app, #13 verification, #12 evals, then #7, #4, #8, #6, #16, #17, #19.

    6. +

  6. Lane owners

    Recommend: name one owner per lane.

    Capture app, geometry, rules, output. Agents then know whose OK they need.

    7. +

  7. Who checks what is hidden behind things?

    Recommend: the app.

    Before the result, show the homeowner the photo of the chosen spot and ask if anything blocks the wall or ground there. On LiDAR phones, also run a depth test (finding 3).

    8. +

  8. What must the scan see, and what can we just ask?

    Recommend: pool and driveway become one yes/no question, labelled as the homeowner's answer.

    Instead of requiring 8 to 13 ft of photographed ground.

    9. +
+ + +

8. Asks: only a person can do these

+
    +
  • Sam: review bots. Only CodeRabbit is installed and it is rate-limited; Greptile, Macroscope, Aikido and Codex review are not installed.
    • +
  • Sam: a one-time App Store Connect key for TestFlight (PR #8).
    • +
  • Sam: web previews on Vercel were turned off because the free plan's 100 deploys a day ran out; reconnect later.
    • +
  • Anyone, 5 minutes: confirm the meter labels on the local review page on Sam's laptop.
    • +
+ + +

9. Reference: what exists, and how to try it

+

Open pull requests. None of these is merged yet. The server (#11) is live.

+
    +
  • #4 scoring harness
    • +
  • #6 research notes
    • +
  • #7 Measure Lab
    • +
  • #8 TestFlight upload
    • +
  • #9 Hunter's camera-metadata probe
    • +
  • #10 iOS app
    • +
  • #11 placement server (live)
    • +
  • #12 evals
    • +
  • #13 independent verification
    • +
  • #14 docs cleanup
    • +
  • #15 web review page (parked)
    • +
  • #16 meter reading
    • +
  • #17 panel labels (blocked on photos)
    • +
  • #18 Hunter's research handoff
    • +
  • #19 this briefing
    • +
+ +

How to try it

+
+

Server health: house-scanning-server.vercel.app/health

+

App in the Simulator: build ios/HouseScan.xcodeproj and run with

+
-replay <folder> -autopilot -serverURL https://house-scanning-server.vercel.app
+

On a real iPhone: set your team in ios/Config/Local.xcconfig (see ios/README.md), plug in, run.

+
+ +
Facts from the team dossier as of 09:10, 26 September 2026. Base's private thresholds do not appear on this page. The drawings are illustrations at a consistent scale, not a real house.
- +