Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
134 changes: 133 additions & 1 deletion experiments/device-field-test/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,137 @@ Other things the recording showed:
has no battery kind.
- **A tilt-prompt distance went backwards**, from 3 ft 3 in to 2 ft 9 in.

## Result, run 2 (build 3.1)

Run on 2026-09-26 with TestFlight build 3.1 (workflow run
[36272848774](https://github.com/SamGu-NRX/house-scanning-master/actions/runs/36272848774), branch `beta` at
[`ad91620`](https://github.com/SamGu-NRX/house-scanning-master/commit/ad91620f41686d18f8699fbc1040d2a754719163),
capture code `t3/ios-mvf` at `e9e405d`, server at `e106acb`). The 4 min 28 s screen recording, run A only, was reviewed
at 1 fps and at 5 fps around each tap. Its narration was transcribed on the tester's laptop. The exported scan was
lost before it was copied off the phone, so:
- the packet wasn't validated;
- the coverage wasn't replayed;
- the scene wasn't sent to the server again.

The numbers below come from the screen and from the code. This repeats the runbook's run 1 (full flow) on the new
build; the runbook's runs 2 and 3 are still to do (#32).

**The first criterion fails again, but the run reached a spot for the first time.**
- **What the phone did.** It walked about 17 ft right of the meter to the end of the mock wall. The photo counter
reached 136 on the walk and 145 after one server request.
- **What the scan said.** The wall ran from 7 ft 6 in left of the meter to 0 ft 6 in right. The window and the AC
stand-in lay outside it.
- **What the server answered.** `manual_review`, "An installer will take a look / 1 ft 3 in left of your meter, 1 ft of
cable", with 10 checks UNSURE.

### What the recording shows

"Can't get there" still sets the end at the meter:
- **00:53, left.** The ground cell just left of the meter was still amber (seen once). The left end landed at the
meter.
- **02:38.** "Can't get there" on "Tilt up to show more wall / Around 16 ft 9 in right of your meter" hatched that
stretch "Skipped". This is intended.
- **02:40, right.** At the end of the wall ("can't get there because I'm at this wall"), only one 6-inch cell next to
the meter was covered. The right end landed at 0 ft 6 in. The strip went blank for the rest of the walk (#24).

The walk then kept asking for stretches past that end: "Tilt up to show more wall / Around 7 ft right of your meter"
and "Tilt down to show the ground / The strip along the wall, 5 ft 3 in right of your meter". None of them could fill.
After 57 s of "I don't know where it wants me to point", the tester tapped "Done with this wall" (#38).

The first upload took about 0.7 s and returned "2 more views to finish".
- **The first view.** "Show the ground around your meter / Keep walking past the left end of the scan (0 ft 4 in left of
the meter)". Its bar stayed at 0%.
- **03:56.** "I can't get there" uploaded again, and the result came about 1.2 s later. The second view was never
shown (#39).
- **The ends had moved.** The result now asked to "Keep walking past the left end of the scan (7 ft 6 in left of the
meter)" and "…the right end of the scan (0 ft 6 in right of the meter)" (#35).

The spot's footprint runs from 2 ft 7 in left of the meter to the meter itself. Its check line read "Clear of the
meter's working space … Measured 1 ft 3 in. The rule is 0 in, and the measurement can be off by about 1 ft 6 in." That
1 ft 3 in is an **overlap** with the 2 ft 6 in by 3 ft working space; the app drops the minus sign (#40). The overlap is
UNSURE rather than FAIL only because the meter's error, 0.79 ft, plus the wall's, 0.71 ft, exceeds it (#44).

The AR view, "The spot an installer will check", stayed in one place relative to the meter for the 11 s the tester
walked around. But:
- it was drawn over a person standing at the wall (#41);
- it was drawn through the post at the wall's end;
- its ground tint ran past the end of the wall.

The tester called it "very not correct".

### How the walk was lost, and how the wall grew

The walk-end code is unchanged since run 1:
- "Can't get there" sets the end at `GuidancePlanner.reach`
([`ScanEngine+Actions.swift` L354–359](https://github.com/SamGu-NRX/house-scanning-master/blob/ad91620f41686d18f8699fbc1040d2a754719163/ios/HouseScan/Runtime/ScanEngine%2BActions.swift#L354-L359)).
- `reach` stops at the first cell next to the meter that isn't covered
([`GuidancePlanner.swift` L102–118](https://github.com/SamGu-NRX/house-scanning-master/blob/ad91620f41686d18f8699fbc1040d2a754719163/ios/HouseScanKit/Sources/HouseScanKit/Guidance/GuidancePlanner.swift#L102-L118)).

New in this run:
- **The past-end request cleared the left end.** `beginServerGap` clears the end it asks past
([`ScanEngine.swift` L1400–1411](https://github.com/SamGu-NRX/house-scanning-master/blob/ad91620f41686d18f8699fbc1040d2a754719163/ios/HouseScan/Runtime/ScanEngine.swift#L1400-L1411)).
- **"I can't get there" didn't set it again.** `skipCurrentGap`
([L1290–1302](https://github.com/SamGu-NRX/house-scanning-master/blob/ad91620f41686d18f8699fbc1040d2a754719163/ios/HouseScan/Runtime/ScanEngine.swift#L1290-L1302))
marks the requested cells skipped on the phone and stops further requests. It never restores the end.
- **The export then ran to the farthest cell seen from one position.** With no left end, `exportSpan` uses
`seenExtent`
([`ScanEngine+Export.swift` L128–133](https://github.com/SamGu-NRX/house-scanning-master/blob/ad91620f41686d18f8699fbc1040d2a754719163/ios/HouseScan/Runtime/ScanEngine%2BExport.swift#L128-L133)).
That was 15 cells, 7 ft 6 in.
- **Guidance ignores the ends.** It asks for stretches past an end because `laggingBand` and `hiddenNearCamera` read
coverage around the camera without checking them
([`GuidancePlanner.swift` L143–166](https://github.com/SamGu-NRX/house-scanning-master/blob/ad91620f41686d18f8699fbc1040d2a754719163/ios/HouseScanKit/Sources/HouseScanKit/Guidance/GuidancePlanner.swift#L143-L166)).
Cells past an end never count as covered and can't be skipped.

The server picked the spot because it ranks UNSURE candidates by the number of UNSURE checks, then cable length, then
position
([`solver.py` L1166–1167](https://github.com/SamGu-NRX/house-scanning-master/blob/ad91620f41686d18f8699fbc1040d2a754719163/server/solver.py#L1166-L1167)).
A 0 ft route that touches the meter wins ties (#44).

None of this changes in `b7f51ba` or `49a676e` on `t3/ios-mvf`.

### Measurements

No tape distances were read aloud, so there is no tape comparison. The app's positions, from the meter's centre line:

| Feature | App |
|---|---|
| AC stand-in | "About 16 ft right" (centre about 16.0 ft from the server's numbers) |
| Window | "About 7 ft right", near edge about 5.9 ft right |
| Spot | From 2 ft 7 in left to 0 ft |

### Device-only claims

The tester said the build and run, and "no haptic" once, but no test IDs. The other haptic claims remain unverified.

| ID | Result | Evidence |
|---|---|---|
| O-1 | Not reached | No camera prompt; permission was granted earlier. A new Motion & Fitness prompt appeared on "Find your electric meter" (#43) |
| F-2 | Unclear | Early taps were narrated, but no "Step a little closer to the wall" appeared |
| C-1 | Pass | The ring filled in about a second; "Meter number saved" followed |
| W-1 | Pass, confusing | All instructions appeared. After the right end was set, "Around 7 ft right" requests couldn't be met |
| W-2 | Pass | The counter flashes; "there is no haptic" |
| W-5 | Fail, B-16 | "Tap the ac unit" (#25) |
| W-7 | Not reached | The wall ends before 20 ft |
| W-8 | Fail, B-06 | The wall ended at about 17 ft and only "Can't get there" was offered (#29) |
| G-2 | Pass, copy bug | "I can't get there" uploaded at once, but "2 more views to finish" showed only one (#39) |
| U-1 | Measured | 0.7 s for the first upload, 1.2 s after the gap request |
| U-2 | Fail, known | The final upload stayed on "Sending measurements, 99%"; "Check clearances" never ticked (#31) |
| R-1 | Pass (haptic unverified) | First result with a spot: "An installer will take a look", 13 check lines |
Comment thread
SamGu-NRX marked this conversation as resolved.
Comment thread
SamGu-NRX marked this conversation as resolved.
| R-3 | Stayed put, wrong spot | Steady relative to the meter; drawn through the post at the wall's end |
| R-4 | Fail, known | The battery is drawn over a person (#41) |

Not tried: W-4, W-9 in the dark, M-2, M-3, R-2, R-5, and runs 2 and 3. The 3.1 fixes for tracking loss on the feature
list and during gap requests are not yet checked on a device.

Other things the recording showed:
- **Marks outside the ends still count.** The window and AC are listed and exported, and the server measures them on
the wall's straight extension (#42).
- **"Slow down" during tilts and while standing still,** seven times. The gate fires above 60°/s of rotation as well
as 1.5 m/s (#26).
- **No way to mark a battery** or other equipment on the wall (#27).
- **The tilt-prompt distance went backwards again,** from 5 ft 3 in to 4 ft 9 in, 4 ft 6 in and then 4 ft, never in
front of the meter (#28).

## What this changes

- **For the capture app (lane A owner's call):** "Can't get there" shouldn't be able to place an end
Expand All @@ -177,6 +308,7 @@ Other things the recording showed:
front of the meter. That's the stretch `reach` needs first. Asking for it before "Walk slowly to your left"
would unblock `reach` early. The strip could also show that an end will land at the meter before the tap.
- **For the next runs:** the runbook now says to fill the strip next to the meter before any "Can't get
there". Testers should say test IDs and haptics aloud. Runs 2 and 3 are still to do.
there". Testers should say test IDs and haptics aloud, and share the scan off the phone before closing the app: a
relaunch deletes it. Runs 2 and 3 are still to do.
Comment thread
SamGu-NRX marked this conversation as resolved.

Raw material stays out of git: the recording, the scan and the tape readings.