diff --git a/experiments/device-field-test/README.md b/experiments/device-field-test/README.md index ee3f26208..fbd383786 100644 --- a/experiments/device-field-test/README.md +++ b/experiments/device-field-test/README.md @@ -163,6 +163,137 @@ Other things the recording showed: has no battery kind. - **A tilt-prompt distance went backwards**, from 3 ft 3 in to 2 ft 9 in. +## Result, run 2 (build 3.1) + +Run on 2026-09-26 with TestFlight build 3.1 (workflow run +[36272848774](https://github.com/SamGu-NRX/house-scanning-master/actions/runs/36272848774), branch `beta` at +[`ad91620`](https://github.com/SamGu-NRX/house-scanning-master/commit/ad91620f41686d18f8699fbc1040d2a754719163), +capture code `t3/ios-mvf` at `e9e405d`, server at `e106acb`). The 4 min 28 s screen recording, run A only, was reviewed +at 1 fps and at 5 fps around each tap. Its narration was transcribed on the tester's laptop. The exported scan was +lost before it was copied off the phone, so: +- the packet wasn't validated; +- the coverage wasn't replayed; +- the scene wasn't sent to the server again. + +The numbers below come from the screen and from the code. This repeats the runbook's run 1 (full flow) on the new +build; the runbook's runs 2 and 3 are still to do (#32). + +**The first criterion fails again, but the run reached a spot for the first time.** +- **What the phone did.** It walked about 17 ft right of the meter to the end of the mock wall. The photo counter + reached 136 on the walk and 145 after one server request. +- **What the scan said.** The wall ran from 7 ft 6 in left of the meter to 0 ft 6 in right. The window and the AC + stand-in lay outside it. +- **What the server answered.** `manual_review`, "An installer will take a look / 1 ft 3 in left of your meter, 1 ft of + cable", with 10 checks UNSURE. + +### What the recording shows + +"Can't get there" still sets the end at the meter: +- **00:53, left.** The ground cell just left of the meter was still amber (seen once). The left end landed at the + meter. +- **02:38.** "Can't get there" on "Tilt up to show more wall / Around 16 ft 9 in right of your meter" hatched that + stretch "Skipped". This is intended. +- **02:40, right.** At the end of the wall ("can't get there because I'm at this wall"), only one 6-inch cell next to + the meter was covered. The right end landed at 0 ft 6 in. The strip went blank for the rest of the walk (#24). + +The walk then kept asking for stretches past that end: "Tilt up to show more wall / Around 7 ft right of your meter" +and "Tilt down to show the ground / The strip along the wall, 5 ft 3 in right of your meter". None of them could fill. +After 57 s of "I don't know where it wants me to point", the tester tapped "Done with this wall" (#38). + +The first upload took about 0.7 s and returned "2 more views to finish". +- **The first view.** "Show the ground around your meter / Keep walking past the left end of the scan (0 ft 4 in left of + the meter)". Its bar stayed at 0%. +- **03:56.** "I can't get there" uploaded again, and the result came about 1.2 s later. The second view was never + shown (#39). +- **The ends had moved.** The result now asked to "Keep walking past the left end of the scan (7 ft 6 in left of the + meter)" and "…the right end of the scan (0 ft 6 in right of the meter)" (#35). + +The spot's footprint runs from 2 ft 7 in left of the meter to the meter itself. Its check line read "Clear of the +meter's working space … Measured 1 ft 3 in. The rule is 0 in, and the measurement can be off by about 1 ft 6 in." That +1 ft 3 in is an **overlap** with the 2 ft 6 in by 3 ft working space; the app drops the minus sign (#40). The overlap is +UNSURE rather than FAIL only because the meter's error, 0.79 ft, plus the wall's, 0.71 ft, exceeds it (#44). + +The AR view, "The spot an installer will check", stayed in one place relative to the meter for the 11 s the tester +walked around. But: +- it was drawn over a person standing at the wall (#41); +- it was drawn through the post at the wall's end; +- its ground tint ran past the end of the wall. + +The tester called it "very not correct". + +### How the walk was lost, and how the wall grew + +The walk-end code is unchanged since run 1: +- "Can't get there" sets the end at `GuidancePlanner.reach` + ([`ScanEngine+Actions.swift` L354–359](https://github.com/SamGu-NRX/house-scanning-master/blob/ad91620f41686d18f8699fbc1040d2a754719163/ios/HouseScan/Runtime/ScanEngine%2BActions.swift#L354-L359)). +- `reach` stops at the first cell next to the meter that isn't covered + ([`GuidancePlanner.swift` L102–118](https://github.com/SamGu-NRX/house-scanning-master/blob/ad91620f41686d18f8699fbc1040d2a754719163/ios/HouseScanKit/Sources/HouseScanKit/Guidance/GuidancePlanner.swift#L102-L118)). + +New in this run: +- **The past-end request cleared the left end.** `beginServerGap` clears the end it asks past + ([`ScanEngine.swift` L1400–1411](https://github.com/SamGu-NRX/house-scanning-master/blob/ad91620f41686d18f8699fbc1040d2a754719163/ios/HouseScan/Runtime/ScanEngine.swift#L1400-L1411)). +- **"I can't get there" didn't set it again.** `skipCurrentGap` + ([L1290–1302](https://github.com/SamGu-NRX/house-scanning-master/blob/ad91620f41686d18f8699fbc1040d2a754719163/ios/HouseScan/Runtime/ScanEngine.swift#L1290-L1302)) + marks the requested cells skipped on the phone and stops further requests. It never restores the end. +- **The export then ran to the farthest cell seen from one position.** With no left end, `exportSpan` uses + `seenExtent` + ([`ScanEngine+Export.swift` L128–133](https://github.com/SamGu-NRX/house-scanning-master/blob/ad91620f41686d18f8699fbc1040d2a754719163/ios/HouseScan/Runtime/ScanEngine%2BExport.swift#L128-L133)). + That was 15 cells, 7 ft 6 in. +- **Guidance ignores the ends.** It asks for stretches past an end because `laggingBand` and `hiddenNearCamera` read + coverage around the camera without checking them + ([`GuidancePlanner.swift` L143–166](https://github.com/SamGu-NRX/house-scanning-master/blob/ad91620f41686d18f8699fbc1040d2a754719163/ios/HouseScanKit/Sources/HouseScanKit/Guidance/GuidancePlanner.swift#L143-L166)). + Cells past an end never count as covered and can't be skipped. + +The server picked the spot because it ranks UNSURE candidates by the number of UNSURE checks, then cable length, then +position +([`solver.py` L1166–1167](https://github.com/SamGu-NRX/house-scanning-master/blob/ad91620f41686d18f8699fbc1040d2a754719163/server/solver.py#L1166-L1167)). +A 0 ft route that touches the meter wins ties (#44). + +None of this changes in `b7f51ba` or `49a676e` on `t3/ios-mvf`. + +### Measurements + +No tape distances were read aloud, so there is no tape comparison. The app's positions, from the meter's centre line: + +| Feature | App | +|---|---| +| AC stand-in | "About 16 ft right" (centre about 16.0 ft from the server's numbers) | +| Window | "About 7 ft right", near edge about 5.9 ft right | +| Spot | From 2 ft 7 in left to 0 ft | + +### Device-only claims + +The tester said the build and run, and "no haptic" once, but no test IDs. The other haptic claims remain unverified. + +| ID | Result | Evidence | +|---|---|---| +| O-1 | Not reached | No camera prompt; permission was granted earlier. A new Motion & Fitness prompt appeared on "Find your electric meter" (#43) | +| F-2 | Unclear | Early taps were narrated, but no "Step a little closer to the wall" appeared | +| C-1 | Pass | The ring filled in about a second; "Meter number saved" followed | +| W-1 | Pass, confusing | All instructions appeared. After the right end was set, "Around 7 ft right" requests couldn't be met | +| W-2 | Pass | The counter flashes; "there is no haptic" | +| W-5 | Fail, B-16 | "Tap the ac unit" (#25) | +| W-7 | Not reached | The wall ends before 20 ft | +| W-8 | Fail, B-06 | The wall ended at about 17 ft and only "Can't get there" was offered (#29) | +| G-2 | Pass, copy bug | "I can't get there" uploaded at once, but "2 more views to finish" showed only one (#39) | +| U-1 | Measured | 0.7 s for the first upload, 1.2 s after the gap request | +| U-2 | Fail, known | The final upload stayed on "Sending measurements, 99%"; "Check clearances" never ticked (#31) | +| R-1 | Pass (haptic unverified) | First result with a spot: "An installer will take a look", 13 check lines | +| R-3 | Stayed put, wrong spot | Steady relative to the meter; drawn through the post at the wall's end | +| R-4 | Fail, known | The battery is drawn over a person (#41) | + +Not tried: W-4, W-9 in the dark, M-2, M-3, R-2, R-5, and runs 2 and 3. The 3.1 fixes for tracking loss on the feature +list and during gap requests are not yet checked on a device. + +Other things the recording showed: +- **Marks outside the ends still count.** The window and AC are listed and exported, and the server measures them on + the wall's straight extension (#42). +- **"Slow down" during tilts and while standing still,** seven times. The gate fires above 60°/s of rotation as well + as 1.5 m/s (#26). +- **No way to mark a battery** or other equipment on the wall (#27). +- **The tilt-prompt distance went backwards again,** from 5 ft 3 in to 4 ft 9 in, 4 ft 6 in and then 4 ft, never in + front of the meter (#28). + ## What this changes - **For the capture app (lane A owner's call):** "Can't get there" shouldn't be able to place an end @@ -177,6 +308,7 @@ Other things the recording showed: front of the meter. That's the stretch `reach` needs first. Asking for it before "Walk slowly to your left" would unblock `reach` early. The strip could also show that an end will land at the meter before the tap. - **For the next runs:** the runbook now says to fill the strip next to the meter before any "Can't get - there". Testers should say test IDs and haptics aloud. Runs 2 and 3 are still to do. + there". Testers should say test IDs and haptics aloud, and share the scan off the phone before closing the app: a + relaunch deletes it. Runs 2 and 3 are still to do. Raw material stays out of git: the recording, the scan and the tape readings.