AR overlays drift off their objects after walking away (window outline drawn over the panel) #73

After walking away and back, the AR overlays drift off the objects they mark: the window's outline was drawn over the panel and disconnect boxes, 2–3 ft left of the window.

Seen in. Device field test of TestFlight build 4.1 (beta at 45017ba), iPhone 14 without LiDAR, iOS 26.6.2. Run 2 (indoor mock wall), video 4:14–4:28, during the server's ground request, after the tester had walked about 5 m away from the wall and back. Report: run 2, A41-2015-05.

What happens.

• The window's dashed outline was drawn over the electrical panel and disconnect boxes, about 2–3 ft left of the window. At 4:20 it partly overlapped the window's left side.
• Tester: "The window's not on the window anymore … Now it's just a floating window. That's bad."
• Tracking was normal throughout, with no pose jumps, so this isn't a relocalization snap.
• The window mark itself (5.43–7.33 ft right) agrees with run 1's window (5.71–7.96 ft) to within about 0.5 ft, so the taps were roughly right when made.

Why it matters. The AR result ("See it on your wall") is drawn the same way, after the same walk. Once it renders (see the issue "See it on your wall" draws nothing on phones without LiDAR (regression from #41)), the battery could be feet off.

Cause (inferred, not confirmed). Either visual-inertial drift over 42 m of walking without LiDAR, or the overlays not following the meter anchor:

• Marked features keep their tapped world points. When ARKit moves the meter anchor, only the meter position is updated and features are re-projected from those fixed world points (ScanEngine.swift#L381-L402, ScanEngine+Actions.swift#L272-L281). So a window outline stays where it was in raw world coordinates, not where it is relative to the meter's anchor. The repo's rule is to place AR content relative to the meter's anchor.
• The anchor update takes the anchor's position only, not its rotation, so the wall's direction never follows ARKit's corrections.
• The outline is then drawn from the wall frame and the re-projected span (WallMarksOverlay.swift#L45-L54).

Fix direction.

• Log the meter anchor's transform over time and how far it has moved since the tap, to tell drift from a placement bug.
• Store features relative to the meter anchor (or give each its own ARAnchor), and re-derive overlay positions from the anchors every frame, rotation included.
• On the result screen, ask the homeowner to aim at the meter again first, re-hit-test the meter plane and correct the offset before drawing the battery.
• Test on a non-LiDAR phone.

Related. A41-2015-05, #41.

Screen recordings are in the comment below.