[AidenJohnston]

Problem. On "See it on your wall" the battery was drawn over a person standing at the wall (field run 2, 4:14) and through the wooden post at the end of the mock wall (4:17). The result was a SwiftUI Canvas (BatteryOverlay) painted on top of the camera view, so nothing in the real scene could ever cover it. The session already had the LiDAR mesh and scene depth, but nothing used them for occlusion.

Fix.

• On the live camera, the battery, the cable run and the clearance zones are now RealityKit content in the ARView that shows the camera (ResultARModel in ResultScene3D.swift, same colors and battery model as the 3D result view). They hang on an AnchorEntity for the meter's ARAnchor, so they follow ARKit's corrections to that anchor as before. World alignment stays .gravity.
• While the AR screen is up, LiveCapture turns on sceneUnderstanding .occlusion when the phone builds a mesh (LiDAR), and .personSegmentationWithDepth when supportsFrameSemantics says the phone can. The configuration is re-run without reset options, so tracking, anchors and the mesh carry on. Both are turned off again when the screen closes, so the capture runs exactly as before. A phone without LiDAR gets people occlusion only; a phone with neither gets the unoccluded model.
• The LiDAR mesh sits a centimeter or two off the real wall and ground, and it would cut into anything drawn flush with them. The battery's back, the cable and the clearance zones therefore stay at least 3 cm off the wall and ground (ResultARModel.meshClearance; the cable was already drawn 3 cm out).
• The battery casts a grounding shadow and still rises out of the ground on entrance (shorter with Reduce Motion). It is hidden while tracking isn't normal, as the overlay was (R-5), and rebuilt if the wall frame changes while the screen is open.
• A replay, or a meter whose anchor isn't in the session, keeps the old Canvas overlay. ScanViewState.resultInCamera tells the screen which one is in use.

Check on a device. CI can build this but can't see rendering, so it needs checking in the next field test. Open "See it on your wall" and confirm:

• the battery sits in the same place as before, beside the meter and against the wall, and doesn't sink into the wall or the ground;
• a person walking between the phone and the battery covers it, with a clean edge around them;
• from beside the end of the wall, a post or other object in front of the battery hides the part behind it (LiDAR phone);
• the battery disappears while the phone relocalizes and comes back in place;
• after "Done", a follow-up capture from the result still works normally.

Worth knowing.

• On uneven ground (grass, gravel) the mesh can still rise above the clearance zones or the battery's base and hide a sliver of them. If that shows up in the field, raise meshClearance.
• The dashed outlines the Canvas drew around each clearance zone aren't in the 3D version; the zones are tinted fills, as in the 3D result view.

Fixes #41. This branch merges into t3/ios-mvf, so GitHub won't close the issue on merge; close it by hand.

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 0088fa60-58e8-4bf4-aca0-12998752f444

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codemetrics-ai]

Note

Review skipped

Codemetrics is installed on this repository, but it hasn't been added to your team yet.

To enable AI code reviews for this repo, go to your dashboard and add house-scanning-master to your team's repositories.

[SamGu-NRX]

I read 2d33cea against the engine and capture. Several things hold up:

• The model hangs on the meter's own ARKit anchor, and world alignment stays .gravity.
• The model is rebuilt from publishWall, so an anchor or wall correction re-reads the anchor's rotation instead of keeping a stale inverse.
• People occlusion is switched on only while the result shows, by re-running the same configuration without reset options, so tracking and the map carry over.
• ResultARScreen changes by two lines, which keeps it clear of ios: live 3D map as the coverage model where depth exists #21.

Two points:

1. The clearance zones may vanish on grass. The zones are planes 3 cm above the wall frame's ground (meshClearance). With .occlusion on, the reconstructed ground mesh hides anything below it. On a lawn or gravel that mesh often sits several centimetres above the ground the wall frame uses, so the zones would be hidden or flicker in patches. On the Canvas they always showed. The zones are what explain a pass or fail on this screen, so they need to stay visible.
   • Keep the zones out of mesh occlusion. People occlusion is fine for them. One way: draw them in the Canvas overlay as before, and move only the battery and cable into RealityKit. Another: raise the zones and fade them into the mesh.
   • Please try it on the mock wall's lawn in the next phone run.
2. Only the phone can test this path. Replays and CI keep resultInCamera false and draw the Canvas, so no automated check reaches showResult, setOcclusion or ResultARModel. Add the phone check to the PR description:
   • a person steps in front of the battery;
   • the post hides it;
   • tracking is lost and comes back;
   • "Done", then "See it on your wall" again.

[AidenJohnston]

Challenger review. The fix is sound. The model hangs on an AnchorEntity for the meter's ARAnchor, and simd_quatf(anchor.transform).inverse puts it back into world axes, so placement stays relative to the anchor. World alignment stays .gravity. Leaving the screen goes through setPhase and turns occlusion off again. A replay, or a meter with no anchor, falls back to the Canvas overlay. No project files changed, so the drift check isn't affected. Three nits:

1. nit: ios/HouseScan/Runtime/LiveCapture.swift:53 and :80. People occlusion is checked by itself, but it is enabled alongside .sceneDepth. supportsFrameSemantics should be asked about the set that will actually run. If a device supports each semantic alone but not both together, the session fails on the one screen this PR is meant to fix. Fix: add if occludingPeople, ARWorldTrackingConfiguration.supportsFrameSemantics(configuration.frameSemantics.union(.personSegmentationWithDepth)) { ... }.
2. nit: ios/HouseScan/Runtime/ScanEngine.swift:1147. publishWall rebuilds the whole model every time refreshMeterFromAnchor sees the anchor move more than 2 cm, or the ground estimate moves more than 1 cm. The AnchorEntity already follows the anchor, so these rebuilds are redundant. A rebuild in the first 0.7 s also cancels the rise animation and makes the battery pop into place. Fix: rebuild only when geometry relative to the meter changes, such as groundY - meter.y, along/outward or cornerSegments. Compare against the last wall you built from.
3. nit: this conflicts with #53 in ScanEngine.publishWall, because both PRs append a line after the WallGeometry(...) initializer. Resolve it by keeping both lines, with publishFeaturesPastEnds() first.

The device checks in the description are the right ones. Please also try a non-LiDAR phone, which should get people occlusion only.

[SamGu-NRX]

Read-only follow-up at 2d33cea: the meter-anchor transform, .gravity alignment and result-screen lifecycle read correctly; CI is still running, and its replay flow does not execute this live AR path. One crash risk is worth fixing before the phone run: LiveCapture.swift:53,77-80 checks .personSegmentationWithDepth by itself, then adds it to .sceneDepth on LiDAR phones. The iOS SDK says supportsFrameSemantics must check the entire requested option set and an unsupported combination throws on session run. Check configuration.frameSemantics.union(.personSegmentationWithDepth) before insertion. The ResultScene3D.swift:319-321 comment and PR description also overstate an unmeasured mesh offset and "at least 3 cm" clearance: the battery base is at ground height, zones begin at the wall, and a 1.5 cm-radius cable centred 3 cm out has only 1.5 cm surface clearance. Narrow those claims. On device, verify zones remain visible on grass and with estimated ground, opening during relocalization still grows the battery fully, the shadow appears, and reopening after Done works. No review-clear gate yet.

[SamGu-NRX]

@AidenJohnston Follow-up after merge at 63d950d: please check LiveCapture.configuration against the combined .sceneDepth and .personSegmentationWithDepth frame semantics before rerunning the AR session; the SDK warns an unsupported combination throws. The merged branch checks only person segmentation alone. Also narrow the unmeasured mesh-offset/"at least 3 cm" claims in ResultScene3D.swift and this PR description; the battery base and zone wall edge are flush, while the cable surface is 1.5 cm out. CI replay tests do not run the live AR path, so keep the person/post occlusion, relocalization, shadow, grass-zone and reopen checks for the next phone build. These findings were open when this PR merged; the merge did not clear them.

[SamGu-NRX]

@AidenJohnston Linking the existing post-merge follow-up to #67: build 4.1 at 45017ba651205e9b3ea51a8faecdfc8d9108a759 now has reported physical-device failures in both iPhone 14 runs that opened AR, including run 2 with correct ground height. PR #60 at c566490c5e3fa0d45803895bb463cb260d11a906 records A41-1-02 and A41-2015-01. I inspected those reports and the pinned source, not the recordings directly. This remains an acceptance blocker even though replay CI passed.

Confirmed source: LiveCapture.swift:130-140 returns true after adding the anchor and requesting occlusion, without checking resolved anchoring or visible output; ResultARScreen.swift:17 then suppresses Canvas. The failing step is still unknown. One hypothesis needs narrowing: .idle only disables image encoding at LiveCapture.swift:468-474; pose/snapshot delivery continues at :396-440, and ScanEngine.swift:334-338 applies tracking changes before the pose-only return. Idle alone therefore does not explain lost visibility updates. Please use #67 for the diagnosis/fix and coordinate through S3; no duplicate issue is needed.

For acceptance, record a non-LiDAR phone opening the result with battery, cable and tint visible, then tracking recovery and Done/reopen. Keep #73 drift verification separate: visible output does not establish correct placement. The prior combined-semantics check also remains a separate concern; these non-LiDAR observations do not demonstrate a sceneDepth/person-segmentation combination failure.