Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
fa19e6d
ios: log why Vision failed before reading a close-up as no number
SamGu-NRX Sep 30, 2026
52ca944
Merge #186 (codex/public-vision-log-20260930 at fa19e6de) into the as…
SamGu-NRX Sep 30, 2026
50e1069
ios: keep AX5 wall-walk actions, aim ring, page dots and badges readable
SamGu-NRX Sep 30, 2026
dcaaf7b
ios: a ground change after a first upload is packaged withdraws its a…
SamGu-NRX Sep 30, 2026
1f7e076
ios: a ground change on a failed or refused first upload keeps the fa…
SamGu-NRX Sep 30, 2026
5484ac3
ios: lay out the capped card in one pass and fix review findings
SamGu-NRX Sep 30, 2026
bfc36e1
ios: square the wrapped sample-result badge at accessibility sizes
SamGu-NRX Sep 30, 2026
33c43fd
ios: hide a ring that can't fit, and prove scrolled ends in AX5 tests
SamGu-NRX Sep 30, 2026
40d7b18
ios: show the card's scroll bar only while scrolling; fix onboarding …
SamGu-NRX Sep 30, 2026
e9f8215
ios: drop the capped card; keep the aim ring out from under the chrome
SamGu-NRX Sep 30, 2026
bec7935
ios: fit the aim ring to its clearance and keep the arrow short of th…
SamGu-NRX Sep 30, 2026
22f7801
ios: draw nothing for an aim target under the card
SamGu-NRX Sep 30, 2026
e297d38
test(ios): give frameless audit findings the edge scroll located ones…
SamGu-NRX Sep 30, 2026
455749d
Revert "test(ios): give frameless audit findings the edge scroll loca…
SamGu-NRX Sep 30, 2026
545448e
ios: drop the AX5 open-camera minimum
SamGu-NRX Sep 30, 2026
30e791a
ios: fold the aim card's how-to words under Details at AX5 (B-36)
SamGu-NRX Oct 1, 2026
e06cec2
test(ios): temporary native contrast audit on AX5 walking states
SamGu-NRX Oct 1, 2026
0228e08
ios: read the aim arrow's progress; wait for a laid-out frame before …
SamGu-NRX Oct 1, 2026
1237ac9
ios: keep action words unfolded at AX5; harden the tap helper
SamGu-NRX Oct 1, 2026
35cc852
ios: fold only a second line that explains
SamGu-NRX Oct 1, 2026
daaafe7
Merge #190 (codex/public-ax5-20260930 at 35cc8521) into the assembly …
SamGu-NRX Oct 1, 2026
57fd4e7
ios: the instruction card snaps to its new height under Reduce Motion…
SamGu-NRX Oct 1, 2026
3a7a29e
ios: declare the required-reason APIs the app calls in a privacy mani…
SamGu-NRX Oct 1, 2026
31d4a34
ios: the privacy manifest check finds C calls with spaces, attributes…
SamGu-NRX Oct 1, 2026
d88f762
ios: a ground change on a failed or refused upload leaves that screen…
SamGu-NRX Oct 1, 2026
00644d8
ios: the ground freshness rule's summary names the packaged first upl…
SamGu-NRX Oct 1, 2026
c23c26d
ci: give the iOS job 75 minutes for the combined suite and its every-…
SamGu-NRX Oct 1, 2026
e292494
ios: at the largest text sizes the see-behind card leads with what to…
SamGu-NRX Oct 1, 2026
efb7ab6
ios: coaching keeps the see-behind card's folded words; its AX5 test …
SamGu-NRX Oct 1, 2026
b6ee153
ios: the folded see-behind card leads with "Look around it"
SamGu-NRX Oct 1, 2026
59ff63a
ios: at the largest text sizes the find-meter card folds and the reti…
SamGu-NRX Oct 1, 2026
97e6c05
ios: the folded find-meter aim is a tested pure rule
SamGu-NRX Oct 1, 2026
0d4512b
ios: the find-meter button drops its pin at the largest text sizes
SamGu-NRX Oct 1, 2026
2695268
ios: the find-meter aim accepts the chrome's zero-width spacer; the p…
SamGu-NRX Oct 1, 2026
4f587f9
ios: the privacy scan reads raw and multi-line strings by their own d…
SamGu-NRX Oct 1, 2026
693271e
Merge #188 (codex/public-coverage-20260930 at 1fa7e23c) into the asse…
SamGu-NRX Oct 1, 2026
a9695be
ios: clear the live dots on a spatial reset (#195)
SamGu-NRX Oct 2, 2026
f9f5935
ios: aiming builds no walking-speed spell, and the dark note says wha…
AidenJohnston Sep 27, 2026
68a509e
ios: an answer that settles the walk's task keeps what stalled earlie…
SamGu-NRX Oct 2, 2026
d837c4e
ios: a request stays available whenever progress still calls its evid…
SamGu-NRX Oct 2, 2026
bd6b933
ios: same-view compares requested spans by exact containment; toleran…
SamGu-NRX Oct 2, 2026
737d921
ios: an upload sends one captured scene and packet, measured against …
SamGu-NRX Oct 2, 2026
b926c92
ios: coverage rebuilds reuse each kept frame's views while their inpu…
SamGu-NRX Oct 2, 2026
0b35f65
ios: the refused-upload test lets the demo finish its gap; skipping i…
SamGu-NRX Oct 2, 2026
7c617dc
ios: a placement answer is shown only when it names the scene that wa…
SamGu-NRX Oct 2, 2026
8c47d67
ios: results say they need an installer's review, never that one is c…
SamGu-NRX Oct 2, 2026
032eea5
ios: say why House Scan asks for local network access
SamGu-NRX Oct 2, 2026
c73aa46
ios: the on-site footnote names no spot; the unsure note is checked i…
SamGu-NRX Oct 2, 2026
6ec6912
ios: a refused placement answer's log line carries no scene hash
SamGu-NRX Oct 2, 2026
94f18ff
ios: the frozen review-sample test drops its headline check (#199)
SamGu-NRX Oct 2, 2026
b3c117d
test(ios): check geometry before tap hit points
SamGu-NRX Oct 2, 2026
ef1a4cf
Merge #198 (codex/result-input-binding at 6ec69122) into the assembly…
SamGu-NRX Oct 2, 2026
8fc2158
Merge #199 (fix/result-installer-review at 94f18ff7) into the assembl…
SamGu-NRX Oct 2, 2026
db3e9d2
Merge #201 (codex/local-network-purpose at 032eea5d) into the assembl…
SamGu-NRX Oct 2, 2026
d52ec50
Merge CODEX-01's tap readiness fix (codex/ui-test-reliability at b3c1…
SamGu-NRX Oct 2, 2026
75f6c37
Merge #188 (codex/public-coverage-20260930 at bc21aced) into the asse…
SamGu-NRX Oct 2, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions .github/workflows/ios.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,10 @@ jobs:
# macos-26 (arm64) is the newest GA macOS image. The xcode-27 image is still a preview.
runs-on: macos-26
# Beta run 36302099002 took 43 minutes before the spot-confirmation and review checks
# were added. Leave room for those checks without dropping the every-state audit.
timeout-minutes: 60
# were added. Full run 36832282114 (#190) then took 58 of 60 minutes, and the ground
# freshness UI tests add about 161 s more (run 36744881974), so 60 would cut the
# every-state audit short. 75 leaves room without dropping any check.
timeout-minutes: 75
env:
# Newest stable Xcode on macos-26 per images/macos/macos-26-arm64-Readme.md (image 20260907).
DEVELOPER_DIR: /Applications/Xcode_26.6.app/Contents/Developer
Expand Down
2 changes: 2 additions & 0 deletions ios/Config/Info.plist
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@
</dict>
<key>NSCameraUsageDescription</key>
<string>House Scan uses the camera to measure the outside wall around your electric meter.</string>
<key>NSLocalNetworkUsageDescription</key>
<string>House Scan connects to a House Scan server on your local network only when a developer has set it up for testing.</string>
<key>NSMotionUsageDescription</key>
<string>House Scan records your phone's motion and air pressure while you scan and saves them with the scan. The readings stay on this phone unless you share the scan.</string>
<key>UIApplicationSceneManifest</key>
Expand Down
47 changes: 47 additions & 0 deletions ios/HouseScan/PrivacyInfo.xcprivacy
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<!--
Required-reason APIs the app calls, with Apple's approved reason for each use. Only first-party
code ships in the app; HouseScanKit has no dependencies. Tracking and collected data types are
not declared here: they belong to the App Store privacy answers, which this file does not decide.
-->
<plist version="1.0">
<dict>
<key>NSPrivacyAccessedAPITypes</key>
<array>
<!-- The Practice meter switch: the app's own key in standard defaults
(DeveloperSettings.swift, DeveloperOptions.swift). -->
<dict>
<key>NSPrivacyAccessedAPIType</key>
<string>NSPrivacyAccessedAPICategoryUserDefaults</string>
<key>NSPrivacyAccessedAPITypeReasons</key>
<array>
<string>CA92.1</string>
</array>
</dict>
<!-- Which earlier scan.zip in the app's Caches folder is newest, to keep it
(ScanFolderCleanup.swift). -->
<dict>
<key>NSPrivacyAccessedAPIType</key>
<string>NSPrivacyAccessedAPICategoryFileTimestamp</string>
<key>NSPrivacyAccessedAPITypeReasons</key>
<array>
<string>C617.1</string>
</array>
</dict>
<!-- systemUptime. 35F9.1: time between a homeowner's "can't get there" taps
(ScanEngine+Ends.swift refusalClock), and dropping recorded frames from before a
restart (CaptureRecorder.restart). 8FFB.1: the wall-clock time of the first recorded
frame, the packet's capture.started_at (CaptureRecorder.append). -->
<dict>
<key>NSPrivacyAccessedAPIType</key>
<string>NSPrivacyAccessedAPICategorySystemBootTime</string>
<key>NSPrivacyAccessedAPITypeReasons</key>
<array>
<string>35F9.1</string>
<string>8FFB.1</string>
</array>
</dict>
</array>
</dict>
</plist>
2 changes: 1 addition & 1 deletion ios/HouseScan/Runtime/LaunchOptions.swift
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ import OSLog
/// - `-autopilotSomethingThere`: the autopilot answers the first spot check "Something's there"
/// instead of "It's clear", so the scan is checked again without that area.
/// - `-injectGroundRise <meters>`: with `-replay`, `-autopilot` and `-autopilotGate`, once the
/// first result shows, each time a file named `inject-ground` appears in the gate folder the
/// first upload starts, each time a file named `inject-ground` appears in the gate folder the
/// app deletes it and hands the engine a detected floor that many meters above its current
/// ground, as ARKit refining the ground would (`ScanEngine.injectGroundForTest`). Replays carry
/// no plane evidence, so this is the only way a UI test reaches that path. Meters must be over
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import CoreGraphics
import Foundation
import HouseScanKit
import ImageIO
import OSLog
import Vision

/// Reads the meter number from a close-up with Apple Vision, configured as the close-up eval
Expand Down Expand Up @@ -46,6 +47,13 @@ struct VisionMeterNumberReader: MeterNumberReader {
do {
try VNImageRequestHandler(cgImage: image, orientation: orientation).perform([text, barcodes])
} catch {
// The homeowner sees this as "no number", so the log is the only place that says Vision
// itself failed (e.g. com.apple.Vision 9, "Could not create inference context", in one
// Mac's simulator). Domain and code are framework identifiers and stay public. The
// message is Vision's free text, so it stays private: nothing vouches that it never
// names a file or carries content.
let failure = error as NSError
RuntimeLog.engine.error("meter number: Vision failed, reading as no number: \(failure.domain, privacy: .public) \(failure.code, privacy: .public) \(failure.localizedDescription, privacy: .private)")
return retakeNoNumber
}

Expand Down
9 changes: 7 additions & 2 deletions ios/HouseScan/Runtime/ResultClient.swift
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,9 @@ protocol ResultClient: AnyObject {
/// the body and `Content-Type: application/json`. Only the JSON goes: the server's solver reads no
/// photos and skips the image check for a bare scene.json, so the keyframes stay on the phone
/// (and in the scan folder's `scan.zip`, which leaves only if the homeowner shares it). The
/// response body is the result JSON.
/// response body is the result JSON, and it is returned only when it names the scene sent: its
/// `stats.input_sha256` must be present and be the hash of these exact bytes (`ResultBinding`).
/// Any other answer throws, and the upload screen treats it as an answer it couldn't read.
@MainActor
final class HTTPResultClient: ResultClient {
let serverURL: URL
Expand Down Expand Up @@ -42,6 +44,7 @@ final class HTTPResultClient: ResultClient {
status: http.statusCode, body: String(decoding: data.prefix(300), as: UTF8.self),
retryAfter: http.value(forHTTPHeaderField: "Retry-After"))
}
try ResultBinding.check(answer: data, submittedScene: scene)
progress(1)
return data
}
Expand Down Expand Up @@ -137,7 +140,9 @@ final class UploadProgressDelegate: NSObject, URLSessionTaskDelegate, Sendable {
}

/// Answers with the bundled SampleResult.json, for tests and demos without a server. The result
/// is flagged `isSample` so the screen says it is not a real analysis.
/// is flagged `isSample` so the screen says it is not a real analysis. It answers no particular
/// scene (its `input_sha256` is all zeros), so it is deliberately not checked with
/// `ResultBinding`.
@MainActor
final class SampleResultClient: ResultClient {
let isSample = true
Expand Down
12 changes: 9 additions & 3 deletions ios/HouseScan/Runtime/ScanEngine+Confirm.swift
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ struct SpotConfirmState {
/// The last upload's scene.json, and the sha256 of it and of the answer to it: what a check
/// asked now is bound to, and the scene the bundle is written again with after "It's clear".
var lastScene: Data?
/// The capture packet's inputs from that same upload, with `lastScene` attached: the packet
/// written again after "It's clear" (`answerSpotCheck`).
var lastPacket: ScanEngine.PacketInputs?
var sceneSHA256: String?
var answerSHA256: String?
/// The check on screen: its area, the photo shown and the exchange it came from.
Expand Down Expand Up @@ -50,8 +53,9 @@ extension ScanEngine {

/// Records the upload's scene and the answer to it, which a check asked about this answer
/// is bound to. Call it once the answer has decoded.
func noteExchange(scene: Data, answer: Data) {
func noteExchange(scene: Data, answer: Data, packet: PacketInputs?) {
spotConfirm.lastScene = scene
spotConfirm.lastPacket = packet
spotConfirm.sceneSHA256 = PacketFiles.sha256(scene)
spotConfirm.answerSHA256 = PacketFiles.sha256(answer)
}
Expand Down Expand Up @@ -125,8 +129,10 @@ extension ScanEngine {
return
}
// The bundle was written at the upload, before this answer: write it again so its
// guidance log holds the answer.
if let scene = spotConfirm.lastScene { saveBundle(scene: scene, mesh: liveCapture?.meshSnapshot()) }
// guidance log holds the answer. The upload's captured packet is reused, with only
// the guidance log read again and placed in that packet's frame, so the packet keeps
// the submitted scene's geometry, mesh and photos rather than mixing in a later wall.
if let packet = spotConfirm.lastPacket { saveBundle(withCurrentGuidance(packet)) }
go(.result)
}
}
Expand Down
44 changes: 23 additions & 21 deletions ios/HouseScan/Runtime/ScanEngine+Export.swift
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,19 @@ import OSLog
import simd

extension ScanEngine {
/// scene.json for the current scan (contract C1), with the LiDAR mesh's measurements on
/// phones that have one.
/// scene.json for the current scan (contract C1), without mesh measurements: the autopilot's
/// gate copy. The upload captures the scene once and measures the mesh against that capture
/// (`captureUpload`, `UploadPackaging`).
func sceneJSON() throws -> Data {
try SceneExport.jsonData(sceneInput())
}

/// The scan's scene as it stands now, without mesh measurements, read in this one turn.
///
/// The scene frame is the AR world frame moved down so the ground at the wall is y = 0: the
/// server reads heights (meter.pos y, pose translations) as height above that ground, and
/// scene.json has no field for the ground's height otherwise.
func sceneJSON(mesh: MeshMeasurements = MeshMeasurements()) throws -> Data {
func sceneInput() throws -> SceneInput {
guard let map = coverage else { throw ExportError.noWall }
let wall = map.wall
let drop = SIMD3<Float>(0, wall.groundY, 0)
Expand Down Expand Up @@ -74,13 +80,23 @@ extension ScanEngine {
),
keyframes: keyframes,
stills: store.stills,
// Written without plus_minus_ft: the server takes its mesh error for both.
meshFacing: mesh.facing,
meshOverheads: mesh.overheads,
// The mesh's measurements are filled in once measured (`UploadPackaging`). Written
// without plus_minus_ft: the server takes its mesh error for both.
// Unanswered exports like "Not sure": no patch, and the server reports the surface unknown.
groundType: state.groundAnswer.flatMap(Self.sceneGroundType)
)
return try SceneExport.jsonData(input)
return input
}

/// Everything one upload sends, read in one main-actor turn: the scene, the wall in world
/// meters, the LiDAR mesh and the capture packet's inputs. Nothing after this rereads the
/// engine's geometry for this upload, so a ground or anchor correction while the mesh is
/// measured can't mix two walls into its scene or packet.
func captureUpload() throws -> UploadPackaging<PacketInputs> {
let scene = try sceneInput()
guard let map = coverage else { throw ExportError.noWall }
let meshSnapshot = liveCapture?.meshSnapshot()
return UploadPackaging(scene: scene, worldWall: map.wall, mesh: meshSnapshot?.mesh, packet: packetInputs(mesh: meshSnapshot))
}

static func sceneGroundType(_ answer: GroundAnswer) -> SceneGroundType? {
Expand Down Expand Up @@ -132,20 +148,6 @@ extension ScanEngine {
return low...high
}

/// What the LiDAR mesh measured over the exported stretch: the gap from the wall out to
/// whatever faces it, and the clear height under anything overhead, each per stretch of s in
/// meters. Empty without a mesh.
struct MeshMeasurements: Sendable {
var facing: [ObservedSpan] = []
var overheads: [ObservedSpan] = []
}

/// Measures a world-space mesh (meters) against the wall (world meters) over `span`, the
/// exported stretch of s.
nonisolated static func measure(_ mesh: TriangleMesh, wall: WallFrame, over span: ClosedRange<Float>) -> MeshMeasurements {
MeshMeasurements(facing: mesh.facingSpans(wall: wall, over: span), overheads: mesh.overheadSpans(wall: wall, over: span))
}

/// Error of the chest-height ground guess (camera height minus 1.4 m), meters. Phones held
/// for scanning sit roughly 1.1 to 1.7 m up, so ±0.3 m. A hypothesis; no measured spread exists.
static let estimatedGroundError: Float = 0.3
Expand Down
21 changes: 18 additions & 3 deletions ios/HouseScan/Runtime/ScanEngine+Packet.swift
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,10 @@ extension ScanEngine {
/// `photoIDs` hold store ids ("meter_close"); `writePacket` swaps in packet ids.
var marks: [PacketMark]
var guidance: [PacketGuidanceEntry]
/// The wall the marks and guidance were placed with, in world meters, captured with them,
/// so a later write that adds guidance places it in the same frame (`withCurrentGuidance`).
var sceneWall: SceneWall
/// The scene.json the upload sent, attached once serialized (`UploadPackaging`).
var scene: Data
/// The camera's frame rate live; nil on a replay, whose rate comes from its frames.
var trajectoryRate: Double?
Expand All @@ -49,8 +53,9 @@ extension ScanEngine {
var frames: [ReplayFrame]
}

/// Nil before there is a wall: the meter frame is built from it.
func packetInputs(scene: Data, mesh: LiveCapture.MeshSnapshot?) -> PacketInputs? {
/// Nil before there is a wall: the meter frame is built from it. The scene is attached by the
/// upload once it is serialized from the same capture (`captureUpload`).
func packetInputs(mesh: LiveCapture.MeshSnapshot?) -> PacketInputs? {
guard let map = coverage else { return nil }
let wall = map.wall
// The wall in world meters, as scene.json's wall type describes it, for the marks.
Expand Down Expand Up @@ -93,12 +98,22 @@ extension ScanEngine {
},
marks: packetMarks(map, wall: sceneWall, frame: frame),
guidance: guidanceLog.entries.map { Self.packetEntry($0, wall: sceneWall, frame: frame) },
scene: scene,
sceneWall: sceneWall,
scene: Data(),
trajectoryRate: settings.map { Double($0.framesPerSecond) },
motionStreams: motionRunsLive ? motionAvailable : []
)
}

/// `inputs` with the guidance log as it is now, placed with the wall and meter frame the
/// inputs were captured with: what the spot check's answer adds to an upload's packet, without
/// rereading the engine's geometry since.
func withCurrentGuidance(_ inputs: PacketInputs) -> PacketInputs {
var inputs = inputs
inputs.guidance = guidanceLog.entries.map { Self.packetEntry($0, wall: inputs.sceneWall, frame: inputs.meterFrame) }
return inputs
}

/// `utsname.machine`, such as "iPhone16,1": the hardware, never the phone's name. "arm64" in
/// the Simulator.
nonisolated static func hardwareModel() -> String {
Expand Down
Loading
Loading