From d82a90339a0a57a4786ce6e8be0dcc05c18578f4 Mon Sep 17 00:00:00 2001 From: SamGu-NRX Date: Sat, 26 Sep 2026 11:30:48 -0500 Subject: [PATCH 1/9] packet: capture packet 1.0 spec, schema, validator and sample builders The contract between the iPhone app and the server team: one scan session as manifest.json, full-resolution photos and sensor files, everything in the meter frame, in meters and device-uptime seconds. `python -m packet validate` checks the schema, file sizes and hashes, rigid poses, intrinsics against each image, time order, depth alignment, stream layout, the mesh and consent. Builders write samples from the app's own export, ADVIO and ETH3D to ~/house-scanning-data/packets/; only a synthetic fixture is committed. Co-Authored-By: Claude Opus 5.5 (1M context) --- packet/.python-version | 1 + packet/README.md | 289 ++++++++++ .../fixtures/synthetic/depth/p00001.conf.u8 | Bin 0 -> 432 bytes packet/fixtures/synthetic/depth/p00001.f32 | Bin 0 -> 1728 bytes .../fixtures/synthetic/depth/p00002.conf.u8 | Bin 0 -> 432 bytes packet/fixtures/synthetic/depth/p00002.f32 | Bin 0 -> 1728 bytes .../fixtures/synthetic/depth/p00003.conf.u8 | Bin 0 -> 432 bytes packet/fixtures/synthetic/depth/p00003.f32 | Bin 0 -> 1728 bytes packet/fixtures/synthetic/lidar/mesh.ply | Bin 0 -> 369 bytes packet/fixtures/synthetic/manifest.json | 516 +++++++++++++++++ packet/fixtures/synthetic/photos/p00001.jpg | Bin 0 -> 2425 bytes packet/fixtures/synthetic/photos/p00002.jpg | Bin 0 -> 2539 bytes packet/fixtures/synthetic/photos/p00003.jpg | Bin 0 -> 2269 bytes packet/fixtures/synthetic/scene.json | 26 + .../synthetic/streams/accelerometer.csv | 201 +++++++ .../fixtures/synthetic/streams/barometer.csv | 3 + .../synthetic/streams/device_motion.csv | 201 +++++++ .../fixtures/synthetic/streams/gyroscope.csv | 201 +++++++ packet/fixtures/synthetic/streams/heading.csv | 2 + .../fixtures/synthetic/streams/location.csv | 2 + .../synthetic/streams/magnetometer.csv | 201 +++++++ .../fixtures/synthetic/streams/trajectory.csv | 122 +++++ packet/manifest.schema.json | 318 +++++++++++ packet/packet/__init__.py | 1 + packet/packet/__main__.py | 46 ++ packet/packet/files.py | 103 ++++ packet/packet/samples/__init__.py | 1 + packet/packet/samples/advio.py | 209 +++++++ packet/packet/samples/app_export.py | 191 +++++++ packet/packet/samples/eth3d.py | 235 ++++++++ packet/packet/samples/synthetic.py | 287 ++++++++++ packet/packet/validate.py | 517 ++++++++++++++++++ packet/packet/write.py | 156 ++++++ packet/pyproject.toml | 32 ++ packet/tests/test_checks.py | 219 ++++++++ packet/tests/test_packet.py | 242 ++++++++ packet/uv.lock | 417 ++++++++++++++ 37 files changed, 4739 insertions(+) create mode 100644 packet/.python-version create mode 100644 packet/README.md create mode 100644 packet/fixtures/synthetic/depth/p00001.conf.u8 create mode 100644 packet/fixtures/synthetic/depth/p00001.f32 create mode 100644 packet/fixtures/synthetic/depth/p00002.conf.u8 create mode 100644 packet/fixtures/synthetic/depth/p00002.f32 create mode 100644 packet/fixtures/synthetic/depth/p00003.conf.u8 create mode 100644 packet/fixtures/synthetic/depth/p00003.f32 create mode 100644 packet/fixtures/synthetic/lidar/mesh.ply create mode 100644 packet/fixtures/synthetic/manifest.json create mode 100644 packet/fixtures/synthetic/photos/p00001.jpg create mode 100644 packet/fixtures/synthetic/photos/p00002.jpg create mode 100644 packet/fixtures/synthetic/photos/p00003.jpg create mode 100644 packet/fixtures/synthetic/scene.json create mode 100644 packet/fixtures/synthetic/streams/accelerometer.csv create mode 100644 packet/fixtures/synthetic/streams/barometer.csv create mode 100644 packet/fixtures/synthetic/streams/device_motion.csv create mode 100644 packet/fixtures/synthetic/streams/gyroscope.csv create mode 100644 packet/fixtures/synthetic/streams/heading.csv create mode 100644 packet/fixtures/synthetic/streams/location.csv create mode 100644 packet/fixtures/synthetic/streams/magnetometer.csv create mode 100644 packet/fixtures/synthetic/streams/trajectory.csv create mode 100644 packet/manifest.schema.json create mode 100644 packet/packet/__init__.py create mode 100644 packet/packet/__main__.py create mode 100644 packet/packet/files.py create mode 100644 packet/packet/samples/__init__.py create mode 100644 packet/packet/samples/advio.py create mode 100644 packet/packet/samples/app_export.py create mode 100644 packet/packet/samples/eth3d.py create mode 100644 packet/packet/samples/synthetic.py create mode 100644 packet/packet/validate.py create mode 100644 packet/packet/write.py create mode 100644 packet/pyproject.toml create mode 100644 packet/tests/test_checks.py create mode 100644 packet/tests/test_packet.py create mode 100644 packet/uv.lock diff --git a/packet/.python-version b/packet/.python-version new file mode 100644 index 000000000..e4fba2183 --- /dev/null +++ b/packet/.python-version @@ -0,0 +1 @@ +3.12 diff --git a/packet/README.md b/packet/README.md new file mode 100644 index 000000000..cab190135 --- /dev/null +++ b/packet/README.md @@ -0,0 +1,289 @@ +# The capture packet + +The contract between the iPhone app and the server. The app guides the homeowner, takes the +photos and records what the phone knows. The server builds a 3D model from them and evaluates +the placement rules. One packet is one scan session: a folder, or a zip of one, holding +`manifest.json`, the photos and the sensor files. This document is the specification. +[`manifest.schema.json`](manifest.schema.json) checks the manifest's shape, and +`python -m packet validate` checks everything below. + +Version 1.0. The **App** column in each table says what `t3/ios-mvf` at `a39d0a5` records: +**now** (exported today), **partial** (computed in the app but not exported, or exported in +another form) or **planned**. [What the app records today](#what-the-app-records-today) lists +the gaps. + +## Layout + +``` +manifest.json what everything is (this document) +photos/p00001.jpg full-resolution photos, one per kept frame +depth/p00001.f32 depth aligned to that photo (LiDAR phones) +depth/p00001.conf.u8 its confidence +streams/*.csv trajectory, IMU, barometer, location, heading +lidar/mesh.ply ARKit mesh with per-face classification (LiDAR phones) +scene.json the app's placement request (C1), unchanged +``` + +Only `manifest.json` has a fixed name. Every other file is named by a manifest entry, +`{"path", "bytes", "sha256"}`. The path is relative to the manifest, uses `/`, and never starts +with `/` or contains `..`. A reader must refuse a zip whose entries leave the packet. A zip may +hold the packet at its root or inside one top-level folder. + +## Units, frames and clocks + +These hold for every field unless its row says otherwise. + +- **Units.** Meters and seconds. Angles are degrees where a person reads them (headings, + latitude and longitude) and radians per second for the gyroscope. The accelerometer is in + m/s² (Core Motion reports g: multiply by 9.80665), the magnetometer in microtesla and + pressure in kPa. +- **Clock.** Seconds of device uptime: the clock of `ARFrame.timestamp`, `CMLogItem.timestamp` + and `ProcessInfo.systemUptime`. `session.capture.started_at` (ISO 8601, UTC) and + `started_at_uptime` are the same instant, so a wall-clock time such as `CLLocation.timestamp` + converts as `uptime = started_at_uptime + (date - started_at)`. +- **World frame.** ARKit's world frame with `worldAlignment = .gravity`: right-handed, +y up + (against gravity), origin and heading wherever the session started. Nothing in the packet is + in this frame except `session.meter_anchor.pose_in_world`, and heading is never used. +- **Meter frame.** Every pose and point in the packet is in the meter frame. Its origin is the + meter anchor, the point the homeowner tapped on the meter. +y is up (the world's +y), and +z + is the wall's outward normal, horizontal, toward where the homeowner stands. +x = y × z runs + along the wall to the right as seen facing it: the same direction as `scene.json`'s `s`. + `session.meter_anchor.pose_in_world` maps meter-frame points into the world frame. Results + are placed relative to the meter because ARKit's world origin is arbitrary per session, and + the meter is the one point every later visit can find again. +- **Camera frame.** ARKit's: +x right and +y up in the stored image, and the camera looks along + −z. For OpenCV's camera axes (y down, looking along +z), multiply by diag(1, −1, −1). +- **Poses.** Rigid 4 × 4 transforms, written as 16 numbers column by column (`simd_float4x4` + layout, as in Measure Lab v2 and `scene.json`). A photo's pose maps camera coordinates into + the meter frame. +- **Images and intrinsics.** The camera image is landscape and the intrinsics match it. A photo + is stored exactly as the sensor delivers it: landscape (width > height), never rotated to the + phone's orientation, EXIF orientation absent or 1. `intrinsics` are `[fx, fy, cx, cy]` in + pixels of that stored image, where (0, 0) is the top-left corner of the top-left pixel (pixel + centres at half-integers). A resized image scales all four by the same factor. Rotating an + image would mean rotating the intrinsics with it, so neither is rotated. +- **Depth alignment.** A depth map covers the same field of view as its photo, at lower + resolution and the same aspect ratio (within 1%). Depth pixel (row r, column c) covers photo + pixels x ∈ [c·W/w, (c+1)·W/w), y ∈ [r·H/h, (r+1)·H/h). Values are float32 little-endian + meters along the camera's −z axis (not the distance along the ray), row by row from the top, + and 0 means no measurement. Confidence is one byte per depth pixel, ARKit's + `ARConfidenceLevel`: 0 low, 1 medium, 2 high. + +## Session + +| Field | Meaning | App | +| --- | --- | --- | +| `packet_version` | `"1.0"`; see [Versioning](#versioning) | planned | +| `session.id` | Unique per scan | planned | +| `session.producer` | `kind` (`app` or `converter`), `name`, `version`, optional `commit` | planned | +| `session.device.model` | Hardware identifier, e.g. `iPhone16,1` (`utsname.machine`), never the phone's name | planned | +| `session.device.ios_version` | `UIDevice.systemVersion` | planned | +| `session.device.lidar` | `ARWorldTrackingConfiguration.supportsFrameSemantics(.sceneDepth)` | planned | +| `session.device.scene_depth_enabled`, `mesh_enabled` | What the session actually ran with | planned | +| `session.capture.started_at`, `started_at_uptime`, `ended_at_uptime` | Wall clock and uptime at the first frame, uptime at the last | planned | +| `session.capture.distance_walked_m` | Horizontal length of the trajectory (meter frame x and z). Must match `streams.trajectory` within 1% | planned | +| `session.world_alignment` | Always `"gravity"` | partial: the app runs with `.gravity` but does not write it | +| `session.meter_anchor.pose_in_world` | The meter frame in the world frame | partial: `scene.json` has the meter's position (feet, ground at y = 0), not its orientation | +| `session.meter_anchor.ground_y_m` | The ground's height at the meter on the meter frame's y (negative) | partial: implied by `scene.json`'s y = 0 | +| `session.consent.location` | The homeowner agreed to share location and heading. Must be true if either stream is present | planned | + +## Photos + +One entry per kept frame, in time order. Required: `id`, `image`, `width`, `height`, `t`, `pose`, +`intrinsics`. + +| Field | Meaning | App | +| --- | --- | --- | +| `image` | JPEG of the full frame: at least ARKit's `capturedImage` at its native size (1920 × 1440 on current iPhones). On iOS 16 and later, prefer `captureHighResolutionFrame`'s still (4032 × 3024) with its own intrinsics. Never a thumbnail | now: `capturedImage` at native size, quality 0.8 | +| `width`, `height` | Pixels of the stored JPEG | now | +| `t` | `ARFrame.timestamp` | partial: recorded, not exported | +| `pose` | Camera to meter frame, from `ARFrame.camera.transform` | partial: exported camera-to-world in feet | +| `intrinsics` | `ARFrame.camera.intrinsics` for the stored image | now | +| `tracking.state`, `tracking.reason` | `normal`, `limited` or `not_available`, with the limited reason | partial: recorded, not exported | +| `exposure.duration_s`, `exposure.iso`, `exposure.offset_ev` | `ARCamera.exposureDuration` and `exposureOffset` (iOS 15); ISO from the high-resolution still's metadata | planned | +| `lens.focal_length_mm`, `lens.f_number`, `lens.camera` | From the still's EXIF; `camera` is `wide`, `ultra_wide` or `telephoto` | planned | +| `sharpness` | `{"method": "laplacian_variance_luma_640", "value"}`, defined below | partial: the app scores sharpness another way and does not export it | +| `depth` | `map`, optional `confidence`, `width`, `height`, `source` (`arkit_scene_depth`, `arkit_smoothed_scene_depth`, or `rendered_from_laser_scan` in samples) | planned (S6, LiDAR) | + +**Sharpness** is the variance of the 4-neighbour Laplacian of the image's luma, after scaling the +long side to 640 px with bilinear filtering (images already smaller are used as they are). One +fixed method makes scores comparable across phones and resolutions. `packet/write.py` has the +reference implementation. + +## Streams + +CSV files with a header row. The first column is always `t`, strictly increasing. Each has a +manifest entry `{"path", "bytes", "sha256", "rows", "nominal_rate_hz"}`, and its columns are +fixed: + +| Stream | Columns | Source | App | +| --- | --- | --- | --- | +| `trajectory` | `t, tracking, px, py, pz, qx, qy, qz, qw` | Every ARFrame (60 Hz): camera-to-meter position and unit quaternion, tracking state | planned: the app keeps pose-only frames at 30 Hz for its overlay and discards them | +| `accelerometer` | `t, x, y, z` (m/s²) | `CMAccelerometerData` | planned | +| `gyroscope` | `t, x, y, z` (rad/s) | `CMGyroData` | planned | +| `magnetometer` | `t, x, y, z` (µT, uncalibrated) | `CMMagnetometerData` | planned | +| `device_motion` | `t, qx, qy, qz, qw, gravity_x, gravity_y, gravity_z, user_accel_x, user_accel_y, user_accel_z, rotation_rate_x, rotation_rate_y, rotation_rate_z, heading_deg` | `CMDeviceMotion` (gravity and user acceleration in g, as Core Motion reports them) | planned | +| `barometer` | `t, pressure_kpa, relative_altitude_m` | `CMAltimeter` | planned | +| `location` | `t, latitude, longitude, altitude_m, horizontal_accuracy_m, vertical_accuracy_m` | `CLLocation`, with consent only | planned | +| `heading` | `t, magnetic_deg, true_deg, accuracy_deg` | `CLHeading`, with consent only | planned | + +IMU axes are Core Motion's device frame: +x to the right and +y toward the top of a phone held in +portrait, +z out of the screen. A photo from a tracked frame must sit on the trajectory: the +trajectory has a sample within 1/60 s of the photo's `t`, at the same position to 1 cm. + +## LiDAR extras + +On a LiDAR phone with scene reconstruction on, `lidar.mesh` is ARKit's mesh (`ARMeshAnchor`s +merged, in the meter frame), as a binary PLY with exactly this header: + +``` +ply +format binary_little_endian 1.0 +element vertex N +property float x +property float y +property float z +element face M +property list uchar int vertex_indices +property uchar classification +end_header +``` + +Every face is a triangle. `classification` is `ARMeshClassification`'s raw value: 0 none, 1 wall, +2 floor, 3 ceiling, 4 table, 5 seat, 6 window, 7 door. `comment` lines may follow `ply`. + +`lidar.planes` lists `ARPlaneAnchor`s: `id`, `alignment` (`horizontal` or `vertical`), +`classification`, `pose` (plane to meter frame; the plane is its local x-z plane and its normal +is +y), and `extent_m` ([x, z]). App: planned. The app reads plane anchors for its coverage and +taps but does not export them. + +## Marks, guidance and the scene + +`marks` are what the homeowner marked, in the meter frame. `kind` is `meter`, `wall_end`, +`gas_meter`, `ac`, `door`, `window`, `garage_door`, `drive_edge` or `fence`. `points` holds one +point for a meter, wall end, gas meter or AC unit, and two for an opening (bottom-left and +top-right corners), a drive edge or a fence. Each mark also has optional `t`, `photo_ids` and +`attrs.operable`. A wall end adds `side` (`left`, `right`) and `end_kind` (`limit` when something +blocks it, `unexplored` for a corner or no answer). App: partial. `scene.json` carries these as +objects and wall ends in C1's feet and `s` coordinates, without times or photo links. + +`guidance` logs every request the homeowner was shown, and whether it was met: + +- `kind`: `walk`, `tilt_to_ground`, `step_back`, `mark_end`, `closeup`, `gap_band` or + `gap_past_end`; +- `origin`: `phone` or `server`; +- `message`, and for a band request `band` and `span_m` (meter frame x); +- `t_shown` and `t_resolved`; +- `outcome`: `met`, `skipped`, `cannot_reach`, `superseded` or `unresolved`. + +The log is how the server learns what the homeowner could not reach, which `scene.json` cannot +say today (bug triage B-08). App: planned; guidance is transient UI state. + +`scene` is the app's `scene.json`, unchanged, with its `schema_version`. It stays the input to +placement; the packet carries it so reconstruction and placement see the same capture. App: now. + +`provenance` (`dataset`, `license`, `source`, `notes`) says where a converted sample came from and +what it lacks. The app does not write it. + +## Versioning + +`packet_version` is `MAJOR.MINOR`. A reader accepts any packet with its own major version and +ignores fields it does not know. A writer bumps the minor version for additions (a new optional +field, stream or enum value a reader may skip) and the major version for anything an old reader +would misread: a renamed or removed field, a changed unit, frame or column order. The validator +accepts `1.x`. + +## Transport + +This is a recommendation; Hunter's team decides. A packet does not fit in one request. The +hosted API refuses bodies over 4.5 MB (Vercel's limit). The app's Simulator export of 77 photos +at 1280 × 720 is already 16.5 MB. At an estimated 0.5 MB per 1920 × 1440 photo and 3 MB per +high-resolution still, 80 photos come to about 40 MB, or 250 MB with stills. These are estimates: +no device capture has been measured. + +1. The phone writes the packet on disk as it captures, so a crash or a lost connection loses + nothing already taken. +2. `POST /v1/packets` with `manifest.json` only (tens of KB). The server validates the manifest + and answers with a packet id and one presigned upload URL per file in object storage (S3, + R2 or Vercel Blob), keyed by sha256. +3. The phone uploads each file with a background `URLSession` upload task, which iOS continues + after the app is suspended. Files over about 8 MB use multipart upload. Because files are + named by their hash, a retry skips any file the server already holds. +4. `POST /v1/packets/{id}/complete`. The server checks every size and hash (this validator), + then queues reconstruction. +5. Placement stays a separate small request. `POST /v1/placements` with `scene.json` (KB) can + answer at once, and a later answer can use the reconstruction. + +## Relation to existing formats + +- **`scene.json` (C1)** is unchanged and travels inside the packet. It stays in feet and in + ARKit's world moved so the ground is y = 0. The rest of the packet is in meters and in the + meter frame. +- **Measure Lab session v2** (the replay format) shares the packet's units, camera axes, + column-major poses and depth encoding. It differs in frame (world, not meter), file names, a + per-keyframe `depth` object `{file, confidenceFile, w, h}`, and a `tracking[]` timeline where + the packet has a trajectory stream. +- **`recon/recon/capture.py`** on `t3/recon` reads `scene.json` bundles (feet, scaled to meters + on load) and Measure Lab v2. Reading a packet needs a third branch: photo poses are already + in meters, and the wall direction and normal are the meter frame's x and z axes. + +## What the app records today + +At `t3/ios-mvf` `a39d0a5`, the app's export (`scan.zip`) is `scene.json` plus one JPEG per kept +keyframe. The upload sends `scene.json` alone. To write a 1.0 packet the app needs, in rough +order of cost: + +1. **Already in memory, not exported:** each photo's timestamp and tracking state, the meter + anchor's orientation, and mark times. +2. **One read at session start:** device model, iOS version, app version, LiDAR support, the + wall-clock start. +3. **Convert:** poses to meters in the meter frame, and marks from C1's `s`/height to meter-frame + points. +4. **Record and write:** the 60 Hz trajectory (today discarded after the overlay uses it), + exposure and ISO per photo, sharpness by the fixed method, distance walked, and the guidance + log. +5. **New sensors:** Core Motion (accelerometer, gyroscope, magnetometer, device motion, + altimeter); location and heading behind a consent prompt. +6. **LiDAR (S6):** `sceneDepth` with confidence per photo, the mesh with classification, plane + anchors. +7. **Transport:** the manifest-then-files upload above. + +## Tools + +From `packet/`: + +``` +uv run python -m packet validate # exit 0 when valid; --json for a report +uv run python -m packet.samples.synthetic --out fixtures/synthetic +uv run python -m packet.samples.app_export --sim-report --replay --out +uv run python -m packet.samples.advio --replay --advio --out +uv run python -m packet.samples.eth3d --eth3d --out +uv run pytest -q && uv run ruff check . && uv run ruff format --check . +``` + +`fixtures/synthetic` is the only packet in git: three invented photos with every optional +section. The dataset samples are written to `~/house-scanning-data/packets/` and never +committed. ADVIO is CC BY-NC 4.0, ETH3D CC BY-NC-SA 4.0, and the app sample carries ADVIO frames. + +| Sample | Source | Contents | What it shows | +| --- | --- | --- | --- | +| `app-export-a39d0a5` | The app's `scan.zip` from a Simulator run on the ADVIO replay | 77 photos (1280 × 720), marks, `scene.json` | Exactly what the app records today. Times are recovered by matching each photo to its replay frame | +| `advio-20-0040-0075` | ADVIO sequence 20, 40 to 75 s, iPhone 6s | 79 photos (1280 × 720), 60 Hz trajectory, accelerometer, gyroscope, magnetometer, barometer, location | Real phone IMU on the same clock as ARKit poses | +| `eth3d-electro` | ETH3D electro, 8 DSLR photos within about 6 m of the walls | 8 photos (6198 × 4132, originals) with laser-rendered depth | Depth aligned to full-resolution photos. ETH3D's own 3-D points reproject through the packet's poses to a median 0.65 to 1.01 px, and the depth agrees with them to a median 0.2 to 0.6% | + +The validator checks: + +- the schema; +- every file's presence, size and sha256; +- that every pose is a rotation plus a translation; +- that intrinsics fit each image (landscape, principal point inside it, square pixels within 5%, + a horizontal field of view between 20° and 150°); +- that JPEG sizes and EXIF orientation match the manifest; +- that times strictly increase and fall inside the capture; +- the stream columns, unit quaternions and tracking values; +- that photos sit on the trajectory, and that distance walked matches it; +- that depth has its photo's aspect and the right byte count, with finite, non-negative values + and confidence of at most 2; +- the mesh layout, face indices and classes; +- mark and guidance consistency; +- consent for location and heading. diff --git a/packet/fixtures/synthetic/depth/p00001.conf.u8 b/packet/fixtures/synthetic/depth/p00001.conf.u8 new file mode 100644 index 0000000000000000000000000000000000000000..f52aa7368938ce3aad8b1dee3ab343d97e5e8989 GIT binary patch literal 432 UcmZQzV8RL*M#aJ5GT6ca0GxsY82|tP literal 0 HcmV?d00001 diff --git a/packet/fixtures/synthetic/depth/p00001.f32 b/packet/fixtures/synthetic/depth/p00001.f32 new file mode 100644 index 0000000000000000000000000000000000000000..3a47744fd4b9738dcded9b1bb2212dfcc75db373 GIT binary patch literal 1728 lcmZQzfB^*u1_oN-QSuC6lh7TqAZZ!Rjrk|0UegK~Clz{*M literal 0 HcmV?d00001 diff --git a/packet/fixtures/synthetic/depth/p00002.conf.u8 b/packet/fixtures/synthetic/depth/p00002.conf.u8 new file mode 100644 index 0000000000000000000000000000000000000000..f52aa7368938ce3aad8b1dee3ab343d97e5e8989 GIT binary patch literal 432 UcmZQzV8RL*M#aJ5GT6ca0GxsY82|tP literal 0 HcmV?d00001 diff --git a/packet/fixtures/synthetic/depth/p00002.f32 b/packet/fixtures/synthetic/depth/p00002.f32 new file mode 100644 index 0000000000000000000000000000000000000000..3a47744fd4b9738dcded9b1bb2212dfcc75db373 GIT binary patch literal 1728 lcmZQzfB^*u1_oN-QSuC6lh7TqAZZ!Rjrk|0UegK~Clz{*M literal 0 HcmV?d00001 diff --git a/packet/fixtures/synthetic/depth/p00003.conf.u8 b/packet/fixtures/synthetic/depth/p00003.conf.u8 new file mode 100644 index 0000000000000000000000000000000000000000..f52aa7368938ce3aad8b1dee3ab343d97e5e8989 GIT binary patch literal 432 UcmZQzV8RL*M#aJ5GT6ca0GxsY82|tP literal 0 HcmV?d00001 diff --git a/packet/fixtures/synthetic/depth/p00003.f32 b/packet/fixtures/synthetic/depth/p00003.f32 new file mode 100644 index 0000000000000000000000000000000000000000..3a47744fd4b9738dcded9b1bb2212dfcc75db373 GIT binary patch literal 1728 lcmZQzfB^*u1_oN-QSuC6lh7TqAZZ!Rjrk|0UegK~Clz{*M literal 0 HcmV?d00001 diff --git a/packet/fixtures/synthetic/lidar/mesh.ply b/packet/fixtures/synthetic/lidar/mesh.ply new file mode 100644 index 0000000000000000000000000000000000000000..02082c8767474b81b22d003a7fa5db132499ed91 GIT binary patch literal 369 zcmZ{eZwkUN48`aC2R%hkAPRo)o4v`XHDiIU9c>irP4qH)G}9?g1{Tu3{Q7uFXOpOX zXf??LIvrAFk;p;??ugpS&1x--Z(DFAV{+60p%GxACuax*7r7H%@Ez+UYwJ1cG6rc3 zuHTGW8<5-K-(nR?1_J;FeE`w}TmTM>RSZ<{=pYaY1U#NVQdd_aYLYZH$s{saOIuG@OPiui zChHpLQuL`b8ckD&ZfHO?)T7d<3L_XCdInF>AP_XDT4XKidkb;{P>8@C1mZ9z087E( zC>W#>7^86#Fs}ssB^WFkqnbKVgG5F*RO$jO28YAqaf+zX-3L%Tz*7i%i&i+R>2C^B zH;JTLCubEBP1lvy(R|v+&1|;sOVJ=1&<*E(WNxw8a>-I#yOs8<99Fx0zTVZ%-DAV% zFMPN7eYw?t$2Y+{LqfyCqoQN>$Hv7cq#itUIPJ*M^b^@9Pvx9Glbcs`{=&seoXf=} zW#tu>zf}Et?RtGfV^ed>&DP)Ubo_C*^WOcgZjt!Wz~jN8C&N!CUQABCoSur0qqsCcLSF4LCF3F_7ARZpoznv#lukm1H2R%A)qx}vK0Z5Ovu82 zs(aE(bwzsfIRdVvRr&REh7nL@b4##OC^e~_kIvc9lAKKC{fxPk?5nkIE(nM(M!;U! zOW~}m9CqiWF-C_FupyYYk!KB^WelWCqTpWE)Fpq;gw;r?(7508Zjz6VvY&s8)U+DD zPGF|6Iq(CavLVNB5n#?0g+NWLsoYQ{A4;n5qeGalr5-P9cD+B z3vBn=?^^E%Gg{B(B7p279)l=W^6KRXkmQMKiHXjV(--Ad?Yk>75BBvWB*eqMGjg+> zPIynq5A26!1A!+TM!uinakwh@E26$`d&ta`=9gwnTPF%*dMfzGaSg`$yfv0`pK+(# zg#`$xcVJxzmknk`22a<;95typQrcT`o}1O|)|Q`tul>}E*!oQAC(M~Oaa;s=6!3lu z^DuWXheFfr6QUu;^srIz>`GQEZF~sgBVgF8B5nru^s5)nrlWt>?Q`fl89|B1Iq>9W zxKExtTsg=r50PgvwOKtv_+(qa*#Kp`mEL0Jy&dNmH%^zxFBmzYCA~Xd4vPgcuS)c6 zfbv3xTPze!5)c^P(6w_ab{?_ND#A*I1iguUcy$stXm|1b zjSy$hzjs|(qa@jI&ldcAUE3W!)V6%R(CZ#elzbz`q5pMo! zkH8{+sII!_fTwthzCXJuCM@z1o9d}KzAA3OwemaeGHv3*6yivpOw4C(8*e@)Vr2A@ zM{S)fT3jpQ?ZW)E)^sgAmfu~l+!x?QYlT;*c!9-x^L!#??X*q)KiWM;Gv%&w@z1vi zUijA*F%-O@OrXVbPua7m z7Kz?V=E+p0L@4-1W#Nc|iPv*ypN literal 0 HcmV?d00001 diff --git a/packet/fixtures/synthetic/photos/p00002.jpg b/packet/fixtures/synthetic/photos/p00002.jpg new file mode 100644 index 0000000000000000000000000000000000000000..a1469413db3ad39154a2b7de45110c4e399bc124 GIT binary patch literal 2539 zcmd6od010N5WqJVS3pQe08bRfoAL>YwL()+1Vp?L5v)YT1JR0za;WeG&*DJ=j{<5m z9uZna)sOr{v<>gwnj8W}X66$TSLl)D&kO(?zRvtwI@g zv5DvOM!Q@uWG=1}jj><7HBR5a%-mw|5bLqy#!r|yb(+KUuV;KSYqskgH+PS@-b;Lz z`Y!YH4_vc$-TI&n!C~9Bhet$4?bx$--~Iy!e~eE`K6d=X$)8hFx#urjyu`blo^c~9 zJ16(%t=oA;#U-Wp%gP@-d|F*o`>d}1xv;sVwe5BLn~u&dabLgWy>vh(S7^8hfb(}HNt2od$|wk)rfjU< zSjk@}kXscDgWx7dJ>f(_sLE5(y)zUg-}QYxQrxRLk-3jEz{Do6&GgE&(;DE;iBBMS6^t}MknH5} z`5@CZF@}}N=7#a3$#9DE1C)mYyqUMGfVuZ5d10)hv@GCv4H+3KjKLr&)S&9#&|ysvSP6* z%48LvW|y*`Lm=d@>_e&?d&gBa+PMu#bEoi~jOQ&Kmw>e8?3xV0txPo8CJ?P>_ic`` z!6+7}C-1+SG(S8Vg7VaNHBAuYK_E#VHoegJAd4>H?^D?1CW_LOAqy6;9u`2bH%Zv| zsIt#gnfv`=x|>tz8A)V*i*ZSQLi`BA?b|Xhsh3Nma#nXZOO&*nB^>yUyWkb2Xp@EG z`^S_W3EfzHV|llfn+4@} zMIkNQ-;@p?y8L#{R7;tgt>j2Rv{g$*PuyGj?90n~eHPW0w~l`Ocz&>!?Psj+ItZqD zqh*hXZ)n9l^tJTkT1&@3HyvKf=vB{$cAWci_?6aqr;zd zneWsoY7^PlR4G}zCW@Bkhz>d5sd2PC%MS3A9vLwIW)0n4cIzj@tO?@)LBfcvYxa%1 z^6ghDhpat?3bI24B4h{MOqdrNnD#^7@mJ@z)H@!QOyx(E-|PsMqu0GRDjSn+Vk+u# z6~{Rx75N@5dD*wbk9m{r?=`n3TFmx2et>M^Oy2$|B<2yP&vn-f>~66X-Zl{dmZJMd znrBYWej-U5KBTJ6E_+Dy4d|#4Tsn*YEJ9;tnFaU7oQ5Sw7g=fA7iEFUSts zO9)oz{=dqCrl#Nvg+w?yAI+n%<2XF@g#ce)9Ct!s{Y&$DbtxiA3Br{goGLh`NIu+H z-Of|<<@SFo_?n7|ZwkSj68_aPM9T`Vn=$`Z**ZA@uqB?+gCOD=0O zWz{5S8`4mcVWdcVgsf{BlO-aT5z{d9?Qd$&?hd=>?C#(D{w{yK-}hdg?|HumwnH({ z-{Ru#0#GObP{;#d7jOa?G+Ol{hDBbSCJu+i;)r;>hNd=ATU(2$rKO`w*4NP`>1t`| z)AUIO6e^Xft!HRNqZpAXREjDI3WLO8aReNWK+)0Cq5NxuRe+=k+`t|TY6(D-P#6*l zJ_II6pE%S91HTyxjr6F2*CY_NkPQNTfJR|3Xe>t6HM098vJS8$9C`7YjT#2q0`W@@ zQ>E)+Tn`7k6bZt)ro^2TlWz{3VhtYbAPn^LY8R})NayvL|} zBb$h{rE7c5##PWhAp2*)PW>ljzXSUNS0~WMpb+vfB)|ajyo!d$!U?}h8mHR_0{4su zc5@t+k-Ja!WREVA?!@L8dk7*J^6jhi zB+Rxw7skwngqHXVoIXL!N(k=qm1(O~!E+p>7ZCWP`PE)65F84glW3761gWiKB}$1< z?s-SOh}xQf5g2nmK`Gs??3v9T6!bGof@f2hx}2^?WqLzEPC(F%9jBf;HlV10V9-y2)yDa@V9iLg~&+gkU(R z9D*jhcRkV5N)I1V<4h8A$kxwyxQA4nD=1rjJ2L86W?$o0zO^EU(U&AUqI}7jyzV#I zDYfb-QM&OH7$XA^YzZphkd^mICSs3Pwzu9Q2*Ef?0poz&SQGdU=avv?gr5 zgUJ=A;3&nlspRO%I+}fHSAqYKp-x;~qfP~`EUPggm#^L*LGHfzBvNxC{QWHm)L1HHH*^N&=8WxjwmWvwFDH?1mQ}rg zm!&*$tG}vmWb;?KPw;`eV$=r2zTVxRwNON-@(f#YCW7bP?)eJPtYXgiv5fqVnI5NB zejXDK?9RT0;8v`Um^TPP*=oM^_~>i~Kb|{HMYsv#BU7U0$V0Rz>x=)TXP)#>wymlh zeQFUC(ae;4I3k)og-Ai_ir&T<7IKi;A$bYoR&R=2|B{m=i}R!M*Hku<&7C>oFgH^D u{FAsvm99<@unG`yS9;F7wRuTAq)>6=-`rb^yw8MhuI!Xq$~hxXVaKmL(PZoZ literal 0 HcmV?d00001 diff --git a/packet/fixtures/synthetic/scene.json b/packet/fixtures/synthetic/scene.json new file mode 100644 index 000000000..68cb6efb7 --- /dev/null +++ b/packet/fixtures/synthetic/scene.json @@ -0,0 +1,26 @@ +{ + "schema_version": "1.0", + "meter": { + "pos": [ + 0.0, + 4.0, + 0.0 + ], + "wall_id": "w1" + }, + "walls": [ + { + "id": "w1", + "baseline": [ + [ + -9.84, + 0.0 + ], + [ + 9.84, + 0.0 + ] + ] + } + ] +} diff --git a/packet/fixtures/synthetic/streams/accelerometer.csv b/packet/fixtures/synthetic/streams/accelerometer.csv new file mode 100644 index 000000000..76fc6acc8 --- /dev/null +++ b/packet/fixtures/synthetic/streams/accelerometer.csv @@ -0,0 +1,201 @@ +t,x,y,z +100.0,-9.81,0.0,0.0 +100.01,-9.81,0.0,0.0 +100.02,-9.81,0.0,0.0 +100.03,-9.81,0.0,0.0 +100.04,-9.81,0.0,0.0 +100.05,-9.81,0.0,0.0 +100.06,-9.81,0.0,0.0 +100.07,-9.81,0.0,0.0 +100.08,-9.81,0.0,0.0 +100.09,-9.81,0.0,0.0 +100.1,-9.81,0.0,0.0 +100.11,-9.81,0.0,0.0 +100.12,-9.81,0.0,0.0 +100.13,-9.81,0.0,0.0 +100.14,-9.81,0.0,0.0 +100.15,-9.81,0.0,0.0 +100.16,-9.81,0.0,0.0 +100.17,-9.81,0.0,0.0 +100.18,-9.81,0.0,0.0 +100.19,-9.81,0.0,0.0 +100.2,-9.81,0.0,0.0 +100.21,-9.81,0.0,0.0 +100.22,-9.81,0.0,0.0 +100.23,-9.81,0.0,0.0 +100.24,-9.81,0.0,0.0 +100.25,-9.81,0.0,0.0 +100.26,-9.81,0.0,0.0 +100.27,-9.81,0.0,0.0 +100.28,-9.81,0.0,0.0 +100.29,-9.81,0.0,0.0 +100.3,-9.81,0.0,0.0 +100.31,-9.81,0.0,0.0 +100.32,-9.81,0.0,0.0 +100.33,-9.81,0.0,0.0 +100.34,-9.81,0.0,0.0 +100.35,-9.81,0.0,0.0 +100.36,-9.81,0.0,0.0 +100.37,-9.81,0.0,0.0 +100.38,-9.81,0.0,0.0 +100.39,-9.81,0.0,0.0 +100.4,-9.81,0.0,0.0 +100.41,-9.81,0.0,0.0 +100.42,-9.81,0.0,0.0 +100.43,-9.81,0.0,0.0 +100.44,-9.81,0.0,0.0 +100.45,-9.81,0.0,0.0 +100.46,-9.81,0.0,0.0 +100.47,-9.81,0.0,0.0 +100.48,-9.81,0.0,0.0 +100.49,-9.81,0.0,0.0 +100.5,-9.81,0.0,0.0 +100.51,-9.81,0.0,0.0 +100.52,-9.81,0.0,0.0 +100.53,-9.81,0.0,0.0 +100.54,-9.81,0.0,0.0 +100.55,-9.81,0.0,0.0 +100.56,-9.81,0.0,0.0 +100.57,-9.81,0.0,0.0 +100.58,-9.81,0.0,0.0 +100.59,-9.81,0.0,0.0 +100.6,-9.81,0.0,0.0 +100.61,-9.81,0.0,0.0 +100.62,-9.81,0.0,0.0 +100.63,-9.81,0.0,0.0 +100.64,-9.81,0.0,0.0 +100.65,-9.81,0.0,0.0 +100.66,-9.81,0.0,0.0 +100.67,-9.81,0.0,0.0 +100.68,-9.81,0.0,0.0 +100.69,-9.81,0.0,0.0 +100.7,-9.81,0.0,0.0 +100.71,-9.81,0.0,0.0 +100.72,-9.81,0.0,0.0 +100.73,-9.81,0.0,0.0 +100.74,-9.81,0.0,0.0 +100.75,-9.81,0.0,0.0 +100.76,-9.81,0.0,0.0 +100.77,-9.81,0.0,0.0 +100.78,-9.81,0.0,0.0 +100.79,-9.81,0.0,0.0 +100.8,-9.81,0.0,0.0 +100.81,-9.81,0.0,0.0 +100.82,-9.81,0.0,0.0 +100.83,-9.81,0.0,0.0 +100.84,-9.81,0.0,0.0 +100.85,-9.81,0.0,0.0 +100.86,-9.81,0.0,0.0 +100.87,-9.81,0.0,0.0 +100.88,-9.81,0.0,0.0 +100.89,-9.81,0.0,0.0 +100.9,-9.81,0.0,0.0 +100.91,-9.81,0.0,0.0 +100.92,-9.81,0.0,0.0 +100.93,-9.81,0.0,0.0 +100.94,-9.81,0.0,0.0 +100.95,-9.81,0.0,0.0 +100.96,-9.81,0.0,0.0 +100.97,-9.81,0.0,0.0 +100.98,-9.81,0.0,0.0 +100.99,-9.81,0.0,0.0 +101.0,-9.81,0.0,0.0 +101.01,-9.81,0.0,0.0 +101.02,-9.81,0.0,0.0 +101.03,-9.81,0.0,0.0 +101.04,-9.81,0.0,0.0 +101.05,-9.81,0.0,0.0 +101.06,-9.81,0.0,0.0 +101.07,-9.81,0.0,0.0 +101.08,-9.81,0.0,0.0 +101.09,-9.81,0.0,0.0 +101.1,-9.81,0.0,0.0 +101.11,-9.81,0.0,0.0 +101.12,-9.81,0.0,0.0 +101.13,-9.81,0.0,0.0 +101.14,-9.81,0.0,0.0 +101.15,-9.81,0.0,0.0 +101.16,-9.81,0.0,0.0 +101.17,-9.81,0.0,0.0 +101.18,-9.81,0.0,0.0 +101.19,-9.81,0.0,0.0 +101.2,-9.81,0.0,0.0 +101.21,-9.81,0.0,0.0 +101.22,-9.81,0.0,0.0 +101.23,-9.81,0.0,0.0 +101.24,-9.81,0.0,0.0 +101.25,-9.81,0.0,0.0 +101.26,-9.81,0.0,0.0 +101.27,-9.81,0.0,0.0 +101.28,-9.81,0.0,0.0 +101.29,-9.81,0.0,0.0 +101.3,-9.81,0.0,0.0 +101.31,-9.81,0.0,0.0 +101.32,-9.81,0.0,0.0 +101.33,-9.81,0.0,0.0 +101.34,-9.81,0.0,0.0 +101.35,-9.81,0.0,0.0 +101.36,-9.81,0.0,0.0 +101.37,-9.81,0.0,0.0 +101.38,-9.81,0.0,0.0 +101.39,-9.81,0.0,0.0 +101.4,-9.81,0.0,0.0 +101.41,-9.81,0.0,0.0 +101.42,-9.81,0.0,0.0 +101.43,-9.81,0.0,0.0 +101.44,-9.81,0.0,0.0 +101.45,-9.81,0.0,0.0 +101.46,-9.81,0.0,0.0 +101.47,-9.81,0.0,0.0 +101.48,-9.81,0.0,0.0 +101.49,-9.81,0.0,0.0 +101.5,-9.81,0.0,0.0 +101.51,-9.81,0.0,0.0 +101.52,-9.81,0.0,0.0 +101.53,-9.81,0.0,0.0 +101.54,-9.81,0.0,0.0 +101.55,-9.81,0.0,0.0 +101.56,-9.81,0.0,0.0 +101.57,-9.81,0.0,0.0 +101.58,-9.81,0.0,0.0 +101.59,-9.81,0.0,0.0 +101.6,-9.81,0.0,0.0 +101.61,-9.81,0.0,0.0 +101.62,-9.81,0.0,0.0 +101.63,-9.81,0.0,0.0 +101.64,-9.81,0.0,0.0 +101.65,-9.81,0.0,0.0 +101.66,-9.81,0.0,0.0 +101.67,-9.81,0.0,0.0 +101.68,-9.81,0.0,0.0 +101.69,-9.81,0.0,0.0 +101.7,-9.81,0.0,0.0 +101.71,-9.81,0.0,0.0 +101.72,-9.81,0.0,0.0 +101.73,-9.81,0.0,0.0 +101.74,-9.81,0.0,0.0 +101.75,-9.81,0.0,0.0 +101.76,-9.81,0.0,0.0 +101.77,-9.81,0.0,0.0 +101.78,-9.81,0.0,0.0 +101.79,-9.81,0.0,0.0 +101.8,-9.81,0.0,0.0 +101.81,-9.81,0.0,0.0 +101.82,-9.81,0.0,0.0 +101.83,-9.81,0.0,0.0 +101.84,-9.81,0.0,0.0 +101.85,-9.81,0.0,0.0 +101.86,-9.81,0.0,0.0 +101.87,-9.81,0.0,0.0 +101.88,-9.81,0.0,0.0 +101.89,-9.81,0.0,0.0 +101.9,-9.81,0.0,0.0 +101.91,-9.81,0.0,0.0 +101.92,-9.81,0.0,0.0 +101.93,-9.81,0.0,0.0 +101.94,-9.81,0.0,0.0 +101.95,-9.81,0.0,0.0 +101.96,-9.81,0.0,0.0 +101.97,-9.81,0.0,0.0 +101.98,-9.81,0.0,0.0 +101.99,-9.81,0.0,0.0 diff --git a/packet/fixtures/synthetic/streams/barometer.csv b/packet/fixtures/synthetic/streams/barometer.csv new file mode 100644 index 000000000..beae7636f --- /dev/null +++ b/packet/fixtures/synthetic/streams/barometer.csv @@ -0,0 +1,3 @@ +t,pressure_kpa,relative_altitude_m +100.5,101.3,0.0 +101.5,101.3,0.0 diff --git a/packet/fixtures/synthetic/streams/device_motion.csv b/packet/fixtures/synthetic/streams/device_motion.csv new file mode 100644 index 000000000..c39486f10 --- /dev/null +++ b/packet/fixtures/synthetic/streams/device_motion.csv @@ -0,0 +1,201 @@ +t,qx,qy,qz,qw,gravity_x,gravity_y,gravity_z,user_accel_x,user_accel_y,user_accel_z,rotation_rate_x,rotation_rate_y,rotation_rate_z,heading_deg +100.0,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.01,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.02,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.03,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.04,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.05,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.06,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.07,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.08,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.09,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.1,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.11,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.12,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.13,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.14,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.15,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.16,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.17,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.18,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.19,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.2,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.21,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.22,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.23,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.24,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.25,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.26,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.27,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.28,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.29,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.3,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.31,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.32,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.33,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.34,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.35,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.36,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.37,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.38,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.39,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.4,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.41,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.42,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.43,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.44,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.45,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.46,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.47,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.48,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.49,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.5,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.51,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.52,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.53,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.54,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.55,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.56,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.57,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.58,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.59,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.6,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.61,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.62,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.63,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.64,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.65,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.66,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.67,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.68,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.69,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.7,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.71,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.72,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.73,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.74,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.75,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.76,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.77,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.78,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.79,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.8,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.81,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.82,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.83,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.84,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.85,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.86,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.87,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.88,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.89,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.9,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.91,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.92,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.93,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.94,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.95,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.96,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.97,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.98,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +100.99,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.0,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.01,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.02,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.03,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.04,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.05,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.06,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.07,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.08,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.09,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.1,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.11,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.12,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.13,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.14,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.15,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.16,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.17,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.18,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.19,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.2,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.21,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.22,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.23,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.24,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.25,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.26,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.27,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.28,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.29,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.3,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.31,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.32,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.33,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.34,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.35,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.36,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.37,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.38,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.39,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.4,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.41,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.42,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.43,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.44,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.45,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.46,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.47,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.48,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.49,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.5,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.51,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.52,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.53,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.54,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.55,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.56,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.57,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.58,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.59,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.6,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.61,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.62,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.63,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.64,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.65,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.66,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.67,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.68,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.69,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.7,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.71,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.72,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.73,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.74,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.75,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.76,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.77,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.78,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.79,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.8,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.81,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.82,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.83,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.84,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.85,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.86,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.87,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.88,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.89,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.9,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.91,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.92,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.93,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.94,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.95,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.96,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.97,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.98,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 +101.99,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 diff --git a/packet/fixtures/synthetic/streams/gyroscope.csv b/packet/fixtures/synthetic/streams/gyroscope.csv new file mode 100644 index 000000000..52c5a41ec --- /dev/null +++ b/packet/fixtures/synthetic/streams/gyroscope.csv @@ -0,0 +1,201 @@ +t,x,y,z +100.0,0.0,0.0,0.0 +100.01,0.0,0.0,0.0 +100.02,0.0,0.0,0.0 +100.03,0.0,0.0,0.0 +100.04,0.0,0.0,0.0 +100.05,0.0,0.0,0.0 +100.06,0.0,0.0,0.0 +100.07,0.0,0.0,0.0 +100.08,0.0,0.0,0.0 +100.09,0.0,0.0,0.0 +100.1,0.0,0.0,0.0 +100.11,0.0,0.0,0.0 +100.12,0.0,0.0,0.0 +100.13,0.0,0.0,0.0 +100.14,0.0,0.0,0.0 +100.15,0.0,0.0,0.0 +100.16,0.0,0.0,0.0 +100.17,0.0,0.0,0.0 +100.18,0.0,0.0,0.0 +100.19,0.0,0.0,0.0 +100.2,0.0,0.0,0.0 +100.21,0.0,0.0,0.0 +100.22,0.0,0.0,0.0 +100.23,0.0,0.0,0.0 +100.24,0.0,0.0,0.0 +100.25,0.0,0.0,0.0 +100.26,0.0,0.0,0.0 +100.27,0.0,0.0,0.0 +100.28,0.0,0.0,0.0 +100.29,0.0,0.0,0.0 +100.3,0.0,0.0,0.0 +100.31,0.0,0.0,0.0 +100.32,0.0,0.0,0.0 +100.33,0.0,0.0,0.0 +100.34,0.0,0.0,0.0 +100.35,0.0,0.0,0.0 +100.36,0.0,0.0,0.0 +100.37,0.0,0.0,0.0 +100.38,0.0,0.0,0.0 +100.39,0.0,0.0,0.0 +100.4,0.0,0.0,0.0 +100.41,0.0,0.0,0.0 +100.42,0.0,0.0,0.0 +100.43,0.0,0.0,0.0 +100.44,0.0,0.0,0.0 +100.45,0.0,0.0,0.0 +100.46,0.0,0.0,0.0 +100.47,0.0,0.0,0.0 +100.48,0.0,0.0,0.0 +100.49,0.0,0.0,0.0 +100.5,0.0,0.0,0.0 +100.51,0.0,0.0,0.0 +100.52,0.0,0.0,0.0 +100.53,0.0,0.0,0.0 +100.54,0.0,0.0,0.0 +100.55,0.0,0.0,0.0 +100.56,0.0,0.0,0.0 +100.57,0.0,0.0,0.0 +100.58,0.0,0.0,0.0 +100.59,0.0,0.0,0.0 +100.6,0.0,0.0,0.0 +100.61,0.0,0.0,0.0 +100.62,0.0,0.0,0.0 +100.63,0.0,0.0,0.0 +100.64,0.0,0.0,0.0 +100.65,0.0,0.0,0.0 +100.66,0.0,0.0,0.0 +100.67,0.0,0.0,0.0 +100.68,0.0,0.0,0.0 +100.69,0.0,0.0,0.0 +100.7,0.0,0.0,0.0 +100.71,0.0,0.0,0.0 +100.72,0.0,0.0,0.0 +100.73,0.0,0.0,0.0 +100.74,0.0,0.0,0.0 +100.75,0.0,0.0,0.0 +100.76,0.0,0.0,0.0 +100.77,0.0,0.0,0.0 +100.78,0.0,0.0,0.0 +100.79,0.0,0.0,0.0 +100.8,0.0,0.0,0.0 +100.81,0.0,0.0,0.0 +100.82,0.0,0.0,0.0 +100.83,0.0,0.0,0.0 +100.84,0.0,0.0,0.0 +100.85,0.0,0.0,0.0 +100.86,0.0,0.0,0.0 +100.87,0.0,0.0,0.0 +100.88,0.0,0.0,0.0 +100.89,0.0,0.0,0.0 +100.9,0.0,0.0,0.0 +100.91,0.0,0.0,0.0 +100.92,0.0,0.0,0.0 +100.93,0.0,0.0,0.0 +100.94,0.0,0.0,0.0 +100.95,0.0,0.0,0.0 +100.96,0.0,0.0,0.0 +100.97,0.0,0.0,0.0 +100.98,0.0,0.0,0.0 +100.99,0.0,0.0,0.0 +101.0,0.0,0.0,0.0 +101.01,0.0,0.0,0.0 +101.02,0.0,0.0,0.0 +101.03,0.0,0.0,0.0 +101.04,0.0,0.0,0.0 +101.05,0.0,0.0,0.0 +101.06,0.0,0.0,0.0 +101.07,0.0,0.0,0.0 +101.08,0.0,0.0,0.0 +101.09,0.0,0.0,0.0 +101.1,0.0,0.0,0.0 +101.11,0.0,0.0,0.0 +101.12,0.0,0.0,0.0 +101.13,0.0,0.0,0.0 +101.14,0.0,0.0,0.0 +101.15,0.0,0.0,0.0 +101.16,0.0,0.0,0.0 +101.17,0.0,0.0,0.0 +101.18,0.0,0.0,0.0 +101.19,0.0,0.0,0.0 +101.2,0.0,0.0,0.0 +101.21,0.0,0.0,0.0 +101.22,0.0,0.0,0.0 +101.23,0.0,0.0,0.0 +101.24,0.0,0.0,0.0 +101.25,0.0,0.0,0.0 +101.26,0.0,0.0,0.0 +101.27,0.0,0.0,0.0 +101.28,0.0,0.0,0.0 +101.29,0.0,0.0,0.0 +101.3,0.0,0.0,0.0 +101.31,0.0,0.0,0.0 +101.32,0.0,0.0,0.0 +101.33,0.0,0.0,0.0 +101.34,0.0,0.0,0.0 +101.35,0.0,0.0,0.0 +101.36,0.0,0.0,0.0 +101.37,0.0,0.0,0.0 +101.38,0.0,0.0,0.0 +101.39,0.0,0.0,0.0 +101.4,0.0,0.0,0.0 +101.41,0.0,0.0,0.0 +101.42,0.0,0.0,0.0 +101.43,0.0,0.0,0.0 +101.44,0.0,0.0,0.0 +101.45,0.0,0.0,0.0 +101.46,0.0,0.0,0.0 +101.47,0.0,0.0,0.0 +101.48,0.0,0.0,0.0 +101.49,0.0,0.0,0.0 +101.5,0.0,0.0,0.0 +101.51,0.0,0.0,0.0 +101.52,0.0,0.0,0.0 +101.53,0.0,0.0,0.0 +101.54,0.0,0.0,0.0 +101.55,0.0,0.0,0.0 +101.56,0.0,0.0,0.0 +101.57,0.0,0.0,0.0 +101.58,0.0,0.0,0.0 +101.59,0.0,0.0,0.0 +101.6,0.0,0.0,0.0 +101.61,0.0,0.0,0.0 +101.62,0.0,0.0,0.0 +101.63,0.0,0.0,0.0 +101.64,0.0,0.0,0.0 +101.65,0.0,0.0,0.0 +101.66,0.0,0.0,0.0 +101.67,0.0,0.0,0.0 +101.68,0.0,0.0,0.0 +101.69,0.0,0.0,0.0 +101.7,0.0,0.0,0.0 +101.71,0.0,0.0,0.0 +101.72,0.0,0.0,0.0 +101.73,0.0,0.0,0.0 +101.74,0.0,0.0,0.0 +101.75,0.0,0.0,0.0 +101.76,0.0,0.0,0.0 +101.77,0.0,0.0,0.0 +101.78,0.0,0.0,0.0 +101.79,0.0,0.0,0.0 +101.8,0.0,0.0,0.0 +101.81,0.0,0.0,0.0 +101.82,0.0,0.0,0.0 +101.83,0.0,0.0,0.0 +101.84,0.0,0.0,0.0 +101.85,0.0,0.0,0.0 +101.86,0.0,0.0,0.0 +101.87,0.0,0.0,0.0 +101.88,0.0,0.0,0.0 +101.89,0.0,0.0,0.0 +101.9,0.0,0.0,0.0 +101.91,0.0,0.0,0.0 +101.92,0.0,0.0,0.0 +101.93,0.0,0.0,0.0 +101.94,0.0,0.0,0.0 +101.95,0.0,0.0,0.0 +101.96,0.0,0.0,0.0 +101.97,0.0,0.0,0.0 +101.98,0.0,0.0,0.0 +101.99,0.0,0.0,0.0 diff --git a/packet/fixtures/synthetic/streams/heading.csv b/packet/fixtures/synthetic/streams/heading.csv new file mode 100644 index 000000000..e510f1870 --- /dev/null +++ b/packet/fixtures/synthetic/streams/heading.csv @@ -0,0 +1,2 @@ +t,magnetic_deg,true_deg,accuracy_deg +100.5,90.0,90.0,10.0 diff --git a/packet/fixtures/synthetic/streams/location.csv b/packet/fixtures/synthetic/streams/location.csv new file mode 100644 index 000000000..b4f4579f3 --- /dev/null +++ b/packet/fixtures/synthetic/streams/location.csv @@ -0,0 +1,2 @@ +t,latitude,longitude,altitude_m,horizontal_accuracy_m,vertical_accuracy_m +100.5,0.0,0.0,10.0,5.0,3.0 diff --git a/packet/fixtures/synthetic/streams/magnetometer.csv b/packet/fixtures/synthetic/streams/magnetometer.csv new file mode 100644 index 000000000..c467e0a56 --- /dev/null +++ b/packet/fixtures/synthetic/streams/magnetometer.csv @@ -0,0 +1,201 @@ +t,x,y,z +100.0,20.0,-5.0,-40.0 +100.01,20.0,-5.0,-40.0 +100.02,20.0,-5.0,-40.0 +100.03,20.0,-5.0,-40.0 +100.04,20.0,-5.0,-40.0 +100.05,20.0,-5.0,-40.0 +100.06,20.0,-5.0,-40.0 +100.07,20.0,-5.0,-40.0 +100.08,20.0,-5.0,-40.0 +100.09,20.0,-5.0,-40.0 +100.1,20.0,-5.0,-40.0 +100.11,20.0,-5.0,-40.0 +100.12,20.0,-5.0,-40.0 +100.13,20.0,-5.0,-40.0 +100.14,20.0,-5.0,-40.0 +100.15,20.0,-5.0,-40.0 +100.16,20.0,-5.0,-40.0 +100.17,20.0,-5.0,-40.0 +100.18,20.0,-5.0,-40.0 +100.19,20.0,-5.0,-40.0 +100.2,20.0,-5.0,-40.0 +100.21,20.0,-5.0,-40.0 +100.22,20.0,-5.0,-40.0 +100.23,20.0,-5.0,-40.0 +100.24,20.0,-5.0,-40.0 +100.25,20.0,-5.0,-40.0 +100.26,20.0,-5.0,-40.0 +100.27,20.0,-5.0,-40.0 +100.28,20.0,-5.0,-40.0 +100.29,20.0,-5.0,-40.0 +100.3,20.0,-5.0,-40.0 +100.31,20.0,-5.0,-40.0 +100.32,20.0,-5.0,-40.0 +100.33,20.0,-5.0,-40.0 +100.34,20.0,-5.0,-40.0 +100.35,20.0,-5.0,-40.0 +100.36,20.0,-5.0,-40.0 +100.37,20.0,-5.0,-40.0 +100.38,20.0,-5.0,-40.0 +100.39,20.0,-5.0,-40.0 +100.4,20.0,-5.0,-40.0 +100.41,20.0,-5.0,-40.0 +100.42,20.0,-5.0,-40.0 +100.43,20.0,-5.0,-40.0 +100.44,20.0,-5.0,-40.0 +100.45,20.0,-5.0,-40.0 +100.46,20.0,-5.0,-40.0 +100.47,20.0,-5.0,-40.0 +100.48,20.0,-5.0,-40.0 +100.49,20.0,-5.0,-40.0 +100.5,20.0,-5.0,-40.0 +100.51,20.0,-5.0,-40.0 +100.52,20.0,-5.0,-40.0 +100.53,20.0,-5.0,-40.0 +100.54,20.0,-5.0,-40.0 +100.55,20.0,-5.0,-40.0 +100.56,20.0,-5.0,-40.0 +100.57,20.0,-5.0,-40.0 +100.58,20.0,-5.0,-40.0 +100.59,20.0,-5.0,-40.0 +100.6,20.0,-5.0,-40.0 +100.61,20.0,-5.0,-40.0 +100.62,20.0,-5.0,-40.0 +100.63,20.0,-5.0,-40.0 +100.64,20.0,-5.0,-40.0 +100.65,20.0,-5.0,-40.0 +100.66,20.0,-5.0,-40.0 +100.67,20.0,-5.0,-40.0 +100.68,20.0,-5.0,-40.0 +100.69,20.0,-5.0,-40.0 +100.7,20.0,-5.0,-40.0 +100.71,20.0,-5.0,-40.0 +100.72,20.0,-5.0,-40.0 +100.73,20.0,-5.0,-40.0 +100.74,20.0,-5.0,-40.0 +100.75,20.0,-5.0,-40.0 +100.76,20.0,-5.0,-40.0 +100.77,20.0,-5.0,-40.0 +100.78,20.0,-5.0,-40.0 +100.79,20.0,-5.0,-40.0 +100.8,20.0,-5.0,-40.0 +100.81,20.0,-5.0,-40.0 +100.82,20.0,-5.0,-40.0 +100.83,20.0,-5.0,-40.0 +100.84,20.0,-5.0,-40.0 +100.85,20.0,-5.0,-40.0 +100.86,20.0,-5.0,-40.0 +100.87,20.0,-5.0,-40.0 +100.88,20.0,-5.0,-40.0 +100.89,20.0,-5.0,-40.0 +100.9,20.0,-5.0,-40.0 +100.91,20.0,-5.0,-40.0 +100.92,20.0,-5.0,-40.0 +100.93,20.0,-5.0,-40.0 +100.94,20.0,-5.0,-40.0 +100.95,20.0,-5.0,-40.0 +100.96,20.0,-5.0,-40.0 +100.97,20.0,-5.0,-40.0 +100.98,20.0,-5.0,-40.0 +100.99,20.0,-5.0,-40.0 +101.0,20.0,-5.0,-40.0 +101.01,20.0,-5.0,-40.0 +101.02,20.0,-5.0,-40.0 +101.03,20.0,-5.0,-40.0 +101.04,20.0,-5.0,-40.0 +101.05,20.0,-5.0,-40.0 +101.06,20.0,-5.0,-40.0 +101.07,20.0,-5.0,-40.0 +101.08,20.0,-5.0,-40.0 +101.09,20.0,-5.0,-40.0 +101.1,20.0,-5.0,-40.0 +101.11,20.0,-5.0,-40.0 +101.12,20.0,-5.0,-40.0 +101.13,20.0,-5.0,-40.0 +101.14,20.0,-5.0,-40.0 +101.15,20.0,-5.0,-40.0 +101.16,20.0,-5.0,-40.0 +101.17,20.0,-5.0,-40.0 +101.18,20.0,-5.0,-40.0 +101.19,20.0,-5.0,-40.0 +101.2,20.0,-5.0,-40.0 +101.21,20.0,-5.0,-40.0 +101.22,20.0,-5.0,-40.0 +101.23,20.0,-5.0,-40.0 +101.24,20.0,-5.0,-40.0 +101.25,20.0,-5.0,-40.0 +101.26,20.0,-5.0,-40.0 +101.27,20.0,-5.0,-40.0 +101.28,20.0,-5.0,-40.0 +101.29,20.0,-5.0,-40.0 +101.3,20.0,-5.0,-40.0 +101.31,20.0,-5.0,-40.0 +101.32,20.0,-5.0,-40.0 +101.33,20.0,-5.0,-40.0 +101.34,20.0,-5.0,-40.0 +101.35,20.0,-5.0,-40.0 +101.36,20.0,-5.0,-40.0 +101.37,20.0,-5.0,-40.0 +101.38,20.0,-5.0,-40.0 +101.39,20.0,-5.0,-40.0 +101.4,20.0,-5.0,-40.0 +101.41,20.0,-5.0,-40.0 +101.42,20.0,-5.0,-40.0 +101.43,20.0,-5.0,-40.0 +101.44,20.0,-5.0,-40.0 +101.45,20.0,-5.0,-40.0 +101.46,20.0,-5.0,-40.0 +101.47,20.0,-5.0,-40.0 +101.48,20.0,-5.0,-40.0 +101.49,20.0,-5.0,-40.0 +101.5,20.0,-5.0,-40.0 +101.51,20.0,-5.0,-40.0 +101.52,20.0,-5.0,-40.0 +101.53,20.0,-5.0,-40.0 +101.54,20.0,-5.0,-40.0 +101.55,20.0,-5.0,-40.0 +101.56,20.0,-5.0,-40.0 +101.57,20.0,-5.0,-40.0 +101.58,20.0,-5.0,-40.0 +101.59,20.0,-5.0,-40.0 +101.6,20.0,-5.0,-40.0 +101.61,20.0,-5.0,-40.0 +101.62,20.0,-5.0,-40.0 +101.63,20.0,-5.0,-40.0 +101.64,20.0,-5.0,-40.0 +101.65,20.0,-5.0,-40.0 +101.66,20.0,-5.0,-40.0 +101.67,20.0,-5.0,-40.0 +101.68,20.0,-5.0,-40.0 +101.69,20.0,-5.0,-40.0 +101.7,20.0,-5.0,-40.0 +101.71,20.0,-5.0,-40.0 +101.72,20.0,-5.0,-40.0 +101.73,20.0,-5.0,-40.0 +101.74,20.0,-5.0,-40.0 +101.75,20.0,-5.0,-40.0 +101.76,20.0,-5.0,-40.0 +101.77,20.0,-5.0,-40.0 +101.78,20.0,-5.0,-40.0 +101.79,20.0,-5.0,-40.0 +101.8,20.0,-5.0,-40.0 +101.81,20.0,-5.0,-40.0 +101.82,20.0,-5.0,-40.0 +101.83,20.0,-5.0,-40.0 +101.84,20.0,-5.0,-40.0 +101.85,20.0,-5.0,-40.0 +101.86,20.0,-5.0,-40.0 +101.87,20.0,-5.0,-40.0 +101.88,20.0,-5.0,-40.0 +101.89,20.0,-5.0,-40.0 +101.9,20.0,-5.0,-40.0 +101.91,20.0,-5.0,-40.0 +101.92,20.0,-5.0,-40.0 +101.93,20.0,-5.0,-40.0 +101.94,20.0,-5.0,-40.0 +101.95,20.0,-5.0,-40.0 +101.96,20.0,-5.0,-40.0 +101.97,20.0,-5.0,-40.0 +101.98,20.0,-5.0,-40.0 +101.99,20.0,-5.0,-40.0 diff --git a/packet/fixtures/synthetic/streams/trajectory.csv b/packet/fixtures/synthetic/streams/trajectory.csv new file mode 100644 index 000000000..8ae4cca9c --- /dev/null +++ b/packet/fixtures/synthetic/streams/trajectory.csv @@ -0,0 +1,122 @@ +t,tracking,px,py,pz,qx,qy,qz,qw +100.0,normal,-1.0,0.0,2.5,0.0,0.0,0.0,1.0 +100.016667,normal,-0.983333,0.0,2.5,0.0,0.0,0.0,1.0 +100.033333,normal,-0.966667,0.0,2.5,0.0,0.0,0.0,1.0 +100.05,normal,-0.95,0.0,2.5,0.0,0.0,0.0,1.0 +100.066667,normal,-0.933333,0.0,2.5,0.0,0.0,0.0,1.0 +100.083333,normal,-0.916667,0.0,2.5,0.0,0.0,0.0,1.0 +100.1,normal,-0.9,0.0,2.5,0.0,0.0,0.0,1.0 +100.116667,normal,-0.883333,0.0,2.5,0.0,0.0,0.0,1.0 +100.133333,normal,-0.866667,0.0,2.5,0.0,0.0,0.0,1.0 +100.15,normal,-0.85,0.0,2.5,0.0,0.0,0.0,1.0 +100.166667,normal,-0.833333,0.0,2.5,0.0,0.0,0.0,1.0 +100.183333,normal,-0.816667,0.0,2.5,0.0,0.0,0.0,1.0 +100.2,normal,-0.8,0.0,2.5,0.0,0.0,0.0,1.0 +100.216667,normal,-0.783333,0.0,2.5,0.0,0.0,0.0,1.0 +100.233333,normal,-0.766667,0.0,2.5,0.0,0.0,0.0,1.0 +100.25,normal,-0.75,0.0,2.5,0.0,0.0,0.0,1.0 +100.266667,normal,-0.733333,0.0,2.5,0.0,0.0,0.0,1.0 +100.283333,normal,-0.716667,0.0,2.5,0.0,0.0,0.0,1.0 +100.3,normal,-0.7,0.0,2.5,0.0,0.0,0.0,1.0 +100.316667,normal,-0.683333,0.0,2.5,0.0,0.0,0.0,1.0 +100.333333,normal,-0.666667,0.0,2.5,0.0,0.0,0.0,1.0 +100.35,normal,-0.65,0.0,2.5,0.0,0.0,0.0,1.0 +100.366667,normal,-0.633333,0.0,2.5,0.0,0.0,0.0,1.0 +100.383333,normal,-0.616667,0.0,2.5,0.0,0.0,0.0,1.0 +100.4,normal,-0.6,0.0,2.5,0.0,0.0,0.0,1.0 +100.416667,normal,-0.583333,0.0,2.5,0.0,0.0,0.0,1.0 +100.433333,normal,-0.566667,0.0,2.5,0.0,0.0,0.0,1.0 +100.45,normal,-0.55,0.0,2.5,0.0,0.0,0.0,1.0 +100.466667,normal,-0.533333,0.0,2.5,0.0,0.0,0.0,1.0 +100.483333,normal,-0.516667,0.0,2.5,0.0,0.0,0.0,1.0 +100.5,normal,-0.5,0.0,2.5,0.0,0.0,0.0,1.0 +100.516667,normal,-0.483333,0.0,2.5,0.0,0.0,0.0,1.0 +100.533333,normal,-0.466667,0.0,2.5,0.0,0.0,0.0,1.0 +100.55,normal,-0.45,0.0,2.5,0.0,0.0,0.0,1.0 +100.566667,normal,-0.433333,0.0,2.5,0.0,0.0,0.0,1.0 +100.583333,normal,-0.416667,0.0,2.5,0.0,0.0,0.0,1.0 +100.6,normal,-0.4,0.0,2.5,0.0,0.0,0.0,1.0 +100.616667,normal,-0.383333,0.0,2.5,0.0,0.0,0.0,1.0 +100.633333,normal,-0.366667,0.0,2.5,0.0,0.0,0.0,1.0 +100.65,normal,-0.35,0.0,2.5,0.0,0.0,0.0,1.0 +100.666667,normal,-0.333333,0.0,2.5,0.0,0.0,0.0,1.0 +100.683333,normal,-0.316667,0.0,2.5,0.0,0.0,0.0,1.0 +100.7,normal,-0.3,0.0,2.5,0.0,0.0,0.0,1.0 +100.716667,normal,-0.283333,0.0,2.5,0.0,0.0,0.0,1.0 +100.733333,normal,-0.266667,0.0,2.5,0.0,0.0,0.0,1.0 +100.75,normal,-0.25,0.0,2.5,0.0,0.0,0.0,1.0 +100.766667,normal,-0.233333,0.0,2.5,0.0,0.0,0.0,1.0 +100.783333,normal,-0.216667,0.0,2.5,0.0,0.0,0.0,1.0 +100.8,normal,-0.2,0.0,2.5,0.0,0.0,0.0,1.0 +100.816667,normal,-0.183333,0.0,2.5,0.0,0.0,0.0,1.0 +100.833333,normal,-0.166667,0.0,2.5,0.0,0.0,0.0,1.0 +100.85,normal,-0.15,0.0,2.5,0.0,0.0,0.0,1.0 +100.866667,normal,-0.133333,0.0,2.5,0.0,0.0,0.0,1.0 +100.883333,normal,-0.116667,0.0,2.5,0.0,0.0,0.0,1.0 +100.9,normal,-0.1,0.0,2.5,0.0,0.0,0.0,1.0 +100.916667,normal,-0.083333,0.0,2.5,0.0,0.0,0.0,1.0 +100.933333,normal,-0.066667,0.0,2.5,0.0,0.0,0.0,1.0 +100.95,normal,-0.05,0.0,2.5,0.0,0.0,0.0,1.0 +100.966667,normal,-0.033333,0.0,2.5,0.0,0.0,0.0,1.0 +100.983333,normal,-0.016667,0.0,2.5,0.0,0.0,0.0,1.0 +101.0,normal,0.0,0.0,2.5,0.0,0.0,0.0,1.0 +101.016667,normal,0.016667,0.0,2.5,0.0,0.0,0.0,1.0 +101.033333,normal,0.033333,0.0,2.5,0.0,0.0,0.0,1.0 +101.05,normal,0.05,0.0,2.5,0.0,0.0,0.0,1.0 +101.066667,normal,0.066667,0.0,2.5,0.0,0.0,0.0,1.0 +101.083333,normal,0.083333,0.0,2.5,0.0,0.0,0.0,1.0 +101.1,normal,0.1,0.0,2.5,0.0,0.0,0.0,1.0 +101.116667,normal,0.116667,0.0,2.5,0.0,0.0,0.0,1.0 +101.133333,normal,0.133333,0.0,2.5,0.0,0.0,0.0,1.0 +101.15,normal,0.15,0.0,2.5,0.0,0.0,0.0,1.0 +101.166667,normal,0.166667,0.0,2.5,0.0,0.0,0.0,1.0 +101.183333,normal,0.183333,0.0,2.5,0.0,0.0,0.0,1.0 +101.2,normal,0.2,0.0,2.5,0.0,0.0,0.0,1.0 +101.216667,normal,0.216667,0.0,2.5,0.0,0.0,0.0,1.0 +101.233333,normal,0.233333,0.0,2.5,0.0,0.0,0.0,1.0 +101.25,normal,0.25,0.0,2.5,0.0,0.0,0.0,1.0 +101.266667,normal,0.266667,0.0,2.5,0.0,0.0,0.0,1.0 +101.283333,normal,0.283333,0.0,2.5,0.0,0.0,0.0,1.0 +101.3,normal,0.3,0.0,2.5,0.0,0.0,0.0,1.0 +101.316667,normal,0.316667,0.0,2.5,0.0,0.0,0.0,1.0 +101.333333,normal,0.333333,0.0,2.5,0.0,0.0,0.0,1.0 +101.35,normal,0.35,0.0,2.5,0.0,0.0,0.0,1.0 +101.366667,normal,0.366667,0.0,2.5,0.0,0.0,0.0,1.0 +101.383333,normal,0.383333,0.0,2.5,0.0,0.0,0.0,1.0 +101.4,normal,0.4,0.0,2.5,0.0,0.0,0.0,1.0 +101.416667,normal,0.416667,0.0,2.5,0.0,0.0,0.0,1.0 +101.433333,normal,0.433333,0.0,2.5,0.0,0.0,0.0,1.0 +101.45,normal,0.45,0.0,2.5,0.0,0.0,0.0,1.0 +101.466667,normal,0.466667,0.0,2.5,0.0,0.0,0.0,1.0 +101.483333,normal,0.483333,0.0,2.5,0.0,0.0,0.0,1.0 +101.5,normal,0.5,0.0,2.5,0.0,0.0,0.0,1.0 +101.516667,normal,0.516667,0.0,2.5,0.0,0.0,0.0,1.0 +101.533333,normal,0.533333,0.0,2.5,0.0,0.0,0.0,1.0 +101.55,normal,0.55,0.0,2.5,0.0,0.0,0.0,1.0 +101.566667,normal,0.566667,0.0,2.5,0.0,0.0,0.0,1.0 +101.583333,normal,0.583333,0.0,2.5,0.0,0.0,0.0,1.0 +101.6,normal,0.6,0.0,2.5,0.0,0.0,0.0,1.0 +101.616667,normal,0.616667,0.0,2.5,0.0,0.0,0.0,1.0 +101.633333,normal,0.633333,0.0,2.5,0.0,0.0,0.0,1.0 +101.65,normal,0.65,0.0,2.5,0.0,0.0,0.0,1.0 +101.666667,normal,0.666667,0.0,2.5,0.0,0.0,0.0,1.0 +101.683333,normal,0.683333,0.0,2.5,0.0,0.0,0.0,1.0 +101.7,normal,0.7,0.0,2.5,0.0,0.0,0.0,1.0 +101.716667,normal,0.716667,0.0,2.5,0.0,0.0,0.0,1.0 +101.733333,normal,0.733333,0.0,2.5,0.0,0.0,0.0,1.0 +101.75,normal,0.75,0.0,2.5,0.0,0.0,0.0,1.0 +101.766667,normal,0.766667,0.0,2.5,0.0,0.0,0.0,1.0 +101.783333,normal,0.783333,0.0,2.5,0.0,0.0,0.0,1.0 +101.8,normal,0.8,0.0,2.5,0.0,0.0,0.0,1.0 +101.816667,normal,0.816667,0.0,2.5,0.0,0.0,0.0,1.0 +101.833333,normal,0.833333,0.0,2.5,0.0,0.0,0.0,1.0 +101.85,normal,0.85,0.0,2.5,0.0,0.0,0.0,1.0 +101.866667,normal,0.866667,0.0,2.5,0.0,0.0,0.0,1.0 +101.883333,normal,0.883333,0.0,2.5,0.0,0.0,0.0,1.0 +101.9,normal,0.9,0.0,2.5,0.0,0.0,0.0,1.0 +101.916667,normal,0.916667,0.0,2.5,0.0,0.0,0.0,1.0 +101.933333,normal,0.933333,0.0,2.5,0.0,0.0,0.0,1.0 +101.95,normal,0.95,0.0,2.5,0.0,0.0,0.0,1.0 +101.966667,normal,0.966667,0.0,2.5,0.0,0.0,0.0,1.0 +101.983333,normal,0.983333,0.0,2.5,0.0,0.0,0.0,1.0 +102.0,normal,1.0,0.0,2.5,0.0,0.0,0.0,1.0 diff --git a/packet/manifest.schema.json b/packet/manifest.schema.json new file mode 100644 index 000000000..ae35cc58e --- /dev/null +++ b/packet/manifest.schema.json @@ -0,0 +1,318 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://github.com/SamGu-NRX/house-scanning-master/packet/manifest.schema.json", + "title": "Capture packet manifest", + "description": "One scan session: the files in the packet and what they mean. README.md in this folder is the full specification; this schema checks shape only. Lengths in meters, times in seconds of device uptime, poses as 16 numbers column by column.", + "type": "object", + "additionalProperties": true, + "required": ["packet_version", "session", "photos"], + "properties": { + "packet_version": { + "description": "MAJOR.MINOR. A reader accepts any packet with its own major version and ignores fields it does not know.", + "type": "string", + "pattern": "^1\\.[0-9]+$" + }, + "session": { "$ref": "#/$defs/session" }, + "photos": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/$defs/photo" } + }, + "streams": { "$ref": "#/$defs/streams" }, + "lidar": { "$ref": "#/$defs/lidar" }, + "marks": { + "type": "array", + "items": { "$ref": "#/$defs/mark" } + }, + "guidance": { + "type": "array", + "items": { "$ref": "#/$defs/guidance_event" } + }, + "scene": { + "description": "The app's own scene.json (C1), exactly as it would be sent for placement.", + "allOf": [{ "$ref": "#/$defs/file" }], + "type": "object", + "properties": { "schema_version": { "type": "string" } } + }, + "provenance": { "$ref": "#/$defs/provenance" } + }, + "$defs": { + "file": { + "description": "A file inside the packet. `path` is relative to the manifest, uses '/', and never leaves the packet.", + "type": "object", + "required": ["path", "bytes", "sha256"], + "properties": { + "path": { + "type": "string", + "minLength": 1, + "pattern": "^(?!/)(?!.*(^|/)\\.\\.(/|$))[A-Za-z0-9._/-]+$" + }, + "bytes": { "type": "integer", "minimum": 0 }, + "sha256": { "type": "string", "pattern": "^[0-9a-f]{64}$" } + } + }, + "pose": { + "description": "Rigid 4x4 transform, 16 numbers column by column (simd_float4x4 layout); translation in meters.", + "type": "array", + "minItems": 16, + "maxItems": 16, + "items": { "type": "number" } + }, + "vec3": { + "type": "array", + "minItems": 3, + "maxItems": 3, + "items": { "type": "number" } + }, + "time": { + "description": "Seconds of device uptime: the clock of ARFrame.timestamp and CMLogItem.timestamp.", + "type": "number", + "minimum": 0 + }, + "tracking": { + "type": "object", + "required": ["state"], + "properties": { + "state": { "enum": ["normal", "limited", "not_available"] }, + "reason": { + "enum": ["initializing", "relocalizing", "excessive_motion", "insufficient_features", null] + } + } + }, + "session": { + "type": "object", + "required": ["id", "producer", "device", "capture", "world_alignment", "meter_anchor"], + "properties": { + "id": { "type": "string", "minLength": 1 }, + "producer": { + "description": "What wrote this packet: the app, or a converter building a sample from a dataset.", + "type": "object", + "required": ["kind", "name", "version"], + "properties": { + "kind": { "enum": ["app", "converter"] }, + "name": { "type": "string" }, + "version": { "type": "string" }, + "commit": { "type": "string" } + } + }, + "device": { + "type": "object", + "required": ["model", "lidar"], + "properties": { + "model": { "description": "Hardware identifier, e.g. iPhone16,1.", "type": "string" }, + "ios_version": { "type": "string" }, + "lidar": { "description": "The device has a LiDAR scanner (sceneDepth supported).", "type": "boolean" }, + "scene_depth_enabled": { "type": "boolean" }, + "mesh_enabled": { "type": "boolean" } + } + }, + "capture": { + "type": "object", + "required": ["started_at_uptime", "ended_at_uptime"], + "properties": { + "started_at": { "description": "Wall clock at started_at_uptime, ISO 8601 UTC.", "type": "string", "format": "date-time" }, + "started_at_uptime": { "$ref": "#/$defs/time" }, + "ended_at_uptime": { "$ref": "#/$defs/time" }, + "distance_walked_m": { + "description": "Horizontal length of the camera trajectory. Must match streams.trajectory within 1%.", + "type": "number", + "minimum": 0 + } + } + }, + "world_alignment": { "const": "gravity" }, + "meter_anchor": { + "description": "The meter frame, as a pose in ARKit's world frame of this session. Every other pose and point in the packet is in the meter frame.", + "type": "object", + "required": ["pose_in_world"], + "properties": { + "pose_in_world": { "$ref": "#/$defs/pose" }, + "ground_y_m": { + "description": "Height of the ground at the meter, on the meter frame's y axis (negative: the ground is below the meter).", + "type": ["number", "null"] + } + } + }, + "consent": { + "type": "object", + "properties": { + "location": { "description": "The homeowner allowed location and heading. Required true if either stream is present.", "type": "boolean" } + } + } + } + }, + "photo": { + "type": "object", + "required": ["id", "image", "width", "height", "t", "pose", "intrinsics"], + "properties": { + "id": { "type": "string", "pattern": "^[A-Za-z0-9_-]+$" }, + "image": { + "description": "The full-resolution photo as captured, JPEG, stored unrotated: landscape as the sensor delivers it, EXIF orientation absent or 1.", + "$ref": "#/$defs/file" + }, + "width": { "type": "integer", "minimum": 1 }, + "height": { "type": "integer", "minimum": 1 }, + "t": { "$ref": "#/$defs/time" }, + "pose": { + "description": "Camera to meter frame. ARKit camera axes: +x right and +y up in the stored image, the camera looks along -z.", + "$ref": "#/$defs/pose" + }, + "intrinsics": { + "description": "[fx, fy, cx, cy] in pixels of this stored image; (0, 0) is the top-left corner of the top-left pixel.", + "type": "array", + "minItems": 4, + "maxItems": 4, + "items": { "type": "number" } + }, + "tracking": { "$ref": "#/$defs/tracking" }, + "exposure": { + "type": "object", + "properties": { + "duration_s": { "type": "number", "exclusiveMinimum": 0 }, + "iso": { "type": "number", "exclusiveMinimum": 0 }, + "offset_ev": { "type": "number" } + } + }, + "lens": { + "type": "object", + "properties": { + "focal_length_mm": { "type": "number", "exclusiveMinimum": 0 }, + "f_number": { "type": "number", "exclusiveMinimum": 0 }, + "camera": { "description": "Which camera: wide, ultra_wide, telephoto, or a dataset's camera name.", "type": "string" } + } + }, + "sharpness": { + "type": "object", + "required": ["method", "value"], + "properties": { + "method": { "const": "laplacian_variance_luma_640" }, + "value": { "type": "number", "minimum": 0 } + } + }, + "depth": { "$ref": "#/$defs/depth" } + } + }, + "depth": { + "description": "Depth aligned to the photo: same field of view and orientation, lower resolution. Float32 little-endian meters along the camera's -z, row by row from the top; 0 means no measurement.", + "type": "object", + "required": ["map", "width", "height"], + "properties": { + "map": { "$ref": "#/$defs/file" }, + "confidence": { + "description": "UInt8 per depth pixel, ARConfidenceLevel: 0 low, 1 medium, 2 high.", + "$ref": "#/$defs/file" + }, + "width": { "type": "integer", "minimum": 1 }, + "height": { "type": "integer", "minimum": 1 }, + "source": { "enum": ["arkit_scene_depth", "arkit_smoothed_scene_depth", "rendered_from_laser_scan"] } + } + }, + "stream": { + "allOf": [{ "$ref": "#/$defs/file" }], + "type": "object", + "required": ["rows"], + "properties": { + "rows": { "type": "integer", "minimum": 0 }, + "nominal_rate_hz": { "type": "number", "exclusiveMinimum": 0 } + } + }, + "streams": { + "description": "CSV files with a header row; the first column is always t. Columns are fixed per stream (README, Streams).", + "type": "object", + "properties": { + "trajectory": { "$ref": "#/$defs/stream" }, + "accelerometer": { "$ref": "#/$defs/stream" }, + "gyroscope": { "$ref": "#/$defs/stream" }, + "magnetometer": { "$ref": "#/$defs/stream" }, + "device_motion": { "$ref": "#/$defs/stream" }, + "barometer": { "$ref": "#/$defs/stream" }, + "location": { "$ref": "#/$defs/stream" }, + "heading": { "$ref": "#/$defs/stream" } + }, + "additionalProperties": false + }, + "lidar": { + "type": "object", + "properties": { + "mesh": { + "description": "Binary little-endian PLY: float x, y, z per vertex in the meter frame; per face a vertex index list and uchar classification (ARMeshClassification raw value 0 to 7).", + "$ref": "#/$defs/file" + }, + "planes": { + "type": "array", + "items": { + "type": "object", + "required": ["id", "alignment", "pose", "extent_m"], + "properties": { + "id": { "type": "string" }, + "alignment": { "enum": ["horizontal", "vertical"] }, + "classification": { "enum": ["none", "wall", "floor", "ceiling", "table", "seat", "window", "door"] }, + "pose": { "description": "Plane to meter frame; the plane is its local x-z plane, +y its normal.", "$ref": "#/$defs/pose" }, + "extent_m": { + "type": "array", + "minItems": 2, + "maxItems": 2, + "items": { "type": "number", "minimum": 0 } + } + } + } + } + } + }, + "mark": { + "type": "object", + "required": ["id", "kind", "points"], + "properties": { + "id": { "type": "string" }, + "kind": { + "enum": ["meter", "wall_end", "gas_meter", "ac", "door", "window", "garage_door", "drive_edge", "fence"] + }, + "points": { + "description": "Meter frame, meters. One point for the meter, a wall end, a gas meter or an AC unit; two for an opening (bottom-left, top-right corners), a drive edge or a fence.", + "type": "array", + "minItems": 1, + "items": { "$ref": "#/$defs/vec3" } + }, + "t": { "$ref": "#/$defs/time" }, + "photo_ids": { "type": "array", "items": { "type": "string" } }, + "side": { "enum": ["left", "right"] }, + "end_kind": { "description": "For a wall end: limit (something blocks it) or unexplored (it turns a corner, or no answer).", "enum": ["limit", "unexplored"] }, + "attrs": { + "type": "object", + "properties": { "operable": { "type": "boolean" } } + } + } + }, + "guidance_event": { + "type": "object", + "required": ["id", "kind", "origin", "t_shown", "outcome"], + "properties": { + "id": { "type": "string" }, + "kind": { "enum": ["walk", "tilt_to_ground", "step_back", "mark_end", "closeup", "gap_band", "gap_past_end"] }, + "origin": { "enum": ["phone", "server"] }, + "message": { "type": "string" }, + "band": { "enum": ["wall", "ground", "overhead", "facing", null] }, + "span_m": { + "description": "Stretch along the wall, meter frame x, meters.", + "type": ["array", "null"], + "minItems": 2, + "maxItems": 2, + "items": { "type": "number" } + }, + "t_shown": { "$ref": "#/$defs/time" }, + "t_resolved": { "type": ["number", "null"] }, + "outcome": { "enum": ["met", "skipped", "cannot_reach", "superseded", "unresolved"] } + } + }, + "provenance": { + "description": "Where a converted sample's data came from, and what it does not contain.", + "type": "object", + "properties": { + "dataset": { "type": "string" }, + "license": { "type": "string" }, + "source": { "type": "string" }, + "notes": { "type": "array", "items": { "type": "string" } } + } + } + } +} diff --git a/packet/packet/__init__.py b/packet/packet/__init__.py new file mode 100644 index 000000000..ea6b38292 --- /dev/null +++ b/packet/packet/__init__.py @@ -0,0 +1 @@ +"""The capture packet: one scan session's photos, poses and sensor streams (README.md).""" diff --git a/packet/packet/__main__.py b/packet/packet/__main__.py new file mode 100644 index 000000000..063e9e66c --- /dev/null +++ b/packet/packet/__main__.py @@ -0,0 +1,46 @@ +"""python -m packet validate [--json]""" + +from __future__ import annotations + +import argparse +import json +import sys +from pathlib import Path + +from packet.validate import validate + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(prog="python -m packet") + sub = parser.add_subparsers(dest="command", required=True) + check = sub.add_parser("validate", help="check a packet folder or zip against the spec") + check.add_argument("packet", type=Path) + check.add_argument("--json", action="store_true", help="print the report as JSON") + args = parser.parse_args(argv) + + report = validate(args.packet) + if args.json: + print( + json.dumps( + { + "ok": report.ok, + "problems": report.problems, + "warnings": report.warnings, + "summary": report.summary, + }, + indent=2, + ) + ) + else: + print(f"{args.packet}: {'valid' if report.ok else 'INVALID'}") + for key, value in report.summary.items(): + print(f" {key}: {value}") + for w in report.warnings: + print(f" warning: {w}") + for p in report.problems: + print(f" problem: {p}") + return 0 if report.ok else 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/packet/packet/files.py b/packet/packet/files.py new file mode 100644 index 000000000..e9b79585d --- /dev/null +++ b/packet/packet/files.py @@ -0,0 +1,103 @@ +"""Read a packet from a folder or a zip through one interface. + +A zip may hold the packet at its root or inside one top-level folder. Entry names that are +absolute or climb out with '..' are refused before anything is read, because a packet comes from +a phone the server does not control. +""" + +from __future__ import annotations + +import hashlib +import zipfile +from pathlib import Path, PurePosixPath + + +class PacketError(Exception): + """The packet cannot be opened at all (no manifest, unsafe zip).""" + + +def safe_relative(path: str) -> bool: + p = PurePosixPath(path) + return bool(path) and not p.is_absolute() and ".." not in p.parts and "\\" not in path + + +class PacketFiles: + def __init__(self, source: Path): + self.source = source + if source.is_dir(): + self._zip = None + self._root = source + self._prefix = "" + if not (source / "manifest.json").is_file(): + raise PacketError(f"{source} has no manifest.json") + return + if not zipfile.is_zipfile(source): + raise PacketError(f"{source} is neither a folder nor a zip") + self._zip = zipfile.ZipFile(source) + self._root = None + names = [i.filename for i in self._zip.infolist() if not i.is_dir()] + unsafe = [n for n in names if not safe_relative(n)] + if unsafe: + raise PacketError(f"zip entry names leave the packet: {unsafe[:3]}") + manifests = [n for n in names if PurePosixPath(n).name == "manifest.json"] + top = [n for n in manifests if len(PurePosixPath(n).parts) <= 2] + if len(top) != 1: + raise PacketError( + f"zip must hold exactly one manifest.json at its root or in one top folder, " + f"found {manifests[:3] or 'none'}" + ) + parent = str(PurePosixPath(top[0]).parent) + self._prefix = "" if parent == "." else parent + "/" + + def names(self) -> set[str]: + """Every file in the packet, as manifest-relative paths.""" + if self._zip is None: + return { + p.relative_to(self._root).as_posix() for p in self._root.rglob("*") if p.is_file() + } + return { + i.filename[len(self._prefix) :] + for i in self._zip.infolist() + if not i.is_dir() and i.filename.startswith(self._prefix) + } + + def exists(self, path: str) -> bool: + if not safe_relative(path): + return False + if self._zip is None: + return (self._root / path).is_file() + try: + self._zip.getinfo(self._prefix + path) + except KeyError: + return False + return True + + def size(self, path: str) -> int: + if self._zip is None: + return (self._root / path).stat().st_size + return self._zip.getinfo(self._prefix + path).file_size + + def read(self, path: str) -> bytes: + if not safe_relative(path): + raise PacketError(f"unsafe path {path!r}") + if self._zip is None: + return (self._root / path).read_bytes() + return self._zip.read(self._prefix + path) + + def sha256(self, path: str) -> str: + digest = hashlib.sha256() + if self._zip is None: + with (self._root / path).open("rb") as f: + for chunk in iter(lambda: f.read(1 << 20), b""): + digest.update(chunk) + else: + with self._zip.open(self._prefix + path) as f: + for chunk in iter(lambda: f.read(1 << 20), b""): + digest.update(chunk) + return digest.hexdigest() + + def open(self, path: str): + """A binary file object, for readers that stream (images, large CSVs).""" + if self._zip is None: + return (self._root / path).open("rb") + return self._zip.open(self._prefix + path) diff --git a/packet/packet/samples/__init__.py b/packet/packet/samples/__init__.py new file mode 100644 index 000000000..8a4455c53 --- /dev/null +++ b/packet/packet/samples/__init__.py @@ -0,0 +1 @@ +"""Builders for sample packets: a synthetic fixture and conversions of public datasets.""" diff --git a/packet/packet/samples/advio.py b/packet/packet/samples/advio.py new file mode 100644 index 000000000..87c05cd30 --- /dev/null +++ b/packet/packet/samples/advio.py @@ -0,0 +1,209 @@ +"""A sample packet from ADVIO sequence 20 (outdoor, iPhone 6s): photos, 60 Hz ARKit trajectory, +and the phone's own accelerometer, gyroscope, magnetometer, barometer and location streams. + +The photos are the evals lane's replay keyframes (`experiments/evals/evals/replay.py` on +t3/evals): ADVIO video frames at their native 1280 x 720, undistorted to the pinhole model with +ADVIO's calibration. Poses and streams come straight from the sequence's CSVs over the same window. + + uv run python -m packet.samples.advio \ + --replay ~/house-scanning-data/replays/advio-20-0040-0075 \ + --advio ~/house-scanning-data/evals/advio/advio-20 \ + --out ~/house-scanning-data/packets/advio-20-0040-0075 +""" + +from __future__ import annotations + +import argparse +import csv +import json +from pathlib import Path + +import numpy as np +from PIL import Image + +from packet.validate import horizontal_distance +from packet.write import ( + PACKET_VERSION, + SHARPNESS_METHOD, + PacketWriter, + column_major, + from_column_major, + meter_frame, + sharpness, + trajectory_row, +) + +# ADVIO's arkit.csv holds the portrait device orientation; the landscape camera the images come +# from is that frame turned about z (image right = device -y, image up = device +x). Same matrix +# as DEVICE_TO_LANDSCAPE_CAMERA in experiments/evals/evals/camera.py, which the replay used. +DEVICE_TO_LANDSCAPE_CAMERA = np.array([[0.0, 1.0, 0.0], [-1.0, 0.0, 0.0], [0.0, 0.0, 1.0]]) +# The app's replay assumes the wall this far to the side the camera faces (ScanKit replay log). +WALL_OFFSET_M = 2.25 +# ADVIO's replay keyframes match arkit.csv rows exactly; allow float32-level rounding only. +POSE_MATCH_TOL = 1e-4 + + +def quat_wxyz_to_matrix(q: np.ndarray) -> np.ndarray: + q = q / np.linalg.norm(q, axis=-1, keepdims=True) + w, x, y, z = q[..., 0], q[..., 1], q[..., 2], q[..., 3] + r = np.empty((*q.shape[:-1], 3, 3)) + r[..., 0, 0] = 1 - 2 * (y * y + z * z) + r[..., 0, 1] = 2 * (x * y - w * z) + r[..., 0, 2] = 2 * (x * z + w * y) + r[..., 1, 0] = 2 * (x * y + w * z) + r[..., 1, 1] = 1 - 2 * (x * x + z * z) + r[..., 1, 2] = 2 * (y * z - w * x) + r[..., 2, 0] = 2 * (x * z - w * y) + r[..., 2, 1] = 2 * (y * z + w * x) + r[..., 2, 2] = 1 - 2 * (x * x + y * y) + return r + + +def read_csv(path: Path) -> np.ndarray: + with path.open() as f: + return np.array([[float(v) for v in row] for row in csv.reader(f) if row]) + + +def world_poses(arkit: np.ndarray) -> np.ndarray: + """Camera-to-world poses (ARKit camera axes, landscape image) from arkit.csv rows.""" + r = quat_wxyz_to_matrix(arkit[:, 4:8]) + poses = np.tile(np.eye(4), (len(arkit), 1, 1)) + poses[:, :3, :3] = r @ DEVICE_TO_LANDSCAPE_CAMERA + poses[:, :3, 3] = arkit[:, 1:4] + return poses + + +def assumed_meter(poses: np.ndarray) -> np.ndarray: + """Meter-to-world pose by the app replay's rule: the wall is vertical, parallel to the walk, + WALL_OFFSET_M to the side the camera faces; the meter is level with the camera, opposite the + middle of the walk.""" + pos = poses[:, :3, 3] + horizontal = pos[:, [0, 2]] - pos[:, [0, 2]].mean(axis=0) + along2 = np.linalg.svd(horizontal, full_matrices=False)[2][0] + along = np.array([along2[0], 0.0, along2[1]]) + looking = -poses[:, :3, 2].mean(axis=0) # cameras look along their -z + side = looking - along * (looking @ along) + side[1] = 0 + side /= np.linalg.norm(side) + middle = pos[len(pos) // 2] + meter = middle + WALL_OFFSET_M * side + # The meter frame's +z points out of the wall, toward the camera: -side. Its +x is then + # y x z; flip `along` if needed so the frame is right-handed with that z. + if np.cross([0.0, 1.0, 0.0], -side) @ along < 0: + along = -along + return meter_frame(meter, along) + + +def build(replay: Path, advio: Path, out: Path) -> Path: + session = json.loads((replay / "session.json").read_text()) + start, end = session["provenance"]["secondsInSequence"] + arkit = read_csv(advio / "iphone" / "arkit.csv") + keep = (arkit[:, 0] >= start) & (arkit[:, 0] <= end) & (np.abs(arkit[:, 1:4]).sum(axis=1) > 0) + arkit = arkit[keep] + world = world_poses(arkit) + anchor = assumed_meter(world) + to_meter = np.linalg.inv(anchor) + in_meter = to_meter @ world + + w = PacketWriter(out) + photos = [] + for kf in session["keyframes"]: + i = int(np.abs(arkit[:, 0] - kf["timestamp"]).argmin()) + replay_pose = from_column_major(kf["pose"]) + if np.abs(replay_pose - world[i]).max() > POSE_MATCH_TOL: + raise ValueError(f"{kf['id']}: replay pose differs from arkit.csv at t={arkit[i, 0]}") + pid = f"p{int(kf['id'][1:]):05d}" + source = replay / kf["img"] + with Image.open(source) as img: + score = sharpness(img) + photos.append( + { + "id": pid, + "image": w.add_file(f"photos/{pid}.jpg", source), + "width": kf["w"], + "height": kf["h"], + "t": float(arkit[i, 0]), + "pose": column_major(in_meter[i]), + "intrinsics": kf["intrinsics"], + "tracking": {"state": "normal", "reason": None}, + "lens": {"camera": "wide"}, + "sharpness": {"method": SHARPNESS_METHOD, "value": round(score, 3)}, + } + ) + + trajectory = [ + trajectory_row(t, "normal", p) for t, p in zip(arkit[:, 0], in_meter, strict=True) + ] + streams = {"trajectory": w.add_stream("trajectory", trajectory, 60)} + notes = [] + for name, file, rate in [ + ("accelerometer", "accelerometer.csv", 100), + ("gyroscope", "gyro.csv", 100), + ("magnetometer", "magnetometer.csv", 100), + ("barometer", "barometer.csv", 1), + ("location", "platform-locations.csv", 1), + ]: + data = read_csv(advio / "iphone" / file) + data = data[(data[:, 0] >= start) & (data[:, 0] <= end)] + repeats = np.flatnonzero(np.diff(data[:, 0]) <= 0) + 1 + if len(repeats): + notes.append(f"{file}: dropped {len(repeats)} rows whose time did not advance") + data = np.delete(data, repeats, axis=0) + rows = [[round(float(v), 6) for v in r] for r in data] + streams[name] = w.add_stream(name, rows, rate) + + manifest = { + "packet_version": PACKET_VERSION, + "session": { + "id": session["session"]["id"], + "producer": {"kind": "converter", "name": "packet.samples.advio", "version": "1"}, + "device": {"model": "iPhone8,1", "ios_version": "11 (ADVIO 2018)", "lidar": False}, + "capture": { + "started_at_uptime": float(start), + "ended_at_uptime": float(end), + "distance_walked_m": round(horizontal_distance(in_meter[:, :3, 3]), 4), + }, + "world_alignment": "gravity", + "meter_anchor": {"pose_in_world": column_major(anchor), "ground_y_m": None}, + "consent": {"location": True}, + }, + "photos": photos, + "streams": streams, + "marks": [ + {"id": "m1", "kind": "meter", "points": [[0.0, 0.0, 0.0]], "t": photos[0]["t"]}, + ], + "provenance": { + "dataset": "ADVIO sequence 20 (Cortés, Solin, Rahtu, Kannala; ECCV 2018)", + "license": "CC BY-NC 4.0: for accuracy testing only; never commit or redistribute", + "source": "https://zenodo.org/record/1476931/files/advio-20.zip", + "notes": [ + "t is ADVIO's own clock (seconds from the start of its recording), which every " + "ADVIO file shares; it stands in for device uptime.", + f"Photos are the evals replay's keyframes ({session['provenance']['keyframeRule']}" + "), undistorted to pinhole; ADVIO's video is 1280 x 720, so this is its full " + "resolution.", + "There is no meter or wall in ADVIO. The meter frame follows the app replay's " + f"assumption: a vertical wall parallel to the walk, {WALL_OFFSET_M} m to the side " + "the camera faces, the meter level with the camera opposite the walk's middle.", + "Streams are in ADVIO's units: accelerometer m/s^2, gyroscope rad/s, magnetometer " + "microtesla, barometer kPa and meters. Location was recorded by the dataset " + "authors and published with it.", + "No exposure, ISO, depth, mesh, guidance or scene.json: ADVIO has none of them.", + *notes, + ], + }, + } + return w.finish(manifest) + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0]) + parser.add_argument("--replay", type=Path, required=True) + parser.add_argument("--advio", type=Path, required=True) + parser.add_argument("--out", type=Path, required=True) + args = parser.parse_args() + print(build(args.replay.expanduser(), args.advio.expanduser(), args.out.expanduser())) + + +if __name__ == "__main__": + main() diff --git a/packet/packet/samples/app_export.py b/packet/packet/samples/app_export.py new file mode 100644 index 000000000..748a31f48 --- /dev/null +++ b/packet/packet/samples/app_export.py @@ -0,0 +1,191 @@ +"""A sample packet from the iOS app's own export (scan.zip) of a Simulator run on the ADVIO +replay, so the server team sees exactly what the app records today, in the packet's layout. + +The app's export is scene.json (C1, feet) plus one JPEG per keyframe with its pose and +intrinsics. It has no timestamps; each photo's time is recovered by matching its pixels to the +replay keyframe it was taken from. Everything else the packet can hold is absent, and +provenance.notes lists it. + + uv run python -m packet.samples.app_export \ + --sim-report ~/house-scanning-data/reports/sim/20260926-071459-t3-ios-mvf-a39d0a50-replay \ + --replay ~/house-scanning-data/replays/advio-20-0040-0075 \ + --out ~/house-scanning-data/packets/app-export-a39d0a5 +""" + +from __future__ import annotations + +import argparse +import io +import json +import zipfile +from pathlib import Path + +import numpy as np +from PIL import Image + +from packet.write import ( + PACKET_VERSION, + SHARPNESS_METHOD, + PacketWriter, + column_major, + from_column_major, + meter_frame, + orthonormalize, + sharpness, +) + +FEET = 0.3048 +MATCH_SIZE = (64, 36) +# The app re-encodes the replay's JPEGs; a true match differs by JPEG noise only (mean absolute +# difference of a few grey levels on a 64 x 36 thumbnail). Anything above this is not a match. +MATCH_MAX_DIFF = 6.0 +# scene.json object types that are also packet mark kinds (fences and drives are not objects). +MARK_OBJECTS = {"gas_meter", "ac", "door", "window", "garage_door"} + + +def thumbnail(data: bytes | Path) -> np.ndarray: + with Image.open(io.BytesIO(data) if isinstance(data, bytes) else data) as img: + return np.asarray(img.convert("L").resize(MATCH_SIZE, Image.Resampling.BILINEAR), float) + + +def build(sim_report: Path, replay: Path, out: Path) -> Path: + report = json.loads((sim_report / "report.json").read_text()) + bundle = zipfile.ZipFile(sim_report / report["app_export"]) + scene_bytes = bundle.read("scene.json") + scene = json.loads(scene_bytes) + + session = json.loads((replay / "session.json").read_text()) + replay_thumbs = [ + (kf["timestamp"], thumbnail(replay / kf["img"])) for kf in session["keyframes"] + ] + + meter_ft = np.asarray(scene["meter"]["pos"], float) + wall = next(w for w in scene["walls"] if w["id"] == scene["meter"]["wall_id"]) + (x0, z0), (x1, z1) = wall["baseline"][0], wall["baseline"][-1] + along = np.array([x1 - x0, 0.0, z1 - z0]) + anchor = meter_frame(meter_ft * FEET, along) + to_meter = np.linalg.inv(anchor) + + w = PacketWriter(out) + photos = [] + for kf in scene["keyframes"]: + data = bundle.read(kf["img"]) + thumb = thumbnail(data) + diffs = [float(np.abs(thumb - t).mean()) for _, t in replay_thumbs] + best = int(np.argmin(diffs)) + if diffs[best] > MATCH_MAX_DIFF: + raise ValueError( + f"{kf['img']}: no replay keyframe matches (best diff {diffs[best]:.1f})" + ) + pose = from_column_major(kf["pose"]) + pose[:3, 3] *= FEET + pose = orthonormalize(to_meter @ pose) # scene.json rounds rotations to 4 decimals + pid = f"p{int(kf['id'][1:]):05d}" + photos.append( + { + "id": pid, + "image": w.add_bytes(f"photos/{pid}.jpg", data), + "width": kf["w"], + "height": kf["h"], + "t": replay_thumbs[best][0], + "pose": column_major(pose), + "intrinsics": kf["intrinsics"], + "sharpness": { + "method": SHARPNESS_METHOD, + "value": round(sharpness(Image.open(io.BytesIO(data))), 3), + }, + } + ) + photos.sort(key=lambda p: p["t"]) + times = [p["t"] for p in photos] + if len(set(times)) != len(times): + raise ValueError("two app photos matched the same replay keyframe") + camera_z = np.median([p["pose"][14] for p in photos]) + if camera_z <= 0: + raise ValueError( + f"cameras sit behind the wall (median z {camera_z:.2f} m): wall direction?" + ) + + def on_wall(s_ft: float, height_ft: float) -> list[float]: + """A point on the wall at s (along it) and a height above the ground, in the meter frame.""" + return [round(s_ft * FEET, 4), round((height_ft - meter_ft[1]) * FEET, 4), 0.0] + + marks = [{"id": "meter", "kind": "meter", "points": [[0.0, 0.0, 0.0]]}] + for i, obj in enumerate(scene.get("objects", [])): + kind = obj["type"] + if kind not in MARK_OBJECTS: + continue + s0, s1 = obj["span_ft"] + bottom, top = obj.get("bottom_ft", 0.0), obj.get("top_ft", obj.get("bottom_ft", 0.0)) + points = ( + [on_wall((s0 + s1) / 2, (bottom + top) / 2)] + if kind in ("gas_meter", "ac") + else [on_wall(s0, bottom), on_wall(s1, top)] + ) + mark = {"id": f"object{i}", "kind": kind, "points": points} + if "operable" in obj.get("attrs", {}): + mark["attrs"] = {"operable": obj["attrs"]["operable"]} + marks.append(mark) + ends = scene.get("coverage", {}).get("ends", {}) + for side, (ex, ez) in (("left", (x0, z0)), ("right", (x1, z1))): + if side in ends: + s = (np.array([ex, 0.0, ez]) - meter_ft * [1, 0, 1]) @ anchor[:3, 0] + marks.append( + { + "id": f"end-{side}", + "kind": "wall_end", + "points": [on_wall(float(s), meter_ft[1])], + "side": side, + "end_kind": ends[side]["kind"], + } + ) + + manifest = { + "packet_version": PACKET_VERSION, + "session": { + "id": f"app-export-{report['sha'][:7]}", + "producer": {"kind": "converter", "name": "packet.samples.app_export", "version": "1"}, + "device": {"model": "iOS Simulator", "lidar": False}, + "capture": {"started_at_uptime": times[0], "ended_at_uptime": times[-1]}, + "world_alignment": "gravity", + "meter_anchor": { + "pose_in_world": column_major(anchor), + "ground_y_m": round(-meter_ft[1] * FEET, 4), + }, + }, + "photos": photos, + "marks": marks, + "scene": w.add_bytes("scene.json", scene_bytes) + | {"schema_version": scene["schema_version"]}, + "provenance": { + "dataset": "ADVIO sequence 20 via the app's replay", + "license": "CC BY-NC 4.0 (ADVIO): accuracy testing only; never commit or share", + "source": f"t3/ios-mvf {report['sha'][:7]}, Simulator run {sim_report.name}, scan.zip", + "notes": [ + "Photos, poses, intrinsics, marks and scene.json are the app's; the JPEGs are " + "unchanged. Poses are converted from scene.json's feet to meters and into the " + "meter frame, with rotations re-orthonormalized after scene.json's rounding.", + "pose_in_world is in scene.json's frame: ARKit's world moved so the ground at the " + "wall is y = 0. The app does not export the offset to ARKit's own origin.", + "t is recovered: each photo matched to the replay keyframe it came from. The " + "app exports no timestamps.", + "Absent because the app does not record them yet: device model and iOS version, " + "wall-clock start, tracking state, exposure, ISO, lens, depth, trajectory, IMU, " + "distance walked, mesh, planes, mark times and the guidance log.", + ], + }, + } + return w.finish(manifest) + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0]) + parser.add_argument("--sim-report", type=Path, required=True) + parser.add_argument("--replay", type=Path, required=True) + parser.add_argument("--out", type=Path, required=True) + args = parser.parse_args() + print(build(args.sim_report.expanduser(), args.replay.expanduser(), args.out.expanduser())) + + +if __name__ == "__main__": + main() diff --git a/packet/packet/samples/eth3d.py b/packet/packet/samples/eth3d.py new file mode 100644 index 000000000..b4b81d329 --- /dev/null +++ b/packet/packet/samples/eth3d.py @@ -0,0 +1,235 @@ +"""A sample packet from ETH3D's electro scene: full-resolution DSLR photos of building walls +within about 6 m, each with a depth map rendered from the terrestrial laser scan as a stand-in +for the depth a LiDAR iPhone records. + +Inputs are the evals lane's extracted scene (scan points, COLMAP calibration) plus the original +undistorted photos from electro_dslr_undistorted.7z (sha256 in experiments/evals/README.md on +t3/evals), unpacked to /full/. + + uv run python -m packet.samples.eth3d --eth3d ~/house-scanning-data/evals/eth3d/electro \ + --out ~/house-scanning-data/packets/eth3d-electro +""" + +from __future__ import annotations + +import argparse +import math +from pathlib import Path + +import numpy as np +from PIL import Image + +from packet.write import ( + PACKET_VERSION, + SHARPNESS_METHOD, + PacketWriter, + column_major, + meter_frame, + sharpness, +) + +PHOTOS = [f"DSC_{n}" for n in range(9257, 9265)] # one walk past the wall, in shooting order +# ARKit's depth map is 256 x 192 for a 1920 x 1440 image (1/7.5); a 1/16 map of a 24 MP photo +# is 387 x 258, the same order of pixel count, so the sample exercises the same alignment rule. +DEPTH_SCALE = 16 +NEAR_M = 0.2 +WALL_RADIUS_M = 0.25 +OPENCV_TO_ARKIT_CAMERA = np.diag([1.0, -1.0, -1.0, 1.0]) + + +def read_cameras(path: Path) -> dict[int, tuple[int, int, float, float, float, float]]: + cams = {} + for line in path.read_text().splitlines(): + if line.startswith("#") or not line.strip(): + continue + cid, model, w, h, *params = line.split() + if model != "PINHOLE": + raise ValueError(f"camera {cid} is {model}; the undistorted set should be PINHOLE") + cams[int(cid)] = (int(w), int(h), *map(float, params)) + return cams + + +def read_images(path: Path) -> dict[str, tuple[np.ndarray, int]]: + """Image name -> (world-to-camera 4x4 in OpenCV axes, camera id). COLMAP lists two lines per + image; the second holds 2-D points and is skipped.""" + out = {} + lines = [ln for ln in path.read_text().splitlines() if not ln.startswith("#")] + for line in lines[::2]: + f = line.split() + qw, qx, qy, qz, tx, ty, tz = map(float, f[1:8]) + r = quaternion_wxyz(qw, qx, qy, qz) + m = np.eye(4) + m[:3, :3], m[:3, 3] = r, [tx, ty, tz] + out[Path(f[9]).stem] = (m, int(f[8])) + return out + + +def quaternion_wxyz(w: float, x: float, y: float, z: float) -> np.ndarray: + n = math.sqrt(w * w + x * x + y * y + z * z) + w, x, y, z = w / n, x / n, y / n, z / n + return np.array( + [ + [1 - 2 * (y * y + z * z), 2 * (x * y - w * z), 2 * (x * z + w * y)], + [2 * (x * y + w * z), 1 - 2 * (x * x + z * z), 2 * (y * z - w * x)], + [2 * (x * z - w * y), 2 * (y * z + w * x), 1 - 2 * (x * x + y * y)], + ] + ) + + +def gravity_world(normals: np.ndarray) -> tuple[np.ndarray, float]: + """World-from-scan rotation with +y up, and the scan's tilt in degrees. The scan is levelled + with z roughly up; ground and roof normals (within 18° of z) give the exact up direction.""" + flat = normals[np.abs(normals[:, 2]) > 0.95] + up = (flat * np.sign(flat[:, 2:3])).mean(axis=0) + up /= np.linalg.norm(up) + swap = np.array([[1.0, 0, 0], [0, 0, 1], [0, -1, 0]]) # scan z -> world y, right-handed + u = swap @ up + axis = np.cross(u, [0.0, 1.0, 0.0]) + s, c = np.linalg.norm(axis), u[1] + if s < 1e-12: + level = np.eye(3) + else: + k = axis / s + kx = np.array([[0, -k[2], k[1]], [k[2], 0, -k[0]], [-k[1], k[0], 0]]) + level = np.eye(3) + s * kx + (1 - c) * kx @ kx + m = np.eye(4) + m[:3, :3] = level @ swap + return m, math.degrees(math.acos(min(1.0, abs(up[2])))) + + +def render_depth( + points: np.ndarray, cam_from_scan: np.ndarray, k: tuple, size: tuple +) -> np.ndarray: + """Nearest scan depth per pixel of a map `size` = (w, h) covering the photo's field of view.""" + w, h, fx, fy, cx, cy = k + dw, dh = size + pc = points @ cam_from_scan[:3, :3].T + cam_from_scan[:3, 3] + z = pc[:, 2] + front = z > NEAR_M + pc, z = pc[front], z[front] + u = fx * pc[:, 0] / z + cx + v = fy * pc[:, 1] / z + cy + inside = (u >= 0) & (u < w) & (v >= 0) & (v < h) + j = (u[inside] * dw / w).astype(int) + i = (v[inside] * dh / h).astype(int) + depth = np.full(dh * dw, np.inf) + np.minimum.at(depth, i * dw + j, z[inside]) + depth[np.isinf(depth)] = 0 + return depth.reshape(dh, dw).astype(np.float32) + + +def build(eth3d: Path, out: Path) -> Path: + cams = read_cameras(eth3d / "dslr_calibration_undistorted" / "cameras.txt") + images = read_images(eth3d / "dslr_calibration_undistorted" / "images.txt") + points = np.load(eth3d / "scan_points_10mm.npy").astype(np.float64) + world_from_scan, tilt = gravity_world(np.load(eth3d / "normals.npy")) + + # Camera-to-world poses in ARKit camera axes, and depth maps, per photo. + frames = [] + for name in PHOTOS: + cam_from_scan, cid = images[name] + k = cams[cid] + size = (round(k[0] / DEPTH_SCALE), round(k[1] / DEPTH_SCALE)) + depth = render_depth(points, cam_from_scan, k, size) + world_pose = world_from_scan @ np.linalg.inv(cam_from_scan) @ OPENCV_TO_ARKIT_CAMERA + frames.append((name, k, depth, world_pose, cam_from_scan)) + + # The meter frame: where the first photo's optical axis meets the wall. + name, k, depth, world_pose, cam_from_scan = frames[0] + centre = depth[depth.shape[0] // 2, depth.shape[1] // 2] + if centre <= 0: + raise ValueError(f"{name}: no scan depth at the image centre to anchor the meter on") + hit_scan = np.linalg.inv(cam_from_scan) @ np.array([0.0, 0.0, centre, 1.0]) + near = points[np.linalg.norm(points - hit_scan[:3], axis=1) < WALL_RADIUS_M] + normal = world_from_scan[:3, :3] @ np.linalg.svd(near - near.mean(axis=0))[2][2] + if abs(normal[1]) > 0.5: + raise ValueError(f"{name}: the image centre hits a surface that is not a wall") + normal[1] = 0 + normal /= np.linalg.norm(normal) + hit_world = (world_from_scan @ hit_scan)[:3] + if normal @ (world_pose[:3, 3] - hit_world) < 0: + normal = -normal # point out of the wall, toward the camera + anchor = meter_frame(hit_world, np.cross([0.0, 1.0, 0.0], normal)) + to_meter = np.linalg.inv(anchor) + + world_points_y = (points @ world_from_scan[:3, :3].T)[:, 1] + horizontal = np.linalg.norm( + (points - hit_scan[:3]) @ world_from_scan[:3, :3].T * [1, 0, 1], axis=1 + ) + ground_y = float(np.percentile(world_points_y[horizontal < 1.0], 2)) - hit_world[1] + + w = PacketWriter(out) + photos = [] + for n, (name, k, depth, world_pose, _) in enumerate(frames): + pid = f"p{n + 1:05d}" + source = eth3d / "full" / f"{name}.JPG" + with Image.open(source) as img: + if img.size != (k[0], k[1]): + raise ValueError(f"{name}: image is {img.size}, calibration says {k[:2]}") + score = sharpness(img) + confidence = np.where(depth > 0, 2, 0) + photos.append( + { + "id": pid, + "image": w.add_file(f"photos/{pid}.jpg", source), + "width": k[0], + "height": k[1], + "t": float(n + 1), + "pose": column_major(to_meter @ world_pose), + "intrinsics": [round(v, 4) for v in k[2:]], + "lens": {"camera": f"ETH3D DSLR, calibration camera {images[name][1]}"}, + "sharpness": {"method": SHARPNESS_METHOD, "value": round(score, 3)}, + "depth": w.add_depth(pid, depth, confidence) + | {"source": "rendered_from_laser_scan"}, + } + ) + + manifest = { + "packet_version": PACKET_VERSION, + "session": { + "id": "eth3d-electro-9257-9264", + "producer": {"kind": "converter", "name": "packet.samples.eth3d", "version": "1"}, + "device": {"model": "ETH3D DSLR (24 MP)", "lidar": False}, + "capture": {"started_at_uptime": 0.0, "ended_at_uptime": float(len(frames) + 1)}, + "world_alignment": "gravity", + "meter_anchor": { + "pose_in_world": column_major(anchor), + "ground_y_m": round(ground_y, 4), + }, + }, + "photos": photos, + "marks": [{"id": "m1", "kind": "meter", "points": [[0.0, 0.0, 0.0]], "t": 0.5}], + "provenance": { + "dataset": "ETH3D high-res multi-view, electro (Schöps et al., CVPR 2017)", + "license": "CC BY-NC-SA 4.0: accuracy testing only; never commit or share", + "source": "https://www.eth3d.net/data/electro_dslr_undistorted.7z and the evals " + "lane's scan_points_10mm.npy / normals.npy", + "notes": [ + "Photos are ETH3D's undistorted originals, unchanged. They carry no EXIF, so " + "there is no capture time, exposure, ISO or lens data: t is the shooting order " + "in seconds.", + f"Depth is rendered from the laser scan (10 mm points, nearest point per pixel) " + f"at 1/{DEPTH_SCALE} of each photo's size; confidence is 2 where a scan point " + "landed and 0 elsewhere. It stands in for LiDAR depth: denser and more accurate " + "than an iPhone's.", + f"World is the scan levelled by its ground normals (tilt {tilt:.2f} degrees), " + "with y up. There is no meter: the meter frame sits where the first photo's " + "optical axis meets the wall, z out of the wall toward the camera. ground_y_m is " + "the 2nd percentile scan height within 1 m of it.", + "No trajectory, IMU, mesh, planes, guidance or scene.json: a DSLR records none.", + ], + }, + } + return w.finish(manifest) + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0]) + parser.add_argument("--eth3d", type=Path, required=True) + parser.add_argument("--out", type=Path, required=True) + args = parser.parse_args() + print(build(args.eth3d.expanduser(), args.out.expanduser())) + + +if __name__ == "__main__": + main() diff --git a/packet/packet/samples/synthetic.py b/packet/packet/samples/synthetic.py new file mode 100644 index 000000000..111706c90 --- /dev/null +++ b/packet/packet/samples/synthetic.py @@ -0,0 +1,287 @@ +"""A tiny synthetic packet that uses every part of the spec: the test fixture in fixtures/. + +Nothing in it is measured. A camera slides 2 m along a flat wall 2.5 m away, facing it, and every +sensor reports the values that motion implies (or a fixed plausible value). It exists so readers +and the validator have one packet with every optional section, small enough to commit. + + uv run python -m packet.samples.synthetic --out fixtures/synthetic +""" + +from __future__ import annotations + +import argparse +import io +import json +from pathlib import Path + +import numpy as np +from PIL import Image, ImageDraw + +from packet.validate import horizontal_distance +from packet.write import ( + PACKET_VERSION, + SHARPNESS_METHOD, + PacketWriter, + column_major, + sharpness, + trajectory_row, +) + +T0, T1, RATE = 100.0, 102.0, 60 +W, H, FX = 96, 72, 80.0 +DEPTH_W, DEPTH_H = 24, 18 +PHOTO_TICKS = (15, 60, 105) # trajectory samples the photos are taken at +WALL_DISTANCE = 2.5 + + +def camera_pose(t: float) -> np.ndarray: + """Camera to meter frame: sliding along +x from -1 to +1 m, 2.5 m out, facing the wall. + The camera looks along its -z, which is the meter frame's -z (into the wall).""" + pose = np.eye(4) + pose[:3, 3] = [-1.0 + 2.0 * (t - T0) / (T1 - T0), 0.0, WALL_DISTANCE] + return pose + + +def photo(index: int) -> bytes: + img = Image.new("RGB", (W, H), (200, 190, 170)) + draw = ImageDraw.Draw(img) + for x in range(0, W, 12): + draw.line([(x + 4 * index, 0), (x + 4 * index, H)], fill=(120, 60, 40), width=2) + draw.rectangle([30, 20, 50, 40], outline=(40, 40, 40), width=2) + buf = io.BytesIO() + img.save(buf, "JPEG", quality=90) + return buf.getvalue() + + +def build(out: Path) -> Path: + w = PacketWriter(out) + ticks = np.arange(0, round((T1 - T0) * RATE) + 1) + times = T0 + ticks / RATE + poses = [camera_pose(t) for t in times] + + photos = [] + for n, k in enumerate(PHOTO_TICKS, start=1): + pid = f"p{n:05d}" + data = photo(n) + # A flat wall straight ahead: every depth pixel is the wall distance. + depth = np.full((DEPTH_H, DEPTH_W), WALL_DISTANCE, dtype=np.float32) + depth[:, :2] = 0 # a strip with no measurement, as at a real depth map's edge + confidence = np.where(depth > 0, 2, 0) + photos.append( + { + "id": pid, + "image": w.add_bytes(f"photos/{pid}.jpg", data), + "width": W, + "height": H, + "t": float(times[k]), + "pose": column_major(poses[k]), + "intrinsics": [FX, FX, W / 2, H / 2], + "tracking": {"state": "normal", "reason": None}, + "exposure": {"duration_s": 1 / 500, "iso": 50}, + "lens": {"focal_length_mm": 5.1, "f_number": 1.8, "camera": "wide"}, + "sharpness": { + "method": SHARPNESS_METHOD, + "value": round(sharpness(Image.open(io.BytesIO(data))), 3), + }, + "depth": w.add_depth(pid, depth, confidence) | {"source": "arkit_scene_depth"}, + } + ) + + traj = [trajectory_row(t, "normal", p) for t, p in zip(times, poses, strict=True)] + imu_t = np.round(T0 + np.arange(0, 2.0, 0.01), 3) + streams = { + "trajectory": w.add_stream("trajectory", traj, RATE), + # Device held still in landscape: gravity along the device's -x. + "accelerometer": w.add_stream("accelerometer", [[t, -9.81, 0.0, 0.0] for t in imu_t], 100), + "gyroscope": w.add_stream("gyroscope", [[t, 0.0, 0.0, 0.0] for t in imu_t], 100), + "magnetometer": w.add_stream("magnetometer", [[t, 20.0, -5.0, -40.0] for t in imu_t], 100), + "device_motion": w.add_stream( + "device_motion", + [[t, 0, 0, 0, 1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 90.0] for t in imu_t], + 100, + ), + "barometer": w.add_stream("barometer", [[T0 + 0.5, 101.3, 0.0], [T0 + 1.5, 101.3, 0.0]], 1), + "location": w.add_stream("location", [[T0 + 0.5, 0.0, 0.0, 10.0, 5.0, 3.0]], 1), + "heading": w.add_stream("heading", [[T0 + 0.5, 90.0, 90.0, 10.0]], 1), + } + + mesh = mesh_ply() + wall_plane = np.eye(4) + wall_plane[:3, :3] = [[1, 0, 0], [0, 0, -1], [0, 1, 0]] # plane normal (+y) = meter +z + ground_plane = np.eye(4) + ground_plane[1, 3] = -1.2 + anchor = np.eye(4) + anchor[:3, :3] = [[0.866025, 0, 0.5], [0, 1, 0], [-0.5, 0, 0.866025]] + anchor[:3, 3] = [3.0, 0.2, -1.5] + + manifest = { + "packet_version": PACKET_VERSION, + "session": { + "id": "synthetic-0001", + "producer": {"kind": "converter", "name": "packet.samples.synthetic", "version": "1"}, + "device": { + "model": "synthetic", + "ios_version": "n/a", + "lidar": True, + "scene_depth_enabled": True, + "mesh_enabled": True, + }, + "capture": { + "started_at": "2026-09-26T12:00:00Z", + "started_at_uptime": T0 - 0.5, + "ended_at_uptime": T1 + 0.5, + "distance_walked_m": round( + horizontal_distance(np.array([p[:3, 3] for p in poses])), 4 + ), + }, + "world_alignment": "gravity", + "meter_anchor": {"pose_in_world": column_major(anchor), "ground_y_m": -1.2}, + "consent": {"location": True}, + }, + "photos": photos, + "streams": streams, + "lidar": { + "mesh": w.add_bytes("lidar/mesh.ply", mesh), + "planes": [ + { + "id": "wall", + "alignment": "vertical", + "classification": "wall", + "pose": column_major(wall_plane), + "extent_m": [6.0, 2.4], + }, + { + "id": "ground", + "alignment": "horizontal", + "classification": "floor", + "pose": column_major(ground_plane), + "extent_m": [6.0, 3.0], + }, + ], + }, + "marks": [ + { + "id": "m1", + "kind": "meter", + "points": [[0.0, 0.0, 0.0]], + "t": T0 - 0.2, + "photo_ids": ["p00001"], + }, + { + "id": "m2", + "kind": "wall_end", + "points": [[-3.0, 0.0, 0.0]], + "t": T0 + 0.1, + "side": "left", + "end_kind": "unexplored", + }, + { + "id": "m3", + "kind": "wall_end", + "points": [[3.0, 0.0, 0.0]], + "t": T0 + 1.9, + "side": "right", + "end_kind": "limit", + }, + {"id": "m4", "kind": "gas_meter", "points": [[1.5, -0.6, 0.1]], "t": T0 + 1.2}, + { + "id": "m5", + "kind": "window", + "points": [[-2.0, 0.2, 0.0], [-1.2, 1.2, 0.0]], + "t": T0 + 0.4, + "attrs": {"operable": True}, + }, + { + "id": "m6", + "kind": "drive_edge", + "points": [[2.0, -1.2, 0.5], [2.0, -1.2, 4.0]], + "t": T0 + 1.5, + }, + ], + "guidance": [ + { + "id": "g1", + "kind": "walk", + "origin": "phone", + "message": "Walk slowly to your left", + "t_shown": T0, + "t_resolved": T0 + 0.8, + "outcome": "met", + }, + { + "id": "g2", + "kind": "tilt_to_ground", + "origin": "phone", + "band": "ground", + "message": "Tilt down to show the ground", + "t_shown": T0 + 0.9, + "t_resolved": T0 + 1.3, + "outcome": "met", + }, + { + "id": "g3", + "kind": "gap_band", + "origin": "server", + "band": "ground", + "span_m": [1.0, 2.5], + "message": "Show the ground around your meter", + "t_shown": T0 + 1.6, + "t_resolved": T0 + 1.9, + "outcome": "cannot_reach", + }, + ], + "scene": w.add_bytes("scene.json", scene_json()) | {"schema_version": "1.0"}, + "provenance": { + "dataset": "synthetic", + "license": "same as this repository", + "source": "packet/packet/samples/synthetic.py", + "notes": ["Every value is invented; nothing was measured."], + }, + } + return w.finish(manifest) + + +def mesh_ply() -> bytes: + """A wall quad (class 1, wall) and a ground quad (class 2, floor), two triangles each.""" + v = np.array( + [ + [-3, -1.2, 0], + [3, -1.2, 0], + [3, 1.2, 0], + [-3, 1.2, 0], + [-3, -1.2, 0], + [3, -1.2, 0], + [3, -1.2, 3], + [-3, -1.2, 3], + ], + dtype=" bytes: + scene = { + "schema_version": "1.0", + "meter": {"pos": [0.0, 4.0, 0.0], "wall_id": "w1"}, + "walls": [{"id": "w1", "baseline": [[-9.84, 0.0], [9.84, 0.0]]}], + } + return (json.dumps(scene, indent=1) + "\n").encode() + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0]) + parser.add_argument("--out", type=Path, required=True) + print(build(parser.parse_args().out)) + + +if __name__ == "__main__": + main() diff --git a/packet/packet/validate.py b/packet/packet/validate.py new file mode 100644 index 000000000..88660dea0 --- /dev/null +++ b/packet/packet/validate.py @@ -0,0 +1,517 @@ +"""Check a capture packet against README.md and manifest.schema.json. + +Each check returns a list of problems; `validate` runs them all and never stops at the first, so +one run shows everything a producer has to fix. Tolerances are stated beside each check. +""" + +from __future__ import annotations + +import csv +import io +import json +import math +from dataclasses import dataclass, field +from pathlib import Path + +import numpy as np +from jsonschema import Draft202012Validator +from PIL import Image + +from packet.files import PacketError, PacketFiles + +SCHEMA_PATH = Path(__file__).resolve().parents[1] / "manifest.schema.json" + +# A float32 simd matrix written to JSON keeps about 7 significant digits, so an orthonormal +# rotation comes back orthonormal to about 1e-6; 1e-3 leaves room for any honest writer. +ROTATION_TOL = 1e-3 +QUATERNION_TOL = 1e-3 +# Real cameras have square pixels to well under 1%; 5% catches fx/fy swapped with a rotation. +PIXEL_ASPECT_TOL = 0.05 +FOV_RANGE_DEG = (20.0, 150.0) +DEPTH_ASPECT_TOL = 0.01 +# Distance walked is a sum over the trajectory; 1% (or 5 cm on a short walk) covers rounding. +DISTANCE_REL_TOL, DISTANCE_ABS_TOL = 0.01, 0.05 +# A photo taken from a tracked frame must sit on the trajectory: its nearest trajectory sample +# within one frame at 60 Hz, and at the same place to a centimetre. +PHOTO_TRAJECTORY_DT, PHOTO_TRAJECTORY_DIST = 1 / 60 + 1e-6, 0.01 + +STREAM_COLUMNS = { + "trajectory": ["t", "tracking", "px", "py", "pz", "qx", "qy", "qz", "qw"], + "accelerometer": ["t", "x", "y", "z"], + "gyroscope": ["t", "x", "y", "z"], + "magnetometer": ["t", "x", "y", "z"], + "device_motion": [ + "t", + "qx", + "qy", + "qz", + "qw", + "gravity_x", + "gravity_y", + "gravity_z", + "user_accel_x", + "user_accel_y", + "user_accel_z", + "rotation_rate_x", + "rotation_rate_y", + "rotation_rate_z", + "heading_deg", + ], + "barometer": ["t", "pressure_kpa", "relative_altitude_m"], + "location": [ + "t", + "latitude", + "longitude", + "altitude_m", + "horizontal_accuracy_m", + "vertical_accuracy_m", + ], + "heading": ["t", "magnetic_deg", "true_deg", "accuracy_deg"], +} +TRACKING_STATES = {"normal", "limited", "not_available"} +CONSENTED_STREAMS = ("location", "heading") +MARK_POINTS = { + "meter": 1, + "wall_end": 1, + "gas_meter": 1, + "ac": 1, + "door": 2, + "window": 2, + "garage_door": 2, + "drive_edge": 2, + "fence": 2, +} +# ARMeshClassification raw values: none, wall, floor, ceiling, table, seat, window, door. +MESH_CLASSIFICATIONS = 8 + + +@dataclass +class Report: + problems: list[str] = field(default_factory=list) + warnings: list[str] = field(default_factory=list) + summary: dict = field(default_factory=dict) + + @property + def ok(self) -> bool: + return not self.problems + + +# --- Geometry ----------------------------------------------------------------------------------- + + +def matrix(pose: list[float]) -> np.ndarray: + """16 numbers column by column to a 4x4 matrix.""" + return np.asarray(pose, dtype=np.float64).reshape(4, 4).T + + +def rigid_problems(pose: list[float], where: str) -> list[str]: + """A rotation (orthonormal, determinant +1), a finite translation and a [0 0 0 1] last row.""" + if len(pose) != 16 or not all(math.isfinite(v) for v in pose): + return [f"{where}: pose must be 16 finite numbers"] + m = matrix(pose) + out = [] + if not np.allclose(m[3], [0, 0, 0, 1], atol=1e-9): + out.append(f"{where}: last row is {m[3].tolist()}, not [0, 0, 0, 1] (column-major?)") + r = m[:3, :3] + err = float(np.abs(r.T @ r - np.eye(3)).max()) + if err > ROTATION_TOL: + out.append(f"{where}: rotation is not orthonormal (max |Ráµ€R - I| = {err:.2g})") + elif np.linalg.det(r) < 0: + out.append(f"{where}: rotation is a reflection (determinant < 0)") + return out + + +def intrinsics_problems(k: list[float], width: int, height: int, where: str) -> list[str]: + fx, fy, cx, cy = k + out = [] + if width <= height: + out.append( + f"{where}: image is {width}x{height}; photos are stored landscape (width > height)" + ) + if fx <= 0 or fy <= 0: + return [*out, f"{where}: focal lengths must be positive, got fx={fx}, fy={fy}"] + if not (0 < cx < width and 0 < cy < height): + out.append(f"{where}: principal point ({cx}, {cy}) is outside the {width}x{height} image") + if abs(fx / fy - 1) > PIXEL_ASPECT_TOL: + out.append(f"{where}: fx/fy = {fx / fy:.3f}; intrinsics may belong to a rotated image") + fov = math.degrees(2 * math.atan(width / (2 * fx))) + if not FOV_RANGE_DEG[0] <= fov <= FOV_RANGE_DEG[1]: + out.append( + f"{where}: horizontal field of view {fov:.1f}° is outside {FOV_RANGE_DEG}; " + "intrinsics do not fit this image size" + ) + return out + + +def increasing_problems(times: list[float], where: str) -> list[str]: + for i in range(1, len(times)): + if not times[i] > times[i - 1]: + return [f"{where}: time goes from {times[i - 1]} to {times[i]} at row {i + 1}"] + return [] + + +# --- Files -------------------------------------------------------------------------------------- + + +def file_problems(files: PacketFiles, ref: dict, where: str) -> list[str]: + path = ref["path"] + if not files.exists(path): + return [f"{where}: {path} is missing"] + size = files.size(path) + if size != ref["bytes"]: + return [f"{where}: {path} is {size} bytes, manifest says {ref['bytes']}"] + if files.sha256(path) != ref["sha256"]: + return [f"{where}: {path} sha256 does not match"] + return [] + + +def file_refs(manifest: dict) -> list[tuple[str, dict]]: + """Every file the manifest names, with where it is named.""" + refs = [] + for i, p in enumerate(manifest.get("photos", [])): + refs.append((f"photos[{i}].image", p["image"])) + depth = p.get("depth") + if depth: + refs.append((f"photos[{i}].depth.map", depth["map"])) + if "confidence" in depth: + refs.append((f"photos[{i}].depth.confidence", depth["confidence"])) + for name, s in manifest.get("streams", {}).items(): + refs.append((f"streams.{name}", s)) + if manifest.get("lidar", {}).get("mesh"): + refs.append(("lidar.mesh", manifest["lidar"]["mesh"])) + if manifest.get("scene"): + refs.append(("scene", manifest["scene"])) + return refs + + +# --- Photos and depth --------------------------------------------------------------------------- + + +def image_problems(files: PacketFiles, photo: dict, where: str) -> list[str]: + try: + with files.open(photo["image"]["path"]) as f, Image.open(f) as img: + fmt, size = img.format, img.size + orientation = img.getexif().get(0x0112) + except (OSError, SyntaxError) as exc: + return [f"{where}: image cannot be read: {exc}"] + out = [] + if fmt != "JPEG": + out.append(f"{where}: image is {fmt}, not JPEG") + if size != (photo["width"], photo["height"]): + out.append( + f"{where}: image is {size[0]}x{size[1]}, manifest says " + f"{photo['width']}x{photo['height']}" + ) + if orientation not in (None, 1): + out.append(f"{where}: EXIF orientation {orientation}; store the unrotated sensor image") + return out + + +def depth_problems(files: PacketFiles, photo: dict, intact: set[str], where: str) -> list[str]: + d = photo["depth"] + w, h = d["width"], d["height"] + out = [] + if w > photo["width"] or h > photo["height"]: + out.append(f"{where}: depth {w}x{h} is larger than the photo") + photo_aspect, depth_aspect = photo["width"] / photo["height"], w / h + if abs(depth_aspect / photo_aspect - 1) > DEPTH_ASPECT_TOL: + out.append( + f"{where}: depth {w}x{h} does not have the photo's aspect " + f"({photo['width']}x{photo['height']}); it cannot be aligned by scaling" + ) + if d["map"]["bytes"] != w * h * 4: + return [ + *out, + f"{where}: depth map is {d['map']['bytes']} bytes, {w}x{h} float32 is {w * h * 4}", + ] + if d["map"]["path"] not in intact: + return out # reported by file_problems + depth = np.frombuffer(files.read(d["map"]["path"]), dtype=" 0).mean() < 0.01: + out.append(f"{where}: depth is empty (under 1% of pixels measured)") + if "confidence" in d: + if d["confidence"]["bytes"] != w * h: + out.append( + f"{where}: confidence is {d['confidence']['bytes']} bytes, {w}x{h} uint8 is {w * h}" + ) + elif d["confidence"]["path"] in intact: + conf = np.frombuffer(files.read(d["confidence"]["path"]), dtype=np.uint8) + if conf.max(initial=0) > 2: + out.append(f"{where}: confidence values must be 0, 1 or 2") + return out + + +# --- Streams ------------------------------------------------------------------------------------ + + +def read_stream(files: PacketFiles, name: str, ref: dict) -> tuple[list[str], list[list[str]]]: + text = io.TextIOWrapper(files.open(ref["path"]), encoding="utf-8", newline="") + rows = list(csv.reader(text)) + return (rows[0] if rows else []), rows[1:] + + +def stream_problems(name: str, header: list[str], rows: list[list[str]], ref: dict) -> list[str]: + where = f"streams.{name}" + want = STREAM_COLUMNS[name] + if header != want: + return [f"{where}: header is {header}, expected {want}"] + if len(rows) != ref["rows"]: + return [f"{where}: {len(rows)} rows, manifest says {ref['rows']}"] + numeric = [i for i, c in enumerate(want) if c != "tracking"] + try: + values = [[float(r[i]) for i in numeric] for r in rows] + except (ValueError, IndexError) as exc: + return [f"{where}: a row is short or not numeric ({exc})"] + if any(not math.isfinite(v) for row in values for v in row): + return [f"{where}: values must be finite"] + out = increasing_problems([row[0] for row in values], where) + if name == "trajectory": + bad = [r[1] for r in rows if r[1] not in TRACKING_STATES] + if bad: + out.append( + f"{where}: tracking must be one of {sorted(TRACKING_STATES)}, got {bad[0]!r}" + ) + q = np.asarray([row[4:8] for row in values]) + norm_err = np.abs(np.linalg.norm(q, axis=1) - 1) if len(q) else np.zeros(0) + if len(q) and norm_err.max() > QUATERNION_TOL: + out.append( + f"{where}: quaternion at row {int(norm_err.argmax()) + 1} is not unit length" + ) + return out + + +def horizontal_distance(positions: np.ndarray) -> float: + """Path length on the ground plane (meter frame x and z; y is up).""" + if len(positions) < 2: + return 0.0 + step = np.diff(positions[:, [0, 2]], axis=0) + return float(np.linalg.norm(step, axis=1).sum()) + + +# --- LiDAR -------------------------------------------------------------------------------------- + +MESH_HEADER = [ + "ply", + "format binary_little_endian 1.0", + None, # element vertex N + "property float x", + "property float y", + "property float z", + None, # element face M + "property list uchar int vertex_indices", + "property uchar classification", + "end_header", +] + + +def mesh_problems(data: bytes) -> list[str]: + """A binary PLY with exactly the layout README.md fixes, indices in range, classes 0 to 7.""" + end = data.find(b"end_header\n") + if not data.startswith(b"ply\n") or end < 0: + return ["lidar.mesh: not a PLY file"] + lines = [*data[:end].decode("ascii", "replace").rstrip("\n").split("\n"), "end_header"] + lines = [ln for ln in lines if not ln.startswith("comment")] + if len(lines) != len(MESH_HEADER): + return [f"lidar.mesh: header has {len(lines)} lines, expected the layout in README.md"] + counts = {} + for got, want in zip(lines, MESH_HEADER, strict=True): + if want is None: + parts = got.split() + if len(parts) != 3 or parts[0] != "element" or parts[1] not in ("vertex", "face"): + return [f"lidar.mesh: expected an element line, got {got!r}"] + counts[parts[1]] = int(parts[2]) + elif got != want: + return [f"lidar.mesh: header line {got!r}, expected {want!r}"] + body = memoryview(data)[end + len(b"end_header\n") :] + nv, nf = counts["vertex"], counts["face"] + vertices = np.frombuffer(body[: nv * 12], dtype="= nv): + out.append("lidar.mesh: a face index is out of range") + if nf and faces["c"].max() >= MESH_CLASSIFICATIONS: + out.append("lidar.mesh: classification must be an ARMeshClassification raw value, 0 to 7") + return out + + +# --- The whole packet --------------------------------------------------------------------------- + + +def schema_problems(manifest: dict) -> list[str]: + schema = json.loads(SCHEMA_PATH.read_text()) + errors = sorted(Draft202012Validator(schema).iter_errors(manifest), key=lambda e: list(e.path)) + return [f"schema: {'/'.join(map(str, e.path)) or '(root)'}: {e.message}" for e in errors] + + +def validate(source: Path) -> Report: + report = Report() + try: + files = PacketFiles(source) + manifest = json.loads(files.read("manifest.json")) + except (PacketError, json.JSONDecodeError, UnicodeDecodeError) as exc: + report.problems.append(str(exc)) + return report + report.problems += schema_problems(manifest) + if report.problems: + return report # the checks below rely on the shape the schema guarantees + + problems = report.problems + session = manifest["session"] + start, end = session["capture"]["started_at_uptime"], session["capture"]["ended_at_uptime"] + if not end > start: + problems.append("session.capture: ended_at_uptime must be after started_at_uptime") + + def in_window(t: float, where: str) -> None: + if not start <= t <= end: + problems.append(f"{where}: t = {t} is outside the capture [{start}, {end}]") + + problems += rigid_problems(session["meter_anchor"]["pose_in_world"], "session.meter_anchor") + + refs = file_refs(manifest) + paths = [ref["path"] for _, ref in refs] + dupes = sorted({p for p in paths if paths.count(p) > 1}) + if dupes: + problems.append(f"files named more than once: {dupes[:3]}") + # Content checks read only files whose size and hash matched, so one bad file is one problem. + intact = set() + for where, ref in refs: + found = file_problems(files, ref, where) + problems += found + if not found: + intact.add(ref["path"]) + unlisted = files.names() - set(paths) - {"manifest.json"} + if unlisted: + report.warnings.append( + f"{len(unlisted)} files are not in the manifest: {sorted(unlisted)[:3]}" + ) + + photos = manifest["photos"] + ids = [p["id"] for p in photos] + if len(set(ids)) != len(ids): + problems.append("photos: ids must be unique") + problems += increasing_problems([p["t"] for p in photos], "photos (in manifest order)") + for i, p in enumerate(photos): + where = f"photos[{i}] {p['id']}" + in_window(p["t"], where) + problems += rigid_problems(p["pose"], where) + problems += intrinsics_problems(p["intrinsics"], p["width"], p["height"], where) + if p["image"]["path"] in intact: + problems += image_problems(files, p, where) + if "depth" in p: + problems += depth_problems(files, p, intact, where) + + streams = manifest.get("streams", {}) + trajectory = None + for name, ref in streams.items(): + if ref["path"] not in intact: + continue + header, rows = read_stream(files, name, ref) + found = stream_problems(name, header, rows, ref) + problems += found + if not found and rows: + first, last = float(rows[0][0]), float(rows[-1][0]) + if first < start or last > end: + report.warnings.append( + f"streams.{name}: runs {first} to {last}, beyond the capture [{start}, {end}]" + ) + if name == "trajectory": + trajectory = np.asarray([[float(v) for v in (r[0], *r[2:5])] for r in rows]) + for name in CONSENTED_STREAMS: + if name in streams and not session.get("consent", {}).get("location", False): + problems.append(f"streams.{name} is present but session.consent.location is not true") + + walked = session["capture"].get("distance_walked_m") + if trajectory is not None: + measured = horizontal_distance(trajectory[:, 1:4]) + if walked is None: + report.warnings.append("session.capture.distance_walked_m is missing") + elif abs(walked - measured) > max(DISTANCE_ABS_TOL, DISTANCE_REL_TOL * measured): + problems.append( + f"session.capture.distance_walked_m is {walked}; " + f"the trajectory walks {measured:.3f}" + ) + for i, p in enumerate(photos): + j = int(np.abs(trajectory[:, 0] - p["t"]).argmin()) + dt = abs(trajectory[j, 0] - p["t"]) + dist = float(np.linalg.norm(trajectory[j, 1:4] - matrix(p["pose"])[:3, 3])) + if dt > PHOTO_TRAJECTORY_DT: + problems.append(f"photos[{i}] {p['id']}: no trajectory sample within {dt:.3f} s") + elif dist > PHOTO_TRAJECTORY_DIST: + problems.append( + f"photos[{i}] {p['id']}: {dist:.3f} m from the trajectory at its time; " + "poses and trajectory disagree (frame or clock)" + ) + + lidar = manifest.get("lidar", {}) + if "mesh" in lidar and lidar["mesh"]["path"] in intact: + problems += mesh_problems(files.read(lidar["mesh"]["path"])) + for i, plane in enumerate(lidar.get("planes", [])): + problems += rigid_problems(plane["pose"], f"lidar.planes[{i}]") + + photo_ids = set(ids) + for i, m in enumerate(manifest.get("marks", [])): + where = f"marks[{i}] {m['kind']}" + if "t" in m: + in_window(m["t"], where) + if len(m["points"]) != MARK_POINTS[m["kind"]]: + problems.append( + f"{where}: {len(m['points'])} points, a {m['kind']} has {MARK_POINTS[m['kind']]}" + ) + missing = [pid for pid in m.get("photo_ids", []) if pid not in photo_ids] + if missing: + problems.append(f"{where}: photo_ids {missing[:3]} are not photos in this packet") + if m["kind"] == "wall_end" and not {"side", "end_kind"} <= m.keys(): + problems.append(f"{where}: a wall end needs side and end_kind") + + for i, g in enumerate(manifest.get("guidance", [])): + where = f"guidance[{i}] {g['kind']}" + in_window(g["t_shown"], where) + resolved = g.get("t_resolved") + if resolved is not None and resolved < g["t_shown"]: + problems.append(f"{where}: resolved before it was shown") + if g["outcome"] != "unresolved" and resolved is None: + problems.append(f"{where}: outcome {g['outcome']} needs t_resolved") + + scene = manifest.get("scene") + if scene and scene["path"] in intact: + try: + doc = json.loads(files.read(scene["path"])) + except json.JSONDecodeError as exc: + problems.append(f"scene: {scene['path']} is not JSON ({exc})") + else: + want = scene.get("schema_version") + if want is not None and doc.get("schema_version") != want: + problems.append( + f"scene: schema_version {doc.get('schema_version')!r}, manifest says {want!r}" + ) + + report.summary = { + "packet_version": manifest["packet_version"], + "producer": f"{session['producer']['name']} {session['producer']['version']}", + "photos": len(photos), + "photo_size": f"{photos[0]['width']}x{photos[0]['height']}", + "photos_with_depth": sum("depth" in p for p in photos), + "streams": {n: s["rows"] for n, s in streams.items()}, + "mesh": "mesh" in lidar, + "planes": len(lidar.get("planes", [])), + "marks": len(manifest.get("marks", [])), + "guidance": len(manifest.get("guidance", [])), + "seconds": round(end - start, 2), + "distance_walked_m": walked, + } + return report diff --git a/packet/packet/write.py b/packet/packet/write.py new file mode 100644 index 000000000..c30291bfd --- /dev/null +++ b/packet/packet/write.py @@ -0,0 +1,156 @@ +"""Write a packet: files with their sizes and hashes, CSV streams, the manifest. + +The sample builders and the test fixture go through this, so every packet they produce is +described by the same code the validator checks. +""" + +from __future__ import annotations + +import csv +import hashlib +import io +import json +import shutil +import zipfile +from pathlib import Path + +import numpy as np +from PIL import Image + +from packet.validate import STREAM_COLUMNS + +PACKET_VERSION = "1.0" +SHARPNESS_METHOD = "laplacian_variance_luma_640" + + +class PacketWriter: + def __init__(self, folder: Path): + if folder.exists(): + shutil.rmtree(folder) + folder.mkdir(parents=True) + self.folder = folder + + def add_bytes(self, path: str, data: bytes) -> dict: + target = self.folder / path + target.parent.mkdir(parents=True, exist_ok=True) + target.write_bytes(data) + return {"path": path, "bytes": len(data), "sha256": hashlib.sha256(data).hexdigest()} + + def add_file(self, path: str, source: Path) -> dict: + """Copy a file in unchanged (an original photo keeps its bytes).""" + target = self.folder / path + target.parent.mkdir(parents=True, exist_ok=True) + shutil.copyfile(source, target) + digest = hashlib.sha256() + with target.open("rb") as f: + for chunk in iter(lambda: f.read(1 << 20), b""): + digest.update(chunk) + return {"path": path, "bytes": target.stat().st_size, "sha256": digest.hexdigest()} + + def add_stream(self, name: str, rows: list[list], rate_hz: float | None = None) -> dict: + buf = io.StringIO() + out = csv.writer(buf, lineterminator="\n") + out.writerow(STREAM_COLUMNS[name]) + out.writerows(rows) + ref = self.add_bytes(f"streams/{name}.csv", buf.getvalue().encode()) + ref["rows"] = len(rows) + if rate_hz is not None: + ref["nominal_rate_hz"] = rate_hz + return ref + + def add_depth(self, photo_id: str, depth: np.ndarray, confidence: np.ndarray | None) -> dict: + h, w = depth.shape + ref = { + "map": self.add_bytes(f"depth/{photo_id}.f32", depth.astype(" Path: + (self.folder / "manifest.json").write_text(json.dumps(manifest, indent=1) + "\n") + if not zip_too: + return self.folder + archive = self.folder.with_suffix(".zip") + with zipfile.ZipFile(archive, "w", zipfile.ZIP_STORED) as z: + for p in sorted(self.folder.rglob("*")): + if p.is_file(): + z.write(p, f"{self.folder.name}/{p.relative_to(self.folder).as_posix()}") + return archive + + +# --- Photo measurements ------------------------------------------------------------------------- + + +def sharpness(image: Image.Image) -> float: + """Variance of the 4-neighbour Laplacian of the luma, after scaling the long side to 640 px + (bilinear; images already smaller are used as they are). README.md fixes this method so scores + compare across phones and resolutions.""" + luma = image.convert("L") + scale = 640 / max(luma.size) + if scale < 1: + size = (round(luma.width * scale), round(luma.height * scale)) + luma = luma.resize(size, Image.Resampling.BILINEAR) + a = np.asarray(luma, dtype=np.float64) + lap = a[1:-1, :-2] + a[1:-1, 2:] + a[:-2, 1:-1] + a[2:, 1:-1] - 4 * a[1:-1, 1:-1] + return float(lap.var()) + + +# --- Frames and poses --------------------------------------------------------------------------- + + +def column_major(m: np.ndarray) -> list[float]: + return [float(v) for v in np.asarray(m).T.reshape(-1)] + + +def from_column_major(pose: list[float]) -> np.ndarray: + return np.asarray(pose, dtype=np.float64).reshape(4, 4).T + + +def orthonormalize(m: np.ndarray) -> np.ndarray: + """The nearest rotation to a rounded one (SVD), keeping the translation.""" + u, _, vt = np.linalg.svd(m[:3, :3]) + out = m.copy() + out[:3, :3] = u @ np.diag([1.0, 1.0, np.linalg.det(u @ vt)]) @ vt + return out + + +def meter_frame(meter_world: np.ndarray, wall_along_world: np.ndarray) -> np.ndarray: + """Meter-to-world pose: origin at the meter, +y up (world +y, gravity aligned), +x along the + wall to the right as seen from outside, +z out of the wall toward the homeowner.""" + x = np.array([wall_along_world[0], 0.0, wall_along_world[2]]) + x /= np.linalg.norm(x) + y = np.array([0.0, 1.0, 0.0]) + z = np.cross(x, y) + t = np.eye(4) + t[:3, 0], t[:3, 1], t[:3, 2], t[:3, 3] = x, y, z, meter_world + return t + + +def quaternion_xyzw(r: np.ndarray) -> list[float]: + """Unit quaternion (x, y, z, w) of a rotation matrix, w >= 0.""" + m = r + tr = m[0, 0] + m[1, 1] + m[2, 2] + if tr > 0: + s = 2 * np.sqrt(tr + 1) + q = [(m[2, 1] - m[1, 2]) / s, (m[0, 2] - m[2, 0]) / s, (m[1, 0] - m[0, 1]) / s, s / 4] + elif m[0, 0] > m[1, 1] and m[0, 0] > m[2, 2]: + s = 2 * np.sqrt(1 + m[0, 0] - m[1, 1] - m[2, 2]) + q = [s / 4, (m[0, 1] + m[1, 0]) / s, (m[0, 2] + m[2, 0]) / s, (m[2, 1] - m[1, 2]) / s] + elif m[1, 1] > m[2, 2]: + s = 2 * np.sqrt(1 + m[1, 1] - m[0, 0] - m[2, 2]) + q = [(m[0, 1] + m[1, 0]) / s, s / 4, (m[1, 2] + m[2, 1]) / s, (m[0, 2] - m[2, 0]) / s] + else: + s = 2 * np.sqrt(1 + m[2, 2] - m[0, 0] - m[1, 1]) + q = [(m[0, 2] + m[2, 0]) / s, (m[1, 2] + m[2, 1]) / s, s / 4, (m[1, 0] - m[0, 1]) / s] + q = np.asarray(q) + q /= np.linalg.norm(q) + return [float(v) for v in (q if q[3] >= 0 else -q)] + + +def trajectory_row(t: float, tracking: str, pose: np.ndarray, digits: int = 6) -> list: + p, q = pose[:3, 3], quaternion_xyzw(pose[:3, :3]) + return [round(t, 6), tracking, *(round(float(v), digits) for v in p), *(round(v, 7) for v in q)] diff --git a/packet/pyproject.toml b/packet/pyproject.toml new file mode 100644 index 000000000..31b0c243f --- /dev/null +++ b/packet/pyproject.toml @@ -0,0 +1,32 @@ +[project] +name = "house-scanning-packet" +version = "0.1.0" +description = "The capture packet: spec, JSON Schema, validator and sample builders" +readme = "README.md" +requires-python = ">=3.12" +dependencies = [ + "jsonschema>=4.23", + "numpy>=2.1", + "pillow>=11.0", +] + +[dependency-groups] +dev = [ + "pytest>=9.1.1", + "ruff>=0.16.9", +] + +[tool.uv] +package = false +required-version = ">=0.11.15,<0.12" + +[tool.ruff] +line-length = 100 + +[tool.ruff.lint] +select = ["E", "W", "F", "I", "B", "UP", "SIM", "RUF"] + +[tool.pytest.ini_options] +testpaths = ["tests"] +pythonpath = ["."] +addopts = ["--strict-markers", "--strict-config"] diff --git a/packet/tests/test_checks.py b/packet/tests/test_checks.py new file mode 100644 index 000000000..d92c63851 --- /dev/null +++ b/packet/tests/test_checks.py @@ -0,0 +1,219 @@ +"""Each validator check on its own, with inputs built by hand.""" + +import math +import zipfile + +import numpy as np +import pytest + +from packet.files import PacketError, PacketFiles, safe_relative +from packet.samples.synthetic import mesh_ply +from packet.validate import ( + STREAM_COLUMNS, + increasing_problems, + intrinsics_problems, + mesh_problems, + rigid_problems, + stream_problems, +) +from packet.write import column_major, quaternion_xyzw, trajectory_row + + +def pose(r=None, t=(0.0, 0.0, 0.0)) -> list[float]: + m = np.eye(4) + if r is not None: + m[:3, :3] = r + m[:3, 3] = t + return column_major(m) + + +def yaw(deg: float) -> np.ndarray: + a = math.radians(deg) + return np.array([[math.cos(a), 0, math.sin(a)], [0, 1, 0], [-math.sin(a), 0, math.cos(a)]]) + + +# --- Poses -------------------------------------------------------------------------------------- + + +def test_a_rotation_with_translation_is_rigid(): + assert rigid_problems(pose(yaw(30), (1, 2, 3)), "p") == [] + + +def test_a_reflection_is_not_a_rotation(): + assert "reflection" in rigid_problems(pose(np.diag([1, 1, -1])), "p")[0] + + +def test_a_scaled_rotation_is_not_orthonormal(): + assert "not orthonormal" in rigid_problems(pose(yaw(30) * 1.01), "p")[0] + + +def test_rotation_rounded_to_seven_digits_passes(): + rounded = [round(v, 7) for v in pose(yaw(33.3), (1, 2, 3))] + assert rigid_problems(rounded, "p") == [] + + +def test_a_row_major_pose_is_caught_by_its_last_row(): + m = np.eye(4) + m[:3, 3] = [1, 2, 3] + row_major = [float(v) for v in m.reshape(-1)] # translation lands in the last row + assert "column-major" in rigid_problems(row_major, "p")[0] + + +def test_nan_in_a_pose_is_refused(): + bad = pose() + bad[12] = math.nan + assert rigid_problems(bad, "p") == ["p: pose must be 16 finite numbers"] + + +# --- Intrinsics --------------------------------------------------------------------------------- + + +def test_phone_intrinsics_fit_a_landscape_image(): + assert intrinsics_problems([1440.0, 1440.0, 960.0, 720.0], 1920, 1440, "p") == [] + + +def test_a_portrait_image_is_refused(): + assert "landscape" in intrinsics_problems([1440.0, 1440.0, 720.0, 960.0], 1440, 1920, "p")[0] + + +def test_principal_point_outside_the_image(): + assert "outside" in intrinsics_problems([1440.0, 1440.0, 2000.0, 720.0], 1920, 1440, "p")[0] + + +def test_unequal_focal_lengths_suggest_a_rotated_image(): + assert "rotated" in intrinsics_problems([1440.0, 1080.0, 960.0, 720.0], 1920, 1440, "p")[0] + + +def test_intrinsics_for_a_larger_image_do_not_fit(): + # Intrinsics of a 4032-wide photo attached to a 480-wide thumbnail: a 19° field of view. + msgs = intrinsics_problems([1440.0, 1440.0, 240.0, 180.0], 480, 360, "p") + assert any("field of view" in m for m in msgs) + + +# --- Time --------------------------------------------------------------------------------------- + + +def test_increasing_times_pass_and_repeats_fail(): + assert increasing_problems([1.0, 2.0, 3.0], "s") == [] + assert "row 3" in increasing_problems([1.0, 2.0, 2.0], "s")[0] + assert "row 2" in increasing_problems([2.0, 1.0], "s")[0] + + +# --- Streams ------------------------------------------------------------------------------------ + + +def trajectory_rows(n: int = 3) -> list[list[str]]: + return [[str(v) for v in trajectory_row(10 + i / 60, "normal", np.eye(4))] for i in range(n)] + + +def test_a_good_trajectory_passes(): + rows = trajectory_rows() + assert stream_problems("trajectory", STREAM_COLUMNS["trajectory"], rows, {"rows": 3}) == [] + + +def test_stream_header_must_match_exactly(): + header = ["t", "x", "y"] + assert "header" in stream_problems("gyroscope", header, [], {"rows": 0})[0] + + +def test_stream_row_count_must_match_the_manifest(): + msg = stream_problems( + "trajectory", STREAM_COLUMNS["trajectory"], trajectory_rows(), {"rows": 4} + ) + assert "3 rows" in msg[0] + + +def test_stream_values_must_be_numeric(): + rows = [["1.0", "0.1", "x", "0.3"]] + assert ( + "not numeric" + in stream_problems("gyroscope", STREAM_COLUMNS["gyroscope"], rows, {"rows": 1})[0] + ) + + +def test_stream_time_must_increase(): + rows = [["1.0", "0", "0", "0"], ["1.0", "0", "0", "0"]] + assert ( + "time goes" + in stream_problems("gyroscope", STREAM_COLUMNS["gyroscope"], rows, {"rows": 2})[0] + ) + + +def test_trajectory_tracking_and_quaternions_are_checked(): + rows = trajectory_rows() + rows[1][1] = "good" + rows[2][8] = "0.9" # qw: no longer unit length + msgs = stream_problems("trajectory", STREAM_COLUMNS["trajectory"], rows, {"rows": 3}) + assert any("tracking" in m for m in msgs) + assert any("unit length" in m for m in msgs) + + +def test_quaternion_round_trip(): + q = quaternion_xyzw(yaw(120)) + x, y, z, w = q + assert math.isclose(math.hypot(x, y, z, w), 1) + assert math.isclose(math.degrees(2 * math.acos(w)), 120, abs_tol=1e-9) + assert math.isclose(y, math.sin(math.radians(60))) + + +# --- Mesh --------------------------------------------------------------------------------------- + + +def test_the_fixture_mesh_passes(): + assert mesh_problems(mesh_ply()) == [] + + +def test_mesh_face_index_out_of_range(): + data = bytearray(mesh_ply()) + # The last face's first index sits 13 bytes from the end: count(1) + 3 x int32 + class(1). + data[-13:-9] = (99).to_bytes(4, "little") + assert "out of range" in mesh_problems(bytes(data))[0] + + +def test_mesh_classification_above_seven(): + data = bytearray(mesh_ply()) + data[-1] = 8 + assert "classification" in mesh_problems(bytes(data))[0] + + +def test_truncated_mesh_is_caught(): + assert "bytes" in mesh_problems(mesh_ply()[:-5])[0] + + +def test_mesh_with_another_layout_is_refused(): + data = mesh_ply().replace(b"property float z", b"property double z") + assert "header line" in mesh_problems(data)[0] + + +# --- Opening packets ---------------------------------------------------------------------------- + + +@pytest.mark.parametrize( + ("path", "ok"), + [("photos/p1.jpg", True), ("/etc/passwd", False), ("a/../../b", False), ("a\\b", False)], +) +def test_safe_relative(path, ok): + assert safe_relative(path) is ok + + +def test_a_zip_entry_that_climbs_out_is_refused(tmp_path): + archive = tmp_path / "p.zip" + with zipfile.ZipFile(archive, "w") as z: + z.writestr("manifest.json", "{}") + z.writestr("../evil.txt", "x") + with pytest.raises(PacketError, match="leave the packet"): + PacketFiles(archive) + + +def test_a_zip_with_two_manifests_is_refused(tmp_path): + archive = tmp_path / "p.zip" + with zipfile.ZipFile(archive, "w") as z: + z.writestr("a/manifest.json", "{}") + z.writestr("b/manifest.json", "{}") + with pytest.raises(PacketError, match="exactly one manifest"): + PacketFiles(archive) + + +def test_a_folder_without_a_manifest_is_refused(tmp_path): + with pytest.raises(PacketError, match="no manifest"): + PacketFiles(tmp_path) diff --git a/packet/tests/test_packet.py b/packet/tests/test_packet.py new file mode 100644 index 000000000..79aa12410 --- /dev/null +++ b/packet/tests/test_packet.py @@ -0,0 +1,242 @@ +"""The validator on the committed fixture, and on copies of it broken one way at a time.""" + +import hashlib +import io +import json +import shutil +import zipfile +from pathlib import Path + +import numpy as np +import pytest +from PIL import Image + +from packet.__main__ import main +from packet.samples import synthetic +from packet.validate import validate + +FIXTURE = Path(__file__).resolve().parents[1] / "fixtures" / "synthetic" + + +@pytest.fixture +def packet(tmp_path) -> Path: + copy = tmp_path / "packet" + shutil.copytree(FIXTURE, copy) + return copy + + +def manifest(folder: Path) -> dict: + return json.loads((folder / "manifest.json").read_text()) + + +def save(folder: Path, m: dict) -> None: + (folder / "manifest.json").write_text(json.dumps(m)) + + +def replace_file(folder: Path, ref: dict, data: bytes) -> None: + """Write new bytes and update the manifest entry, so only the content check can object.""" + (folder / ref["path"]).write_bytes(data) + ref["bytes"], ref["sha256"] = len(data), hashlib.sha256(data).hexdigest() + + +def only_problem(folder: Path) -> str: + report = validate(folder) + assert len(report.problems) == 1, report.problems + return report.problems[0] + + +def test_the_committed_fixture_is_what_the_builder_writes(tmp_path): + rebuilt = synthetic.build(tmp_path / "synthetic") + for f in FIXTURE.rglob("*"): + if f.is_file(): + assert f.read_bytes() == (rebuilt / f.relative_to(FIXTURE)).read_bytes(), f + + +def test_the_fixture_is_valid_as_a_folder_and_as_a_zip(tmp_path): + assert validate(FIXTURE).problems == [] + archive = tmp_path / "p.zip" + with zipfile.ZipFile(archive, "w") as z: + for f in FIXTURE.rglob("*"): + if f.is_file(): + z.write(f, f"synthetic/{f.relative_to(FIXTURE).as_posix()}") + report = validate(archive) + assert report.ok, report.problems + assert report.summary["photos"] == 3 + + +def test_cli_exit_codes(packet, capsys): + assert main(["validate", str(packet)]) == 0 + assert capsys.readouterr().out.startswith(f"{packet}: valid") + m = manifest(packet) + m["packet_version"] = "2.0" + save(packet, m) + assert main(["validate", str(packet), "--json"]) == 1 + report = json.loads(capsys.readouterr().out) + assert report["ok"] is False and report["problems"][0].startswith("schema: packet_version") + + +def test_a_newer_major_version_is_refused(packet): + m = manifest(packet) + m["packet_version"] = "2.0" + save(packet, m) + assert "packet_version" in only_problem(packet) + + +def test_an_unknown_field_is_accepted(packet): + m = manifest(packet) + m["session"]["something_new"] = {"added_in": "1.1"} + save(packet, m) + assert validate(packet).ok + + +def test_a_missing_file(packet): + (packet / "photos" / "p00002.jpg").unlink() + assert "is missing" in only_problem(packet) + + +def test_a_file_whose_size_changed(packet): + with (packet / "streams" / "gyroscope.csv").open("a") as f: + f.write("\n") + assert "bytes, manifest says" in only_problem(packet) + + +def test_a_file_whose_content_changed(packet): + path = packet / "lidar" / "mesh.ply" + data = bytearray(path.read_bytes()) + data[-1] ^= 1 # same size, different bytes + path.write_bytes(bytes(data)) + assert "sha256" in validate(packet).problems[0] + + +def test_an_unlisted_file_is_a_warning_only(packet): + (packet / "notes.txt").write_text("hello") + report = validate(packet) + assert report.ok and "not in the manifest" in report.warnings[0] + + +def test_image_size_must_match_the_manifest(packet): + m = manifest(packet) + buf = io.BytesIO() + Image.new("RGB", (80, 60)).save(buf, "JPEG") + replace_file(packet, m["photos"][0]["image"], buf.getvalue()) + save(packet, m) + assert "80x60" in only_problem(packet) + + +def test_a_rotated_photo_is_refused(packet): + m = manifest(packet) + img = Image.open(packet / m["photos"][0]["image"]["path"]) + exif = Image.Exif() + exif[0x0112] = 6 # "rotate 90° clockwise to display" + buf = io.BytesIO() + img.save(buf, "JPEG", exif=exif.tobytes()) + replace_file(packet, m["photos"][0]["image"], buf.getvalue()) + save(packet, m) + assert "EXIF orientation 6" in only_problem(packet) + + +def test_photos_out_of_time_order(packet): + m = manifest(packet) + m["photos"][0]["t"], m["photos"][1]["t"] = m["photos"][1]["t"], m["photos"][0]["t"] + save(packet, m) + assert any("time goes" in p for p in validate(packet).problems) + + +def test_a_photo_off_its_trajectory(packet): + m = manifest(packet) + m["photos"][1]["pose"][12] += 0.1 # 10 cm along x + save(packet, m) + assert "from the trajectory" in only_problem(packet) + + +def test_a_photo_between_trajectory_samples(packet): + m = manifest(packet) + m["photos"][2]["t"] = 102.3 # after the trajectory's last sample at 102.0, inside the capture + save(packet, m) + assert "no trajectory sample" in only_problem(packet) + + +def test_distance_walked_must_match_the_trajectory(packet): + m = manifest(packet) + m["session"]["capture"]["distance_walked_m"] = 2.5 + save(packet, m) + assert "trajectory walks 2.000" in only_problem(packet) + + +def test_location_without_consent(packet): + m = manifest(packet) + m["session"]["consent"]["location"] = False + save(packet, m) + problems = validate(packet).problems + assert [p.split(" is present")[0] for p in problems] == ["streams.location", "streams.heading"] + + +def test_depth_with_the_wrong_aspect(packet): + m = manifest(packet) + d = m["photos"][0]["depth"] + replace_file(packet, d["map"], np.full((20, 20), 2.5, " Date: Sat, 26 Sep 2026 12:08:23 -0500 Subject: [PATCH 2/9] =?UTF-8?q?packet:=201.1=20=E2=80=94=20depth=20frames,?= =?UTF-8?q?=20estimated=20depth=20with=20sigma,=20top-level=20planes,=20me?= =?UTF-8?q?sh=20classification=20flag?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit From S6's proposals on #22. All additive: a 1.0 packet still validates, and the two new requirements (confidence for ARKit depth, the flag with a mesh) apply only to packets that say 1.1. lidar.planes stays readable but deprecated. The validator checks sigma, depth frames against the trajectory and their own grid, plane boundaries inside their extent, and class 0 everywhere when classification was off. Co-Authored-By: Claude Opus 5.5 (1M context) --- packet/README.md | 62 +- .../synthetic/depth_frames/d00001.conf.u8 | 1 + .../synthetic/depth_frames/d00001.f32 | Bin 0 -> 1728 bytes .../synthetic/depth_frames/d00002.f32 | Bin 0 -> 1728 bytes .../synthetic/depth_frames/d00002.sigma.f32 | 1 + packet/fixtures/synthetic/manifest.json | 229 ++++-- packet/manifest.schema.json | 675 +++++++++++++++--- packet/packet/samples/synthetic.py | 78 +- packet/packet/validate.py | 135 +++- packet/packet/write.py | 19 +- packet/tests/test_checks.py | 8 + packet/tests/test_packet.py | 105 ++- 12 files changed, 1075 insertions(+), 238 deletions(-) create mode 100644 packet/fixtures/synthetic/depth_frames/d00001.conf.u8 create mode 100644 packet/fixtures/synthetic/depth_frames/d00001.f32 create mode 100644 packet/fixtures/synthetic/depth_frames/d00002.f32 create mode 100644 packet/fixtures/synthetic/depth_frames/d00002.sigma.f32 diff --git a/packet/README.md b/packet/README.md index cab190135..0f74f683a 100644 --- a/packet/README.md +++ b/packet/README.md @@ -7,7 +7,7 @@ the placement rules. One packet is one scan session: a folder, or a zip of one, [`manifest.schema.json`](manifest.schema.json) checks the manifest's shape, and `python -m packet validate` checks everything below. -Version 1.0. The **App** column in each table says what `t3/ios-mvf` at `a39d0a5` records: +Version 1.1 ([what changed from 1.0](#versioning)). The **App** column in each table says what `t3/ios-mvf` at `a39d0a5` records: **now** (exported today), **partial** (computed in the app but not exported, or exported in another form) or **planned**. [What the app records today](#what-the-app-records-today) lists the gaps. @@ -19,6 +19,7 @@ manifest.json what everything is (this document) photos/p00001.jpg full-resolution photos, one per kept frame depth/p00001.f32 depth aligned to that photo (LiDAR phones) depth/p00001.conf.u8 its confidence +depth_frames/d00001.* depth recorded between photos, without an image (1.1) streams/*.csv trajectory, IMU, barometer, location, heading lidar/mesh.ply ARKit mesh with per-face classification (LiDAR phones) scene.json the app's placement request (C1), unchanged @@ -67,7 +68,9 @@ These hold for every field unless its row says otherwise. pixels x ∈ [c·W/w, (c+1)·W/w), y ∈ [r·H/h, (r+1)·H/h). Values are float32 little-endian meters along the camera's −z axis (not the distance along the ray), row by row from the top, and 0 means no measurement. Confidence is one byte per depth pixel, ARKit's - `ARConfidenceLevel`: 0 low, 1 medium, 2 high. + `ARConfidenceLevel`: 0 low, 1 medium, 2 high. Sigma (1.1) is float32 meters per depth pixel, + one standard deviation, in the same layout. A depth frame has no photo: its `intrinsics` + describe the depth map's own pixel grid. ## Session @@ -80,6 +83,7 @@ These hold for every field unless its row says otherwise. | `session.device.ios_version` | `UIDevice.systemVersion` | planned | | `session.device.lidar` | `ARWorldTrackingConfiguration.supportsFrameSemantics(.sceneDepth)` | planned | | `session.device.scene_depth_enabled`, `mesh_enabled` | What the session actually ran with | planned | +| `session.device.mesh_classification_enabled` | 1.1. `sceneReconstruction` included `.meshWithClassification`. Required with a mesh; when false every face's class is 0, which otherwise could not be told from ARKit's own "none" | planned (S6) | | `session.capture.started_at`, `started_at_uptime`, `ended_at_uptime` | Wall clock and uptime at the first frame, uptime at the last | planned | | `session.capture.distance_walked_m` | Horizontal length of the trajectory (meter frame x and z). Must match `streams.trajectory` within 1% | planned | | `session.world_alignment` | Always `"gravity"` | partial: the app runs with `.gravity` but does not write it | @@ -103,7 +107,7 @@ One entry per kept frame, in time order. Required: `id`, `image`, `width`, `heig | `exposure.duration_s`, `exposure.iso`, `exposure.offset_ev` | `ARCamera.exposureDuration` and `exposureOffset` (iOS 15); ISO from the high-resolution still's metadata | planned | | `lens.focal_length_mm`, `lens.f_number`, `lens.camera` | From the still's EXIF; `camera` is `wide`, `ultra_wide` or `telephoto` | planned | | `sharpness` | `{"method": "laplacian_variance_luma_640", "value"}`, defined below | partial: the app scores sharpness another way and does not export it | -| `depth` | `map`, optional `confidence`, `width`, `height`, `source` (`arkit_scene_depth`, `arkit_smoothed_scene_depth`, or `rendered_from_laser_scan` in samples) | planned (S6, LiDAR) | +| `depth` | `map`, `confidence`, `sigma`, `width`, `height`, `source`. Sources: `arkit_scene_depth` and `arkit_smoothed_scene_depth` (LiDAR; from 1.1 they need `confidence`, which marks glass, dark surfaces and edges low); `estimated` (1.1: metric depth inferred from the image on a phone without LiDAR; needs `sigma`, since the uncertainty decides what counts as a surface); `rendered_from_laser_scan` (samples only) | planned (S6) | **Sharpness** is the variance of the 4-neighbour Laplacian of the image's luma, after scaling the long side to 640 px with bilinear filtering (images already smaller are used as they are). One @@ -150,12 +154,33 @@ end_header ``` Every face is a triangle. `classification` is `ARMeshClassification`'s raw value: 0 none, 1 wall, -2 floor, 3 ceiling, 4 table, 5 seat, 6 window, 7 door. `comment` lines may follow `ply`. - -`lidar.planes` lists `ARPlaneAnchor`s: `id`, `alignment` (`horizontal` or `vertical`), -`classification`, `pose` (plane to meter frame; the plane is its local x-z plane and its normal -is +y), and `extent_m` ([x, z]). App: planned. The app reads plane anchors for its coverage and -taps but does not export them. +2 floor, 3 ceiling, 4 table, 5 seat, 6 window, 7 door. When classification was off, every class +is 0 and `session.device.mesh_classification_enabled` is false (1.1). `comment` lines may follow +`ply`. + +## Depth frames + +`depth_frames` (1.1) holds depth recorded between photos, with no image: `id`, `t`, `pose` +(camera to meter frame), `intrinsics` for the depth map's grid, `width`, `height`, `map`, and +`confidence` or `sigma` as for a photo's depth. Photos are kept about every 0.5 m, but an +on-device map fuses depth at about 10 Hz, so photo-rate depth alone cannot rebuild the +occupancy the phone used to decide what was seen. Each frame must sit on the trajectory like a +photo. Cost: at 256 × 192 a frame with confidence is about 246 KB, so 10 Hz is about 2.5 MB/s; +record a few hertz. App: planned (S6). + +## Planes + +`planes` (1.1, top level) lists `ARPlaneAnchor`s, which every ARKit phone detects, LiDAR or not: +`id`, `alignment` (`horizontal` or `vertical`), `classification`, `pose`, `extent_m` and optional +`boundary_m`. `pose` maps the plane into the meter frame and already includes the extent's +centre and `planeExtent.rotationOnYAxis`, so `extent_m` ([x, z]) is a rectangle centred on it. +ARKit's anchor transform alone is not that centre. The plane is its local x-z plane and its +normal is +y. `boundary_m` is `geometry.boundaryVertices` as [x, z] in the same frame as +`extent_m`, since ARKit's extent often claims a full rectangle over an L of wall or a wall broken +by a window. The validator checks that the boundary lies inside the extent, which catches a +boundary left in the anchor's frame. 1.0 put planes in `lidar.planes`; 1.x readers still accept +it there, but a packet may not use both. App: planned. The app reads plane anchors for its +coverage and taps but does not export them. ## Marks, guidance and the scene @@ -193,6 +218,13 @@ field, stream or enum value a reader may skip) and the major version for anythin would misread: a renamed or removed field, a changed unit, frame or column order. The validator accepts `1.x`. +1.1 adds, from S6's proposals on #22: `depth_frames`; the `estimated` depth source with +`sigma`; top-level `planes` with `boundary_m`, and the statement of what a plane's `pose` +includes; and `session.device.mesh_classification_enabled`. Two rules apply only to packets +that say 1.1 or later, so 1.0 packets stay valid: ARKit depth needs `confidence`, and a mesh +needs the classification flag. A 1.0 reader of a 1.1 packet misses depth frames, sigma and +top-level planes, and reads everything else as before. + ## Transport This is a recommendation; Hunter's team decides. A packet does not fit in one request. The @@ -244,8 +276,9 @@ order of cost: log. 5. **New sensors:** Core Motion (accelerometer, gyroscope, magnetometer, device motion, altimeter); location and heading behind a consent prompt. -6. **LiDAR (S6):** `sceneDepth` with confidence per photo, the mesh with classification, plane - anchors. +6. **LiDAR and the live map (S6):** `sceneDepth` with confidence per photo, depth frames between + photos, estimated depth with sigma on phones without LiDAR, the mesh with its + classification flag, plane anchors with boundaries. 7. **Transport:** the manifest-then-files upload above. ## Tools @@ -283,7 +316,10 @@ The validator checks: - the stream columns, unit quaternions and tracking values; - that photos sit on the trajectory, and that distance walked matches it; - that depth has its photo's aspect and the right byte count, with finite, non-negative values - and confidence of at most 2; -- the mesh layout, face indices and classes; + and sigma, confidence of at most 2, and confidence for ARKit depth and sigma for estimated + depth; +- that depth frames sit on the trajectory, in time order, with intrinsics that fit their grid; +- that plane boundaries lie inside their extent, and planes appear in one place only; +- the mesh layout, face indices and classes, and the classification flag; - mark and guidance consistency; - consent for location and heading. diff --git a/packet/fixtures/synthetic/depth_frames/d00001.conf.u8 b/packet/fixtures/synthetic/depth_frames/d00001.conf.u8 new file mode 100644 index 000000000..5e25ef9e5 --- /dev/null +++ b/packet/fixtures/synthetic/depth_frames/d00001.conf.u8 @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/packet/fixtures/synthetic/depth_frames/d00001.f32 b/packet/fixtures/synthetic/depth_frames/d00001.f32 new file mode 100644 index 0000000000000000000000000000000000000000..602e2160e5dcc3e531c86c68dbe1cce17eee4cae GIT binary patch literal 1728 ecmZQzP;g)v#iL;`ng&MGz-Ss6O#?$L4FCXozM=sD literal 0 HcmV?d00001 diff --git a/packet/fixtures/synthetic/depth_frames/d00002.f32 b/packet/fixtures/synthetic/depth_frames/d00002.f32 new file mode 100644 index 0000000000000000000000000000000000000000..602e2160e5dcc3e531c86c68dbe1cce17eee4cae GIT binary patch literal 1728 ecmZQzP;g)v#iL;`ng&MGz-Ss6O#?$L4FCXozM=sD literal 0 HcmV?d00001 diff --git a/packet/fixtures/synthetic/depth_frames/d00002.sigma.f32 b/packet/fixtures/synthetic/depth_frames/d00002.sigma.f32 new file mode 100644 index 000000000..aaaa711c4 --- /dev/null +++ b/packet/fixtures/synthetic/depth_frames/d00002.sigma.f32 @@ -0,0 +1 @@ +š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™> \ No newline at end of file diff --git a/packet/fixtures/synthetic/manifest.json b/packet/fixtures/synthetic/manifest.json index f078b4199..3067f2447 100644 --- a/packet/fixtures/synthetic/manifest.json +++ b/packet/fixtures/synthetic/manifest.json @@ -1,5 +1,5 @@ { - "packet_version": "1.0", + "packet_version": "1.1", "session": { "id": "synthetic-0001", "producer": { @@ -12,7 +12,8 @@ "ios_version": "n/a", "lidar": true, "scene_depth_enabled": true, - "mesh_enabled": true + "mesh_enabled": true, + "mesh_classification_enabled": true }, "capture": { "started_at": "2026-09-26T12:00:00Z", @@ -307,69 +308,187 @@ "nominal_rate_hz": 1 } }, + "depth_frames": [ + { + "id": "d00001", + "t": 100.5, + "pose": [ + 1.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0, + 0.0, + -0.5, + 0.0, + 2.5, + 1.0 + ], + "intrinsics": [ + 20.0, + 20.0, + 12.0, + 9.0 + ], + "tracking": { + "state": "normal", + "reason": null + }, + "map": { + "path": "depth_frames/d00001.f32", + "bytes": 1728, + "sha256": "6bedc12ac8ed368df03325e5ee0a12a34db6c5d26bdd43ad86665ba1cc11a5b9" + }, + "width": 24, + "height": 18, + "confidence": { + "path": "depth_frames/d00001.conf.u8", + "bytes": 432, + "sha256": "50df04e966aa60b11660f8a8c2ca6456aea34ee7137ddea4c1687989085439b8" + }, + "source": "arkit_scene_depth" + }, + { + "id": "d00002", + "t": 101.25, + "pose": [ + 1.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0, + 0.0, + 0.25, + 0.0, + 2.5, + 1.0 + ], + "intrinsics": [ + 20.0, + 20.0, + 12.0, + 9.0 + ], + "tracking": { + "state": "normal", + "reason": null + }, + "map": { + "path": "depth_frames/d00002.f32", + "bytes": 1728, + "sha256": "6bedc12ac8ed368df03325e5ee0a12a34db6c5d26bdd43ad86665ba1cc11a5b9" + }, + "width": 24, + "height": 18, + "sigma": { + "path": "depth_frames/d00002.sigma.f32", + "bytes": 1728, + "sha256": "3cd6948e48938afac643b908d3f182743276f0a59e205fe2d6986475d060810d" + }, + "source": "estimated" + } + ], "lidar": { "mesh": { "path": "lidar/mesh.ply", "bytes": 369, "sha256": "6424380bcc053a1e5ea20a26d196ffa5d1140e3f6e70d0def5b9b20f8c2a97b4" - }, - "planes": [ - { - "id": "wall", - "alignment": "vertical", - "classification": "wall", - "pose": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 + } + }, + "planes": [ + { + "id": "wall", + "alignment": "vertical", + "classification": "wall", + "pose": [ + 1.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0, + 0.0, + 0.0, + -1.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0 + ], + "extent_m": [ + 6.0, + 2.4 + ], + "boundary_m": [ + [ + -3.0, + -1.2 ], - "extent_m": [ - 6.0, - 2.4 - ] - }, - { - "id": "ground", - "alignment": "horizontal", - "classification": "floor", - "pose": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, + [ + 3.0, + -1.2 + ], + [ + 3.0, + 0.2 + ], + [ 0.0, - -1.2, + 0.2 + ], + [ 0.0, - 1.0 + 1.2 ], - "extent_m": [ - 6.0, - 3.0 + [ + -3.0, + 1.2 ] - } - ] - }, + ] + }, + { + "id": "ground", + "alignment": "horizontal", + "classification": "floor", + "pose": [ + 1.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0, + 0.0, + 0.0, + -1.2, + 0.0, + 1.0 + ], + "extent_m": [ + 6.0, + 3.0 + ] + } + ], "marks": [ { "id": "m1", diff --git a/packet/manifest.schema.json b/packet/manifest.schema.json index ae35cc58e..dd235455b 100644 --- a/packet/manifest.schema.json +++ b/packet/manifest.schema.json @@ -5,50 +5,99 @@ "description": "One scan session: the files in the packet and what they mean. README.md in this folder is the full specification; this schema checks shape only. Lengths in meters, times in seconds of device uptime, poses as 16 numbers column by column.", "type": "object", "additionalProperties": true, - "required": ["packet_version", "session", "photos"], + "required": [ + "packet_version", + "session", + "photos" + ], "properties": { "packet_version": { - "description": "MAJOR.MINOR. A reader accepts any packet with its own major version and ignores fields it does not know.", + "description": "MAJOR.MINOR. A reader accepts any packet with its own major version and ignores fields it does not know. 1.1 adds depth frames, estimated depth with sigma, top-level planes with boundaries, and the mesh classification flag.", "type": "string", "pattern": "^1\\.[0-9]+$" }, - "session": { "$ref": "#/$defs/session" }, + "session": { + "$ref": "#/$defs/session" + }, "photos": { "type": "array", "minItems": 1, - "items": { "$ref": "#/$defs/photo" } + "items": { + "$ref": "#/$defs/photo" + } + }, + "depth_frames": { + "type": "array", + "items": { + "$ref": "#/$defs/depth_frame" + } + }, + "streams": { + "$ref": "#/$defs/streams" + }, + "planes": { + "description": "1.1. ARPlaneAnchors, with or without LiDAR.", + "type": "array", + "items": { + "$ref": "#/$defs/plane" + } + }, + "lidar": { + "$ref": "#/$defs/lidar" }, - "streams": { "$ref": "#/$defs/streams" }, - "lidar": { "$ref": "#/$defs/lidar" }, "marks": { "type": "array", - "items": { "$ref": "#/$defs/mark" } + "items": { + "$ref": "#/$defs/mark" + } }, "guidance": { "type": "array", - "items": { "$ref": "#/$defs/guidance_event" } + "items": { + "$ref": "#/$defs/guidance_event" + } }, "scene": { "description": "The app's own scene.json (C1), exactly as it would be sent for placement.", - "allOf": [{ "$ref": "#/$defs/file" }], + "allOf": [ + { + "$ref": "#/$defs/file" + } + ], "type": "object", - "properties": { "schema_version": { "type": "string" } } + "properties": { + "schema_version": { + "type": "string" + } + } }, - "provenance": { "$ref": "#/$defs/provenance" } + "provenance": { + "$ref": "#/$defs/provenance" + } }, "$defs": { "file": { "description": "A file inside the packet. `path` is relative to the manifest, uses '/', and never leaves the packet.", "type": "object", - "required": ["path", "bytes", "sha256"], + "required": [ + "path", + "bytes", + "sha256" + ], "properties": { "path": { "type": "string", "minLength": 1, "pattern": "^(?!/)(?!.*(^|/)\\.\\.(/|$))[A-Za-z0-9._/-]+$" }, - "bytes": { "type": "integer", "minimum": 0 }, - "sha256": { "type": "string", "pattern": "^[0-9a-f]{64}$" } + "bytes": { + "type": "integer", + "minimum": 0 + }, + "sha256": { + "type": "string", + "pattern": "^[0-9a-f]{64}$" + } } }, "pose": { @@ -56,13 +105,17 @@ "type": "array", "minItems": 16, "maxItems": 16, - "items": { "type": "number" } + "items": { + "type": "number" + } }, "vec3": { "type": "array", "minItems": 3, "maxItems": 3, - "items": { "type": "number" } + "items": { + "type": "number" + } }, "time": { "description": "Seconds of device uptime: the clock of ARFrame.timestamp and CMLogItem.timestamp.", @@ -71,48 +124,117 @@ }, "tracking": { "type": "object", - "required": ["state"], + "required": [ + "state" + ], "properties": { - "state": { "enum": ["normal", "limited", "not_available"] }, + "state": { + "enum": [ + "normal", + "limited", + "not_available" + ] + }, "reason": { - "enum": ["initializing", "relocalizing", "excessive_motion", "insufficient_features", null] + "enum": [ + "initializing", + "relocalizing", + "excessive_motion", + "insufficient_features", + null + ] } } }, "session": { "type": "object", - "required": ["id", "producer", "device", "capture", "world_alignment", "meter_anchor"], + "required": [ + "id", + "producer", + "device", + "capture", + "world_alignment", + "meter_anchor" + ], "properties": { - "id": { "type": "string", "minLength": 1 }, + "id": { + "type": "string", + "minLength": 1 + }, "producer": { "description": "What wrote this packet: the app, or a converter building a sample from a dataset.", "type": "object", - "required": ["kind", "name", "version"], + "required": [ + "kind", + "name", + "version" + ], "properties": { - "kind": { "enum": ["app", "converter"] }, - "name": { "type": "string" }, - "version": { "type": "string" }, - "commit": { "type": "string" } + "kind": { + "enum": [ + "app", + "converter" + ] + }, + "name": { + "type": "string" + }, + "version": { + "type": "string" + }, + "commit": { + "type": "string" + } } }, "device": { "type": "object", - "required": ["model", "lidar"], + "required": [ + "model", + "lidar" + ], "properties": { - "model": { "description": "Hardware identifier, e.g. iPhone16,1.", "type": "string" }, - "ios_version": { "type": "string" }, - "lidar": { "description": "The device has a LiDAR scanner (sceneDepth supported).", "type": "boolean" }, - "scene_depth_enabled": { "type": "boolean" }, - "mesh_enabled": { "type": "boolean" } + "model": { + "description": "Hardware identifier, e.g. iPhone16,1.", + "type": "string" + }, + "ios_version": { + "type": "string" + }, + "lidar": { + "description": "The device has a LiDAR scanner (sceneDepth supported).", + "type": "boolean" + }, + "scene_depth_enabled": { + "type": "boolean" + }, + "mesh_enabled": { + "type": "boolean" + }, + "mesh_classification_enabled": { + "type": "boolean", + "description": "1.1. ARKit's sceneReconstruction included .meshWithClassification. Required with a mesh from 1.1; when false every face's class is 0." + } } }, "capture": { "type": "object", - "required": ["started_at_uptime", "ended_at_uptime"], + "required": [ + "started_at_uptime", + "ended_at_uptime" + ], "properties": { - "started_at": { "description": "Wall clock at started_at_uptime, ISO 8601 UTC.", "type": "string", "format": "date-time" }, - "started_at_uptime": { "$ref": "#/$defs/time" }, - "ended_at_uptime": { "$ref": "#/$defs/time" }, + "started_at": { + "description": "Wall clock at started_at_uptime, ISO 8601 UTC.", + "type": "string", + "format": "date-time" + }, + "started_at_uptime": { + "$ref": "#/$defs/time" + }, + "ended_at_uptime": { + "$ref": "#/$defs/time" + }, "distance_walked_m": { "description": "Horizontal length of the camera trajectory. Must match streams.trajectory within 1%.", "type": "number", @@ -120,39 +242,70 @@ } } }, - "world_alignment": { "const": "gravity" }, + "world_alignment": { + "const": "gravity" + }, "meter_anchor": { "description": "The meter frame, as a pose in ARKit's world frame of this session. Every other pose and point in the packet is in the meter frame.", "type": "object", - "required": ["pose_in_world"], + "required": [ + "pose_in_world" + ], "properties": { - "pose_in_world": { "$ref": "#/$defs/pose" }, + "pose_in_world": { + "$ref": "#/$defs/pose" + }, "ground_y_m": { "description": "Height of the ground at the meter, on the meter frame's y axis (negative: the ground is below the meter).", - "type": ["number", "null"] + "type": [ + "number", + "null" + ] } } }, "consent": { "type": "object", "properties": { - "location": { "description": "The homeowner allowed location and heading. Required true if either stream is present.", "type": "boolean" } + "location": { + "description": "The homeowner allowed location and heading. Required true if either stream is present.", + "type": "boolean" + } } } } }, "photo": { "type": "object", - "required": ["id", "image", "width", "height", "t", "pose", "intrinsics"], + "required": [ + "id", + "image", + "width", + "height", + "t", + "pose", + "intrinsics" + ], "properties": { - "id": { "type": "string", "pattern": "^[A-Za-z0-9_-]+$" }, + "id": { + "type": "string", + "pattern": "^[A-Za-z0-9_-]+$" + }, "image": { "description": "The full-resolution photo as captured, JPEG, stored unrotated: landscape as the sensor delivers it, EXIF orientation absent or 1.", "$ref": "#/$defs/file" }, - "width": { "type": "integer", "minimum": 1 }, - "height": { "type": "integer", "minimum": 1 }, - "t": { "$ref": "#/$defs/time" }, + "width": { + "type": "integer", + "minimum": 1 + }, + "height": { + "type": "integer", + "minimum": 1 + }, + "t": { + "$ref": "#/$defs/time" + }, "pose": { "description": "Camera to meter frame. ARKit camera axes: +x right and +y up in the stored image, the camera looks along -z.", "$ref": "#/$defs/pose" @@ -162,72 +315,174 @@ "type": "array", "minItems": 4, "maxItems": 4, - "items": { "type": "number" } + "items": { + "type": "number" + } + }, + "tracking": { + "$ref": "#/$defs/tracking" }, - "tracking": { "$ref": "#/$defs/tracking" }, "exposure": { "type": "object", "properties": { - "duration_s": { "type": "number", "exclusiveMinimum": 0 }, - "iso": { "type": "number", "exclusiveMinimum": 0 }, - "offset_ev": { "type": "number" } + "duration_s": { + "type": "number", + "exclusiveMinimum": 0 + }, + "iso": { + "type": "number", + "exclusiveMinimum": 0 + }, + "offset_ev": { + "type": "number" + } } }, "lens": { "type": "object", "properties": { - "focal_length_mm": { "type": "number", "exclusiveMinimum": 0 }, - "f_number": { "type": "number", "exclusiveMinimum": 0 }, - "camera": { "description": "Which camera: wide, ultra_wide, telephoto, or a dataset's camera name.", "type": "string" } + "focal_length_mm": { + "type": "number", + "exclusiveMinimum": 0 + }, + "f_number": { + "type": "number", + "exclusiveMinimum": 0 + }, + "camera": { + "description": "Which camera: wide, ultra_wide, telephoto, or a dataset's camera name.", + "type": "string" + } } }, "sharpness": { "type": "object", - "required": ["method", "value"], + "required": [ + "method", + "value" + ], "properties": { - "method": { "const": "laplacian_variance_luma_640" }, - "value": { "type": "number", "minimum": 0 } + "method": { + "const": "laplacian_variance_luma_640" + }, + "value": { + "type": "number", + "minimum": 0 + } } }, - "depth": { "$ref": "#/$defs/depth" } + "depth": { + "$ref": "#/$defs/depth" + } } }, "depth": { - "description": "Depth aligned to the photo: same field of view and orientation, lower resolution. Float32 little-endian meters along the camera's -z, row by row from the top; 0 means no measurement.", + "description": "Depth aligned to an image: float32 little-endian meters along the camera's -z, row by row from the top; 0 means no measurement. For a photo it covers the photo's field of view at the same aspect; for a depth frame its own intrinsics describe it.", "type": "object", - "required": ["map", "width", "height"], + "required": [ + "map", + "width", + "height" + ], "properties": { - "map": { "$ref": "#/$defs/file" }, + "map": { + "$ref": "#/$defs/file" + }, "confidence": { "description": "UInt8 per depth pixel, ARConfidenceLevel: 0 low, 1 medium, 2 high.", "$ref": "#/$defs/file" }, - "width": { "type": "integer", "minimum": 1 }, - "height": { "type": "integer", "minimum": 1 }, - "source": { "enum": ["arkit_scene_depth", "arkit_smoothed_scene_depth", "rendered_from_laser_scan"] } - } + "width": { + "type": "integer", + "minimum": 1 + }, + "height": { + "type": "integer", + "minimum": 1 + }, + "source": { + "enum": [ + "arkit_scene_depth", + "arkit_smoothed_scene_depth", + "estimated", + "rendered_from_laser_scan" + ], + "description": "estimated: metric depth inferred from the image (no LiDAR); needs sigma. From 1.1, the ARKit sources need confidence." + }, + "sigma": { + "description": "1.1. Float32 little-endian meters, one standard deviation per depth pixel, same layout as map. Required when source is estimated.", + "$ref": "#/$defs/file" + } + }, + "allOf": [ + { + "if": { + "properties": { + "source": { + "const": "estimated" + } + }, + "required": [ + "source" + ] + }, + "then": { + "required": [ + "sigma" + ] + } + } + ] }, "stream": { - "allOf": [{ "$ref": "#/$defs/file" }], + "allOf": [ + { + "$ref": "#/$defs/file" + } + ], "type": "object", - "required": ["rows"], + "required": [ + "rows" + ], "properties": { - "rows": { "type": "integer", "minimum": 0 }, - "nominal_rate_hz": { "type": "number", "exclusiveMinimum": 0 } + "rows": { + "type": "integer", + "minimum": 0 + }, + "nominal_rate_hz": { + "type": "number", + "exclusiveMinimum": 0 + } } }, "streams": { "description": "CSV files with a header row; the first column is always t. Columns are fixed per stream (README, Streams).", "type": "object", "properties": { - "trajectory": { "$ref": "#/$defs/stream" }, - "accelerometer": { "$ref": "#/$defs/stream" }, - "gyroscope": { "$ref": "#/$defs/stream" }, - "magnetometer": { "$ref": "#/$defs/stream" }, - "device_motion": { "$ref": "#/$defs/stream" }, - "barometer": { "$ref": "#/$defs/stream" }, - "location": { "$ref": "#/$defs/stream" }, - "heading": { "$ref": "#/$defs/stream" } + "trajectory": { + "$ref": "#/$defs/stream" + }, + "accelerometer": { + "$ref": "#/$defs/stream" + }, + "gyroscope": { + "$ref": "#/$defs/stream" + }, + "magnetometer": { + "$ref": "#/$defs/stream" + }, + "device_motion": { + "$ref": "#/$defs/stream" + }, + "barometer": { + "$ref": "#/$defs/stream" + }, + "location": { + "$ref": "#/$defs/stream" + }, + "heading": { + "$ref": "#/$defs/stream" + } }, "additionalProperties": false }, @@ -239,79 +494,269 @@ "$ref": "#/$defs/file" }, "planes": { + "description": "Deprecated in 1.1: use top-level planes. Still read in 1.x; a packet may not use both.", "type": "array", "items": { - "type": "object", - "required": ["id", "alignment", "pose", "extent_m"], - "properties": { - "id": { "type": "string" }, - "alignment": { "enum": ["horizontal", "vertical"] }, - "classification": { "enum": ["none", "wall", "floor", "ceiling", "table", "seat", "window", "door"] }, - "pose": { "description": "Plane to meter frame; the plane is its local x-z plane, +y its normal.", "$ref": "#/$defs/pose" }, - "extent_m": { - "type": "array", - "minItems": 2, - "maxItems": 2, - "items": { "type": "number", "minimum": 0 } - } - } + "$ref": "#/$defs/plane" } } } }, "mark": { "type": "object", - "required": ["id", "kind", "points"], + "required": [ + "id", + "kind", + "points" + ], "properties": { - "id": { "type": "string" }, + "id": { + "type": "string" + }, "kind": { - "enum": ["meter", "wall_end", "gas_meter", "ac", "door", "window", "garage_door", "drive_edge", "fence"] + "enum": [ + "meter", + "wall_end", + "gas_meter", + "ac", + "door", + "window", + "garage_door", + "drive_edge", + "fence" + ] }, "points": { "description": "Meter frame, meters. One point for the meter, a wall end, a gas meter or an AC unit; two for an opening (bottom-left, top-right corners), a drive edge or a fence.", "type": "array", "minItems": 1, - "items": { "$ref": "#/$defs/vec3" } + "items": { + "$ref": "#/$defs/vec3" + } + }, + "t": { + "$ref": "#/$defs/time" + }, + "photo_ids": { + "type": "array", + "items": { + "type": "string" + } + }, + "side": { + "enum": [ + "left", + "right" + ] + }, + "end_kind": { + "description": "For a wall end: limit (something blocks it) or unexplored (it turns a corner, or no answer).", + "enum": [ + "limit", + "unexplored" + ] }, - "t": { "$ref": "#/$defs/time" }, - "photo_ids": { "type": "array", "items": { "type": "string" } }, - "side": { "enum": ["left", "right"] }, - "end_kind": { "description": "For a wall end: limit (something blocks it) or unexplored (it turns a corner, or no answer).", "enum": ["limit", "unexplored"] }, "attrs": { "type": "object", - "properties": { "operable": { "type": "boolean" } } + "properties": { + "operable": { + "type": "boolean" + } + } } } }, "guidance_event": { "type": "object", - "required": ["id", "kind", "origin", "t_shown", "outcome"], + "required": [ + "id", + "kind", + "origin", + "t_shown", + "outcome" + ], "properties": { - "id": { "type": "string" }, - "kind": { "enum": ["walk", "tilt_to_ground", "step_back", "mark_end", "closeup", "gap_band", "gap_past_end"] }, - "origin": { "enum": ["phone", "server"] }, - "message": { "type": "string" }, - "band": { "enum": ["wall", "ground", "overhead", "facing", null] }, + "id": { + "type": "string" + }, + "kind": { + "enum": [ + "walk", + "tilt_to_ground", + "step_back", + "mark_end", + "closeup", + "gap_band", + "gap_past_end" + ] + }, + "origin": { + "enum": [ + "phone", + "server" + ] + }, + "message": { + "type": "string" + }, + "band": { + "enum": [ + "wall", + "ground", + "overhead", + "facing", + null + ] + }, "span_m": { "description": "Stretch along the wall, meter frame x, meters.", - "type": ["array", "null"], + "type": [ + "array", + "null" + ], "minItems": 2, "maxItems": 2, - "items": { "type": "number" } + "items": { + "type": "number" + } + }, + "t_shown": { + "$ref": "#/$defs/time" + }, + "t_resolved": { + "type": [ + "number", + "null" + ] }, - "t_shown": { "$ref": "#/$defs/time" }, - "t_resolved": { "type": ["number", "null"] }, - "outcome": { "enum": ["met", "skipped", "cannot_reach", "superseded", "unresolved"] } + "outcome": { + "enum": [ + "met", + "skipped", + "cannot_reach", + "superseded", + "unresolved" + ] + } } }, "provenance": { "description": "Where a converted sample's data came from, and what it does not contain.", "type": "object", "properties": { - "dataset": { "type": "string" }, - "license": { "type": "string" }, - "source": { "type": "string" }, - "notes": { "type": "array", "items": { "type": "string" } } + "dataset": { + "type": "string" + }, + "license": { + "type": "string" + }, + "source": { + "type": "string" + }, + "notes": { + "type": "array", + "items": { + "type": "string" + } + } + } + }, + "plane": { + "type": "object", + "required": [ + "id", + "alignment", + "pose", + "extent_m" + ], + "properties": { + "id": { + "type": "string" + }, + "alignment": { + "enum": [ + "horizontal", + "vertical" + ] + }, + "classification": { + "enum": [ + "none", + "wall", + "floor", + "ceiling", + "table", + "seat", + "window", + "door" + ] + }, + "pose": { + "description": "Plane to meter frame, including the extent's centre and planeExtent.rotationOnYAxis, so extent_m is a rectangle centred on this pose. The plane is its local x-z plane, +y its normal.", + "$ref": "#/$defs/pose" + }, + "extent_m": { + "type": "array", + "minItems": 2, + "maxItems": 2, + "items": { + "type": "number", + "minimum": 0 + } + }, + "boundary_m": { + "description": "1.1. ARKit's geometry.boundaryVertices as [x, z] in this pose's frame (moved by the centre and rotation, like extent_m).", + "type": "array", + "minItems": 3, + "items": { + "type": "array", + "minItems": 2, + "maxItems": 2, + "items": { + "type": "number" + } + } + } + } + }, + "depth_frame": { + "description": "1.1. Depth recorded between photos, with no image. Intrinsics are for the depth map's own pixel grid.", + "allOf": [ + { + "$ref": "#/$defs/depth" + } + ], + "type": "object", + "required": [ + "id", + "t", + "pose", + "intrinsics" + ], + "properties": { + "id": { + "type": "string", + "pattern": "^[A-Za-z0-9_-]+$" + }, + "t": { + "$ref": "#/$defs/time" + }, + "pose": { + "description": "Camera to meter frame, ARKit camera axes.", + "$ref": "#/$defs/pose" + }, + "intrinsics": { + "description": "[fx, fy, cx, cy] in pixels of the depth map.", + "type": "array", + "minItems": 4, + "maxItems": 4, + "items": { + "type": "number" + } + }, + "tracking": { + "$ref": "#/$defs/tracking" + } } } } diff --git a/packet/packet/samples/synthetic.py b/packet/packet/samples/synthetic.py index 111706c90..71c9341b3 100644 --- a/packet/packet/samples/synthetic.py +++ b/packet/packet/samples/synthetic.py @@ -31,6 +31,7 @@ W, H, FX = 96, 72, 80.0 DEPTH_W, DEPTH_H = 24, 18 PHOTO_TICKS = (15, 60, 105) # trajectory samples the photos are taken at +DEPTH_FRAME_TICKS = (30, 75) # depth recorded between photos (1.1) WALL_DISTANCE = 2.5 @@ -125,6 +126,7 @@ def build(out: Path) -> Path: "lidar": True, "scene_depth_enabled": True, "mesh_enabled": True, + "mesh_classification_enabled": True, }, "capture": { "started_at": "2026-09-26T12:00:00Z", @@ -140,25 +142,33 @@ def build(out: Path) -> Path: }, "photos": photos, "streams": streams, - "lidar": { - "mesh": w.add_bytes("lidar/mesh.ply", mesh), - "planes": [ - { - "id": "wall", - "alignment": "vertical", - "classification": "wall", - "pose": column_major(wall_plane), - "extent_m": [6.0, 2.4], - }, - { - "id": "ground", - "alignment": "horizontal", - "classification": "floor", - "pose": column_major(ground_plane), - "extent_m": [6.0, 3.0], - }, - ], - }, + "depth_frames": depth_frames(w, times, poses), + "lidar": {"mesh": w.add_bytes("lidar/mesh.ply", mesh)}, + "planes": [ + { + "id": "wall", + "alignment": "vertical", + "classification": "wall", + "pose": column_major(wall_plane), + "extent_m": [6.0, 2.4], + # An L of wall: a window cut out of the top right (plane x, z = -height). + "boundary_m": [ + [-3.0, -1.2], + [3.0, -1.2], + [3.0, 0.2], + [0.0, 0.2], + [0.0, 1.2], + [-3.0, 1.2], + ], + }, + { + "id": "ground", + "alignment": "horizontal", + "classification": "floor", + "pose": column_major(ground_plane), + "extent_m": [6.0, 3.0], + }, + ], "marks": [ { "id": "m1", @@ -241,6 +251,36 @@ def build(out: Path) -> Path: return w.finish(manifest) +def depth_frames(w: PacketWriter, times: np.ndarray, poses: list[np.ndarray]) -> list[dict]: + """Depth between photos: one frame from ARKit (with confidence), one estimated from the image + (with sigma), at the depth map's own resolution.""" + scale = DEPTH_W / W + frames = [] + for n, k in enumerate(DEPTH_FRAME_TICKS, start=1): + fid = f"d{n:05d}" + depth = np.full((DEPTH_H, DEPTH_W), WALL_DISTANCE, dtype=np.float32) + if n == 1: + data = w.add_depth( + fid, depth, confidence=np.full(depth.shape, 2), folder="depth_frames" + ) + data["source"] = "arkit_scene_depth" + else: + sigma = np.full(depth.shape, 0.15, dtype=np.float32) + data = w.add_depth(fid, depth, sigma=sigma, folder="depth_frames") + data["source"] = "estimated" + frames.append( + { + "id": fid, + "t": float(times[k]), + "pose": column_major(poses[k]), + "intrinsics": [FX * scale, FX * scale, DEPTH_W / 2, DEPTH_H / 2], + "tracking": {"state": "normal", "reason": None}, + **data, + } + ) + return frames + + def mesh_ply() -> bytes: """A wall quad (class 1, wall) and a ground quad (class 2, floor), two triangles each.""" v = np.array( diff --git a/packet/packet/validate.py b/packet/packet/validate.py index 88660dea0..f27363cf9 100644 --- a/packet/packet/validate.py +++ b/packet/packet/validate.py @@ -165,16 +165,19 @@ def file_problems(files: PacketFiles, ref: dict, where: str) -> list[str]: return [] +def depth_refs(d: dict, where: str) -> list[tuple[str, dict]]: + return [(f"{where}.{key}", d[key]) for key in ("map", "confidence", "sigma") if key in d] + + def file_refs(manifest: dict) -> list[tuple[str, dict]]: """Every file the manifest names, with where it is named.""" refs = [] for i, p in enumerate(manifest.get("photos", [])): refs.append((f"photos[{i}].image", p["image"])) - depth = p.get("depth") - if depth: - refs.append((f"photos[{i}].depth.map", depth["map"])) - if "confidence" in depth: - refs.append((f"photos[{i}].depth.confidence", depth["confidence"])) + if "depth" in p: + refs += depth_refs(p["depth"], f"photos[{i}].depth") + for i, f in enumerate(manifest.get("depth_frames", [])): + refs += depth_refs(f, f"depth_frames[{i}]") for name, s in manifest.get("streams", {}).items(): refs.append((f"streams.{name}", s)) if manifest.get("lidar", {}).get("mesh"): @@ -207,7 +210,13 @@ def image_problems(files: PacketFiles, photo: dict, where: str) -> list[str]: return out -def depth_problems(files: PacketFiles, photo: dict, intact: set[str], where: str) -> list[str]: +ARKIT_DEPTH = {"arkit_scene_depth", "arkit_smoothed_scene_depth"} +# A boundary vertex may sit on the extent's edge; allow float rounding and 1 cm. +BOUNDARY_TOL_M = 0.01 + + +def photo_depth_problems(photo: dict, where: str) -> list[str]: + """A photo's depth covers the photo's field of view: same aspect, no larger.""" d = photo["depth"] w, h = d["width"], d["height"] out = [] @@ -219,20 +228,37 @@ def depth_problems(files: PacketFiles, photo: dict, intact: set[str], where: str f"{where}: depth {w}x{h} does not have the photo's aspect " f"({photo['width']}x{photo['height']}); it cannot be aligned by scaling" ) - if d["map"]["bytes"] != w * h * 4: - return [ - *out, - f"{where}: depth map is {d['map']['bytes']} bytes, {w}x{h} float32 is {w * h * 4}", - ] - if d["map"]["path"] not in intact: - return out # reported by file_problems - depth = np.frombuffer(files.read(d["map"]["path"]), dtype=" 0).mean() < 0.01: - out.append(f"{where}: depth is empty (under 1% of pixels measured)") + return out + + +def depth_data_problems( + files: PacketFiles, d: dict, intact: set[str], minor: int, where: str +) -> list[str]: + """The depth map, its confidence and its sigma: sizes, values, and which are required.""" + w, h = d["width"], d["height"] + out = [] + source = d.get("source") + if source in ARKIT_DEPTH and minor >= 1 and "confidence" not in d: + out.append(f"{where}: {source} depth needs confidence (required from 1.1)") + if source == "estimated" and "sigma" not in d: + out.append(f"{where}: estimated depth needs sigma") + for key, dtype, name in (("map", " 0).mean() < 0.01: + out.append(f"{where}: depth is empty (under 1% of pixels measured)") if "confidence" in d: if d["confidence"]["bytes"] != w * h: out.append( @@ -245,6 +271,23 @@ def depth_problems(files: PacketFiles, photo: dict, intact: set[str], where: str return out +def plane_problems(plane: dict, where: str) -> list[str]: + out = rigid_problems(plane["pose"], where) + if "boundary_m" in plane: + ex, ez = plane["extent_m"] + b = np.asarray(plane["boundary_m"], dtype=float) + if not np.isfinite(b).all(): + out.append(f"{where}: boundary_m must be finite") + elif (np.abs(b[:, 0]) > ex / 2 + BOUNDARY_TOL_M).any() or ( + np.abs(b[:, 1]) > ez / 2 + BOUNDARY_TOL_M + ).any(): + out.append( + f"{where}: boundary_m leaves the {ex} x {ez} m extent centred on the pose; " + "is it in ARKit's anchor frame instead of the pose's?" + ) + return out + + # --- Streams ------------------------------------------------------------------------------------ @@ -308,8 +351,9 @@ def horizontal_distance(positions: np.ndarray) -> float: ] -def mesh_problems(data: bytes) -> list[str]: - """A binary PLY with exactly the layout README.md fixes, indices in range, classes 0 to 7.""" +def mesh_problems(data: bytes, classified: bool = True) -> list[str]: + """A binary PLY with exactly the layout README.md fixes, indices in range, classes 0 to 7, + and all 0 when classification was off.""" end = data.find(b"end_header\n") if not data.startswith(b"ply\n") or end < 0: return ["lidar.mesh: not a PLY file"] @@ -346,6 +390,8 @@ def mesh_problems(data: bytes) -> list[str]: out.append("lidar.mesh: a face index is out of range") if nf and faces["c"].max() >= MESH_CLASSIFICATIONS: out.append("lidar.mesh: classification must be an ARMeshClassification raw value, 0 to 7") + elif nf and not classified and faces["c"].any(): + out.append("lidar.mesh: classification was off, so every face's class must be 0") return out @@ -371,6 +417,7 @@ def validate(source: Path) -> Report: return report # the checks below rely on the shape the schema guarantees problems = report.problems + minor = int(manifest["packet_version"].split(".")[1]) session = manifest["session"] start, end = session["capture"]["started_at_uptime"], session["capture"]["ended_at_uptime"] if not end > start: @@ -413,7 +460,20 @@ def in_window(t: float, where: str) -> None: if p["image"]["path"] in intact: problems += image_problems(files, p, where) if "depth" in p: - problems += depth_problems(files, p, intact, where) + problems += photo_depth_problems(p, where) + problems += depth_data_problems(files, p["depth"], intact, minor, where) + + frames = manifest.get("depth_frames", []) + frame_ids = [f["id"] for f in frames] + if len(set(frame_ids)) != len(frame_ids): + problems.append("depth_frames: ids must be unique") + problems += increasing_problems([f["t"] for f in frames], "depth_frames (in manifest order)") + for i, f in enumerate(frames): + where = f"depth_frames[{i}] {f['id']}" + in_window(f["t"], where) + problems += rigid_problems(f["pose"], where) + problems += intrinsics_problems(f["intrinsics"], f["width"], f["height"], where) + problems += depth_data_problems(files, f, intact, minor, where) streams = manifest.get("streams", {}) trajectory = None @@ -445,23 +505,35 @@ def in_window(t: float, where: str) -> None: f"session.capture.distance_walked_m is {walked}; " f"the trajectory walks {measured:.3f}" ) - for i, p in enumerate(photos): + posed = [(f"photos[{i}] {p['id']}", p) for i, p in enumerate(photos)] + posed += [(f"depth_frames[{i}] {f['id']}", f) for i, f in enumerate(frames)] + for where, p in posed: j = int(np.abs(trajectory[:, 0] - p["t"]).argmin()) dt = abs(trajectory[j, 0] - p["t"]) dist = float(np.linalg.norm(trajectory[j, 1:4] - matrix(p["pose"])[:3, 3])) if dt > PHOTO_TRAJECTORY_DT: - problems.append(f"photos[{i}] {p['id']}: no trajectory sample within {dt:.3f} s") + problems.append(f"{where}: no trajectory sample within {dt:.3f} s") elif dist > PHOTO_TRAJECTORY_DIST: problems.append( - f"photos[{i}] {p['id']}: {dist:.3f} m from the trajectory at its time; " + f"{where}: {dist:.3f} m from the trajectory at its time; " "poses and trajectory disagree (frame or clock)" ) lidar = manifest.get("lidar", {}) - if "mesh" in lidar and lidar["mesh"]["path"] in intact: - problems += mesh_problems(files.read(lidar["mesh"]["path"])) - for i, plane in enumerate(lidar.get("planes", [])): - problems += rigid_problems(plane["pose"], f"lidar.planes[{i}]") + classified = session["device"].get("mesh_classification_enabled") + if "mesh" in lidar: + if minor >= 1 and classified is None: + problems.append( + "session.device.mesh_classification_enabled is required with a mesh (from 1.1)" + ) + if lidar["mesh"]["path"] in intact: + problems += mesh_problems(files.read(lidar["mesh"]["path"]), classified is not False) + if "planes" in manifest and "planes" in lidar: + problems.append("planes appear at the top level and in lidar.planes; use one (top level)") + planes = [("planes", pl) for pl in manifest.get("planes", [])] + planes += [("lidar.planes", pl) for pl in lidar.get("planes", [])] + for i, (key, plane) in enumerate(planes): + problems += plane_problems(plane, f"{key}[{i}]") photo_ids = set(ids) for i, m in enumerate(manifest.get("marks", [])): @@ -506,9 +578,10 @@ def in_window(t: float, where: str) -> None: "photos": len(photos), "photo_size": f"{photos[0]['width']}x{photos[0]['height']}", "photos_with_depth": sum("depth" in p for p in photos), + "depth_frames": len(frames), "streams": {n: s["rows"] for n, s in streams.items()}, "mesh": "mesh" in lidar, - "planes": len(lidar.get("planes", [])), + "planes": len(planes), "marks": len(manifest.get("marks", [])), "guidance": len(manifest.get("guidance", [])), "seconds": round(end - start, 2), diff --git a/packet/packet/write.py b/packet/packet/write.py index c30291bfd..6722a4443 100644 --- a/packet/packet/write.py +++ b/packet/packet/write.py @@ -19,7 +19,7 @@ from packet.validate import STREAM_COLUMNS -PACKET_VERSION = "1.0" +PACKET_VERSION = "1.1" SHARPNESS_METHOD = "laplacian_variance_luma_640" @@ -58,16 +58,27 @@ def add_stream(self, name: str, rows: list[list], rate_hz: float | None = None) ref["nominal_rate_hz"] = rate_hz return ref - def add_depth(self, photo_id: str, depth: np.ndarray, confidence: np.ndarray | None) -> dict: + def add_depth( + self, + name: str, + depth: np.ndarray, + confidence: np.ndarray | None = None, + sigma: np.ndarray | None = None, + folder: str = "depth", + ) -> dict: h, w = depth.shape ref = { - "map": self.add_bytes(f"depth/{photo_id}.f32", depth.astype(" Path: diff --git a/packet/tests/test_checks.py b/packet/tests/test_checks.py index d92c63851..b1be902c5 100644 --- a/packet/tests/test_checks.py +++ b/packet/tests/test_checks.py @@ -217,3 +217,11 @@ def test_a_zip_with_two_manifests_is_refused(tmp_path): def test_a_folder_without_a_manifest_is_refused(tmp_path): with pytest.raises(PacketError, match="no manifest"): PacketFiles(tmp_path) + + +def test_an_unclassified_mesh_has_only_class_zero(): + data = bytearray(mesh_ply()) + for i in range(1, 5): # the class byte ends each 14-byte face + data[-14 * i + 13] = 0 + assert mesh_problems(bytes(data), classified=False) == [] + assert mesh_problems(mesh_ply(), classified=False) != [] diff --git a/packet/tests/test_packet.py b/packet/tests/test_packet.py index 79aa12410..d982ace32 100644 --- a/packet/tests/test_packet.py +++ b/packet/tests/test_packet.py @@ -196,7 +196,9 @@ def test_depth_bytes_must_match_its_size(packet): d = m["photos"][0]["depth"] d["width"], d["height"] = 12, 9 # same aspect, a quarter of the pixels save(packet, m) - assert "float32 is 432" in only_problem(packet) + problems = validate(packet).problems + assert len(problems) == 2 + assert "float32 is 432" in problems[0] and "uint8 is 108" in problems[1] def test_confidence_above_high(packet): @@ -240,3 +242,104 @@ def test_a_path_that_leaves_the_packet_breaks_the_schema(packet): m["scene"]["path"] = "../scene.json" save(packet, m) assert validate(packet).problems[0].startswith("schema: scene/path") + + +# --- 1.1 additions ------------------------------------------------------------------------------ + + +def as_1_0(m: dict) -> dict: + """The fixture as a 1.0 writer would have written it: no 1.1 fields.""" + m["packet_version"] = "1.0" + del m["depth_frames"] + del m["session"]["device"]["mesh_classification_enabled"] + m["lidar"]["planes"] = m.pop("planes") + for plane in m["lidar"]["planes"]: + plane.pop("boundary_m", None) + for p in m["photos"]: + del p["depth"]["confidence"] # optional in 1.0 even for ARKit depth + return m + + +def test_a_1_0_packet_still_validates(packet): + save(packet, as_1_0(manifest(packet))) + report = validate(packet) + assert report.ok, report.problems + assert len(report.warnings) == 1 # the files only 1.1 names are now unlisted + + +def test_arkit_depth_needs_confidence_from_1_1(packet): + m = manifest(packet) + del m["photos"][0]["depth"]["confidence"] + (packet / "depth" / "p00001.conf.u8").unlink() + save(packet, m) + assert "needs confidence (required from 1.1)" in only_problem(packet) + + +def test_estimated_depth_needs_sigma(packet): + m = manifest(packet) + del m["depth_frames"][1]["sigma"] + save(packet, m) + assert only_problem(packet).startswith("schema: depth_frames/1") + + +def test_sigma_must_be_finite_and_sized(packet): + m = manifest(packet) + f = m["depth_frames"][1] + sigma = np.full((f["height"], f["width"]), 0.15, " Date: Sat, 26 Sep 2026 12:08:39 -0500 Subject: [PATCH 3/9] packet: mark the fixture's raw sensor files binary Co-Authored-By: Claude Opus 5.5 (1M context) --- packet/.gitattributes | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 packet/.gitattributes diff --git a/packet/.gitattributes b/packet/.gitattributes new file mode 100644 index 000000000..747c65e22 --- /dev/null +++ b/packet/.gitattributes @@ -0,0 +1,4 @@ +# Packet sensor files are raw binary; never convert line endings or diff them as text. +*.f32 binary +*.u8 binary +*.ply binary From 6a12700844814657783796e4fca6c218257fb2d9 Mon Sep 17 00:00:00 2001 From: SamGu-NRX Date: Sat, 26 Sep 2026 20:00:54 -0500 Subject: [PATCH 4/9] packet: point to the server's capture packet 0.4, keep our 0.5 proposals The team adopted the server's capture packet spec 0.4 and its intake API as the single upload contract. This folder no longer defines a format: the 1.1 schema, validator, sample builders, fixture and tests are removed (still readable at d5439cf). What remains is the app side's proposals for 0.5 and three arrival checks offered for the server's validator. Co-Authored-By: Claude Opus 5.5 (1M context) --- packet/.gitattributes | 4 - packet/.python-version | 1 - packet/README.md | 383 ++------- .../fixtures/synthetic/depth/p00001.conf.u8 | Bin 432 -> 0 bytes packet/fixtures/synthetic/depth/p00001.f32 | Bin 1728 -> 0 bytes .../fixtures/synthetic/depth/p00002.conf.u8 | Bin 432 -> 0 bytes packet/fixtures/synthetic/depth/p00002.f32 | Bin 1728 -> 0 bytes .../fixtures/synthetic/depth/p00003.conf.u8 | Bin 432 -> 0 bytes packet/fixtures/synthetic/depth/p00003.f32 | Bin 1728 -> 0 bytes .../synthetic/depth_frames/d00001.conf.u8 | 1 - .../synthetic/depth_frames/d00001.f32 | Bin 1728 -> 0 bytes .../synthetic/depth_frames/d00002.f32 | Bin 1728 -> 0 bytes .../synthetic/depth_frames/d00002.sigma.f32 | 1 - packet/fixtures/synthetic/lidar/mesh.ply | Bin 369 -> 0 bytes packet/fixtures/synthetic/manifest.json | 635 --------------- packet/fixtures/synthetic/photos/p00001.jpg | Bin 2425 -> 0 bytes packet/fixtures/synthetic/photos/p00002.jpg | Bin 2539 -> 0 bytes packet/fixtures/synthetic/photos/p00003.jpg | Bin 2269 -> 0 bytes packet/fixtures/synthetic/scene.json | 26 - .../synthetic/streams/accelerometer.csv | 201 ----- .../fixtures/synthetic/streams/barometer.csv | 3 - .../synthetic/streams/device_motion.csv | 201 ----- .../fixtures/synthetic/streams/gyroscope.csv | 201 ----- packet/fixtures/synthetic/streams/heading.csv | 2 - .../fixtures/synthetic/streams/location.csv | 2 - .../synthetic/streams/magnetometer.csv | 201 ----- .../fixtures/synthetic/streams/trajectory.csv | 122 --- packet/manifest.schema.json | 763 ------------------ packet/packet/__init__.py | 1 - packet/packet/__main__.py | 46 -- packet/packet/files.py | 103 --- packet/packet/samples/__init__.py | 1 - packet/packet/samples/advio.py | 209 ----- packet/packet/samples/app_export.py | 191 ----- packet/packet/samples/eth3d.py | 235 ------ packet/packet/samples/synthetic.py | 327 -------- packet/packet/validate.py | 590 -------------- packet/packet/write.py | 167 ---- packet/pyproject.toml | 32 - packet/tests/test_checks.py | 227 ------ packet/tests/test_packet.py | 345 -------- packet/uv.lock | 417 ---------- packet/validator-checks.md | 59 ++ 43 files changed, 119 insertions(+), 5578 deletions(-) delete mode 100644 packet/.gitattributes delete mode 100644 packet/.python-version delete mode 100644 packet/fixtures/synthetic/depth/p00001.conf.u8 delete mode 100644 packet/fixtures/synthetic/depth/p00001.f32 delete mode 100644 packet/fixtures/synthetic/depth/p00002.conf.u8 delete mode 100644 packet/fixtures/synthetic/depth/p00002.f32 delete mode 100644 packet/fixtures/synthetic/depth/p00003.conf.u8 delete mode 100644 packet/fixtures/synthetic/depth/p00003.f32 delete mode 100644 packet/fixtures/synthetic/depth_frames/d00001.conf.u8 delete mode 100644 packet/fixtures/synthetic/depth_frames/d00001.f32 delete mode 100644 packet/fixtures/synthetic/depth_frames/d00002.f32 delete mode 100644 packet/fixtures/synthetic/depth_frames/d00002.sigma.f32 delete mode 100644 packet/fixtures/synthetic/lidar/mesh.ply delete mode 100644 packet/fixtures/synthetic/manifest.json delete mode 100644 packet/fixtures/synthetic/photos/p00001.jpg delete mode 100644 packet/fixtures/synthetic/photos/p00002.jpg delete mode 100644 packet/fixtures/synthetic/photos/p00003.jpg delete mode 100644 packet/fixtures/synthetic/scene.json delete mode 100644 packet/fixtures/synthetic/streams/accelerometer.csv delete mode 100644 packet/fixtures/synthetic/streams/barometer.csv delete mode 100644 packet/fixtures/synthetic/streams/device_motion.csv delete mode 100644 packet/fixtures/synthetic/streams/gyroscope.csv delete mode 100644 packet/fixtures/synthetic/streams/heading.csv delete mode 100644 packet/fixtures/synthetic/streams/location.csv delete mode 100644 packet/fixtures/synthetic/streams/magnetometer.csv delete mode 100644 packet/fixtures/synthetic/streams/trajectory.csv delete mode 100644 packet/manifest.schema.json delete mode 100644 packet/packet/__init__.py delete mode 100644 packet/packet/__main__.py delete mode 100644 packet/packet/files.py delete mode 100644 packet/packet/samples/__init__.py delete mode 100644 packet/packet/samples/advio.py delete mode 100644 packet/packet/samples/app_export.py delete mode 100644 packet/packet/samples/eth3d.py delete mode 100644 packet/packet/samples/synthetic.py delete mode 100644 packet/packet/validate.py delete mode 100644 packet/packet/write.py delete mode 100644 packet/pyproject.toml delete mode 100644 packet/tests/test_checks.py delete mode 100644 packet/tests/test_packet.py delete mode 100644 packet/uv.lock create mode 100644 packet/validator-checks.md diff --git a/packet/.gitattributes b/packet/.gitattributes deleted file mode 100644 index 747c65e22..000000000 --- a/packet/.gitattributes +++ /dev/null @@ -1,4 +0,0 @@ -# Packet sensor files are raw binary; never convert line endings or diff them as text. -*.f32 binary -*.u8 binary -*.ply binary diff --git a/packet/.python-version b/packet/.python-version deleted file mode 100644 index e4fba2183..000000000 --- a/packet/.python-version +++ /dev/null @@ -1 +0,0 @@ -3.12 diff --git a/packet/README.md b/packet/README.md index 0f74f683a..7cab25f4c 100644 --- a/packet/README.md +++ b/packet/README.md @@ -1,325 +1,62 @@ # The capture packet -The contract between the iPhone app and the server. The app guides the homeowner, takes the -photos and records what the phone knows. The server builds a 3D model from them and evaluates -the placement rules. One packet is one scan session: a folder, or a zip of one, holding -`manifest.json`, the photos and the sensor files. This document is the specification. -[`manifest.schema.json`](manifest.schema.json) checks the manifest's shape, and -`python -m packet validate` checks everything below. - -Version 1.1 ([what changed from 1.0](#versioning)). The **App** column in each table says what `t3/ios-mvf` at `a39d0a5` records: -**now** (exported today), **partial** (computed in the app but not exported, or exported in -another form) or **planned**. [What the app records today](#what-the-app-records-today) lists -the gaps. - -## Layout - -``` -manifest.json what everything is (this document) -photos/p00001.jpg full-resolution photos, one per kept frame -depth/p00001.f32 depth aligned to that photo (LiDAR phones) -depth/p00001.conf.u8 its confidence -depth_frames/d00001.* depth recorded between photos, without an image (1.1) -streams/*.csv trajectory, IMU, barometer, location, heading -lidar/mesh.ply ARKit mesh with per-face classification (LiDAR phones) -scene.json the app's placement request (C1), unchanged -``` - -Only `manifest.json` has a fixed name. Every other file is named by a manifest entry, -`{"path", "bytes", "sha256"}`. The path is relative to the manifest, uses `/`, and never starts -with `/` or contains `..`. A reader must refuse a zip whose entries leave the packet. A zip may -hold the packet at its root or inside one top-level folder. - -## Units, frames and clocks - -These hold for every field unless its row says otherwise. - -- **Units.** Meters and seconds. Angles are degrees where a person reads them (headings, - latitude and longitude) and radians per second for the gyroscope. The accelerometer is in - m/s² (Core Motion reports g: multiply by 9.80665), the magnetometer in microtesla and - pressure in kPa. -- **Clock.** Seconds of device uptime: the clock of `ARFrame.timestamp`, `CMLogItem.timestamp` - and `ProcessInfo.systemUptime`. `session.capture.started_at` (ISO 8601, UTC) and - `started_at_uptime` are the same instant, so a wall-clock time such as `CLLocation.timestamp` - converts as `uptime = started_at_uptime + (date - started_at)`. -- **World frame.** ARKit's world frame with `worldAlignment = .gravity`: right-handed, +y up - (against gravity), origin and heading wherever the session started. Nothing in the packet is - in this frame except `session.meter_anchor.pose_in_world`, and heading is never used. -- **Meter frame.** Every pose and point in the packet is in the meter frame. Its origin is the - meter anchor, the point the homeowner tapped on the meter. +y is up (the world's +y), and +z - is the wall's outward normal, horizontal, toward where the homeowner stands. +x = y × z runs - along the wall to the right as seen facing it: the same direction as `scene.json`'s `s`. - `session.meter_anchor.pose_in_world` maps meter-frame points into the world frame. Results - are placed relative to the meter because ARKit's world origin is arbitrary per session, and - the meter is the one point every later visit can find again. -- **Camera frame.** ARKit's: +x right and +y up in the stored image, and the camera looks along - −z. For OpenCV's camera axes (y down, looking along +z), multiply by diag(1, −1, −1). -- **Poses.** Rigid 4 × 4 transforms, written as 16 numbers column by column (`simd_float4x4` - layout, as in Measure Lab v2 and `scene.json`). A photo's pose maps camera coordinates into - the meter frame. -- **Images and intrinsics.** The camera image is landscape and the intrinsics match it. A photo - is stored exactly as the sensor delivers it: landscape (width > height), never rotated to the - phone's orientation, EXIF orientation absent or 1. `intrinsics` are `[fx, fy, cx, cy]` in - pixels of that stored image, where (0, 0) is the top-left corner of the top-left pixel (pixel - centres at half-integers). A resized image scales all four by the same factor. Rotating an - image would mean rotating the intrinsics with it, so neither is rotated. -- **Depth alignment.** A depth map covers the same field of view as its photo, at lower - resolution and the same aspect ratio (within 1%). Depth pixel (row r, column c) covers photo - pixels x ∈ [c·W/w, (c+1)·W/w), y ∈ [r·H/h, (r+1)·H/h). Values are float32 little-endian - meters along the camera's −z axis (not the distance along the ray), row by row from the top, - and 0 means no measurement. Confidence is one byte per depth pixel, ARKit's - `ARConfidenceLevel`: 0 low, 1 medium, 2 high. Sigma (1.1) is float32 meters per depth pixel, - one standard deviation, in the same layout. A depth frame has no photo: its `intrinsics` - describe the depth map's own pixel grid. - -## Session - -| Field | Meaning | App | -| --- | --- | --- | -| `packet_version` | `"1.0"`; see [Versioning](#versioning) | planned | -| `session.id` | Unique per scan | planned | -| `session.producer` | `kind` (`app` or `converter`), `name`, `version`, optional `commit` | planned | -| `session.device.model` | Hardware identifier, e.g. `iPhone16,1` (`utsname.machine`), never the phone's name | planned | -| `session.device.ios_version` | `UIDevice.systemVersion` | planned | -| `session.device.lidar` | `ARWorldTrackingConfiguration.supportsFrameSemantics(.sceneDepth)` | planned | -| `session.device.scene_depth_enabled`, `mesh_enabled` | What the session actually ran with | planned | -| `session.device.mesh_classification_enabled` | 1.1. `sceneReconstruction` included `.meshWithClassification`. Required with a mesh; when false every face's class is 0, which otherwise could not be told from ARKit's own "none" | planned (S6) | -| `session.capture.started_at`, `started_at_uptime`, `ended_at_uptime` | Wall clock and uptime at the first frame, uptime at the last | planned | -| `session.capture.distance_walked_m` | Horizontal length of the trajectory (meter frame x and z). Must match `streams.trajectory` within 1% | planned | -| `session.world_alignment` | Always `"gravity"` | partial: the app runs with `.gravity` but does not write it | -| `session.meter_anchor.pose_in_world` | The meter frame in the world frame | partial: `scene.json` has the meter's position (feet, ground at y = 0), not its orientation | -| `session.meter_anchor.ground_y_m` | The ground's height at the meter on the meter frame's y (negative) | partial: implied by `scene.json`'s y = 0 | -| `session.consent.location` | The homeowner agreed to share location and heading. Must be true if either stream is present | planned | - -## Photos - -One entry per kept frame, in time order. Required: `id`, `image`, `width`, `height`, `t`, `pose`, -`intrinsics`. - -| Field | Meaning | App | -| --- | --- | --- | -| `image` | JPEG of the full frame: at least ARKit's `capturedImage` at its native size (1920 × 1440 on current iPhones). On iOS 16 and later, prefer `captureHighResolutionFrame`'s still (4032 × 3024) with its own intrinsics. Never a thumbnail | now: `capturedImage` at native size, quality 0.8 | -| `width`, `height` | Pixels of the stored JPEG | now | -| `t` | `ARFrame.timestamp` | partial: recorded, not exported | -| `pose` | Camera to meter frame, from `ARFrame.camera.transform` | partial: exported camera-to-world in feet | -| `intrinsics` | `ARFrame.camera.intrinsics` for the stored image | now | -| `tracking.state`, `tracking.reason` | `normal`, `limited` or `not_available`, with the limited reason | partial: recorded, not exported | -| `exposure.duration_s`, `exposure.iso`, `exposure.offset_ev` | `ARCamera.exposureDuration` and `exposureOffset` (iOS 15); ISO from the high-resolution still's metadata | planned | -| `lens.focal_length_mm`, `lens.f_number`, `lens.camera` | From the still's EXIF; `camera` is `wide`, `ultra_wide` or `telephoto` | planned | -| `sharpness` | `{"method": "laplacian_variance_luma_640", "value"}`, defined below | partial: the app scores sharpness another way and does not export it | -| `depth` | `map`, `confidence`, `sigma`, `width`, `height`, `source`. Sources: `arkit_scene_depth` and `arkit_smoothed_scene_depth` (LiDAR; from 1.1 they need `confidence`, which marks glass, dark surfaces and edges low); `estimated` (1.1: metric depth inferred from the image on a phone without LiDAR; needs `sigma`, since the uncertainty decides what counts as a surface); `rendered_from_laser_scan` (samples only) | planned (S6) | - -**Sharpness** is the variance of the 4-neighbour Laplacian of the image's luma, after scaling the -long side to 640 px with bilinear filtering (images already smaller are used as they are). One -fixed method makes scores comparable across phones and resolutions. `packet/write.py` has the -reference implementation. - -## Streams - -CSV files with a header row. The first column is always `t`, strictly increasing. Each has a -manifest entry `{"path", "bytes", "sha256", "rows", "nominal_rate_hz"}`, and its columns are -fixed: - -| Stream | Columns | Source | App | -| --- | --- | --- | --- | -| `trajectory` | `t, tracking, px, py, pz, qx, qy, qz, qw` | Every ARFrame (60 Hz): camera-to-meter position and unit quaternion, tracking state | planned: the app keeps pose-only frames at 30 Hz for its overlay and discards them | -| `accelerometer` | `t, x, y, z` (m/s²) | `CMAccelerometerData` | planned | -| `gyroscope` | `t, x, y, z` (rad/s) | `CMGyroData` | planned | -| `magnetometer` | `t, x, y, z` (µT, uncalibrated) | `CMMagnetometerData` | planned | -| `device_motion` | `t, qx, qy, qz, qw, gravity_x, gravity_y, gravity_z, user_accel_x, user_accel_y, user_accel_z, rotation_rate_x, rotation_rate_y, rotation_rate_z, heading_deg` | `CMDeviceMotion` (gravity and user acceleration in g, as Core Motion reports them) | planned | -| `barometer` | `t, pressure_kpa, relative_altitude_m` | `CMAltimeter` | planned | -| `location` | `t, latitude, longitude, altitude_m, horizontal_accuracy_m, vertical_accuracy_m` | `CLLocation`, with consent only | planned | -| `heading` | `t, magnetic_deg, true_deg, accuracy_deg` | `CLHeading`, with consent only | planned | - -IMU axes are Core Motion's device frame: +x to the right and +y toward the top of a phone held in -portrait, +z out of the screen. A photo from a tracked frame must sit on the trajectory: the -trajectory has a sample within 1/60 s of the photo's `t`, at the same position to 1 cm. - -## LiDAR extras - -On a LiDAR phone with scene reconstruction on, `lidar.mesh` is ARKit's mesh (`ARMeshAnchor`s -merged, in the meter frame), as a binary PLY with exactly this header: - -``` -ply -format binary_little_endian 1.0 -element vertex N -property float x -property float y -property float z -element face M -property list uchar int vertex_indices -property uchar classification -end_header -``` - -Every face is a triangle. `classification` is `ARMeshClassification`'s raw value: 0 none, 1 wall, -2 floor, 3 ceiling, 4 table, 5 seat, 6 window, 7 door. When classification was off, every class -is 0 and `session.device.mesh_classification_enabled` is false (1.1). `comment` lines may follow -`ply`. - -## Depth frames - -`depth_frames` (1.1) holds depth recorded between photos, with no image: `id`, `t`, `pose` -(camera to meter frame), `intrinsics` for the depth map's grid, `width`, `height`, `map`, and -`confidence` or `sigma` as for a photo's depth. Photos are kept about every 0.5 m, but an -on-device map fuses depth at about 10 Hz, so photo-rate depth alone cannot rebuild the -occupancy the phone used to decide what was seen. Each frame must sit on the trajectory like a -photo. Cost: at 256 × 192 a frame with confidence is about 246 KB, so 10 Hz is about 2.5 MB/s; -record a few hertz. App: planned (S6). - -## Planes - -`planes` (1.1, top level) lists `ARPlaneAnchor`s, which every ARKit phone detects, LiDAR or not: -`id`, `alignment` (`horizontal` or `vertical`), `classification`, `pose`, `extent_m` and optional -`boundary_m`. `pose` maps the plane into the meter frame and already includes the extent's -centre and `planeExtent.rotationOnYAxis`, so `extent_m` ([x, z]) is a rectangle centred on it. -ARKit's anchor transform alone is not that centre. The plane is its local x-z plane and its -normal is +y. `boundary_m` is `geometry.boundaryVertices` as [x, z] in the same frame as -`extent_m`, since ARKit's extent often claims a full rectangle over an L of wall or a wall broken -by a window. The validator checks that the boundary lies inside the extent, which catches a -boundary left in the anchor's frame. 1.0 put planes in `lidar.planes`; 1.x readers still accept -it there, but a packet may not use both. App: planned. The app reads plane anchors for its -coverage and taps but does not export them. - -## Marks, guidance and the scene - -`marks` are what the homeowner marked, in the meter frame. `kind` is `meter`, `wall_end`, -`gas_meter`, `ac`, `door`, `window`, `garage_door`, `drive_edge` or `fence`. `points` holds one -point for a meter, wall end, gas meter or AC unit, and two for an opening (bottom-left and -top-right corners), a drive edge or a fence. Each mark also has optional `t`, `photo_ids` and -`attrs.operable`. A wall end adds `side` (`left`, `right`) and `end_kind` (`limit` when something -blocks it, `unexplored` for a corner or no answer). App: partial. `scene.json` carries these as -objects and wall ends in C1's feet and `s` coordinates, without times or photo links. - -`guidance` logs every request the homeowner was shown, and whether it was met: - -- `kind`: `walk`, `tilt_to_ground`, `step_back`, `mark_end`, `closeup`, `gap_band` or - `gap_past_end`; -- `origin`: `phone` or `server`; -- `message`, and for a band request `band` and `span_m` (meter frame x); -- `t_shown` and `t_resolved`; -- `outcome`: `met`, `skipped`, `cannot_reach`, `superseded` or `unresolved`. - -The log is how the server learns what the homeowner could not reach, which `scene.json` cannot -say today (bug triage B-08). App: planned; guidance is transient UI state. - -`scene` is the app's `scene.json`, unchanged, with its `schema_version`. It stays the input to -placement; the packet carries it so reconstruction and placement see the same capture. App: now. - -`provenance` (`dataset`, `license`, `source`, `notes`) says where a converted sample came from and -what it lacks. The app does not write it. - -## Versioning - -`packet_version` is `MAJOR.MINOR`. A reader accepts any packet with its own major version and -ignores fields it does not know. A writer bumps the minor version for additions (a new optional -field, stream or enum value a reader may skip) and the major version for anything an old reader -would misread: a renamed or removed field, a changed unit, frame or column order. The validator -accepts `1.x`. - -1.1 adds, from S6's proposals on #22: `depth_frames`; the `estimated` depth source with -`sigma`; top-level `planes` with `boundary_m`, and the statement of what a plane's `pose` -includes; and `session.device.mesh_classification_enabled`. Two rules apply only to packets -that say 1.1 or later, so 1.0 packets stay valid: ARKit depth needs `confidence`, and a mesh -needs the classification flag. A 1.0 reader of a 1.1 packet misses depth frames, sigma and -top-level planes, and reads everything else as before. - -## Transport - -This is a recommendation; Hunter's team decides. A packet does not fit in one request. The -hosted API refuses bodies over 4.5 MB (Vercel's limit). The app's Simulator export of 77 photos -at 1280 × 720 is already 16.5 MB. At an estimated 0.5 MB per 1920 × 1440 photo and 3 MB per -high-resolution still, 80 photos come to about 40 MB, or 250 MB with stills. These are estimates: -no device capture has been measured. - -1. The phone writes the packet on disk as it captures, so a crash or a lost connection loses - nothing already taken. -2. `POST /v1/packets` with `manifest.json` only (tens of KB). The server validates the manifest - and answers with a packet id and one presigned upload URL per file in object storage (S3, - R2 or Vercel Blob), keyed by sha256. -3. The phone uploads each file with a background `URLSession` upload task, which iOS continues - after the app is suspended. Files over about 8 MB use multipart upload. Because files are - named by their hash, a retry skips any file the server already holds. -4. `POST /v1/packets/{id}/complete`. The server checks every size and hash (this validator), - then queues reconstruction. -5. Placement stays a separate small request. `POST /v1/placements` with `scene.json` (KB) can - answer at once, and a later answer can use the reconstruction. - -## Relation to existing formats - -- **`scene.json` (C1)** is unchanged and travels inside the packet. It stays in feet and in - ARKit's world moved so the ground is y = 0. The rest of the packet is in meters and in the - meter frame. -- **Measure Lab session v2** (the replay format) shares the packet's units, camera axes, - column-major poses and depth encoding. It differs in frame (world, not meter), file names, a - per-keyframe `depth` object `{file, confidenceFile, w, h}`, and a `tracking[]` timeline where - the packet has a trajectory stream. -- **`recon/recon/capture.py`** on `t3/recon` reads `scene.json` bundles (feet, scaled to meters - on load) and Measure Lab v2. Reading a packet needs a third branch: photo poses are already - in meters, and the wall direction and normal are the meter frame's x and z axes. - -## What the app records today - -At `t3/ios-mvf` `a39d0a5`, the app's export (`scan.zip`) is `scene.json` plus one JPEG per kept -keyframe. The upload sends `scene.json` alone. To write a 1.0 packet the app needs, in rough -order of cost: - -1. **Already in memory, not exported:** each photo's timestamp and tracking state, the meter - anchor's orientation, and mark times. -2. **One read at session start:** device model, iOS version, app version, LiDAR support, the - wall-clock start. -3. **Convert:** poses to meters in the meter frame, and marks from C1's `s`/height to meter-frame - points. -4. **Record and write:** the 60 Hz trajectory (today discarded after the overlay uses it), - exposure and ISO per photo, sharpness by the fixed method, distance walked, and the guidance - log. -5. **New sensors:** Core Motion (accelerometer, gyroscope, magnetometer, device motion, - altimeter); location and heading behind a consent prompt. -6. **LiDAR and the live map (S6):** `sceneDepth` with confidence per photo, depth frames between - photos, estimated depth with sigma on phones without LiDAR, the mesh with its - classification flag, plane anchors with boundaries. -7. **Transport:** the manifest-then-files upload above. - -## Tools - -From `packet/`: - -``` -uv run python -m packet validate # exit 0 when valid; --json for a report -uv run python -m packet.samples.synthetic --out fixtures/synthetic -uv run python -m packet.samples.app_export --sim-report --replay --out -uv run python -m packet.samples.advio --replay --advio --out -uv run python -m packet.samples.eth3d --eth3d --out -uv run pytest -q && uv run ruff check . && uv run ruff format --check . -``` - -`fixtures/synthetic` is the only packet in git: three invented photos with every optional -section. The dataset samples are written to `~/house-scanning-data/packets/` and never -committed. ADVIO is CC BY-NC 4.0, ETH3D CC BY-NC-SA 4.0, and the app sample carries ADVIO frames. - -| Sample | Source | Contents | What it shows | -| --- | --- | --- | --- | -| `app-export-a39d0a5` | The app's `scan.zip` from a Simulator run on the ADVIO replay | 77 photos (1280 × 720), marks, `scene.json` | Exactly what the app records today. Times are recovered by matching each photo to its replay frame | -| `advio-20-0040-0075` | ADVIO sequence 20, 40 to 75 s, iPhone 6s | 79 photos (1280 × 720), 60 Hz trajectory, accelerometer, gyroscope, magnetometer, barometer, location | Real phone IMU on the same clock as ARKit poses | -| `eth3d-electro` | ETH3D electro, 8 DSLR photos within about 6 m of the walls | 8 photos (6198 × 4132, originals) with laser-rendered depth | Depth aligned to full-resolution photos. ETH3D's own 3-D points reproject through the packet's poses to a median 0.65 to 1.01 px, and the depth agrees with them to a median 0.2 to 0.6% | - -The validator checks: - -- the schema; -- every file's presence, size and sha256; -- that every pose is a rotation plus a translation; -- that intrinsics fit each image (landscape, principal point inside it, square pixels within 5%, - a horizontal field of view between 20° and 150°); -- that JPEG sizes and EXIF orientation match the manifest; -- that times strictly increase and fall inside the capture; -- the stream columns, unit quaternions and tracking values; -- that photos sit on the trajectory, and that distance walked matches it; -- that depth has its photo's aspect and the right byte count, with finite, non-negative values - and sigma, confidence of at most 2, and confidence for ARKit depth and sigma for estimated - depth; -- that depth frames sit on the trajectory, in time order, with intrinsics that fit their grid; -- that plane boundaries lie inside their extent, and planes appear in one place only; -- the mesh layout, face indices and classes, and the classification flag; -- mark and guidance consistency; -- consent for location and heading. +The capture packet is how a scan leaves the phone. It is specified by the server team's capture +packet spec, `formatVersion` 0.4, together with its intake API. The spec is +`docs/capture-packet-spec.md` in the private repository `huntertcarver/house-scanning-server`; +the API is its Appendix D (`/v1/captures`, `/files`, `/files:commit`, `/finalize`). That spec is +the one contract; this folder does not define a format of its own. + +This folder holds the app side's proposals for the next revision (0.5): what the app records +that 0.4 has no field for, and three arrival checks for the server's validator +([validator-checks.md](validator-checks.md)). Packet 1.1, which this folder used to define, is +retired; it is still readable at `t3/packet` commit `d5439cf`. + +## Proposals for 0.5 + +Each item is something the app knows at capture time and the server would otherwise have to +guess. All are additive, so 0.4 readers keep working. + +1. **The meter anchor's orientation.** 0.4 records the meter tap's point, not the wall it sits + on. Every placement distance is measured along that wall from the meter (C1's `s = 0`), and + the app knows the wall at the tap: the vertical plane its raycast hit. Proposal: on the + `meter` tap, a `meterAnchor` with `transform` (anchor to world in the tap's epoch: origin at + the meter, +y up, +z the wall's outward normal, +x along the wall to the right seen from + outside) and `normalSource` (`plane`, `mesh` or `estimated`). +2. **The ground's height at the meter.** 0.4 notes that world y = 0 is the phone's height at + session start, not the ground. The app measures the ground where it can: the horizontal plane + below the meter. Proposal: `meterAnchor.groundY` (world meters) with `groundSource` + (`plane`, `mesh`, `assumed`), so headroom and "ground under the battery" start from a + measurement when there is one. +3. **What a marked end or feature is.** 0.4's tap labels name the point, not the answers the + homeowner gave about it: + - whether a wall end is a real end (something blocks it) or unexplored (it turns a corner, + or no answer), which decides whether the server may reject for want of space; + - whether a window opens, which decides whether the opening clearance applies; + - which corner taps belong to the same door or window. + + Proposal: `taps[].attrs` with `endKind` (`limit`, `unexplored`) and `operable`, and + `taps[].group` for taps that outline one object. +4. **The guidance log.** Every request the homeowner was shown (walk, tilt to the ground, mark + an end, close-up, a gap the phone or the server asked for), when, and its outcome: `met`, + `skipped`, `cannot_reach`, `superseded` or `unresolved`. Today "I can't get there" never + reaches the server, so the server can ask for the same unreachable view again. Proposal: + `guidance[]` with `id`, `kind`, `origin` (`phone`, `server`), `message`, optional `band` and + `span`, `tShown`, `tResolved` and `outcome`. +5. **Depth between keyframes** (from S6's live map, draft PR #21). Keyframes are kept about every + 0.5 m, but the phone fuses depth at about 10 Hz to decide what was seen. With keyframe depth + alone the server cannot rebuild that decision. Proposal: optional `depthFrames[]` with `t`, + `epoch`, pose, intrinsics of the depth map's own grid, and the same depth and confidence files + as a keyframe. At 256 × 192, one frame with confidence is about 246 KB, so a few hertz rather + than ten. +6. **Estimated depth with its uncertainty** (S6). A phone without LiDAR can infer metric depth + from the image, but used without its uncertainty it creates surfaces that are not there. + Proposal: a depth `source` of `estimated`, which requires a `sigma` file (float32 meters, one + standard deviation per depth pixel, the depth's layout). +7. **The frame of a plane's boundary.** 0.4 lists `planes[].boundary` without saying its frame. + ARKit's `boundaryVertices` are relative to the anchor transform, not to `planeExtent`'s centre + and `rotationOnYAxis`, and packet 1.1 found that mixing the two is an easy mistake. Proposal: + state that `boundary` is `[x, z]` in the anchor's frame, and have the validator check it lies + within `planeExtent` once the centre and rotation are applied. + +Packet 1.1 also had a fixed sharpness score, distance walked and a location-consent flag. They are +left out: 0.4's exposure and EXIF, `arkitPoses` and `locationAuthorization` cover them. diff --git a/packet/fixtures/synthetic/depth/p00001.conf.u8 b/packet/fixtures/synthetic/depth/p00001.conf.u8 deleted file mode 100644 index f52aa7368938ce3aad8b1dee3ab343d97e5e8989..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 432 UcmZQzV8RL*M#aJ5GT6ca0GxsY82|tP diff --git a/packet/fixtures/synthetic/depth/p00001.f32 b/packet/fixtures/synthetic/depth/p00001.f32 deleted file mode 100644 index 3a47744fd4b9738dcded9b1bb2212dfcc75db373..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 1728 lcmZQzfB^*u1_oN-QSuC6lh7TqAZZ!Rjrk|0UegK~Clz{*M diff --git a/packet/fixtures/synthetic/depth/p00002.conf.u8 b/packet/fixtures/synthetic/depth/p00002.conf.u8 deleted file mode 100644 index f52aa7368938ce3aad8b1dee3ab343d97e5e8989..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 432 UcmZQzV8RL*M#aJ5GT6ca0GxsY82|tP diff --git a/packet/fixtures/synthetic/depth/p00002.f32 b/packet/fixtures/synthetic/depth/p00002.f32 deleted file mode 100644 index 3a47744fd4b9738dcded9b1bb2212dfcc75db373..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 1728 lcmZQzfB^*u1_oN-QSuC6lh7TqAZZ!Rjrk|0UegK~Clz{*M diff --git a/packet/fixtures/synthetic/depth/p00003.conf.u8 b/packet/fixtures/synthetic/depth/p00003.conf.u8 deleted file mode 100644 index f52aa7368938ce3aad8b1dee3ab343d97e5e8989..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 432 UcmZQzV8RL*M#aJ5GT6ca0GxsY82|tP diff --git a/packet/fixtures/synthetic/depth/p00003.f32 b/packet/fixtures/synthetic/depth/p00003.f32 deleted file mode 100644 index 3a47744fd4b9738dcded9b1bb2212dfcc75db373..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 1728 lcmZQzfB^*u1_oN-QSuC6lh7TqAZZ!Rjrk|0UegK~Clz{*M diff --git a/packet/fixtures/synthetic/depth_frames/d00001.conf.u8 b/packet/fixtures/synthetic/depth_frames/d00001.conf.u8 deleted file mode 100644 index 5e25ef9e5..000000000 --- a/packet/fixtures/synthetic/depth_frames/d00001.conf.u8 +++ /dev/null @@ -1 +0,0 @@ - \ No newline at end of file diff --git a/packet/fixtures/synthetic/depth_frames/d00001.f32 b/packet/fixtures/synthetic/depth_frames/d00001.f32 deleted file mode 100644 index 602e2160e5dcc3e531c86c68dbe1cce17eee4cae..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 1728 ecmZQzP;g)v#iL;`ng&MGz-Ss6O#?$L4FCXozM=sD diff --git a/packet/fixtures/synthetic/depth_frames/d00002.f32 b/packet/fixtures/synthetic/depth_frames/d00002.f32 deleted file mode 100644 index 602e2160e5dcc3e531c86c68dbe1cce17eee4cae..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 1728 ecmZQzP;g)v#iL;`ng&MGz-Ss6O#?$L4FCXozM=sD diff --git a/packet/fixtures/synthetic/depth_frames/d00002.sigma.f32 b/packet/fixtures/synthetic/depth_frames/d00002.sigma.f32 deleted file mode 100644 index aaaa711c4..000000000 --- a/packet/fixtures/synthetic/depth_frames/d00002.sigma.f32 +++ /dev/null @@ -1 +0,0 @@ -š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™>š™> \ No newline at end of file diff --git a/packet/fixtures/synthetic/lidar/mesh.ply b/packet/fixtures/synthetic/lidar/mesh.ply deleted file mode 100644 index 02082c8767474b81b22d003a7fa5db132499ed91..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 369 zcmZ{eZwkUN48`aC2R%hkAPRo)o4v`XHDiIU9c>irP4qH)G}9?g1{Tu3{Q7uFXOpOX zXf??LIvrAFk;p;??ugpS&1x--Z(DFAV{+60p%GxACuax*7r7H%@Ez+UYwJ1cG6rc3 zuHTGW8<5-K-(nR?1_J;FeE`w}TmTM>RSZ<{=pYaY1U#NVQdd_aYLYZH$s{saOIuG@OPiui zChHpLQuL`b8ckD&ZfHO?)T7d<3L_XCdInF>AP_XDT4XKidkb;{P>8@C1mZ9z087E( zC>W#>7^86#Fs}ssB^WFkqnbKVgG5F*RO$jO28YAqaf+zX-3L%Tz*7i%i&i+R>2C^B zH;JTLCubEBP1lvy(R|v+&1|;sOVJ=1&<*E(WNxw8a>-I#yOs8<99Fx0zTVZ%-DAV% zFMPN7eYw?t$2Y+{LqfyCqoQN>$Hv7cq#itUIPJ*M^b^@9Pvx9Glbcs`{=&seoXf=} zW#tu>zf}Et?RtGfV^ed>&DP)Ubo_C*^WOcgZjt!Wz~jN8C&N!CUQABCoSur0qqsCcLSF4LCF3F_7ARZpoznv#lukm1H2R%A)qx}vK0Z5Ovu82 zs(aE(bwzsfIRdVvRr&REh7nL@b4##OC^e~_kIvc9lAKKC{fxPk?5nkIE(nM(M!;U! zOW~}m9CqiWF-C_FupyYYk!KB^WelWCqTpWE)Fpq;gw;r?(7508Zjz6VvY&s8)U+DD zPGF|6Iq(CavLVNB5n#?0g+NWLsoYQ{A4;n5qeGalr5-P9cD+B z3vBn=?^^E%Gg{B(B7p279)l=W^6KRXkmQMKiHXjV(--Ad?Yk>75BBvWB*eqMGjg+> zPIynq5A26!1A!+TM!uinakwh@E26$`d&ta`=9gwnTPF%*dMfzGaSg`$yfv0`pK+(# zg#`$xcVJxzmknk`22a<;95typQrcT`o}1O|)|Q`tul>}E*!oQAC(M~Oaa;s=6!3lu z^DuWXheFfr6QUu;^srIz>`GQEZF~sgBVgF8B5nru^s5)nrlWt>?Q`fl89|B1Iq>9W zxKExtTsg=r50PgvwOKtv_+(qa*#Kp`mEL0Jy&dNmH%^zxFBmzYCA~Xd4vPgcuS)c6 zfbv3xTPze!5)c^P(6w_ab{?_ND#A*I1iguUcy$stXm|1b zjSy$hzjs|(qa@jI&ldcAUE3W!)V6%R(CZ#elzbz`q5pMo! zkH8{+sII!_fTwthzCXJuCM@z1o9d}KzAA3OwemaeGHv3*6yivpOw4C(8*e@)Vr2A@ zM{S)fT3jpQ?ZW)E)^sgAmfu~l+!x?QYlT;*c!9-x^L!#??X*q)KiWM;Gv%&w@z1vi zUijA*F%-O@OrXVbPua7m z7Kz?V=E+p0L@4-1W#Nc|iPv*ypN diff --git a/packet/fixtures/synthetic/photos/p00002.jpg b/packet/fixtures/synthetic/photos/p00002.jpg deleted file mode 100644 index a1469413db3ad39154a2b7de45110c4e399bc124..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 2539 zcmd6od010N5WqJVS3pQe08bRfoAL>YwL()+1Vp?L5v)YT1JR0za;WeG&*DJ=j{<5m z9uZna)sOr{v<>gwnj8W}X66$TSLl)D&kO(?zRvtwI@g zv5DvOM!Q@uWG=1}jj><7HBR5a%-mw|5bLqy#!r|yb(+KUuV;KSYqskgH+PS@-b;Lz z`Y!YH4_vc$-TI&n!C~9Bhet$4?bx$--~Iy!e~eE`K6d=X$)8hFx#urjyu`blo^c~9 zJ16(%t=oA;#U-Wp%gP@-d|F*o`>d}1xv;sVwe5BLn~u&dabLgWy>vh(S7^8hfb(}HNt2od$|wk)rfjU< zSjk@}kXscDgWx7dJ>f(_sLE5(y)zUg-}QYxQrxRLk-3jEz{Do6&GgE&(;DE;iBBMS6^t}MknH5} z`5@CZF@}}N=7#a3$#9DE1C)mYyqUMGfVuZ5d10)hv@GCv4H+3KjKLr&)S&9#&|ysvSP6* z%48LvW|y*`Lm=d@>_e&?d&gBa+PMu#bEoi~jOQ&Kmw>e8?3xV0txPo8CJ?P>_ic`` z!6+7}C-1+SG(S8Vg7VaNHBAuYK_E#VHoegJAd4>H?^D?1CW_LOAqy6;9u`2bH%Zv| zsIt#gnfv`=x|>tz8A)V*i*ZSQLi`BA?b|Xhsh3Nma#nXZOO&*nB^>yUyWkb2Xp@EG z`^S_W3EfzHV|llfn+4@} zMIkNQ-;@p?y8L#{R7;tgt>j2Rv{g$*PuyGj?90n~eHPW0w~l`Ocz&>!?Psj+ItZqD zqh*hXZ)n9l^tJTkT1&@3HyvKf=vB{$cAWci_?6aqr;zd zneWsoY7^PlR4G}zCW@Bkhz>d5sd2PC%MS3A9vLwIW)0n4cIzj@tO?@)LBfcvYxa%1 z^6ghDhpat?3bI24B4h{MOqdrNnD#^7@mJ@z)H@!QOyx(E-|PsMqu0GRDjSn+Vk+u# z6~{Rx75N@5dD*wbk9m{r?=`n3TFmx2et>M^Oy2$|B<2yP&vn-f>~66X-Zl{dmZJMd znrBYWej-U5KBTJ6E_+Dy4d|#4Tsn*YEJ9;tnFaU7oQ5Sw7g=fA7iEFUSts zO9)oz{=dqCrl#Nvg+w?yAI+n%<2XF@g#ce)9Ct!s{Y&$DbtxiA3Br{goGLh`NIu+H z-Of|<<@SFo_?n7|ZwkSj68_aPM9T`Vn=$`Z**ZA@uqB?+gCOD=0O zWz{5S8`4mcVWdcVgsf{BlO-aT5z{d9?Qd$&?hd=>?C#(D{w{yK-}hdg?|HumwnH({ z-{Ru#0#GObP{;#d7jOa?G+Ol{hDBbSCJu+i;)r;>hNd=ATU(2$rKO`w*4NP`>1t`| z)AUIO6e^Xft!HRNqZpAXREjDI3WLO8aReNWK+)0Cq5NxuRe+=k+`t|TY6(D-P#6*l zJ_II6pE%S91HTyxjr6F2*CY_NkPQNTfJR|3Xe>t6HM098vJS8$9C`7YjT#2q0`W@@ zQ>E)+Tn`7k6bZt)ro^2TlWz{3VhtYbAPn^LY8R})NayvL|} zBb$h{rE7c5##PWhAp2*)PW>ljzXSUNS0~WMpb+vfB)|ajyo!d$!U?}h8mHR_0{4su zc5@t+k-Ja!WREVA?!@L8dk7*J^6jhi zB+Rxw7skwngqHXVoIXL!N(k=qm1(O~!E+p>7ZCWP`PE)65F84glW3761gWiKB}$1< z?s-SOh}xQf5g2nmK`Gs??3v9T6!bGof@f2hx}2^?WqLzEPC(F%9jBf;HlV10V9-y2)yDa@V9iLg~&+gkU(R z9D*jhcRkV5N)I1V<4h8A$kxwyxQA4nD=1rjJ2L86W?$o0zO^EU(U&AUqI}7jyzV#I zDYfb-QM&OH7$XA^YzZphkd^mICSs3Pwzu9Q2*Ef?0poz&SQGdU=avv?gr5 zgUJ=A;3&nlspRO%I+}fHSAqYKp-x;~qfP~`EUPggm#^L*LGHfzBvNxC{QWHm)L1HHH*^N&=8WxjwmWvwFDH?1mQ}rg zm!&*$tG}vmWb;?KPw;`eV$=r2zTVxRwNON-@(f#YCW7bP?)eJPtYXgiv5fqVnI5NB zejXDK?9RT0;8v`Um^TPP*=oM^_~>i~Kb|{HMYsv#BU7U0$V0Rz>x=)TXP)#>wymlh zeQFUC(ae;4I3k)og-Ai_ir&T<7IKi;A$bYoR&R=2|B{m=i}R!M*Hku<&7C>oFgH^D u{FAsvm99<@unG`yS9;F7wRuTAq)>6=-`rb^yw8MhuI!Xq$~hxXVaKmL(PZoZ diff --git a/packet/fixtures/synthetic/scene.json b/packet/fixtures/synthetic/scene.json deleted file mode 100644 index 68cb6efb7..000000000 --- a/packet/fixtures/synthetic/scene.json +++ /dev/null @@ -1,26 +0,0 @@ -{ - "schema_version": "1.0", - "meter": { - "pos": [ - 0.0, - 4.0, - 0.0 - ], - "wall_id": "w1" - }, - "walls": [ - { - "id": "w1", - "baseline": [ - [ - -9.84, - 0.0 - ], - [ - 9.84, - 0.0 - ] - ] - } - ] -} diff --git a/packet/fixtures/synthetic/streams/accelerometer.csv b/packet/fixtures/synthetic/streams/accelerometer.csv deleted file mode 100644 index 76fc6acc8..000000000 --- a/packet/fixtures/synthetic/streams/accelerometer.csv +++ /dev/null @@ -1,201 +0,0 @@ -t,x,y,z -100.0,-9.81,0.0,0.0 -100.01,-9.81,0.0,0.0 -100.02,-9.81,0.0,0.0 -100.03,-9.81,0.0,0.0 -100.04,-9.81,0.0,0.0 -100.05,-9.81,0.0,0.0 -100.06,-9.81,0.0,0.0 -100.07,-9.81,0.0,0.0 -100.08,-9.81,0.0,0.0 -100.09,-9.81,0.0,0.0 -100.1,-9.81,0.0,0.0 -100.11,-9.81,0.0,0.0 -100.12,-9.81,0.0,0.0 -100.13,-9.81,0.0,0.0 -100.14,-9.81,0.0,0.0 -100.15,-9.81,0.0,0.0 -100.16,-9.81,0.0,0.0 -100.17,-9.81,0.0,0.0 -100.18,-9.81,0.0,0.0 -100.19,-9.81,0.0,0.0 -100.2,-9.81,0.0,0.0 -100.21,-9.81,0.0,0.0 -100.22,-9.81,0.0,0.0 -100.23,-9.81,0.0,0.0 -100.24,-9.81,0.0,0.0 -100.25,-9.81,0.0,0.0 -100.26,-9.81,0.0,0.0 -100.27,-9.81,0.0,0.0 -100.28,-9.81,0.0,0.0 -100.29,-9.81,0.0,0.0 -100.3,-9.81,0.0,0.0 -100.31,-9.81,0.0,0.0 -100.32,-9.81,0.0,0.0 -100.33,-9.81,0.0,0.0 -100.34,-9.81,0.0,0.0 -100.35,-9.81,0.0,0.0 -100.36,-9.81,0.0,0.0 -100.37,-9.81,0.0,0.0 -100.38,-9.81,0.0,0.0 -100.39,-9.81,0.0,0.0 -100.4,-9.81,0.0,0.0 -100.41,-9.81,0.0,0.0 -100.42,-9.81,0.0,0.0 -100.43,-9.81,0.0,0.0 -100.44,-9.81,0.0,0.0 -100.45,-9.81,0.0,0.0 -100.46,-9.81,0.0,0.0 -100.47,-9.81,0.0,0.0 -100.48,-9.81,0.0,0.0 -100.49,-9.81,0.0,0.0 -100.5,-9.81,0.0,0.0 -100.51,-9.81,0.0,0.0 -100.52,-9.81,0.0,0.0 -100.53,-9.81,0.0,0.0 -100.54,-9.81,0.0,0.0 -100.55,-9.81,0.0,0.0 -100.56,-9.81,0.0,0.0 -100.57,-9.81,0.0,0.0 -100.58,-9.81,0.0,0.0 -100.59,-9.81,0.0,0.0 -100.6,-9.81,0.0,0.0 -100.61,-9.81,0.0,0.0 -100.62,-9.81,0.0,0.0 -100.63,-9.81,0.0,0.0 -100.64,-9.81,0.0,0.0 -100.65,-9.81,0.0,0.0 -100.66,-9.81,0.0,0.0 -100.67,-9.81,0.0,0.0 -100.68,-9.81,0.0,0.0 -100.69,-9.81,0.0,0.0 -100.7,-9.81,0.0,0.0 -100.71,-9.81,0.0,0.0 -100.72,-9.81,0.0,0.0 -100.73,-9.81,0.0,0.0 -100.74,-9.81,0.0,0.0 -100.75,-9.81,0.0,0.0 -100.76,-9.81,0.0,0.0 -100.77,-9.81,0.0,0.0 -100.78,-9.81,0.0,0.0 -100.79,-9.81,0.0,0.0 -100.8,-9.81,0.0,0.0 -100.81,-9.81,0.0,0.0 -100.82,-9.81,0.0,0.0 -100.83,-9.81,0.0,0.0 -100.84,-9.81,0.0,0.0 -100.85,-9.81,0.0,0.0 -100.86,-9.81,0.0,0.0 -100.87,-9.81,0.0,0.0 -100.88,-9.81,0.0,0.0 -100.89,-9.81,0.0,0.0 -100.9,-9.81,0.0,0.0 -100.91,-9.81,0.0,0.0 -100.92,-9.81,0.0,0.0 -100.93,-9.81,0.0,0.0 -100.94,-9.81,0.0,0.0 -100.95,-9.81,0.0,0.0 -100.96,-9.81,0.0,0.0 -100.97,-9.81,0.0,0.0 -100.98,-9.81,0.0,0.0 -100.99,-9.81,0.0,0.0 -101.0,-9.81,0.0,0.0 -101.01,-9.81,0.0,0.0 -101.02,-9.81,0.0,0.0 -101.03,-9.81,0.0,0.0 -101.04,-9.81,0.0,0.0 -101.05,-9.81,0.0,0.0 -101.06,-9.81,0.0,0.0 -101.07,-9.81,0.0,0.0 -101.08,-9.81,0.0,0.0 -101.09,-9.81,0.0,0.0 -101.1,-9.81,0.0,0.0 -101.11,-9.81,0.0,0.0 -101.12,-9.81,0.0,0.0 -101.13,-9.81,0.0,0.0 -101.14,-9.81,0.0,0.0 -101.15,-9.81,0.0,0.0 -101.16,-9.81,0.0,0.0 -101.17,-9.81,0.0,0.0 -101.18,-9.81,0.0,0.0 -101.19,-9.81,0.0,0.0 -101.2,-9.81,0.0,0.0 -101.21,-9.81,0.0,0.0 -101.22,-9.81,0.0,0.0 -101.23,-9.81,0.0,0.0 -101.24,-9.81,0.0,0.0 -101.25,-9.81,0.0,0.0 -101.26,-9.81,0.0,0.0 -101.27,-9.81,0.0,0.0 -101.28,-9.81,0.0,0.0 -101.29,-9.81,0.0,0.0 -101.3,-9.81,0.0,0.0 -101.31,-9.81,0.0,0.0 -101.32,-9.81,0.0,0.0 -101.33,-9.81,0.0,0.0 -101.34,-9.81,0.0,0.0 -101.35,-9.81,0.0,0.0 -101.36,-9.81,0.0,0.0 -101.37,-9.81,0.0,0.0 -101.38,-9.81,0.0,0.0 -101.39,-9.81,0.0,0.0 -101.4,-9.81,0.0,0.0 -101.41,-9.81,0.0,0.0 -101.42,-9.81,0.0,0.0 -101.43,-9.81,0.0,0.0 -101.44,-9.81,0.0,0.0 -101.45,-9.81,0.0,0.0 -101.46,-9.81,0.0,0.0 -101.47,-9.81,0.0,0.0 -101.48,-9.81,0.0,0.0 -101.49,-9.81,0.0,0.0 -101.5,-9.81,0.0,0.0 -101.51,-9.81,0.0,0.0 -101.52,-9.81,0.0,0.0 -101.53,-9.81,0.0,0.0 -101.54,-9.81,0.0,0.0 -101.55,-9.81,0.0,0.0 -101.56,-9.81,0.0,0.0 -101.57,-9.81,0.0,0.0 -101.58,-9.81,0.0,0.0 -101.59,-9.81,0.0,0.0 -101.6,-9.81,0.0,0.0 -101.61,-9.81,0.0,0.0 -101.62,-9.81,0.0,0.0 -101.63,-9.81,0.0,0.0 -101.64,-9.81,0.0,0.0 -101.65,-9.81,0.0,0.0 -101.66,-9.81,0.0,0.0 -101.67,-9.81,0.0,0.0 -101.68,-9.81,0.0,0.0 -101.69,-9.81,0.0,0.0 -101.7,-9.81,0.0,0.0 -101.71,-9.81,0.0,0.0 -101.72,-9.81,0.0,0.0 -101.73,-9.81,0.0,0.0 -101.74,-9.81,0.0,0.0 -101.75,-9.81,0.0,0.0 -101.76,-9.81,0.0,0.0 -101.77,-9.81,0.0,0.0 -101.78,-9.81,0.0,0.0 -101.79,-9.81,0.0,0.0 -101.8,-9.81,0.0,0.0 -101.81,-9.81,0.0,0.0 -101.82,-9.81,0.0,0.0 -101.83,-9.81,0.0,0.0 -101.84,-9.81,0.0,0.0 -101.85,-9.81,0.0,0.0 -101.86,-9.81,0.0,0.0 -101.87,-9.81,0.0,0.0 -101.88,-9.81,0.0,0.0 -101.89,-9.81,0.0,0.0 -101.9,-9.81,0.0,0.0 -101.91,-9.81,0.0,0.0 -101.92,-9.81,0.0,0.0 -101.93,-9.81,0.0,0.0 -101.94,-9.81,0.0,0.0 -101.95,-9.81,0.0,0.0 -101.96,-9.81,0.0,0.0 -101.97,-9.81,0.0,0.0 -101.98,-9.81,0.0,0.0 -101.99,-9.81,0.0,0.0 diff --git a/packet/fixtures/synthetic/streams/barometer.csv b/packet/fixtures/synthetic/streams/barometer.csv deleted file mode 100644 index beae7636f..000000000 --- a/packet/fixtures/synthetic/streams/barometer.csv +++ /dev/null @@ -1,3 +0,0 @@ -t,pressure_kpa,relative_altitude_m -100.5,101.3,0.0 -101.5,101.3,0.0 diff --git a/packet/fixtures/synthetic/streams/device_motion.csv b/packet/fixtures/synthetic/streams/device_motion.csv deleted file mode 100644 index c39486f10..000000000 --- a/packet/fixtures/synthetic/streams/device_motion.csv +++ /dev/null @@ -1,201 +0,0 @@ -t,qx,qy,qz,qw,gravity_x,gravity_y,gravity_z,user_accel_x,user_accel_y,user_accel_z,rotation_rate_x,rotation_rate_y,rotation_rate_z,heading_deg -100.0,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.01,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.02,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.03,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.04,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.05,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.06,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.07,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.08,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.09,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.1,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.11,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.12,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.13,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.14,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.15,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.16,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.17,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.18,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.19,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.2,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.21,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.22,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.23,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.24,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.25,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.26,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.27,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.28,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.29,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.3,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.31,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.32,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.33,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.34,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.35,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.36,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.37,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.38,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.39,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.4,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.41,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.42,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.43,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.44,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.45,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.46,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.47,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.48,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.49,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.5,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.51,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.52,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.53,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.54,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.55,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.56,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.57,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.58,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.59,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.6,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.61,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.62,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.63,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.64,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.65,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.66,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.67,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.68,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.69,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.7,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.71,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.72,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.73,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.74,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.75,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.76,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.77,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.78,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.79,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.8,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.81,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.82,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.83,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.84,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.85,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.86,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.87,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.88,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.89,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.9,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.91,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.92,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.93,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.94,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.95,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.96,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.97,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.98,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -100.99,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.0,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.01,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.02,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.03,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.04,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.05,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.06,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.07,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.08,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.09,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.1,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.11,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.12,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.13,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.14,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.15,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.16,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.17,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.18,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.19,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.2,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.21,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.22,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.23,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.24,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.25,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.26,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.27,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.28,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.29,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.3,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.31,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.32,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.33,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.34,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.35,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.36,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.37,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.38,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.39,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.4,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.41,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.42,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.43,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.44,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.45,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.46,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.47,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.48,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.49,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.5,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.51,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.52,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.53,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.54,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.55,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.56,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.57,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.58,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.59,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.6,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.61,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.62,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.63,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.64,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.65,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.66,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.67,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.68,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.69,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.7,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.71,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.72,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.73,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.74,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.75,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.76,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.77,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.78,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.79,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.8,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.81,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.82,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.83,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.84,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.85,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.86,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.87,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.88,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.89,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.9,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.91,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.92,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.93,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.94,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.95,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.96,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.97,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.98,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 -101.99,0,0,0,1,-1,0,0,0,0,0,0,0,0,90.0 diff --git a/packet/fixtures/synthetic/streams/gyroscope.csv b/packet/fixtures/synthetic/streams/gyroscope.csv deleted file mode 100644 index 52c5a41ec..000000000 --- a/packet/fixtures/synthetic/streams/gyroscope.csv +++ /dev/null @@ -1,201 +0,0 @@ -t,x,y,z -100.0,0.0,0.0,0.0 -100.01,0.0,0.0,0.0 -100.02,0.0,0.0,0.0 -100.03,0.0,0.0,0.0 -100.04,0.0,0.0,0.0 -100.05,0.0,0.0,0.0 -100.06,0.0,0.0,0.0 -100.07,0.0,0.0,0.0 -100.08,0.0,0.0,0.0 -100.09,0.0,0.0,0.0 -100.1,0.0,0.0,0.0 -100.11,0.0,0.0,0.0 -100.12,0.0,0.0,0.0 -100.13,0.0,0.0,0.0 -100.14,0.0,0.0,0.0 -100.15,0.0,0.0,0.0 -100.16,0.0,0.0,0.0 -100.17,0.0,0.0,0.0 -100.18,0.0,0.0,0.0 -100.19,0.0,0.0,0.0 -100.2,0.0,0.0,0.0 -100.21,0.0,0.0,0.0 -100.22,0.0,0.0,0.0 -100.23,0.0,0.0,0.0 -100.24,0.0,0.0,0.0 -100.25,0.0,0.0,0.0 -100.26,0.0,0.0,0.0 -100.27,0.0,0.0,0.0 -100.28,0.0,0.0,0.0 -100.29,0.0,0.0,0.0 -100.3,0.0,0.0,0.0 -100.31,0.0,0.0,0.0 -100.32,0.0,0.0,0.0 -100.33,0.0,0.0,0.0 -100.34,0.0,0.0,0.0 -100.35,0.0,0.0,0.0 -100.36,0.0,0.0,0.0 -100.37,0.0,0.0,0.0 -100.38,0.0,0.0,0.0 -100.39,0.0,0.0,0.0 -100.4,0.0,0.0,0.0 -100.41,0.0,0.0,0.0 -100.42,0.0,0.0,0.0 -100.43,0.0,0.0,0.0 -100.44,0.0,0.0,0.0 -100.45,0.0,0.0,0.0 -100.46,0.0,0.0,0.0 -100.47,0.0,0.0,0.0 -100.48,0.0,0.0,0.0 -100.49,0.0,0.0,0.0 -100.5,0.0,0.0,0.0 -100.51,0.0,0.0,0.0 -100.52,0.0,0.0,0.0 -100.53,0.0,0.0,0.0 -100.54,0.0,0.0,0.0 -100.55,0.0,0.0,0.0 -100.56,0.0,0.0,0.0 -100.57,0.0,0.0,0.0 -100.58,0.0,0.0,0.0 -100.59,0.0,0.0,0.0 -100.6,0.0,0.0,0.0 -100.61,0.0,0.0,0.0 -100.62,0.0,0.0,0.0 -100.63,0.0,0.0,0.0 -100.64,0.0,0.0,0.0 -100.65,0.0,0.0,0.0 -100.66,0.0,0.0,0.0 -100.67,0.0,0.0,0.0 -100.68,0.0,0.0,0.0 -100.69,0.0,0.0,0.0 -100.7,0.0,0.0,0.0 -100.71,0.0,0.0,0.0 -100.72,0.0,0.0,0.0 -100.73,0.0,0.0,0.0 -100.74,0.0,0.0,0.0 -100.75,0.0,0.0,0.0 -100.76,0.0,0.0,0.0 -100.77,0.0,0.0,0.0 -100.78,0.0,0.0,0.0 -100.79,0.0,0.0,0.0 -100.8,0.0,0.0,0.0 -100.81,0.0,0.0,0.0 -100.82,0.0,0.0,0.0 -100.83,0.0,0.0,0.0 -100.84,0.0,0.0,0.0 -100.85,0.0,0.0,0.0 -100.86,0.0,0.0,0.0 -100.87,0.0,0.0,0.0 -100.88,0.0,0.0,0.0 -100.89,0.0,0.0,0.0 -100.9,0.0,0.0,0.0 -100.91,0.0,0.0,0.0 -100.92,0.0,0.0,0.0 -100.93,0.0,0.0,0.0 -100.94,0.0,0.0,0.0 -100.95,0.0,0.0,0.0 -100.96,0.0,0.0,0.0 -100.97,0.0,0.0,0.0 -100.98,0.0,0.0,0.0 -100.99,0.0,0.0,0.0 -101.0,0.0,0.0,0.0 -101.01,0.0,0.0,0.0 -101.02,0.0,0.0,0.0 -101.03,0.0,0.0,0.0 -101.04,0.0,0.0,0.0 -101.05,0.0,0.0,0.0 -101.06,0.0,0.0,0.0 -101.07,0.0,0.0,0.0 -101.08,0.0,0.0,0.0 -101.09,0.0,0.0,0.0 -101.1,0.0,0.0,0.0 -101.11,0.0,0.0,0.0 -101.12,0.0,0.0,0.0 -101.13,0.0,0.0,0.0 -101.14,0.0,0.0,0.0 -101.15,0.0,0.0,0.0 -101.16,0.0,0.0,0.0 -101.17,0.0,0.0,0.0 -101.18,0.0,0.0,0.0 -101.19,0.0,0.0,0.0 -101.2,0.0,0.0,0.0 -101.21,0.0,0.0,0.0 -101.22,0.0,0.0,0.0 -101.23,0.0,0.0,0.0 -101.24,0.0,0.0,0.0 -101.25,0.0,0.0,0.0 -101.26,0.0,0.0,0.0 -101.27,0.0,0.0,0.0 -101.28,0.0,0.0,0.0 -101.29,0.0,0.0,0.0 -101.3,0.0,0.0,0.0 -101.31,0.0,0.0,0.0 -101.32,0.0,0.0,0.0 -101.33,0.0,0.0,0.0 -101.34,0.0,0.0,0.0 -101.35,0.0,0.0,0.0 -101.36,0.0,0.0,0.0 -101.37,0.0,0.0,0.0 -101.38,0.0,0.0,0.0 -101.39,0.0,0.0,0.0 -101.4,0.0,0.0,0.0 -101.41,0.0,0.0,0.0 -101.42,0.0,0.0,0.0 -101.43,0.0,0.0,0.0 -101.44,0.0,0.0,0.0 -101.45,0.0,0.0,0.0 -101.46,0.0,0.0,0.0 -101.47,0.0,0.0,0.0 -101.48,0.0,0.0,0.0 -101.49,0.0,0.0,0.0 -101.5,0.0,0.0,0.0 -101.51,0.0,0.0,0.0 -101.52,0.0,0.0,0.0 -101.53,0.0,0.0,0.0 -101.54,0.0,0.0,0.0 -101.55,0.0,0.0,0.0 -101.56,0.0,0.0,0.0 -101.57,0.0,0.0,0.0 -101.58,0.0,0.0,0.0 -101.59,0.0,0.0,0.0 -101.6,0.0,0.0,0.0 -101.61,0.0,0.0,0.0 -101.62,0.0,0.0,0.0 -101.63,0.0,0.0,0.0 -101.64,0.0,0.0,0.0 -101.65,0.0,0.0,0.0 -101.66,0.0,0.0,0.0 -101.67,0.0,0.0,0.0 -101.68,0.0,0.0,0.0 -101.69,0.0,0.0,0.0 -101.7,0.0,0.0,0.0 -101.71,0.0,0.0,0.0 -101.72,0.0,0.0,0.0 -101.73,0.0,0.0,0.0 -101.74,0.0,0.0,0.0 -101.75,0.0,0.0,0.0 -101.76,0.0,0.0,0.0 -101.77,0.0,0.0,0.0 -101.78,0.0,0.0,0.0 -101.79,0.0,0.0,0.0 -101.8,0.0,0.0,0.0 -101.81,0.0,0.0,0.0 -101.82,0.0,0.0,0.0 -101.83,0.0,0.0,0.0 -101.84,0.0,0.0,0.0 -101.85,0.0,0.0,0.0 -101.86,0.0,0.0,0.0 -101.87,0.0,0.0,0.0 -101.88,0.0,0.0,0.0 -101.89,0.0,0.0,0.0 -101.9,0.0,0.0,0.0 -101.91,0.0,0.0,0.0 -101.92,0.0,0.0,0.0 -101.93,0.0,0.0,0.0 -101.94,0.0,0.0,0.0 -101.95,0.0,0.0,0.0 -101.96,0.0,0.0,0.0 -101.97,0.0,0.0,0.0 -101.98,0.0,0.0,0.0 -101.99,0.0,0.0,0.0 diff --git a/packet/fixtures/synthetic/streams/heading.csv b/packet/fixtures/synthetic/streams/heading.csv deleted file mode 100644 index e510f1870..000000000 --- a/packet/fixtures/synthetic/streams/heading.csv +++ /dev/null @@ -1,2 +0,0 @@ -t,magnetic_deg,true_deg,accuracy_deg -100.5,90.0,90.0,10.0 diff --git a/packet/fixtures/synthetic/streams/location.csv b/packet/fixtures/synthetic/streams/location.csv deleted file mode 100644 index b4f4579f3..000000000 --- a/packet/fixtures/synthetic/streams/location.csv +++ /dev/null @@ -1,2 +0,0 @@ -t,latitude,longitude,altitude_m,horizontal_accuracy_m,vertical_accuracy_m -100.5,0.0,0.0,10.0,5.0,3.0 diff --git a/packet/fixtures/synthetic/streams/magnetometer.csv b/packet/fixtures/synthetic/streams/magnetometer.csv deleted file mode 100644 index c467e0a56..000000000 --- a/packet/fixtures/synthetic/streams/magnetometer.csv +++ /dev/null @@ -1,201 +0,0 @@ -t,x,y,z -100.0,20.0,-5.0,-40.0 -100.01,20.0,-5.0,-40.0 -100.02,20.0,-5.0,-40.0 -100.03,20.0,-5.0,-40.0 -100.04,20.0,-5.0,-40.0 -100.05,20.0,-5.0,-40.0 -100.06,20.0,-5.0,-40.0 -100.07,20.0,-5.0,-40.0 -100.08,20.0,-5.0,-40.0 -100.09,20.0,-5.0,-40.0 -100.1,20.0,-5.0,-40.0 -100.11,20.0,-5.0,-40.0 -100.12,20.0,-5.0,-40.0 -100.13,20.0,-5.0,-40.0 -100.14,20.0,-5.0,-40.0 -100.15,20.0,-5.0,-40.0 -100.16,20.0,-5.0,-40.0 -100.17,20.0,-5.0,-40.0 -100.18,20.0,-5.0,-40.0 -100.19,20.0,-5.0,-40.0 -100.2,20.0,-5.0,-40.0 -100.21,20.0,-5.0,-40.0 -100.22,20.0,-5.0,-40.0 -100.23,20.0,-5.0,-40.0 -100.24,20.0,-5.0,-40.0 -100.25,20.0,-5.0,-40.0 -100.26,20.0,-5.0,-40.0 -100.27,20.0,-5.0,-40.0 -100.28,20.0,-5.0,-40.0 -100.29,20.0,-5.0,-40.0 -100.3,20.0,-5.0,-40.0 -100.31,20.0,-5.0,-40.0 -100.32,20.0,-5.0,-40.0 -100.33,20.0,-5.0,-40.0 -100.34,20.0,-5.0,-40.0 -100.35,20.0,-5.0,-40.0 -100.36,20.0,-5.0,-40.0 -100.37,20.0,-5.0,-40.0 -100.38,20.0,-5.0,-40.0 -100.39,20.0,-5.0,-40.0 -100.4,20.0,-5.0,-40.0 -100.41,20.0,-5.0,-40.0 -100.42,20.0,-5.0,-40.0 -100.43,20.0,-5.0,-40.0 -100.44,20.0,-5.0,-40.0 -100.45,20.0,-5.0,-40.0 -100.46,20.0,-5.0,-40.0 -100.47,20.0,-5.0,-40.0 -100.48,20.0,-5.0,-40.0 -100.49,20.0,-5.0,-40.0 -100.5,20.0,-5.0,-40.0 -100.51,20.0,-5.0,-40.0 -100.52,20.0,-5.0,-40.0 -100.53,20.0,-5.0,-40.0 -100.54,20.0,-5.0,-40.0 -100.55,20.0,-5.0,-40.0 -100.56,20.0,-5.0,-40.0 -100.57,20.0,-5.0,-40.0 -100.58,20.0,-5.0,-40.0 -100.59,20.0,-5.0,-40.0 -100.6,20.0,-5.0,-40.0 -100.61,20.0,-5.0,-40.0 -100.62,20.0,-5.0,-40.0 -100.63,20.0,-5.0,-40.0 -100.64,20.0,-5.0,-40.0 -100.65,20.0,-5.0,-40.0 -100.66,20.0,-5.0,-40.0 -100.67,20.0,-5.0,-40.0 -100.68,20.0,-5.0,-40.0 -100.69,20.0,-5.0,-40.0 -100.7,20.0,-5.0,-40.0 -100.71,20.0,-5.0,-40.0 -100.72,20.0,-5.0,-40.0 -100.73,20.0,-5.0,-40.0 -100.74,20.0,-5.0,-40.0 -100.75,20.0,-5.0,-40.0 -100.76,20.0,-5.0,-40.0 -100.77,20.0,-5.0,-40.0 -100.78,20.0,-5.0,-40.0 -100.79,20.0,-5.0,-40.0 -100.8,20.0,-5.0,-40.0 -100.81,20.0,-5.0,-40.0 -100.82,20.0,-5.0,-40.0 -100.83,20.0,-5.0,-40.0 -100.84,20.0,-5.0,-40.0 -100.85,20.0,-5.0,-40.0 -100.86,20.0,-5.0,-40.0 -100.87,20.0,-5.0,-40.0 -100.88,20.0,-5.0,-40.0 -100.89,20.0,-5.0,-40.0 -100.9,20.0,-5.0,-40.0 -100.91,20.0,-5.0,-40.0 -100.92,20.0,-5.0,-40.0 -100.93,20.0,-5.0,-40.0 -100.94,20.0,-5.0,-40.0 -100.95,20.0,-5.0,-40.0 -100.96,20.0,-5.0,-40.0 -100.97,20.0,-5.0,-40.0 -100.98,20.0,-5.0,-40.0 -100.99,20.0,-5.0,-40.0 -101.0,20.0,-5.0,-40.0 -101.01,20.0,-5.0,-40.0 -101.02,20.0,-5.0,-40.0 -101.03,20.0,-5.0,-40.0 -101.04,20.0,-5.0,-40.0 -101.05,20.0,-5.0,-40.0 -101.06,20.0,-5.0,-40.0 -101.07,20.0,-5.0,-40.0 -101.08,20.0,-5.0,-40.0 -101.09,20.0,-5.0,-40.0 -101.1,20.0,-5.0,-40.0 -101.11,20.0,-5.0,-40.0 -101.12,20.0,-5.0,-40.0 -101.13,20.0,-5.0,-40.0 -101.14,20.0,-5.0,-40.0 -101.15,20.0,-5.0,-40.0 -101.16,20.0,-5.0,-40.0 -101.17,20.0,-5.0,-40.0 -101.18,20.0,-5.0,-40.0 -101.19,20.0,-5.0,-40.0 -101.2,20.0,-5.0,-40.0 -101.21,20.0,-5.0,-40.0 -101.22,20.0,-5.0,-40.0 -101.23,20.0,-5.0,-40.0 -101.24,20.0,-5.0,-40.0 -101.25,20.0,-5.0,-40.0 -101.26,20.0,-5.0,-40.0 -101.27,20.0,-5.0,-40.0 -101.28,20.0,-5.0,-40.0 -101.29,20.0,-5.0,-40.0 -101.3,20.0,-5.0,-40.0 -101.31,20.0,-5.0,-40.0 -101.32,20.0,-5.0,-40.0 -101.33,20.0,-5.0,-40.0 -101.34,20.0,-5.0,-40.0 -101.35,20.0,-5.0,-40.0 -101.36,20.0,-5.0,-40.0 -101.37,20.0,-5.0,-40.0 -101.38,20.0,-5.0,-40.0 -101.39,20.0,-5.0,-40.0 -101.4,20.0,-5.0,-40.0 -101.41,20.0,-5.0,-40.0 -101.42,20.0,-5.0,-40.0 -101.43,20.0,-5.0,-40.0 -101.44,20.0,-5.0,-40.0 -101.45,20.0,-5.0,-40.0 -101.46,20.0,-5.0,-40.0 -101.47,20.0,-5.0,-40.0 -101.48,20.0,-5.0,-40.0 -101.49,20.0,-5.0,-40.0 -101.5,20.0,-5.0,-40.0 -101.51,20.0,-5.0,-40.0 -101.52,20.0,-5.0,-40.0 -101.53,20.0,-5.0,-40.0 -101.54,20.0,-5.0,-40.0 -101.55,20.0,-5.0,-40.0 -101.56,20.0,-5.0,-40.0 -101.57,20.0,-5.0,-40.0 -101.58,20.0,-5.0,-40.0 -101.59,20.0,-5.0,-40.0 -101.6,20.0,-5.0,-40.0 -101.61,20.0,-5.0,-40.0 -101.62,20.0,-5.0,-40.0 -101.63,20.0,-5.0,-40.0 -101.64,20.0,-5.0,-40.0 -101.65,20.0,-5.0,-40.0 -101.66,20.0,-5.0,-40.0 -101.67,20.0,-5.0,-40.0 -101.68,20.0,-5.0,-40.0 -101.69,20.0,-5.0,-40.0 -101.7,20.0,-5.0,-40.0 -101.71,20.0,-5.0,-40.0 -101.72,20.0,-5.0,-40.0 -101.73,20.0,-5.0,-40.0 -101.74,20.0,-5.0,-40.0 -101.75,20.0,-5.0,-40.0 -101.76,20.0,-5.0,-40.0 -101.77,20.0,-5.0,-40.0 -101.78,20.0,-5.0,-40.0 -101.79,20.0,-5.0,-40.0 -101.8,20.0,-5.0,-40.0 -101.81,20.0,-5.0,-40.0 -101.82,20.0,-5.0,-40.0 -101.83,20.0,-5.0,-40.0 -101.84,20.0,-5.0,-40.0 -101.85,20.0,-5.0,-40.0 -101.86,20.0,-5.0,-40.0 -101.87,20.0,-5.0,-40.0 -101.88,20.0,-5.0,-40.0 -101.89,20.0,-5.0,-40.0 -101.9,20.0,-5.0,-40.0 -101.91,20.0,-5.0,-40.0 -101.92,20.0,-5.0,-40.0 -101.93,20.0,-5.0,-40.0 -101.94,20.0,-5.0,-40.0 -101.95,20.0,-5.0,-40.0 -101.96,20.0,-5.0,-40.0 -101.97,20.0,-5.0,-40.0 -101.98,20.0,-5.0,-40.0 -101.99,20.0,-5.0,-40.0 diff --git a/packet/fixtures/synthetic/streams/trajectory.csv b/packet/fixtures/synthetic/streams/trajectory.csv deleted file mode 100644 index 8ae4cca9c..000000000 --- a/packet/fixtures/synthetic/streams/trajectory.csv +++ /dev/null @@ -1,122 +0,0 @@ -t,tracking,px,py,pz,qx,qy,qz,qw -100.0,normal,-1.0,0.0,2.5,0.0,0.0,0.0,1.0 -100.016667,normal,-0.983333,0.0,2.5,0.0,0.0,0.0,1.0 -100.033333,normal,-0.966667,0.0,2.5,0.0,0.0,0.0,1.0 -100.05,normal,-0.95,0.0,2.5,0.0,0.0,0.0,1.0 -100.066667,normal,-0.933333,0.0,2.5,0.0,0.0,0.0,1.0 -100.083333,normal,-0.916667,0.0,2.5,0.0,0.0,0.0,1.0 -100.1,normal,-0.9,0.0,2.5,0.0,0.0,0.0,1.0 -100.116667,normal,-0.883333,0.0,2.5,0.0,0.0,0.0,1.0 -100.133333,normal,-0.866667,0.0,2.5,0.0,0.0,0.0,1.0 -100.15,normal,-0.85,0.0,2.5,0.0,0.0,0.0,1.0 -100.166667,normal,-0.833333,0.0,2.5,0.0,0.0,0.0,1.0 -100.183333,normal,-0.816667,0.0,2.5,0.0,0.0,0.0,1.0 -100.2,normal,-0.8,0.0,2.5,0.0,0.0,0.0,1.0 -100.216667,normal,-0.783333,0.0,2.5,0.0,0.0,0.0,1.0 -100.233333,normal,-0.766667,0.0,2.5,0.0,0.0,0.0,1.0 -100.25,normal,-0.75,0.0,2.5,0.0,0.0,0.0,1.0 -100.266667,normal,-0.733333,0.0,2.5,0.0,0.0,0.0,1.0 -100.283333,normal,-0.716667,0.0,2.5,0.0,0.0,0.0,1.0 -100.3,normal,-0.7,0.0,2.5,0.0,0.0,0.0,1.0 -100.316667,normal,-0.683333,0.0,2.5,0.0,0.0,0.0,1.0 -100.333333,normal,-0.666667,0.0,2.5,0.0,0.0,0.0,1.0 -100.35,normal,-0.65,0.0,2.5,0.0,0.0,0.0,1.0 -100.366667,normal,-0.633333,0.0,2.5,0.0,0.0,0.0,1.0 -100.383333,normal,-0.616667,0.0,2.5,0.0,0.0,0.0,1.0 -100.4,normal,-0.6,0.0,2.5,0.0,0.0,0.0,1.0 -100.416667,normal,-0.583333,0.0,2.5,0.0,0.0,0.0,1.0 -100.433333,normal,-0.566667,0.0,2.5,0.0,0.0,0.0,1.0 -100.45,normal,-0.55,0.0,2.5,0.0,0.0,0.0,1.0 -100.466667,normal,-0.533333,0.0,2.5,0.0,0.0,0.0,1.0 -100.483333,normal,-0.516667,0.0,2.5,0.0,0.0,0.0,1.0 -100.5,normal,-0.5,0.0,2.5,0.0,0.0,0.0,1.0 -100.516667,normal,-0.483333,0.0,2.5,0.0,0.0,0.0,1.0 -100.533333,normal,-0.466667,0.0,2.5,0.0,0.0,0.0,1.0 -100.55,normal,-0.45,0.0,2.5,0.0,0.0,0.0,1.0 -100.566667,normal,-0.433333,0.0,2.5,0.0,0.0,0.0,1.0 -100.583333,normal,-0.416667,0.0,2.5,0.0,0.0,0.0,1.0 -100.6,normal,-0.4,0.0,2.5,0.0,0.0,0.0,1.0 -100.616667,normal,-0.383333,0.0,2.5,0.0,0.0,0.0,1.0 -100.633333,normal,-0.366667,0.0,2.5,0.0,0.0,0.0,1.0 -100.65,normal,-0.35,0.0,2.5,0.0,0.0,0.0,1.0 -100.666667,normal,-0.333333,0.0,2.5,0.0,0.0,0.0,1.0 -100.683333,normal,-0.316667,0.0,2.5,0.0,0.0,0.0,1.0 -100.7,normal,-0.3,0.0,2.5,0.0,0.0,0.0,1.0 -100.716667,normal,-0.283333,0.0,2.5,0.0,0.0,0.0,1.0 -100.733333,normal,-0.266667,0.0,2.5,0.0,0.0,0.0,1.0 -100.75,normal,-0.25,0.0,2.5,0.0,0.0,0.0,1.0 -100.766667,normal,-0.233333,0.0,2.5,0.0,0.0,0.0,1.0 -100.783333,normal,-0.216667,0.0,2.5,0.0,0.0,0.0,1.0 -100.8,normal,-0.2,0.0,2.5,0.0,0.0,0.0,1.0 -100.816667,normal,-0.183333,0.0,2.5,0.0,0.0,0.0,1.0 -100.833333,normal,-0.166667,0.0,2.5,0.0,0.0,0.0,1.0 -100.85,normal,-0.15,0.0,2.5,0.0,0.0,0.0,1.0 -100.866667,normal,-0.133333,0.0,2.5,0.0,0.0,0.0,1.0 -100.883333,normal,-0.116667,0.0,2.5,0.0,0.0,0.0,1.0 -100.9,normal,-0.1,0.0,2.5,0.0,0.0,0.0,1.0 -100.916667,normal,-0.083333,0.0,2.5,0.0,0.0,0.0,1.0 -100.933333,normal,-0.066667,0.0,2.5,0.0,0.0,0.0,1.0 -100.95,normal,-0.05,0.0,2.5,0.0,0.0,0.0,1.0 -100.966667,normal,-0.033333,0.0,2.5,0.0,0.0,0.0,1.0 -100.983333,normal,-0.016667,0.0,2.5,0.0,0.0,0.0,1.0 -101.0,normal,0.0,0.0,2.5,0.0,0.0,0.0,1.0 -101.016667,normal,0.016667,0.0,2.5,0.0,0.0,0.0,1.0 -101.033333,normal,0.033333,0.0,2.5,0.0,0.0,0.0,1.0 -101.05,normal,0.05,0.0,2.5,0.0,0.0,0.0,1.0 -101.066667,normal,0.066667,0.0,2.5,0.0,0.0,0.0,1.0 -101.083333,normal,0.083333,0.0,2.5,0.0,0.0,0.0,1.0 -101.1,normal,0.1,0.0,2.5,0.0,0.0,0.0,1.0 -101.116667,normal,0.116667,0.0,2.5,0.0,0.0,0.0,1.0 -101.133333,normal,0.133333,0.0,2.5,0.0,0.0,0.0,1.0 -101.15,normal,0.15,0.0,2.5,0.0,0.0,0.0,1.0 -101.166667,normal,0.166667,0.0,2.5,0.0,0.0,0.0,1.0 -101.183333,normal,0.183333,0.0,2.5,0.0,0.0,0.0,1.0 -101.2,normal,0.2,0.0,2.5,0.0,0.0,0.0,1.0 -101.216667,normal,0.216667,0.0,2.5,0.0,0.0,0.0,1.0 -101.233333,normal,0.233333,0.0,2.5,0.0,0.0,0.0,1.0 -101.25,normal,0.25,0.0,2.5,0.0,0.0,0.0,1.0 -101.266667,normal,0.266667,0.0,2.5,0.0,0.0,0.0,1.0 -101.283333,normal,0.283333,0.0,2.5,0.0,0.0,0.0,1.0 -101.3,normal,0.3,0.0,2.5,0.0,0.0,0.0,1.0 -101.316667,normal,0.316667,0.0,2.5,0.0,0.0,0.0,1.0 -101.333333,normal,0.333333,0.0,2.5,0.0,0.0,0.0,1.0 -101.35,normal,0.35,0.0,2.5,0.0,0.0,0.0,1.0 -101.366667,normal,0.366667,0.0,2.5,0.0,0.0,0.0,1.0 -101.383333,normal,0.383333,0.0,2.5,0.0,0.0,0.0,1.0 -101.4,normal,0.4,0.0,2.5,0.0,0.0,0.0,1.0 -101.416667,normal,0.416667,0.0,2.5,0.0,0.0,0.0,1.0 -101.433333,normal,0.433333,0.0,2.5,0.0,0.0,0.0,1.0 -101.45,normal,0.45,0.0,2.5,0.0,0.0,0.0,1.0 -101.466667,normal,0.466667,0.0,2.5,0.0,0.0,0.0,1.0 -101.483333,normal,0.483333,0.0,2.5,0.0,0.0,0.0,1.0 -101.5,normal,0.5,0.0,2.5,0.0,0.0,0.0,1.0 -101.516667,normal,0.516667,0.0,2.5,0.0,0.0,0.0,1.0 -101.533333,normal,0.533333,0.0,2.5,0.0,0.0,0.0,1.0 -101.55,normal,0.55,0.0,2.5,0.0,0.0,0.0,1.0 -101.566667,normal,0.566667,0.0,2.5,0.0,0.0,0.0,1.0 -101.583333,normal,0.583333,0.0,2.5,0.0,0.0,0.0,1.0 -101.6,normal,0.6,0.0,2.5,0.0,0.0,0.0,1.0 -101.616667,normal,0.616667,0.0,2.5,0.0,0.0,0.0,1.0 -101.633333,normal,0.633333,0.0,2.5,0.0,0.0,0.0,1.0 -101.65,normal,0.65,0.0,2.5,0.0,0.0,0.0,1.0 -101.666667,normal,0.666667,0.0,2.5,0.0,0.0,0.0,1.0 -101.683333,normal,0.683333,0.0,2.5,0.0,0.0,0.0,1.0 -101.7,normal,0.7,0.0,2.5,0.0,0.0,0.0,1.0 -101.716667,normal,0.716667,0.0,2.5,0.0,0.0,0.0,1.0 -101.733333,normal,0.733333,0.0,2.5,0.0,0.0,0.0,1.0 -101.75,normal,0.75,0.0,2.5,0.0,0.0,0.0,1.0 -101.766667,normal,0.766667,0.0,2.5,0.0,0.0,0.0,1.0 -101.783333,normal,0.783333,0.0,2.5,0.0,0.0,0.0,1.0 -101.8,normal,0.8,0.0,2.5,0.0,0.0,0.0,1.0 -101.816667,normal,0.816667,0.0,2.5,0.0,0.0,0.0,1.0 -101.833333,normal,0.833333,0.0,2.5,0.0,0.0,0.0,1.0 -101.85,normal,0.85,0.0,2.5,0.0,0.0,0.0,1.0 -101.866667,normal,0.866667,0.0,2.5,0.0,0.0,0.0,1.0 -101.883333,normal,0.883333,0.0,2.5,0.0,0.0,0.0,1.0 -101.9,normal,0.9,0.0,2.5,0.0,0.0,0.0,1.0 -101.916667,normal,0.916667,0.0,2.5,0.0,0.0,0.0,1.0 -101.933333,normal,0.933333,0.0,2.5,0.0,0.0,0.0,1.0 -101.95,normal,0.95,0.0,2.5,0.0,0.0,0.0,1.0 -101.966667,normal,0.966667,0.0,2.5,0.0,0.0,0.0,1.0 -101.983333,normal,0.983333,0.0,2.5,0.0,0.0,0.0,1.0 -102.0,normal,1.0,0.0,2.5,0.0,0.0,0.0,1.0 diff --git a/packet/manifest.schema.json b/packet/manifest.schema.json deleted file mode 100644 index dd235455b..000000000 --- a/packet/manifest.schema.json +++ /dev/null @@ -1,763 +0,0 @@ -{ - "$schema": "https://json-schema.org/draft/2020-12/schema", - "$id": "https://github.com/SamGu-NRX/house-scanning-master/packet/manifest.schema.json", - "title": "Capture packet manifest", - "description": "One scan session: the files in the packet and what they mean. README.md in this folder is the full specification; this schema checks shape only. Lengths in meters, times in seconds of device uptime, poses as 16 numbers column by column.", - "type": "object", - "additionalProperties": true, - "required": [ - "packet_version", - "session", - "photos" - ], - "properties": { - "packet_version": { - "description": "MAJOR.MINOR. A reader accepts any packet with its own major version and ignores fields it does not know. 1.1 adds depth frames, estimated depth with sigma, top-level planes with boundaries, and the mesh classification flag.", - "type": "string", - "pattern": "^1\\.[0-9]+$" - }, - "session": { - "$ref": "#/$defs/session" - }, - "photos": { - "type": "array", - "minItems": 1, - "items": { - "$ref": "#/$defs/photo" - } - }, - "depth_frames": { - "type": "array", - "items": { - "$ref": "#/$defs/depth_frame" - } - }, - "streams": { - "$ref": "#/$defs/streams" - }, - "planes": { - "description": "1.1. ARPlaneAnchors, with or without LiDAR.", - "type": "array", - "items": { - "$ref": "#/$defs/plane" - } - }, - "lidar": { - "$ref": "#/$defs/lidar" - }, - "marks": { - "type": "array", - "items": { - "$ref": "#/$defs/mark" - } - }, - "guidance": { - "type": "array", - "items": { - "$ref": "#/$defs/guidance_event" - } - }, - "scene": { - "description": "The app's own scene.json (C1), exactly as it would be sent for placement.", - "allOf": [ - { - "$ref": "#/$defs/file" - } - ], - "type": "object", - "properties": { - "schema_version": { - "type": "string" - } - } - }, - "provenance": { - "$ref": "#/$defs/provenance" - } - }, - "$defs": { - "file": { - "description": "A file inside the packet. `path` is relative to the manifest, uses '/', and never leaves the packet.", - "type": "object", - "required": [ - "path", - "bytes", - "sha256" - ], - "properties": { - "path": { - "type": "string", - "minLength": 1, - "pattern": "^(?!/)(?!.*(^|/)\\.\\.(/|$))[A-Za-z0-9._/-]+$" - }, - "bytes": { - "type": "integer", - "minimum": 0 - }, - "sha256": { - "type": "string", - "pattern": "^[0-9a-f]{64}$" - } - } - }, - "pose": { - "description": "Rigid 4x4 transform, 16 numbers column by column (simd_float4x4 layout); translation in meters.", - "type": "array", - "minItems": 16, - "maxItems": 16, - "items": { - "type": "number" - } - }, - "vec3": { - "type": "array", - "minItems": 3, - "maxItems": 3, - "items": { - "type": "number" - } - }, - "time": { - "description": "Seconds of device uptime: the clock of ARFrame.timestamp and CMLogItem.timestamp.", - "type": "number", - "minimum": 0 - }, - "tracking": { - "type": "object", - "required": [ - "state" - ], - "properties": { - "state": { - "enum": [ - "normal", - "limited", - "not_available" - ] - }, - "reason": { - "enum": [ - "initializing", - "relocalizing", - "excessive_motion", - "insufficient_features", - null - ] - } - } - }, - "session": { - "type": "object", - "required": [ - "id", - "producer", - "device", - "capture", - "world_alignment", - "meter_anchor" - ], - "properties": { - "id": { - "type": "string", - "minLength": 1 - }, - "producer": { - "description": "What wrote this packet: the app, or a converter building a sample from a dataset.", - "type": "object", - "required": [ - "kind", - "name", - "version" - ], - "properties": { - "kind": { - "enum": [ - "app", - "converter" - ] - }, - "name": { - "type": "string" - }, - "version": { - "type": "string" - }, - "commit": { - "type": "string" - } - } - }, - "device": { - "type": "object", - "required": [ - "model", - "lidar" - ], - "properties": { - "model": { - "description": "Hardware identifier, e.g. iPhone16,1.", - "type": "string" - }, - "ios_version": { - "type": "string" - }, - "lidar": { - "description": "The device has a LiDAR scanner (sceneDepth supported).", - "type": "boolean" - }, - "scene_depth_enabled": { - "type": "boolean" - }, - "mesh_enabled": { - "type": "boolean" - }, - "mesh_classification_enabled": { - "type": "boolean", - "description": "1.1. ARKit's sceneReconstruction included .meshWithClassification. Required with a mesh from 1.1; when false every face's class is 0." - } - } - }, - "capture": { - "type": "object", - "required": [ - "started_at_uptime", - "ended_at_uptime" - ], - "properties": { - "started_at": { - "description": "Wall clock at started_at_uptime, ISO 8601 UTC.", - "type": "string", - "format": "date-time" - }, - "started_at_uptime": { - "$ref": "#/$defs/time" - }, - "ended_at_uptime": { - "$ref": "#/$defs/time" - }, - "distance_walked_m": { - "description": "Horizontal length of the camera trajectory. Must match streams.trajectory within 1%.", - "type": "number", - "minimum": 0 - } - } - }, - "world_alignment": { - "const": "gravity" - }, - "meter_anchor": { - "description": "The meter frame, as a pose in ARKit's world frame of this session. Every other pose and point in the packet is in the meter frame.", - "type": "object", - "required": [ - "pose_in_world" - ], - "properties": { - "pose_in_world": { - "$ref": "#/$defs/pose" - }, - "ground_y_m": { - "description": "Height of the ground at the meter, on the meter frame's y axis (negative: the ground is below the meter).", - "type": [ - "number", - "null" - ] - } - } - }, - "consent": { - "type": "object", - "properties": { - "location": { - "description": "The homeowner allowed location and heading. Required true if either stream is present.", - "type": "boolean" - } - } - } - } - }, - "photo": { - "type": "object", - "required": [ - "id", - "image", - "width", - "height", - "t", - "pose", - "intrinsics" - ], - "properties": { - "id": { - "type": "string", - "pattern": "^[A-Za-z0-9_-]+$" - }, - "image": { - "description": "The full-resolution photo as captured, JPEG, stored unrotated: landscape as the sensor delivers it, EXIF orientation absent or 1.", - "$ref": "#/$defs/file" - }, - "width": { - "type": "integer", - "minimum": 1 - }, - "height": { - "type": "integer", - "minimum": 1 - }, - "t": { - "$ref": "#/$defs/time" - }, - "pose": { - "description": "Camera to meter frame. ARKit camera axes: +x right and +y up in the stored image, the camera looks along -z.", - "$ref": "#/$defs/pose" - }, - "intrinsics": { - "description": "[fx, fy, cx, cy] in pixels of this stored image; (0, 0) is the top-left corner of the top-left pixel.", - "type": "array", - "minItems": 4, - "maxItems": 4, - "items": { - "type": "number" - } - }, - "tracking": { - "$ref": "#/$defs/tracking" - }, - "exposure": { - "type": "object", - "properties": { - "duration_s": { - "type": "number", - "exclusiveMinimum": 0 - }, - "iso": { - "type": "number", - "exclusiveMinimum": 0 - }, - "offset_ev": { - "type": "number" - } - } - }, - "lens": { - "type": "object", - "properties": { - "focal_length_mm": { - "type": "number", - "exclusiveMinimum": 0 - }, - "f_number": { - "type": "number", - "exclusiveMinimum": 0 - }, - "camera": { - "description": "Which camera: wide, ultra_wide, telephoto, or a dataset's camera name.", - "type": "string" - } - } - }, - "sharpness": { - "type": "object", - "required": [ - "method", - "value" - ], - "properties": { - "method": { - "const": "laplacian_variance_luma_640" - }, - "value": { - "type": "number", - "minimum": 0 - } - } - }, - "depth": { - "$ref": "#/$defs/depth" - } - } - }, - "depth": { - "description": "Depth aligned to an image: float32 little-endian meters along the camera's -z, row by row from the top; 0 means no measurement. For a photo it covers the photo's field of view at the same aspect; for a depth frame its own intrinsics describe it.", - "type": "object", - "required": [ - "map", - "width", - "height" - ], - "properties": { - "map": { - "$ref": "#/$defs/file" - }, - "confidence": { - "description": "UInt8 per depth pixel, ARConfidenceLevel: 0 low, 1 medium, 2 high.", - "$ref": "#/$defs/file" - }, - "width": { - "type": "integer", - "minimum": 1 - }, - "height": { - "type": "integer", - "minimum": 1 - }, - "source": { - "enum": [ - "arkit_scene_depth", - "arkit_smoothed_scene_depth", - "estimated", - "rendered_from_laser_scan" - ], - "description": "estimated: metric depth inferred from the image (no LiDAR); needs sigma. From 1.1, the ARKit sources need confidence." - }, - "sigma": { - "description": "1.1. Float32 little-endian meters, one standard deviation per depth pixel, same layout as map. Required when source is estimated.", - "$ref": "#/$defs/file" - } - }, - "allOf": [ - { - "if": { - "properties": { - "source": { - "const": "estimated" - } - }, - "required": [ - "source" - ] - }, - "then": { - "required": [ - "sigma" - ] - } - } - ] - }, - "stream": { - "allOf": [ - { - "$ref": "#/$defs/file" - } - ], - "type": "object", - "required": [ - "rows" - ], - "properties": { - "rows": { - "type": "integer", - "minimum": 0 - }, - "nominal_rate_hz": { - "type": "number", - "exclusiveMinimum": 0 - } - } - }, - "streams": { - "description": "CSV files with a header row; the first column is always t. Columns are fixed per stream (README, Streams).", - "type": "object", - "properties": { - "trajectory": { - "$ref": "#/$defs/stream" - }, - "accelerometer": { - "$ref": "#/$defs/stream" - }, - "gyroscope": { - "$ref": "#/$defs/stream" - }, - "magnetometer": { - "$ref": "#/$defs/stream" - }, - "device_motion": { - "$ref": "#/$defs/stream" - }, - "barometer": { - "$ref": "#/$defs/stream" - }, - "location": { - "$ref": "#/$defs/stream" - }, - "heading": { - "$ref": "#/$defs/stream" - } - }, - "additionalProperties": false - }, - "lidar": { - "type": "object", - "properties": { - "mesh": { - "description": "Binary little-endian PLY: float x, y, z per vertex in the meter frame; per face a vertex index list and uchar classification (ARMeshClassification raw value 0 to 7).", - "$ref": "#/$defs/file" - }, - "planes": { - "description": "Deprecated in 1.1: use top-level planes. Still read in 1.x; a packet may not use both.", - "type": "array", - "items": { - "$ref": "#/$defs/plane" - } - } - } - }, - "mark": { - "type": "object", - "required": [ - "id", - "kind", - "points" - ], - "properties": { - "id": { - "type": "string" - }, - "kind": { - "enum": [ - "meter", - "wall_end", - "gas_meter", - "ac", - "door", - "window", - "garage_door", - "drive_edge", - "fence" - ] - }, - "points": { - "description": "Meter frame, meters. One point for the meter, a wall end, a gas meter or an AC unit; two for an opening (bottom-left, top-right corners), a drive edge or a fence.", - "type": "array", - "minItems": 1, - "items": { - "$ref": "#/$defs/vec3" - } - }, - "t": { - "$ref": "#/$defs/time" - }, - "photo_ids": { - "type": "array", - "items": { - "type": "string" - } - }, - "side": { - "enum": [ - "left", - "right" - ] - }, - "end_kind": { - "description": "For a wall end: limit (something blocks it) or unexplored (it turns a corner, or no answer).", - "enum": [ - "limit", - "unexplored" - ] - }, - "attrs": { - "type": "object", - "properties": { - "operable": { - "type": "boolean" - } - } - } - } - }, - "guidance_event": { - "type": "object", - "required": [ - "id", - "kind", - "origin", - "t_shown", - "outcome" - ], - "properties": { - "id": { - "type": "string" - }, - "kind": { - "enum": [ - "walk", - "tilt_to_ground", - "step_back", - "mark_end", - "closeup", - "gap_band", - "gap_past_end" - ] - }, - "origin": { - "enum": [ - "phone", - "server" - ] - }, - "message": { - "type": "string" - }, - "band": { - "enum": [ - "wall", - "ground", - "overhead", - "facing", - null - ] - }, - "span_m": { - "description": "Stretch along the wall, meter frame x, meters.", - "type": [ - "array", - "null" - ], - "minItems": 2, - "maxItems": 2, - "items": { - "type": "number" - } - }, - "t_shown": { - "$ref": "#/$defs/time" - }, - "t_resolved": { - "type": [ - "number", - "null" - ] - }, - "outcome": { - "enum": [ - "met", - "skipped", - "cannot_reach", - "superseded", - "unresolved" - ] - } - } - }, - "provenance": { - "description": "Where a converted sample's data came from, and what it does not contain.", - "type": "object", - "properties": { - "dataset": { - "type": "string" - }, - "license": { - "type": "string" - }, - "source": { - "type": "string" - }, - "notes": { - "type": "array", - "items": { - "type": "string" - } - } - } - }, - "plane": { - "type": "object", - "required": [ - "id", - "alignment", - "pose", - "extent_m" - ], - "properties": { - "id": { - "type": "string" - }, - "alignment": { - "enum": [ - "horizontal", - "vertical" - ] - }, - "classification": { - "enum": [ - "none", - "wall", - "floor", - "ceiling", - "table", - "seat", - "window", - "door" - ] - }, - "pose": { - "description": "Plane to meter frame, including the extent's centre and planeExtent.rotationOnYAxis, so extent_m is a rectangle centred on this pose. The plane is its local x-z plane, +y its normal.", - "$ref": "#/$defs/pose" - }, - "extent_m": { - "type": "array", - "minItems": 2, - "maxItems": 2, - "items": { - "type": "number", - "minimum": 0 - } - }, - "boundary_m": { - "description": "1.1. ARKit's geometry.boundaryVertices as [x, z] in this pose's frame (moved by the centre and rotation, like extent_m).", - "type": "array", - "minItems": 3, - "items": { - "type": "array", - "minItems": 2, - "maxItems": 2, - "items": { - "type": "number" - } - } - } - } - }, - "depth_frame": { - "description": "1.1. Depth recorded between photos, with no image. Intrinsics are for the depth map's own pixel grid.", - "allOf": [ - { - "$ref": "#/$defs/depth" - } - ], - "type": "object", - "required": [ - "id", - "t", - "pose", - "intrinsics" - ], - "properties": { - "id": { - "type": "string", - "pattern": "^[A-Za-z0-9_-]+$" - }, - "t": { - "$ref": "#/$defs/time" - }, - "pose": { - "description": "Camera to meter frame, ARKit camera axes.", - "$ref": "#/$defs/pose" - }, - "intrinsics": { - "description": "[fx, fy, cx, cy] in pixels of the depth map.", - "type": "array", - "minItems": 4, - "maxItems": 4, - "items": { - "type": "number" - } - }, - "tracking": { - "$ref": "#/$defs/tracking" - } - } - } - } -} diff --git a/packet/packet/__init__.py b/packet/packet/__init__.py deleted file mode 100644 index ea6b38292..000000000 --- a/packet/packet/__init__.py +++ /dev/null @@ -1 +0,0 @@ -"""The capture packet: one scan session's photos, poses and sensor streams (README.md).""" diff --git a/packet/packet/__main__.py b/packet/packet/__main__.py deleted file mode 100644 index 063e9e66c..000000000 --- a/packet/packet/__main__.py +++ /dev/null @@ -1,46 +0,0 @@ -"""python -m packet validate [--json]""" - -from __future__ import annotations - -import argparse -import json -import sys -from pathlib import Path - -from packet.validate import validate - - -def main(argv: list[str] | None = None) -> int: - parser = argparse.ArgumentParser(prog="python -m packet") - sub = parser.add_subparsers(dest="command", required=True) - check = sub.add_parser("validate", help="check a packet folder or zip against the spec") - check.add_argument("packet", type=Path) - check.add_argument("--json", action="store_true", help="print the report as JSON") - args = parser.parse_args(argv) - - report = validate(args.packet) - if args.json: - print( - json.dumps( - { - "ok": report.ok, - "problems": report.problems, - "warnings": report.warnings, - "summary": report.summary, - }, - indent=2, - ) - ) - else: - print(f"{args.packet}: {'valid' if report.ok else 'INVALID'}") - for key, value in report.summary.items(): - print(f" {key}: {value}") - for w in report.warnings: - print(f" warning: {w}") - for p in report.problems: - print(f" problem: {p}") - return 0 if report.ok else 1 - - -if __name__ == "__main__": - sys.exit(main()) diff --git a/packet/packet/files.py b/packet/packet/files.py deleted file mode 100644 index e9b79585d..000000000 --- a/packet/packet/files.py +++ /dev/null @@ -1,103 +0,0 @@ -"""Read a packet from a folder or a zip through one interface. - -A zip may hold the packet at its root or inside one top-level folder. Entry names that are -absolute or climb out with '..' are refused before anything is read, because a packet comes from -a phone the server does not control. -""" - -from __future__ import annotations - -import hashlib -import zipfile -from pathlib import Path, PurePosixPath - - -class PacketError(Exception): - """The packet cannot be opened at all (no manifest, unsafe zip).""" - - -def safe_relative(path: str) -> bool: - p = PurePosixPath(path) - return bool(path) and not p.is_absolute() and ".." not in p.parts and "\\" not in path - - -class PacketFiles: - def __init__(self, source: Path): - self.source = source - if source.is_dir(): - self._zip = None - self._root = source - self._prefix = "" - if not (source / "manifest.json").is_file(): - raise PacketError(f"{source} has no manifest.json") - return - if not zipfile.is_zipfile(source): - raise PacketError(f"{source} is neither a folder nor a zip") - self._zip = zipfile.ZipFile(source) - self._root = None - names = [i.filename for i in self._zip.infolist() if not i.is_dir()] - unsafe = [n for n in names if not safe_relative(n)] - if unsafe: - raise PacketError(f"zip entry names leave the packet: {unsafe[:3]}") - manifests = [n for n in names if PurePosixPath(n).name == "manifest.json"] - top = [n for n in manifests if len(PurePosixPath(n).parts) <= 2] - if len(top) != 1: - raise PacketError( - f"zip must hold exactly one manifest.json at its root or in one top folder, " - f"found {manifests[:3] or 'none'}" - ) - parent = str(PurePosixPath(top[0]).parent) - self._prefix = "" if parent == "." else parent + "/" - - def names(self) -> set[str]: - """Every file in the packet, as manifest-relative paths.""" - if self._zip is None: - return { - p.relative_to(self._root).as_posix() for p in self._root.rglob("*") if p.is_file() - } - return { - i.filename[len(self._prefix) :] - for i in self._zip.infolist() - if not i.is_dir() and i.filename.startswith(self._prefix) - } - - def exists(self, path: str) -> bool: - if not safe_relative(path): - return False - if self._zip is None: - return (self._root / path).is_file() - try: - self._zip.getinfo(self._prefix + path) - except KeyError: - return False - return True - - def size(self, path: str) -> int: - if self._zip is None: - return (self._root / path).stat().st_size - return self._zip.getinfo(self._prefix + path).file_size - - def read(self, path: str) -> bytes: - if not safe_relative(path): - raise PacketError(f"unsafe path {path!r}") - if self._zip is None: - return (self._root / path).read_bytes() - return self._zip.read(self._prefix + path) - - def sha256(self, path: str) -> str: - digest = hashlib.sha256() - if self._zip is None: - with (self._root / path).open("rb") as f: - for chunk in iter(lambda: f.read(1 << 20), b""): - digest.update(chunk) - else: - with self._zip.open(self._prefix + path) as f: - for chunk in iter(lambda: f.read(1 << 20), b""): - digest.update(chunk) - return digest.hexdigest() - - def open(self, path: str): - """A binary file object, for readers that stream (images, large CSVs).""" - if self._zip is None: - return (self._root / path).open("rb") - return self._zip.open(self._prefix + path) diff --git a/packet/packet/samples/__init__.py b/packet/packet/samples/__init__.py deleted file mode 100644 index 8a4455c53..000000000 --- a/packet/packet/samples/__init__.py +++ /dev/null @@ -1 +0,0 @@ -"""Builders for sample packets: a synthetic fixture and conversions of public datasets.""" diff --git a/packet/packet/samples/advio.py b/packet/packet/samples/advio.py deleted file mode 100644 index 87c05cd30..000000000 --- a/packet/packet/samples/advio.py +++ /dev/null @@ -1,209 +0,0 @@ -"""A sample packet from ADVIO sequence 20 (outdoor, iPhone 6s): photos, 60 Hz ARKit trajectory, -and the phone's own accelerometer, gyroscope, magnetometer, barometer and location streams. - -The photos are the evals lane's replay keyframes (`experiments/evals/evals/replay.py` on -t3/evals): ADVIO video frames at their native 1280 x 720, undistorted to the pinhole model with -ADVIO's calibration. Poses and streams come straight from the sequence's CSVs over the same window. - - uv run python -m packet.samples.advio \ - --replay ~/house-scanning-data/replays/advio-20-0040-0075 \ - --advio ~/house-scanning-data/evals/advio/advio-20 \ - --out ~/house-scanning-data/packets/advio-20-0040-0075 -""" - -from __future__ import annotations - -import argparse -import csv -import json -from pathlib import Path - -import numpy as np -from PIL import Image - -from packet.validate import horizontal_distance -from packet.write import ( - PACKET_VERSION, - SHARPNESS_METHOD, - PacketWriter, - column_major, - from_column_major, - meter_frame, - sharpness, - trajectory_row, -) - -# ADVIO's arkit.csv holds the portrait device orientation; the landscape camera the images come -# from is that frame turned about z (image right = device -y, image up = device +x). Same matrix -# as DEVICE_TO_LANDSCAPE_CAMERA in experiments/evals/evals/camera.py, which the replay used. -DEVICE_TO_LANDSCAPE_CAMERA = np.array([[0.0, 1.0, 0.0], [-1.0, 0.0, 0.0], [0.0, 0.0, 1.0]]) -# The app's replay assumes the wall this far to the side the camera faces (ScanKit replay log). -WALL_OFFSET_M = 2.25 -# ADVIO's replay keyframes match arkit.csv rows exactly; allow float32-level rounding only. -POSE_MATCH_TOL = 1e-4 - - -def quat_wxyz_to_matrix(q: np.ndarray) -> np.ndarray: - q = q / np.linalg.norm(q, axis=-1, keepdims=True) - w, x, y, z = q[..., 0], q[..., 1], q[..., 2], q[..., 3] - r = np.empty((*q.shape[:-1], 3, 3)) - r[..., 0, 0] = 1 - 2 * (y * y + z * z) - r[..., 0, 1] = 2 * (x * y - w * z) - r[..., 0, 2] = 2 * (x * z + w * y) - r[..., 1, 0] = 2 * (x * y + w * z) - r[..., 1, 1] = 1 - 2 * (x * x + z * z) - r[..., 1, 2] = 2 * (y * z - w * x) - r[..., 2, 0] = 2 * (x * z - w * y) - r[..., 2, 1] = 2 * (y * z + w * x) - r[..., 2, 2] = 1 - 2 * (x * x + y * y) - return r - - -def read_csv(path: Path) -> np.ndarray: - with path.open() as f: - return np.array([[float(v) for v in row] for row in csv.reader(f) if row]) - - -def world_poses(arkit: np.ndarray) -> np.ndarray: - """Camera-to-world poses (ARKit camera axes, landscape image) from arkit.csv rows.""" - r = quat_wxyz_to_matrix(arkit[:, 4:8]) - poses = np.tile(np.eye(4), (len(arkit), 1, 1)) - poses[:, :3, :3] = r @ DEVICE_TO_LANDSCAPE_CAMERA - poses[:, :3, 3] = arkit[:, 1:4] - return poses - - -def assumed_meter(poses: np.ndarray) -> np.ndarray: - """Meter-to-world pose by the app replay's rule: the wall is vertical, parallel to the walk, - WALL_OFFSET_M to the side the camera faces; the meter is level with the camera, opposite the - middle of the walk.""" - pos = poses[:, :3, 3] - horizontal = pos[:, [0, 2]] - pos[:, [0, 2]].mean(axis=0) - along2 = np.linalg.svd(horizontal, full_matrices=False)[2][0] - along = np.array([along2[0], 0.0, along2[1]]) - looking = -poses[:, :3, 2].mean(axis=0) # cameras look along their -z - side = looking - along * (looking @ along) - side[1] = 0 - side /= np.linalg.norm(side) - middle = pos[len(pos) // 2] - meter = middle + WALL_OFFSET_M * side - # The meter frame's +z points out of the wall, toward the camera: -side. Its +x is then - # y x z; flip `along` if needed so the frame is right-handed with that z. - if np.cross([0.0, 1.0, 0.0], -side) @ along < 0: - along = -along - return meter_frame(meter, along) - - -def build(replay: Path, advio: Path, out: Path) -> Path: - session = json.loads((replay / "session.json").read_text()) - start, end = session["provenance"]["secondsInSequence"] - arkit = read_csv(advio / "iphone" / "arkit.csv") - keep = (arkit[:, 0] >= start) & (arkit[:, 0] <= end) & (np.abs(arkit[:, 1:4]).sum(axis=1) > 0) - arkit = arkit[keep] - world = world_poses(arkit) - anchor = assumed_meter(world) - to_meter = np.linalg.inv(anchor) - in_meter = to_meter @ world - - w = PacketWriter(out) - photos = [] - for kf in session["keyframes"]: - i = int(np.abs(arkit[:, 0] - kf["timestamp"]).argmin()) - replay_pose = from_column_major(kf["pose"]) - if np.abs(replay_pose - world[i]).max() > POSE_MATCH_TOL: - raise ValueError(f"{kf['id']}: replay pose differs from arkit.csv at t={arkit[i, 0]}") - pid = f"p{int(kf['id'][1:]):05d}" - source = replay / kf["img"] - with Image.open(source) as img: - score = sharpness(img) - photos.append( - { - "id": pid, - "image": w.add_file(f"photos/{pid}.jpg", source), - "width": kf["w"], - "height": kf["h"], - "t": float(arkit[i, 0]), - "pose": column_major(in_meter[i]), - "intrinsics": kf["intrinsics"], - "tracking": {"state": "normal", "reason": None}, - "lens": {"camera": "wide"}, - "sharpness": {"method": SHARPNESS_METHOD, "value": round(score, 3)}, - } - ) - - trajectory = [ - trajectory_row(t, "normal", p) for t, p in zip(arkit[:, 0], in_meter, strict=True) - ] - streams = {"trajectory": w.add_stream("trajectory", trajectory, 60)} - notes = [] - for name, file, rate in [ - ("accelerometer", "accelerometer.csv", 100), - ("gyroscope", "gyro.csv", 100), - ("magnetometer", "magnetometer.csv", 100), - ("barometer", "barometer.csv", 1), - ("location", "platform-locations.csv", 1), - ]: - data = read_csv(advio / "iphone" / file) - data = data[(data[:, 0] >= start) & (data[:, 0] <= end)] - repeats = np.flatnonzero(np.diff(data[:, 0]) <= 0) + 1 - if len(repeats): - notes.append(f"{file}: dropped {len(repeats)} rows whose time did not advance") - data = np.delete(data, repeats, axis=0) - rows = [[round(float(v), 6) for v in r] for r in data] - streams[name] = w.add_stream(name, rows, rate) - - manifest = { - "packet_version": PACKET_VERSION, - "session": { - "id": session["session"]["id"], - "producer": {"kind": "converter", "name": "packet.samples.advio", "version": "1"}, - "device": {"model": "iPhone8,1", "ios_version": "11 (ADVIO 2018)", "lidar": False}, - "capture": { - "started_at_uptime": float(start), - "ended_at_uptime": float(end), - "distance_walked_m": round(horizontal_distance(in_meter[:, :3, 3]), 4), - }, - "world_alignment": "gravity", - "meter_anchor": {"pose_in_world": column_major(anchor), "ground_y_m": None}, - "consent": {"location": True}, - }, - "photos": photos, - "streams": streams, - "marks": [ - {"id": "m1", "kind": "meter", "points": [[0.0, 0.0, 0.0]], "t": photos[0]["t"]}, - ], - "provenance": { - "dataset": "ADVIO sequence 20 (Cortés, Solin, Rahtu, Kannala; ECCV 2018)", - "license": "CC BY-NC 4.0: for accuracy testing only; never commit or redistribute", - "source": "https://zenodo.org/record/1476931/files/advio-20.zip", - "notes": [ - "t is ADVIO's own clock (seconds from the start of its recording), which every " - "ADVIO file shares; it stands in for device uptime.", - f"Photos are the evals replay's keyframes ({session['provenance']['keyframeRule']}" - "), undistorted to pinhole; ADVIO's video is 1280 x 720, so this is its full " - "resolution.", - "There is no meter or wall in ADVIO. The meter frame follows the app replay's " - f"assumption: a vertical wall parallel to the walk, {WALL_OFFSET_M} m to the side " - "the camera faces, the meter level with the camera opposite the walk's middle.", - "Streams are in ADVIO's units: accelerometer m/s^2, gyroscope rad/s, magnetometer " - "microtesla, barometer kPa and meters. Location was recorded by the dataset " - "authors and published with it.", - "No exposure, ISO, depth, mesh, guidance or scene.json: ADVIO has none of them.", - *notes, - ], - }, - } - return w.finish(manifest) - - -def main() -> None: - parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0]) - parser.add_argument("--replay", type=Path, required=True) - parser.add_argument("--advio", type=Path, required=True) - parser.add_argument("--out", type=Path, required=True) - args = parser.parse_args() - print(build(args.replay.expanduser(), args.advio.expanduser(), args.out.expanduser())) - - -if __name__ == "__main__": - main() diff --git a/packet/packet/samples/app_export.py b/packet/packet/samples/app_export.py deleted file mode 100644 index 748a31f48..000000000 --- a/packet/packet/samples/app_export.py +++ /dev/null @@ -1,191 +0,0 @@ -"""A sample packet from the iOS app's own export (scan.zip) of a Simulator run on the ADVIO -replay, so the server team sees exactly what the app records today, in the packet's layout. - -The app's export is scene.json (C1, feet) plus one JPEG per keyframe with its pose and -intrinsics. It has no timestamps; each photo's time is recovered by matching its pixels to the -replay keyframe it was taken from. Everything else the packet can hold is absent, and -provenance.notes lists it. - - uv run python -m packet.samples.app_export \ - --sim-report ~/house-scanning-data/reports/sim/20260926-071459-t3-ios-mvf-a39d0a50-replay \ - --replay ~/house-scanning-data/replays/advio-20-0040-0075 \ - --out ~/house-scanning-data/packets/app-export-a39d0a5 -""" - -from __future__ import annotations - -import argparse -import io -import json -import zipfile -from pathlib import Path - -import numpy as np -from PIL import Image - -from packet.write import ( - PACKET_VERSION, - SHARPNESS_METHOD, - PacketWriter, - column_major, - from_column_major, - meter_frame, - orthonormalize, - sharpness, -) - -FEET = 0.3048 -MATCH_SIZE = (64, 36) -# The app re-encodes the replay's JPEGs; a true match differs by JPEG noise only (mean absolute -# difference of a few grey levels on a 64 x 36 thumbnail). Anything above this is not a match. -MATCH_MAX_DIFF = 6.0 -# scene.json object types that are also packet mark kinds (fences and drives are not objects). -MARK_OBJECTS = {"gas_meter", "ac", "door", "window", "garage_door"} - - -def thumbnail(data: bytes | Path) -> np.ndarray: - with Image.open(io.BytesIO(data) if isinstance(data, bytes) else data) as img: - return np.asarray(img.convert("L").resize(MATCH_SIZE, Image.Resampling.BILINEAR), float) - - -def build(sim_report: Path, replay: Path, out: Path) -> Path: - report = json.loads((sim_report / "report.json").read_text()) - bundle = zipfile.ZipFile(sim_report / report["app_export"]) - scene_bytes = bundle.read("scene.json") - scene = json.loads(scene_bytes) - - session = json.loads((replay / "session.json").read_text()) - replay_thumbs = [ - (kf["timestamp"], thumbnail(replay / kf["img"])) for kf in session["keyframes"] - ] - - meter_ft = np.asarray(scene["meter"]["pos"], float) - wall = next(w for w in scene["walls"] if w["id"] == scene["meter"]["wall_id"]) - (x0, z0), (x1, z1) = wall["baseline"][0], wall["baseline"][-1] - along = np.array([x1 - x0, 0.0, z1 - z0]) - anchor = meter_frame(meter_ft * FEET, along) - to_meter = np.linalg.inv(anchor) - - w = PacketWriter(out) - photos = [] - for kf in scene["keyframes"]: - data = bundle.read(kf["img"]) - thumb = thumbnail(data) - diffs = [float(np.abs(thumb - t).mean()) for _, t in replay_thumbs] - best = int(np.argmin(diffs)) - if diffs[best] > MATCH_MAX_DIFF: - raise ValueError( - f"{kf['img']}: no replay keyframe matches (best diff {diffs[best]:.1f})" - ) - pose = from_column_major(kf["pose"]) - pose[:3, 3] *= FEET - pose = orthonormalize(to_meter @ pose) # scene.json rounds rotations to 4 decimals - pid = f"p{int(kf['id'][1:]):05d}" - photos.append( - { - "id": pid, - "image": w.add_bytes(f"photos/{pid}.jpg", data), - "width": kf["w"], - "height": kf["h"], - "t": replay_thumbs[best][0], - "pose": column_major(pose), - "intrinsics": kf["intrinsics"], - "sharpness": { - "method": SHARPNESS_METHOD, - "value": round(sharpness(Image.open(io.BytesIO(data))), 3), - }, - } - ) - photos.sort(key=lambda p: p["t"]) - times = [p["t"] for p in photos] - if len(set(times)) != len(times): - raise ValueError("two app photos matched the same replay keyframe") - camera_z = np.median([p["pose"][14] for p in photos]) - if camera_z <= 0: - raise ValueError( - f"cameras sit behind the wall (median z {camera_z:.2f} m): wall direction?" - ) - - def on_wall(s_ft: float, height_ft: float) -> list[float]: - """A point on the wall at s (along it) and a height above the ground, in the meter frame.""" - return [round(s_ft * FEET, 4), round((height_ft - meter_ft[1]) * FEET, 4), 0.0] - - marks = [{"id": "meter", "kind": "meter", "points": [[0.0, 0.0, 0.0]]}] - for i, obj in enumerate(scene.get("objects", [])): - kind = obj["type"] - if kind not in MARK_OBJECTS: - continue - s0, s1 = obj["span_ft"] - bottom, top = obj.get("bottom_ft", 0.0), obj.get("top_ft", obj.get("bottom_ft", 0.0)) - points = ( - [on_wall((s0 + s1) / 2, (bottom + top) / 2)] - if kind in ("gas_meter", "ac") - else [on_wall(s0, bottom), on_wall(s1, top)] - ) - mark = {"id": f"object{i}", "kind": kind, "points": points} - if "operable" in obj.get("attrs", {}): - mark["attrs"] = {"operable": obj["attrs"]["operable"]} - marks.append(mark) - ends = scene.get("coverage", {}).get("ends", {}) - for side, (ex, ez) in (("left", (x0, z0)), ("right", (x1, z1))): - if side in ends: - s = (np.array([ex, 0.0, ez]) - meter_ft * [1, 0, 1]) @ anchor[:3, 0] - marks.append( - { - "id": f"end-{side}", - "kind": "wall_end", - "points": [on_wall(float(s), meter_ft[1])], - "side": side, - "end_kind": ends[side]["kind"], - } - ) - - manifest = { - "packet_version": PACKET_VERSION, - "session": { - "id": f"app-export-{report['sha'][:7]}", - "producer": {"kind": "converter", "name": "packet.samples.app_export", "version": "1"}, - "device": {"model": "iOS Simulator", "lidar": False}, - "capture": {"started_at_uptime": times[0], "ended_at_uptime": times[-1]}, - "world_alignment": "gravity", - "meter_anchor": { - "pose_in_world": column_major(anchor), - "ground_y_m": round(-meter_ft[1] * FEET, 4), - }, - }, - "photos": photos, - "marks": marks, - "scene": w.add_bytes("scene.json", scene_bytes) - | {"schema_version": scene["schema_version"]}, - "provenance": { - "dataset": "ADVIO sequence 20 via the app's replay", - "license": "CC BY-NC 4.0 (ADVIO): accuracy testing only; never commit or share", - "source": f"t3/ios-mvf {report['sha'][:7]}, Simulator run {sim_report.name}, scan.zip", - "notes": [ - "Photos, poses, intrinsics, marks and scene.json are the app's; the JPEGs are " - "unchanged. Poses are converted from scene.json's feet to meters and into the " - "meter frame, with rotations re-orthonormalized after scene.json's rounding.", - "pose_in_world is in scene.json's frame: ARKit's world moved so the ground at the " - "wall is y = 0. The app does not export the offset to ARKit's own origin.", - "t is recovered: each photo matched to the replay keyframe it came from. The " - "app exports no timestamps.", - "Absent because the app does not record them yet: device model and iOS version, " - "wall-clock start, tracking state, exposure, ISO, lens, depth, trajectory, IMU, " - "distance walked, mesh, planes, mark times and the guidance log.", - ], - }, - } - return w.finish(manifest) - - -def main() -> None: - parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0]) - parser.add_argument("--sim-report", type=Path, required=True) - parser.add_argument("--replay", type=Path, required=True) - parser.add_argument("--out", type=Path, required=True) - args = parser.parse_args() - print(build(args.sim_report.expanduser(), args.replay.expanduser(), args.out.expanduser())) - - -if __name__ == "__main__": - main() diff --git a/packet/packet/samples/eth3d.py b/packet/packet/samples/eth3d.py deleted file mode 100644 index b4b81d329..000000000 --- a/packet/packet/samples/eth3d.py +++ /dev/null @@ -1,235 +0,0 @@ -"""A sample packet from ETH3D's electro scene: full-resolution DSLR photos of building walls -within about 6 m, each with a depth map rendered from the terrestrial laser scan as a stand-in -for the depth a LiDAR iPhone records. - -Inputs are the evals lane's extracted scene (scan points, COLMAP calibration) plus the original -undistorted photos from electro_dslr_undistorted.7z (sha256 in experiments/evals/README.md on -t3/evals), unpacked to /full/. - - uv run python -m packet.samples.eth3d --eth3d ~/house-scanning-data/evals/eth3d/electro \ - --out ~/house-scanning-data/packets/eth3d-electro -""" - -from __future__ import annotations - -import argparse -import math -from pathlib import Path - -import numpy as np -from PIL import Image - -from packet.write import ( - PACKET_VERSION, - SHARPNESS_METHOD, - PacketWriter, - column_major, - meter_frame, - sharpness, -) - -PHOTOS = [f"DSC_{n}" for n in range(9257, 9265)] # one walk past the wall, in shooting order -# ARKit's depth map is 256 x 192 for a 1920 x 1440 image (1/7.5); a 1/16 map of a 24 MP photo -# is 387 x 258, the same order of pixel count, so the sample exercises the same alignment rule. -DEPTH_SCALE = 16 -NEAR_M = 0.2 -WALL_RADIUS_M = 0.25 -OPENCV_TO_ARKIT_CAMERA = np.diag([1.0, -1.0, -1.0, 1.0]) - - -def read_cameras(path: Path) -> dict[int, tuple[int, int, float, float, float, float]]: - cams = {} - for line in path.read_text().splitlines(): - if line.startswith("#") or not line.strip(): - continue - cid, model, w, h, *params = line.split() - if model != "PINHOLE": - raise ValueError(f"camera {cid} is {model}; the undistorted set should be PINHOLE") - cams[int(cid)] = (int(w), int(h), *map(float, params)) - return cams - - -def read_images(path: Path) -> dict[str, tuple[np.ndarray, int]]: - """Image name -> (world-to-camera 4x4 in OpenCV axes, camera id). COLMAP lists two lines per - image; the second holds 2-D points and is skipped.""" - out = {} - lines = [ln for ln in path.read_text().splitlines() if not ln.startswith("#")] - for line in lines[::2]: - f = line.split() - qw, qx, qy, qz, tx, ty, tz = map(float, f[1:8]) - r = quaternion_wxyz(qw, qx, qy, qz) - m = np.eye(4) - m[:3, :3], m[:3, 3] = r, [tx, ty, tz] - out[Path(f[9]).stem] = (m, int(f[8])) - return out - - -def quaternion_wxyz(w: float, x: float, y: float, z: float) -> np.ndarray: - n = math.sqrt(w * w + x * x + y * y + z * z) - w, x, y, z = w / n, x / n, y / n, z / n - return np.array( - [ - [1 - 2 * (y * y + z * z), 2 * (x * y - w * z), 2 * (x * z + w * y)], - [2 * (x * y + w * z), 1 - 2 * (x * x + z * z), 2 * (y * z - w * x)], - [2 * (x * z - w * y), 2 * (y * z + w * x), 1 - 2 * (x * x + y * y)], - ] - ) - - -def gravity_world(normals: np.ndarray) -> tuple[np.ndarray, float]: - """World-from-scan rotation with +y up, and the scan's tilt in degrees. The scan is levelled - with z roughly up; ground and roof normals (within 18° of z) give the exact up direction.""" - flat = normals[np.abs(normals[:, 2]) > 0.95] - up = (flat * np.sign(flat[:, 2:3])).mean(axis=0) - up /= np.linalg.norm(up) - swap = np.array([[1.0, 0, 0], [0, 0, 1], [0, -1, 0]]) # scan z -> world y, right-handed - u = swap @ up - axis = np.cross(u, [0.0, 1.0, 0.0]) - s, c = np.linalg.norm(axis), u[1] - if s < 1e-12: - level = np.eye(3) - else: - k = axis / s - kx = np.array([[0, -k[2], k[1]], [k[2], 0, -k[0]], [-k[1], k[0], 0]]) - level = np.eye(3) + s * kx + (1 - c) * kx @ kx - m = np.eye(4) - m[:3, :3] = level @ swap - return m, math.degrees(math.acos(min(1.0, abs(up[2])))) - - -def render_depth( - points: np.ndarray, cam_from_scan: np.ndarray, k: tuple, size: tuple -) -> np.ndarray: - """Nearest scan depth per pixel of a map `size` = (w, h) covering the photo's field of view.""" - w, h, fx, fy, cx, cy = k - dw, dh = size - pc = points @ cam_from_scan[:3, :3].T + cam_from_scan[:3, 3] - z = pc[:, 2] - front = z > NEAR_M - pc, z = pc[front], z[front] - u = fx * pc[:, 0] / z + cx - v = fy * pc[:, 1] / z + cy - inside = (u >= 0) & (u < w) & (v >= 0) & (v < h) - j = (u[inside] * dw / w).astype(int) - i = (v[inside] * dh / h).astype(int) - depth = np.full(dh * dw, np.inf) - np.minimum.at(depth, i * dw + j, z[inside]) - depth[np.isinf(depth)] = 0 - return depth.reshape(dh, dw).astype(np.float32) - - -def build(eth3d: Path, out: Path) -> Path: - cams = read_cameras(eth3d / "dslr_calibration_undistorted" / "cameras.txt") - images = read_images(eth3d / "dslr_calibration_undistorted" / "images.txt") - points = np.load(eth3d / "scan_points_10mm.npy").astype(np.float64) - world_from_scan, tilt = gravity_world(np.load(eth3d / "normals.npy")) - - # Camera-to-world poses in ARKit camera axes, and depth maps, per photo. - frames = [] - for name in PHOTOS: - cam_from_scan, cid = images[name] - k = cams[cid] - size = (round(k[0] / DEPTH_SCALE), round(k[1] / DEPTH_SCALE)) - depth = render_depth(points, cam_from_scan, k, size) - world_pose = world_from_scan @ np.linalg.inv(cam_from_scan) @ OPENCV_TO_ARKIT_CAMERA - frames.append((name, k, depth, world_pose, cam_from_scan)) - - # The meter frame: where the first photo's optical axis meets the wall. - name, k, depth, world_pose, cam_from_scan = frames[0] - centre = depth[depth.shape[0] // 2, depth.shape[1] // 2] - if centre <= 0: - raise ValueError(f"{name}: no scan depth at the image centre to anchor the meter on") - hit_scan = np.linalg.inv(cam_from_scan) @ np.array([0.0, 0.0, centre, 1.0]) - near = points[np.linalg.norm(points - hit_scan[:3], axis=1) < WALL_RADIUS_M] - normal = world_from_scan[:3, :3] @ np.linalg.svd(near - near.mean(axis=0))[2][2] - if abs(normal[1]) > 0.5: - raise ValueError(f"{name}: the image centre hits a surface that is not a wall") - normal[1] = 0 - normal /= np.linalg.norm(normal) - hit_world = (world_from_scan @ hit_scan)[:3] - if normal @ (world_pose[:3, 3] - hit_world) < 0: - normal = -normal # point out of the wall, toward the camera - anchor = meter_frame(hit_world, np.cross([0.0, 1.0, 0.0], normal)) - to_meter = np.linalg.inv(anchor) - - world_points_y = (points @ world_from_scan[:3, :3].T)[:, 1] - horizontal = np.linalg.norm( - (points - hit_scan[:3]) @ world_from_scan[:3, :3].T * [1, 0, 1], axis=1 - ) - ground_y = float(np.percentile(world_points_y[horizontal < 1.0], 2)) - hit_world[1] - - w = PacketWriter(out) - photos = [] - for n, (name, k, depth, world_pose, _) in enumerate(frames): - pid = f"p{n + 1:05d}" - source = eth3d / "full" / f"{name}.JPG" - with Image.open(source) as img: - if img.size != (k[0], k[1]): - raise ValueError(f"{name}: image is {img.size}, calibration says {k[:2]}") - score = sharpness(img) - confidence = np.where(depth > 0, 2, 0) - photos.append( - { - "id": pid, - "image": w.add_file(f"photos/{pid}.jpg", source), - "width": k[0], - "height": k[1], - "t": float(n + 1), - "pose": column_major(to_meter @ world_pose), - "intrinsics": [round(v, 4) for v in k[2:]], - "lens": {"camera": f"ETH3D DSLR, calibration camera {images[name][1]}"}, - "sharpness": {"method": SHARPNESS_METHOD, "value": round(score, 3)}, - "depth": w.add_depth(pid, depth, confidence) - | {"source": "rendered_from_laser_scan"}, - } - ) - - manifest = { - "packet_version": PACKET_VERSION, - "session": { - "id": "eth3d-electro-9257-9264", - "producer": {"kind": "converter", "name": "packet.samples.eth3d", "version": "1"}, - "device": {"model": "ETH3D DSLR (24 MP)", "lidar": False}, - "capture": {"started_at_uptime": 0.0, "ended_at_uptime": float(len(frames) + 1)}, - "world_alignment": "gravity", - "meter_anchor": { - "pose_in_world": column_major(anchor), - "ground_y_m": round(ground_y, 4), - }, - }, - "photos": photos, - "marks": [{"id": "m1", "kind": "meter", "points": [[0.0, 0.0, 0.0]], "t": 0.5}], - "provenance": { - "dataset": "ETH3D high-res multi-view, electro (Schöps et al., CVPR 2017)", - "license": "CC BY-NC-SA 4.0: accuracy testing only; never commit or share", - "source": "https://www.eth3d.net/data/electro_dslr_undistorted.7z and the evals " - "lane's scan_points_10mm.npy / normals.npy", - "notes": [ - "Photos are ETH3D's undistorted originals, unchanged. They carry no EXIF, so " - "there is no capture time, exposure, ISO or lens data: t is the shooting order " - "in seconds.", - f"Depth is rendered from the laser scan (10 mm points, nearest point per pixel) " - f"at 1/{DEPTH_SCALE} of each photo's size; confidence is 2 where a scan point " - "landed and 0 elsewhere. It stands in for LiDAR depth: denser and more accurate " - "than an iPhone's.", - f"World is the scan levelled by its ground normals (tilt {tilt:.2f} degrees), " - "with y up. There is no meter: the meter frame sits where the first photo's " - "optical axis meets the wall, z out of the wall toward the camera. ground_y_m is " - "the 2nd percentile scan height within 1 m of it.", - "No trajectory, IMU, mesh, planes, guidance or scene.json: a DSLR records none.", - ], - }, - } - return w.finish(manifest) - - -def main() -> None: - parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0]) - parser.add_argument("--eth3d", type=Path, required=True) - parser.add_argument("--out", type=Path, required=True) - args = parser.parse_args() - print(build(args.eth3d.expanduser(), args.out.expanduser())) - - -if __name__ == "__main__": - main() diff --git a/packet/packet/samples/synthetic.py b/packet/packet/samples/synthetic.py deleted file mode 100644 index 71c9341b3..000000000 --- a/packet/packet/samples/synthetic.py +++ /dev/null @@ -1,327 +0,0 @@ -"""A tiny synthetic packet that uses every part of the spec: the test fixture in fixtures/. - -Nothing in it is measured. A camera slides 2 m along a flat wall 2.5 m away, facing it, and every -sensor reports the values that motion implies (or a fixed plausible value). It exists so readers -and the validator have one packet with every optional section, small enough to commit. - - uv run python -m packet.samples.synthetic --out fixtures/synthetic -""" - -from __future__ import annotations - -import argparse -import io -import json -from pathlib import Path - -import numpy as np -from PIL import Image, ImageDraw - -from packet.validate import horizontal_distance -from packet.write import ( - PACKET_VERSION, - SHARPNESS_METHOD, - PacketWriter, - column_major, - sharpness, - trajectory_row, -) - -T0, T1, RATE = 100.0, 102.0, 60 -W, H, FX = 96, 72, 80.0 -DEPTH_W, DEPTH_H = 24, 18 -PHOTO_TICKS = (15, 60, 105) # trajectory samples the photos are taken at -DEPTH_FRAME_TICKS = (30, 75) # depth recorded between photos (1.1) -WALL_DISTANCE = 2.5 - - -def camera_pose(t: float) -> np.ndarray: - """Camera to meter frame: sliding along +x from -1 to +1 m, 2.5 m out, facing the wall. - The camera looks along its -z, which is the meter frame's -z (into the wall).""" - pose = np.eye(4) - pose[:3, 3] = [-1.0 + 2.0 * (t - T0) / (T1 - T0), 0.0, WALL_DISTANCE] - return pose - - -def photo(index: int) -> bytes: - img = Image.new("RGB", (W, H), (200, 190, 170)) - draw = ImageDraw.Draw(img) - for x in range(0, W, 12): - draw.line([(x + 4 * index, 0), (x + 4 * index, H)], fill=(120, 60, 40), width=2) - draw.rectangle([30, 20, 50, 40], outline=(40, 40, 40), width=2) - buf = io.BytesIO() - img.save(buf, "JPEG", quality=90) - return buf.getvalue() - - -def build(out: Path) -> Path: - w = PacketWriter(out) - ticks = np.arange(0, round((T1 - T0) * RATE) + 1) - times = T0 + ticks / RATE - poses = [camera_pose(t) for t in times] - - photos = [] - for n, k in enumerate(PHOTO_TICKS, start=1): - pid = f"p{n:05d}" - data = photo(n) - # A flat wall straight ahead: every depth pixel is the wall distance. - depth = np.full((DEPTH_H, DEPTH_W), WALL_DISTANCE, dtype=np.float32) - depth[:, :2] = 0 # a strip with no measurement, as at a real depth map's edge - confidence = np.where(depth > 0, 2, 0) - photos.append( - { - "id": pid, - "image": w.add_bytes(f"photos/{pid}.jpg", data), - "width": W, - "height": H, - "t": float(times[k]), - "pose": column_major(poses[k]), - "intrinsics": [FX, FX, W / 2, H / 2], - "tracking": {"state": "normal", "reason": None}, - "exposure": {"duration_s": 1 / 500, "iso": 50}, - "lens": {"focal_length_mm": 5.1, "f_number": 1.8, "camera": "wide"}, - "sharpness": { - "method": SHARPNESS_METHOD, - "value": round(sharpness(Image.open(io.BytesIO(data))), 3), - }, - "depth": w.add_depth(pid, depth, confidence) | {"source": "arkit_scene_depth"}, - } - ) - - traj = [trajectory_row(t, "normal", p) for t, p in zip(times, poses, strict=True)] - imu_t = np.round(T0 + np.arange(0, 2.0, 0.01), 3) - streams = { - "trajectory": w.add_stream("trajectory", traj, RATE), - # Device held still in landscape: gravity along the device's -x. - "accelerometer": w.add_stream("accelerometer", [[t, -9.81, 0.0, 0.0] for t in imu_t], 100), - "gyroscope": w.add_stream("gyroscope", [[t, 0.0, 0.0, 0.0] for t in imu_t], 100), - "magnetometer": w.add_stream("magnetometer", [[t, 20.0, -5.0, -40.0] for t in imu_t], 100), - "device_motion": w.add_stream( - "device_motion", - [[t, 0, 0, 0, 1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 90.0] for t in imu_t], - 100, - ), - "barometer": w.add_stream("barometer", [[T0 + 0.5, 101.3, 0.0], [T0 + 1.5, 101.3, 0.0]], 1), - "location": w.add_stream("location", [[T0 + 0.5, 0.0, 0.0, 10.0, 5.0, 3.0]], 1), - "heading": w.add_stream("heading", [[T0 + 0.5, 90.0, 90.0, 10.0]], 1), - } - - mesh = mesh_ply() - wall_plane = np.eye(4) - wall_plane[:3, :3] = [[1, 0, 0], [0, 0, -1], [0, 1, 0]] # plane normal (+y) = meter +z - ground_plane = np.eye(4) - ground_plane[1, 3] = -1.2 - anchor = np.eye(4) - anchor[:3, :3] = [[0.866025, 0, 0.5], [0, 1, 0], [-0.5, 0, 0.866025]] - anchor[:3, 3] = [3.0, 0.2, -1.5] - - manifest = { - "packet_version": PACKET_VERSION, - "session": { - "id": "synthetic-0001", - "producer": {"kind": "converter", "name": "packet.samples.synthetic", "version": "1"}, - "device": { - "model": "synthetic", - "ios_version": "n/a", - "lidar": True, - "scene_depth_enabled": True, - "mesh_enabled": True, - "mesh_classification_enabled": True, - }, - "capture": { - "started_at": "2026-09-26T12:00:00Z", - "started_at_uptime": T0 - 0.5, - "ended_at_uptime": T1 + 0.5, - "distance_walked_m": round( - horizontal_distance(np.array([p[:3, 3] for p in poses])), 4 - ), - }, - "world_alignment": "gravity", - "meter_anchor": {"pose_in_world": column_major(anchor), "ground_y_m": -1.2}, - "consent": {"location": True}, - }, - "photos": photos, - "streams": streams, - "depth_frames": depth_frames(w, times, poses), - "lidar": {"mesh": w.add_bytes("lidar/mesh.ply", mesh)}, - "planes": [ - { - "id": "wall", - "alignment": "vertical", - "classification": "wall", - "pose": column_major(wall_plane), - "extent_m": [6.0, 2.4], - # An L of wall: a window cut out of the top right (plane x, z = -height). - "boundary_m": [ - [-3.0, -1.2], - [3.0, -1.2], - [3.0, 0.2], - [0.0, 0.2], - [0.0, 1.2], - [-3.0, 1.2], - ], - }, - { - "id": "ground", - "alignment": "horizontal", - "classification": "floor", - "pose": column_major(ground_plane), - "extent_m": [6.0, 3.0], - }, - ], - "marks": [ - { - "id": "m1", - "kind": "meter", - "points": [[0.0, 0.0, 0.0]], - "t": T0 - 0.2, - "photo_ids": ["p00001"], - }, - { - "id": "m2", - "kind": "wall_end", - "points": [[-3.0, 0.0, 0.0]], - "t": T0 + 0.1, - "side": "left", - "end_kind": "unexplored", - }, - { - "id": "m3", - "kind": "wall_end", - "points": [[3.0, 0.0, 0.0]], - "t": T0 + 1.9, - "side": "right", - "end_kind": "limit", - }, - {"id": "m4", "kind": "gas_meter", "points": [[1.5, -0.6, 0.1]], "t": T0 + 1.2}, - { - "id": "m5", - "kind": "window", - "points": [[-2.0, 0.2, 0.0], [-1.2, 1.2, 0.0]], - "t": T0 + 0.4, - "attrs": {"operable": True}, - }, - { - "id": "m6", - "kind": "drive_edge", - "points": [[2.0, -1.2, 0.5], [2.0, -1.2, 4.0]], - "t": T0 + 1.5, - }, - ], - "guidance": [ - { - "id": "g1", - "kind": "walk", - "origin": "phone", - "message": "Walk slowly to your left", - "t_shown": T0, - "t_resolved": T0 + 0.8, - "outcome": "met", - }, - { - "id": "g2", - "kind": "tilt_to_ground", - "origin": "phone", - "band": "ground", - "message": "Tilt down to show the ground", - "t_shown": T0 + 0.9, - "t_resolved": T0 + 1.3, - "outcome": "met", - }, - { - "id": "g3", - "kind": "gap_band", - "origin": "server", - "band": "ground", - "span_m": [1.0, 2.5], - "message": "Show the ground around your meter", - "t_shown": T0 + 1.6, - "t_resolved": T0 + 1.9, - "outcome": "cannot_reach", - }, - ], - "scene": w.add_bytes("scene.json", scene_json()) | {"schema_version": "1.0"}, - "provenance": { - "dataset": "synthetic", - "license": "same as this repository", - "source": "packet/packet/samples/synthetic.py", - "notes": ["Every value is invented; nothing was measured."], - }, - } - return w.finish(manifest) - - -def depth_frames(w: PacketWriter, times: np.ndarray, poses: list[np.ndarray]) -> list[dict]: - """Depth between photos: one frame from ARKit (with confidence), one estimated from the image - (with sigma), at the depth map's own resolution.""" - scale = DEPTH_W / W - frames = [] - for n, k in enumerate(DEPTH_FRAME_TICKS, start=1): - fid = f"d{n:05d}" - depth = np.full((DEPTH_H, DEPTH_W), WALL_DISTANCE, dtype=np.float32) - if n == 1: - data = w.add_depth( - fid, depth, confidence=np.full(depth.shape, 2), folder="depth_frames" - ) - data["source"] = "arkit_scene_depth" - else: - sigma = np.full(depth.shape, 0.15, dtype=np.float32) - data = w.add_depth(fid, depth, sigma=sigma, folder="depth_frames") - data["source"] = "estimated" - frames.append( - { - "id": fid, - "t": float(times[k]), - "pose": column_major(poses[k]), - "intrinsics": [FX * scale, FX * scale, DEPTH_W / 2, DEPTH_H / 2], - "tracking": {"state": "normal", "reason": None}, - **data, - } - ) - return frames - - -def mesh_ply() -> bytes: - """A wall quad (class 1, wall) and a ground quad (class 2, floor), two triangles each.""" - v = np.array( - [ - [-3, -1.2, 0], - [3, -1.2, 0], - [3, 1.2, 0], - [-3, 1.2, 0], - [-3, -1.2, 0], - [3, -1.2, 0], - [3, -1.2, 3], - [-3, -1.2, 3], - ], - dtype=" bytes: - scene = { - "schema_version": "1.0", - "meter": {"pos": [0.0, 4.0, 0.0], "wall_id": "w1"}, - "walls": [{"id": "w1", "baseline": [[-9.84, 0.0], [9.84, 0.0]]}], - } - return (json.dumps(scene, indent=1) + "\n").encode() - - -def main() -> None: - parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0]) - parser.add_argument("--out", type=Path, required=True) - print(build(parser.parse_args().out)) - - -if __name__ == "__main__": - main() diff --git a/packet/packet/validate.py b/packet/packet/validate.py deleted file mode 100644 index f27363cf9..000000000 --- a/packet/packet/validate.py +++ /dev/null @@ -1,590 +0,0 @@ -"""Check a capture packet against README.md and manifest.schema.json. - -Each check returns a list of problems; `validate` runs them all and never stops at the first, so -one run shows everything a producer has to fix. Tolerances are stated beside each check. -""" - -from __future__ import annotations - -import csv -import io -import json -import math -from dataclasses import dataclass, field -from pathlib import Path - -import numpy as np -from jsonschema import Draft202012Validator -from PIL import Image - -from packet.files import PacketError, PacketFiles - -SCHEMA_PATH = Path(__file__).resolve().parents[1] / "manifest.schema.json" - -# A float32 simd matrix written to JSON keeps about 7 significant digits, so an orthonormal -# rotation comes back orthonormal to about 1e-6; 1e-3 leaves room for any honest writer. -ROTATION_TOL = 1e-3 -QUATERNION_TOL = 1e-3 -# Real cameras have square pixels to well under 1%; 5% catches fx/fy swapped with a rotation. -PIXEL_ASPECT_TOL = 0.05 -FOV_RANGE_DEG = (20.0, 150.0) -DEPTH_ASPECT_TOL = 0.01 -# Distance walked is a sum over the trajectory; 1% (or 5 cm on a short walk) covers rounding. -DISTANCE_REL_TOL, DISTANCE_ABS_TOL = 0.01, 0.05 -# A photo taken from a tracked frame must sit on the trajectory: its nearest trajectory sample -# within one frame at 60 Hz, and at the same place to a centimetre. -PHOTO_TRAJECTORY_DT, PHOTO_TRAJECTORY_DIST = 1 / 60 + 1e-6, 0.01 - -STREAM_COLUMNS = { - "trajectory": ["t", "tracking", "px", "py", "pz", "qx", "qy", "qz", "qw"], - "accelerometer": ["t", "x", "y", "z"], - "gyroscope": ["t", "x", "y", "z"], - "magnetometer": ["t", "x", "y", "z"], - "device_motion": [ - "t", - "qx", - "qy", - "qz", - "qw", - "gravity_x", - "gravity_y", - "gravity_z", - "user_accel_x", - "user_accel_y", - "user_accel_z", - "rotation_rate_x", - "rotation_rate_y", - "rotation_rate_z", - "heading_deg", - ], - "barometer": ["t", "pressure_kpa", "relative_altitude_m"], - "location": [ - "t", - "latitude", - "longitude", - "altitude_m", - "horizontal_accuracy_m", - "vertical_accuracy_m", - ], - "heading": ["t", "magnetic_deg", "true_deg", "accuracy_deg"], -} -TRACKING_STATES = {"normal", "limited", "not_available"} -CONSENTED_STREAMS = ("location", "heading") -MARK_POINTS = { - "meter": 1, - "wall_end": 1, - "gas_meter": 1, - "ac": 1, - "door": 2, - "window": 2, - "garage_door": 2, - "drive_edge": 2, - "fence": 2, -} -# ARMeshClassification raw values: none, wall, floor, ceiling, table, seat, window, door. -MESH_CLASSIFICATIONS = 8 - - -@dataclass -class Report: - problems: list[str] = field(default_factory=list) - warnings: list[str] = field(default_factory=list) - summary: dict = field(default_factory=dict) - - @property - def ok(self) -> bool: - return not self.problems - - -# --- Geometry ----------------------------------------------------------------------------------- - - -def matrix(pose: list[float]) -> np.ndarray: - """16 numbers column by column to a 4x4 matrix.""" - return np.asarray(pose, dtype=np.float64).reshape(4, 4).T - - -def rigid_problems(pose: list[float], where: str) -> list[str]: - """A rotation (orthonormal, determinant +1), a finite translation and a [0 0 0 1] last row.""" - if len(pose) != 16 or not all(math.isfinite(v) for v in pose): - return [f"{where}: pose must be 16 finite numbers"] - m = matrix(pose) - out = [] - if not np.allclose(m[3], [0, 0, 0, 1], atol=1e-9): - out.append(f"{where}: last row is {m[3].tolist()}, not [0, 0, 0, 1] (column-major?)") - r = m[:3, :3] - err = float(np.abs(r.T @ r - np.eye(3)).max()) - if err > ROTATION_TOL: - out.append(f"{where}: rotation is not orthonormal (max |Ráµ€R - I| = {err:.2g})") - elif np.linalg.det(r) < 0: - out.append(f"{where}: rotation is a reflection (determinant < 0)") - return out - - -def intrinsics_problems(k: list[float], width: int, height: int, where: str) -> list[str]: - fx, fy, cx, cy = k - out = [] - if width <= height: - out.append( - f"{where}: image is {width}x{height}; photos are stored landscape (width > height)" - ) - if fx <= 0 or fy <= 0: - return [*out, f"{where}: focal lengths must be positive, got fx={fx}, fy={fy}"] - if not (0 < cx < width and 0 < cy < height): - out.append(f"{where}: principal point ({cx}, {cy}) is outside the {width}x{height} image") - if abs(fx / fy - 1) > PIXEL_ASPECT_TOL: - out.append(f"{where}: fx/fy = {fx / fy:.3f}; intrinsics may belong to a rotated image") - fov = math.degrees(2 * math.atan(width / (2 * fx))) - if not FOV_RANGE_DEG[0] <= fov <= FOV_RANGE_DEG[1]: - out.append( - f"{where}: horizontal field of view {fov:.1f}° is outside {FOV_RANGE_DEG}; " - "intrinsics do not fit this image size" - ) - return out - - -def increasing_problems(times: list[float], where: str) -> list[str]: - for i in range(1, len(times)): - if not times[i] > times[i - 1]: - return [f"{where}: time goes from {times[i - 1]} to {times[i]} at row {i + 1}"] - return [] - - -# --- Files -------------------------------------------------------------------------------------- - - -def file_problems(files: PacketFiles, ref: dict, where: str) -> list[str]: - path = ref["path"] - if not files.exists(path): - return [f"{where}: {path} is missing"] - size = files.size(path) - if size != ref["bytes"]: - return [f"{where}: {path} is {size} bytes, manifest says {ref['bytes']}"] - if files.sha256(path) != ref["sha256"]: - return [f"{where}: {path} sha256 does not match"] - return [] - - -def depth_refs(d: dict, where: str) -> list[tuple[str, dict]]: - return [(f"{where}.{key}", d[key]) for key in ("map", "confidence", "sigma") if key in d] - - -def file_refs(manifest: dict) -> list[tuple[str, dict]]: - """Every file the manifest names, with where it is named.""" - refs = [] - for i, p in enumerate(manifest.get("photos", [])): - refs.append((f"photos[{i}].image", p["image"])) - if "depth" in p: - refs += depth_refs(p["depth"], f"photos[{i}].depth") - for i, f in enumerate(manifest.get("depth_frames", [])): - refs += depth_refs(f, f"depth_frames[{i}]") - for name, s in manifest.get("streams", {}).items(): - refs.append((f"streams.{name}", s)) - if manifest.get("lidar", {}).get("mesh"): - refs.append(("lidar.mesh", manifest["lidar"]["mesh"])) - if manifest.get("scene"): - refs.append(("scene", manifest["scene"])) - return refs - - -# --- Photos and depth --------------------------------------------------------------------------- - - -def image_problems(files: PacketFiles, photo: dict, where: str) -> list[str]: - try: - with files.open(photo["image"]["path"]) as f, Image.open(f) as img: - fmt, size = img.format, img.size - orientation = img.getexif().get(0x0112) - except (OSError, SyntaxError) as exc: - return [f"{where}: image cannot be read: {exc}"] - out = [] - if fmt != "JPEG": - out.append(f"{where}: image is {fmt}, not JPEG") - if size != (photo["width"], photo["height"]): - out.append( - f"{where}: image is {size[0]}x{size[1]}, manifest says " - f"{photo['width']}x{photo['height']}" - ) - if orientation not in (None, 1): - out.append(f"{where}: EXIF orientation {orientation}; store the unrotated sensor image") - return out - - -ARKIT_DEPTH = {"arkit_scene_depth", "arkit_smoothed_scene_depth"} -# A boundary vertex may sit on the extent's edge; allow float rounding and 1 cm. -BOUNDARY_TOL_M = 0.01 - - -def photo_depth_problems(photo: dict, where: str) -> list[str]: - """A photo's depth covers the photo's field of view: same aspect, no larger.""" - d = photo["depth"] - w, h = d["width"], d["height"] - out = [] - if w > photo["width"] or h > photo["height"]: - out.append(f"{where}: depth {w}x{h} is larger than the photo") - photo_aspect, depth_aspect = photo["width"] / photo["height"], w / h - if abs(depth_aspect / photo_aspect - 1) > DEPTH_ASPECT_TOL: - out.append( - f"{where}: depth {w}x{h} does not have the photo's aspect " - f"({photo['width']}x{photo['height']}); it cannot be aligned by scaling" - ) - return out - - -def depth_data_problems( - files: PacketFiles, d: dict, intact: set[str], minor: int, where: str -) -> list[str]: - """The depth map, its confidence and its sigma: sizes, values, and which are required.""" - w, h = d["width"], d["height"] - out = [] - source = d.get("source") - if source in ARKIT_DEPTH and minor >= 1 and "confidence" not in d: - out.append(f"{where}: {source} depth needs confidence (required from 1.1)") - if source == "estimated" and "sigma" not in d: - out.append(f"{where}: estimated depth needs sigma") - for key, dtype, name in (("map", " 0).mean() < 0.01: - out.append(f"{where}: depth is empty (under 1% of pixels measured)") - if "confidence" in d: - if d["confidence"]["bytes"] != w * h: - out.append( - f"{where}: confidence is {d['confidence']['bytes']} bytes, {w}x{h} uint8 is {w * h}" - ) - elif d["confidence"]["path"] in intact: - conf = np.frombuffer(files.read(d["confidence"]["path"]), dtype=np.uint8) - if conf.max(initial=0) > 2: - out.append(f"{where}: confidence values must be 0, 1 or 2") - return out - - -def plane_problems(plane: dict, where: str) -> list[str]: - out = rigid_problems(plane["pose"], where) - if "boundary_m" in plane: - ex, ez = plane["extent_m"] - b = np.asarray(plane["boundary_m"], dtype=float) - if not np.isfinite(b).all(): - out.append(f"{where}: boundary_m must be finite") - elif (np.abs(b[:, 0]) > ex / 2 + BOUNDARY_TOL_M).any() or ( - np.abs(b[:, 1]) > ez / 2 + BOUNDARY_TOL_M - ).any(): - out.append( - f"{where}: boundary_m leaves the {ex} x {ez} m extent centred on the pose; " - "is it in ARKit's anchor frame instead of the pose's?" - ) - return out - - -# --- Streams ------------------------------------------------------------------------------------ - - -def read_stream(files: PacketFiles, name: str, ref: dict) -> tuple[list[str], list[list[str]]]: - text = io.TextIOWrapper(files.open(ref["path"]), encoding="utf-8", newline="") - rows = list(csv.reader(text)) - return (rows[0] if rows else []), rows[1:] - - -def stream_problems(name: str, header: list[str], rows: list[list[str]], ref: dict) -> list[str]: - where = f"streams.{name}" - want = STREAM_COLUMNS[name] - if header != want: - return [f"{where}: header is {header}, expected {want}"] - if len(rows) != ref["rows"]: - return [f"{where}: {len(rows)} rows, manifest says {ref['rows']}"] - numeric = [i for i, c in enumerate(want) if c != "tracking"] - try: - values = [[float(r[i]) for i in numeric] for r in rows] - except (ValueError, IndexError) as exc: - return [f"{where}: a row is short or not numeric ({exc})"] - if any(not math.isfinite(v) for row in values for v in row): - return [f"{where}: values must be finite"] - out = increasing_problems([row[0] for row in values], where) - if name == "trajectory": - bad = [r[1] for r in rows if r[1] not in TRACKING_STATES] - if bad: - out.append( - f"{where}: tracking must be one of {sorted(TRACKING_STATES)}, got {bad[0]!r}" - ) - q = np.asarray([row[4:8] for row in values]) - norm_err = np.abs(np.linalg.norm(q, axis=1) - 1) if len(q) else np.zeros(0) - if len(q) and norm_err.max() > QUATERNION_TOL: - out.append( - f"{where}: quaternion at row {int(norm_err.argmax()) + 1} is not unit length" - ) - return out - - -def horizontal_distance(positions: np.ndarray) -> float: - """Path length on the ground plane (meter frame x and z; y is up).""" - if len(positions) < 2: - return 0.0 - step = np.diff(positions[:, [0, 2]], axis=0) - return float(np.linalg.norm(step, axis=1).sum()) - - -# --- LiDAR -------------------------------------------------------------------------------------- - -MESH_HEADER = [ - "ply", - "format binary_little_endian 1.0", - None, # element vertex N - "property float x", - "property float y", - "property float z", - None, # element face M - "property list uchar int vertex_indices", - "property uchar classification", - "end_header", -] - - -def mesh_problems(data: bytes, classified: bool = True) -> list[str]: - """A binary PLY with exactly the layout README.md fixes, indices in range, classes 0 to 7, - and all 0 when classification was off.""" - end = data.find(b"end_header\n") - if not data.startswith(b"ply\n") or end < 0: - return ["lidar.mesh: not a PLY file"] - lines = [*data[:end].decode("ascii", "replace").rstrip("\n").split("\n"), "end_header"] - lines = [ln for ln in lines if not ln.startswith("comment")] - if len(lines) != len(MESH_HEADER): - return [f"lidar.mesh: header has {len(lines)} lines, expected the layout in README.md"] - counts = {} - for got, want in zip(lines, MESH_HEADER, strict=True): - if want is None: - parts = got.split() - if len(parts) != 3 or parts[0] != "element" or parts[1] not in ("vertex", "face"): - return [f"lidar.mesh: expected an element line, got {got!r}"] - counts[parts[1]] = int(parts[2]) - elif got != want: - return [f"lidar.mesh: header line {got!r}, expected {want!r}"] - body = memoryview(data)[end + len(b"end_header\n") :] - nv, nf = counts["vertex"], counts["face"] - vertices = np.frombuffer(body[: nv * 12], dtype="= nv): - out.append("lidar.mesh: a face index is out of range") - if nf and faces["c"].max() >= MESH_CLASSIFICATIONS: - out.append("lidar.mesh: classification must be an ARMeshClassification raw value, 0 to 7") - elif nf and not classified and faces["c"].any(): - out.append("lidar.mesh: classification was off, so every face's class must be 0") - return out - - -# --- The whole packet --------------------------------------------------------------------------- - - -def schema_problems(manifest: dict) -> list[str]: - schema = json.loads(SCHEMA_PATH.read_text()) - errors = sorted(Draft202012Validator(schema).iter_errors(manifest), key=lambda e: list(e.path)) - return [f"schema: {'/'.join(map(str, e.path)) or '(root)'}: {e.message}" for e in errors] - - -def validate(source: Path) -> Report: - report = Report() - try: - files = PacketFiles(source) - manifest = json.loads(files.read("manifest.json")) - except (PacketError, json.JSONDecodeError, UnicodeDecodeError) as exc: - report.problems.append(str(exc)) - return report - report.problems += schema_problems(manifest) - if report.problems: - return report # the checks below rely on the shape the schema guarantees - - problems = report.problems - minor = int(manifest["packet_version"].split(".")[1]) - session = manifest["session"] - start, end = session["capture"]["started_at_uptime"], session["capture"]["ended_at_uptime"] - if not end > start: - problems.append("session.capture: ended_at_uptime must be after started_at_uptime") - - def in_window(t: float, where: str) -> None: - if not start <= t <= end: - problems.append(f"{where}: t = {t} is outside the capture [{start}, {end}]") - - problems += rigid_problems(session["meter_anchor"]["pose_in_world"], "session.meter_anchor") - - refs = file_refs(manifest) - paths = [ref["path"] for _, ref in refs] - dupes = sorted({p for p in paths if paths.count(p) > 1}) - if dupes: - problems.append(f"files named more than once: {dupes[:3]}") - # Content checks read only files whose size and hash matched, so one bad file is one problem. - intact = set() - for where, ref in refs: - found = file_problems(files, ref, where) - problems += found - if not found: - intact.add(ref["path"]) - unlisted = files.names() - set(paths) - {"manifest.json"} - if unlisted: - report.warnings.append( - f"{len(unlisted)} files are not in the manifest: {sorted(unlisted)[:3]}" - ) - - photos = manifest["photos"] - ids = [p["id"] for p in photos] - if len(set(ids)) != len(ids): - problems.append("photos: ids must be unique") - problems += increasing_problems([p["t"] for p in photos], "photos (in manifest order)") - for i, p in enumerate(photos): - where = f"photos[{i}] {p['id']}" - in_window(p["t"], where) - problems += rigid_problems(p["pose"], where) - problems += intrinsics_problems(p["intrinsics"], p["width"], p["height"], where) - if p["image"]["path"] in intact: - problems += image_problems(files, p, where) - if "depth" in p: - problems += photo_depth_problems(p, where) - problems += depth_data_problems(files, p["depth"], intact, minor, where) - - frames = manifest.get("depth_frames", []) - frame_ids = [f["id"] for f in frames] - if len(set(frame_ids)) != len(frame_ids): - problems.append("depth_frames: ids must be unique") - problems += increasing_problems([f["t"] for f in frames], "depth_frames (in manifest order)") - for i, f in enumerate(frames): - where = f"depth_frames[{i}] {f['id']}" - in_window(f["t"], where) - problems += rigid_problems(f["pose"], where) - problems += intrinsics_problems(f["intrinsics"], f["width"], f["height"], where) - problems += depth_data_problems(files, f, intact, minor, where) - - streams = manifest.get("streams", {}) - trajectory = None - for name, ref in streams.items(): - if ref["path"] not in intact: - continue - header, rows = read_stream(files, name, ref) - found = stream_problems(name, header, rows, ref) - problems += found - if not found and rows: - first, last = float(rows[0][0]), float(rows[-1][0]) - if first < start or last > end: - report.warnings.append( - f"streams.{name}: runs {first} to {last}, beyond the capture [{start}, {end}]" - ) - if name == "trajectory": - trajectory = np.asarray([[float(v) for v in (r[0], *r[2:5])] for r in rows]) - for name in CONSENTED_STREAMS: - if name in streams and not session.get("consent", {}).get("location", False): - problems.append(f"streams.{name} is present but session.consent.location is not true") - - walked = session["capture"].get("distance_walked_m") - if trajectory is not None: - measured = horizontal_distance(trajectory[:, 1:4]) - if walked is None: - report.warnings.append("session.capture.distance_walked_m is missing") - elif abs(walked - measured) > max(DISTANCE_ABS_TOL, DISTANCE_REL_TOL * measured): - problems.append( - f"session.capture.distance_walked_m is {walked}; " - f"the trajectory walks {measured:.3f}" - ) - posed = [(f"photos[{i}] {p['id']}", p) for i, p in enumerate(photos)] - posed += [(f"depth_frames[{i}] {f['id']}", f) for i, f in enumerate(frames)] - for where, p in posed: - j = int(np.abs(trajectory[:, 0] - p["t"]).argmin()) - dt = abs(trajectory[j, 0] - p["t"]) - dist = float(np.linalg.norm(trajectory[j, 1:4] - matrix(p["pose"])[:3, 3])) - if dt > PHOTO_TRAJECTORY_DT: - problems.append(f"{where}: no trajectory sample within {dt:.3f} s") - elif dist > PHOTO_TRAJECTORY_DIST: - problems.append( - f"{where}: {dist:.3f} m from the trajectory at its time; " - "poses and trajectory disagree (frame or clock)" - ) - - lidar = manifest.get("lidar", {}) - classified = session["device"].get("mesh_classification_enabled") - if "mesh" in lidar: - if minor >= 1 and classified is None: - problems.append( - "session.device.mesh_classification_enabled is required with a mesh (from 1.1)" - ) - if lidar["mesh"]["path"] in intact: - problems += mesh_problems(files.read(lidar["mesh"]["path"]), classified is not False) - if "planes" in manifest and "planes" in lidar: - problems.append("planes appear at the top level and in lidar.planes; use one (top level)") - planes = [("planes", pl) for pl in manifest.get("planes", [])] - planes += [("lidar.planes", pl) for pl in lidar.get("planes", [])] - for i, (key, plane) in enumerate(planes): - problems += plane_problems(plane, f"{key}[{i}]") - - photo_ids = set(ids) - for i, m in enumerate(manifest.get("marks", [])): - where = f"marks[{i}] {m['kind']}" - if "t" in m: - in_window(m["t"], where) - if len(m["points"]) != MARK_POINTS[m["kind"]]: - problems.append( - f"{where}: {len(m['points'])} points, a {m['kind']} has {MARK_POINTS[m['kind']]}" - ) - missing = [pid for pid in m.get("photo_ids", []) if pid not in photo_ids] - if missing: - problems.append(f"{where}: photo_ids {missing[:3]} are not photos in this packet") - if m["kind"] == "wall_end" and not {"side", "end_kind"} <= m.keys(): - problems.append(f"{where}: a wall end needs side and end_kind") - - for i, g in enumerate(manifest.get("guidance", [])): - where = f"guidance[{i}] {g['kind']}" - in_window(g["t_shown"], where) - resolved = g.get("t_resolved") - if resolved is not None and resolved < g["t_shown"]: - problems.append(f"{where}: resolved before it was shown") - if g["outcome"] != "unresolved" and resolved is None: - problems.append(f"{where}: outcome {g['outcome']} needs t_resolved") - - scene = manifest.get("scene") - if scene and scene["path"] in intact: - try: - doc = json.loads(files.read(scene["path"])) - except json.JSONDecodeError as exc: - problems.append(f"scene: {scene['path']} is not JSON ({exc})") - else: - want = scene.get("schema_version") - if want is not None and doc.get("schema_version") != want: - problems.append( - f"scene: schema_version {doc.get('schema_version')!r}, manifest says {want!r}" - ) - - report.summary = { - "packet_version": manifest["packet_version"], - "producer": f"{session['producer']['name']} {session['producer']['version']}", - "photos": len(photos), - "photo_size": f"{photos[0]['width']}x{photos[0]['height']}", - "photos_with_depth": sum("depth" in p for p in photos), - "depth_frames": len(frames), - "streams": {n: s["rows"] for n, s in streams.items()}, - "mesh": "mesh" in lidar, - "planes": len(planes), - "marks": len(manifest.get("marks", [])), - "guidance": len(manifest.get("guidance", [])), - "seconds": round(end - start, 2), - "distance_walked_m": walked, - } - return report diff --git a/packet/packet/write.py b/packet/packet/write.py deleted file mode 100644 index 6722a4443..000000000 --- a/packet/packet/write.py +++ /dev/null @@ -1,167 +0,0 @@ -"""Write a packet: files with their sizes and hashes, CSV streams, the manifest. - -The sample builders and the test fixture go through this, so every packet they produce is -described by the same code the validator checks. -""" - -from __future__ import annotations - -import csv -import hashlib -import io -import json -import shutil -import zipfile -from pathlib import Path - -import numpy as np -from PIL import Image - -from packet.validate import STREAM_COLUMNS - -PACKET_VERSION = "1.1" -SHARPNESS_METHOD = "laplacian_variance_luma_640" - - -class PacketWriter: - def __init__(self, folder: Path): - if folder.exists(): - shutil.rmtree(folder) - folder.mkdir(parents=True) - self.folder = folder - - def add_bytes(self, path: str, data: bytes) -> dict: - target = self.folder / path - target.parent.mkdir(parents=True, exist_ok=True) - target.write_bytes(data) - return {"path": path, "bytes": len(data), "sha256": hashlib.sha256(data).hexdigest()} - - def add_file(self, path: str, source: Path) -> dict: - """Copy a file in unchanged (an original photo keeps its bytes).""" - target = self.folder / path - target.parent.mkdir(parents=True, exist_ok=True) - shutil.copyfile(source, target) - digest = hashlib.sha256() - with target.open("rb") as f: - for chunk in iter(lambda: f.read(1 << 20), b""): - digest.update(chunk) - return {"path": path, "bytes": target.stat().st_size, "sha256": digest.hexdigest()} - - def add_stream(self, name: str, rows: list[list], rate_hz: float | None = None) -> dict: - buf = io.StringIO() - out = csv.writer(buf, lineterminator="\n") - out.writerow(STREAM_COLUMNS[name]) - out.writerows(rows) - ref = self.add_bytes(f"streams/{name}.csv", buf.getvalue().encode()) - ref["rows"] = len(rows) - if rate_hz is not None: - ref["nominal_rate_hz"] = rate_hz - return ref - - def add_depth( - self, - name: str, - depth: np.ndarray, - confidence: np.ndarray | None = None, - sigma: np.ndarray | None = None, - folder: str = "depth", - ) -> dict: - h, w = depth.shape - ref = { - "map": self.add_bytes(f"{folder}/{name}.f32", depth.astype(" Path: - (self.folder / "manifest.json").write_text(json.dumps(manifest, indent=1) + "\n") - if not zip_too: - return self.folder - archive = self.folder.with_suffix(".zip") - with zipfile.ZipFile(archive, "w", zipfile.ZIP_STORED) as z: - for p in sorted(self.folder.rglob("*")): - if p.is_file(): - z.write(p, f"{self.folder.name}/{p.relative_to(self.folder).as_posix()}") - return archive - - -# --- Photo measurements ------------------------------------------------------------------------- - - -def sharpness(image: Image.Image) -> float: - """Variance of the 4-neighbour Laplacian of the luma, after scaling the long side to 640 px - (bilinear; images already smaller are used as they are). README.md fixes this method so scores - compare across phones and resolutions.""" - luma = image.convert("L") - scale = 640 / max(luma.size) - if scale < 1: - size = (round(luma.width * scale), round(luma.height * scale)) - luma = luma.resize(size, Image.Resampling.BILINEAR) - a = np.asarray(luma, dtype=np.float64) - lap = a[1:-1, :-2] + a[1:-1, 2:] + a[:-2, 1:-1] + a[2:, 1:-1] - 4 * a[1:-1, 1:-1] - return float(lap.var()) - - -# --- Frames and poses --------------------------------------------------------------------------- - - -def column_major(m: np.ndarray) -> list[float]: - return [float(v) for v in np.asarray(m).T.reshape(-1)] - - -def from_column_major(pose: list[float]) -> np.ndarray: - return np.asarray(pose, dtype=np.float64).reshape(4, 4).T - - -def orthonormalize(m: np.ndarray) -> np.ndarray: - """The nearest rotation to a rounded one (SVD), keeping the translation.""" - u, _, vt = np.linalg.svd(m[:3, :3]) - out = m.copy() - out[:3, :3] = u @ np.diag([1.0, 1.0, np.linalg.det(u @ vt)]) @ vt - return out - - -def meter_frame(meter_world: np.ndarray, wall_along_world: np.ndarray) -> np.ndarray: - """Meter-to-world pose: origin at the meter, +y up (world +y, gravity aligned), +x along the - wall to the right as seen from outside, +z out of the wall toward the homeowner.""" - x = np.array([wall_along_world[0], 0.0, wall_along_world[2]]) - x /= np.linalg.norm(x) - y = np.array([0.0, 1.0, 0.0]) - z = np.cross(x, y) - t = np.eye(4) - t[:3, 0], t[:3, 1], t[:3, 2], t[:3, 3] = x, y, z, meter_world - return t - - -def quaternion_xyzw(r: np.ndarray) -> list[float]: - """Unit quaternion (x, y, z, w) of a rotation matrix, w >= 0.""" - m = r - tr = m[0, 0] + m[1, 1] + m[2, 2] - if tr > 0: - s = 2 * np.sqrt(tr + 1) - q = [(m[2, 1] - m[1, 2]) / s, (m[0, 2] - m[2, 0]) / s, (m[1, 0] - m[0, 1]) / s, s / 4] - elif m[0, 0] > m[1, 1] and m[0, 0] > m[2, 2]: - s = 2 * np.sqrt(1 + m[0, 0] - m[1, 1] - m[2, 2]) - q = [s / 4, (m[0, 1] + m[1, 0]) / s, (m[0, 2] + m[2, 0]) / s, (m[2, 1] - m[1, 2]) / s] - elif m[1, 1] > m[2, 2]: - s = 2 * np.sqrt(1 + m[1, 1] - m[0, 0] - m[2, 2]) - q = [(m[0, 1] + m[1, 0]) / s, s / 4, (m[1, 2] + m[2, 1]) / s, (m[0, 2] - m[2, 0]) / s] - else: - s = 2 * np.sqrt(1 + m[2, 2] - m[0, 0] - m[1, 1]) - q = [(m[0, 2] + m[2, 0]) / s, (m[1, 2] + m[2, 1]) / s, s / 4, (m[1, 0] - m[0, 1]) / s] - q = np.asarray(q) - q /= np.linalg.norm(q) - return [float(v) for v in (q if q[3] >= 0 else -q)] - - -def trajectory_row(t: float, tracking: str, pose: np.ndarray, digits: int = 6) -> list: - p, q = pose[:3, 3], quaternion_xyzw(pose[:3, :3]) - return [round(t, 6), tracking, *(round(float(v), digits) for v in p), *(round(v, 7) for v in q)] diff --git a/packet/pyproject.toml b/packet/pyproject.toml deleted file mode 100644 index 31b0c243f..000000000 --- a/packet/pyproject.toml +++ /dev/null @@ -1,32 +0,0 @@ -[project] -name = "house-scanning-packet" -version = "0.1.0" -description = "The capture packet: spec, JSON Schema, validator and sample builders" -readme = "README.md" -requires-python = ">=3.12" -dependencies = [ - "jsonschema>=4.23", - "numpy>=2.1", - "pillow>=11.0", -] - -[dependency-groups] -dev = [ - "pytest>=9.1.1", - "ruff>=0.16.9", -] - -[tool.uv] -package = false -required-version = ">=0.11.15,<0.12" - -[tool.ruff] -line-length = 100 - -[tool.ruff.lint] -select = ["E", "W", "F", "I", "B", "UP", "SIM", "RUF"] - -[tool.pytest.ini_options] -testpaths = ["tests"] -pythonpath = ["."] -addopts = ["--strict-markers", "--strict-config"] diff --git a/packet/tests/test_checks.py b/packet/tests/test_checks.py deleted file mode 100644 index b1be902c5..000000000 --- a/packet/tests/test_checks.py +++ /dev/null @@ -1,227 +0,0 @@ -"""Each validator check on its own, with inputs built by hand.""" - -import math -import zipfile - -import numpy as np -import pytest - -from packet.files import PacketError, PacketFiles, safe_relative -from packet.samples.synthetic import mesh_ply -from packet.validate import ( - STREAM_COLUMNS, - increasing_problems, - intrinsics_problems, - mesh_problems, - rigid_problems, - stream_problems, -) -from packet.write import column_major, quaternion_xyzw, trajectory_row - - -def pose(r=None, t=(0.0, 0.0, 0.0)) -> list[float]: - m = np.eye(4) - if r is not None: - m[:3, :3] = r - m[:3, 3] = t - return column_major(m) - - -def yaw(deg: float) -> np.ndarray: - a = math.radians(deg) - return np.array([[math.cos(a), 0, math.sin(a)], [0, 1, 0], [-math.sin(a), 0, math.cos(a)]]) - - -# --- Poses -------------------------------------------------------------------------------------- - - -def test_a_rotation_with_translation_is_rigid(): - assert rigid_problems(pose(yaw(30), (1, 2, 3)), "p") == [] - - -def test_a_reflection_is_not_a_rotation(): - assert "reflection" in rigid_problems(pose(np.diag([1, 1, -1])), "p")[0] - - -def test_a_scaled_rotation_is_not_orthonormal(): - assert "not orthonormal" in rigid_problems(pose(yaw(30) * 1.01), "p")[0] - - -def test_rotation_rounded_to_seven_digits_passes(): - rounded = [round(v, 7) for v in pose(yaw(33.3), (1, 2, 3))] - assert rigid_problems(rounded, "p") == [] - - -def test_a_row_major_pose_is_caught_by_its_last_row(): - m = np.eye(4) - m[:3, 3] = [1, 2, 3] - row_major = [float(v) for v in m.reshape(-1)] # translation lands in the last row - assert "column-major" in rigid_problems(row_major, "p")[0] - - -def test_nan_in_a_pose_is_refused(): - bad = pose() - bad[12] = math.nan - assert rigid_problems(bad, "p") == ["p: pose must be 16 finite numbers"] - - -# --- Intrinsics --------------------------------------------------------------------------------- - - -def test_phone_intrinsics_fit_a_landscape_image(): - assert intrinsics_problems([1440.0, 1440.0, 960.0, 720.0], 1920, 1440, "p") == [] - - -def test_a_portrait_image_is_refused(): - assert "landscape" in intrinsics_problems([1440.0, 1440.0, 720.0, 960.0], 1440, 1920, "p")[0] - - -def test_principal_point_outside_the_image(): - assert "outside" in intrinsics_problems([1440.0, 1440.0, 2000.0, 720.0], 1920, 1440, "p")[0] - - -def test_unequal_focal_lengths_suggest_a_rotated_image(): - assert "rotated" in intrinsics_problems([1440.0, 1080.0, 960.0, 720.0], 1920, 1440, "p")[0] - - -def test_intrinsics_for_a_larger_image_do_not_fit(): - # Intrinsics of a 4032-wide photo attached to a 480-wide thumbnail: a 19° field of view. - msgs = intrinsics_problems([1440.0, 1440.0, 240.0, 180.0], 480, 360, "p") - assert any("field of view" in m for m in msgs) - - -# --- Time --------------------------------------------------------------------------------------- - - -def test_increasing_times_pass_and_repeats_fail(): - assert increasing_problems([1.0, 2.0, 3.0], "s") == [] - assert "row 3" in increasing_problems([1.0, 2.0, 2.0], "s")[0] - assert "row 2" in increasing_problems([2.0, 1.0], "s")[0] - - -# --- Streams ------------------------------------------------------------------------------------ - - -def trajectory_rows(n: int = 3) -> list[list[str]]: - return [[str(v) for v in trajectory_row(10 + i / 60, "normal", np.eye(4))] for i in range(n)] - - -def test_a_good_trajectory_passes(): - rows = trajectory_rows() - assert stream_problems("trajectory", STREAM_COLUMNS["trajectory"], rows, {"rows": 3}) == [] - - -def test_stream_header_must_match_exactly(): - header = ["t", "x", "y"] - assert "header" in stream_problems("gyroscope", header, [], {"rows": 0})[0] - - -def test_stream_row_count_must_match_the_manifest(): - msg = stream_problems( - "trajectory", STREAM_COLUMNS["trajectory"], trajectory_rows(), {"rows": 4} - ) - assert "3 rows" in msg[0] - - -def test_stream_values_must_be_numeric(): - rows = [["1.0", "0.1", "x", "0.3"]] - assert ( - "not numeric" - in stream_problems("gyroscope", STREAM_COLUMNS["gyroscope"], rows, {"rows": 1})[0] - ) - - -def test_stream_time_must_increase(): - rows = [["1.0", "0", "0", "0"], ["1.0", "0", "0", "0"]] - assert ( - "time goes" - in stream_problems("gyroscope", STREAM_COLUMNS["gyroscope"], rows, {"rows": 2})[0] - ) - - -def test_trajectory_tracking_and_quaternions_are_checked(): - rows = trajectory_rows() - rows[1][1] = "good" - rows[2][8] = "0.9" # qw: no longer unit length - msgs = stream_problems("trajectory", STREAM_COLUMNS["trajectory"], rows, {"rows": 3}) - assert any("tracking" in m for m in msgs) - assert any("unit length" in m for m in msgs) - - -def test_quaternion_round_trip(): - q = quaternion_xyzw(yaw(120)) - x, y, z, w = q - assert math.isclose(math.hypot(x, y, z, w), 1) - assert math.isclose(math.degrees(2 * math.acos(w)), 120, abs_tol=1e-9) - assert math.isclose(y, math.sin(math.radians(60))) - - -# --- Mesh --------------------------------------------------------------------------------------- - - -def test_the_fixture_mesh_passes(): - assert mesh_problems(mesh_ply()) == [] - - -def test_mesh_face_index_out_of_range(): - data = bytearray(mesh_ply()) - # The last face's first index sits 13 bytes from the end: count(1) + 3 x int32 + class(1). - data[-13:-9] = (99).to_bytes(4, "little") - assert "out of range" in mesh_problems(bytes(data))[0] - - -def test_mesh_classification_above_seven(): - data = bytearray(mesh_ply()) - data[-1] = 8 - assert "classification" in mesh_problems(bytes(data))[0] - - -def test_truncated_mesh_is_caught(): - assert "bytes" in mesh_problems(mesh_ply()[:-5])[0] - - -def test_mesh_with_another_layout_is_refused(): - data = mesh_ply().replace(b"property float z", b"property double z") - assert "header line" in mesh_problems(data)[0] - - -# --- Opening packets ---------------------------------------------------------------------------- - - -@pytest.mark.parametrize( - ("path", "ok"), - [("photos/p1.jpg", True), ("/etc/passwd", False), ("a/../../b", False), ("a\\b", False)], -) -def test_safe_relative(path, ok): - assert safe_relative(path) is ok - - -def test_a_zip_entry_that_climbs_out_is_refused(tmp_path): - archive = tmp_path / "p.zip" - with zipfile.ZipFile(archive, "w") as z: - z.writestr("manifest.json", "{}") - z.writestr("../evil.txt", "x") - with pytest.raises(PacketError, match="leave the packet"): - PacketFiles(archive) - - -def test_a_zip_with_two_manifests_is_refused(tmp_path): - archive = tmp_path / "p.zip" - with zipfile.ZipFile(archive, "w") as z: - z.writestr("a/manifest.json", "{}") - z.writestr("b/manifest.json", "{}") - with pytest.raises(PacketError, match="exactly one manifest"): - PacketFiles(archive) - - -def test_a_folder_without_a_manifest_is_refused(tmp_path): - with pytest.raises(PacketError, match="no manifest"): - PacketFiles(tmp_path) - - -def test_an_unclassified_mesh_has_only_class_zero(): - data = bytearray(mesh_ply()) - for i in range(1, 5): # the class byte ends each 14-byte face - data[-14 * i + 13] = 0 - assert mesh_problems(bytes(data), classified=False) == [] - assert mesh_problems(mesh_ply(), classified=False) != [] diff --git a/packet/tests/test_packet.py b/packet/tests/test_packet.py deleted file mode 100644 index d982ace32..000000000 --- a/packet/tests/test_packet.py +++ /dev/null @@ -1,345 +0,0 @@ -"""The validator on the committed fixture, and on copies of it broken one way at a time.""" - -import hashlib -import io -import json -import shutil -import zipfile -from pathlib import Path - -import numpy as np -import pytest -from PIL import Image - -from packet.__main__ import main -from packet.samples import synthetic -from packet.validate import validate - -FIXTURE = Path(__file__).resolve().parents[1] / "fixtures" / "synthetic" - - -@pytest.fixture -def packet(tmp_path) -> Path: - copy = tmp_path / "packet" - shutil.copytree(FIXTURE, copy) - return copy - - -def manifest(folder: Path) -> dict: - return json.loads((folder / "manifest.json").read_text()) - - -def save(folder: Path, m: dict) -> None: - (folder / "manifest.json").write_text(json.dumps(m)) - - -def replace_file(folder: Path, ref: dict, data: bytes) -> None: - """Write new bytes and update the manifest entry, so only the content check can object.""" - (folder / ref["path"]).write_bytes(data) - ref["bytes"], ref["sha256"] = len(data), hashlib.sha256(data).hexdigest() - - -def only_problem(folder: Path) -> str: - report = validate(folder) - assert len(report.problems) == 1, report.problems - return report.problems[0] - - -def test_the_committed_fixture_is_what_the_builder_writes(tmp_path): - rebuilt = synthetic.build(tmp_path / "synthetic") - for f in FIXTURE.rglob("*"): - if f.is_file(): - assert f.read_bytes() == (rebuilt / f.relative_to(FIXTURE)).read_bytes(), f - - -def test_the_fixture_is_valid_as_a_folder_and_as_a_zip(tmp_path): - assert validate(FIXTURE).problems == [] - archive = tmp_path / "p.zip" - with zipfile.ZipFile(archive, "w") as z: - for f in FIXTURE.rglob("*"): - if f.is_file(): - z.write(f, f"synthetic/{f.relative_to(FIXTURE).as_posix()}") - report = validate(archive) - assert report.ok, report.problems - assert report.summary["photos"] == 3 - - -def test_cli_exit_codes(packet, capsys): - assert main(["validate", str(packet)]) == 0 - assert capsys.readouterr().out.startswith(f"{packet}: valid") - m = manifest(packet) - m["packet_version"] = "2.0" - save(packet, m) - assert main(["validate", str(packet), "--json"]) == 1 - report = json.loads(capsys.readouterr().out) - assert report["ok"] is False and report["problems"][0].startswith("schema: packet_version") - - -def test_a_newer_major_version_is_refused(packet): - m = manifest(packet) - m["packet_version"] = "2.0" - save(packet, m) - assert "packet_version" in only_problem(packet) - - -def test_an_unknown_field_is_accepted(packet): - m = manifest(packet) - m["session"]["something_new"] = {"added_in": "1.1"} - save(packet, m) - assert validate(packet).ok - - -def test_a_missing_file(packet): - (packet / "photos" / "p00002.jpg").unlink() - assert "is missing" in only_problem(packet) - - -def test_a_file_whose_size_changed(packet): - with (packet / "streams" / "gyroscope.csv").open("a") as f: - f.write("\n") - assert "bytes, manifest says" in only_problem(packet) - - -def test_a_file_whose_content_changed(packet): - path = packet / "lidar" / "mesh.ply" - data = bytearray(path.read_bytes()) - data[-1] ^= 1 # same size, different bytes - path.write_bytes(bytes(data)) - assert "sha256" in validate(packet).problems[0] - - -def test_an_unlisted_file_is_a_warning_only(packet): - (packet / "notes.txt").write_text("hello") - report = validate(packet) - assert report.ok and "not in the manifest" in report.warnings[0] - - -def test_image_size_must_match_the_manifest(packet): - m = manifest(packet) - buf = io.BytesIO() - Image.new("RGB", (80, 60)).save(buf, "JPEG") - replace_file(packet, m["photos"][0]["image"], buf.getvalue()) - save(packet, m) - assert "80x60" in only_problem(packet) - - -def test_a_rotated_photo_is_refused(packet): - m = manifest(packet) - img = Image.open(packet / m["photos"][0]["image"]["path"]) - exif = Image.Exif() - exif[0x0112] = 6 # "rotate 90° clockwise to display" - buf = io.BytesIO() - img.save(buf, "JPEG", exif=exif.tobytes()) - replace_file(packet, m["photos"][0]["image"], buf.getvalue()) - save(packet, m) - assert "EXIF orientation 6" in only_problem(packet) - - -def test_photos_out_of_time_order(packet): - m = manifest(packet) - m["photos"][0]["t"], m["photos"][1]["t"] = m["photos"][1]["t"], m["photos"][0]["t"] - save(packet, m) - assert any("time goes" in p for p in validate(packet).problems) - - -def test_a_photo_off_its_trajectory(packet): - m = manifest(packet) - m["photos"][1]["pose"][12] += 0.1 # 10 cm along x - save(packet, m) - assert "from the trajectory" in only_problem(packet) - - -def test_a_photo_between_trajectory_samples(packet): - m = manifest(packet) - m["photos"][2]["t"] = 102.3 # after the trajectory's last sample at 102.0, inside the capture - save(packet, m) - assert "no trajectory sample" in only_problem(packet) - - -def test_distance_walked_must_match_the_trajectory(packet): - m = manifest(packet) - m["session"]["capture"]["distance_walked_m"] = 2.5 - save(packet, m) - assert "trajectory walks 2.000" in only_problem(packet) - - -def test_location_without_consent(packet): - m = manifest(packet) - m["session"]["consent"]["location"] = False - save(packet, m) - problems = validate(packet).problems - assert [p.split(" is present")[0] for p in problems] == ["streams.location", "streams.heading"] - - -def test_depth_with_the_wrong_aspect(packet): - m = manifest(packet) - d = m["photos"][0]["depth"] - replace_file(packet, d["map"], np.full((20, 20), 2.5, " dict: - """The fixture as a 1.0 writer would have written it: no 1.1 fields.""" - m["packet_version"] = "1.0" - del m["depth_frames"] - del m["session"]["device"]["mesh_classification_enabled"] - m["lidar"]["planes"] = m.pop("planes") - for plane in m["lidar"]["planes"]: - plane.pop("boundary_m", None) - for p in m["photos"]: - del p["depth"]["confidence"] # optional in 1.0 even for ARKit depth - return m - - -def test_a_1_0_packet_still_validates(packet): - save(packet, as_1_0(manifest(packet))) - report = validate(packet) - assert report.ok, report.problems - assert len(report.warnings) == 1 # the files only 1.1 names are now unlisted - - -def test_arkit_depth_needs_confidence_from_1_1(packet): - m = manifest(packet) - del m["photos"][0]["depth"]["confidence"] - (packet / "depth" / "p00001.conf.u8").unlink() - save(packet, m) - assert "needs confidence (required from 1.1)" in only_problem(packet) - - -def test_estimated_depth_needs_sigma(packet): - m = manifest(packet) - del m["depth_frames"][1]["sigma"] - save(packet, m) - assert only_problem(packet).startswith("schema: depth_frames/1") - - -def test_sigma_must_be_finite_and_sized(packet): - m = manifest(packet) - f = m["depth_frames"][1] - sigma = np.full((f["height"], f["width"]), 0.15, " Date: Sat, 26 Sep 2026 20:02:27 -0500 Subject: [PATCH 5/9] packet: list the validator hazards #22's review found, for the server's validator Co-Authored-By: Claude Opus 5.5 (1M context) --- packet/validator-checks.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/packet/validator-checks.md b/packet/validator-checks.md index 5bc08d93e..94d1decaa 100644 --- a/packet/validator-checks.md +++ b/packet/validator-checks.md @@ -57,3 +57,22 @@ ARKit's own 0 ("none"). **Evidence.** 1.1's `tests/test_checks.py` breaks each of these in turn (index out of range, class 8, a truncated body, a changed property type). The ARKit mesh has not yet been exercised from a device. + +## Hazards the 1.1 review found + +Reviewers of the 1.1 validator (#22) found ways a packet passed that it should not. Each applies to +any validator that reads packets the same way: + +- **Symlinks.** A folder packet whose file is a symlink to an identical file outside it passes + a path-spelling check. Resolve each path and require it to stay inside the packet. +- **Duplicate zip entries.** Python's `ZipFile` reads the last entry of a repeated name, while + other unzip tools take the first, so a bad first copy can hide behind a good second one. Refuse + a zip with repeated names. +- **Truncated JPEGs.** Pillow's `Image.open` reads only the header, so format, size and EXIF can + look right on a truncated file. Decode the pixels (`Image.load()`) before accepting it. +- **Malformed binaries.** Parsing a short mesh or depth body can raise before the length check + runs. Report it as a problem, not an exception. +- **Non-JSON numbers.** `json.loads` accepts `NaN` and `Infinity` by default. Parse with + `parse_constant` raising, so a pose or intrinsic cannot be `NaN`. +- **Stream times outside the capture.** A stream with samples far outside the session's time span + passed with only a warning. Decide whether that is an error. From ad093defb3e6bcbd4c2819777c036f8e87a52683 Mon Sep 17 00:00:00 2001 From: SamGu-NRX Date: Sat, 26 Sep 2026 21:32:39 -0500 Subject: [PATCH 6/9] packet: add the signals lab's 0.5 proposals; describe the checks without the server's paths MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit From #59: the wall plane's source (single raycast yaw 2.2° median, 6.7° p90), span photos, scale references in view, the meter anchor's pose over time, and an optional UWB distance stream. The server's repository is private, so validator-checks.md no longer names its internal files. Co-Authored-By: Claude Opus 5.5 (1M context) --- packet/README.md | 27 +++++++++++++++++++++++++-- packet/validator-checks.md | 11 +++++------ 2 files changed, 30 insertions(+), 8 deletions(-) diff --git a/packet/README.md b/packet/README.md index 7cab25f4c..d2bd9effc 100644 --- a/packet/README.md +++ b/packet/README.md @@ -21,7 +21,10 @@ guess. All are additive, so 0.4 readers keep working. the app knows the wall at the tap: the vertical plane its raycast hit. Proposal: on the `meter` tap, a `meterAnchor` with `transform` (anchor to world in the tap's epoch: origin at the meter, +y up, +z the wall's outward normal, +x along the wall to the right seen from - outside) and `normalSource` (`plane`, `mesh` or `estimated`). + outside) and `normalSource`: `single_raycast` (one plane hit at the tap), `fitted` (fitted + across the walk), `mesh` or `estimated`. The source matters: the signals lab (#59, + `experiments/edge-geometry`) measured the yaw of the app's single-raycast plane at 2.2° median + and 6.7° p90, which at 10 ft along the wall moves a point about 1 ft. 2. **The ground's height at the meter.** 0.4 notes that world y = 0 is the phone's height at session start, not the ground. The app measures the ground where it can: the horizontal plane below the meter. Proposal: `meterAnchor.groundY` (world meters) with `groundSource` @@ -58,5 +61,25 @@ guess. All are additive, so 0.4 readers keep working. state that `boundary` is `[x, z]` in the anchor's frame, and have the validator check it lies within `planeExtent` once the centre and rotation are applied. -Packet 1.1 also had a fixed sharpness score, distance walked and a location-consent flag. They are +8. **Photos that frame one span.** An along-wall edge is 0.9 in p90 when seen within 15° of + face-on and 24 in beyond 45° (#59, `experiments/edge-geometry`), so a distance measured inside + one face-on photo is far better than one chained across the walk. Proposal: a still or + keyframe purpose `span` that names the marks the photo frames. +9. **Scale references in view.** One reference near the meter is not enough: ARKit's scale drifts + 1.8% (robust SD) within a walk (#59, `experiments/drift-anatomy`), so a reference helps where + it appears. Proposal: `references[]` with `kind` (door, brick course, and hand-held ones such + as an ID card or a Letter sheet), nominal size, and its corners in each photo that shows it. + Hand-held references ask the homeowner to hold something; #59 rates them worth a device test + (`experiments/sensor-budget`), and whether the flow may ask is Sam's and Hunter's call. +10. **The meter anchor's pose over time.** ARKit re-estimates an anchor as tracking corrects, so + the anchor pose at tap time goes stale. Proposal: an optional stream of the meter anchor's + world pose at each ARKit update, so the server can see how far the meter frame moved. There + is no measurement of that movement yet. +11. **Distances from a second phone (low priority).** When a second phone with UWB is present, + an optional stream of phone-to-phone distances. #59 (`experiments/drift-anatomy`) finds it + cuts the p90 error at 20 ft from 10.4 in to 7.7 in at 10 cm ranging noise, and to 4.9 in at + 5 cm. + +Already in 0.4, so not proposed: the kind and distance of each tap's hit, a keyframe for every +tap, and feature-point identifiers. Packet 1.1 also had a fixed sharpness score, distance walked and a location-consent flag. They are left out: 0.4's exposure and EXIF, `arkitPoses` and `locationAuthorization` cover them. diff --git a/packet/validator-checks.md b/packet/validator-checks.md index 94d1decaa..f05019a7f 100644 --- a/packet/validator-checks.md +++ b/packet/validator-checks.md @@ -1,11 +1,10 @@ # Three arrival checks for the capture packet validator -The server's validator (`packet/packetgen/validate.py` and `server/contracts` in -`huntertcarver/house-scanning-server`) already checks the schema, every file's size and SHA-256, -rigid poses, plausible intrinsics, epochs, depth and confidence byte counts, and tap replay. These -three checks catch mistakes those do not. Each ran in the packet 1.1 validator (`t3/packet` -`d5439cf`, `packet/validate.py`) with a test that breaks it on purpose. Whether they reach the -server as a pull request or stay a proposal is Sam's and Hunter's call. +Three checks the server's validator does not make at 0.4, each catching a mistake that passes +every other check. They are described here in our own words; nothing is copied from the server's +private code. Each ran in the packet 1.1 validator (`t3/packet` `d5439cf`, `packet/validate.py`) +with a test that breaks it on purpose. Whether they reach the server as a pull request or stay a +proposal is Sam's and Hunter's call. ## 1. Keyframes lie on the pose stream From b2fefa568d80f807b4310bd3f64ecd58c8b20f81 Mon Sep 17 00:00:00 2001 From: SamGu-NRX Date: Sat, 26 Sep 2026 21:36:30 -0500 Subject: [PATCH 7/9] packet: fix the proposal list's formatting Co-Authored-By: Claude Opus 5.5 (1M context) --- packet/README.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/packet/README.md b/packet/README.md index d2bd9effc..06aa8bc7f 100644 --- a/packet/README.md +++ b/packet/README.md @@ -60,7 +60,6 @@ guess. All are additive, so 0.4 readers keep working. and `rotationOnYAxis`, and packet 1.1 found that mixing the two is an easy mistake. Proposal: state that `boundary` is `[x, z]` in the anchor's frame, and have the validator check it lies within `planeExtent` once the centre and rotation are applied. - 8. **Photos that frame one span.** An along-wall edge is 0.9 in p90 when seen within 15° of face-on and 24 in beyond 45° (#59, `experiments/edge-geometry`), so a distance measured inside one face-on photo is far better than one chained across the walk. Proposal: a still or @@ -72,14 +71,15 @@ guess. All are additive, so 0.4 readers keep working. Hand-held references ask the homeowner to hold something; #59 rates them worth a device test (`experiments/sensor-budget`), and whether the flow may ask is Sam's and Hunter's call. 10. **The meter anchor's pose over time.** ARKit re-estimates an anchor as tracking corrects, so - the anchor pose at tap time goes stale. Proposal: an optional stream of the meter anchor's - world pose at each ARKit update, so the server can see how far the meter frame moved. There - is no measurement of that movement yet. + the anchor pose at tap time goes stale. Proposal: an optional stream of the meter anchor's + world pose at each ARKit update, so the server can see how far the meter frame moved. There + is no measurement of that movement yet. 11. **Distances from a second phone (low priority).** When a second phone with UWB is present, - an optional stream of phone-to-phone distances. #59 (`experiments/drift-anatomy`) finds it - cuts the p90 error at 20 ft from 10.4 in to 7.7 in at 10 cm ranging noise, and to 4.9 in at - 5 cm. + an optional stream of phone-to-phone distances. #59 (`experiments/drift-anatomy`) finds it + cuts the p90 error at 20 ft from 10.4 in to 7.7 in at 10 cm ranging noise, and to 4.9 in at + 5 cm. Already in 0.4, so not proposed: the kind and distance of each tap's hit, a keyframe for every -tap, and feature-point identifiers. Packet 1.1 also had a fixed sharpness score, distance walked and a location-consent flag. They are -left out: 0.4's exposure and EXIF, `arkitPoses` and `locationAuthorization` cover them. +tap, and feature-point identifiers. Packet 1.1 also had a fixed sharpness score, distance walked +and a location-consent flag; 0.4's exposure and EXIF, `arkitPoses` and `locationAuthorization` +cover them. From 3792a8ff5137bf774222316886d03e1c4ffb652b Mon Sep 17 00:00:00 2001 From: SamGu-NRX Date: Sat, 26 Sep 2026 23:00:56 -0500 Subject: [PATCH 8/9] packet: mark the proposals unapproved, state the studies' limits and the checks' limits From #22's review and Codex review: the eleven items are proposals, not contract; #59's numbers name their proxy, dataset and simulation; the retired sharpness and consent fields are not claimed to be covered; window operability is evidence for the rules; the boundary frame goes in a new field. In the proposed checks: positions only (not rotation), shape not alignment for depth, non-negative indices and finite vertices for the mesh, finiteness of every parsed number. The pointer no longer lists the intake routes. Co-Authored-By: Claude Opus 5.5 (1M context) --- packet/README.md | 49 ++++++++++++++++++++++---------------- packet/validator-checks.md | 36 ++++++++++++++++++---------- 2 files changed, 53 insertions(+), 32 deletions(-) diff --git a/packet/README.md b/packet/README.md index 06aa8bc7f..ddbfd5f24 100644 --- a/packet/README.md +++ b/packet/README.md @@ -2,9 +2,9 @@ The capture packet is how a scan leaves the phone. It is specified by the server team's capture packet spec, `formatVersion` 0.4, together with its intake API. The spec is -`docs/capture-packet-spec.md` in the private repository `huntertcarver/house-scanning-server`; -the API is its Appendix D (`/v1/captures`, `/files`, `/files:commit`, `/finalize`). That spec is -the one contract; this folder does not define a format of its own. +`docs/capture-packet-spec.md` in the private repository `huntertcarver/house-scanning-server`, +and the intake API is its Appendix D. That spec is the one contract; this folder does not define +a format of its own. This folder holds the app side's proposals for the next revision (0.5): what the app records that 0.4 has no field for, and three arrival checks for the server's validator @@ -14,7 +14,10 @@ retired; it is still readable at `t3/packet` commit `d5439cf`. ## Proposals for 0.5 Each item is something the app knows at capture time and the server would otherwise have to -guess. All are additive, so 0.4 readers keep working. +guess. All eleven are unapproved proposals. None is part of the current contract, and the app must +not send them until the contract's owners adopt them in a version and with a compatibility rule +of their choosing. The study numbers cited motivate device tests; none is measured phone +performance. 1. **The meter anchor's orientation.** 0.4 records the meter tap's point, not the wall it sits on. Every placement distance is measured along that wall from the meter (C1's `s = 0`), and @@ -23,8 +26,10 @@ guess. All are additive, so 0.4 readers keep working. the meter, +y up, +z the wall's outward normal, +x along the wall to the right seen from outside) and `normalSource`: `single_raycast` (one plane hit at the tap), `fitted` (fitted across the walk), `mesh` or `estimated`. The source matters: the signals lab (#59, - `experiments/edge-geometry`) measured the yaw of the app's single-raycast plane at 2.2° median - and 6.7° p90, which at 10 ft along the wall moves a point about 1 ft. + `experiments/edge-geometry`) found a yaw error of 2.2° median and 6.7° p90 for a stand-in for + the app's plane, a 0.3 m plane fitted to depth on ETH3D photos, not a device raycast. At that + p90, 10 ft along the wall moves a point about 1 ft. A per-capture yaw uncertainty is worth + adding once a device study measures it. 2. **The ground's height at the meter.** 0.4 notes that world y = 0 is the phone's height at session start, not the ground. The app measures the ground where it can: the horizontal plane below the meter. Proposal: `meterAnchor.groundY` (world meters) with `groundSource` @@ -34,7 +39,8 @@ guess. All are additive, so 0.4 readers keep working. homeowner gave about it: - whether a wall end is a real end (something blocks it) or unexplored (it turns a corner, or no answer), which decides whether the server may reject for want of space; - - whether a window opens, which decides whether the opening clearance applies; + - whether a window opens, which the rules may use (whether a fixed window is exempt from the + opening clearance is the rules' choice, not the app's); - which corner taps belong to the same door or window. Proposal: `taps[].attrs` with `endKind` (`limit`, `unexplored`) and `operable`, and @@ -58,15 +64,17 @@ guess. All are additive, so 0.4 readers keep working. 7. **The frame of a plane's boundary.** 0.4 lists `planes[].boundary` without saying its frame. ARKit's `boundaryVertices` are relative to the anchor transform, not to `planeExtent`'s centre and `rotationOnYAxis`, and packet 1.1 found that mixing the two is an easy mistake. Proposal: - state that `boundary` is `[x, z]` in the anchor's frame, and have the validator check it lies - within `planeExtent` once the centre and rotation are applied. -8. **Photos that frame one span.** An along-wall edge is 0.9 in p90 when seen within 15° of - face-on and 24 in beyond 45° (#59, `experiments/edge-geometry`), so a distance measured inside - one face-on photo is far better than one chained across the walk. Proposal: a still or + carry the anchor-frame boundary in a new field whose frame is stated, rather than giving the + existing `boundary` a meaning an existing reader might not share, and have the validator check + it lies within `planeExtent` once the centre and rotation are applied. +8. **Photos that frame one span.** With learned depth on ETH3D photos, an along-wall edge came to + 0.9 in p90 when seen within 15° of face-on and 24 in beyond 45° (#59, + `experiments/edge-geometry`), so a distance measured inside one face-on photo is likely far + better than one chained across the walk. Proposal: a still or keyframe purpose `span` that names the marks the photo frames. -9. **Scale references in view.** One reference near the meter is not enough: ARKit's scale drifts - 1.8% (robust SD) within a walk (#59, `experiments/drift-anatomy`), so a reference helps where - it appears. Proposal: `references[]` with `kind` (door, brick course, and hand-held ones such +9. **Scale references in view.** One reference near the meter may not be enough: on MARViN's + walks ARKit's scale varied 1.8% (robust SD) within a walk (#59, `experiments/drift-anatomy`), + so a reference helps where it appears. Proposal: `references[]` with `kind` (door, brick course, and hand-held ones such as an ID card or a Letter sheet), nominal size, and its corners in each photo that shows it. Hand-held references ask the homeowner to hold something; #59 rates them worth a device test (`experiments/sensor-budget`), and whether the flow may ask is Sam's and Hunter's call. @@ -75,11 +83,12 @@ guess. All are additive, so 0.4 readers keep working. world pose at each ARKit update, so the server can see how far the meter frame moved. There is no measurement of that movement yet. 11. **Distances from a second phone (low priority).** When a second phone with UWB is present, - an optional stream of phone-to-phone distances. #59 (`experiments/drift-anatomy`) finds it - cuts the p90 error at 20 ft from 10.4 in to 7.7 in at 10 cm ranging noise, and to 4.9 in at - 5 cm. + an optional stream of phone-to-phone distances. In #59's simulation on MARViN walks + (`experiments/drift-anatomy`), a simulated range cut the p90 error at 20 ft from 10.4 in to + 7.7 in at 10 cm ranging noise, and to 4.9 in at 5 cm. Already in 0.4, so not proposed: the kind and distance of each tap's hit, a keyframe for every tap, and feature-point identifiers. Packet 1.1 also had a fixed sharpness score, distance walked -and a location-consent flag; 0.4's exposure and EXIF, `arkitPoses` and `locationAuthorization` -cover them. +and a flag for consent to share location. They are not proposed here: distance walked can be +derived from the pose stream, and whether a sharpness score or a sharing-consent record belongs +in the contract is for its owners. diff --git a/packet/validator-checks.md b/packet/validator-checks.md index f05019a7f..9e3829ef4 100644 --- a/packet/validator-checks.md +++ b/packet/validator-checks.md @@ -1,8 +1,9 @@ # Three arrival checks for the capture packet validator -Three checks the server's validator does not make at 0.4, each catching a mistake that passes -every other check. They are described here in our own words; nothing is copied from the server's -private code. Each ran in the packet 1.1 validator (`t3/packet` `d5439cf`, `packet/validate.py`) +Three checks we propose for the server's validator, each catching a mistake that each part of a +packet can pass on its own. We read the server's spec and parts of its code but did not audit its +whole validator, so some of these may already exist there. They are described here in our own +words; nothing is copied from the server's private code. Each ran in the packet 1.1 validator (`t3/packet` `d5439cf`, `packet/validate.py`) with a test that breaks it on purpose. Whether they reach the server as a pull request or stay a proposal is Sam's and Hunter's call. @@ -17,21 +18,30 @@ clocks: for example, keyframe poses converted to another frame while the stream timestamps taken from `Date` on one side and uptime on the other. Every other check passes such a packet, because each part is valid on its own. +**Limits.** It compares positions only: a keyframe with the right position and a wrong rotation +(an inverted transform, other camera axes) passes. A rotation tolerance needs a measured basis +first. Packet 1.1 had one world per packet, so matching within an epoch is part of the proposal, +not something 1.1 tested. + **Evidence.** On the ADVIO sample, 79 keyframes matched their stream rows exactly. Moving one keyframe 10 cm, or its timestamp past the stream's end, fails the check (1.1 tests `test_a_photo_off_its_trajectory` and `test_a_photo_between_trajectory_samples`). -## 2. Depth is aligned to its keyframe +## 2. Depth has its keyframe's shape -A depth map covers its keyframe's field of view at lower resolution: +A depth map covers its keyframe's field of view at lower resolution, so: - its `w / h` equals the image's within 1%, and it is no larger than the image; - values are finite and non-negative, with 0 for no measurement, and at least 1% are measured; - confidence values are at most 2. -**What it catches.** A depth map from another orientation or crop (for example a portrait-rotated -map with a landscape image), which the byte-count check passes because `w × h` is unchanged. It +**What it catches.** A depth map rotated 90° or cropped to another aspect (for example a portrait +map with a landscape image), which a byte-count check passes because `w × h` is unchanged. It also catches NaN written for "no reading", and an all-zero map. +**Limits.** It checks shape, not alignment. A map flipped 180°, mirrored, or taken from another +keyframe with the same size passes. Proving alignment would need orientation and capture-link +fields the contract does not have. + **Evidence.** On ETH3D electro with laser-rendered depth, ETH3D's own 3-D points reprojected through the packet's poses to a median 0.65 to 1.01 px, and the depth agreed with them to a median 0.2 to 0.6%. The aspect, NaN and confidence tests in 1.1's `tests/test_packet.py` each fail their @@ -44,12 +54,13 @@ this check proposes one layout as well: - `float x, y, z` per vertex, and per face a `uchar` count followed by that many `int` indices, then an optional `uchar classification`; - the body length equals exactly what the vertex and face counts imply; -- every face is a triangle, and every index is below the vertex count; +- vertex and face counts are non-negative, and every vertex coordinate is finite; +- every face is a triangle, and every index satisfies `0 <= index < vertex_count`; - classification values are 0 to 7 (`ARMeshClassification`), and all 0 when `mesh.classified` is false. -**What it catches.** A truncated upload that still hashes as sent, an index past the vertex list -(which crashes or corrupts a reconstruction reader), `double` coordinates or `uint` counts from a +**What it catches.** A truncated upload that still hashes as sent, a negative index or one past +the vertex list (which crashes or corrupts a reconstruction reader), `NaN` or infinite vertices, `double` coordinates or `uint` counts from a different exporter, and classes written when classification was off, which cannot be told from ARKit's own 0 ("none"). @@ -71,7 +82,8 @@ any validator that reads packets the same way: look right on a truncated file. Decode the pixels (`Image.load()`) before accepting it. - **Malformed binaries.** Parsing a short mesh or depth body can raise before the length check runs. Report it as a problem, not an exception. -- **Non-JSON numbers.** `json.loads` accepts `NaN` and `Infinity` by default. Parse with - `parse_constant` raising, so a pose or intrinsic cannot be `NaN`. +- **Non-finite numbers.** `json.loads` accepts `NaN` and `Infinity` by default, and parses the + valid JSON number `1e400` as infinity without calling `parse_constant`. Check every parsed + number with `math.isfinite`, so no pose or intrinsic is `NaN` or infinite. - **Stream times outside the capture.** A stream with samples far outside the session's time span passed with only a warning. Decide whether that is an error. From 89372a0a4143918f0d0e67c7e2bbae5ac60e8023 Mon Sep 17 00:00:00 2001 From: SamGu-NRX Date: Sat, 26 Sep 2026 23:24:32 -0500 Subject: [PATCH 9/9] packet: tighten three proposals after the Codex review of 3792a8f - Hazards: a huge JSON integer parses as an int, and math.isfinite raises on it; check integers against their field's range and report failures as problems. - Proposal 10: each anchor-pose sample needs its timestamp and epoch. - Proposal 11: the #59 result assumes a still second phone at a known place; the proposal requires that, a time per range and the range's uncertainty. Co-Authored-By: Claude Opus 5.5 (1M context) --- packet/README.md | 15 +++++++++++---- packet/validator-checks.md | 9 ++++++--- 2 files changed, 17 insertions(+), 7 deletions(-) diff --git a/packet/README.md b/packet/README.md index ddbfd5f24..98fa39619 100644 --- a/packet/README.md +++ b/packet/README.md @@ -80,12 +80,19 @@ performance. (`experiments/sensor-budget`), and whether the flow may ask is Sam's and Hunter's call. 10. **The meter anchor's pose over time.** ARKit re-estimates an anchor as tracking corrects, so the anchor pose at tap time goes stale. Proposal: an optional stream of the meter anchor's - world pose at each ARKit update, so the server can see how far the meter frame moved. There - is no measurement of that movement yet. + world pose at each ARKit update, so the server can see how far the meter frame moved. Each + sample needs its timestamp, on the pose stream's clock, and its epoch: the epoch tells a + world reset from a correction within one world, and the timestamp ties the sample to the + keyframes and taps around it. There is no measurement of that movement yet. 11. **Distances from a second phone (low priority).** When a second phone with UWB is present, an optional stream of phone-to-phone distances. In #59's simulation on MARViN walks - (`experiments/drift-anatomy`), a simulated range cut the p90 error at 20 ft from 10.4 in to - 7.7 in at 10 cm ranging noise, and to 4.9 in at 5 cm. + (`experiments/drift-anatomy`, question A4), a second phone left at the meter cut the p90 error + at 20 ft from 10.4 in to 7.7 in at 10 cm ranging noise, and to 4.9 in at 5 cm. That result + depends on the second phone staying still at a known place, so the proposal requires three + things: where the second phone stands (at the meter, or a stated position) and that it did + not move; a time for each range on the capture's clock, so it matches a pose; and the range's + uncertainty. Ranges from a second phone that moves, or whose place is unknown, are not what + the study simulated. Already in 0.4, so not proposed: the kind and distance of each tap's hit, a keyframe for every tap, and feature-point identifiers. Packet 1.1 also had a fixed sharpness score, distance walked diff --git a/packet/validator-checks.md b/packet/validator-checks.md index 9e3829ef4..37a8cf041 100644 --- a/packet/validator-checks.md +++ b/packet/validator-checks.md @@ -82,8 +82,11 @@ any validator that reads packets the same way: look right on a truncated file. Decode the pixels (`Image.load()`) before accepting it. - **Malformed binaries.** Parsing a short mesh or depth body can raise before the length check runs. Report it as a problem, not an exception. -- **Non-finite numbers.** `json.loads` accepts `NaN` and `Infinity` by default, and parses the - valid JSON number `1e400` as infinity without calling `parse_constant`. Check every parsed - number with `math.isfinite`, so no pose or intrinsic is `NaN` or infinite. +- **Out-of-range numbers.** `json.loads` accepts `NaN` and `Infinity` by default, and parses the + valid JSON number `1e400` as infinity without calling `parse_constant`. It parses a 400-digit + integer as an exact `int`, on which `math.isfinite` raises `OverflowError` rather than + returning `False`. So check floats with `math.isfinite`, check integers against the range their + field allows (a count, an index, a size) before any conversion to float, and report a value + that fails either check, or cannot be converted, as a problem, not an exception. - **Stream times outside the capture.** A stream with samples far outside the session's time span passed with only a warning. Decide whether that is an error.