Repository navigation
experiments: live dots, a macOS prototype of the holographic capture view - #86
Merged
Merged
Conversation
A Swift package in experiments/live-dots. LiveDotsCore fuses the synthetic wall fixture's depth into 5 cm voxels, marks edges from creases, seen-empty boundaries and a voxel-scale colour gradient, and times each dot's birth and evidence rise. LiveDots draws the replay in a phone-sized window with Metal, and exports stills and frames at 1170 x 2532. The no-LiDAR mode is simulated from the same depth. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Dots now grow from 2.5 pt (flat) and 3.5 pt (edge) at 2.5 m to 4 and 6 pt at 1 m, and every dot carries an additive halo that holds its colour: a light blue glow around the Hologram core, violet on the bin. The simulated no-LiDAR mode keeps points seen in 2 of the last 6 keyframes for 4 s, and the oblique-edge opacity cap applies only to near-vertical surfaces. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The no-LiDAR simulation now takes feature candidates at a 0.1 gradient instead of the edges' 0.2, because ARKit's raw feature points land on any high-contrast texture, brick joints included. Edge classification stays at 0.2. Live points in view during the walk rise from 34 to 84 to 96 to 186. results/ holds the four stills downscaled to 800 px and the LiDAR video, all rendered from the synthetic fixture. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Warning Review limit reachedNext included review available in 59 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: ⛔ Files ignored due to path filters (19)
📒 Files selected for processing (55)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
hologram is the current look. constellation draws edge dots only, larger, joined to their two nearest edge neighbours on the same surface by hairline links that are born and fade with them. ember draws every dot amber when a keyframe observes it and cools it to the hologram colours over 6 s. LiveDots takes --scheme and the controls gain a Look picker; make-video.sh renders one LiDAR video per scheme, and results/ holds 800 px copies. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
LiDAR re-measures every voxel in view on every keyframe, so re-warming on observation kept the whole screen amber. An ember dot now cools over the 6 s after its birth and never re-warms, including when it turns from flat to edge, so the newest part of the map glows. The last-seen bookkeeping the old rule needed is gone. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Fog: every live dot splats a soft disc into a 96 x 208 reveal mask, which is blurred, smoothstepped and eased with a lag (600 ms lifting, 120 ms returning) before a chalk fog with slow domain-warped noise and a depth term is drawn over the camera image and under the dots. --fog on|off|veil keeps the old flat veil for comparison. Boxes: the fixture's meter, gas meter, door and window, simulated from the scene. The meter is solid Signal blue from its first view; the others appear dashed at 60% in view and settle solid when seen face-on with both sides in frame. Guidance: point at the meter, hold still while the ring fills (the third keyframe now holds 1.15 s), walk left, walk right, tilt back down with a chevron, done, plus a distance-from-meter badge during the walk. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2 tasks done
3 of 4 tasks
Owner
Author
 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
AidenJohnston
marked this pull request as ready for review
September 27, 2026 15:27
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
|
Note Review skipped Codemetrics is installed on this repository, but it hasn't been added to your team yet. To enable AI code reviews for this repo, go to your dashboard and add house-scanning-master to your team's repositories. |
This was referenced Oct 13, 2026
SamGu-NRX
pushed a commit
that referenced
this pull request
Oct 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
The camera screen on
t3/ios-map3dcovers the wall the phone has not seen with a frosted haze that lifts as the walk proceeds. Sam asked for the opposite: nothing where the phone has not looked, and dots that sit on the edges and surfaces it has detected. This addsexperiments/live-dots, a macOS prototype of that view, so the rules could be seen and tuned before anyone builds them intoMap3DOverlay.It is a Swift package with two targets.
LiveDotsCoredecodes the synthetic LiDAR replay fixture from #10 (fetch-fixtures.shpulls it out oforigin/t3/ios-mvf), unprojects each keyframe's 256 × 192 depth into 5 cm voxels, marks a voxel as an edge when its normal creases, when it borders space the camera saw as empty, or when a voxel-scale image gradient says the wall is painted there (the fixture's meter, door and window are paint, not relief), and times each dot's birth and evidence rise.LiveDotsplays the 41 keyframes in a phone-shaped window and draws the dots with Metal point sprites over the camera image, with a ring that fills as the requirement zone is covered and one instruction line. A "no LiDAR" mode simulates ARKit's feature points from the same depth and image texture; it is a simulation and says so.--exportrenders frames offscreen at 1170 × 2532 andmake-video.shstreams them into ffmpeg.The rules the prototype implements are listed at the top of
Sources/LiveDotsCore/DotField.swift, and the design they come from is in the direction document (~/house-scanning-data/reports/experience/direction.html, section 4).Why
Two things had to be seen rather than argued. Whether a pixel-scale image gradient could separate painted edges from brick joints (it could not: 31% of outline voxels and 19% of brick voxels qualified; a pyramid level at voxel scale gives 100% against 1%), and whether dots sized by 1/distance would read at a walking distance of 2.6 m (they did not; every dot hit the floor and the field read as speckle, so sizes now floor at 2.5 m and halos carry the colour).
UI changes
New screen, no before. Stills and videos rendered by the package itself, from the synthetic fixture:
Three looks behind one switch (
--scheme hologram|constellation|ember, or the Look picker), so they can be compared on the same replay. Each has its own video inresults/.Fog that the dots replace (
--fog on, default), with recognised boxes and guidance states driven by the replay. Borrowings are credited in the fog section's header inShaders.swift.Videos: fog-lidar.mp4, fog-nolidar.mp4, fog-reduce-motion.mp4.
Video of the LiDAR mode, 30 frames per keyframe-second so the births are visible: results/live-dots-lidar.mp4. The no-LiDAR and Reduce Motion videos are in the report folder, not the repository.
Verification
swift testinexperiments/live-dots: 33 tests pass (unprojection against a hand-computed point, hash jitter bound and determinism, edge classification on a crease and on a seen-empty boundary, evidence to opacity, the 15° view dedupe, the 30° face-on cap on wall edges and its absence on ground edges, the ring fraction, and fixture errors that name the path and byte counts).swift buildwith no warnings, Swift 6 language mode.LiveDots --benchmark: 6,000 dots with halos, 0.54 ms GPU per 1170 × 2532 frame on an M4 Pro (p95 0.91 ms), 1.0 ms with fog on. Not measured on a phone.Risk
None to the app: nothing outside
experiments/live-dotschanges, and the package depends on no other target. Building this into #21 needs two additions toMap3D: an iterator of observed surface voxels with per-frame deltas, and a 5 cm point set beside the 10 cm map (direction document, section 4.7).Checklist
🤖 Generated with Claude Code