diff --git a/experiments/evals/METHODS.md b/experiments/evals/METHODS.md new file mode 100644 index 000000000..e2ad69ba6 --- /dev/null +++ b/experiments/evals/METHODS.md @@ -0,0 +1,420 @@ +# Evals on real data: methods and results + +The front door is [README.md](README.md). This file keeps each question's method, pass criteria and results. + +## Questions + +1. How far is ARKit's tracking off after walking 3, 10, 20 and 30 ft outdoors? The untested default for an AR tap is plus or minus 0.3 ft (3.6 in; `docs/00`, Conventions the code relies on). +2. How accurately can phone photos reconstruct a real building wall, with the model's own scale or with one taped distance, from one photo or several? +3. Can the phone's AR poses fix the learned models' scale, well enough for about 4 in at p90 on 1 to 3 m spans, the error that decides a 3 ft clearance? +4. Which photos are worth keeping? +5. On a real field session with a tape survey, how do the phone's AR taps and the learned-depth methods compare, and what is the phone's AR scale error? +6. How large is a current iPhone's ARKit scale error, from public data? +7. Does the app's coverage map claim wall and ground that no photo saw? + +Every dataset evaluation uses real captures. Two checks add synthetic inputs, and are labelled where they appear: section 3's pose-prior rows degrade ETH3D's true poses with simulated AR-like error, and `make field-dryrun` scores a synthetic session to prove the field command. The unit tests of the metric code also use synthetic cases. + +## Answers + +What the ETH3D numbers measure: the error in the distance between two scanned surface points, against a laser scan, reported separately for surface interiors, vertical surfaces (walls, fences) and depth edges (window frames, equipment, fence edges). Clearance endpoints are often edges, and edges score worst. Every "p90" below counts failures (no prediction, or a taped reference that no scale could match) as infinitely wrong. + +1. **ARKit drift: far worse than plus or minus 0.3 ft beyond a few feet, on the only phone measured.** On a 2018 iPhone 6s outdoors, ARKit and ARCore on the same rig disagree about a walked distance by a median 2.8 in after 3 ft (p90 8.2 in), 8.6 in after 10 ft (p90 22.3) and 25.7 in after 30 ft (p90 60.1). That is a disagreement between two trackers, not a bound on ARKit's error, but most of it is one steady offset that GPS attributes to ARKit: it reads distances 5% to 17% short. Position error, which also counts sideways drift, is larger against ADVIO's truth: p90 18.6, 55.7, 94.6 and 132.6 in after 3, 10, 20 and 30 ft. That is two to three times the server's allowance of 0.16 ft per ft, though the truth's own heading and position noise inflate it. One of four walks lost tracking entirely. +2. **Photos alone: no.** Each model's own metric scale is off. Within 6 m of the camera, MoGe-2 reads 4% to 12% long and Depth Anything 3 metric 7% to 11% short: median errors of 6 to 8 in on 1 to 3 m spans, p90 about 20 in. +3. **One photo plus one taped distance: good median, loose tail, worst on walls and edges.** A single MoGe-2 photo scaled by a taped 1 to 3 m distance is off a median 1.6 in on 1 to 3 m spans (electro, within 6 m). The p90 is 9.3 in on surfaces overall, 13.6 in on vertical surfaces and 25.3 in at edges. That settles only clear-cut placements. +4. **More photos with true poses and no rescale: worse.** On the same 7 seed groups and pairs, per-photo depth averaged over 1, 2, 4 and 8 photos with the true poses gives, with one tape, medians of 1.4, 2.0, 2.2 and 2.8 in and p90s of 5.2, 8.3, 10.0 and 11.9 in on 1 to 3 m spans. Each photo carries its own scale error, and averaging mixes them. +5. **AR poses fixing scale: not to about 4 in, even at an assumed 2% pose error.** Rescaling each photo's MoGe-2 depth to points triangulated with the poses, then fusing 8 photos, gives these p90s on 1 to 3 m spans (electro, within 6 m, no tape, 95% intervals over seed groups and 5 pose-noise draws): + - exact poses: 4.0 in [3.3, 4.8] overall, 2.8 in [2.0, 4.6] on walls; + - an assumed modern phone (2% scale error, 1 cm and 0.1° of noise): 5.6 in [4.6, 6.7] overall, 5.0 in [3.7, 6.9] on walls; + - the errors measured on the 2018 phone: 14.6 in [10.6, 18.7] overall, 10.8 in [8.6, 13.1] on walls. + + These assume perfect feature matching across photos; without it, walls reach 6.3 in even with exact poses. Edges stay at 8 in or worse everywhere. The pose's scale error passes straight into every length. MapAnything given the poses fixes its scale on one scene but not the other, and leaves 10% to 20% of pairs without a prediction. + + **Verdict:** neither way reaches about 4 in at p90 with realistic pose error. An earlier single noise draw showed walls at 3.9 in; with five draws that was the lucky end of the spread. +6. **Which photos to keep: the close ones.** Keeping only photos with the wall within 6 m cuts a single photo's p90 from 11.4 to 6.8 in on 1 to 3 m spans, and from 38.7 to 14.8 in on 3 to 10 m spans. Viewing angle barely mattered. +7. **The field session: not measured yet.** `make field` (section 5) puts the phone's AR taps and three learned-depth rows in one table against tomorrow's tape survey, with the phone's AR scale error beside it. It runs end to end on the ADVIO replay and on a synthetic survey, but the numbers need the real session. +8. **A current iPhone's ARKit scale: within 2% of the only public reference, which cannot itself be checked to 2%.** On MARViN's 35 outdoor walks (iPhone 14 Pro Max, ARKit 6, 45 to 255 m each), ARKit's scale matched the dataset's COLMAP reference within 2% on 30. Scene by scene the median walk reads +0.3%, +1.2% and −1.7%. So section 3's modern_assumed 2% is plausible for this phone, though even that setting leaves walls at a p90 of 5.0 in [3.7, 6.9]. It is not proven: that reference gets its meters from its authors, not from a tape or a laser, and GPS can only check it to several percent. The phone also has LiDAR, so it may track better than the LiDAR-less phones we target. Its position error p90 over trusted walks is 8.6, 13.4 and 18.5 in after 10, 20 and 30 ft (50.0 in at 30 ft counting the three doubtful walks), inside the server's 0.16 ft per ft allowance. The field tape test stays decisive (section 6). + +9. **Does the app's coverage map claim surface no photo saw? Yes, on the one wall tested: 1.1 ft of 19.3 ft claimed (6%), against a bar of 0.5 ft.** All of it sits behind equipment standing in front of the wall (a scaffold's footings and cables), because the coverage code checks range, angle and image bounds but not occlusion (`CoverageMap.swift` lines 241 to 248 at beede15). The ground band and facade could not be tested: ETH3D's photographers stood too far from the walls for the app to credit any ground, or any of facade (section 7). Requiring views farther apart or at wider angles does not fix it without discarding good wall (6 to 8.5 ft here). A depth test against true depth does, which needs LiDAR (section 7b). + +10. **The app's 3D map (Map3D), LiDAR path, ideal depth: no wall claimed that no photo saw, where `CoverageMap` claimed 1.1 ft.** It misses 8.9 ft, 5.5 ft of it behind pilasters that stand out past its 10 cm face window. Its wall line runs 1.76° off and 4.8 in out at the meter. The non-LiDAR path, ground and overhead are untested, and its one facing claim sits below the truth's resolution (section 7c). + +## Reproduce + +From a clean checkout. Prerequisites: +- **Everything:** [uv](https://docs.astral.sh/uv/) 0.11.15 or newer below 0.12 (`tool.uv.required-version`), which installs Python 3.12; git, with network access for the dataset downloads; at least 6 GB of free disk before a download. +- **Model runs** (`make recon-models`, `make pose-priors`, `make field`): macOS on Apple silicon, for MPS. They were measured on a 24 GB M4 Pro, with every process kept near or under 4 GB. +- **The app's coverage code** (`make coverage`, `make coverage-options`, `make map3d`): macOS 26 and Swift 6.2 or newer (Xcode 26), because HouseScanKit and the drivers declare `.macOS(.v26)` and swift-tools-version 6.2. Each target checks out the app code read-only at its pinned commit, as a git worktree under `~/house-scanning-data/evals/`. +- **Linux** runs everything except the model runs and the Swift drivers. + +Reproducibility limit: MapAnything, and MoGe-2's backbone loader, fetch DINOv2 at first run, the code from github.com/facebookresearch/dinov2 at branch `main` through `torch.hub` and weights from dl.fbaipublicfiles.com. Neither is pinned by these evals. The copies cached under `~/house-scanning-data/evals/torch-cache` are what the published results used. + + +```sh +cd experiments/evals +uv sync +make test # metric code against hand-computed cases, lint, format +make drift # ADVIO download (about 800 MB) -> results/advio_drift.md +make replay # the replay session -> ~/house-scanning-data/replays/ +make recon # ETH3D download (about 2.4 GB), MoGe-2 and Depth Anything 3 runs -> results/eth3d_recon.md +make sensitivity # after recon -> results/eth3d_visibility_sensitivity.md +make pose-priors # after recon -> results/pose_priors.md +make frames # after recon -> results/frames.md +make field SESSION=... TRUTH=... MAP=... # a field session, section 5; PR #4's harness is checked out at a pinned commit +make modern-arkit # MARViN pose files (about 3 MB) -> results/modern_arkit.md, section 6 +make coverage # after recon-data; the app's coverage code at a pinned commit -> results/coverage.md, section 7 +make coverage-options # after coverage -> results/coverage_options.md, section 7b +make map3d # after coverage; the app's Map3D at a pinned commit -> results/map3d.md, section 7c +``` + +Data lives in `~/house-scanning-data/` (override with `HOUSE_SCANNING_DATA`). `evals/datasets.py` checks each archive's size and sha256, unpacks only what the evals read, deletes the archive, and refuses to download with less than 6 GB free. Once every expected file is in place it writes a completion record tied to the archive's sha256; a later run skips the download only if that record matches and every recorded file is still there at its recorded size, so an interrupted unpack is redone. Both datasets are licensed for non-commercial research: they measure accuracy here and are never committed, redistributed or used for training. + +Every process stays near or under 4 GB, because the machine is shared. MapAnything's 1.2 B parameters are loaded with their transformer weights in bf16 (`models/map_anything.py`), and even then only 2 and 4 views at 392 px fit (peak 4.05 GB). An earlier run at its native 518 px, with fp32 weights, read 10% to 33% short, where the 392 px run reads 17% long on electro's near walls and 14% short on facade. So its metric scale depends on input resolution, and neither run is a stable baseline. + +## Datasets + +| Dataset and license | File | URL | Bytes | sha256 | +| --- | --- | --- | --- | --- | +| ADVIO, CC BY-NC 4.0 | advio-20.zip | https://zenodo.org/record/1476931/files/advio-20.zip | 195538352 | be21154394df09d6ecebe1894062f290bb53d74a5c6ccbca4b3a69f91ee2c8a2 | +| | advio-21.zip | https://zenodo.org/record/1476931/files/advio-21.zip | 209791503 | fb8a1cf3f645bbd9ea848e66924e83ae75b5cd18d632986e5629af1eecb58570 | +| | advio-22.zip | https://zenodo.org/record/1476931/files/advio-22.zip | 254664089 | 96c7212fc9cb610a88ba9fa62ac2581f0cdae49e80007888698ee2cf5ebd56f4 | +| | advio-23.zip | https://zenodo.org/record/1476931/files/advio-23.zip | 134340011 | 726d9b80d30036a8585f6cecaae385236b689cc4b8b19be6362c6df223825dc1 | +| | iphone-04.yaml (calibration, sequences 20 to 23; copied into `evals/advio.py`) | https://raw.githubusercontent.com/AaltoVision/ADVIO/master/calibration/iphone-04.yaml | | 6f312ace74f23e41b755fa40829e30a4f96e8abd3c28e6bdbac2e5e39f960813 | +| ETH3D, CC BY-NC-SA 4.0 | facade_dslr_undistorted.7z | https://www.eth3d.net/data/facade_dslr_undistorted.7z | 1252088400 | 046e577388db0633eeb2d8d72da6a2de53857434b4a7a98e4c97485795f9ce82 | +| | facade_dslr_scan_eval.7z | https://www.eth3d.net/data/facade_dslr_scan_eval.7z | 176517224 | d686462d417fbea010d0021f918bec8b881444929ec06e3bd4e8a5230ebf59a8 | +| | facade_dslr_occlusion.7z | https://www.eth3d.net/data/facade_dslr_occlusion.7z | 144292883 | 26a9d2076ba16b7f8e24311579c47b0b7b826b180906ced20fa1cba8f3dd130d | +| | electro_dslr_undistorted.7z | https://www.eth3d.net/data/electro_dslr_undistorted.7z | 514345623 | 0d2bc31fec0032b8fb20703abba8e45ef7a395c13d2d3476adc1f4dd4ffd7d8b | +| | electro_dslr_scan_eval.7z | https://www.eth3d.net/data/electro_dslr_scan_eval.7z | 250703286 | 5ca10a73e7da0e3c511e255bba5656cca4c372dc00c1417c97b75228a5bb3bac | +| | electro_dslr_occlusion.7z | https://www.eth3d.net/data/electro_dslr_occlusion.7z | 47214879 | cc25a22de9fc27251a5178454c24785671a2b2c199756a0d03297413290830b3 | + +MARViN (iPhone 14 Pro Max; https://github.com/XRIM-Lab/MarViN) states no data license, so it is used only to measure accuracy and never redistributed. Only its pose, GPS and ground-truth text files are fetched, from its public Google Drive folder. The 89 files the results used are pinned by path and sha256 in [evals/marvin_manifest.json](evals/marvin_manifest.json): only those are downloaded and scored, a missing or changed one stops the run, and other walks on disk are never read. + +Model checkpoints (MIT and Apache-2.0) are pinned by revision and checked against Hugging Face's sha256 (`models/README.md`). + +## 1. ARKit drift outdoors (ADVIO) + +[results/advio_drift.md](results/advio_drift.md), `make drift`. + +**Data.** ADVIO's four outdoor walks (sequences 20 to 23, 3 to 6 minutes and 380 to 510 m each), recorded in 2018 on a rig carrying an iPhone 6s (ARKit 1.0, video, GPS) and a Google Pixel running ARCore. The ground truth is an inertial track pinned to fix points marked on a map. + +**Method.** From every half second of a walk, the walk continues until exactly 3, 10, 20 or 30 ft of path has been covered. The error is ARKit's straight-line displacement minus the reference's. It needs no heading alignment, and it is what a span measured by walking it would be off by. + +**Why ARKit is not scored against ADVIO's ground truth alone.** Its scale is wrong in walks 20 and 21. ARKit covers only 0.70 and 0.75 of the truth's distance there, which is why ARKit's path totals 345 m against the truth's 474 m in walk 20. GPS scales the truth by 0.835 and 0.799 (95% intervals 0.81 to 0.86 and 0.78 to 0.82), and ARCore agrees (0.80 and 0.82). In walks 22 and 23 both agree with the truth's scale. The fix points explain it: walks 20 and 21 were pinned on one map at 0.0422 m per map pixel, and 22 and 23 on another at 0.2256 m per pixel. So most of the path-length gap is the reference's error; the rest is ARKit reading 5% to 17% short (ARKit over GPS: 0.83, 0.94, 0.95 in walks 20 to 22). + +The truth also wanders between fix points. The results file splits the variance of each pairwise difference into per-tracker parts, but that split assumes the three trackers' errors are independent. Walk 22 violates it: it gives negative variances, and at 3 ft it puts ARCore's spread (16.3 in) above the truth's (8.1 in). Read the split as conditional, not as a resolution limit or a bound. + +**Results, pooled over walks 20 to 22** (walk 23: ARKit jumped at 826 m/s 5.5 s in and never recovered). |ARKit − reference|, inches, median / p90: + +| Walked | vs truth as published | vs truth rescaled to GPS | ARKit − ARCore disagreement | Signed median, ARKit − ARCore | +| --- | --- | --- | --- | --- | +| 3 ft | 9.7 / 17.8 | 4.7 / 13.1 | 2.8 / 8.2 | −2.4 | +| 10 ft | 31.6 / 55.8 | 15.4 / 40.5 | 8.6 / 22.3 | −8.1 | +| 20 ft | 60.4 / 106.0 | 30.9 / 70.1 | 16.9 / 41.5 | −15.9 | +| 30 ft | 88.2 / 156.8 | 44.5 / 92.9 | 25.7 / 60.1 | −24.8 | + +The ARCore column is the tightest comparison, but it is not a conservative estimate of ARKit's error: two trackers on one rig can share errors that cancel in their difference. + +**Position error.** Distance error only compares how far each track moved, so it cannot see sideways drift, and placement relative to the meter needs the whole displacement. Here ARKit's displacement is turned by the heading difference at each window's start, against the GPS-rescaled truth, both as tracked and with each walk's own ARKit scale divided out (0.838, 0.943 and 0.951 in walks 20 to 22). Pooled over walks 20 to 22, median / p90 inches: + +| Walked | As tracked | Scale removed | Server allowance, 0.16 ft per ft | +| --- | --- | --- | --- | +| 3 ft | 8.3 / 18.6 | 7.7 / 20.4 | 5.8 | +| 10 ft | 26.3 / 55.7 | 24.5 / 59.6 | 19.2 | +| 20 ft | 50.4 / 94.6 | 47.7 / 102.5 | 38.4 | +| 30 ft | 73.8 / 132.6 | 70.4 / 144.6 | 57.6 | + +Removing the scale barely changes the numbers, so what remains is heading and random error. Much of it is the reference's own: its random error is the largest of the three trackers', and its heading at each window's start enters directly. These numbers therefore overstate ARKit's error, but they are the only position errors this data supports. ARCore cannot serve as a position reference, because its orientation convention is undocumented. + +**Limits.** A 2018 phone with ARKit 1.0, walking briskly with the camera pointed along the path. Current phones may track better; there is no data on them here. + +## 2. Reconstruction accuracy on building walls (ETH3D) + +[results/eth3d_recon.md](results/eth3d_recon.md), `make recon`; [results/eth3d_visibility_sensitivity.md](results/eth3d_visibility_sensitivity.md), `make sensitivity`. + +**Data.** ETH3D's facade (76 photos) and electro (45 photos) scenes: building walls photographed with a 24 MP DSLR, camera poses registered to terrestrial laser scans. Photos are resized to 1024 px wide and fed to each model with their known intrinsics. + +**What is scored.** The error in the distance between two scanned points, for pairs 1 to 3 m and 3 to 10 m apart. Distances do not change under rotation or translation, so no alignment step can hide error. Scale error is the median of predicted over true length, minus one. + +**How points are chosen** (`evals/eth3d.py`, `evals/recon.py`): +- **Visibility.** Scan points are projected into each view. A point is visible when it is within 4% of the nearest scanned depth along its pixel, in a z-buffer that also holds ETH3D's occlusion splats. Points in ETH3D's masked regions are dropped: glass, and objects missing from the scan such as trees. The masks are drawn on the distorted originals, so they are stretched to the undistorted frame and dilated 13 px. +- **Cohorts.** Points where the depth jumps by more than 10% within 5 pixels are depth edges; the rest are surface interior, and interior points whose scan normal is within 17° of horizontal form the vertical cohort. Horizontal is taken from the ground plane under the cameras (`level_world` in `evals/coverage.py`), not the cameras' mean up, which leans 5.4° on electro and 10.0° on facade; the change moved 0.7% and 2.7% of the vertical points in or out of the cohort (`vertical_cohort` in the results file). +- **Fixed evaluation sets.** Each of 8 seed photos per scene fixes a set of points (those it sees; within 6 m of it for the near slice), test pairs and 25 taped reference pairs. The same set scores every method and every group size (the seed plus its 1, 3 or 7 nearest cameras facing the same way). A seed that lacks a group at any size, or whose slice is empty at any size, is dropped from every size (`comparable_groups` in `evals/recon.py`), so rows with different photo counts score the same seeds. Other photos only add predictions for those points, and in the near slice a photo contributes a point only if its own camera is within 6 m of it. +- **Failures.** A pair with no prediction, or a taped reference no scale can match, counts as infinitely wrong. The tables report the calibration success rate. A percentile that lands on a failure shows as "fails" in the tables and as the string `"failed"` in the JSON files, which are strict JSON; a statistic with nothing to compute is `null`. + +**Checks.** Scan depths agree with ETH3D's own sparse 3-D points to a median 0.2% to 0.4%. Depth rendered from the scan itself, scored through the same pipeline, is off 0.2 to 0.8 in median on 1 to 3 m spans (p90 under 2.5 in): the evaluation's floor. Changing the visibility tolerance to 2% or 8% moves model scale errors by about 1 point and MoGe-2's medians by at most 0.6 in, but MoGe-2's p90s by up to 31% (electro, one photo, 3 to 10 m: 21.8 in at 2% against 16.6 in at 4%) and Depth Anything 3's by more ([results/eth3d_visibility_sensitivity.md](results/eth3d_visibility_sensitivity.md)). That check compares 1 and 2 photos only, so it scores all 8 electro seed groups; the table below compares up to 8 photos and scores the 7 present at every size. Treat p90s as carrying roughly that much evaluation uncertainty. + +**Results at phone range: electro, within 6 m** (facade's photos were taken from 10 to 25 m). |length error|, inches, median / p90; "all photos" rows score every photo on its own evaluation set: + +| Method | Photos | Cohort | Model scale: 1-3 m | Scale error, 1-3 m pairs | One taped distance: 1-3 m | 3-10 m | +| --- | --- | --- | --- | --- | --- | --- | +| MoGe-2, one photo | all | surface interior | 6.8 / 19.5 | +3.5% | 1.6 / 9.3 | 3.0 / 16.9 | +| | | vertical interior | 7.1 / 23.9 | +1.1% | 1.8 / 13.6 | 3.6 / 44.7 | +| | | edges | 8.3 / 31.7 | +3.6% | 3.6 / 25.3 | 5.8 / 33.8 | +| Depth Anything 3 metric, one photo | all | surface interior | 6.0 / 20.2 | −6.6% | 2.9 / 24.4 | 6.0 / 57.9 | +| MoGe-2 per photo + true poses | 1 | surface interior | 6.8 / 13.8 | +4.2% | 1.4 / 5.2 | 2.8 / 10.1 | +| | 2 | | 6.6 / 13.9 | +2.0% | 2.0 / 8.3 | 3.5 / 15.6 | +| | 4 | | 5.7 / 13.0 | +2.9% | 2.2 / 10.0 | 3.9 / 18.9 | +| | 8 | | 5.4 / 12.5 | +0.7% | 2.8 / 11.9 | 4.5 / 24.3 | +| Scan rendered as depth (floor) | 1 | surface interior | 0.7 / 1.6 | −0.9% | 0.5 / 1.6 | 0.9 / 2.8 | + +Back-projection uses ETH3D's own intrinsics, not the centred ones MoGe-2 reports (a 0.8 to 3.6 px difference that moves medians by at most 0.1 in). + +Over all points (4 to 23 m away), MoGe-2 reads 8% short on facade and 0.5% long on electro, and Depth Anything 3 metric 10% to 15% short. With a tape, one MoGe-2 photo is off a median 2.5 to 2.9 in on 1 to 3 m spans. + +**Limits.** Two institutional buildings, not houses, photographed with a DSLR that is sharper than a phone camera. The taped distance is simulated as exact. Facade's near slice holds only 2 of its 8 groups, too few to conclude from. + +## 3. Pose priors: can AR poses fix the scale? (ETH3D) + +[results/pose_priors.md](results/pose_priors.md), `make pose-priors`. + +**Poses.** ETH3D's true poses, degraded per group of photos (`evals/ar_poses.py`). Each setting is an assumption: +- **exact:** the true poses. +- **advio_2018:** each group takes one ADVIO walk's measured ARKit scale against the GPS-rescaled truth (0.838, 0.943 or 0.951; ARCore itself reads about 4% short of GPS, so it is not the reference). It adds 5 cm and 0.2° of noise per camera, from ARKit's own 3 in spread over 3 ft and the 0.22° image-vs-pose disagreement on the replay. +- **modern_assumed:** 2% short, 1 cm and 0.1°, a guess for a current iPhone with no data behind it. + +**Methods.** +- **(a) MapAnything** given the poses and intrinsics as priors, at 392 px with 2 and 4 photos (the memory limit); the same runs without poses are its baseline. +- **(b) Triangulation rescale.** MoGe-2 depth for each photo is rescaled to the points triangulated from SIFT matches across the group with those poses (`evals/triangulate.py`: one factor per photo, the median ratio over at least 20 points that reproject within 2 px plus the pose noise), then placed with the same poses and fused. +- **Control:** MoGe-2 placed with the same poses without the rescale. + +**Results: electro, within 6 m, 1 to 3 m spans, no tape, 8 photos.** Median / p90 inches with 95% intervals. The intervals bootstrap over the seed groups (7 for surface interior, 6 for walls) and over 5 pose-noise draws for each AR-like setting; exact poses have no noise to draw. + +| (b) MoGe-2 rescaled by triangulation | Surface interior | Vertical interior (walls) | Scale error | +| --- | --- | --- | --- | +| exact poses | 1.3 [1.0, 1.5] / 4.0 [3.3, 4.8] | 1.1 [0.8, 1.4] / 2.8 [2.0, 4.6] | −0.6% | +| modern_assumed (2% short, 1 cm, 0.1°) | 2.5 [2.1, 2.9] / 5.6 [4.6, 6.7] | 2.3 [1.7, 2.8] / 5.0 [3.7, 6.9] | −3.0% | +| advio_2018 (measured on the 2018 phone) | 6.2 [4.6, 8.6] / 14.6 [10.6, 18.7] | 4.9 [3.7, 6.5] / 10.8 [8.6, 13.1] | −7.3% | + +Edges stay at 8 in or worse at p90 in every setting. The control, MoGe-2 placed with the same poses and no rescale, has a p90 of 12.5 in with exact poses. With exact poses, which carry no noise, 4 photos give (b) a surface p90 of 4.5 in against 4.0 in with 8. The AR-like rows can't be compared across photo counts: each group draws its pose noise in turn from one generator shared by every group size, and advio_2018 hands out its three scales by group order, so the 4- and 8-photo groups from the same seed photo carry different pose errors ([#159](https://github.com/SamGu-NRX/house-scanning-master/issues/159) tracks drawing them per seed). With 4 photos those rows read 7.9 in with modern_assumed (its interval reaches failure) and 26.5 in with advio_2018; the difference from 8 photos mixes photo count with pose error. The 8-photo rows above, and the README's wall result taken from them, don't depend on this comparison. + +These rows assume perfect cross-view correspondence: each scan point is found in every photo that sees it through the true projection, as a perfect feature matcher would. An app without a matcher would instead reproject the tapped photo's own estimate into the others. Done that way, the wall p90 with 8 photos rises from 2.8 to 6.3 in with exact poses, from 4.3 to 6.4 in with modern_assumed and from 10.3 to 14.8 in with advio_2018, on one noise draw each. Medians move by 0.2 in or less. + +(a) MapAnything given the poses, at 392 px with 4 photos, fixes most of its scale on electro (+17% becomes +2% to +4%) but not on facade (still 11% to 13% short). On electro within 6 m its p90 without a tape is 12 to 19 in on surfaces, and 13% to 17% of taped pairs go without a prediction, so every taped row fails. Its rows have a single noise draw: its weights were removed to free disk. Those runs predate the grouped runner's checkpoint check, so their `run.json` files record no checkpoint hash, and with the weights gone it can't be checked now; a new grouped run verifies the pinned sha256 and records it. + +**What decides it.** Triangulation fixes each photo's own scale error, which is why 8 rescaled photos beat 8 unscaled ones by a factor of three. But the triangulated points take their scale from the poses, so the pose's scale error passes straight into every length: 2% short in, 3% short out. A taped distance on top makes it worse, not better (p90 7.6 in with exact poses): the reference pair carries its own reconstruction error, which outweighs the scale error that is left. + +## 4. Which photos to keep (ETH3D) + +[results/frames.md](results/frames.md), `make frames`. Each photo is scored alone (MoGe-2, one taped distance, surface interior) and described by what an app can measure while capturing: the median distance to the wall points in view, and the median angle between the viewing ray and the scanned surface. Both scenes pooled, median / p90 inches: + +| Keep | Photos kept | 1-3 m spans | 3-10 m spans | +| --- | --- | --- | --- | +| every photo | 121 of 121 | 2.7 / 11.4 | 7.6 / 38.7 | +| wall within 6 m | 15 of 121 | 1.9 / 6.8 | 4.3 / 14.8 | +| wall within 8 m | 27 of 121 | 2.1 / 7.4 | 4.8 / 17.0 | +| wall within 12 m | 42 of 121 | 2.3 / 8.1 | 5.4 / 20.4 | +| wall seen within 40 degrees of head-on | 52 of 121 | 2.3 / 9.1 | 7.0 / 34.5 | +| wall seen more than 50 degrees off head-on | 40 of 121 | 2.8 / 11.7 | 7.1 / 32.8 | + +No photo here is closer to head-on than 26 degrees, so a head-on shot is untested, and all 15 photos within 6 m are from electro. + +## 5. A field session against the tape + +`make field SESSION=session.zip TRUTH=survey.json MAP=map.json` turns one Measure Lab session (PR #7's one-hour tape protocol) and its tape survey into one table of every method, scored by the scoring harness (PR #4, `experiments/scoring`). The map is the same file `score import-measure-lab` reads. The command runs these steps (`evals/field.py`): + +1. It turns each keyframe upright and runs MoGe-2 on it. Session images are sideways sensor images, and the model expects upright photos. **Run `make field` with nothing else heavy on the machine:** MoGe-2 at its default resolution peaks at 4.07 GB, just over the shared machine's 4 GB per process. Depth maps are written as each keyframe finishes, so the peak does not grow with session length, and a 3.6 GB GPU cap stops a run that would grow instead of letting it swell. Storing its encoder in half precision to get under 4 GB failed: parts of the encoder run outside autocast and reject fp16 weights. +2. It gives every point the rig made from taps a second position: MoGe-2's depth at the tapped pixel, placed with that keyframe's AR pose. It then recomputes walls and every session measurement with Measure Lab's own formulas (`Wall.swift`, `Measurements.swift`). +3. It writes three scoring-harness results files: + - `moge2`: the model's own scale; + - `moge2-triangulated`: method (b) of section 3, each keyframe rescaled to features triangulated with the session's AR poses across it and its 7 nearest keyframes; + - `moge2-tape`: one depth scale for the session, solved so the survey's scale reference comes out at its taped length. Each point moves along its ray from its own keyframe's camera and camera positions don't scale, so it solves along the rays rather than dividing lengths; a length ratio is exact only when both taps share a keyframe. The reference must be a straight point-to-point measurement. + + They state no uncertainty and make no decisions: none of them has a validated error bar. +4. It imports the rig's own row with `score import-measure-lab` and scores every row with `score`. The harness is PR #4 at 77ce830dd08acf0c7982edcec91cd3240e0565ea, fetched by that full SHA (a shallow or single-branch clone lacks it) and checked out read-only outside the repo. Before any of this, `stamp` refuses a map already bound to another session, because its M numbers belong to that walk. A reused harness checkout must still be at that commit with `experiments/scoring` unmodified. The session zip is checked before anything is written: plain relative member paths only, no duplicates, at most 10,000 files and 2 GiB expanded, with 3 GB left free. Each keyframe's size is read from its JPEG or PNG header before decoding, and an image over 50 megapixels is refused. Scale fitting holds only one keyframe's neighbour group in memory at a time and refuses a group of more than 200 megapixels (about 1 GB decoded) before decoding it, so memory doesn't grow with session length. +5. It writes `field_report.md` with the phone's AR scale error, the number section 3's verdict hinges on: + - Only measurements the rig accepted count, as in PR #4's importer. + - The scale is a least-squares fit over straight spans of 10 ft or more, weighted by length, with a 95% bound per span and overall. The bound combines the tape's ± with an assumed 2 in of tapping error per span, which the field test itself will check (`--tap-error-in`). + - It needs an accepted span of about 30 ft (29 ft or more), and answers "within 2%" as yes, no or cannot tell, under the same strict rule as placements. + - Each session writes to its own folder, cleared on a re-run, so `score` never picks up an earlier run's rows. + +MapAnything is left out. In section 3, given poses, it was worse than (b) everywhere. It left 10% to 25% of pairs without a prediction and did not fix its scale on facade. It also cannot take a phone's full keyframe set within the 4 GB limit. + +**Checked.** A hand-computed synthetic session (`tests/test_field.py`) gets all four rows through PR #4's own `score`, including its import of the rig's row, in one table: +- The rig row reads 2% short, as planted. +- MoGe-2 with depth 10% long gives every length 10% long. +- The tape row comes out exact. +- The triangulated row reports every value as failed: one keyframe has no neighbours to triangulate against. + +**On the real ADVIO replay** ([results/field-replay/field_report.md](results/field-replay/field_report.md)), the pipeline runs through MoGe-2 and triangulation. All 79 keyframes get a scale, and MoGe-2 needs a median ×0.774 (0.66 to 1.03) to agree with the iPhone 6s's ARKit poses; ARKit itself read about 16% short on this walk. Nothing more comes out, because the replay has no taps, measurements, survey or map. + +**In the field:** [field/FIELD_SHEET.md](field/FIELD_SHEET.md) lists what to bring, mark, tape and tap, in order, with the Measure Lab numbers the map expects. `field/survey.template.json` and `field/map.template.json` name every item, so the team fills in only tape readings, typed as the tape shows them (`30 0 1/8`). `make field` converts them to feet and fills in the zip's sha256 and session id. `make field-dryrun` builds a synthetic session in the sheet's order, fills the templates, and runs `make field` with them. It recovers the planted 1.5% AR scale error (−1.50% ± 0.68%) and gets all four rows through PR #4's `score`. + +## 6. A current iPhone's ARKit scale (MARViN) + +[results/modern_arkit.md](results/modern_arkit.md), `make modern-arkit`. + +**Which data, and why.** The only public data I found with a 2020-or-newer iPhone's own ARKit poses and a separate reference trajectory, downloadable today, is MARViN (Liu et al., IEEE VRW 2024). It has an iPhone 14 Pro Max (ARKit 6) walking outdoors around three sites, 45 to 255 m per walk, one image per second, with a COLMAP reconstruction of each site as the reference. Checked and not used: +- **LaMAR** (ETH and Microsoft) needs an access request. +- **ScanNet++** needs an application. +- **ARKitScenes** (iPad Pro, indoor) ships ARKit's trajectory but not its laser-registered camera poses. +- **ADVIO**'s phone is from 2016. + +**Method.** The same windows as section 1: from every image, the walk continues until the reference has covered 3, 10, 20 or 30 ft. ARKit's straight-line displacement is compared with the reference's. A walk's scale is the median ratio over windows of 10 ft or more. Distances need no axis alignment, which matters because ARKit's poses here are in Unity's axes. + +**The reference's limits.** COLMAP from one moving camera has no scale of its own, and the dataset does not say how its reconstructions were put into meters. They could have been scaled to ARKit itself, which would hide any ARKit scale error. The phone's GPS is the only independent check shipped. Over these walks GPS is 1 to 9 m off, and it puts the reference's scale between 0.92 and 1.07 even on its 11 cleanest walks, so it cannot confirm or refute 2%. The reference's own accuracy is not published. + +**Results.** Scale error of ARKit against the reference, per walk: + +| Site | Walks | Median walk | Range | +| --- | --- | --- | --- | +| atrium | 10 | +0.3% | −50.7% to +1.0% | +| bar | 13 | +1.2% | −0.4% to +2.1% | +| church | 12 | −1.7% | −2.2% to −1.0% | + +Thirty of 35 walks are within 2%. Three atrium walks read 18% to 51% short; on them, ARKit and the reference disagree about whether the phone moved at all between images, so the reference is in doubt there. Distance errors pooled over all walks, median / p90 inches: + +| Walked | As tracked | Without the three doubtful walks | +| --- | --- | --- | +| 10 ft | 2.7 / 16.4 | 2.4 / 7.1 | +| 20 ft | 4.5 / 34.8 | 4.2 / 11.9 | +| 30 ft | 5.9 / 45.8 | 5.5 / 16.2 | + +The 3 ft row in the results file is below what this reference resolves (images 0.2 to 1.4 m apart, and reference jitter up to 0.4 m), so it is not repeated here. + +**Position error**, the same windows with ARKit's displacement turned by the heading offset at each window's start. ARKit's quaternions are (w, x, y, z) in Unity's left-handed axes, and each walk is paired with the reference by the handedness that keeps the heading offset steady: it holds to 0.6° over a median walk (worst 3.1°). Median / p90 inches: + +| Walked | As tracked | Without the three doubtful walks | Server allowance, 0.16 ft per ft | +| --- | --- | --- | --- | +| 10 ft | 3.3 / 18.7 | 3.1 / 8.6 | 19.2 | +| 20 ft | 5.4 / 37.5 | 5.1 / 13.4 | 38.4 | +| 30 ft | 7.2 / 50.0 | 6.9 / 18.5 | 57.6 | + +**Verdict.** +- **Within 2%?** Probably, for this phone; not proven. Against the only reference available, this 2022 iPhone's ARKit holds its scale within 2% on 30 of 35 outdoor walks, and each site's walks agree with each other to within about 1 to 2.5%. +- **Walk-to-walk disagreement.** Within one site, ARKit's scale ratio to the reference spreads 0.28% [0.12, 0.36] at atrium, 0.75% [0.46, 0.94] at bar and 0.34% [0.19, 0.44] at church (1 SD, 95% interval). That is disagreement between ARKit and the reference, not a bound on ARKit's error: the reference's local scale error could produce all of it, and nothing here bounds that error independently. +- **What it means for section 3.** A pose scale error of about 2% is the first thing walls need, and this is the first evidence from a current phone that ARKit meets it. It is not enough: at 2%, walls still come out at a p90 of 5.0 in [3.7, 6.9] (section 3). The 2018 phone's 5% to 17% does not describe current hardware. +- **What stops it being settled:** + - the reference's scale cannot be checked independently to 2%; + - one site sits at a steady −1.7%, which is the phone's error or the reference's; + - the phone has LiDAR, which ARKit may use. + +The tape test on the team's phone is still the number to trust. + +## 7. Does the app's coverage map claim surface no photo saw? (ETH3D) + +[results/coverage.md](results/coverage.md), `make coverage`. + +**Why.** Every placement check rests on "unseen is not clear": the server passes a check only over the wall and ground the app exports as observed. That export is `CoverageMap.coveredIntervals` (`ios/HouseScanKit`), whose own comment says occlusion is not modelled. Nobody had tested it against what the photos actually saw. + +**Method.** +- **The app's code, unmodified.** `ios/HouseScanKit` at commit beede15 of `t3/ios-mvf`, checked out read-only outside the repo and run by `coverage_driver/` with its default `CoverageConfig`. Every ETH3D photo is a kept keyframe with normal tracking, fed in capture order. +- **The wall, as taps would give it.** Per scene, the straight stretch of wall that the most photos see within the app's 6 m. The meter sits mid-stretch, 1.5 m up. The outward normal comes from a plane fit to the scan, the ground height from the scan at the wall's foot, and the marked ends are the stretch's ends. The world is levelled on the ground plane under the cameras, because ARKit's y is up. Poses and intrinsics go in unrotated, in ARKit's camera axes. +- **Truth.** Columns 2 cm wide along the wall, each band sampled every 5 cm up the wall or out across the ground. A photo saw a sample when all of these hold: + - it lands inside the full image; + - it is within the app's 6 m; + - the camera is on the room side of the wall; + - nothing nearer hides it. "Nearer" means more than max(10 cm, 4% of depth) in front in the laser scan's depth, with ETH3D's occlusion splats included; ETH3D's masks of objects missing from the scan (people, vegetation) also count as hiding. + + Grazing angle is not part of the truth: an oblique photo still saw the surface, and the app's 65° limit shows up only as missed length. + +**Pass criteria, fixed before the run.** +- **False-observed length**: wall claimed as observed where at least 10 cm of the band (2 samples) was seen by no photo. Must be at most 0.5 ft, one 6-in cell, per scene and band; that is what "about zero" means here. It is also reported as a share of claimed length, and split by the cause at the photos the app credited: occlusion, range, frame edge. Grazing angle cannot cause it under this truth. A stricter variant counts band seen from fewer than two positions 0.25 m apart, the app's own bar for "covered"; it is reported, not graded. +- **Missed length**: band every sample of which two photos 0.25 m apart saw, which the app did not credit. No pass bar, since it costs the homeowner extra photos, not a wrong answer. It is split by why the app turned those photos down: frame edge (including its 3% margin and the band's top or bottom out of view), range, grazing angle, or only one position. +- **Validity checks**: my replica of the app's view gates (used only to name causes) must reproduce the app's own sightings, and the wall plane must fit the scan within a few centimetres. + +**Results.** Lengths in feet along the wall. + +| Scene | Band | Claimed | False-observed (share) | Pass | False-observed, 2-position truth | Missed | +| --- | --- | --- | --- | --- | --- | --- | +| electro | wall | 19.3 | 1.1 (6%), all occlusion | no | 1.3 (7%) | 0.0 | +| electro | ground | 0.0 | 0.0 | untested | 0.0 | 18.1, all grazing angle | +| facade | both | nothing: no wall within 6 m of any photo | | untested | | | + +- **Where the 1.1 ft is.** 1.05 ft is the bottom 20 cm of the wall at the stretch's left end, behind a scaffold's wooden footings and a cable bundle standing in front of the wall. The app credits it because the photos that frame it are within range and angle; no photo sees behind the footings. The other 0.07 ft is one 2 cm column at a pilaster's edge, within the scan's resolution. +- **Ground: untested.** The app claimed no ground on electro. The ETH3D photographers stood 2.0 to 7.6 m from the wall (median 4.9 m). A camera 1.5 m up must be within 3.2 m for the ground at the wall's foot to be inside the app's 65°, so every ground view was turned down: 18.1 ft seen from two positions and not credited. At the app's intended 2.6 m standoff the ground would be claimed, so these photos cannot say whether that claim would be right. +- **Facade: untested.** The building stands beyond the app's 6 m from every photo: 99% of its wall-height scan points are 6.5 m or more from the nearest photo, with a median of 11.6 m. Within range are only stair flanks, sculptures and tree trunks, none a straight wall that a photo sees 1 m of. The app claims nothing there. +- **Validity.** The replica of the app's view gates reproduces all 337 of its sightings exactly. The flat parts of the wall fit the plane to 1.8 cm RMS. Two pilasters stand 0.36 m proud of it (3.8 ft of the 19.3), so the truth samples their front faces; with the plane alone they read as 2.8 ft of false occlusion. A photo counts as seeing a sample only where the scan has depth at its pixel; no claimed sample was left unknown for lack of scan. Peak memory 1.3 GB. + +**Verdict.** +- **Does coverage claim unseen wall? Yes: 1.1 ft of 19.3 on electro (6%), where "about zero" allowed 0.5 ft. It fails.** All of it is occlusion, and that is the rule, not bad luck. At commit beede15, `CoverageMap.swift` lines 241 to 248 (`sees`, inside `visibleRows`) accept a sample on range, angle and image bounds alone; nothing checks whether something stands between camera and wall. The type's own comment (lines 57 to 59) says so, and adds that "the server re-checks what matters from the images". The server's contract says the opposite: it passes a check over whatever the app reports as observed. The loss is small here only because 17 photos from many angles see around most of the scaffold. A bush, a bin or an AC unit in front of a wall is claimed as observed whenever the phone stands in front of it. +- **What would fix it (for S3):** + - Give `CameraFrame` an optional per-frame depth map: ARKit's `sceneDepth` on LiDAR phones, or depth rendered from ARKit's mesh. + - In `sees`, reject a sample when that depth at its pixel is nearer than the sample by more than max(10 cm, 4%), the tolerance used here. + - Sample more rows, or check depth along each row-to-row segment. With `rowsPerBand = 3` (lines 43 to 47) the rows are 0.99 m apart, so something mounted between them passes even a depth check at the rows. + - Phones without depth can't make this check on the device. Until one side changes, either the server re-checks occlusion from the images, as the comment assumes, or the export must not be read as "seen and clear". +- **Missed wall: none.** Everything two photos saw, the app credited. +- **Limits.** One scene, one 19 ft wall, 17 contributing DSLR photos, not a walking phone capture. The truth ignores anything within max(10 cm, 4% of depth) of the wall (downspouts, conduit), which can only have hidden more false-observed length, not less. + +### 7b. Options for occlusion (ETH3D electro) + +[results/coverage_options.md](results/coverage_options.md), `make coverage-options`. + +**Question.** Which rule change stops coverage from claiming unseen wall, and at what cost in extra photos? The same harness, wall and truth as section 7. Each option is modelled in `coverage_driver/`, around the app's code at beede15, which stays unedited: +- **Position baseline b**: `CoverageConfig.coveringBaseline` (`CoverageMap.swift` line 49, used at line 181), swept at 0.25 (today), 0.5, 1 and 2 m. No LiDAR needed. +- **Angle diversity θ**: the predicate in `record` (lines 180 and 181) also requires the two views' directions to the row's centre to differ by at least θ, swept at 0°, 15°, 30° and 45°. It is crossed with b. No LiDAR needed. At θ = 0 and b = 0.25 the driver's copy of `record` must reproduce the app's own answer exactly. +- **Depth test (LiDAR)**: `sees` (lines 241 to 248) also rejects a row sample that the laser scan, standing in for LiDAR depth, shows hidden. The same test and tolerance as the truth, and ETH3D's masks count, because LiDAR sees people and trees too. Three settings: + - true depth out to the app's 6 m, 3 rows (the ceiling); + - the same with 9 rows (`rowsPerBand`, lines 43 to 47); + - 9 rows with no depth beyond 5 m, the iPhone LiDAR's stated range. + +**Pre-registered, before any run.** An option passes when false-observed length on electro's wall is at most 0.5 ft, as in section 7. Missed length is measured against the same truth for every option: band seen from two positions at least 0.25 m apart. The recommendation is the passing option with the least missed length, preferring one that needs no LiDAR when two are within 1 ft of each other. + +**Results** on electro's 19.3 ft wall, in feet: false-observed (pass at 0.5 or less) and missed (band two photos saw that the option doesn't claim). The full grid is in the results file. + +| Option | Needs LiDAR | False-observed | Pass | Missed | +| --- | --- | --- | --- | --- | +| The app today (b = 0.25 m, θ = 0°) | no | 1.1 | no | 0.0 | +| Views ≥ 2 m apart, any angle up to 45° | no | 1.1 | no | 0.0 | +| Views ≥ 5 m apart* | no | 0.0 | yes | 6.0 | +| Views ≥ 60° apart* | no | 0.0 | yes | 8.5 | +| Depth test, true depth to 6 m, 3 or 9 rows | yes | 0.0 | yes | 3.3 | +| Depth test, 9 rows, no depth past 5 m | yes | 0.0 | yes | 8.9 | +| Depth test, 9 rows, 0.4 m of wall relief allowed* | yes | 0.1 | yes | 0.0 | + +\* Added after the pre-registered grid (b 0.25 to 2 m, θ 0 to 45°) changed nothing. + +- **Spacing or angle between views: no.** Nothing in the pre-registered grid moved the 1.1 ft. The footings hide the wall's base from every direction the photos were taken from, and seeing a cell from far-apart or differently angled views says nothing about whether something stands in front of it. The patch goes only when the rule refuses cells for want of spread: at 5 m apart (6.0 ft lost) or 60° apart (8.5 ft lost, 47% of what was seen). It is removed along with good wall, not singled out. At 75°, nothing is claimed. +- **Depth test: yes, at a price the rule's shape sets.** With true depth it claims nothing unseen. The 3.3 ft it misses is the two pilasters. A tapped wall is a plane, so a pilaster 0.36 m proud of it reads as something in front of the wall, and a real LiDAR test would do the same. Allowing 0.4 m of relief keeps them (0 missed) and still rejects the footings (0.1 ft). At 0.5 m the footings get through (1.0 ft). That margin was found on this one wall and says nothing about bushes closer than 0.4 m. Nine rows instead of three changed nothing here, because the hidden patch touches the bottom row. It would matter for something mounted between rows. +- **LiDAR's reach costs coverage.** iPhone LiDAR reads to about 5 m; photos 5 to 6 m out then earn nothing, which misses 8.9 ft on these DSLR photos. At the app's 2.6 m standoff that limit would rarely bind. +- **Ground band: still untested.** Every option keeps the 65° gate, so none claims ground from these photos. None of the datasets on disk can test it: ADVIO and MARViN have no surface truth, and facade's photos stand even farther back. ETH3D's other outdoor scenes carry the same laser truth, at 0.3 to 0.7 GB each with images (courtyard, delivery_area, meadow, playground, terrace). Whether their photographers stood within 3.2 m of a wall can only be read after downloading, so I did not. + +**Recommendation.** Stay with "unseen is not clear" and make occlusion someone's job, because no rule about how the phone moved can stand in for it: +- **LiDAR phones:** a depth test in `sees` (`CoverageMap.swift` lines 241 to 248), with rows closer than 0.99 m and an allowance for wall relief. The allowance needs field data before a number ships. +- **Phones without LiDAR:** the app cannot know what hides the wall. Either the homeowner confirms nothing stands in front of each claimed stretch (for example, by marking obstacles on a still), or checks that need clear ground treat claims from those phones as unconfirmed. + +Tightening the baseline or angle would only ask for more photos: 6 to 8.5 extra feet of wall here, with no guarantee that the hidden patch goes. + +### 7c. The app's 3D map (Map3D) against the same truth (ETH3D electro) + +[results/map3d.md](results/map3d.md), `make map3d`. + +**Question.** The app's live 3D map (`ios/HouseScanKit/Sources/HouseScanKit/Map3D`, t3/ios-map3d at 66cdcba, draft PR #21) is to replace `CoverageMap` on every phone. Does it claim wall, ground, facing or overhead space no photo saw? + +**Method.** +- **The app's code, unmodified**, checked out read-only and driven by `map3d_driver/`: `Map3D.integrate(DepthFrame)` per photo, then `coverage(along:)` and `measuredWalls()`, default `Map3DConfig`. +- **The same wall as section 7:** the map frame and the `WallFrame` coverage is read along are section 7's (meter, outward normal, ground height). +- **LiDAR path:** one depth frame per photo, 256 x 192, rendered from the laser scan at the photo's pose (nearest scan depth along -z; 0 where nothing was scanned or ETH3D masks an object the scanner missed), with no confidence. +- **Non-LiDAR path: not tested.** It needs ARKit's feature points and detected planes. COLMAP's tracks on 24-megapixel DSLR photos are far denser and more exact than ARKit's feature points, and ARKit's plane detector can't be reproduced from them without guessing. +- **Truth.** Wall and ground: section 7's laser-scan visibility, on the same wall face and ground band (1.2 m, the depth `CoverageMap` claims). Map3D's ground counts as claimed where its reach is at least 1.2 m. Facing and overhead space: a point counts as seen empty when some photo within 5 m frames it and the laser scan's depth at its pixel lies beyond it by more than the section 7 tolerance, max(10 cm, 4%), so the ray passed through it. + +**Pre-registered, before any run.** +- **False-observed** (claimed columns where at least 10 cm of what the band claims was seen by no photo) must be at most **0.5 ft** per band, as in section 7, where `CoverageMap` claimed 1.1 ft of wall. +- **Missed** (wall and ground band seen from two positions 0.25 m apart but not claimed) is reported without a bar. +- **Wall chain:** the measured wall piece through the meter against the laser's wall line (angle, offset at the meter, length), reported without a bar. + +**Results** (ideal depth: exact, dense, no sensor noise). Feet along the wall. + +| Band | Claimed | False-observed | Pass | Missed | +| --- | --- | --- | --- | --- | +| Wall, `CoverageMap` (section 7's 19.3 ft stretch) | 19.3 | 1.1 | no | 0.0 | +| Wall, Map3D, same stretch | 11.1 | 0.0 | yes | 6.9 (4.7 with a 5 m truth) | +| Wall, Map3D, everything it claims (s −9.7 to +21 ft) | 19.4 | 0.0 | yes | 8.9 of 25.2 seen, 5.5 of it on pilaster faces | +| Ground | 0.0 | 0.0 | untested | 18.1 | +| Facing | 0.5 | 0.5 (see below) | at the bar, unresolved | n/a | +| Overhead | 0.0 | 0.0 | untested | n/a | + +- **Wall: Map3D claims no wall that no photo saw, where `CoverageMap` claimed 1.1 ft.** The 1.1 ft behind the scaffold's footings is left unclaimed, as the depth test of section 7b predicted. +- **The cost is missed wall.** 5.5 ft of the 8.9 ft missed is behind pilasters 0.36 m proud of the wall line. Map3D counts the face only within 0.10 m in front of the line (`faceFront`, `Map3DConfig.swift` line 85 at 66cdcba, used in `Map3DCoverage.swift` lines 73 to 86), so a pilaster hides the wall behind it. That is conservative, not an over-claim. It asks for views that cannot exist, so the gap loop could ask forever for a pilaster-backed stretch. +- **Ground and overhead: untested.** Map3D claims no ground: reach 0 everywhere, the same grazing-view limit as section 7. +- **Facing: unresolved.** Map3D's one claim is a single 6 in cell seen clear 0.1 m out from a door face. By the pre-registered metric it reads 0.50 ft, exactly at the bar. An earlier count gave the boundary column a full 2 cm width and read 0.52 ft (`column_weights` in `evals/map3d.py` now clips it). The truth, however, cannot resolve space that close to a surface: it needs the laser surface to lie beyond a point by 4% of the distance, 12 to 20 cm here. So the claim is neither confirmed nor refuted, and a 0.1 m reach settles no check (the server needs more than 4.83 ft). +- **Wall chain:** one piece through the meter, 28.4 ft long, where the laser's wall face (pilasters included) runs 30.8 ft. The left end is 0.3 ft short and the right 2.2 ft short. The line is 1.76° off the laser face and 4.8 in outside it at the meter, more than the server's 3.6 in default meter error. The cause is not established. +- **Not measured:** the non-LiDAR path, real LiDAR noise and dropouts, and coverage along Map3D's own chain instead of section 7's wall line. An independent review of the method also flagged a rule to check: `VoxelGrid.swift` lines 376 to 378 and 409 to 412 at 66cdcba take the nearest view distance and the best view angle from possibly different views, so a surface can pass the 5 m and 65° tests with no single view meeting both. It caused no false-observed wall here. + +## Replay session from a real walk + +`~/house-scanning-data/replays/advio-20-0040-0075.zip` holds 35 s of ADVIO walk 20 (seconds 40 to 75, a path past a brick building) in Measure Lab session format v2, for the app's replay mode. It has 79 keyframes chosen by Measure Lab's rule (0.5 m or 15° since the last), plus `ground_truth.json` with ADVIO's pose for each keyframe. `make replay` rebuilds and checks it. + +- **Images.** `frames.mov` stores unrotated 1280 × 720 sensor frames behind a portrait display tag. They are decoded by index, since seeking by time lands frames off, and undistorted with ADVIO's calibration. +- **Intrinsics.** `[1082.1, 1081.1, 641.29, 359.91]`, from the portrait OpenCV calibration rotated to landscape and shifted half a pixel (`evals/camera.py`). +- **Poses.** `arkit.csv` orientations are in the portrait device frame. The converter turns them a quarter turn about the viewing axis. Image-derived rotations agree with this reading to a median 0.22° (alternatives: 4.5° to 6.5°). +- **Check.** `check_replay` reads only the session files and measures matched features' distance from the epipolar lines the poses predict: median 3.1 px, against 61 px with the axes deliberately wrong. +- **Missing from ADVIO.** No tracking state (written as `normal` once ARKit reports a position), no capture time (`startedAt` is the publication date), no taps or measurements. The camera points along the path, not at a wall. diff --git a/experiments/evals/Makefile b/experiments/evals/Makefile index 497517525..cf8b0b31b 100644 --- a/experiments/evals/Makefile +++ b/experiments/evals/Makefile @@ -68,16 +68,30 @@ pose-priors: # (experiments/scoring, PR #4) and scores every row in one table. FIELD_OUT ?= $(DATA)/evals/field/results/$(basename $(notdir $(SESSION))) RULES ?= field/rules.json -# PR #4's harness, checked out read-only at a pinned commit outside the repo. -SCORING_COMMIT := 77ce830 -SCORING_CHECKOUT := $(DATA)/evals/scoring-$(SCORING_COMMIT) +# PR #4's harness, checked out read-only at a pinned commit outside the repo. The full SHA is +# fetched by name first, because a shallow or single-branch clone doesn't have it. +SCORING_COMMIT := 77ce830dd08acf0c7982edcec91cd3240e0565ea +SCORING_CHECKOUT := $(DATA)/evals/scoring-77ce830 SCORING ?= $(SCORING_CHECKOUT)/experiments/scoring STAMPED_MAP = $(FIELD_OUT)/inputs/map.json field: - @# The pinned checkout is created only at its default place; an overridden SCORING must exist. - test -d "$(SCORING)" || { test "$(SCORING)" = "$(SCORING_CHECKOUT)/experiments/scoring" && \ - git -C ../.. worktree add --detach "$(SCORING_CHECKOUT)" $(SCORING_COMMIT); } || \ - { echo "SCORING=$(SCORING) is not a scoring harness checkout" >&2; exit 1; } + @# The default checkout is created at the pinned commit and, when reused, must still be there + @# with the harness unmodified. An overridden SCORING is the caller's choice and must exist. + @if [ "$(SCORING)" = "$(SCORING_CHECKOUT)/experiments/scoring" ]; then \ + if [ ! -d "$(SCORING_CHECKOUT)" ]; then \ + { git -C ../.. cat-file -e $(SCORING_COMMIT)^{commit} 2>/dev/null || \ + git -C ../.. fetch --no-tags origin $(SCORING_COMMIT); } && \ + git -C ../.. worktree add --detach "$(SCORING_CHECKOUT)" $(SCORING_COMMIT) || exit 1; \ + fi; \ + head=$$(git -C "$(SCORING_CHECKOUT)" rev-parse HEAD) && \ + dirty=$$(git -C "$(SCORING_CHECKOUT)" status --porcelain -- experiments/scoring) || exit 1; \ + if [ "$$head" != "$(SCORING_COMMIT)" ] || [ -n "$$dirty" ]; then \ + echo "$(SCORING_CHECKOUT) is at $$head with changes [$$dirty]; expected $(SCORING_COMMIT)," \ + "unmodified. Remove it (git worktree remove) and rerun." >&2; exit 1; \ + fi; \ + elif [ ! -d "$(SCORING)" ]; then \ + echo "SCORING=$(SCORING) is not a scoring harness checkout" >&2; exit 1; \ + fi uv sync --project models $(if $(TRUTH),uv run python -m evals.field stamp "$(SESSION)" --truth "$(TRUTH)" --map "$(MAP)" --out-dir "$(FIELD_OUT)" &&) \ W=$$(uv run python -m evals.field prepare "$(SESSION)") && \ diff --git a/experiments/evals/README.md b/experiments/evals/README.md index ba310eca6..6357eea27 100644 --- a/experiments/evals/README.md +++ b/experiments/evals/README.md @@ -1,411 +1,12 @@ # Evals on real data -## Questions +**Question.** Can a phone's photos and AR tracking measure a house wall well enough for placement: about 4 in at p90 on 1 to 3 m spans? And does the app's coverage map claim only wall it saw (false-observed at most 0.5 ft)? Each bar was fixed before its run. -1. How far is ARKit's tracking off after walking 3, 10, 20 and 30 ft outdoors? The untested default for an AR tap is plus or minus 0.3 ft (3.6 in; `docs/00`, Conventions the code relies on). -2. How accurately can phone photos reconstruct a real building wall, with the model's own scale or with one taped distance, from one photo or several? -3. Can the phone's AR poses fix the learned models' scale, well enough for about 4 in at p90 on 1 to 3 m spans, the error that decides a 3 ft clearance? -4. Which photos are worth keeping? -5. On a real field session with a tape survey, how do the phone's AR taps and the learned-depth methods compare, and what is the phone's AR scale error? -6. How large is a current iPhone's ARKit scale error, from public data? -7. Does the app's coverage map claim wall and ground that no photo saw? +**Run.** `cd experiments/evals && uv sync && make test`; each result has its own target (`make drift`, `make recon`, `make coverage`, …). Model runs need an Apple-silicon Mac, and the app-code checks need macOS 26 with Swift 6.2. [METHODS.md](METHODS.md) lists every target, prerequisite and dataset sha256. -Every dataset evaluation uses real captures. Two checks add synthetic inputs, and are labelled where they appear: section 3's pose-prior rows degrade ETH3D's true poses with simulated AR-like error, and `make field-dryrun` scores a synthetic session to prove the field command. The unit tests of the metric code also use synthetic cases. +**Results.** +- **Tracking:** a 2018 iPhone's position error, p90 18.6 to 132.6 in over 3 to 30 ft, is 2 to 3 times the server's allowance. An iPhone 14 Pro Max stays inside it: 8.6, 13.4 and 18.5 in at 10, 20 and 30 ft. +- **Depth:** learned depth is 4% to 12% off scale. Rescaled with AR poses, walls reach p90 5.0 in [3.7, 6.9] at a simulated 2% pose error, which misses 4 in. +- **Coverage:** `CoverageMap` claimed 1.1 ft that no photo saw, and fails. Map3D's LiDAR path claimed none on ideal depth, and passes. -## Answers - -What the ETH3D numbers measure: the error in the distance between two scanned surface points, against a laser scan, reported separately for surface interiors, vertical surfaces (walls, fences) and depth edges (window frames, equipment, fence edges). Clearance endpoints are often edges, and edges score worst. Every "p90" below counts failures (no prediction, or a taped reference that no scale could match) as infinitely wrong. - -1. **ARKit drift: far worse than plus or minus 0.3 ft beyond a few feet, on the only phone measured.** On a 2018 iPhone 6s outdoors, ARKit and ARCore on the same rig disagree about a walked distance by a median 2.8 in after 3 ft (p90 8.2 in), 8.6 in after 10 ft (p90 22.3) and 25.7 in after 30 ft (p90 60.1). That is a disagreement between two trackers, not a bound on ARKit's error, but most of it is one steady offset that GPS attributes to ARKit: it reads distances 5% to 17% short. Position error, which also counts sideways drift, is larger against ADVIO's truth: p90 18.6, 55.7, 94.6 and 132.6 in after 3, 10, 20 and 30 ft. That is two to three times the server's allowance of 0.16 ft per ft, though the truth's own heading and position noise inflate it. One of four walks lost tracking entirely. -2. **Photos alone: no.** Each model's own metric scale is off. Within 6 m of the camera, MoGe-2 reads 4% to 12% long and Depth Anything 3 metric 7% to 11% short: median errors of 6 to 8 in on 1 to 3 m spans, p90 about 20 in. -3. **One photo plus one taped distance: good median, loose tail, worst on walls and edges.** A single MoGe-2 photo scaled by a taped 1 to 3 m distance is off a median 1.6 in on 1 to 3 m spans (electro, within 6 m). The p90 is 9.6 in on surfaces overall, 15.1 in on vertical surfaces and 26.9 in at edges. That settles only clear-cut placements. -4. **More photos with true poses and no rescale: no gain.** On a fixed set of points and pairs, per-photo depth averaged over 1, 2, 4 and 8 photos with the true poses gives, with one tape, medians of 1.8, 2.3, 2.5 and 2.8 in and p90s of 8.6, 11.4, 15.7 and 10.8 in on 1 to 3 m spans. Each photo carries its own scale error, and averaging mixes them. -5. **AR poses fixing scale: not to about 4 in, even at an assumed 2% pose error.** Rescaling each photo's MoGe-2 depth to points triangulated with the poses, then fusing 8 photos, gives these p90s on 1 to 3 m spans (electro, within 6 m, no tape, 95% intervals over seed groups and 5 pose-noise draws): - - exact poses: 4.0 in [3.3, 4.8] overall, 2.8 in [2.0, 4.4] on walls; - - an assumed modern phone (2% scale error, 1 cm and 0.1° of noise): 5.6 in [4.6, 6.7] overall, 5.0 in [3.8, 6.8] on walls; - - the errors measured on the 2018 phone: 14.6 in [10.6, 18.7] overall, 10.8 in [8.6, 13.1] on walls. - - These assume perfect feature matching across photos; without it, walls reach 6.3 in even with exact poses. Edges stay at 8 in or worse everywhere. The pose's scale error passes straight into every length. MapAnything given the poses fixes its scale on one scene but not the other, and leaves 10% to 20% of pairs without a prediction. - - **Verdict:** neither way reaches about 4 in at p90 with realistic pose error. An earlier single noise draw showed walls at 3.9 in; with five draws that was the lucky end of the spread. -6. **Which photos to keep: the close ones.** Keeping only photos with the wall within 6 m cuts a single photo's p90 from 11.4 to 6.9 in on 1 to 3 m spans, and from 39.0 to 14.8 in on 3 to 10 m spans. Viewing angle barely mattered. -7. **The field session: not measured yet.** `make field` (section 5) puts the phone's AR taps and three learned-depth rows in one table against tomorrow's tape survey, with the phone's AR scale error beside it. It runs end to end on the ADVIO replay and on a synthetic survey, but the numbers need the real session. -8. **A current iPhone's ARKit scale: within 2% of the only public reference, which cannot itself be checked to 2%.** On MARViN's 35 outdoor walks (iPhone 14 Pro Max, ARKit 6, 45 to 255 m each), ARKit's scale matched the dataset's COLMAP reference within 2% on 30. Scene by scene the median walk reads +0.3%, +1.2% and −1.7%. So section 3's modern_assumed 2% is plausible for this phone, though even that setting leaves walls at a p90 of 5.0 in [3.8, 6.8]. It is not proven: that reference gets its meters from its authors, not from a tape or a laser, and GPS can only check it to several percent. The phone also has LiDAR, so it may track better than the LiDAR-less phones we target. Its position error p90 over trusted walks is 8.6, 13.4 and 18.5 in after 10, 20 and 30 ft (50.0 in at 30 ft counting the three doubtful walks), inside the server's 0.16 ft per ft allowance. The field tape test stays decisive (section 6). - -9. **Does the app's coverage map claim surface no photo saw? Yes, on the one wall tested: 1.1 ft of 19.3 ft claimed (6%), against a bar of 0.5 ft.** All of it sits behind equipment standing in front of the wall (a scaffold's footings and cables), because the coverage code checks range, angle and image bounds but not occlusion (`CoverageMap.swift` lines 241 to 248 at beede15). The ground band and facade could not be tested: ETH3D's photographers stood too far from the walls for the app to credit any ground, or any of facade (section 7). Requiring views farther apart or at wider angles does not fix it without discarding good wall (6 to 8.5 ft here). A depth test against true depth does, which needs LiDAR (section 7b). - -10. **The app's 3D map (Map3D), LiDAR path, ideal depth: no wall claimed that no photo saw, where `CoverageMap` claimed 1.1 ft.** It misses 8.9 ft, 5.5 ft of it behind pilasters that stand out past its 10 cm face window. Its wall line runs 1.76° off and 4.8 in out at the meter. The non-LiDAR path, ground and overhead are untested, and its one facing claim sits below the truth's resolution (section 7c). - -## Reproduce - -From a clean checkout (macOS with Apple silicon, or Linux for everything but the models): - -```sh -cd experiments/evals -uv sync -make test # metric code against hand-computed cases, lint, format -make drift # ADVIO download (about 800 MB) -> results/advio_drift.md -make replay # the replay session -> ~/house-scanning-data/replays/ -make recon # ETH3D download (about 2.4 GB), MoGe-2 and Depth Anything 3 runs -> results/eth3d_recon.md -make sensitivity # after recon -> results/eth3d_visibility_sensitivity.md -make pose-priors # after recon -> results/pose_priors.md -make frames # after recon -> results/frames.md -make field SESSION=... TRUTH=... MAP=... # a field session, section 5; PR #4's harness is checked out at a pinned commit -make modern-arkit # MARViN pose files (about 3 MB) -> results/modern_arkit.md, section 6 -make coverage # after recon-data; the app's coverage code at a pinned commit -> results/coverage.md, section 7 -make coverage-options # after coverage -> results/coverage_options.md, section 7b -make map3d # after coverage; the app's Map3D at a pinned commit -> results/map3d.md, section 7c -``` - -Data lives in `~/house-scanning-data/` (override with `HOUSE_SCANNING_DATA`). `evals/datasets.py` checks each archive's size and sha256, unpacks only what the evals read, deletes the archive, and refuses to download with less than 6 GB free. Both datasets are licensed for non-commercial research: they measure accuracy here and are never committed, redistributed or used for training. - -Every process stays near or under 4 GB, because the machine is shared. MapAnything's 1.2 B parameters are loaded with their transformer weights in bf16 (`models/map_anything.py`), and even then only 2 and 4 views at 392 px fit (peak 4.05 GB). An earlier run at its native 518 px, with fp32 weights, read 10% to 33% short, where the 392 px run reads 17% long on electro's near walls and 14% short on facade. So its metric scale depends on input resolution, and neither run is a stable baseline. - -## Datasets - -| Dataset and license | File | URL | Bytes | sha256 | -| --- | --- | --- | --- | --- | -| ADVIO, CC BY-NC 4.0 | advio-20.zip | https://zenodo.org/record/1476931/files/advio-20.zip | 195538352 | be21154394df09d6ecebe1894062f290bb53d74a5c6ccbca4b3a69f91ee2c8a2 | -| | advio-21.zip | https://zenodo.org/record/1476931/files/advio-21.zip | 209791503 | fb8a1cf3f645bbd9ea848e66924e83ae75b5cd18d632986e5629af1eecb58570 | -| | advio-22.zip | https://zenodo.org/record/1476931/files/advio-22.zip | 254664089 | 96c7212fc9cb610a88ba9fa62ac2581f0cdae49e80007888698ee2cf5ebd56f4 | -| | advio-23.zip | https://zenodo.org/record/1476931/files/advio-23.zip | 134340011 | 726d9b80d30036a8585f6cecaae385236b689cc4b8b19be6362c6df223825dc1 | -| | iphone-04.yaml (calibration, sequences 20 to 23; copied into `evals/advio.py`) | https://raw.githubusercontent.com/AaltoVision/ADVIO/master/calibration/iphone-04.yaml | | 6f312ace74f23e41b755fa40829e30a4f96e8abd3c28e6bdbac2e5e39f960813 | -| ETH3D, CC BY-NC-SA 4.0 | facade_dslr_undistorted.7z | https://www.eth3d.net/data/facade_dslr_undistorted.7z | 1252088400 | 046e577388db0633eeb2d8d72da6a2de53857434b4a7a98e4c97485795f9ce82 | -| | facade_dslr_scan_eval.7z | https://www.eth3d.net/data/facade_dslr_scan_eval.7z | 176517224 | d686462d417fbea010d0021f918bec8b881444929ec06e3bd4e8a5230ebf59a8 | -| | facade_dslr_occlusion.7z | https://www.eth3d.net/data/facade_dslr_occlusion.7z | 144292883 | 26a9d2076ba16b7f8e24311579c47b0b7b826b180906ced20fa1cba8f3dd130d | -| | electro_dslr_undistorted.7z | https://www.eth3d.net/data/electro_dslr_undistorted.7z | 514345623 | 0d2bc31fec0032b8fb20703abba8e45ef7a395c13d2d3476adc1f4dd4ffd7d8b | -| | electro_dslr_scan_eval.7z | https://www.eth3d.net/data/electro_dslr_scan_eval.7z | 250703286 | 5ca10a73e7da0e3c511e255bba5656cca4c372dc00c1417c97b75228a5bb3bac | -| | electro_dslr_occlusion.7z | https://www.eth3d.net/data/electro_dslr_occlusion.7z | 47214879 | cc25a22de9fc27251a5178454c24785671a2b2c199756a0d03297413290830b3 | - -MARViN (iPhone 14 Pro Max; https://github.com/XRIM-Lab/MarViN) states no data license, so it is used only to measure accuracy and never redistributed. Only its pose and GPS text files are fetched, from its public Google Drive folder, and each file's sha256 is listed in [results/modern_arkit.md](results/modern_arkit.md). - -Model checkpoints (MIT and Apache-2.0) are pinned by revision and checked against Hugging Face's sha256 (`models/README.md`). - -## 1. ARKit drift outdoors (ADVIO) - -[results/advio_drift.md](results/advio_drift.md), `make drift`. - -**Data.** ADVIO's four outdoor walks (sequences 20 to 23, 3 to 6 minutes and 380 to 510 m each), recorded in 2018 on a rig carrying an iPhone 6s (ARKit 1.0, video, GPS) and a Google Pixel running ARCore. The ground truth is an inertial track pinned to fix points marked on a map. - -**Method.** From every half second of a walk, the walk continues until exactly 3, 10, 20 or 30 ft of path has been covered. The error is ARKit's straight-line displacement minus the reference's. It needs no heading alignment, and it is what a span measured by walking it would be off by. - -**Why ARKit is not scored against ADVIO's ground truth alone.** Its scale is wrong in walks 20 and 21. ARKit covers only 0.70 and 0.75 of the truth's distance there, which is why ARKit's path totals 345 m against the truth's 474 m in walk 20. GPS scales the truth by 0.835 and 0.799 (95% intervals 0.81 to 0.86 and 0.78 to 0.82), and ARCore agrees (0.80 and 0.82). In walks 22 and 23 both agree with the truth's scale. The fix points explain it: walks 20 and 21 were pinned on one map at 0.0422 m per map pixel, and 22 and 23 on another at 0.2256 m per pixel. So most of the path-length gap is the reference's error; the rest is ARKit reading 5% to 17% short (ARKit over GPS: 0.83, 0.94, 0.95 in walks 20 to 22). - -The truth also wanders between fix points. The results file splits the variance of each pairwise difference into per-tracker parts, but that split assumes the three trackers' errors are independent. Walk 22 violates it: it gives negative variances, and at 3 ft it puts ARCore's spread (16.3 in) above the truth's (8.1 in). Read the split as conditional, not as a resolution limit or a bound. - -**Results, pooled over walks 20 to 22** (walk 23: ARKit jumped at 826 m/s 5.5 s in and never recovered). |ARKit − reference|, inches, median / p90: - -| Walked | vs truth as published | vs truth rescaled to GPS | ARKit − ARCore disagreement | Signed median, ARKit − ARCore | -| --- | --- | --- | --- | --- | -| 3 ft | 9.7 / 17.8 | 4.7 / 13.1 | 2.8 / 8.2 | −2.4 | -| 10 ft | 31.6 / 55.8 | 15.4 / 40.5 | 8.6 / 22.3 | −8.1 | -| 20 ft | 60.4 / 106.0 | 30.9 / 70.1 | 16.9 / 41.5 | −15.9 | -| 30 ft | 88.2 / 156.8 | 44.5 / 92.9 | 25.7 / 60.1 | −24.8 | - -The ARCore column is the tightest comparison, but it is not a conservative estimate of ARKit's error: two trackers on one rig can share errors that cancel in their difference. - -**Position error.** Distance error only compares how far each track moved, so it cannot see sideways drift, and placement relative to the meter needs the whole displacement. Here ARKit's displacement is turned by the heading difference at each window's start, against the GPS-rescaled truth, both as tracked and with each walk's own ARKit scale divided out (0.838, 0.943 and 0.951 in walks 20 to 22). Pooled over walks 20 to 22, median / p90 inches: - -| Walked | As tracked | Scale removed | Server allowance, 0.16 ft per ft | -| --- | --- | --- | --- | -| 3 ft | 8.3 / 18.6 | 7.7 / 20.4 | 5.8 | -| 10 ft | 26.3 / 55.7 | 24.5 / 59.6 | 19.2 | -| 20 ft | 50.4 / 94.6 | 47.7 / 102.5 | 38.4 | -| 30 ft | 73.8 / 132.6 | 70.4 / 144.6 | 57.6 | - -Removing the scale barely changes the numbers, so what remains is heading and random error. Much of it is the reference's own: its random error is the largest of the three trackers', and its heading at each window's start enters directly. These numbers therefore overstate ARKit's error, but they are the only position errors this data supports. ARCore cannot serve as a position reference, because its orientation convention is undocumented. - -**Limits.** A 2018 phone with ARKit 1.0, walking briskly with the camera pointed along the path. Current phones may track better; there is no data on them here. - -## 2. Reconstruction accuracy on building walls (ETH3D) - -[results/eth3d_recon.md](results/eth3d_recon.md), `make recon`; [results/eth3d_visibility_sensitivity.md](results/eth3d_visibility_sensitivity.md), `make sensitivity`. - -**Data.** ETH3D's facade (76 photos) and electro (45 photos) scenes: building walls photographed with a 24 MP DSLR, camera poses registered to terrestrial laser scans. Photos are resized to 1024 px wide and fed to each model with their known intrinsics. - -**What is scored.** The error in the distance between two scanned points, for pairs 1 to 3 m and 3 to 10 m apart. Distances do not change under rotation or translation, so no alignment step can hide error. Scale error is the median of predicted over true length, minus one. - -**How points are chosen** (`evals/eth3d.py`, `evals/recon.py`): -- **Visibility.** Scan points are projected into each view. A point is visible when it is within 4% of the nearest scanned depth along its pixel, in a z-buffer that also holds ETH3D's occlusion splats. Points in ETH3D's masked regions are dropped: glass, and objects missing from the scan such as trees. The masks are drawn on the distorted originals, so they are stretched to the undistorted frame and dilated 13 px. -- **Cohorts.** Points where the depth jumps by more than 10% within 5 pixels are depth edges; the rest are surface interior, and interior points whose scan normal is within 17° of horizontal form the vertical cohort. -- **Fixed evaluation sets.** Each of 8 seed photos per scene fixes a set of points (those it sees; within 6 m of it for the near slice), test pairs and 25 taped reference pairs. The same set scores every method and every group size (the seed plus its 1, 3 or 7 nearest cameras facing the same way). Other photos only add predictions for those points, and in the near slice a photo contributes a point only if its own camera is within 6 m of it. -- **Failures.** A pair with no prediction, or a taped reference no scale can match, counts as infinitely wrong. The tables report the calibration success rate. - -**Checks.** Scan depths agree with ETH3D's own sparse 3-D points to a median 0.2% to 0.4%. Depth rendered from the scan itself, scored through the same pipeline, is off 0.2 to 0.8 in median on 1 to 3 m spans (p90 under 2.5 in): the evaluation's floor. Changing the visibility tolerance to 2% or 8% moves model scale errors by about 1 point and medians by about 0.5 in, but p90s by up to 20% (MoGe-2) and more for Depth Anything 3. Treat p90s as carrying roughly that much evaluation uncertainty. - -**Results at phone range: electro, within 6 m** (facade's photos were taken from 10 to 25 m). |length error|, inches, median / p90; "all photos" rows score every photo on its own evaluation set: - -| Method | Photos | Cohort | Model scale: 1-3 m | Scale error, 1-3 m pairs | One taped distance: 1-3 m | 3-10 m | -| --- | --- | --- | --- | --- | --- | --- | -| MoGe-2, one photo | all | surface interior | 6.8 / 19.5 | +3.5% | 1.6 / 9.6 | 3.0 / 17.8 | -| | | vertical interior | 7.1 / 24.7 | +1.0% | 1.8 / 15.1 | 3.6 / 58.2 | -| | | edges | 8.3 / 31.8 | +3.6% | 3.7 / 26.9 | 5.8 / 36.8 | -| Depth Anything 3 metric, one photo | all | surface interior | 6.0 / 20.2 | −6.6% | 2.9 / 25.3 | 6.1 / 62.6 | -| MoGe-2 per photo + true poses | 1 | surface interior | 6.9 / 14.9 | +4.4% | 1.8 / 8.7 | 3.5 / 17.5 | -| | 2 | | 6.8 / 15.0 | +2.4% | 2.3 / 11.3 | 4.0 / 21.1 | -| | 4 | | 6.1 / 14.4 | +2.5% | 2.5 / 15.7 | 4.4 / 33.1 | -| | 8 | | 5.4 / 12.5 | +0.7% | 2.8 / 10.8 | 4.3 / 21.1 | -| Scan rendered as depth (floor) | 1 | surface interior | 0.7 / 1.6 | −0.9% | 0.5 / 1.6 | 0.9 / 2.8 | - -Back-projection uses ETH3D's own intrinsics, not the centred ones MoGe-2 reports (a 0.8 to 3.6 px difference that moves medians by at most 0.1 in). - -Over all points (4 to 23 m away), MoGe-2 reads 8% short on facade and 0.5% long on electro, and Depth Anything 3 metric 10% to 15% short. With a tape, one MoGe-2 photo is off a median 2.5 to 2.9 in on 1 to 3 m spans. - -**Limits.** Two institutional buildings, not houses, photographed with a DSLR that is sharper than a phone camera. The taped distance is simulated as exact. Facade's near slice holds only 2 of its 8 groups, too few to conclude from. - -## 3. Pose priors: can AR poses fix the scale? (ETH3D) - -[results/pose_priors.md](results/pose_priors.md), `make pose-priors`. - -**Poses.** ETH3D's true poses, degraded per group of photos (`evals/ar_poses.py`). Each setting is an assumption: -- **exact:** the true poses. -- **advio_2018:** each group takes one ADVIO walk's measured ARKit scale against the GPS-rescaled truth (0.838, 0.943 or 0.951; ARCore itself reads about 4% short of GPS, so it is not the reference). It adds 5 cm and 0.2° of noise per camera, from ARKit's own 3 in spread over 3 ft and the 0.22° image-vs-pose disagreement on the replay. -- **modern_assumed:** 2% short, 1 cm and 0.1°, a guess for a current iPhone with no data behind it. - -**Methods.** -- **(a) MapAnything** given the poses and intrinsics as priors, at 392 px with 2 and 4 photos (the memory limit); the same runs without poses are its baseline. -- **(b) Triangulation rescale.** MoGe-2 depth for each photo is rescaled to the points triangulated from SIFT matches across the group with those poses (`evals/triangulate.py`: one factor per photo, the median ratio over at least 20 points that reproject within 2 px plus the pose noise), then placed with the same poses and fused. -- **Control:** MoGe-2 placed with the same poses without the rescale. - -**Results: electro, within 6 m, 1 to 3 m spans, no tape, 8 photos.** Median / p90 inches with 95% intervals. The intervals bootstrap over the seed groups (7 for surface interior, 6 for walls) and over 5 pose-noise draws for each AR-like setting; exact poses have no noise to draw. - -| (b) MoGe-2 rescaled by triangulation | Surface interior | Vertical interior (walls) | Scale error | -| --- | --- | --- | --- | -| exact poses | 1.3 [1.0, 1.5] / 4.0 [3.3, 4.8] | 1.1 [0.8, 1.4] / 2.8 [2.0, 4.4] | −0.6% | -| modern_assumed (2% short, 1 cm, 0.1°) | 2.5 [2.1, 2.9] / 5.6 [4.6, 6.7] | 2.3 [1.7, 2.8] / 5.0 [3.8, 6.8] | −3.0% | -| advio_2018 (measured on the 2018 phone) | 6.2 [4.6, 8.6] / 14.6 [10.6, 18.7] | 4.9 [3.7, 6.5] / 10.8 [8.6, 13.1] | −7.3% | - -Edges stay at 8 in or worse at p90 in every setting. The control, MoGe-2 placed with the same poses and no rescale, has a p90 of 12.5 in with exact poses. With 4 photos, (b)'s p90 more than doubles. - -These rows assume perfect cross-view correspondence: each scan point is found in every photo that sees it through the true projection, as a perfect feature matcher would. An app without a matcher would instead reproject the tapped photo's own estimate into the others. Done that way, the wall p90 with 8 photos rises from 2.8 to 6.3 in with exact poses, from 4.3 to 6.4 in with modern_assumed and from 10.3 to 14.8 in with advio_2018, on one noise draw each. Medians move by 0.2 in or less. - -(a) MapAnything given the poses, at 392 px with 4 photos, fixes most of its scale on electro (+17% becomes about +3%) but not on facade (still 12% to 16% short). It leaves 10% to 20% of pairs without a prediction, so its p90 fails. Its rows have a single noise draw: its weights were removed to free disk. - -**What decides it.** Triangulation fixes each photo's own scale error, which is why 8 rescaled photos beat 8 unscaled ones by a factor of three. But the triangulated points take their scale from the poses, so the pose's scale error passes straight into every length: 2% short in, 3% short out. A taped distance on top makes it worse, not better (p90 7.6 in with exact poses): the reference pair carries its own reconstruction error, which outweighs the scale error that is left. - -## 4. Which photos to keep (ETH3D) - -[results/frames.md](results/frames.md), `make frames`. Each photo is scored alone (MoGe-2, one taped distance, surface interior) and described by what an app can measure while capturing: the median distance to the wall points in view, and the median angle between the viewing ray and the scanned surface. Both scenes pooled, median / p90 inches: - -| Keep | Photos kept | 1-3 m spans | 3-10 m spans | -| --- | --- | --- | --- | -| every photo | 121 of 121 | 2.7 / 11.4 | 7.6 / 39.0 | -| wall within 6 m | 15 of 121 | 1.9 / 6.9 | 4.3 / 14.8 | -| wall within 8 m | 27 of 121 | 2.1 / 7.4 | 4.8 / 17.0 | -| wall within 12 m | 42 of 121 | 2.3 / 8.1 | 5.4 / 20.4 | -| wall seen within 40 degrees of head-on | 52 of 121 | 2.3 / 9.3 | 7.1 / 35.2 | -| wall seen more than 50 degrees off head-on | 40 of 121 | 2.8 / 11.7 | 7.1 / 32.8 | - -No photo here is closer to head-on than 26 degrees, so a head-on shot is untested, and all 15 photos within 6 m are from electro. - -## 5. A field session against the tape - -`make field SESSION=session.zip TRUTH=survey.json MAP=map.json RULES=rules.json SCORING=../scoring` turns one Measure Lab session (PR #7's one-hour tape protocol) and its tape survey into one table of every method, scored by the scoring harness (PR #4, `experiments/scoring`). The map is the same file `score import-measure-lab` reads. The command runs these steps (`evals/field.py`): - -1. It turns each keyframe upright and runs MoGe-2 on it. Session images are sideways sensor images, and the model expects upright photos. **Run `make field` with nothing else heavy on the machine:** MoGe-2 at its default resolution peaks at 4.07 GB, just over the shared machine's 4 GB per process. Depth maps are written as each keyframe finishes, so the peak does not grow with session length, and a 3.6 GB GPU cap stops a run that would grow instead of letting it swell. Storing its encoder in half precision to get under 4 GB failed: parts of the encoder run outside autocast and reject fp16 weights. -2. It gives every point the rig made from taps a second position: MoGe-2's depth at the tapped pixel, placed with that keyframe's AR pose. It then recomputes walls and every session measurement with Measure Lab's own formulas (`Wall.swift`, `Measurements.swift`). -3. It writes three scoring-harness results files: - - `moge2`: the model's own scale; - - `moge2-triangulated`: method (b) of section 3, each keyframe rescaled to features triangulated with the session's AR poses across it and its 7 nearest keyframes; - - `moge2-tape`: one depth scale for the session, solved so the survey's scale reference comes out at its taped length. Each point moves along its ray from its own keyframe's camera and camera positions don't scale, so it solves along the rays rather than dividing lengths; a length ratio is exact only when both taps share a keyframe. The reference must be a straight point-to-point measurement. - - They state no uncertainty and make no decisions: none of them has a validated error bar. -4. It imports the rig's own row with `score import-measure-lab` and scores every row with `score`. -5. It writes `field_report.md` with the phone's AR scale error, the number section 3's verdict hinges on: - - Only measurements the rig accepted count, as in PR #4's importer. - - The scale is a least-squares fit over straight spans of 10 ft or more, weighted by length, with a 95% bound per span and overall. The bound combines the tape's ± with an assumed 2 in of tapping error per span, which the field test itself will check (`--tap-error-in`). - - It needs an accepted span of about 30 ft (29 ft or more), and answers "within 2%" as yes, no or cannot tell, under the same strict rule as placements. - - Each session writes to its own folder, cleared on a re-run, so `score` never picks up an earlier run's rows. - -MapAnything is left out. In section 3, given poses, it was worse than (b) everywhere. It left 10% to 20% of pairs without a prediction and did not fix its scale on facade. It also cannot take a phone's full keyframe set within the 4 GB limit. - -**Checked.** A hand-computed synthetic session (`tests/test_field.py`) gets all four rows through PR #4's own `score`, including its import of the rig's row, in one table: -- The rig row reads 2% short, as planted. -- MoGe-2 with depth 10% long gives every length 10% long. -- The tape row comes out exact. -- The triangulated row reports every value as failed: one keyframe has no neighbours to triangulate against. - -**On the real ADVIO replay** ([results/field-replay/field_report.md](results/field-replay/field_report.md)), the pipeline runs through MoGe-2 and triangulation. All 79 keyframes get a scale, and MoGe-2 needs a median ×0.774 (0.66 to 1.03) to agree with the iPhone 6s's ARKit poses; ARKit itself read about 16% short on this walk. Nothing more comes out, because the replay has no taps, measurements, survey or map. - -**In the field:** [field/FIELD_SHEET.md](field/FIELD_SHEET.md) lists what to bring, mark, tape and tap, in order, with the Measure Lab numbers the map expects. `field/survey.template.json` and `field/map.template.json` name every item, so the team fills in only tape readings, typed as the tape shows them (`30 0 1/8`). `make field` converts them to feet and fills in the zip's sha256 and session id. `make field-dryrun` builds a synthetic session in the sheet's order, fills the templates, and runs `make field` with them. It recovers the planted 1.5% AR scale error (−1.50% ± 0.68%) and gets all four rows through PR #4's `score`. - -## 6. A current iPhone's ARKit scale (MARViN) - -[results/modern_arkit.md](results/modern_arkit.md), `make modern-arkit`. - -**Which data, and why.** The only public data I found with a 2020-or-newer iPhone's own ARKit poses and a separate reference trajectory, downloadable today, is MARViN (Liu et al., IEEE VRW 2024). It has an iPhone 14 Pro Max (ARKit 6) walking outdoors around three sites, 45 to 255 m per walk, one image per second, with a COLMAP reconstruction of each site as the reference. Checked and not used: -- **LaMAR** (ETH and Microsoft) needs an access request. -- **ScanNet++** needs an application. -- **ARKitScenes** (iPad Pro, indoor) ships ARKit's trajectory but not its laser-registered camera poses. -- **ADVIO**'s phone is from 2016. - -**Method.** The same windows as section 1: from every image, the walk continues until the reference has covered 3, 10, 20 or 30 ft. ARKit's straight-line displacement is compared with the reference's. A walk's scale is the median ratio over windows of 10 ft or more. Distances need no axis alignment, which matters because ARKit's poses here are in Unity's axes. - -**The reference's limits.** COLMAP from one moving camera has no scale of its own, and the dataset does not say how its reconstructions were put into meters. They could have been scaled to ARKit itself, which would hide any ARKit scale error. The phone's GPS is the only independent check shipped. Over these walks GPS is 1 to 9 m off, and it puts the reference's scale between 0.92 and 1.07 even on its 11 cleanest walks, so it cannot confirm or refute 2%. The reference's own accuracy is not published. - -**Results.** Scale error of ARKit against the reference, per walk: - -| Site | Walks | Median walk | Range | -| --- | --- | --- | --- | -| atrium | 10 | +0.3% | −50.7% to +1.0% | -| bar | 13 | +1.2% | −0.4% to +2.1% | -| church | 12 | −1.7% | −2.2% to −1.0% | - -Thirty of 35 walks are within 2%. Three atrium walks read 18% to 51% short; on them, ARKit and the reference disagree about whether the phone moved at all between images, so the reference is in doubt there. Distance errors pooled over all walks, median / p90 inches: - -| Walked | As tracked | Without the three doubtful walks | -| --- | --- | --- | -| 10 ft | 2.7 / 16.4 | 2.4 / 7.1 | -| 20 ft | 4.5 / 34.8 | 4.2 / 11.9 | -| 30 ft | 5.9 / 45.8 | 5.5 / 16.2 | - -The 3 ft row in the results file is below what this reference resolves (images 0.2 to 1.4 m apart, and reference jitter up to 0.4 m), so it is not repeated here. - -**Position error**, the same windows with ARKit's displacement turned by the heading offset at each window's start. ARKit's quaternions are (w, x, y, z) in Unity's left-handed axes, and each walk is paired with the reference by the handedness that keeps the heading offset steady: it holds to 0.6° over a median walk (worst 3.1°). Median / p90 inches: - -| Walked | As tracked | Without the three doubtful walks | Server allowance, 0.16 ft per ft | -| --- | --- | --- | --- | -| 10 ft | 3.3 / 18.7 | 3.1 / 8.6 | 19.2 | -| 20 ft | 5.4 / 37.5 | 5.1 / 13.4 | 38.4 | -| 30 ft | 7.2 / 50.0 | 6.9 / 18.5 | 57.6 | - -**Verdict.** -- **Within 2%?** Probably, for this phone; not proven. Against the only reference available, this 2022 iPhone's ARKit holds its scale within 2% on 30 of 35 outdoor walks, and each site's walks agree with each other to within about 1 to 2.5%. -- **Walk-to-walk floor.** A reference scaled to ARKit would carry one scale per site, so the spread between one site's trusted walks survives it. That spread is 0.28% [0.12, 0.36] at atrium, 0.75% [0.46, 0.94] at bar and 0.34% [0.19, 0.44] at church (1 SD, 95% interval), so a single walk's scale error is at least about 0.3% to 0.8%. -- **What it means for section 3.** A pose scale error of about 2% is the first thing walls need, and this is the first evidence from a current phone that ARKit meets it. It is not enough: at 2%, walls still come out at a p90 of 5.0 in [3.8, 6.8] (section 3). The 2018 phone's 5% to 17% does not describe current hardware. -- **What stops it being settled:** - - the reference's scale cannot be checked independently to 2%; - - one site sits at a steady −1.7%, which is the phone's error or the reference's; - - the phone has LiDAR, which ARKit may use. - -The tape test on the team's phone is still the number to trust. - -## 7. Does the app's coverage map claim surface no photo saw? (ETH3D) - -[results/coverage.md](results/coverage.md), `make coverage`. - -**Why.** Every placement check rests on "unseen is not clear": the server passes a check only over the wall and ground the app exports as observed. That export is `CoverageMap.coveredIntervals` (`ios/HouseScanKit`), whose own comment says occlusion is not modelled. Nobody had tested it against what the photos actually saw. - -**Method.** -- **The app's code, unmodified.** `ios/HouseScanKit` at commit beede15 of `t3/ios-mvf`, checked out read-only outside the repo and run by `coverage_driver/` with its default `CoverageConfig`. Every ETH3D photo is a kept keyframe with normal tracking, fed in capture order. -- **The wall, as taps would give it.** Per scene, the straight stretch of wall that the most photos see within the app's 6 m. The meter sits mid-stretch, 1.5 m up. The outward normal comes from a plane fit to the scan, the ground height from the scan at the wall's foot, and the marked ends are the stretch's ends. The world is levelled on the ground plane under the cameras, because ARKit's y is up. Poses and intrinsics go in unrotated, in ARKit's camera axes. -- **Truth.** Columns 2 cm wide along the wall, each band sampled every 5 cm up the wall or out across the ground. A photo saw a sample when all of these hold: - - it lands inside the full image; - - it is within the app's 6 m; - - the camera is on the room side of the wall; - - nothing nearer hides it. "Nearer" means more than max(10 cm, 4% of depth) in front in the laser scan's depth, with ETH3D's occlusion splats included; ETH3D's masks of objects missing from the scan (people, vegetation) also count as hiding. - - Grazing angle is not part of the truth: an oblique photo still saw the surface, and the app's 65° limit shows up only as missed length. - -**Pass criteria, fixed before the run.** -- **False-observed length**: wall claimed as observed where at least 10 cm of the band (2 samples) was seen by no photo. Must be at most 0.5 ft, one 6-in cell, per scene and band; that is what "about zero" means here. It is also reported as a share of claimed length, and split by the cause at the photos the app credited: occlusion, range, frame edge. Grazing angle cannot cause it under this truth. A stricter variant counts band seen from fewer than two positions 0.25 m apart, the app's own bar for "covered"; it is reported, not graded. -- **Missed length**: band every sample of which two photos 0.25 m apart saw, which the app did not credit. No pass bar, since it costs the homeowner extra photos, not a wrong answer. It is split by why the app turned those photos down: frame edge (including its 3% margin and the band's top or bottom out of view), range, grazing angle, or only one position. -- **Validity checks**: my replica of the app's view gates (used only to name causes) must reproduce the app's own sightings, and the wall plane must fit the scan within a few centimetres. - -**Results.** Lengths in feet along the wall. - -| Scene | Band | Claimed | False-observed (share) | Pass | False-observed, 2-position truth | Missed | -| --- | --- | --- | --- | --- | --- | --- | -| electro | wall | 19.3 | 1.1 (6%), all occlusion | no | 1.3 (7%) | 0.0 | -| electro | ground | 0.0 | 0.0 | untested | 0.0 | 18.1, all grazing angle | -| facade | both | nothing: no wall within 6 m of any photo | | untested | | | - -- **Where the 1.1 ft is.** 1.05 ft is the bottom 20 cm of the wall at the stretch's left end, behind a scaffold's wooden footings and a cable bundle standing in front of the wall. The app credits it because the photos that frame it are within range and angle; no photo sees behind the footings. The other 0.07 ft is one 2 cm column at a pilaster's edge, within the scan's resolution. -- **Ground: untested.** The app claimed no ground on electro. The ETH3D photographers stood 2.0 to 7.6 m from the wall (median 4.9 m). A camera 1.5 m up must be within 3.2 m for the ground at the wall's foot to be inside the app's 65°, so every ground view was turned down: 18.1 ft seen from two positions and not credited. At the app's intended 2.6 m standoff the ground would be claimed, so these photos cannot say whether that claim would be right. -- **Facade: untested.** The building stands beyond the app's 6 m from every photo: 99% of its wall-height scan points are 6.5 m or more from the nearest photo, with a median of 11.6 m. Within range are only stair flanks, sculptures and tree trunks, none a straight wall that a photo sees 1 m of. The app claims nothing there. -- **Validity.** The replica of the app's view gates reproduces all 337 of its sightings exactly. The flat parts of the wall fit the plane to 1.8 cm RMS. Two pilasters stand 0.36 m proud of it (3.8 ft of the 19.3), so the truth samples their front faces; with the plane alone they read as 2.8 ft of false occlusion. No claimed sample was credited only where the scan is empty. Peak memory 1.3 GB. - -**Verdict.** -- **Does coverage claim unseen wall? Yes: 1.1 ft of 19.3 on electro (6%), where "about zero" allowed 0.5 ft. It fails.** All of it is occlusion, and that is the rule, not bad luck. At commit beede15, `CoverageMap.swift` lines 241 to 248 (`sees`, inside `visibleRows`) accept a sample on range, angle and image bounds alone; nothing checks whether something stands between camera and wall. The type's own comment (lines 57 to 59) says so, and adds that "the server re-checks what matters from the images". The server's contract says the opposite: it passes a check over whatever the app reports as observed. The loss is small here only because 17 photos from many angles see around most of the scaffold. A bush, a bin or an AC unit in front of a wall is claimed as observed whenever the phone stands in front of it. -- **What would fix it (for S3):** - - Give `CameraFrame` an optional per-frame depth map: ARKit's `sceneDepth` on LiDAR phones, or depth rendered from ARKit's mesh. - - In `sees`, reject a sample when that depth at its pixel is nearer than the sample by more than max(10 cm, 4%), the tolerance used here. - - Sample more rows, or check depth along each row-to-row segment. With `rowsPerBand = 3` (lines 43 to 47) the rows are 0.99 m apart, so something mounted between them passes even a depth check at the rows. - - Phones without depth can't make this check on the device. Until one side changes, either the server re-checks occlusion from the images, as the comment assumes, or the export must not be read as "seen and clear". -- **Missed wall: none.** Everything two photos saw, the app credited. -- **Limits.** One scene, one 19 ft wall, 17 contributing DSLR photos, not a walking phone capture. The truth ignores anything within max(10 cm, 4% of depth) of the wall (downspouts, conduit), which can only have hidden more false-observed length, not less. - -### 7b. Options for occlusion (ETH3D electro) - -[results/coverage_options.md](results/coverage_options.md), `make coverage-options`. - -**Question.** Which rule change stops coverage from claiming unseen wall, and at what cost in extra photos? The same harness, wall and truth as section 7. Each option is modelled in `coverage_driver/`, around the app's code at beede15, which stays unedited: -- **Position baseline b**: `CoverageConfig.coveringBaseline` (`CoverageMap.swift` line 49, used at line 181), swept at 0.25 (today), 0.5, 1 and 2 m. No LiDAR needed. -- **Angle diversity θ**: the predicate in `record` (lines 180 and 181) also requires the two views' directions to the row's centre to differ by at least θ, swept at 0°, 15°, 30° and 45°. It is crossed with b. No LiDAR needed. At θ = 0 and b = 0.25 the driver's copy of `record` must reproduce the app's own answer exactly. -- **Depth test (LiDAR)**: `sees` (lines 241 to 248) also rejects a row sample that the laser scan, standing in for LiDAR depth, shows hidden. The same test and tolerance as the truth, and ETH3D's masks count, because LiDAR sees people and trees too. Three settings: - - true depth out to the app's 6 m, 3 rows (the ceiling); - - the same with 9 rows (`rowsPerBand`, lines 43 to 47); - - 9 rows with no depth beyond 5 m, the iPhone LiDAR's stated range. - -**Pre-registered, before any run.** An option passes when false-observed length on electro's wall is at most 0.5 ft, as in section 7. Missed length is measured against the same truth for every option: band seen from two positions at least 0.25 m apart. The recommendation is the passing option with the least missed length, preferring one that needs no LiDAR when two are within 1 ft of each other. - -**Results** on electro's 19.3 ft wall, in feet: false-observed (pass at 0.5 or less) and missed (band two photos saw that the option doesn't claim). The full grid is in the results file. - -| Option | Needs LiDAR | False-observed | Pass | Missed | -| --- | --- | --- | --- | --- | -| The app today (b = 0.25 m, θ = 0°) | no | 1.1 | no | 0.0 | -| Views ≥ 2 m apart, any angle up to 45° | no | 1.1 | no | 0.0 | -| Views ≥ 5 m apart* | no | 0.0 | yes | 6.0 | -| Views ≥ 60° apart* | no | 0.0 | yes | 8.5 | -| Depth test, true depth to 6 m, 3 or 9 rows | yes | 0.0 | yes | 3.3 | -| Depth test, 9 rows, no depth past 5 m | yes | 0.0 | yes | 7.8 | -| Depth test, 9 rows, 0.4 m of wall relief allowed* | yes | 0.1 | yes | 0.0 | - -\* Added after the pre-registered grid (b 0.25 to 2 m, θ 0 to 45°) changed nothing. - -- **Spacing or angle between views: no.** Nothing in the pre-registered grid moved the 1.1 ft. The footings hide the wall's base from every direction the photos were taken from, and seeing a cell from far-apart or differently angled views says nothing about whether something stands in front of it. The patch goes only when the rule refuses cells for want of spread: at 5 m apart (6.0 ft lost) or 60° apart (8.5 ft lost, 47% of what was seen). It is removed along with good wall, not singled out. At 75°, nothing is claimed. -- **Depth test: yes, at a price the rule's shape sets.** With true depth it claims nothing unseen. The 3.3 ft it misses is the two pilasters. A tapped wall is a plane, so a pilaster 0.36 m proud of it reads as something in front of the wall, and a real LiDAR test would do the same. Allowing 0.4 m of relief keeps them (0 missed) and still rejects the footings (0.1 ft). At 0.5 m the footings get through (1.0 ft). That margin was found on this one wall and says nothing about bushes closer than 0.4 m. Nine rows instead of three changed nothing here, because the hidden patch touches the bottom row. It would matter for something mounted between rows. -- **LiDAR's reach costs coverage.** iPhone LiDAR reads to about 5 m; photos 5 to 6 m out then earn nothing, which misses 7.8 ft on these DSLR photos. At the app's 2.6 m standoff that limit would rarely bind. -- **Ground band: still untested.** Every option keeps the 65° gate, so none claims ground from these photos. None of the datasets on disk can test it: ADVIO and MARViN have no surface truth, and facade's photos stand even farther back. ETH3D's other outdoor scenes carry the same laser truth, at 0.3 to 0.7 GB each with images (courtyard, delivery_area, meadow, playground, terrace). Whether their photographers stood within 3.2 m of a wall can only be read after downloading, so I did not. - -**Recommendation.** Stay with "unseen is not clear" and make occlusion someone's job, because no rule about how the phone moved can stand in for it: -- **LiDAR phones:** a depth test in `sees` (`CoverageMap.swift` lines 241 to 248), with rows closer than 0.99 m and an allowance for wall relief. The allowance needs field data before a number ships. -- **Phones without LiDAR:** the app cannot know what hides the wall. Either the homeowner confirms nothing stands in front of each claimed stretch (for example, by marking obstacles on a still), or checks that need clear ground treat claims from those phones as unconfirmed. - -Tightening the baseline or angle would only ask for more photos: 6 to 8.5 extra feet of wall here, with no guarantee that the hidden patch goes. - -### 7c. The app's 3D map (Map3D) against the same truth (ETH3D electro) - -[results/map3d.md](results/map3d.md), `make map3d`. - -**Question.** The app's live 3D map (`ios/HouseScanKit/Sources/HouseScanKit/Map3D`, t3/ios-map3d at 66cdcba, draft PR #21) is to replace `CoverageMap` on every phone. Does it claim wall, ground, facing or overhead space no photo saw? - -**Method.** -- **The app's code, unmodified**, checked out read-only and driven by `map3d_driver/`: `Map3D.integrate(DepthFrame)` per photo, then `coverage(along:)` and `measuredWalls()`, default `Map3DConfig`. -- **The same wall as section 7:** the map frame and the `WallFrame` coverage is read along are section 7's (meter, outward normal, ground height). -- **LiDAR path:** one depth frame per photo, 256 x 192, rendered from the laser scan at the photo's pose (nearest scan depth along -z; 0 where nothing was scanned or ETH3D masks an object the scanner missed), with no confidence. -- **Non-LiDAR path: not tested.** It needs ARKit's feature points and detected planes. COLMAP's tracks on 24-megapixel DSLR photos are far denser and more exact than ARKit's feature points, and ARKit's plane detector can't be reproduced from them without guessing. -- **Truth.** Wall and ground: section 7's laser-scan visibility, on the same wall face and ground band (1.2 m, the depth `CoverageMap` claims). Map3D's ground counts as claimed where its reach is at least 1.2 m. Facing and overhead space: a point counts as seen empty when some photo within 5 m frames it and the laser scan's depth at its pixel lies beyond it by more than the section 7 tolerance, max(10 cm, 4%), so the ray passed through it. - -**Pre-registered, before any run.** -- **False-observed** (claimed columns where at least 10 cm of what the band claims was seen by no photo) must be at most **0.5 ft** per band, as in section 7, where `CoverageMap` claimed 1.1 ft of wall. -- **Missed** (wall and ground band seen from two positions 0.25 m apart but not claimed) is reported without a bar. -- **Wall chain:** the measured wall piece through the meter against the laser's wall line (angle, offset at the meter, length), reported without a bar. - -**Results** (ideal depth: exact, dense, no sensor noise). Feet along the wall. - -| Band | Claimed | False-observed | Pass | Missed | -| --- | --- | --- | --- | --- | -| Wall, `CoverageMap` (section 7's 19.3 ft stretch) | 19.3 | 1.1 | no | 0.0 | -| Wall, Map3D, same stretch | 11.1 | 0.0 | yes | 6.9 (6.1 with a 5 m truth) | -| Wall, Map3D, everything it claims (s −9.7 to +21 ft) | 19.4 | 0.0 | yes | 8.9 of 25.2 seen, 5.5 of it on pilaster faces | -| Ground | 0.0 | 0.0 | untested | 18.1 | -| Facing | 0.5 | 0.5 (see below) | unresolved | n/a | -| Overhead | 0.0 | 0.0 | untested | n/a | - -- **Wall: Map3D claims no wall that no photo saw, where `CoverageMap` claimed 1.1 ft.** The 1.1 ft behind the scaffold's footings is left unclaimed, as the depth test of section 7b predicted. -- **The cost is missed wall.** 5.5 ft of the 8.9 ft missed is behind pilasters 0.36 m proud of the wall line. Map3D counts the face only within 0.10 m in front of the line (`faceFront`, `Map3DConfig.swift` line 85 at 66cdcba, used in `Map3DCoverage.swift` lines 73 to 86), so a pilaster hides the wall behind it. That is conservative, not an over-claim. It asks for views that cannot exist, so the gap loop could ask forever for a pilaster-backed stretch. -- **Ground and overhead: untested.** Map3D claims no ground: reach 0 everywhere, the same grazing-view limit as section 7. -- **Facing: unresolved.** Map3D's one claim is a single 6 in cell seen clear 0.1 m out from a door face. By the pre-registered metric it reads 0.52 ft, just over the bar. The truth, however, cannot resolve space that close to a surface: it needs the laser surface to lie beyond a point by 4% of the distance, 12 to 20 cm here. So the claim is neither confirmed nor refuted, and a 0.1 m reach settles no check (the server needs more than 4.83 ft). -- **Wall chain:** one piece through the meter, 28.4 ft long, where the laser's wall face (pilasters included) runs 30.8 ft. The left end is 0.3 ft short and the right 2.2 ft short. The line is 1.76° off the laser face and 4.8 in outside it at the meter, more than the server's 3.6 in default meter error. The cause is not established. -- **Not measured:** the non-LiDAR path, real LiDAR noise and dropouts, and coverage along Map3D's own chain instead of section 7's wall line. An independent review of the method also flagged a rule to check: `VoxelGrid.swift` lines 376 to 378 and 409 to 412 at 66cdcba take the nearest view distance and the best view angle from possibly different views, so a surface can pass the 5 m and 65° tests with no single view meeting both. It caused no false-observed wall here. - -## Replay session from a real walk - -`~/house-scanning-data/replays/advio-20-0040-0075.zip` holds 35 s of ADVIO walk 20 (seconds 40 to 75, a path past a brick building) in Measure Lab session format v2, for the app's replay mode. It has 79 keyframes chosen by Measure Lab's rule (0.5 m or 15° since the last), plus `ground_truth.json` with ADVIO's pose for each keyframe. `make replay` rebuilds and checks it. - -- **Images.** `frames.mov` stores unrotated 1280 × 720 sensor frames behind a portrait display tag. They are decoded by index, since seeking by time lands frames off, and undistorted with ADVIO's calibration. -- **Intrinsics.** `[1082.1, 1081.1, 641.29, 359.91]`, from the portrait OpenCV calibration rotated to landscape and shifted half a pixel (`evals/camera.py`). -- **Poses.** `arkit.csv` orientations are in the portrait device frame. The converter turns them a quarter turn about the viewing axis. Image-derived rotations agree with this reading to a median 0.22° (alternatives: 4.5° to 6.5°). -- **Check.** `check_replay` reads only the session files and measures matched features' distance from the epipolar lines the poses predict: median 3.1 px, against 61 px with the axes deliberately wrong. -- **Missing from ADVIO.** No tracking state (written as `normal` once ARKit reports a position), no capture time (`startedAt` is the publication date), no taps or measurements. The camera points along the path, not at a wall. +**Changed.** The server kept its drift allowance. Map3D replaces `CoverageMap`. Occlusion needs a depth test. The field kit scores the phone against a tape. diff --git a/experiments/evals/coverage_driver/Sources/coverage-driver/main.swift b/experiments/evals/coverage_driver/Sources/coverage-driver/main.swift index 9618fbc32..8c3c36443 100644 --- a/experiments/evals/coverage_driver/Sources/coverage-driver/main.swift +++ b/experiments/evals/coverage_driver/Sources/coverage-driver/main.swift @@ -3,7 +3,7 @@ // each keyframe's sightings so the caller can attribute errors to the frames the app credited. // Without a wall it writes only the config. // -// Options for occlusion (README section 7b), modelled here so HouseScanKit stays unedited: +// Options for occlusion (METHODS.md section 7b), modelled here so HouseScanKit stays unedited: // - `variant.coveringBaseline` and `variant.rowsPerBand` set the app's own CoverageConfig fields. // - `hidden` removes rows from a keyframe's sightings before the app's `record`: a depth test that // `sees` (CoverageMap.swift lines 241-248) would make, computed by the caller. diff --git a/experiments/evals/evals/coverage.py b/experiments/evals/evals/coverage.py index 433a6d9c3..10338535c 100644 --- a/experiments/evals/evals/coverage.py +++ b/experiments/evals/evals/coverage.py @@ -1,4 +1,4 @@ -"""Does the app's coverage map claim wall and ground that no photo saw? (README section 7) +"""Does the app's coverage map claim wall and ground that no photo saw? (METHODS.md section 7) The app exports as observed the stretches `CoverageMap.coveredIntervals` returns (`ios/HouseScanKit` on t3/ios-mvf), and the server passes a check only over them. This runs that code, unmodified, on @@ -10,7 +10,7 @@ to the app in ARKit's camera axes (+x right, +y up, looking along -z), intrinsics unrotated, with (0, 0) at the image's top-left corner, the convention of `CameraFrame`. -The truth, the pass criteria and the cause labels are defined in README section 7. The only copy of +The truth, the pass criteria and the cause labels are defined in METHODS.md section 7. The only copy of app logic here is `app_rows`, a replica of `CoverageMap.visibleRows` used to name why the app turned a photo down; `main` refuses to report unless it reproduces the app's own sightings exactly. """ @@ -37,6 +37,7 @@ scan_points, up_direction, ) +from evals.pairs import results_json from evals.paths import ETH3D_DIR, EVALS_DIR KIT_COMMIT = "beede15f568b3a4d694fb275caf9eaaa882c546b" @@ -412,7 +413,8 @@ class PhotoTruth: @property def saw(self) -> np.ndarray: - return self.framed & self.in_range & self.facing & ~self.hidden + # No scan point near the pixel leaves occlusion unknown, which is not seen. + return self.framed & self.in_range & self.facing & ~self.hidden & ~self.no_scan def photo_truth( @@ -602,8 +604,10 @@ def evaluate_band( ft = COLUMN_M / FEET claimed_ft = claimed.sum() * ft - no_scan = np.stack([t.no_scan & t.saw for t in truths], axis=-1) - only_unknown = seen1 & ~(saw & ~no_scan).any(axis=-1) + # Samples no photo saw, but where some photo framed them over an empty scan pixel: unknown + # rather than unseen. Counted apart so a laser hole is visible in the result. + unknown = np.stack([t.framed & t.in_range & t.facing & t.no_scan for t in truths], axis=-1) + only_unknown = ~seen1 & unknown.any(axis=-1) return { "claimed_ft": claimed_ft, "false_observed_ft": false_cols.sum() * ft, @@ -614,7 +618,7 @@ def evaluate_band( "seen_two_view_ft": seen2.all(axis=1).sum() * ft, "missed_ft": missed_cols.sum() * ft, "missed_by_cause_ft": {k: missed_causes[k] * ft for k in MISSED_CAUSES}, - "claimed_samples_seen_only_where_scan_is_empty": float(only_unknown[claimed].mean()) + "claimed_samples_unknown_for_lack_of_scan": float(only_unknown[claimed].mean()) if claimed.any() else 0.0, # Nothing claimed tests nothing. @@ -816,7 +820,7 @@ def markdown(runs: list[dict]) -> str: f"HouseScanKit `CoverageMap` at {KIT_COMMIT} (t3/ios-mvf), unmodified, default " "`CoverageConfig`. ETH3D photos with true poses, " "every photo a kept keyframe. Lengths in feet along the wall. Definitions and pass criteria: " - "README section 7.", + "METHODS.md section 7.", "", "| Scene | Band | Claimed | False-observed (share) | Pass (<= 0.5 ft) | Causes: occlusion / frame edge / range | False-observed, 2-position truth (share) | Seen from 2 positions | Missed | Missed causes: frame edge / grazing / range / one position |", "| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |", @@ -839,7 +843,7 @@ def markdown(runs: list[dict]) -> str: "", "## Setup and validity checks", "", - "| Scene | Photos (with a sighting) | Wall stretch | Of it, pilaster faces | Wall plane fit RMS | Ground fit RMS | App sightings | Replica mismatches | Claimed samples seen only where the scan is empty (wall / ground) |", + "| Scene | Photos (with a sighting) | Wall stretch | Of it, pilaster faces | Wall plane fit RMS | Ground fit RMS | App sightings | Replica mismatches | Claimed samples unknown for lack of scan (wall / ground) |", "| --- | --- | --- | --- | --- | --- | --- | --- | --- |", ] for r in runs: @@ -854,8 +858,8 @@ def markdown(runs: list[dict]) -> str: f"{r['wall_length_ft']:.1f} ft | {r['pilaster_ft']:.1f} ft | " f"{r['wall_fit_rms_cm']:.1f} cm | {r['ground_fit_rms_cm']:.1f} cm | " f"{r['sightings']} | {r['replica_mismatches']} | " - f"{100 * b['wall']['claimed_samples_seen_only_where_scan_is_empty']:.1f}% / " - f"{100 * b['ground']['claimed_samples_seen_only_where_scan_is_empty']:.1f}% |" + f"{100 * b['wall']['claimed_samples_unknown_for_lack_of_scan']:.1f}% / " + f"{100 * b['ground']['claimed_samples_unknown_for_lack_of_scan']:.1f}% |" ) return "\n".join(lines) + "\n" @@ -865,7 +869,7 @@ def main() -> None: runs = [evaluate_scene(scene) for scene in RUNS] bad = [r["scene"] for r in runs if r["wall"] and r["replica_mismatches"]] RESULTS.mkdir(exist_ok=True) - (RESULTS / "coverage.json").write_text(json.dumps(runs, indent=1)) + (RESULTS / "coverage.json").write_text(results_json(runs)) if bad: raise SystemExit(f"cause replica disagrees with the app on {bad}; causes would be wrong") md = markdown(runs) diff --git a/experiments/evals/evals/coverage_options.py b/experiments/evals/evals/coverage_options.py index 4c738987e..c397b341b 100644 --- a/experiments/evals/evals/coverage_options.py +++ b/experiments/evals/evals/coverage_options.py @@ -1,4 +1,4 @@ -"""Options for occlusion in the app's coverage map, on section 7's wall and truth (README section 7b). +"""Options for occlusion in the app's coverage map, on section 7's wall and truth (METHODS.md section 7b). Each option runs the app's CoverageMap (HouseScanKit at `coverage.KIT_COMMIT`, unedited) through `coverage_driver/`, which models the rule change around it: @@ -11,8 +11,6 @@ from __future__ import annotations -import json - from evals.coverage import ( FEET, HIDE_ABS_M, @@ -25,6 +23,7 @@ run_app, setup_scene, ) +from evals.pairs import results_json SCENE = "electro" # Pre-registered: 0-45 degrees and 0.25-2 m. 60 and 75 degrees and 5 m were added after that grid @@ -64,7 +63,8 @@ def depth_hidden( setup.cfg["maxDistance"] if reach is None else reach, relief, ) - bad = (t.hidden | ~t.in_range).any(axis=1) + # A depth test with no depth there (no scan return) cannot credit the row. + bad = (t.hidden | t.no_scan | ~t.in_range).any(axis=1) if bad.any(): out.append({**x, "rows": [r for r, b in zip(x["rows"], bad, strict=True) if b]}) return out @@ -80,7 +80,9 @@ def _row(label: str, bands: dict, lidar: bool) -> dict: "missed_ft": wall["missed_ft"], "ground_claimed_ft": bands["ground"]["claimed_ft"], "ground_false_observed_ft": bands["ground"]["false_observed_ft"], - "passes": bool(wall["false_observed_ft"] <= PASS_FT), + # An option that claims no wall tested nothing, so it neither passes nor fails; the main + # evaluator (`evals.coverage`) records the same case as untested. + "passes": bool(wall["false_observed_ft"] <= PASS_FT) if wall["claimed_ft"] > 0 else None, } @@ -123,6 +125,14 @@ def depth_option(label: str, n_rows: int, reach: float | None, relief: float) -> } +PASS_CELL = {True: "yes", False: "no", None: "untested"} + + +def false_observed_cell(row: dict) -> str: + """False-observed feet, or "untested" when the option claimed no wall to test.""" + return "untested" if row["passes"] is None else f"{row['false_observed_ft']:.1f}" + + def markdown(res: dict) -> str: lines = [ "# Options for occlusion in the coverage map (generated by `make coverage-options`)", @@ -133,14 +143,14 @@ def markdown(res: dict) -> str: "", "## Position baseline b and angle diversity theta (no LiDAR): false-observed / missed", "", + "An option that claims no wall is untested, not passed.", + "", "| b \\ theta | " + " | ".join(f"{t} deg" for t in ANGLES_DEG) + " |", "| --- |" + " --- |" * len(ANGLES_DEG), ] for b in BASELINES_M: cells = [ - f"{g['false_observed_ft']:.1f} / {g['missed_ft']:.1f}" - for g in res["grid"] - if g["b_m"] == b + f"{false_observed_cell(g)} / {g['missed_ft']:.1f}" for g in res["grid"] if g["b_m"] == b ] lines.append(f"| {b:g} m | " + " | ".join(cells) + " |") lines += [ @@ -148,7 +158,7 @@ def markdown(res: dict) -> str: "## Options", "", "The last four rows allow that much wall relief in front of the tapped plane, a setting " - "chosen after seeing this wall; see README section 7b.", + "chosen after seeing this wall; see METHODS.md section 7b.", "", "| Option | Needs LiDAR | Claimed | False-observed | Pass | Missed | Ground claimed |", "| --- | --- | --- | --- | --- | --- | --- |", @@ -156,7 +166,7 @@ def markdown(res: dict) -> str: for r in res["options"] + res["relief"]: lines.append( f"| {r['option']} | {'yes' if r['needs_lidar'] else 'no'} | {r['claimed_ft']:.1f} | " - f"{r['false_observed_ft']:.1f} | {'yes' if r['passes'] else 'no'} | " + f"{false_observed_cell(r)} | {PASS_CELL[r['passes']]} | " f"{r['missed_ft']:.1f} | {r['ground_claimed_ft']:.1f} |" ) return "\n".join(lines) + "\n" @@ -166,7 +176,7 @@ def main() -> None: check_kit() res = evaluate() RESULTS.mkdir(exist_ok=True) - (RESULTS / "coverage_options.json").write_text(json.dumps(res, indent=1)) + (RESULTS / "coverage_options.json").write_text(results_json(res)) md = markdown(res) (RESULTS / "coverage_options.md").write_text(md) print(md) diff --git a/experiments/evals/evals/datasets.py b/experiments/evals/evals/datasets.py index b222b639b..14c8d4739 100644 --- a/experiments/evals/evals/datasets.py +++ b/experiments/evals/evals/datasets.py @@ -4,7 +4,9 @@ uv run python -m evals.datasets eth3d # ETH3D facade and electro (about 2.4 GB of archives) Each archive is checked against its size and sha256 below, unpacked (only the files the evals use), -then deleted. A download refuses to start with less than `MIN_FREE_GB` free, because the Mac these +then deleted. A completion record, written only after every expected file is in place, ties the +unpacked files to the archive's sha256; a later run skips the archive only if that record matches +and every recorded file is still there at its recorded size. A download refuses to start with less than `MIN_FREE_GB` free, because the Mac these run on is shared. Both datasets are for non-commercial research: they measure accuracy here and are never committed or redistributed. """ @@ -13,6 +15,7 @@ import argparse import hashlib +import json import shutil import urllib.request import zipfile @@ -152,6 +155,7 @@ def fetch(archive: Archive) -> None: raise SystemExit( f"{archive.name}: got {size} bytes sha256 {digest}, expected {archive.size} {archive.sha256}" ) + record_path(archive).unlink(missing_ok=True) if archive.name.endswith(".zip"): with zipfile.ZipFile(tmp) as z: names = [ @@ -160,11 +164,48 @@ def fetch(archive: Archive) -> None: z.extractall(archive.dest, members=names) else: with py7zr.SevenZipFile(tmp) as z: + names = z.getnames() z.extractall(archive.dest) tmp.unlink() + write_record(archive, names) print(f" verified and unpacked into {archive.dest}") +def record_path(archive: Archive) -> Path: + return archive.dest / ".complete" / f"{archive.name}.json" + + +def write_record(archive: Archive, names: list[str]) -> None: + """Record the unpacked files once every expected one is present: the marker, and at least one + file under each requested member.""" + files = {n: (archive.dest / n).stat().st_size for n in names if (archive.dest / n).is_file()} + missing = [m for m in archive.members if not any(n.startswith(m) for n in files)] + if not (archive.dest / archive.marker).exists(): + missing.append(archive.marker) + if missing: + raise SystemExit(f"{archive.name}: unpacked without {missing}; not recording it complete") + path = record_path(archive) + path.parent.mkdir(exist_ok=True) + path.write_text(json.dumps({"sha256": archive.sha256, "files": files}, indent=0)) + + +def is_complete(archive: Archive) -> bool: + """True when this archive's record matches its pinned sha256 and every recorded file exists at + its recorded size. Sizes catch a truncated or partly deleted copy without rehashing gigabytes; + an edit that keeps a file's size is not caught.""" + path = record_path(archive) + if not path.exists(): + return False + record = json.loads(path.read_text()) + if record.get("sha256") != archive.sha256 or not record.get("files"): + return False + for name, size in record["files"].items(): + f = archive.dest / name + if not f.is_file() or f.stat().st_size != size: + return False + return True + + def main() -> None: ap = argparse.ArgumentParser( description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter @@ -175,9 +216,8 @@ def main() -> None: ) args = ap.parse_args() for a in ADVIO if args.dataset == "advio" else ETH3D: - marker = a.dest / a.marker - if marker.exists() and not args.force: - print(f"{a.name}: already unpacked ({marker})") + if is_complete(a) and not args.force: + print(f"{a.name}: already unpacked and recorded complete ({record_path(a)})") continue fetch(a) diff --git a/experiments/evals/evals/drift.py b/experiments/evals/evals/drift.py index 846e322c1..76ab59774 100644 --- a/experiments/evals/evals/drift.py +++ b/experiments/evals/evals/drift.py @@ -32,13 +32,13 @@ from __future__ import annotations import argparse -import json from dataclasses import asdict, dataclass from pathlib import Path import numpy as np from evals.advio import PoseTrack, _read_pose_csv, load_sequence +from evals.pairs import results_json from evals.paths import ADVIO_DIR FEET = 0.3048 @@ -415,9 +415,7 @@ def main() -> None: pooled = _pooled(results) args.out.mkdir(parents=True, exist_ok=True) slim = [{k: v for k, v in r.items() if k != "errors_in"} for r in results] - (args.out / "advio_drift.json").write_text( - json.dumps({"sequences": slim, "pooled": pooled}, indent=1) - ) + (args.out / "advio_drift.json").write_text(results_json({"sequences": slim, "pooled": pooled})) md = _markdown(results, pooled) (args.out / "advio_drift.md").write_text(md) print(md) diff --git a/experiments/evals/evals/eth3d.py b/experiments/evals/evals/eth3d.py index f3fcecd6c..bfda31f64 100644 --- a/experiments/evals/evals/eth3d.py +++ b/experiments/evals/evals/eth3d.py @@ -138,7 +138,10 @@ def read_ply_xyz(path: Path) -> np.ndarray: with path.open("rb") as fh: header = [] while True: - line = fh.readline().decode("ascii").strip() + raw = fh.readline() + if not raw: + raise ValueError(f"{path}: file ends before the PLY header's end_header line") + line = raw.decode("ascii").strip() header.append(line) if line == "end_header": break diff --git a/experiments/evals/evals/field.py b/experiments/evals/evals/field.py index 888301ecd..42f010cd9 100644 --- a/experiments/evals/evals/field.py +++ b/experiments/evals/evals/field.py @@ -26,14 +26,16 @@ from __future__ import annotations import argparse +import functools import hashlib import json import re +import shutil import sys import time import zipfile from dataclasses import dataclass -from pathlib import Path +from pathlib import Path, PurePosixPath import cv2 import numpy as np @@ -59,20 +61,146 @@ # --- Session --------------------------------------------------------------------------------- +# Bounds on a session zip before anything is written. A real session is a few hundred files and +# well under 1 GB (the ADVIO replay: 81 files, 17 MB); these leave room for long LiDAR sessions while +# stopping a crafted archive from filling the shared disk. +MAX_ZIP_MEMBERS = 10_000 +MAX_ZIP_BYTES = 2 * 1024**3 +# The archive itself, checked before it is hashed or opened, and every central-directory entry +# (folders and __MACOSX included), so neither the file nor its index can be arbitrarily large. +MAX_ZIP_ARCHIVE_BYTES = 2 * 1024**3 +MAX_ZIP_ENTRIES = 2 * MAX_ZIP_MEMBERS +# The central directory ZipFile reads whole and parses into one object per entry. 20,000 entries +# with 100-byte names take about 3 MB; the cap also bounds entries with minimal 46-byte records. +MAX_ZIP_DIRECTORY_BYTES = 8 * 1024**2 +MIN_FREE_AFTER_UNPACK = 3 * 1024**3 +# Pixels in one session image, read from its header before OpenCV decodes it. The zip limits count +# encoded bytes, and a few-KB JPEG can declare 65,535 x 65,535 pixels (13 GB decoded in colour). +# ARKit keyframes are 1920 x 1440 (2.8 MP) and a 48 MP iPhone photo is 8064 x 6048 (48.8 MP), so +# 50 MP admits both, and decoding it in colour takes about 150 MB of the 4 GB per-process budget. +MAX_IMAGE_PIXELS = 50_000_000 +# Pixels held at once while fitting scales. Triangulation needs a keyframe and its NEIGHBOURS nearest +# together, so only that group's grayscale image (1 byte per pixel) and depth (4) are kept, about +# 5 bytes a pixel whatever the session's length. 200 MP is then about 1 GB, leaving the rest of the +# 4 GB per-process budget for feature matching. Eight ARKit keyframes are 22 MP; eight 48 MP photos +# are 390 MP and are refused from their headers before any is decoded. +MAX_GROUP_PIXELS = 200_000_000 + + +def safe_member(name: str) -> PurePosixPath: + """A zip member name as a relative path with no way out of the extraction root.""" + path = PurePosixPath(name) + parts = [p for p in path.parts if p not in ("", ".")] + if ( + not parts + or path.is_absolute() + or "\\" in name + or ":" in parts[0] + or any(p == ".." for p in parts) + ): + raise ValueError(f"zip member {name!r} is not a plain relative path") + return PurePosixPath(*parts) + + +def file_sha256(path: Path) -> str: + """sha256 of a file, read 1 MB at a time so a large file is never held in memory.""" + h = hashlib.sha256() + with path.open("rb") as f: + while block := f.read(1 << 20): + h.update(block) + return h.hexdigest() + + +def check_zip_directory(session: Path) -> None: + """Refuse a zip whose central directory is too large or lists too many entries, reading only + its end record. `zipfile.ZipFile` reads the whole directory and builds every entry while it is + constructed, so a later count comes too late. The end record is read by `zipfile._EndRecData`, + the function ZipFile itself uses (ZIP64 included), so the sizes checked here are the ones it + will read.""" + with session.open("rb") as f: + try: + end = zipfile._EndRecData(f) + except OSError as e: + raise ValueError(f"{session}: not a zip file ({e})") from e + if not end: + raise ValueError(f"{session}: not a zip file (no end-of-central-directory record)") + entries, directory_bytes = end[zipfile._ECD_ENTRIES_TOTAL], end[zipfile._ECD_SIZE] + if entries > MAX_ZIP_ENTRIES: + raise ValueError(f"{session}: {entries} zip entries, more than {MAX_ZIP_ENTRIES}") + if directory_bytes > MAX_ZIP_DIRECTORY_BYTES: + raise ValueError( + f"{session}: {directory_bytes}-byte zip directory, more than {MAX_ZIP_DIRECTORY_BYTES}" + ) + + def unpack(session: Path) -> tuple[Path, str | None]: - """The session folder, and the zip's sha256 (the scoring harness's capture id) if zipped.""" + """The session folder, and the zip's sha256 (the scoring harness's capture id) if zipped. + + Every member is checked before anything is written: a plain relative path, no duplicates, at + most MAX_ZIP_MEMBERS files and MAX_ZIP_BYTES in all, with MIN_FREE_AFTER_UNPACK left free. Files + are copied with a running byte count, so a member whose header understates its size still + stops at the bound. Extraction goes to a temporary folder renamed into place only when complete. + """ if session.is_dir(): return session, None - digest = hashlib.sha256(session.read_bytes()).hexdigest() + size = session.stat().st_size + if size > MAX_ZIP_ARCHIVE_BYTES: + raise ValueError(f"{session}: {size} bytes, more than {MAX_ZIP_ARCHIVE_BYTES}") + check_zip_directory(session) + digest = file_sha256(session) out = FIELD_DIR / digest[:16] with zipfile.ZipFile(session) as z: - names = [n for n in z.namelist() if not n.startswith("__MACOSX/")] - roots = [n for n in names if n.split("/")[-1] == "session.json" and n.count("/") <= 1] + infos = z.infolist() + if len(infos) > MAX_ZIP_ENTRIES: + raise ValueError(f"{session}: {len(infos)} zip entries, more than {MAX_ZIP_ENTRIES}") + members, seen = [], set() + for info in infos: + if info.filename.startswith("__MACOSX/") or info.is_dir(): + continue + rel = safe_member(info.filename) + if rel in seen: + raise ValueError(f"{session}: {rel} appears twice") + seen.add(rel) + members.append((info, rel)) + if len(members) > MAX_ZIP_MEMBERS: + raise ValueError(f"{session}: {len(members)} files, more than {MAX_ZIP_MEMBERS}") + declared = sum(info.file_size for info, _ in members) + if declared > MAX_ZIP_BYTES: + raise ValueError(f"{session}: expands to {declared} bytes, more than {MAX_ZIP_BYTES}") + roots = [rel for _, rel in members if rel.name == "session.json" and len(rel.parts) <= 2] if len(roots) != 1: raise ValueError(f"{session}: expected one session.json, found {roots}") - if not (out / roots[0]).exists(): - z.extractall(out, members=names) - return (out / roots[0]).parent, digest + root = out / roots[0] + if not root.exists(): + FIELD_DIR.mkdir(parents=True, exist_ok=True) + free = shutil.disk_usage(FIELD_DIR).free + if free - declared < MIN_FREE_AFTER_UNPACK: + raise ValueError( + f"{session}: unpacking {declared / 1e9:.2f} GB would leave less than " + f"{MIN_FREE_AFTER_UNPACK / 1024**3:.0f} GB free ({free / 1e9:.1f} GB free now)" + ) + tmp = out.with_name(out.name + ".partial") + shutil.rmtree(tmp, ignore_errors=True) + try: + written = 0 + for info, rel in members: + dest = tmp.joinpath(*rel.parts) + dest.parent.mkdir(parents=True, exist_ok=True) + with z.open(info) as src, dest.open("wb") as dst: + while block := src.read(1 << 20): + written += len(block) + if written > MAX_ZIP_BYTES: + raise ValueError(f"{session}: expands past {MAX_ZIP_BYTES} bytes") + dst.write(block) + shutil.rmtree(out, ignore_errors=True) + tmp.rename(out) + except BaseException: + shutil.rmtree(tmp, ignore_errors=True) + raise + folder = root.parent.resolve() + if not folder.is_relative_to(out.resolve()): + raise ValueError(f"{session}: session folder {folder} is outside {out}") + return folder, digest def load_session(folder: Path) -> dict: @@ -133,20 +261,103 @@ def safe_id(value: str, what: str) -> str: return value +# JPEG start-of-frame markers carry the image size; C4 (DHT), C8 (JPG) and CC (DAC) share the range +# but are not frames. +_JPEG_SOF = set(range(0xC0, 0xD0)) - {0xC4, 0xC8, 0xCC} + + +def image_size(path: Path) -> tuple[int, int]: + """(width, height) from a JPEG or PNG header, reading no pixel data. Other formats are + refused: sessions carry JPEG keyframes.""" + with path.open("rb") as f: + head = f.read(24) + if head[:8] == b"\x89PNG\r\n\x1a\n" and head[12:16] == b"IHDR": + return int.from_bytes(head[16:20], "big"), int.from_bytes(head[20:24], "big") + if head[:2] != b"\xff\xd8": + raise ValueError(f"{path}: not a JPEG or PNG image") + f.seek(2) + while True: + byte = f.read(1) + if not byte: + break + if byte != b"\xff": + continue + marker = f.read(1) + while marker == b"\xff": # fill bytes before a marker + marker = f.read(1) + if not marker: + break + m = marker[0] + if m in (0x01, *range(0xD0, 0xD8)): # markers with no length field + continue + if m in (0xD9, 0xDA): # end of image, or start of scan before any frame header + break + length = int.from_bytes(f.read(2), "big") + if m in _JPEG_SOF: + frame = f.read(5) + if len(frame) == 5: + return int.from_bytes(frame[3:5], "big"), int.from_bytes(frame[1:3], "big") + break + if length < 2: + break + f.seek(length - 2, 1) + raise ValueError(f"{path}: JPEG with no readable frame header") + + +def read_session_image(path: Path, flags: int = cv2.IMREAD_COLOR) -> np.ndarray: + """Decode a session image after checking its header size against MAX_IMAGE_PIXELS.""" + w, h = image_size(path) + if w <= 0 or h <= 0 or w * h > MAX_IMAGE_PIXELS: + raise ValueError(f"{path}: {w} x {h} pixels, more than {MAX_IMAGE_PIXELS} or empty") + img = cv2.imread(str(path), flags) + if img is None: + raise ValueError(f"{path}: OpenCV could not decode it") + return img + + +def session_file(folder: Path, relative: str) -> Path: + """A file the session names, which must lie inside the session folder: an absolute path or + `..` in session.json would otherwise read an unrelated local photo.""" + root = folder.resolve() + path = (root / relative).resolve() + if Path(relative).is_absolute() or not path.is_relative_to(root): + raise ValueError(f"{folder}: session names {relative!r}, outside the session folder") + return path + + +def capture_id(folder: Path, digest: str | None) -> str: + """The zip's sha256 (the scoring harness's capture id), or for a folder a sha256 over + session.json and every keyframe image, so predictions can be tied to the exact capture.""" + if digest is not None: + return digest + h = hashlib.sha256((folder / "session.json").read_bytes()) + for kf in sorted(load_session(folder)["keyframes"], key=lambda k: k["id"]): + with session_file(folder, kf["img"]).open("rb") as f: + while block := f.read(1 << 20): + h.update(block) + return h.hexdigest() + + def work_dir(folder: Path) -> Path: return FIELD_DIR / "work" / safe_id(load_session(folder)["session"]["id"], "session id") -def prepare(folder: Path) -> Path: - """Upright copies of every keyframe plus the model runner's image list and intrinsics.""" +def prepare(folder: Path, digest: str | None = None) -> Path: + """Upright copies of every keyframe plus the model runner's image list and intrinsics, in a + work folder tied to this exact capture: a different capture with the same session id clears + it, so no earlier prediction survives under the new capture's name.""" session = load_session(folder) out = work_dir(folder) + capture = capture_id(folder, digest) + stamp = out / "capture.json" + if out.exists() and (not stamp.exists() or json.loads(stamp.read_text())["capture"] != capture): + shutil.rmtree(out) + out.mkdir(parents=True, exist_ok=True) + stamp.write_text(json.dumps({"capture": capture}) + "\n") (out / "upright").mkdir(parents=True, exist_ok=True) listing, intrinsics, turns = [], {}, {} for kf in session["keyframes"]: - img = cv2.imread(str(folder / kf["img"])) - if img is None: - raise FileNotFoundError(folder / kf["img"]) + img = read_session_image(session_file(folder, kf["img"])) k = upright_turns(keyframe_pose_cv(kf)) path = out / "upright" / f"{safe_id(kf['id'], 'keyframe id')}.jpg" cv2.imwrite(str(path), np.rot90(img, k), [cv2.IMWRITE_JPEG_QUALITY, 95]) @@ -183,28 +394,54 @@ def triangulated_scales(folder: Path, out: Path, session: dict, kids: list[str]) kfs = {kf["id"]: kf for kf in session["keyframes"]} turns = json.loads((out / "turns.json").read_text()) poses = {k: keyframe_pose_cv(kf) for k, kf in kfs.items()} - cache: dict[str, tuple] = {} - def inputs(k): - if k not in cache: - gray = cv2.imread(str(folder / kfs[k]["img"]), cv2.IMREAD_GRAYSCALE) - cache[k] = (gray, keyframe_K_cv(kfs[k]), keyframe_depth(out, k, turns[k])) - return cache[k] + def load(k): + gray = read_session_image(session_file(folder, kfs[k]["img"]), cv2.IMREAD_GRAYSCALE) + return gray, keyframe_K_cv(kfs[k]), keyframe_depth(out, k, turns[k]) + def pixels(k): + w, h = image_size(session_file(folder, kfs[k]["img"])) + return w * h + + frames = GroupFrames(load, pixels) fits = {} for kid in kids: group = neighbours(poses, kid) if len(group) < 2: continue + held = frames.hold(group) fits[kid] = view_scales( - {k: inputs(k)[0] for k in group}, - {k: inputs(k)[1] for k in group}, + {k: held[k][0] for k in group}, + {k: held[k][1] for k in group}, {k: poses[k] for k in group}, - {k: inputs(k)[2] for k in group}, + {k: held[k][2] for k in group}, )[kid] return fits +class GroupFrames: + """The decoded inputs of one neighbour group at a time. Frames outside the requested group are + dropped before new ones load, so memory follows the group, not the session; a group over + MAX_GROUP_PIXELS is refused from its image headers before anything in it is decoded.""" + + def __init__(self, load, pixels): + self.load, self.pixels = load, pixels + self.held: dict[str, tuple] = {} + + def hold(self, group: list[str]) -> dict[str, tuple]: + total = sum(self.pixels(k) for k in group) + if total > MAX_GROUP_PIXELS: + raise ValueError( + f"keyframes {group} hold {total} pixels together, more than {MAX_GROUP_PIXELS}" + ) + for k in [k for k in self.held if k not in group]: + del self.held[k] + for k in group: + if k not in self.held: + self.held[k] = self.load(k) + return self.held + + # --- Points, walls, values (Measure Lab's definitions) ------------------------------------------ @@ -458,17 +695,37 @@ def parse_tape(text: str) -> float: return round(feet + inches / 12, 6) +# The session field of field/map.template.json before `stamp` binds it to a session. +MAP_SESSION_PLACEHOLDER = "filled in by make field" + + def stamp(session_path: Path, truth_path: Path, map_path: Path, out_dir: Path) -> Path: """Makes the team's files match this session: tape readings typed as text become feet and the zip's sha256 joins the survey's captures (both in place), and a copy of the map naming this - session goes to out_dir/inputs/map.json, which is returned.""" + session goes to out_dir/inputs/map.json, which is returned. A map already bound to another + session is refused before anything is written, since its M numbers belong to that walk. So is + a survey that already lists another capture: the scoring harness's survey format has no + session field, so the captures list is its binding, and a survey stamped for one walk would + otherwise score another.""" folder, capture = unpack(session_path) if capture is None: raise ValueError( f"{session_path}: pass the zip Measure Lab shared; its sha256 is the capture id" ) session_id = load_session(folder)["session"]["id"] + mapping = json.loads(map_path.read_text()) + if mapping.get("session") not in (MAP_SESSION_PLACEHOLDER, session_id): + raise ValueError( + f"{map_path} is the map for session {mapping.get('session')!r}, not {session_id!r}; " + f"start from field/map.template.json for a new session" + ) survey = json.loads(truth_path.read_text()) + others = [c for c in survey["captures"] if c != capture] + if others: + raise ValueError( + f"{truth_path} is already the survey for capture {others[0]}, not this session's " + f"{capture}; copy field/survey.template.json for each session" + ) for m in survey["measurements"]: if m["status"] == "measured" and isinstance(m.get("value_ft"), str): if "FILL" in m["value_ft"]: @@ -481,7 +738,6 @@ def stamp(session_path: Path, truth_path: Path, map_path: Path, out_dir: Path) - survey["captures"].append(capture) print(f"survey captures += {capture}", file=sys.stderr) truth_path.write_text(json.dumps(survey, indent=2) + "\n") - mapping = json.loads(map_path.read_text()) mapping["session"] = session_id stamped = out_dir / "inputs" / "map.json" stamped.parent.mkdir(parents=True, exist_ok=True) @@ -511,18 +767,26 @@ def score( folder, capture = unpack(session_path) session = load_session(folder) out = work_dir(folder) + stamp = out / "capture.json" + if not stamp.exists() or json.loads(stamp.read_text())["capture"] != capture_id( + folder, capture + ): + raise ValueError( + f"{out} was prepared for another capture (or not at all): run `prepare` on " + f"{session_path} and the model again before scoring it" + ) turns = json.loads((out / "turns.json").read_text()) started = time.perf_counter() tapped = sorted({t["keyframe"] for t in session["taps"]}) kids = tapped or [kf["id"] for kf in session["keyframes"]] fits = triangulated_scales(folder, out, session, kids) - depth_cache: dict[str, np.ndarray] = {} + # Taps read depth one keyframe at a time; keeping the last two bounds memory to two depth + # maps however many keyframes were tapped. + @functools.lru_cache(maxsize=2) def depth_of(k): - if k not in depth_cache: - depth_cache[k] = keyframe_depth(out, k, turns[k]) - return depth_cache[k] + return keyframe_depth(out, k, turns[k]) tri = [f.scale for f in fits.values() if f.scale is not None] lines = [ @@ -560,6 +824,13 @@ def depth_of(k): truth = json.loads(Path(truth_path).read_text()) mapping = json.loads(Path(map_path).read_text()) + if mapping.get("session") != session["session"]["id"]: + raise ValueError( + f"{map_path}: map is for session {mapping.get('session')!r}, the capture is " + f"{session['session']['id']!r}; run `stamp` first" + ) + if capture not in truth.get("captures", []): + raise ValueError(f"{truth_path}: survey does not list capture {capture}; run `stamp` first") rules_sha = hashlib.sha256(Path(rules_path).read_bytes()).hexdigest() ref = mapping["measurements"].get(truth["scale_reference"]) tape = {m["id"]: m for m in truth["measurements"]}.get(truth["scale_reference"]) @@ -569,9 +840,16 @@ def depth_of(k): and tape and tape["status"] == "measured" ): - s = tape_scale( - session, native, ref["session_measurement"], ref["key"], tape["value_ft"] * FEET - ) + try: + s = tape_scale( + session, native, ref["session_measurement"], ref["key"], tape["value_ft"] * FEET + ) + except ValueError as e: + # The tape row fails on its own; the native and triangulated rows still stand. + s = float("nan") + lines.append( + f"- `moge2-tape`: every value failed, because the scale reference did: {e}" + ) rows["moge2-tape"] = ("scale_reference", learned_values(session, positions(native, s))) else: lines.append( @@ -619,7 +897,7 @@ def main() -> None: print(stamp(args.session, args.truth, args.map, out_dir)) return if args.step == "prepare": - print(prepare(unpack(args.session)[0])) + print(prepare(*unpack(args.session))) return report = score(args.session, args.truth, args.map, args.rules, out_dir, args.tap_error_in) out_dir.mkdir(parents=True, exist_ok=True) diff --git a/experiments/evals/evals/frames.py b/experiments/evals/evals/frames.py index 3e2b7e40b..19edd4e23 100644 --- a/experiments/evals/evals/frames.py +++ b/experiments/evals/evals/frames.py @@ -12,13 +12,12 @@ from __future__ import annotations import argparse -import json from pathlib import Path import numpy as np from evals.eth3d import SCENES -from evals.pairs import evaluate_fixed, pool +from evals.pairs import evaluate_fixed, pool, results_json from evals.recon import ( PREDICTIONS, WIDTH, @@ -142,9 +141,7 @@ def main() -> None: for r in rows ] args.out.mkdir(parents=True, exist_ok=True) - (args.out / "frames.json").write_text( - json.dumps({"policies": summary, "photos": per}, indent=1) - ) + (args.out / "frames.json").write_text(results_json({"policies": summary, "photos": per})) md = "\n".join(lines) + "\n" (args.out / "frames.md").write_text(md) print(md) diff --git a/experiments/evals/evals/map3d.py b/experiments/evals/evals/map3d.py index c008ae7f1..e8ea8dc18 100644 --- a/experiments/evals/evals/map3d.py +++ b/experiments/evals/evals/map3d.py @@ -1,4 +1,4 @@ -"""The app's 3D map (Map3D, t3/ios-map3d) against the same truth as CoverageMap (README section 7c). +"""The app's 3D map (Map3D, t3/ios-map3d) against the same truth as CoverageMap (METHODS.md section 7c). The app's code runs unmodified through `map3d_driver/`, built against a read-only checkout of HouseScanKit at `KIT_COMMIT`. Its LiDAR path is fed one depth frame per ETH3D photo, 256 x 192, @@ -36,10 +36,11 @@ two_positions, ) from evals.eth3d import excluded_pixels, occluder_points, scan_points +from evals.pairs import results_json from evals.paths import EVALS_DIR -KIT_COMMIT = "66cdcba" -KIT_CHECKOUT = EVALS_DIR / f"housescankit-{KIT_COMMIT}" +KIT_COMMIT = "66cdcbaaca5de3ca4812e7073fd5490b477713a2" # t3/ios-map3d, draft PR #21 +KIT_CHECKOUT = EVALS_DIR / f"housescankit-{KIT_COMMIT[:7]}" KIT_DIR = KIT_CHECKOUT / "ios" / "HouseScanKit" DRIVER_DIR = Path(__file__).resolve().parents[1] / "map3d_driver" DRIVER_BIN = DRIVER_DIR / ".build" / "release" / "map3d-driver" @@ -59,7 +60,14 @@ def prepare() -> None: """Checks out HouseScanKit at the pinned commit, read-only, and builds the driver.""" repo = Path(__file__).resolve().parents[3] if not KIT_CHECKOUT.exists(): - subprocess.run(["git", "-C", str(repo), "fetch", "origin", "t3/ios-map3d"], check=True) + # Fetch the commit itself: the branch can move or be rebased away from it. + have = subprocess.run( + ["git", "-C", str(repo), "cat-file", "-e", f"{KIT_COMMIT}^{{commit}}"], check=False + ) + if have.returncode: + subprocess.run( + ["git", "-C", str(repo), "fetch", "--no-tags", "origin", KIT_COMMIT], check=True + ) subprocess.run( ["git", "-C", str(repo), "worktree", "add", "--detach", str(KIT_CHECKOUT), KIT_COMMIT], check=True, @@ -151,6 +159,19 @@ def seen_empty(setup: Setup, points: np.ndarray, tol_abs: float = HIDE_ABS_M) -> return seen +def span_columns(lo: float, hi: float) -> tuple[np.ndarray, np.ndarray]: + """2 cm columns that tile [lo, hi] exactly: their centres and widths in meters. The last column + is the partial strip left over, so the widths sum to the span's length.""" + edges = np.append(np.arange(lo, hi - 1e-9, 0.02), hi) + return (edges[:-1] + edges[1:]) / 2, np.diff(edges) + + +def within_bar(false_observed_ft: float) -> bool: + """The pre-registered 0.5 ft bar, inclusive. Rounding to a micro-foot keeps a claim of exactly + the bar from failing on floating-point sums of column widths.""" + return round(false_observed_ft, 6) <= PASS_FT + + def space_claims(setup: Setup, spans: list[dict], kind: str, tol_abs: float = HIDE_ABS_M) -> dict: """False-observed length of Map3D's facing or overhead claims: claimed 2 cm columns where at least two 5 cm samples of the claimed space were never seen empty.""" @@ -160,7 +181,7 @@ def space_claims(setup: Setup, spans: list[dict], kind: str, tol_abs: float = HI for sp in spans: lo, hi = sp["s"] reach = sp["out"] - cols = np.arange(lo + 0.01, hi, 0.02) + cols, w = span_columns(lo, hi) if kind == "facing": outs = np.arange(VOXEL_M, reach + 1e-6, 0.05) heights = np.arange(GROUND_CLEARANCE_M, HEADROOM_M + 1e-6, 0.05) @@ -173,16 +194,14 @@ def space_claims(setup: Setup, spans: list[dict], kind: str, tol_abs: float = HI pts = wall.world(grid_s, grid_h, grid_o).reshape(-1, 3) unseen = (~seen_empty(setup, pts, tol_abs)).reshape(len(cols), -1).sum(axis=1) bad = unseen >= 2 - claimed_ft += len(cols) * 0.02 / FEET - bad_ft += bad.sum() * 0.02 / FEET + claimed_ft += w.sum() / FEET + bad_ft += w[bad].sum() / FEET if bad.any(): - runs.append( - [round(lo / FEET, 2), round(hi / FEET, 2), round(bad.sum() * 0.02 / FEET, 2)] - ) + runs.append([round(lo / FEET, 2), round(hi / FEET, 2), round(w[bad].sum() / FEET, 2)]) return { "claimed_ft": round(claimed_ft, 2), "false_observed_ft": round(bad_ft, 2), - "passes": bad_ft <= PASS_FT, + "passes": within_bar(bad_ft), "spans_with_unseen_ft": runs, } @@ -195,7 +214,7 @@ def ground_claims(setup: Setup, spans: list[dict]) -> dict: claimed_ft, bad_ft, runs = 0.0, 0.0, [] for sp in spans: lo, hi = sp["s"] - cols = np.arange(lo + 0.01, hi, 0.02) + cols, w = span_columns(lo, hi) outs = np.arange(VOXEL_M, sp["out"] + 1e-6, 0.05) if not len(cols) or not len(outs): continue @@ -210,21 +229,21 @@ def ground_claims(setup: Setup, spans: list[dict]) -> dict: ) seen |= t.saw & ~t.no_scan bad = (~seen).reshape(len(cols), -1).sum(axis=1) >= 2 - claimed_ft += len(cols) * 0.02 / FEET - bad_ft += bad.sum() * 0.02 / FEET + claimed_ft += w.sum() / FEET + bad_ft += w[bad].sum() / FEET if bad.any(): runs.append( [ round(lo / FEET, 2), round(hi / FEET, 2), round(sp["out"] / FEET, 2), - round(bad.sum() * 0.02 / FEET, 2), + round(w[bad].sum() / FEET, 2), ] ) return { "claimed_ft": round(claimed_ft, 2), "false_observed_ft": round(bad_ft, 2), - "passes": bad_ft <= PASS_FT, + "passes": within_bar(bad_ft), "spans_with_unseen_ft_s_s_reach_bad": runs, } @@ -332,7 +351,7 @@ def evaluate() -> dict: "false_observed_share", "missed_ft", "seen_two_view_ft", - "claimed_samples_seen_only_where_scan_is_empty", + "claimed_samples_unknown_for_lack_of_scan", ) stretch = (setup.wall.left, setup.wall.right) s_cols = np.arange(stretch[0] + 0.01, stretch[1], 0.02) @@ -371,7 +390,7 @@ def markdown(r: dict) -> str: "256 x 192 depth frame per ETH3D electro photo, rendered from the laser scan. Coverage read " f"along section 7's {r['wall_stretch_ft']} ft wall. Lengths in feet along the wall; " f"false-observed passes at {PASS_FT} ft or less. Ideal depth: exact, dense, no sensor " - "noise, so this is an upper bound on what real LiDAR gives. Definitions: README section 7c.", + "noise, so this is an upper bound on what real LiDAR gives. Definitions: METHODS.md section 7c.", "", "| Band | Claimed | False-observed (share) | Pass | Missed | Seen from 2 positions |", "| --- | --- | --- | --- | --- | --- |", @@ -453,18 +472,15 @@ def check_kit() -> None: head = subprocess.run( [*git, "rev-parse", "HEAD"], capture_output=True, text=True, check=True ).stdout.strip() - pinned = subprocess.run( - [*git, "rev-parse", f"{KIT_COMMIT}^{{commit}}"], capture_output=True, text=True, check=True - ).stdout.strip() dirty = subprocess.run( [*git, "status", "--porcelain", "--", "ios/HouseScanKit"], capture_output=True, text=True, check=True, ).stdout.strip() - if head != pinned or dirty: + if head != KIT_COMMIT or dirty: raise SystemExit( - f"{KIT_CHECKOUT} is at {head} with changes {dirty!r}; expected {pinned}, clean" + f"{KIT_CHECKOUT} is at {head} with changes {dirty!r}; expected {KIT_COMMIT}, clean" ) @@ -472,7 +488,7 @@ def main() -> None: check_kit() r = evaluate() RESULTS.mkdir(exist_ok=True) - (RESULTS / "map3d.json").write_text(json.dumps(r, indent=1, default=float)) + (RESULTS / "map3d.json").write_text(results_json(r, default=float)) md = markdown(r) (RESULTS / "map3d.md").write_text(md) print(md) diff --git a/experiments/evals/evals/marvin_manifest.json b/experiments/evals/evals/marvin_manifest.json new file mode 100644 index 000000000..a8d1a9f45 --- /dev/null +++ b/experiments/evals/evals/marvin_manifest.json @@ -0,0 +1,91 @@ +{ + "atrium/seq1/ARkitPose.txt": "94da66e774d2e9f3fb00c00de594405ad2b315c905dfdeff353949cd445b925e", + "atrium/seq1/GPSData.txt": "76d261047acae6775d8a9a5ca8c1a146459798a4dd8410d71f585b2047ac0f5d", + "atrium/seq10/ARkitPose.txt": "41baf11f1dd0ba826906f3baf9586aca9370f811e5ee2176b17919aa85723d2b", + "atrium/seq10/GPSData.txt": "734de27d016d37f15e05715fd2eb7ec36a8c772243539120d1b1a1932a993ce1", + "atrium/seq11/GPSData.txt": "b1c63742812e6ae71e6535a3083c67622a2016f548f71a0ce1779d78a9f0361d", + "atrium/seq12/GPSData.txt": "184727caaf7b8c1af1b9f683498c6f9a4bb2fa561270fa63bb6fbf3c047ded5c", + "atrium/seq13/GPSData.txt": "8f8d6b794c111c19b491f0046e97ed89a194695107c5529ccec6872429176599", + "atrium/seq14/GPSData.txt": "29c86afd50b8e7abbd2902bce3f947a5b28dcb15d8c1631afbd16596b5860339", + "atrium/seq2/ARkitPose.txt": "7469a46401fd7960b80e7bda0f753a15ab8674ad212812dd493dfb3f62ed682c", + "atrium/seq2/GPSData.txt": "25149d03aecb8897d63a6d4f86eb539983b1648b531247fecddf094256f3f549", + "atrium/seq3/ARkitPose.txt": "27d89f410162bcf8428b6ba9a186931a0894d80ad9831bcf032b7f9dba8da084", + "atrium/seq3/GPSData.txt": "42c746578bc1a63c890fb138c87ee1ee4db8c5cce41c066aaa26b42e0a9ab77c", + "atrium/seq4/ARkitPose.txt": "911a388e02f4d4515f8bb389bd1d7bdab4ca222632571a5996b9e86d76eeaa23", + "atrium/seq4/GPSData.txt": "59c026b0944ac370cb5ab0334a823597d07d2ae7817eb82d096946228d7d090f", + "atrium/seq5/ARkitPose.txt": "6b294f77be6e5cccf6b3e57e5b3b2bb9f0632d6c13d45864ede4767c69791c33", + "atrium/seq5/GPSData.txt": "558c2ff5338a860e4c9cf1abc0085b31868387c59a13b78db33d4d39d38caaa7", + "atrium/seq6/ARkitPose.txt": "c55f9ec4f0654018154058c95aaa3f3c209f4d73cde351412139c4d23ed23540", + "atrium/seq6/GPSData.txt": "6fb8e236172a7f4d890ee05dd4ce43719171fc75e95d2a2633c54ad576124360", + "atrium/seq7/ARkitPose.txt": "b53ce3320bc15952a9ea7fb1c7997e8af5932e02662314e3fe4f1eec33a1b545", + "atrium/seq7/GPSData.txt": "765226f20c87c0ae5cc10b1d913e1083f48d2ee3b3e8330f5b9958bcd28000a1", + "atrium/seq8/ARkitPose.txt": "1a6d28c06a6526f1c63bb394e694375eab654c827a5abe69227a16cc0df86761", + "atrium/seq8/GPSData.txt": "cc01043cfa6dd91049336fd2f7f7113a494f3e6c9500675ef017540f1e753b2e", + "atrium/seq9/ARkitPose.txt": "ce39b215a2af7975da9ea8cfbde4d8b23a99151b44432b42efc2fa5918641efe", + "atrium/seq9/GPSData.txt": "d58ed7f01167bd7c58b610c0bacfcf3a0f61874d6dea877249ef5f0072f8d95b", + "atrium/test.txt": "68c164ce7b54cb643459de211565d0c7c79a35a3095ba04a9211d5ce1bc82a17", + "atrium/train.txt": "bfa42bcf326234e6081020b8754607c23a7ea423762ad645e7bfb558e914059b", + "bar/seq1/ARkitPose.txt": "4b9415baf9d5bb87afba356c84a3c6f5b9efec7c3cc26a52326bfc1795b206c2", + "bar/seq1/GPSData.txt": "1937cf62c7078cf2230d4061ef99c922387e74b89df39209a0e290cab3d782c7", + "bar/seq10/ARkitPose.txt": "570e043a3634e27ee1bb27d43eb49578f465718ef85a7ac866121d0ccb9b0ea9", + "bar/seq10/GPSData.txt": "56fb8535f1e18915e86e983175885828b2cd38b33e45d0d095b86319a57864eb", + "bar/seq11/ARkitPose.txt": "3fec8a1c23a65a0fadeb82bd968aead5f9093a7f64f9216ef2bb9f8338d3e1e7", + "bar/seq11/GPSData.txt": "8c65dd00fb34fa63a0fb3b8c57c1433b4d3d1232810337bf17d86be0ab71f999", + "bar/seq12/ARkitPose.txt": "9c8187fc1f22331953d1ceb34a0b8b4cf53854631301c31ad391b17b768379fc", + "bar/seq12/GPSData.txt": "3501a0ef8a23e9a960acc497dbe329f1ed6c0357aca52816ecc12aa3d03920f8", + "bar/seq13/ARkitPose.txt": "dfa84dc040000f4441e7db0293923b8c39109b32d0cdb9b2af4344cbd187f41c", + "bar/seq13/GPSData.txt": "cef05123f41f021461165957dc197c743e556d43ef365af7a69946d5797be084", + "bar/seq14/GPSData.txt": "716725e25fc37542bc95ba100c78c534432ffafb3000921f81e1dcd1a245fa0e", + "bar/seq15/GPSData.txt": "8858f21f1787802ae215825c5c206e036850cce027aafcd99bd38ad619f6c056", + "bar/seq16/GPSData.txt": "a420e9bd516fef1fd837cc37d3cee81547312e81225b751ab7bf3fb5af21eddd", + "bar/seq17/GPSData.txt": "153c3f5e378bd4dcdb5511ab9a6d4f540197a7adb58c14b45962e74f43226d63", + "bar/seq2/ARkitPose.txt": "6618f45531685dfa7bfd4dd264c2b402cdede9bf2ce15c28f78e6b8dc2b3bf7b", + "bar/seq2/GPSData.txt": "b5f819c7c5332a7c57393c2015a695b1f41a7890484a7ae04b8b91596fe49007", + "bar/seq3/ARkitPose.txt": "2689949d5a07e89ae767056960165ed413cbf7cf782199d96e13d63f2db7abf9", + "bar/seq3/GPSData.txt": "50a69113c0ee71734d0c1124981d985f07f2b208b746c961ef0bc462c62a3edf", + "bar/seq4/ARkitPose.txt": "73e39c3ff8a7cd86572d98d31dc4881a9cc61a224e3b9504d5a5f6b93652d2d8", + "bar/seq4/GPSData.txt": "cde25642c60015970c7dc3f8970d9b895267f816f2000b1d8129858f3f0bbcb3", + "bar/seq5/ARkitPose.txt": "39883912564585f767d6da8ab45386eebd885708d62635a2d3fcfb7fb77c547b", + "bar/seq5/GPSData.txt": "e326cd4de1435599e74e9f2623686c1a7b05422a37360da4bc7106dfbc8a73d2", + "bar/seq6/ARkitPose.txt": "3045d393a58b7def77f704cc4c06cc45bf3c0b85d2b12871c69cfa3eb1f75a43", + "bar/seq6/GPSData.txt": "8f3c9f0de0184c013978493d50fe1112f279bd9a068b9941352a09191f99e7be", + "bar/seq7/ARkitPose.txt": "e5b2008f9302555809f9e953a73799bba5960ac178aa42b4ad14899c850ae29d", + "bar/seq7/GPSData.txt": "fd81c401d52d442a84a4818a6703860b60f36be37c12740384f620414735063e", + "bar/seq8/ARkitPose.txt": "6f14ebaae0d2aef7b37d91245571587667f65d2907805399288bd32b8f929f07", + "bar/seq8/GPSData.txt": "bd951b540308c9e36ffc2a078f326499c224be6f962b2ec72ae3b2eb7fccef88", + "bar/seq9/ARkitPose.txt": "c27c312f398aa31b37e5be99093d64fea0282b864e526ab4db25ea57dd388f34", + "bar/seq9/GPSData.txt": "bc3ab47836d4ec52449a877f363853158dbf0720bfd4e8efc0876f27851842f9", + "bar/test.txt": "1008f38c7de007f48da9ec91e90e22083073978353050b37ce88d183b98979a2", + "bar/train.txt": "16c7124a4b4b980108c91eeb61a8174c6dcdc7b040177eb88ad19cbb96ad89a5", + "church/seq1/ARkitPose.txt": "0daaeabc4456d005b9f799ce349b4acd1514c6f8713d91e9e20515c6431a161c", + "church/seq1/GPSData.txt": "a0a172b7a5420cbea3e9f7913d344b3595f759d258a0cf7924a5b53dd60521ed", + "church/seq10/ARkitPose.txt": "5e3c0bb89bf231020ff067265a40a86291ae5986e5c79dfc6b3ab26805919526", + "church/seq10/GPSData.txt": "25857d4b94788499db005343bbae06edfebe7745ed00261af928ac1dcf5d6af5", + "church/seq11/ARkitPose.txt": "37f2450e410e65de942a5c6a89f8b0dc19a655a84f2dc944a61db6a21066c820", + "church/seq11/GPSData.txt": "220f94d2a79d93a8a3a3f9f7e294d9189baa2440e18580af2476d9ace9009aae", + "church/seq12/ARkitPose.txt": "1036da4acda117b876a789e9cafdae599344abe227a178c2a958d15bc261e0f2", + "church/seq12/GPSData.txt": "b79a9c2689d65a3e98eb3cf5b27d5317cbb806515cddd2b7660303a420961ae9", + "church/seq13/GPSData.txt": "1d8b4818aca106f3886996914fac519fa470d279c8f8b08f98ff602a67af7330", + "church/seq14/GPSData.txt": "fa6e9b0ad02e2b9a79a2a1b1e485708b28d7a929eca6a3cad17628c7a5fe4e00", + "church/seq15/GPSData.txt": "c7ac35d0715ee0155371d2bec9b9a60eb44409552b9ad2a62b7cf3bf3f78b944", + "church/seq16/GPSData.txt": "4290ce22af745fb8755eb72e8a54338f14a1f82561b03fe782a4aca2acce1266", + "church/seq17/GPSData.txt": "13ec050e4b7ac8d04367d6173430bf47011a266bdd1430877b950de3c460b64c", + "church/seq2/ARkitPose.txt": "6c7f3ae229ab7856909027e89611d1a8dad7eef9dfa6e430370b2a83baf35295", + "church/seq2/GPSData.txt": "16972e7ecf964898b0103b21102c37d3a8973f4a7ef02951dc9ec31535cfba7e", + "church/seq3/ARkitPose.txt": "5dd90bbabb79b15724c3029d81aad7ed0c7cd4d00f84d82b47cac7cf2e647c1e", + "church/seq3/GPSData.txt": "0407b87909c7e750dfb81eba801385cb0d4d1cd30162e469d56d9e5b85c3d835", + "church/seq4/ARkitPose.txt": "615337c38105aa674db0edf9c666408299f7444a19596b40c4760b7bb0ff7881", + "church/seq4/GPSData.txt": "dc79dc391d336ab6a0ef04da95adba2147449ebe81a5c994a42ee521fd63eeb2", + "church/seq5/ARkitPose.txt": "a1af901cbecb76a3eed32e24b56aa01cd16694fb4267403dbfde682fc827bfcf", + "church/seq5/GPSData.txt": "23ecb699d4e5346c61f67aec2afb98f6a9d11506b13846b5ce855fe9ca2a3524", + "church/seq6/ARkitPose.txt": "7055a979c8b4674e187f17461d744e42baa78dc795effbedf93f6e870bc19bf3", + "church/seq6/GPSData.txt": "ed8cfb2b86e5594dd6d255bec45416cc1bbfc36c8261150726602a2370413b86", + "church/seq7/ARkitPose.txt": "761c9183821dcc218031211d843cb21b0ae77c2ffb3f3cd4042f4f88434693c6", + "church/seq7/GPSData.txt": "49254c1ca0ce0a95420e4ecdedbc11b40734c9fb1a389a382a426507405361ab", + "church/seq8/ARkitPose.txt": "f7d0194a16b229351e9cbf6cdf090d56c0703216b962afd4b93c16c615bb5c58", + "church/seq8/GPSData.txt": "bdb6f15e445401b5694a6701fb66a0cd8dfea6c3747fe7a73fee91ca00fb3682", + "church/seq9/ARkitPose.txt": "bacdf1795e6de3e19d51ca0a51b712ff7355c0b6ea45fa8d4ada513867743bb1", + "church/seq9/GPSData.txt": "a29cf3011f8afe8dd69195076d866dc46918b6d92c4e5226d5674d8b61b9aa38", + "church/test.txt": "e4aae74db2397457d99ee8ba4a8b0d8553bca6f0abf5ce258d795216620afcdf", + "church/train.txt": "9119e3d102dc8dd1cd0741ee7e9eb36e522f46be94f84a711b51695ace5061db" +} diff --git a/experiments/evals/evals/modern_arkit.py b/experiments/evals/evals/modern_arkit.py index 7082563f2..84743a339 100644 --- a/experiments/evals/evals/modern_arkit.py +++ b/experiments/evals/evals/modern_arkit.py @@ -15,7 +15,7 @@ The limit: COLMAP from one moving camera has no scale of its own, and the dataset does not say how its reconstructions were put into meters. The phone's GPS, the one independent reference shipped, -cannot pin that scale better than several percent over these 25 to 160 m walks (checked here). So +cannot pin that scale better than several percent over these 45 to 255 m walks (checked here). So ARKit's scale is measured against the ground truth's scale, not against the tape. """ @@ -39,6 +39,7 @@ window_pairs, ) from evals.geometry import quat_wxyz_to_matrix +from evals.pairs import results_json from evals.paths import EVALS_DIR MARVIN_DIR = EVALS_DIR / "marvin" @@ -52,24 +53,64 @@ RESULTS = Path(__file__).resolve().parents[1] / "results" -def fetch() -> dict[str, str]: - """Download only the pose, GPS and ground-truth text files; return {path: sha256}.""" - files = gdown.download_folder(FOLDER, skip_download=True, quiet=True, remaining_ok=True) - wanted = [ - f - for f in files - if f.path.split("/")[0] in SCENES - and f.path.endswith(("ARkitPose.txt", "GPSData.txt", "train.txt", "test.txt")) +# The pose, GPS and ground-truth files the published results used, with their sha256. The Drive +# folder can change under us, so only these files are fetched and scored, and each must match. +MANIFEST = Path(__file__).with_name("marvin_manifest.json") + + +def manifest() -> dict[str, str]: + return json.loads(MANIFEST.read_text()) + + +def file_sha256(path: Path) -> str: + return hashlib.sha256(path.read_bytes()).hexdigest() + + +def verify(pinned: dict[str, str], root: Path) -> None: + """Refuse to score unless every pinned file is present with its pinned content.""" + wrong = [ + rel + for rel, sha in pinned.items() + if not (root / rel).exists() or file_sha256(root / rel) != sha ] - hashes = {} - for f in wanted: - out = MARVIN_DIR / f.path - if not out.exists(): + if wrong: + raise RuntimeError( + f"{len(wrong)} MARViN files are missing or differ from {MANIFEST.name}, " + f"e.g. {wrong[:3]}; delete them from {root} and fetch again" + ) + + +def fetch() -> dict[str, str]: + """Download the pinned files that are not on disk yet, then check every one against its hash.""" + pinned = manifest() + missing = [rel for rel in pinned if not (MARVIN_DIR / rel).exists()] + if missing: + listed = { + f.path: f.id + for f in gdown.download_folder( + FOLDER, skip_download=True, quiet=True, remaining_ok=True + ) + } + for rel in missing: + if rel not in listed: + raise RuntimeError(f"{rel} is no longer in the MARViN Drive folder") + out = MARVIN_DIR / rel out.parent.mkdir(parents=True, exist_ok=True) - if gdown.download(id=f.id, output=str(out), quiet=True) is None: - raise RuntimeError(f"Google Drive refused {f.path} ({f.id}); retry later") - hashes[f.path] = hashlib.sha256(out.read_bytes()).hexdigest() - return hashes + if gdown.download(id=listed[rel], output=str(out), quiet=True) is None: + raise RuntimeError(f"Google Drive refused {rel} ({listed[rel]}); retry later") + verify(pinned, MARVIN_DIR) + return pinned + + +def pinned_walks(pinned: dict[str, str], scene: str) -> list[str]: + """The scene's walks named in the manifest (`seqN`), in walk order. Other walks on disk are + never read.""" + walks = [ + rel.split("/")[1] + for rel in pinned + if rel.startswith(f"{scene}/") and rel.endswith("/ARkitPose.txt") + ] + return sorted(walks, key=lambda name: int(name[3:])) def read_table(path: Path, columns: int) -> dict[str, np.ndarray]: @@ -156,15 +197,16 @@ def gps_scale(gps: dict, truth_by_image: dict, frame: np.ndarray) -> tuple[float return s, float(np.median(res)) -def evaluate() -> dict: +def evaluate(pinned: dict[str, str]) -> dict: + verify(pinned, MARVIN_DIR) scenes = {} for scene in SCENES: root = MARVIN_DIR / scene truth = {**read_table(root / "train.txt", 8), **read_table(root / "test.txt", 8)} frame = level_frame(np.array([v[:3] for v in truth.values()])) walks = [] - for seq in sorted(root.glob("seq*/ARkitPose.txt"), key=lambda p: int(p.parent.name[3:])): - name = seq.parent.name + for name in pinned_walks(pinned, scene): + seq = root / name / "ARkitPose.txt" ark_rows = read_table(seq, 8) keys = sorted(k for k in ark_rows if f"{name}/{k}" in truth) if len(keys) < 30: @@ -189,7 +231,7 @@ def evaluate() -> dict: scale = float(np.median(long)) if len(long) else float("nan") gps_path = root / name / "GPSData.txt" gps = None - if gps_path.exists(): + if f"{scene}/{name}/GPSData.txt" in pinned: by_image = {k: truth[f"{name}/{k}"][:3] for k in keys} gps = gps_scale(read_table(gps_path, 4), by_image, frame) walks.append( @@ -210,7 +252,7 @@ def evaluate() -> dict: def site_spread( scales: np.ndarray, draws: int = 10_000, seed: int = 0 ) -> tuple[float, float, float]: - """Walk-to-walk standard deviation of ARKit's scale within one site, and its 95% bootstrap + """Walk-to-walk standard deviation of the ARKit / reference scale ratio within one site, and its 95% bootstrap interval over walks. Walks off by more than `OUTLIER` are left out: their reference is in doubt. """ s = scales[np.abs(scales - 1) <= OUTLIER] @@ -253,13 +295,14 @@ def markdown(scenes: dict, hashes: dict[str, str]) -> str: f"Walks within 2% of the ground truth's scale: {int(np.sum(np.abs(s_all - 1) <= 0.02))} of " f"{len(all_walks)}.", "", - "## Walk-to-walk spread of ARKit's scale within a site", + "## Walk-to-walk spread of the ARKit / reference scale ratio within a site", "", f"Standard deviation over each site's walks within {OUTLIER:.0%} of the reference, with a " - "95% bootstrap interval over walks. A reference scaled to ARKit would carry one scale per " - "site, which shifts every walk of that site alike, so this spread survives it: a single " - "walk's scale error is at least this large, unless the reference's own scale varies walk " - "to walk in step with ARKit's.", + "95% bootstrap interval over walks. It measures how much ARKit and the reference disagree " + "from walk to walk, not ARKit's error alone: a perfect ARKit paired with a reference whose " + "local scale varies would give the same spread. A single site scale for the reference " + "cannot remove its local reconstruction error, and no independent bound on that error " + "exists here.", "", "| Site | Walks used | Spread (1 SD) | 95% interval |", "| --- | --- | --- | --- |", @@ -333,7 +376,7 @@ def markdown(scenes: dict, hashes: dict[str, str]) -> str: f"GPS / truth over each walk: median {np.median(g):.3f}, range {g.min():.3f} to " f"{g.max():.3f} ({len(g)} walks). On the {len(tight)} walks where GPS fits within 1.5 m, " f"median {np.median(tight):.3f}, range {tight.min():.3f} to {tight.max():.3f}. GPS noise of " - "1 to 9 m over 25 to 160 m walks cannot pin the scale to 2%.", + "1 to 9 m over 45 to 255 m walks cannot pin the scale to 2%.", "", "## Files (Google Drive folder " + FOLDER + ")", "", @@ -346,13 +389,13 @@ def markdown(scenes: dict, hashes: dict[str, str]) -> str: def main() -> None: hashes = fetch() - scenes = evaluate() + scenes = evaluate(hashes) summary = { scene: [{k: v for k, v in w.items() if k != "errors"} for w in walks] for scene, walks in scenes.items() } RESULTS.mkdir(exist_ok=True) - (RESULTS / "modern_arkit.json").write_text(json.dumps(summary, indent=1)) + (RESULTS / "modern_arkit.json").write_text(results_json(summary)) md = markdown(scenes, hashes) (RESULTS / "modern_arkit.md").write_text(md) print(md) diff --git a/experiments/evals/evals/pairs.py b/experiments/evals/evals/pairs.py index 5f3dd5fc9..db8f16468 100644 --- a/experiments/evals/evals/pairs.py +++ b/experiments/evals/evals/pairs.py @@ -15,6 +15,8 @@ from __future__ import annotations +import json +import math from dataclasses import dataclass import numpy as np @@ -177,3 +179,27 @@ def pool(raws: list[dict]) -> dict[str, dict]: round(100 * (1 - sum(r["ref_failures"] for r in raws) / refs), 1) if refs else None ) return out + + +def strict_json_value(value): + """`value` with every non-finite float spelled out, so the results files are strict JSON: a + failed percentile (+inf, see `summarize`) becomes "failed", and a statistic with nothing to + compute (NaN) becomes null. A -inf has no meaning in these results and is refused.""" + if isinstance(value, dict): + return {k: strict_json_value(v) for k, v in value.items()} + if isinstance(value, list | tuple): + return [strict_json_value(v) for v in value] + if isinstance(value, float) and not math.isfinite(value): + if math.isnan(value): + return None + if value > 0: + return "failed" + raise ValueError("-inf in a results file") + return value + + +def results_json(value, indent: int = 1, default=None) -> str: + """Serialise a results document as strict JSON (see `strict_json_value`).""" + if default is not None: + value = json.loads(json.dumps(value, default=default)) + return json.dumps(strict_json_value(value), indent=indent, allow_nan=False) diff --git a/experiments/evals/evals/pose_priors.py b/experiments/evals/evals/pose_priors.py index 6d58314bf..7622cbef0 100644 --- a/experiments/evals/evals/pose_priors.py +++ b/experiments/evals/evals/pose_priors.py @@ -30,6 +30,7 @@ from __future__ import annotations import argparse +import functools import hashlib import json from pathlib import Path @@ -39,7 +40,7 @@ from evals.ar_poses import ADVIO_ARKIT_SCALES, SETTINGS, group_poses, noise_draws from evals.eth3d import SCENES, read_views -from evals.pairs import INCH, evaluate_fixed, pool +from evals.pairs import INCH, evaluate_fixed, pool, results_json from evals.paths import ETH3D_DIR from evals.recon import ( COHORTS, @@ -51,9 +52,11 @@ Method, Scene, cell, + comparable_groups, load_prediction, model_frame_method, predict, + require_complete, ) from evals.triangulate import view_scales @@ -63,9 +66,14 @@ # Percentiles interpolate and inf - inf is NaN, so failures become a finite sentinel (as in # `evals.pairs.summarize`) and any result that touches one is reported as a failure. SENTINEL = 1e12 -# The MapAnything advio_2018 runs were given the poses of the earlier advio_2018 setting, whose -# scales were ARKit against ARCore (0.883, 0.936, 0.958); the weights are gone, so it cannot rerun. -MAPANYTHING_LABELS = {"advio_2018": "advio_2018 poses, earlier ARCore-referenced scales"} +# The published MapAnything advio_2018 runs were given the poses of the earlier advio_2018 setting, +# whose scales were ARKit against ARCore (0.883, 0.936, 0.958). Those runs predate the pose hash in +# run.json, so a run without one keeps this label; a run that records its poses is labelled by them. +HISTORICAL_MAPANYTHING = {"advio_2018": "advio_2018 poses, earlier ARCore-referenced scales"} +HISTORICAL_NOTE = ( + " Its advio_2018 row was run on the earlier advio_2018 poses, whose scales were ARKit against " + "ARCore (0.883, 0.936, 0.958), with the same noise as draw 0 here." +) def pose_file(scene: str, setting: str, draw: int = 0) -> Path: @@ -110,31 +118,78 @@ def save(self) -> None: self.changed = False -def fit_inputs(T: dict, K: dict, max_reproj_px: float) -> str: - """What a group's scale fit depends on besides the images and depth maps on disk.""" +@functools.cache +def _file_digest(path: Path, mtime_ns: int, size: int) -> str: + return hashlib.sha256(path.read_bytes()).hexdigest() + + +def file_digest(path: Path) -> str: + """sha256 of a file's bytes, computed once per (path, modification time, size).""" + st = path.stat() + return _file_digest(path, st.st_mtime_ns, st.st_size) + + +def fit_inputs( + T: dict, K: dict, max_reproj_px: float, images: dict[str, Path], depths: dict[str, Path] +) -> str: + """Everything a group's scale fit reads: poses, intrinsics, the threshold, and the bytes of + each member's image and depth prediction.""" spec = { "model": MODEL, "poses": {m: np.asarray(t).tolist() for m, t in T.items()}, "K": {m: np.asarray(k).tolist() for m, k in K.items()}, "max_reproj_px": max_reproj_px, + "images": {m: file_digest(p) for m, p in images.items()}, + "depths": {m: file_digest(p) for m, p in depths.items()}, } return hashlib.sha256(json.dumps(spec, sort_keys=True).encode()).hexdigest()[:16] +def mapanything_label(run_root: Path, setting: str, poses_file: Path) -> str: + """A MapAnything row's label, from the poses its groups' run.json files record. Runs that + record no pose hash are the published historical ones; runs that record one must all match + the current pose file, or the row would mix poses.""" + recorded = {json.loads(p.read_text()).get("poses_sha256") for p in run_root.glob("*/run.json")} + if recorded <= {None}: + return HISTORICAL_MAPANYTHING.get(setting, f"{setting} poses") + if recorded != {file_digest(poses_file)}: + raise SystemExit( + f"{run_root}: its groups were run on poses other than {poses_file} (or a mix of " + f"recorded and unrecorded runs); rerun `make pose-priors` for {setting}" + ) + return f"{setting} poses" + + +def uses_historical_rows(results: dict) -> bool: + labels = {f"mapanything@392 + {label}" for label in HISTORICAL_MAPANYTHING.values()} + return any( + method in labels + for ranges in results.values() + for per_method in ranges.values() + for method in per_method + ) + + def pose_methods(scene: Scene, fits: FitCache) -> dict[str, list[Method]]: """{row label: one Method per noise draw}.""" root = PREDICTIONS / scene.name gray: dict[str, np.ndarray] = {} depths: dict[str, np.ndarray] = {} + def image_path(name): + return scene.dir / f"images_{WIDTH}" / f"{name}.jpg" + + def depth_path(name): + return root / MODEL / f"{name}.npz" + def image(name): if name not in gray: - gray[name] = cv2.imread(str(scene.dir / f"images_{WIDTH}" / f"{name}.jpg"), 0) + gray[name] = cv2.imread(str(image_path(name)), 0) return gray[name] def depth(name): if name not in depths: - depths[name] = load_prediction(root / MODEL / f"{name}.npz")[0] + depths[name] = load_prediction(depth_path(name))[0] return depths[name] def fused_with(setting: str, draw: int, rescale: bool) -> Method: @@ -159,7 +214,14 @@ def fit(): ) return {m: f[m].scale for m in members} - scale = fits.get(f"{setting}/draw{draw}/{gid}", fit_inputs(T, K, px), fit) + key = fit_inputs( + T, + K, + px, + {m: image_path(m) for m in members}, + {m: depth_path(m) for m in members}, + ) + scale = fits.get(f"{setting}/draw{draw}/{gid}", key, fit) def pv(name): d, s = depth(name), scale[name] @@ -178,7 +240,9 @@ def pv(name): "mapanything@392, images only": [model_frame_method(root / "ma392_images")] } for setting in SETTINGS: - label = MAPANYTHING_LABELS.get(setting, f"{setting} poses") + label = mapanything_label( + root / f"ma392_{setting}", setting, pose_file(scene.name, setting) + ) methods[f"mapanything@392 + {label}"] = [model_frame_method(root / f"ma392_{setting}")] for setting in SETTINGS: draws = noise_draws(setting) @@ -271,14 +335,17 @@ def score_scene(scene: Scene, methods: dict[str, list[Method]], sizes=GROUP_SIZE out[range_name] = {} for method, draws in methods.items(): res: dict = {} + groups = comparable_groups(scene.groups, sizes) for n in map(str, sizes): units: dict[str, list[list[dict]]] = {c: [] for c in COHORTS} - for members in scene.groups.get(n, []): + missing = [] + for members in groups.get(n, []): ev = sets[members[0]] per_cohort: dict[str, list[dict]] = {c: [] for c in COHORTS} - for factory in draws: + for d, factory in enumerate(draws): per_view = factory(f"n{n}-{members[0]}", members) if per_view is None: + missing.append(f"n{n}-{members[0]}" + (f" draw {d}" if d else "")) continue pts = predict(scene, ev, members, per_view, max_range) for c in COHORTS: @@ -287,6 +354,9 @@ def score_scene(scene: Scene, methods: dict[str, list[Method]], sizes=GROUP_SIZE for c, raws in per_cohort.items(): if raws: units[c].append(raws) + require_complete( + scene.name, method, n, missing, len(groups.get(n, [])) * len(draws) + ) if units["surface interior"]: res[n] = {c: summarize_groups(u) for c, u in units.items() if u} if res: @@ -350,8 +420,7 @@ def markdown(results: dict) -> str: "estimate pools every seed group and noise draw; the interval is a bootstrap over seed " f"groups and, within each, noise draws ({BOOTSTRAP_REPS} replicates). 'Groups x noise " "draws' counts both. MapAnything rows have one noise draw: its weights are no longer on " - "this machine. Its advio_2018 row was run on the earlier advio_2018 poses, whose scales " - "were ARKit against ARCore (0.883, 0.936, 0.958), with the same noise as draw 0 here." + "this machine." + (HISTORICAL_NOTE if uses_historical_rows(results) else "") ) for scene, ranges in results.items(): for range_name in RANGES: @@ -379,7 +448,7 @@ def main() -> None: results[s] = score_scene(scene, pose_methods(scene, fits)) fits.save() out = Path(__file__).resolve().parents[1] / "results" - (out / "pose_priors.json").write_text(json.dumps(results, indent=1)) + (out / "pose_priors.json").write_text(results_json(results)) md = markdown(results) (out / "pose_priors.md").write_text(md) print(md) diff --git a/experiments/evals/evals/recon.py b/experiments/evals/evals/recon.py index 77ec4b13e..bbe6f52ec 100644 --- a/experiments/evals/evals/recon.py +++ b/experiments/evals/evals/recon.py @@ -29,6 +29,7 @@ import cv2 import numpy as np +from evals.coverage import level_world from evals.eth3d import ( SCENES, View, @@ -41,7 +42,7 @@ up_direction, visible_scan_points, ) -from evals.pairs import BINS_M, Points, bin_key, evaluate_fixed, pool, sample_pairs +from evals.pairs import BINS_M, Points, bin_key, evaluate_fixed, pool, results_json, sample_pairs from evals.paths import ETH3D_DIR, EVALS_DIR from evals.triangulate import sample_depth @@ -95,6 +96,17 @@ def view_groups(views: list[View]) -> dict[str, list[list[str]]]: return groups +def comparable_groups(groups: dict[str, list[list[str]]], sizes) -> dict[str, list[list[str]]]: + """Each size's groups restricted to the seeds that have a group at every size compared, so + the photo-count comparison changes the number of photos and nothing else. A seed without + enough neighbours for the largest size would otherwise leave that row alone.""" + sizes = [str(n) for n in sizes if groups.get(str(n))] + if not sizes: + return {} + common = set.intersection(*({m[0] for m in groups[n]} for n in sizes)) + return {n: [m for m in groups[n] if m[0] in common] for n in sizes} + + def visibility_dir(scene: str, tolerance: float) -> Path: return ETH3D_DIR / scene / f"visibility_{WIDTH}_tol{round(tolerance * 100)}" @@ -152,8 +164,23 @@ def __init__(self, name: str, tolerance: float = TOLERANCE): self.views = {v.name: v for v in read_views(self.dir)} self.cands = eval_candidates(name) self.groups = json.loads((self.dir / "subsets.json").read_text()) - up = up_direction(list(self.views.values())) - self.vertical = np.abs(np.load(self.dir / "normals.npy") @ up) < VERTICAL_MAX_UP + # Up is the normal of the ground plane under the cameras (`coverage.level_world`). The + # cameras' mean up axis, used before, was 5 degrees off on electro and 10 on facade; + # the change in cohort membership is recorded in the results. + views = list(self.views.values()) + R, _, _ = level_world(views, scan_points(name)[::3].astype(np.float64)) + up = R.T @ np.array([0.0, 1.0, 0.0]) + normals = np.load(self.dir / "normals.npy") + self.vertical = np.abs(normals @ up) < VERTICAL_MAX_UP + before = np.abs(normals @ up_direction(views)) < VERTICAL_MAX_UP + self.cohort_change = { + "vertical_points": int(self.vertical.sum()), + "vertical_points_with_camera_up": int(before.sum()), + "points_changing_cohort": int((self.vertical != before).sum()), + "up_change_deg": round( + float(np.degrees(np.arccos(np.clip(up @ up_direction(views), -1, 1)))), 2 + ), + } self.vis_dir = visibility_dir(name, tolerance) self._vis: dict[str, tuple[np.ndarray, np.ndarray, np.ndarray]] = {} self._oracle: dict[str, tuple[np.ndarray, np.ndarray]] = {} @@ -271,6 +298,23 @@ def _evaluate_into(raws: dict, scene: Scene, ev: EvalSet, members, per_view, max raws[cohort].append(evaluate_fixed(pts, ev.pairs[cohort], ev.refs)) +class PartialRun(SystemExit): + """A method has output for some of a size's seed groups but not all.""" + + +def require_complete(scene: str, method: str, n: str, missing: list[str], total: int) -> None: + """Refuse to score a partial run. A size with no output at all is a run that was never made + (MapAnything's 8 views don't fit in memory) and is left out of the table. A size with some + groups missing is a run that stopped partway: scoring the rest would report percentiles over + only the groups that finished, and counting the missing ones as failed pairs would blame the + method for a crash, so the run must be completed instead.""" + if missing and len(missing) < total: + raise PartialRun( + f"{scene}: {method} has no output for {len(missing)} of {total} {n}-photo groups " + f"({', '.join(missing)}); finish that run before scoring" + ) + + def score_groups(scene: Scene, methods: dict[str, Method], sizes=GROUP_SIZES) -> dict: """{range: {method: {views: {cohort: pooled summary}}}} on the fixed per-seed evaluation sets.""" out: dict = {} @@ -279,12 +323,17 @@ def score_groups(scene: Scene, methods: dict[str, Method], sizes=GROUP_SIZES) -> out[range_name] = {} for method, factory in methods.items(): res: dict = {} + groups = comparable_groups(scene.groups, sizes) for n in map(str, sizes): raws: dict[str, list] = {c: [] for c in COHORTS} - for members in scene.groups.get(n, []): + missing = [] + for members in groups.get(n, []): per_view = factory(f"n{n}-{members[0]}", members) - if per_view is not None: - _evaluate_into(raws, scene, sets[members[0]], members, per_view, max_range) + if per_view is None: + missing.append(f"n{n}-{members[0]}") + continue + _evaluate_into(raws, scene, sets[members[0]], members, per_view, max_range) + require_complete(scene.name, method, n, missing, len(groups.get(n, []))) if raws["surface interior"]: res[n] = {c: pool(r) | {"groups": len(r)} for c, r in raws.items() if r} if res: @@ -432,6 +481,7 @@ def score(tolerance: float = TOLERANCE) -> dict: for s in SCENES: scene = Scene(s, tolerance) results[s] = { + "vertical_cohort": scene.cohort_change, "groups": score_groups(scene, true_pose_methods(scene)), "single": { m: score_single_photos(scene, m) @@ -484,7 +534,7 @@ def main() -> None: prepare(s) elif args.step == "score": results = score() - (args.out / "eth3d_recon.json").write_text(json.dumps(results, indent=1)) + (args.out / "eth3d_recon.json").write_text(results_json(results)) md = markdown(results) (args.out / "eth3d_recon.md").write_text(md) print(md) diff --git a/experiments/evals/evals/replay.py b/experiments/evals/evals/replay.py index 6435790ad..7c46ef934 100644 --- a/experiments/evals/evals/replay.py +++ b/experiments/evals/evals/replay.py @@ -107,6 +107,28 @@ def read_frames(video: Path, wanted: list[int]) -> dict[int, np.ndarray]: return out +def replay_identity(sequence: int, start: float, end: float) -> tuple[str, dict]: + """The replay's folder name and recorded inputs. Bounds must be whole seconds: the name keeps + only whole seconds, so 40.1 and 40.9 would otherwise share a folder.""" + if start != int(start) or end != int(end): + raise ValueError(f"start {start} and end {end} must be whole seconds: they name the replay") + source = {"sequence": sequence, "start": int(start), "end": int(end)} + return f"advio-{sequence:02d}-{int(start):04d}-{int(end):04d}", source + + +def clear_replay(folder: Path, source: dict) -> None: + """Removes an earlier build of the same replay; refuses a folder built from other inputs.""" + if not folder.exists(): + return + manifest = folder / "session.json" + found = json.loads(manifest.read_text()).get("replaySource") if manifest.exists() else None + if found != source: + raise ValueError( + f"{folder} exists and was not built from {source} (found {found}); move it first" + ) + shutil.rmtree(folder) + + def build_session(seq: Sequence, start: float, end: float, out_dir: Path) -> dict: t = seq.frame_times ark = seq.arkit @@ -124,10 +146,9 @@ def build_session(seq: Sequence, start: float, end: float, out_dir: Path) -> dic cal = seq.calibration fx, fy, cx, cy = landscape_intrinsics_from_portrait(cal.fx, cal.fy, cal.cx, cal.cy, cal.width) - session_id = f"advio-{seq.number:02d}-{int(start):04d}-{int(end):04d}" + session_id, source = replay_identity(seq.number, start, end) folder = out_dir / session_id - if folder.exists(): - shutil.rmtree(folder) + clear_replay(folder, source) (folder / "keyframes").mkdir(parents=True) frames = read_frames(seq.video, picks) @@ -182,6 +203,7 @@ def build_session(seq: Sequence, start: float, end: float, out_dir: Path) -> dic "pixel": "[u, v] continuous image coordinates: (0, 0) is the top-left corner of the JPEG, v grows down", "ray": "origin is the camera position; direction is a unit vector through the tapped pixel", }, + "replaySource": source, "session": { "id": session_id, # ADVIO publishes no wall-clock capture time; this is the Zenodo record's publication date. @@ -235,8 +257,8 @@ def main() -> None: description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter ) ap.add_argument("--sequence", type=int, default=20) - ap.add_argument("--start", type=float, default=40.0, help="seconds into the sequence") - ap.add_argument("--end", type=float, default=75.0) + ap.add_argument("--start", type=int, default=40, help="whole seconds into the sequence") + ap.add_argument("--end", type=int, default=75) ap.add_argument("--out", type=Path, default=REPLAYS_DIR) args = ap.parse_args() seq = load_sequence(ADVIO_DIR / f"advio-{args.sequence:02d}") diff --git a/experiments/evals/field/rules.json b/experiments/evals/field/rules.json index 8f5acafc2..8f4ea3bfa 100644 --- a/experiments/evals/field/rules.json +++ b/experiments/evals/field/rules.json @@ -6,7 +6,7 @@ "facing_gap_ft": { "value_ft": 3, "pass_when": "at_least", - "source": "The 3 ft fence clearance PR #7's pass criteria cite from Base's public help page" + "source": "Base's public placement help page, https://help.basepowercompany.com/en/articles/10280705 (read 2026-09-27): in narrow spots such as alleyways, 'maintaining a minimum clearance of 3 feet from walls or other fixed features'" } } } diff --git a/experiments/evals/models/README.md b/experiments/evals/models/README.md index ccdf5ce65..33f9f84f9 100644 --- a/experiments/evals/models/README.md +++ b/experiments/evals/models/README.md @@ -12,7 +12,7 @@ uv sync --project models uv run --project models pytest -q models/tests ``` -Checkpoints download on first use into `$HOUSE_SCANNING_DATA/evals/hf-cache` (`HF_HOME`) and `evals/torch-cache` (`TORCH_HOME`) unless those variables are already set. A run refuses to start a download with less than 6 GB free, and checks each checkpoint's sha256 against the pinned value. +Checkpoints download on first use into `$HOUSE_SCANNING_DATA/evals/hf-cache` (`HF_HOME`) and `evals/torch-cache` (`TORCH_HOME`) unless those variables are already set. A run refuses to start a download with less than 6 GB free on the volume holding `HF_HOME`, and checks each checkpoint's sha256 against the pinned value. The grouped MapAnything runner (`models.run_groups`) does both too, records the checkpoint in each group's `run.json`, and reuses a group's output only when a fingerprint of its members, image bytes, intrinsics, poses, input size and checkpoint matches. ## Run @@ -43,7 +43,7 @@ Each run loads one model and processes images one at a time; mapanything runs al | `cam_to_world` | mapanything only. float64 4x4, OpenCV camera, metres. Frame: the first image's camera, or the `--poses` world when poses are given | | `intrinsics_predicted` | mapanything only. What its output rays imply, in input pixels | -`DIR/run.json` records the model, checkpoint repo, revision and sha256, code commit, device, torch version, flags, load time and, per image, seconds, the network input size, the exact resize and crop, and a depth summary. +A run writes into `DIR.staging` and replaces `DIR` only when every image is done, so a run that fails midway leaves the previous run whole. `DIR/run.json` records the model, checkpoint repo, revision and sha256, code commit, device, torch version, flags, load time and, per image, seconds, the network input size, the exact resize and crop, and a depth summary. Network outputs are resampled to the input grid bilinearly. A pixel is valid only if every network pixel it draws on is valid and it lies inside the region the network saw; pixels removed by a crop are invalid. diff --git a/experiments/evals/models/common.py b/experiments/evals/models/common.py index bb2af509e..060c94887 100644 --- a/experiments/evals/models/common.py +++ b/experiments/evals/models/common.py @@ -34,20 +34,36 @@ def set_cache_dirs() -> dict[str, str]: return {k: os.environ[k] for k in ("HF_HOME", "TORCH_HOME")} +def hf_cache_dir() -> Path: + """Where Hugging Face writes downloads: HF_HOME, which `set_cache_dirs` fills in unless the + caller chose it.""" + return Path(os.environ.get("HF_HOME", CACHE_ROOT / "hf-cache")).expanduser() + + +def require_free_space(destination: Path, what: str) -> None: + """Refuse to write `what` to `destination` with less than 6 GB free on its volume: the disk is + shared and has run out before. `destination` need not exist yet; its nearest existing parent is + measured, which is on the volume the download will land on.""" + probe = destination + while not probe.exists(): + probe = probe.parent + free = shutil.disk_usage(probe).free + if free < MIN_FREE_BYTES_FOR_DOWNLOAD: + raise RuntimeError( + f"{what} is not cached and only {free / 1024**3:.1f} GB is free at {probe}; " + f"refusing to download with less than {MIN_FREE_BYTES_FOR_DOWNLOAD / 1024**3:.0f} GB free" + ) + + def require_free_space_for_download(repo_id: str, filename: str, revision: str) -> None: - """Refuse to start a checkpoint download with less than 6 GB free: the disk is shared and has - run out before. A checkpoint already in the cache needs no space.""" + """Refuse to start a checkpoint download into the Hugging Face cache with less than 6 GB free + there. A checkpoint already in the cache needs no space.""" from huggingface_hub import try_to_load_from_cache cached = try_to_load_from_cache(repo_id, filename, revision=revision) if isinstance(cached, str) and Path(cached).exists(): return - free = shutil.disk_usage(Path.home()).free - if free < MIN_FREE_BYTES_FOR_DOWNLOAD: - raise RuntimeError( - f"{repo_id}/{filename} is not cached and only {free / 1024**3:.1f} GB is free; " - f"refusing to download with less than {MIN_FREE_BYTES_FOR_DOWNLOAD / 1024**3:.0f} GB free" - ) + require_free_space(hf_cache_dir(), f"{repo_id}/{filename}") def checkpoint_record(repo_id: str, filename: str, revision: str, sha256: str) -> dict: @@ -377,3 +393,24 @@ def depth_summary(depth: np.ndarray, valid: np.ndarray) -> dict: "p5_m": round(float(np.percentile(d, 5)), 3), "p95_m": round(float(np.percentile(d, 95)), 3), } + + +def fingerprint( + members: list[str], + images: list[Path], + intrinsics, + poses, + max_side: int | None, + checkpoint_sha256: str, +) -> str: + """sha256 over everything a grouped output depends on: members, image bytes, the intrinsics + given to the model, poses, input size and the checkpoint.""" + header = {"members": members, "max_side": max_side, "checkpoint": checkpoint_sha256} + h = hashlib.sha256(json.dumps(header).encode()) + for path in images: + h.update(path.read_bytes()) + h.update(np.asarray(intrinsics, np.float64).tobytes()) + if poses is not None: + for pose in poses: + h.update(np.asarray(pose, np.float64).tobytes()) + return h.hexdigest() diff --git a/experiments/evals/models/run.py b/experiments/evals/models/run.py index 8314ecf7f..db89aa64a 100644 --- a/experiments/evals/models/run.py +++ b/experiments/evals/models/run.py @@ -16,6 +16,7 @@ import importlib import json import platform +import shutil import sys import time from pathlib import Path @@ -129,11 +130,56 @@ def main(argv: list[str] | None = None) -> None: module = importlib.import_module(MODULES[args.model]) device = pick_device(args.device) require_free_space_for_download(module.REPO, module.FILENAME, module.REVISION) - args.out.mkdir(parents=True, exist_ok=True) + # Everything is written to a staging folder and swapped in only when the whole run succeeds, + # so a run that stops midway (out of memory, cancelled) leaves the previous run whole instead of + # a mix of its depth maps and new ones under the old run.json. + stage = args.out.with_name(args.out.name + ".staging") + shutil.rmtree(stage, ignore_errors=True) + stage.mkdir(parents=True) + try: + run_into(args, stage, images, intrinsics, poses, caches, module, device, torch) + except BaseException: + shutil.rmtree(stage, ignore_errors=True) + raise + publish(stage, args.out) + print(f"wrote the run's npz files and run.json to {args.out}", file=sys.stderr) + +def publish(stage: Path, out: Path) -> None: + """Replace `out` with the complete run in `stage`, all or nothing. + + The previous run is set aside as `out.previous` and put back if the swap fails. A + `.previous` is deleted only when a complete run (with its run.json) is published at `out`. A + `.previous` with nothing at `out` is a run a killed publication set aside, and is restored + before anything else, so a failed publication never loses the last complete run.""" + previous = out.with_name(out.name + ".previous") + if previous.exists(): + if (out / "run.json").exists(): + shutil.rmtree(previous) # a later publication completed; this one is superseded + elif not out.exists(): + previous.rename(out) + else: + raise RuntimeError( + f"both {out} (without run.json) and {previous} exist; inspect them by hand" + ) + if out.exists(): + out.rename(previous) + try: + stage.rename(out) + except BaseException: + if previous.exists() and not out.exists(): + previous.rename(out) + raise + shutil.rmtree(previous, ignore_errors=True) + + +def run_into(args, out: Path, images, intrinsics, poses, caches, module, device, torch) -> None: + """Load the model, verify its checkpoint, and write every prediction and run.json to `out`.""" start = time.perf_counter() model = module.load(device) load_seconds = time.perf_counter() - start + # Before any output: a replaced cached checkpoint must not leave depth maps behind. + checkpoint = checkpoint_record(module.REPO, module.FILENAME, module.REVISION, module.SHA256) if device == "mps": torch.mps.empty_cache() torch.mps.set_per_process_memory_fraction( @@ -153,7 +199,7 @@ def main(argv: list[str] | None = None) -> None: per_image = [] for res in module.run(model, inputs): write_npz( - args.out, + out, res.path.stem, res.depth, res.valid, @@ -188,9 +234,7 @@ def main(argv: list[str] | None = None) -> None: run = { "model": args.model, "license": module.LICENSE, - "checkpoint": checkpoint_record( - module.REPO, module.FILENAME, module.REVISION, module.SHA256 - ), + "checkpoint": checkpoint, "code": module.CODE, "device": device, "torch": torch.__version__, @@ -208,8 +252,7 @@ def main(argv: list[str] | None = None) -> None: } if args.model == "mapanything": run["timing_note"] = "views run jointly; per-image seconds = joint seconds / views" - (args.out / "run.json").write_text(json.dumps(run, indent=2) + "\n") - print(f"wrote {len(per_image)} npz and run.json to {args.out}", file=sys.stderr) + (out / "run.json").write_text(json.dumps(run, indent=2) + "\n") if __name__ == "__main__": diff --git a/experiments/evals/models/run_groups.py b/experiments/evals/models/run_groups.py index edb34f82f..e0e46afcf 100644 --- a/experiments/evals/models/run_groups.py +++ b/experiments/evals/models/run_groups.py @@ -15,16 +15,52 @@ from __future__ import annotations import argparse +import hashlib import json +import shutil import sys import time from pathlib import Path import numpy as np -from models.common import RunInputs, read_intrinsics, set_cache_dirs, write_npz +from models import map_anything +from models.common import ( + RunInputs, + checkpoint_record, + fingerprint, + read_intrinsics, + require_free_space_for_download, + set_cache_dirs, + write_npz, +) from models.map_anything import load, run -from models.run import pick_device +from models.run import pick_device, publish + + +def write_group(out: Path, results, summary: dict) -> None: + """Write one group's depth maps and run.json, all or nothing: they go to a staging folder + that `publish` swaps in for `out` only once every file is written, so a failure partway + leaves the previous outputs and run.json as they were, never a mix of old and new views.""" + stage = out.with_name(out.name + ".staging") + shutil.rmtree(stage, ignore_errors=True) + stage.mkdir(parents=True) + try: + for res in results: + write_npz( + stage, + res.path.stem, + res.depth, + res.valid, + res.intrinsics, + res.cam_to_world, + res.arrays, + ) + (stage / "run.json").write_text(json.dumps(summary, indent=1) + "\n") + except BaseException: + shutil.rmtree(stage, ignore_errors=True) + raise + publish(stage, out) def main() -> None: @@ -60,8 +96,11 @@ def main() -> None: images_dir = args.scene_dir / "images_1024" intr_file = args.scene_dir / "model_inputs" / "intrinsics.json" device = pick_device(args.device) + ckpt = (map_anything.REPO, map_anything.FILENAME, map_anything.REVISION) + require_free_space_for_download(*ckpt) t0 = time.perf_counter() model = load(device) + checkpoint = checkpoint_record(*ckpt, map_anything.SHA256) if device == "mps": torch.mps.empty_cache() # loading leaves about 0.6 GB of freed blocks cached torch.mps.set_per_process_memory_fraction( @@ -72,15 +111,21 @@ def main() -> None: for members in members_list: gid = f"n{n}-{members[0]}" out = args.out / gid - if (out / "run.json").exists(): - continue poses = None if pose_sets is not None: poses = [np.array(pose_sets[gid]["poses"][m]) for m in members] images = [images_dir / f"{m}.jpg" for m in members] + intrinsics = read_intrinsics(intr_file, images) + key = fingerprint( + members, images, intrinsics, poses, args.max_side, checkpoint["sha256"] + ) + done = out / "run.json" + # Reuse only an output made from these exact inputs and this checkpoint. + if done.exists() and json.loads(done.read_text()).get("fingerprint") == key: + continue inputs = RunInputs( images=images, - intrinsics=read_intrinsics(intr_file, images), + intrinsics=intrinsics, intrinsics_mode="known", poses=poses, max_side=args.max_side, @@ -88,26 +133,23 @@ def main() -> None: fp32=False, ) results = run(model, inputs) - out.mkdir(parents=True, exist_ok=True) - for res in results: - write_npz( - out, - res.path.stem, - res.depth, - res.valid, - res.intrinsics, - res.cam_to_world, - res.arrays, - ) summary = { + "fingerprint": key, + "checkpoint": checkpoint, "members": members, "seconds_per_view": round(results[0].seconds, 3), "network_wh": list(results[0].network_wh), "weights": "transformer stacks bf16, rest fp32 (models/map_anything.py load)", "metric_scaling_factor": [r.extra["metric_scaling_factor"] for r in results], "poses_file": str(args.poses_file) if args.poses_file else None, + # What evals.pose_priors labels the row by: the exact pose file this run used. + "poses_sha256": ( + hashlib.sha256(args.poses_file.read_bytes()).hexdigest() + if args.poses_file + else None + ), } - (out / "run.json").write_text(json.dumps(summary, indent=1) + "\n") + write_group(out, results, summary) print(f"n={n} {members[0]}: {results[0].seconds:.2f} s/view", file=sys.stderr) if device == "mps": torch.mps.empty_cache() diff --git a/experiments/evals/models/tests/test_common.py b/experiments/evals/models/tests/test_common.py index bb5f862b0..e6d6ef04c 100644 --- a/experiments/evals/models/tests/test_common.py +++ b/experiments/evals/models/tests/test_common.py @@ -3,6 +3,8 @@ from __future__ import annotations import json +import shutil +from pathlib import Path import numpy as np import pytest @@ -117,3 +119,299 @@ def test_intrinsics_json_must_cover_every_image(tmp_path): path.write_text(json.dumps([[1, 1, 0, 0]])) with pytest.raises(ValueError, match="1 intrinsics entries for 2 images"): read_intrinsics(path, images) + + +def test_fingerprint_changes_with_every_input(tmp_path): + from models.common import fingerprint + + a, b = tmp_path / "a.jpg", tmp_path / "b.jpg" + a.write_bytes(b"a") + b.write_bytes(b"b") + pose = np.eye(4) + k = [np.array([500.0, 500, 320, 240])] * 2 + + def key(members=("a", "b"), intrinsics=k, poses=None, max_side=392, ckpt="c1"): + images = [tmp_path / f"{m}.jpg" for m in members] + return fingerprint(list(members), images, intrinsics, poses, max_side, ckpt) + + base = key() + assert base == key() + assert base != key(poses=[pose, pose]) # poses added + assert base != key(members=("a",), intrinsics=k[:1]) # members changed + assert base != key(max_side=518) # resolution changed + assert base != key(ckpt="c2") # checkpoint changed + focal = [np.array([510.0, 500, 320, 240]), k[1]] + assert base != key(intrinsics=focal) # only one focal length changed + b.write_bytes(b"B") + assert base != key() # an image changed + + +def test_free_space_is_measured_where_the_download_lands(tmp_path, monkeypatch): + import shutil + from collections import namedtuple + + from models import common + + Usage = namedtuple("Usage", "total used free") + measured = [] + + def fake_usage(path): + measured.append(Path(path)) + free = 100 * 1024**3 if Path(path) == tmp_path / "big" else 1024**3 + return Usage(0, 0, free) + + monkeypatch.setattr(shutil, "disk_usage", fake_usage) + (tmp_path / "big").mkdir() + # The cache folder does not exist yet: its nearest existing parent is on the roomy volume. + monkeypatch.setenv("HF_HOME", str(tmp_path / "big" / "hf-cache")) + common.require_free_space(common.hf_cache_dir(), "ckpt") + assert measured == [tmp_path / "big"] + monkeypatch.setenv("HF_HOME", str(tmp_path / "small")) + with pytest.raises(RuntimeError, match=r"only 1\.0 GB is free"): + common.require_free_space(common.hf_cache_dir(), "ckpt") + + +def test_checkpoint_record_rejects_a_replaced_checkpoint(tmp_path, monkeypatch): + import hashlib + import sys + import types + + from models.common import checkpoint_record + + weights = tmp_path / "model.safetensors" + weights.write_bytes(b"pinned weights") + hub = types.ModuleType("huggingface_hub") + hub.try_to_load_from_cache = lambda repo, name, revision: str(weights) + monkeypatch.setitem(sys.modules, "huggingface_hub", hub) + pinned = hashlib.sha256(b"pinned weights").hexdigest() + record = checkpoint_record("org/model", "model.safetensors", "rev", pinned) + assert record["sha256"] == pinned and record["bytes"] == 14 + weights.write_bytes(b"other weights") + with pytest.raises(RuntimeError, match="is not the pinned"): + checkpoint_record("org/model", "model.safetensors", "rev", pinned) + + +def test_a_replaced_checkpoint_stops_the_run_before_any_output(tmp_path, monkeypatch): + import sys + import types + + from models import run + + weights = tmp_path / "model.safetensors" + weights.write_bytes(b"some other loadable file") + hub = types.ModuleType("huggingface_hub") + hub.try_to_load_from_cache = lambda repo, name, revision: str(weights) + torch = types.ModuleType("torch") + torch.__version__ = "fake" + torch.cuda = types.SimpleNamespace(is_available=lambda: False) + torch.backends = types.SimpleNamespace(mps=types.SimpleNamespace(is_available=lambda: False)) + model = types.ModuleType("fake_model") + model.REPO, model.FILENAME, model.REVISION = "org/model", "model.safetensors", "rev" + model.SHA256 = "0" * 64 + model.load = lambda device: object() + + def never_run(*args): + raise AssertionError("inference ran with an unverified checkpoint") + + model.run = never_run + for name, module in (("huggingface_hub", hub), ("torch", torch), ("fake_model", model)): + monkeypatch.setitem(sys.modules, name, module) + monkeypatch.setitem(run.MODULES, "moge2", "fake_model") + image = tmp_path / "a.jpg" + image.write_bytes(b"") + (tmp_path / "images.txt").write_text(f"{image}\n") + out = tmp_path / "out" + with pytest.raises(RuntimeError, match="is not the pinned"): + run.main( + [ + "--model", + "moge2", + "--images", + str(tmp_path / "images.txt"), + "--device", + "cpu", + "--out", + str(out), + ] + ) + assert not list(out.glob("*.npz")) + + +def _fake_model_run(tmp_path, monkeypatch, fail_on: str | None): + """models.run.main on two images with a stand-in model whose checkpoint verifies; it raises + on the image named `fail_on` after the images before it were written.""" + import hashlib + import sys + import types + + from models import common, run + + weights = tmp_path / "model.safetensors" + weights.write_bytes(b"pinned") + hub = types.ModuleType("huggingface_hub") + hub.try_to_load_from_cache = lambda repo, name, revision: str(weights) + torch = types.ModuleType("torch") + torch.__version__ = "fake" + torch.cuda = types.SimpleNamespace(is_available=lambda: False) + torch.backends = types.SimpleNamespace(mps=types.SimpleNamespace(is_available=lambda: False)) + model = types.ModuleType("fake_model") + model.REPO, model.FILENAME, model.REVISION = "org/model", "model.safetensors", "rev" + model.SHA256 = hashlib.sha256(b"pinned").hexdigest() + model.LICENSE, model.CODE = "test", "test" + model.load = lambda device: object() + + def infer(_, inputs): + for path in inputs.images: + if path.stem == fail_on: + raise MemoryError(f"out of memory on {path.name}") + depth = np.full((2, 2), 9.0, np.float32) + yield types.SimpleNamespace( + path=path, + depth=depth, + valid=np.ones((2, 2), bool), + intrinsics=np.array([1.0, 1, 0.5, 0.5]), + cam_to_world=None, + arrays={}, + seconds=0.0, + network_wh=(2, 2), + resample=common.Resample(2, 2, 2, 2, 0, 0, 2, 2), + extra={}, + ) + + model.run = infer + for name, module in (("huggingface_hub", hub), ("torch", torch), ("fake_model", model)): + monkeypatch.setitem(sys.modules, name, module) + monkeypatch.setitem(run.MODULES, "moge2", "fake_model") + listing = tmp_path / "images.txt" + listing.write_text("".join(f"{tmp_path / n}.jpg\n" for n in ("a", "b"))) + for n in ("a", "b"): + (tmp_path / f"{n}.jpg").write_bytes(b"") + out = tmp_path / "out" + return lambda: run.main( + ["--model", "moge2", "--images", str(listing), "--device", "cpu", "--out", str(out)] + ), out + + +def _snapshot(folder): + return {p.name: p.read_bytes() for p in sorted(folder.iterdir())} + + +def test_a_run_that_fails_midway_leaves_the_previous_run_intact(tmp_path, monkeypatch): + main, out = _fake_model_run(tmp_path, monkeypatch, fail_on="b") + out.mkdir() + for name in ("a.npz", "b.npz", "run.json"): + (out / name).write_bytes(f"previous {name}".encode()) + before = _snapshot(out) + with pytest.raises(MemoryError): + main() # image a is predicted, then b fails + assert _snapshot(out) == before + assert sorted(p.name for p in tmp_path.iterdir() if p.name.startswith("out")) == ["out"] + + +def test_a_completed_run_replaces_the_previous_one_whole(tmp_path, monkeypatch): + main, out = _fake_model_run(tmp_path, monkeypatch, fail_on=None) + out.mkdir() + (out / "stale.npz").write_bytes(b"from an older image list") + (out / "run.json").write_text("{}") + main() + assert sorted(p.name for p in out.iterdir()) == ["a.npz", "b.npz", "run.json"] + assert json.loads((out / "run.json").read_text())["images"][1]["stem"] == "b" + assert sorted(p.name for p in tmp_path.iterdir() if p.name.startswith("out")) == ["out"] + + +def _complete_run(folder, tag): + folder.mkdir() + (folder / "a.npz").write_bytes(f"{tag} a".encode()) + (folder / "run.json").write_text(json.dumps({"run": tag})) + + +def test_a_publication_that_fails_twice_keeps_the_previous_run(tmp_path, monkeypatch): + from models import run + + out = tmp_path / "out" + _complete_run(out, "previous") + before = _snapshot(out) + real_rename = Path.rename + + def failing_rename(self, target): + if self.name == "out.staging": + raise OSError("injected: the new run could not be moved into place") + return real_rename(self, target) + + monkeypatch.setattr(Path, "rename", failing_rename) + for attempt in range(2): + stage = tmp_path / "out.staging" + _complete_run(stage, f"new {attempt}") + with pytest.raises(OSError, match="injected"): + run.publish(stage, out) + assert _snapshot(out) == before + assert not (tmp_path / "out.previous").exists() + shutil.rmtree(stage) + + +def test_a_run_left_aside_by_a_killed_publication_is_restored_not_deleted(tmp_path, monkeypatch): + from models import run + + # A kill between setting the old run aside and moving the new one in leaves only .previous. + _complete_run(tmp_path / "out.previous", "previous") + before = _snapshot(tmp_path / "out.previous") + stage = tmp_path / "out.staging" + _complete_run(stage, "new") + run.publish(stage, tmp_path / "out") + assert json.loads((tmp_path / "out" / "run.json").read_text()) == {"run": "new"} + assert not (tmp_path / "out.previous").exists() + # Restored first, then superseded by a complete publication: never deleted while alone. + _complete_run(tmp_path / "out.previous", "previous") + shutil.rmtree(tmp_path / "out") + stage = tmp_path / "out.staging" + _complete_run(stage, "new again") + + real = Path.rename + + def refuse_staging(self, target): + if self.name == "out.staging": + raise OSError("injected") + return real(self, target) + + monkeypatch.setattr(Path, "rename", refuse_staging) + with pytest.raises(OSError, match="injected"): + run.publish(stage, tmp_path / "out") + assert _snapshot(tmp_path / "out") == before + + +def test_a_group_write_that_fails_midway_keeps_the_previous_outputs(tmp_path, monkeypatch): + import types + + from models import run_groups + + out = tmp_path / "n2-a" + _complete_run(out, "previous") + (out / "b.npz").write_bytes(b"previous b") + before = _snapshot(out) + real_write = run_groups.write_npz + + def fail_on_b(folder, stem, *args): + if stem == "b": + raise OSError("injected: disk full while writing b") + return real_write(folder, stem, *args) + + monkeypatch.setattr(run_groups, "write_npz", fail_on_b) + results = [ + types.SimpleNamespace( + path=tmp_path / f"{n}.jpg", + depth=np.ones((2, 2), np.float32), + valid=np.ones((2, 2), bool), + intrinsics=np.array([1.0, 1, 0.5, 0.5]), + cam_to_world=np.eye(4), + arrays={}, + ) + for n in ("a", "b") + ] + with pytest.raises(OSError, match="injected"): + run_groups.write_group(out, results, {"run": "new"}) # a is written, then b fails + assert _snapshot(out) == before + assert sorted(p.name for p in tmp_path.iterdir()) == ["n2-a"] + monkeypatch.setattr(run_groups, "write_npz", real_write) + run_groups.write_group(out, results, {"run": "new"}) + assert sorted(p.name for p in out.iterdir()) == ["a.npz", "b.npz", "run.json"] + assert json.loads((out / "run.json").read_text()) == {"run": "new"} diff --git a/experiments/evals/results/coverage.json b/experiments/evals/results/coverage.json index fc32688d8..ee2d5fbc7 100644 --- a/experiments/evals/results/coverage.json +++ b/experiments/evals/results/coverage.json @@ -40,7 +40,7 @@ "range": 0.0, "one position": 0.0 }, - "claimed_samples_seen_only_where_scan_is_empty": 0.0, + "claimed_samples_unknown_for_lack_of_scan": 0.0, "passes": false }, "ground": { @@ -62,7 +62,7 @@ "range": 0.0, "one position": 0.0 }, - "claimed_samples_seen_only_where_scan_is_empty": 0.0, + "claimed_samples_unknown_for_lack_of_scan": 0.0, "passes": null } } diff --git a/experiments/evals/results/coverage.md b/experiments/evals/results/coverage.md index f9df72f4d..4b8568ce3 100644 --- a/experiments/evals/results/coverage.md +++ b/experiments/evals/results/coverage.md @@ -1,6 +1,6 @@ # Does the app's coverage map claim surface no photo saw? (generated by `make coverage`) -HouseScanKit `CoverageMap` at beede15f568b3a4d694fb275caf9eaaa882c546b (t3/ios-mvf), unmodified, default `CoverageConfig`. ETH3D photos with true poses, every photo a kept keyframe. Lengths in feet along the wall. Definitions and pass criteria: README section 7. +HouseScanKit `CoverageMap` at beede15f568b3a4d694fb275caf9eaaa882c546b (t3/ios-mvf), unmodified, default `CoverageConfig`. ETH3D photos with true poses, every photo a kept keyframe. Lengths in feet along the wall. Definitions and pass criteria: METHODS.md section 7. | Scene | Band | Claimed | False-observed (share) | Pass (<= 0.5 ft) | Causes: occlusion / frame edge / range | False-observed, 2-position truth (share) | Seen from 2 positions | Missed | Missed causes: frame edge / grazing / range / one position | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | @@ -9,7 +9,7 @@ HouseScanKit `CoverageMap` at beede15f568b3a4d694fb275caf9eaaa882c546b (t3/ios-m ## Setup and validity checks -| Scene | Photos (with a sighting) | Wall stretch | Of it, pilaster faces | Wall plane fit RMS | Ground fit RMS | App sightings | Replica mismatches | Claimed samples seen only where the scan is empty (wall / ground) | +| Scene | Photos (with a sighting) | Wall stretch | Of it, pilaster faces | Wall plane fit RMS | Ground fit RMS | App sightings | Replica mismatches | Claimed samples unknown for lack of scan (wall / ground) | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | electro | 45 (17) | 19.3 ft | 3.8 ft | 1.8 cm | 0.5 cm | 337 | 0 | 0.0% / 0.0% | | facade | 76 (0) | none within the app's range | | | | | | | diff --git a/experiments/evals/results/coverage_options.json b/experiments/evals/results/coverage_options.json index 04b758f14..b6cfc4e5b 100644 --- a/experiments/evals/results/coverage_options.json +++ b/experiments/evals/results/coverage_options.json @@ -72,7 +72,7 @@ "missed_ft": 17.979002624671917, "ground_claimed_ft": 0.0, "ground_false_observed_ft": 0.0, - "passes": true + "passes": null }, { "b_m": 0.5, @@ -144,7 +144,7 @@ "missed_ft": 17.979002624671917, "ground_claimed_ft": 0.0, "ground_false_observed_ft": 0.0, - "passes": true + "passes": null }, { "b_m": 1.0, @@ -216,7 +216,7 @@ "missed_ft": 17.979002624671917, "ground_claimed_ft": 0.0, "ground_false_observed_ft": 0.0, - "passes": true + "passes": null }, { "b_m": 2.0, @@ -288,7 +288,7 @@ "missed_ft": 17.979002624671917, "ground_claimed_ft": 0.0, "ground_false_observed_ft": 0.0, - "passes": true + "passes": null }, { "b_m": 5.0, @@ -360,7 +360,7 @@ "missed_ft": 17.979002624671917, "ground_claimed_ft": 0.0, "ground_false_observed_ft": 0.0, - "passes": true + "passes": null } ], "options": [ @@ -397,9 +397,9 @@ { "option": "depth test, no depth past 5 m, 9 rows", "needs_lidar": true, - "claimed_ft": 10.170603674540683, + "claimed_ft": 9.120734908136482, "false_observed_ft": 0.0, - "missed_ft": 7.808398950131234, + "missed_ft": 8.858267716535433, "ground_claimed_ft": 0.0, "ground_false_observed_ft": 0.0, "passes": true diff --git a/experiments/evals/results/coverage_options.md b/experiments/evals/results/coverage_options.md index 88b5f3cec..9a622f9ca 100644 --- a/experiments/evals/results/coverage_options.md +++ b/experiments/evals/results/coverage_options.md @@ -4,24 +4,26 @@ ETH3D electro, the 19.3 ft wall of section 7. Feet along the wall. False-observe ## Position baseline b and angle diversity theta (no LiDAR): false-observed / missed +An option that claims no wall is untested, not passed. + | b \ theta | 0 deg | 15 deg | 30 deg | 45 deg | 60 deg | 75 deg | | --- | --- | --- | --- | --- | --- | --- | -| 0.25 m | 1.1 / 0.0 | 1.1 / 0.0 | 1.1 / 0.0 | 1.1 / 0.0 | 0.0 / 8.5 | 0.0 / 18.0 | -| 0.5 m | 1.1 / 0.0 | 1.1 / 0.0 | 1.1 / 0.0 | 1.1 / 0.0 | 0.0 / 8.5 | 0.0 / 18.0 | -| 1 m | 1.1 / 0.0 | 1.1 / 0.0 | 1.1 / 0.0 | 1.1 / 0.0 | 0.0 / 8.5 | 0.0 / 18.0 | -| 2 m | 1.1 / 0.0 | 1.1 / 0.0 | 1.1 / 0.0 | 1.1 / 0.0 | 0.0 / 8.5 | 0.0 / 18.0 | -| 5 m | 0.0 / 6.0 | 0.0 / 6.0 | 0.0 / 6.0 | 0.0 / 6.0 | 0.0 / 8.5 | 0.0 / 18.0 | +| 0.25 m | 1.1 / 0.0 | 1.1 / 0.0 | 1.1 / 0.0 | 1.1 / 0.0 | 0.0 / 8.5 | untested / 18.0 | +| 0.5 m | 1.1 / 0.0 | 1.1 / 0.0 | 1.1 / 0.0 | 1.1 / 0.0 | 0.0 / 8.5 | untested / 18.0 | +| 1 m | 1.1 / 0.0 | 1.1 / 0.0 | 1.1 / 0.0 | 1.1 / 0.0 | 0.0 / 8.5 | untested / 18.0 | +| 2 m | 1.1 / 0.0 | 1.1 / 0.0 | 1.1 / 0.0 | 1.1 / 0.0 | 0.0 / 8.5 | untested / 18.0 | +| 5 m | 0.0 / 6.0 | 0.0 / 6.0 | 0.0 / 6.0 | 0.0 / 6.0 | 0.0 / 8.5 | untested / 18.0 | ## Options -The last four rows allow that much wall relief in front of the tapped plane, a setting chosen after seeing this wall; see README section 7b. +The last four rows allow that much wall relief in front of the tapped plane, a setting chosen after seeing this wall; see METHODS.md section 7b. | Option | Needs LiDAR | Claimed | False-observed | Pass | Missed | Ground claimed | | --- | --- | --- | --- | --- | --- | --- | | the app today (b = 0.25 m, theta = 0) | no | 19.3 | 1.1 | no | 0.0 | 0.0 | | depth test, true depth to 6 m, 3 rows | yes | 14.6 | 0.0 | yes | 3.3 | 0.0 | | depth test, true depth to 6 m, 9 rows | yes | 14.6 | 0.0 | yes | 3.3 | 0.0 | -| depth test, no depth past 5 m, 9 rows | yes | 10.2 | 0.0 | yes | 7.8 | 0.0 | +| depth test, no depth past 5 m, 9 rows | yes | 9.1 | 0.0 | yes | 8.9 | 0.0 | | depth test, true depth to 6 m, 9 rows, 0.2 m of relief | yes | 14.6 | 0.0 | yes | 3.3 | 0.0 | | depth test, true depth to 6 m, 9 rows, 0.3 m of relief | yes | 15.1 | 0.0 | yes | 2.9 | 0.0 | | depth test, true depth to 6 m, 9 rows, 0.4 m of relief | yes | 18.1 | 0.1 | yes | 0.0 | 0.0 | diff --git a/experiments/evals/results/eth3d_recon.json b/experiments/evals/results/eth3d_recon.json index 3d2b74609..21ed57bb5 100644 --- a/experiments/evals/results/eth3d_recon.json +++ b/experiments/evals/results/eth3d_recon.json @@ -1,5 +1,11 @@ { "facade": { + "vertical_cohort": { + "vertical_points": 856369, + "vertical_points_with_camera_up": 855248, + "points_changing_cohort": 23145, + "up_change_deg": 10.02 + }, "groups": { "all points": { "oracle": { @@ -26,15 +32,15 @@ "pairs": 600000, "failed_pct": 0.0, "median_in": 0.29, - "p90_in": 2.69, + "p90_in": 2.56, "scale_error_pct": 0.01 }, "3-10m": { "pairs": 600000, "failed_pct": 0.0, "median_in": 0.73, - "p90_in": 6.15, - "scale_error_pct": 0.02 + "p90_in": 5.96, + "scale_error_pct": 0.01 } }, "tape_calibration_success_pct": 100.0, @@ -46,8 +52,8 @@ "pairs": 24000, "failed_pct": 0.0, "median_in": 0.2, - "p90_in": 0.92, - "scale_error_pct": -0.21 + "p90_in": 0.97, + "scale_error_pct": -0.2 }, "3-10m": { "pairs": 24000, @@ -61,16 +67,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 0.0, - "median_in": 0.21, - "p90_in": 2.16, - "scale_error_pct": 0.02 + "median_in": 0.22, + "p90_in": 2.11, + "scale_error_pct": 0.01 }, "3-10m": { "pairs": 600000, "failed_pct": 0.0, - "median_in": 0.53, - "p90_in": 5.6, - "scale_error_pct": 0.02 + "median_in": 0.54, + "p90_in": 5.36, + "scale_error_pct": 0.01 } }, "tape_calibration_success_pct": 100.0, @@ -98,14 +104,14 @@ "pairs": 600000, "failed_pct": 0.0, "median_in": 0.64, - "p90_in": 4.15, - "scale_error_pct": -0.0 + "p90_in": 4.03, + "scale_error_pct": -0.01 }, "3-10m": { "pairs": 600000, "failed_pct": 0.0, "median_in": 1.42, - "p90_in": 7.74, + "p90_in": 7.55, "scale_error_pct": 0.02 } }, @@ -136,14 +142,14 @@ "pairs": 600000, "failed_pct": 0.0, "median_in": 0.3, - "p90_in": 2.8, + "p90_in": 2.68, "scale_error_pct": 0.02 }, "3-10m": { "pairs": 600000, "failed_pct": 0.0, - "median_in": 0.74, - "p90_in": 6.19, + "median_in": 0.75, + "p90_in": 5.99, "scale_error_pct": 0.02 } }, @@ -156,14 +162,14 @@ "pairs": 24000, "failed_pct": 0.0, "median_in": 0.21, - "p90_in": 0.99, - "scale_error_pct": -0.22 + "p90_in": 1.04, + "scale_error_pct": -0.21 }, "3-10m": { "pairs": 24000, "failed_pct": 0.0, - "median_in": 0.57, - "p90_in": 2.05, + "median_in": 0.58, + "p90_in": 2.01, "scale_error_pct": -0.2 } }, @@ -172,14 +178,14 @@ "pairs": 600000, "failed_pct": 0.0, "median_in": 0.22, - "p90_in": 2.29, + "p90_in": 2.22, "scale_error_pct": 0.02 }, "3-10m": { "pairs": 600000, "failed_pct": 0.0, - "median_in": 0.53, - "p90_in": 5.69, + "median_in": 0.54, + "p90_in": 5.41, "scale_error_pct": 0.02 } }, @@ -208,15 +214,15 @@ "pairs": 600000, "failed_pct": 0.0, "median_in": 0.65, - "p90_in": 4.34, + "p90_in": 4.22, "scale_error_pct": -0.01 }, "3-10m": { "pairs": 600000, "failed_pct": 0.0, "median_in": 1.42, - "p90_in": 7.77, - "scale_error_pct": 0.02 + "p90_in": 7.56, + "scale_error_pct": 0.01 } }, "tape_calibration_success_pct": 100.0, @@ -246,15 +252,15 @@ "pairs": 600000, "failed_pct": 0.5, "median_in": 0.31, - "p90_in": 2.89, - "scale_error_pct": 0.03 + "p90_in": 2.78, + "scale_error_pct": 0.02 }, "3-10m": { "pairs": 600000, "failed_pct": 0.5, - "median_in": 0.74, - "p90_in": 6.59, - "scale_error_pct": 0.03 + "median_in": 0.75, + "p90_in": 6.34, + "scale_error_pct": 0.02 } }, "tape_calibration_success_pct": 99.5, @@ -266,14 +272,14 @@ "pairs": 24000, "failed_pct": 0.0, "median_in": 0.2, - "p90_in": 0.96, + "p90_in": 1.0, "scale_error_pct": -0.21 }, "3-10m": { "pairs": 24000, "failed_pct": 0.0, "median_in": 0.56, - "p90_in": 1.87, + "p90_in": 1.84, "scale_error_pct": -0.2 } }, @@ -281,15 +287,15 @@ "1-3m": { "pairs": 600000, "failed_pct": 0.5, - "median_in": 0.22, - "p90_in": 2.51, + "median_in": 0.23, + "p90_in": 2.45, "scale_error_pct": 0.03 }, "3-10m": { "pairs": 600000, "failed_pct": 0.5, - "median_in": 0.54, - "p90_in": 6.06, + "median_in": 0.55, + "p90_in": 5.79, "scale_error_pct": 0.02 } }, @@ -318,15 +324,15 @@ "pairs": 600000, "failed_pct": 0.5, "median_in": 0.63, - "p90_in": 4.21, + "p90_in": 4.1, "scale_error_pct": -0.01 }, "3-10m": { "pairs": 600000, "failed_pct": 0.5, "median_in": 1.35, - "p90_in": 7.92, - "scale_error_pct": 0.02 + "p90_in": 7.73, + "scale_error_pct": 0.01 } }, "tape_calibration_success_pct": 99.5, @@ -356,15 +362,15 @@ "pairs": 600000, "failed_pct": 0.0, "median_in": 0.31, - "p90_in": 2.94, + "p90_in": 2.84, "scale_error_pct": 0.03 }, "3-10m": { "pairs": 600000, "failed_pct": 0.0, - "median_in": 0.75, - "p90_in": 6.68, - "scale_error_pct": 0.04 + "median_in": 0.76, + "p90_in": 6.41, + "scale_error_pct": 0.03 } }, "tape_calibration_success_pct": 100.0, @@ -376,14 +382,14 @@ "pairs": 24000, "failed_pct": 0.0, "median_in": 0.21, - "p90_in": 0.98, + "p90_in": 1.02, "scale_error_pct": -0.23 }, "3-10m": { "pairs": 24000, "failed_pct": 0.0, - "median_in": 0.59, - "p90_in": 1.86, + "median_in": 0.6, + "p90_in": 1.85, "scale_error_pct": -0.22 } }, @@ -392,14 +398,14 @@ "pairs": 600000, "failed_pct": 0.0, "median_in": 0.23, - "p90_in": 2.62, + "p90_in": 2.53, "scale_error_pct": 0.03 }, "3-10m": { "pairs": 600000, "failed_pct": 0.0, - "median_in": 0.55, - "p90_in": 6.14, + "median_in": 0.56, + "p90_in": 5.84, "scale_error_pct": 0.03 } }, @@ -428,14 +434,14 @@ "pairs": 600000, "failed_pct": 0.0, "median_in": 0.66, - "p90_in": 4.34, - "scale_error_pct": -0.01 + "p90_in": 4.25, + "scale_error_pct": -0.02 }, "3-10m": { "pairs": 600000, "failed_pct": 0.0, "median_in": 1.37, - "p90_in": 8.24, + "p90_in": 8.02, "scale_error_pct": 0.01 } }, @@ -467,16 +473,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 0.72, - "median_in": 3.02, - "p90_in": 12.12, - "scale_error_pct": -0.16 + "median_in": 2.98, + "p90_in": 11.9, + "scale_error_pct": -0.05 }, "3-10m": { "pairs": 600000, "failed_pct": 0.81, - "median_in": 10.32, - "p90_in": 44.76, - "scale_error_pct": -1.23 + "median_in": 10.12, + "p90_in": 44.21, + "scale_error_pct": -1.08 } }, "tape_calibration_success_pct": 99.5, @@ -486,33 +492,33 @@ "none": { "1-3m": { "pairs": 24000, - "failed_pct": 0.01, - "median_in": 5.55, - "p90_in": 10.92, - "scale_error_pct": -5.92 + "failed_pct": 0.0, + "median_in": 5.57, + "p90_in": 11.01, + "scale_error_pct": -5.93 }, "3-10m": { "pairs": 24000, "failed_pct": 0.01, - "median_in": 19.26, - "p90_in": 46.21, - "scale_error_pct": -7.07 + "median_in": 19.38, + "p90_in": 46.33, + "scale_error_pct": -7.1 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 0.51, - "median_in": 2.77, - "p90_in": 10.95, - "scale_error_pct": -0.42 + "failed_pct": 0.5, + "median_in": 2.73, + "p90_in": 10.66, + "scale_error_pct": -0.35 }, "3-10m": { "pairs": 600000, "failed_pct": 0.51, - "median_in": 9.69, - "p90_in": 43.92, - "scale_error_pct": -1.26 + "median_in": 9.55, + "p90_in": 43.23, + "scale_error_pct": -1.16 } }, "tape_calibration_success_pct": 99.5, @@ -539,16 +545,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 0.52, - "median_in": 4.93, - "p90_in": 28.19, - "scale_error_pct": 3.57 + "median_in": 4.91, + "p90_in": 28.18, + "scale_error_pct": 3.7 }, "3-10m": { "pairs": 600000, "failed_pct": 0.6, - "median_in": 14.01, - "p90_in": 55.42, - "scale_error_pct": 2.24 + "median_in": 14.03, + "p90_in": 55.43, + "scale_error_pct": 2.38 } }, "tape_calibration_success_pct": 99.5, @@ -577,16 +583,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 0.68, - "median_in": 3.04, - "p90_in": 12.62, - "scale_error_pct": -0.17 + "median_in": 3.0, + "p90_in": 12.53, + "scale_error_pct": -0.05 }, "3-10m": { "pairs": 600000, "failed_pct": 0.72, - "median_in": 9.82, - "p90_in": 43.65, - "scale_error_pct": -1.23 + "median_in": 9.64, + "p90_in": 43.29, + "scale_error_pct": -1.08 } }, "tape_calibration_success_pct": 99.5, @@ -597,32 +603,32 @@ "1-3m": { "pairs": 24000, "failed_pct": 0.0, - "median_in": 5.9, - "p90_in": 11.25, - "scale_error_pct": -6.47 + "median_in": 5.87, + "p90_in": 11.34, + "scale_error_pct": -6.46 }, "3-10m": { "pairs": 24000, - "failed_pct": 0.01, - "median_in": 19.57, - "p90_in": 46.74, - "scale_error_pct": -7.38 + "failed_pct": 0.0, + "median_in": 19.59, + "p90_in": 46.71, + "scale_error_pct": -7.44 } }, "one_known_distance": { "1-3m": { "pairs": 600000, "failed_pct": 0.5, - "median_in": 2.78, - "p90_in": 11.26, - "scale_error_pct": -0.44 + "median_in": 2.73, + "p90_in": 11.1, + "scale_error_pct": -0.32 }, "3-10m": { "pairs": 600000, - "failed_pct": 0.51, - "median_in": 9.21, - "p90_in": 42.66, - "scale_error_pct": -1.24 + "failed_pct": 0.5, + "median_in": 9.09, + "p90_in": 42.47, + "scale_error_pct": -1.16 } }, "tape_calibration_success_pct": 99.5, @@ -634,7 +640,7 @@ "pairs": 24000, "failed_pct": 0.0, "median_in": 5.51, - "p90_in": 26.11, + "p90_in": 26.1, "scale_error_pct": -2.26 }, "3-10m": { @@ -642,23 +648,23 @@ "failed_pct": 0.03, "median_in": 16.03, "p90_in": 48.48, - "scale_error_pct": -3.98 + "scale_error_pct": -3.99 } }, "one_known_distance": { "1-3m": { "pairs": 600000, "failed_pct": 0.5, - "median_in": 5.01, - "p90_in": 30.48, - "scale_error_pct": 3.92 + "median_in": 4.98, + "p90_in": 30.56, + "scale_error_pct": 4.01 }, "3-10m": { "pairs": 600000, "failed_pct": 0.53, - "median_in": 13.78, - "p90_in": 55.8, - "scale_error_pct": 2.18 + "median_in": 13.76, + "p90_in": 55.88, + "scale_error_pct": 2.29 } }, "tape_calibration_success_pct": 99.5, @@ -687,16 +693,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 0.65, - "median_in": 3.34, - "p90_in": 12.34, - "scale_error_pct": -0.32 + "median_in": 3.31, + "p90_in": 12.31, + "scale_error_pct": -0.19 }, "3-10m": { "pairs": 600000, "failed_pct": 0.68, - "median_in": 10.79, - "p90_in": 42.09, - "scale_error_pct": -1.62 + "median_in": 10.64, + "p90_in": 41.92, + "scale_error_pct": -1.5 } }, "tape_calibration_success_pct": 99.5, @@ -708,31 +714,31 @@ "pairs": 24000, "failed_pct": 0.0, "median_in": 5.61, - "p90_in": 10.92, - "scale_error_pct": -6.16 + "p90_in": 11.04, + "scale_error_pct": -6.17 }, "3-10m": { "pairs": 24000, - "failed_pct": 0.01, - "median_in": 18.28, - "p90_in": 46.46, - "scale_error_pct": -7.0 + "failed_pct": 0.0, + "median_in": 18.5, + "p90_in": 46.86, + "scale_error_pct": -7.09 } }, "one_known_distance": { "1-3m": { "pairs": 600000, "failed_pct": 0.5, - "median_in": 3.08, - "p90_in": 11.1, - "scale_error_pct": -0.62 + "median_in": 3.06, + "p90_in": 11.04, + "scale_error_pct": -0.5 }, "3-10m": { "pairs": 600000, - "failed_pct": 0.51, - "median_in": 10.29, - "p90_in": 40.99, - "scale_error_pct": -1.66 + "failed_pct": 0.5, + "median_in": 10.16, + "p90_in": 40.9, + "scale_error_pct": -1.62 } }, "tape_calibration_success_pct": 99.5, @@ -743,14 +749,14 @@ "1-3m": { "pairs": 24000, "failed_pct": 0.0, - "median_in": 5.09, - "p90_in": 24.59, + "median_in": 5.1, + "p90_in": 24.58, "scale_error_pct": -2.27 }, "3-10m": { "pairs": 24000, "failed_pct": 0.02, - "median_in": 14.79, + "median_in": 14.78, "p90_in": 45.42, "scale_error_pct": -3.87 } @@ -760,15 +766,15 @@ "pairs": 600000, "failed_pct": 0.5, "median_in": 5.18, - "p90_in": 28.49, - "scale_error_pct": 4.06 + "p90_in": 28.56, + "scale_error_pct": 4.17 }, "3-10m": { "pairs": 600000, "failed_pct": 0.52, - "median_in": 13.74, - "p90_in": 52.73, - "scale_error_pct": 2.02 + "median_in": 13.75, + "p90_in": 52.82, + "scale_error_pct": 2.14 } }, "tape_calibration_success_pct": 99.5, @@ -797,16 +803,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 2.13, - "median_in": 3.49, - "p90_in": 14.43, - "scale_error_pct": 0.01 + "median_in": 3.47, + "p90_in": 14.39, + "scale_error_pct": 0.13 }, "3-10m": { "pairs": 600000, "failed_pct": 2.15, - "median_in": 11.03, - "p90_in": 49.0, - "scale_error_pct": -1.36 + "median_in": 10.94, + "p90_in": 48.76, + "scale_error_pct": -1.22 } }, "tape_calibration_success_pct": 98.0, @@ -817,32 +823,32 @@ "1-3m": { "pairs": 24000, "failed_pct": 0.0, - "median_in": 5.86, - "p90_in": 11.24, - "scale_error_pct": -6.32 + "median_in": 5.84, + "p90_in": 11.27, + "scale_error_pct": -6.31 }, "3-10m": { "pairs": 24000, - "failed_pct": 0.01, - "median_in": 19.06, + "failed_pct": 0.0, + "median_in": 19.23, "p90_in": 46.46, - "scale_error_pct": -7.3 + "scale_error_pct": -7.36 } }, "one_known_distance": { "1-3m": { "pairs": 600000, "failed_pct": 2.0, - "median_in": 3.21, - "p90_in": 13.02, - "scale_error_pct": -0.31 + "median_in": 3.2, + "p90_in": 12.97, + "scale_error_pct": -0.21 }, "3-10m": { "pairs": 600000, - "failed_pct": 2.01, - "median_in": 10.57, - "p90_in": 48.53, - "scale_error_pct": -1.39 + "failed_pct": 2.0, + "median_in": 10.48, + "p90_in": 48.11, + "scale_error_pct": -1.34 } }, "tape_calibration_success_pct": 98.0, @@ -869,16 +875,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 2.0, - "median_in": 5.67, - "p90_in": 35.99, - "scale_error_pct": 4.52 + "median_in": 5.69, + "p90_in": 36.07, + "scale_error_pct": 4.66 }, "3-10m": { "pairs": 600000, "failed_pct": 2.02, - "median_in": 14.67, - "p90_in": 65.25, - "scale_error_pct": 2.26 + "median_in": 14.76, + "p90_in": 65.21, + "scale_error_pct": 2.41 } }, "tape_calibration_success_pct": 98.0, @@ -909,16 +915,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 1.68, - "median_in": 3.65, - "p90_in": 13.91, - "scale_error_pct": -0.19 + "median_in": 3.52, + "p90_in": 13.66, + "scale_error_pct": -0.15 }, "3-10m": { "pairs": 600000, "failed_pct": 2.03, - "median_in": 12.09, - "p90_in": 54.69, - "scale_error_pct": -1.24 + "median_in": 11.58, + "p90_in": 54.75, + "scale_error_pct": -1.15 } }, "tape_calibration_success_pct": 99.5, @@ -928,33 +934,33 @@ "none": { "1-3m": { "pairs": 24000, - "failed_pct": 0.17, - "median_in": 11.68, - "p90_in": 20.69, - "scale_error_pct": -15.35 + "failed_pct": 0.12, + "median_in": 11.7, + "p90_in": 20.58, + "scale_error_pct": -15.33 }, "3-10m": { "pairs": 24000, - "failed_pct": 0.12, - "median_in": 39.25, - "p90_in": 79.31, + "failed_pct": 0.15, + "median_in": 39.45, + "p90_in": 79.26, "scale_error_pct": -17.01 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 0.67, - "median_in": 3.44, - "p90_in": 12.46, - "scale_error_pct": -0.47 + "failed_pct": 0.62, + "median_in": 3.29, + "p90_in": 12.03, + "scale_error_pct": -0.43 }, "3-10m": { "pairs": 600000, - "failed_pct": 0.62, - "median_in": 11.26, - "p90_in": 51.19, - "scale_error_pct": -1.49 + "failed_pct": 0.64, + "median_in": 10.71, + "p90_in": 51.14, + "scale_error_pct": -1.44 } }, "tape_calibration_success_pct": 99.5, @@ -967,13 +973,13 @@ "failed_pct": 0.72, "median_in": 9.8, "p90_in": 29.18, - "scale_error_pct": -8.51 + "scale_error_pct": -8.52 }, "3-10m": { "pairs": 24000, "failed_pct": 1.06, - "median_in": 30.43, - "p90_in": 71.9, + "median_in": 30.44, + "p90_in": 71.89, "scale_error_pct": -11.64 } }, @@ -981,16 +987,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 1.22, - "median_in": 6.82, - "p90_in": 42.7, - "scale_error_pct": 6.07 + "median_in": 6.66, + "p90_in": 42.73, + "scale_error_pct": 6.08 }, "3-10m": { "pairs": 600000, "failed_pct": 1.55, - "median_in": 19.53, - "p90_in": 81.99, - "scale_error_pct": 3.33 + "median_in": 19.12, + "p90_in": 81.81, + "scale_error_pct": 3.42 } }, "tape_calibration_success_pct": 99.5, @@ -1019,16 +1025,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 1.38, - "median_in": 4.01, - "p90_in": 16.92, - "scale_error_pct": 0.01 + "median_in": 3.91, + "p90_in": 16.88, + "scale_error_pct": 0.06 }, "3-10m": { "pairs": 600000, "failed_pct": 1.71, - "median_in": 13.29, - "p90_in": 59.16, - "scale_error_pct": -1.15 + "median_in": 12.85, + "p90_in": 59.32, + "scale_error_pct": -1.06 } }, "tape_calibration_success_pct": 99.5, @@ -1038,33 +1044,33 @@ "none": { "1-3m": { "pairs": 24000, - "failed_pct": 0.09, + "failed_pct": 0.05, "median_in": 11.56, - "p90_in": 21.07, - "scale_error_pct": -14.87 + "p90_in": 21.0, + "scale_error_pct": -14.84 }, "3-10m": { "pairs": 24000, - "failed_pct": 0.05, - "median_in": 38.41, - "p90_in": 81.04, - "scale_error_pct": -16.61 + "failed_pct": 0.06, + "median_in": 38.57, + "p90_in": 80.95, + "scale_error_pct": -16.63 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 0.59, - "median_in": 3.74, - "p90_in": 15.35, - "scale_error_pct": -0.3 + "failed_pct": 0.55, + "median_in": 3.62, + "p90_in": 15.3, + "scale_error_pct": -0.25 }, "3-10m": { "pairs": 600000, - "failed_pct": 0.55, - "median_in": 12.49, - "p90_in": 56.14, - "scale_error_pct": -1.46 + "failed_pct": 0.56, + "median_in": 12.03, + "p90_in": 56.33, + "scale_error_pct": -1.42 } }, "tape_calibration_success_pct": 99.5, @@ -1082,7 +1088,7 @@ "3-10m": { "pairs": 24000, "failed_pct": 0.77, - "median_in": 29.43, + "median_in": 29.42, "p90_in": 70.76, "scale_error_pct": -10.78 } @@ -1091,16 +1097,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 0.9, - "median_in": 7.96, - "p90_in": 45.5, - "scale_error_pct": 7.93 + "median_in": 7.84, + "p90_in": 45.53, + "scale_error_pct": 7.92 }, "3-10m": { "pairs": 600000, "failed_pct": 1.26, - "median_in": 21.85, - "p90_in": 87.66, - "scale_error_pct": 4.64 + "median_in": 21.61, + "p90_in": 87.51, + "scale_error_pct": 4.72 } }, "tape_calibration_success_pct": 99.5, @@ -1129,16 +1135,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 2.05, - "median_in": 4.11, - "p90_in": 15.78, - "scale_error_pct": -0.15 + "median_in": 4.01, + "p90_in": 15.65, + "scale_error_pct": -0.14 }, "3-10m": { "pairs": 600000, "failed_pct": 2.28, - "median_in": 13.46, - "p90_in": 55.46, - "scale_error_pct": -1.34 + "median_in": 13.11, + "p90_in": 55.54, + "scale_error_pct": -1.28 } }, "tape_calibration_success_pct": 98.5, @@ -1148,33 +1154,33 @@ "none": { "1-3m": { "pairs": 24000, - "failed_pct": 0.04, - "median_in": 11.2, - "p90_in": 20.44, - "scale_error_pct": -14.6 + "failed_pct": 0.01, + "median_in": 11.24, + "p90_in": 20.54, + "scale_error_pct": -14.58 }, "3-10m": { "pairs": 24000, "failed_pct": 0.02, - "median_in": 37.09, - "p90_in": 77.66, - "scale_error_pct": -16.18 + "median_in": 37.21, + "p90_in": 77.43, + "scale_error_pct": -16.23 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 1.53, - "median_in": 3.87, - "p90_in": 14.38, - "scale_error_pct": -0.47 + "failed_pct": 1.51, + "median_in": 3.79, + "p90_in": 14.28, + "scale_error_pct": -0.52 }, "3-10m": { "pairs": 600000, "failed_pct": 1.52, - "median_in": 12.88, - "p90_in": 53.93, - "scale_error_pct": -1.77 + "median_in": 12.51, + "p90_in": 54.18, + "scale_error_pct": -1.74 } }, "tape_calibration_success_pct": 98.5, @@ -1201,16 +1207,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 1.71, - "median_in": 7.59, - "p90_in": 44.5, - "scale_error_pct": 7.08 + "median_in": 7.45, + "p90_in": 44.43, + "scale_error_pct": 7.03 }, "3-10m": { "pairs": 600000, "failed_pct": 1.91, - "median_in": 20.62, - "p90_in": 83.55, - "scale_error_pct": 4.37 + "median_in": 20.43, + "p90_in": 83.16, + "scale_error_pct": 4.41 } }, "tape_calibration_success_pct": 98.5, @@ -1239,16 +1245,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 1.93, - "median_in": 4.22, - "p90_in": 15.45, - "scale_error_pct": -0.14 + "median_in": 4.15, + "p90_in": 15.28, + "scale_error_pct": -0.09 }, "3-10m": { "pairs": 600000, "failed_pct": 2.14, - "median_in": 13.68, - "p90_in": 55.96, - "scale_error_pct": -1.34 + "median_in": 13.41, + "p90_in": 55.83, + "scale_error_pct": -1.25 } }, "tape_calibration_success_pct": 98.5, @@ -1258,33 +1264,33 @@ "none": { "1-3m": { "pairs": 24000, - "failed_pct": 0.04, - "median_in": 10.75, - "p90_in": 19.68, - "scale_error_pct": -13.78 + "failed_pct": 0.01, + "median_in": 10.86, + "p90_in": 19.8, + "scale_error_pct": -13.87 }, "3-10m": { "pairs": 24000, "failed_pct": 0.02, - "median_in": 36.22, - "p90_in": 77.08, - "scale_error_pct": -15.58 + "median_in": 36.45, + "p90_in": 77.06, + "scale_error_pct": -15.68 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 1.54, - "median_in": 4.03, - "p90_in": 14.25, - "scale_error_pct": -0.43 + "failed_pct": 1.51, + "median_in": 3.98, + "p90_in": 14.1, + "scale_error_pct": -0.41 }, "3-10m": { "pairs": 600000, "failed_pct": 1.52, - "median_in": 13.12, - "p90_in": 54.83, - "scale_error_pct": -1.75 + "median_in": 12.82, + "p90_in": 54.59, + "scale_error_pct": -1.72 } }, "tape_calibration_success_pct": 98.5, @@ -1295,14 +1301,14 @@ "1-3m": { "pairs": 24000, "failed_pct": 0.18, - "median_in": 9.29, + "median_in": 9.3, "p90_in": 28.54, - "scale_error_pct": -8.4 + "scale_error_pct": -8.41 }, "3-10m": { "pairs": 24000, "failed_pct": 0.36, - "median_in": 28.17, + "median_in": 28.18, "p90_in": 69.57, "scale_error_pct": -10.31 } @@ -1311,16 +1317,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 1.68, - "median_in": 7.4, - "p90_in": 43.7, - "scale_error_pct": 6.24 + "median_in": 7.29, + "p90_in": 43.68, + "scale_error_pct": 6.33 }, "3-10m": { "pairs": 600000, "failed_pct": 1.84, - "median_in": 19.71, - "p90_in": 78.52, - "scale_error_pct": 3.57 + "median_in": 19.57, + "p90_in": 78.2, + "scale_error_pct": 3.73 } }, "tape_calibration_success_pct": 98.5, @@ -1374,14 +1380,14 @@ "pairs": 3000, "failed_pct": 0.0, "median_in": 0.04, - "p90_in": 0.37, + "p90_in": 0.35, "scale_error_pct": -0.04 }, "3-10m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 0.08, - "p90_in": 0.56, + "median_in": 0.07, + "p90_in": 0.5, "scale_error_pct": -0.05 } }, @@ -1390,14 +1396,14 @@ "pairs": 75000, "failed_pct": 0.0, "median_in": 0.53, - "p90_in": 2.05, + "p90_in": 2.06, "scale_error_pct": 0.42 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, "median_in": 0.97, - "p90_in": 3.95, + "p90_in": 3.97, "scale_error_pct": 0.4 } }, @@ -1484,30 +1490,30 @@ "pairs": 3000, "failed_pct": 0.0, "median_in": 0.04, - "p90_in": 0.38, - "scale_error_pct": -0.04 + "p90_in": 0.36, + "scale_error_pct": -0.05 }, "3-10m": { "pairs": 3000, "failed_pct": 0.0, "median_in": 0.08, - "p90_in": 0.57, - "scale_error_pct": -0.06 + "p90_in": 0.51, + "scale_error_pct": -0.05 } }, "one_known_distance": { "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 0.54, - "p90_in": 2.13, + "median_in": 0.53, + "p90_in": 2.14, "scale_error_pct": 0.4 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, "median_in": 0.99, - "p90_in": 3.98, + "p90_in": 4.0, "scale_error_pct": 0.38 } }, @@ -1601,7 +1607,7 @@ "pairs": 3000, "failed_pct": 0.0, "median_in": 0.1, - "p90_in": 0.61, + "p90_in": 0.55, "scale_error_pct": -0.07 } }, @@ -1617,8 +1623,8 @@ "pairs": 75000, "failed_pct": 0.0, "median_in": 1.04, - "p90_in": 4.06, - "scale_error_pct": 0.29 + "p90_in": 4.07, + "scale_error_pct": 0.3 } }, "tape_calibration_success_pct": 100.0, @@ -1704,31 +1710,31 @@ "pairs": 3000, "failed_pct": 0.0, "median_in": 0.05, - "p90_in": 0.31, + "p90_in": 0.32, "scale_error_pct": -0.06 }, "3-10m": { "pairs": 3000, "failed_pct": 0.0, "median_in": 0.11, - "p90_in": 0.6, - "scale_error_pct": -0.08 + "p90_in": 0.55, + "scale_error_pct": -0.07 } }, "one_known_distance": { "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 0.44, - "p90_in": 2.07, + "median_in": 0.45, + "p90_in": 2.09, "scale_error_pct": 0.2 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 0.94, + "median_in": 0.95, "p90_in": 4.11, - "scale_error_pct": 0.22 + "scale_error_pct": 0.23 } }, "tape_calibration_success_pct": 100.0, @@ -1815,32 +1821,32 @@ "1-3m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 16.31, - "p90_in": 25.65, - "scale_error_pct": 21.36 + "median_in": 16.18, + "p90_in": 25.34, + "scale_error_pct": 20.98 }, "3-10m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 26.63, - "p90_in": 32.89, - "scale_error_pct": 17.96 + "median_in": 26.2, + "p90_in": 32.84, + "scale_error_pct": 17.69 } }, "one_known_distance": { "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 3.34, - "p90_in": 8.75, - "scale_error_pct": 2.52 + "median_in": 3.25, + "p90_in": 8.55, + "scale_error_pct": 2.27 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 4.73, - "p90_in": 11.22, - "scale_error_pct": -0.52 + "median_in": 4.69, + "p90_in": 11.28, + "scale_error_pct": -0.69 } }, "tape_calibration_success_pct": 100.0, @@ -1925,32 +1931,32 @@ "1-3m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 15.83, - "p90_in": 24.19, - "scale_error_pct": 20.71 + "median_in": 15.67, + "p90_in": 23.97, + "scale_error_pct": 20.35 }, "3-10m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 26.12, - "p90_in": 32.33, - "scale_error_pct": 17.68 + "median_in": 25.79, + "p90_in": 32.2, + "scale_error_pct": 17.45 } }, "one_known_distance": { "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 3.36, - "p90_in": 8.69, - "scale_error_pct": 2.22 + "median_in": 3.25, + "p90_in": 8.45, + "scale_error_pct": 1.97 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 4.87, - "p90_in": 12.11, - "scale_error_pct": -0.56 + "median_in": 4.83, + "p90_in": 12.18, + "scale_error_pct": -0.73 } }, "tape_calibration_success_pct": 100.0, @@ -2035,32 +2041,32 @@ "1-3m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 15.81, - "p90_in": 23.32, - "scale_error_pct": 20.41 + "median_in": 15.54, + "p90_in": 23.18, + "scale_error_pct": 20.18 }, "3-10m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 25.41, - "p90_in": 31.08, - "scale_error_pct": 17.29 + "median_in": 25.24, + "p90_in": 30.97, + "scale_error_pct": 17.04 } }, "one_known_distance": { "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 3.43, - "p90_in": 8.76, - "scale_error_pct": 2.07 + "median_in": 3.34, + "p90_in": 8.63, + "scale_error_pct": 1.87 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 5.01, - "p90_in": 13.12, - "scale_error_pct": -0.71 + "median_in": 4.96, + "p90_in": 13.29, + "scale_error_pct": -0.86 } }, "tape_calibration_success_pct": 100.0, @@ -2145,32 +2151,32 @@ "1-3m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 15.36, - "p90_in": 22.55, - "scale_error_pct": 20.32 + "median_in": 15.24, + "p90_in": 22.54, + "scale_error_pct": 20.21 }, "3-10m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 25.26, - "p90_in": 30.98, - "scale_error_pct": 17.26 + "median_in": 25.15, + "p90_in": 30.54, + "scale_error_pct": 17.07 } }, "one_known_distance": { "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 5.05, - "p90_in": 13.49, - "scale_error_pct": 2.76 + "median_in": 4.91, + "p90_in": 13.46, + "scale_error_pct": 2.51 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 6.63, - "p90_in": 28.42, - "scale_error_pct": -0.81 + "median_in": 6.57, + "p90_in": 28.83, + "scale_error_pct": -1.01 } }, "tape_calibration_success_pct": 100.0, @@ -2258,31 +2264,31 @@ "pairs": 3000, "failed_pct": 0.0, "median_in": 1.48, - "p90_in": 4.51, - "scale_error_pct": 0.29 + "p90_in": 4.31, + "scale_error_pct": 0.15 }, "3-10m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 1.79, - "p90_in": 4.6, - "scale_error_pct": -0.92 + "median_in": 1.86, + "p90_in": 4.69, + "scale_error_pct": -1.03 } }, "one_known_distance": { "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 2.78, - "p90_in": 8.09, - "scale_error_pct": -0.57 + "median_in": 2.75, + "p90_in": 7.98, + "scale_error_pct": -0.71 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 4.88, - "p90_in": 14.6, - "scale_error_pct": -1.66 + "median_in": 4.87, + "p90_in": 14.47, + "scale_error_pct": -1.75 } }, "tape_calibration_success_pct": 100.0, @@ -2367,32 +2373,32 @@ "1-3m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 2.25, - "p90_in": 6.69, - "scale_error_pct": 1.09 + "median_in": 2.15, + "p90_in": 6.35, + "scale_error_pct": 0.67 }, "3-10m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 2.53, - "p90_in": 7.33, - "scale_error_pct": -1.27 + "median_in": 2.71, + "p90_in": 7.49, + "scale_error_pct": -1.39 } }, "one_known_distance": { "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 4.07, - "p90_in": 11.09, - "scale_error_pct": 0.48 + "median_in": 4.03, + "p90_in": 10.94, + "scale_error_pct": 0.24 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 6.34, - "p90_in": 20.33, - "scale_error_pct": -1.73 + "median_in": 6.37, + "p90_in": 20.46, + "scale_error_pct": -1.87 } }, "tape_calibration_success_pct": 100.0, @@ -2477,32 +2483,32 @@ "1-3m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 2.47, - "p90_in": 7.0, - "scale_error_pct": 1.62 + "median_in": 2.26, + "p90_in": 6.89, + "scale_error_pct": 1.23 }, "3-10m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 2.41, - "p90_in": 6.96, - "scale_error_pct": -0.79 + "median_in": 2.46, + "p90_in": 7.0, + "scale_error_pct": -0.99 } }, "one_known_distance": { "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 4.31, - "p90_in": 11.76, - "scale_error_pct": 0.83 + "median_in": 4.26, + "p90_in": 11.62, + "scale_error_pct": 0.66 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 6.63, - "p90_in": 21.33, - "scale_error_pct": -1.4 + "median_in": 6.62, + "p90_in": 21.39, + "scale_error_pct": -1.55 } }, "tape_calibration_success_pct": 100.0, @@ -2587,32 +2593,32 @@ "1-3m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 2.44, - "p90_in": 5.66, - "scale_error_pct": -0.64 + "median_in": 2.47, + "p90_in": 5.56, + "scale_error_pct": -0.94 }, "3-10m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 3.99, - "p90_in": 7.93, - "scale_error_pct": -2.58 + "median_in": 4.25, + "p90_in": 8.17, + "scale_error_pct": -2.75 } }, "one_known_distance": { "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 3.74, - "p90_in": 11.19, - "scale_error_pct": 1.02 + "median_in": 3.75, + "p90_in": 11.18, + "scale_error_pct": 0.9 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, "median_in": 5.33, - "p90_in": 24.32, - "scale_error_pct": -0.54 + "p90_in": 24.56, + "scale_error_pct": -0.66 } }, "tape_calibration_success_pct": 100.0, @@ -2682,16 +2688,16 @@ "1-3m": { "pairs": 5700000, "failed_pct": 0.46, - "median_in": 2.92, - "p90_in": 12.33, - "scale_error_pct": 0.03 + "median_in": 2.91, + "p90_in": 12.22, + "scale_error_pct": 0.05 }, "3-10m": { "pairs": 5700000, "failed_pct": 0.57, - "median_in": 9.21, - "p90_in": 45.35, - "scale_error_pct": -0.75 + "median_in": 9.16, + "p90_in": 45.06, + "scale_error_pct": -0.74 } }, "tape_calibration_success_pct": 99.7, @@ -2702,32 +2708,32 @@ "1-3m": { "pairs": 228000, "failed_pct": 0.0, - "median_in": 6.32, - "p90_in": 12.89, - "scale_error_pct": -7.0 + "median_in": 6.31, + "p90_in": 12.9, + "scale_error_pct": -6.98 }, "3-10m": { "pairs": 228000, - "failed_pct": 0.01, - "median_in": 21.57, - "p90_in": 54.08, - "scale_error_pct": -8.14 + "failed_pct": 0.0, + "median_in": 21.62, + "p90_in": 54.0, + "scale_error_pct": -8.17 } }, "one_known_distance": { "1-3m": { "pairs": 5700000, "failed_pct": 0.32, - "median_in": 2.75, - "p90_in": 11.29, - "scale_error_pct": -0.19 + "median_in": 2.72, + "p90_in": 11.16, + "scale_error_pct": -0.18 }, "3-10m": { "pairs": 5700000, "failed_pct": 0.32, - "median_in": 8.93, - "p90_in": 46.11, - "scale_error_pct": -0.86 + "median_in": 8.92, + "p90_in": 45.92, + "scale_error_pct": -0.88 } }, "tape_calibration_success_pct": 99.7, @@ -2739,31 +2745,31 @@ "pairs": 228000, "failed_pct": 0.04, "median_in": 5.65, - "p90_in": 20.82, + "p90_in": 20.83, "scale_error_pct": -2.87 }, "3-10m": { "pairs": 225783, "failed_pct": 0.08, - "median_in": 15.84, - "p90_in": 48.4, - "scale_error_pct": -4.67 + "median_in": 15.83, + "p90_in": 48.41, + "scale_error_pct": -4.68 } }, "one_known_distance": { "1-3m": { "pairs": 5700000, "failed_pct": 0.35, - "median_in": 4.43, - "p90_in": 23.78, - "scale_error_pct": 3.3 + "median_in": 4.45, + "p90_in": 23.8, + "scale_error_pct": 3.36 }, "3-10m": { "pairs": 5644575, "failed_pct": 0.4, - "median_in": 11.67, - "p90_in": 50.67, - "scale_error_pct": 1.82 + "median_in": 11.72, + "p90_in": 50.53, + "scale_error_pct": 1.88 } }, "tape_calibration_success_pct": 99.7, @@ -2793,15 +2799,15 @@ "pairs": 2100000, "failed_pct": 0.0, "median_in": 0.94, - "p90_in": 6.49, - "scale_error_pct": 0.05 + "p90_in": 6.45, + "scale_error_pct": 0.03 }, "3-10m": { "pairs": 1875000, "failed_pct": 0.0, - "median_in": 2.2, - "p90_in": 12.12, - "scale_error_pct": -0.09 + "median_in": 2.22, + "p90_in": 12.3, + "scale_error_pct": -0.11 } }, "tape_calibration_success_pct": 100.0, @@ -2812,32 +2818,32 @@ "1-3m": { "pairs": 36000, "failed_pct": 0.0, - "median_in": 14.9, - "p90_in": 25.37, - "scale_error_pct": 20.89 + "median_in": 14.85, + "p90_in": 25.34, + "scale_error_pct": 20.68 }, "3-10m": { "pairs": 33000, "failed_pct": 0.0, - "median_in": 26.08, - "p90_in": 37.03, - "scale_error_pct": 17.5 + "median_in": 25.93, + "p90_in": 37.2, + "scale_error_pct": 17.39 } }, "one_known_distance": { "1-3m": { "pairs": 900000, "failed_pct": 0.0, - "median_in": 2.7, - "p90_in": 8.24, - "scale_error_pct": 1.66 + "median_in": 2.69, + "p90_in": 8.18, + "scale_error_pct": 1.41 }, "3-10m": { "pairs": 825000, "failed_pct": 0.0, - "median_in": 4.77, - "p90_in": 13.04, - "scale_error_pct": -1.2 + "median_in": 4.76, + "p90_in": 13.2, + "scale_error_pct": -1.44 } }, "tape_calibration_success_pct": 100.0, @@ -2848,15 +2854,15 @@ "1-3m": { "pairs": 36000, "failed_pct": 0.0, - "median_in": 11.18, - "p90_in": 24.32, - "scale_error_pct": 19.4 + "median_in": 11.19, + "p90_in": 24.17, + "scale_error_pct": 19.41 }, "3-10m": { "pairs": 33000, "failed_pct": 0.0, "median_in": 30.19, - "p90_in": 43.72, + "p90_in": 43.74, "scale_error_pct": 16.66 } }, @@ -2864,16 +2870,16 @@ "1-3m": { "pairs": 900000, "failed_pct": 0.0, - "median_in": 2.95, - "p90_in": 9.28, - "scale_error_pct": 0.54 + "median_in": 2.91, + "p90_in": 9.1, + "scale_error_pct": 0.35 }, "3-10m": { "pairs": 825000, "failed_pct": 0.0, - "median_in": 5.78, - "p90_in": 17.59, - "scale_error_pct": -1.33 + "median_in": 5.88, + "p90_in": 18.07, + "scale_error_pct": -1.53 } }, "tape_calibration_success_pct": 100.0, @@ -2903,16 +2909,16 @@ "one_known_distance": { "1-3m": { "pairs": 5700000, - "failed_pct": 1.13, - "median_in": 3.79, - "p90_in": 14.76, + "failed_pct": 1.08, + "median_in": 3.76, + "p90_in": 14.61, "scale_error_pct": 0.03 }, "3-10m": { "pairs": 5700000, - "failed_pct": 1.48, - "median_in": 12.02, - "p90_in": 56.52, + "failed_pct": 1.43, + "median_in": 11.92, + "p90_in": 56.12, "scale_error_pct": -0.7 } }, @@ -2925,31 +2931,31 @@ "pairs": 228000, "failed_pct": 0.09, "median_in": 10.83, - "p90_in": 19.86, - "scale_error_pct": -14.11 + "p90_in": 19.8, + "scale_error_pct": -14.12 }, "3-10m": { "pairs": 228000, "failed_pct": 0.1, - "median_in": 37.01, - "p90_in": 78.1, - "scale_error_pct": -15.51 + "median_in": 37.1, + "p90_in": 78.11, + "scale_error_pct": -15.55 } }, "one_known_distance": { "1-3m": { "pairs": 5700000, - "failed_pct": 0.62, - "median_in": 3.58, - "p90_in": 13.66, - "scale_error_pct": -0.16 + "failed_pct": 0.56, + "median_in": 3.53, + "p90_in": 13.47, + "scale_error_pct": -0.17 }, "3-10m": { "pairs": 5700000, - "failed_pct": 0.62, - "median_in": 11.51, - "p90_in": 56.8, - "scale_error_pct": -0.91 + "failed_pct": 0.57, + "median_in": 11.45, + "p90_in": 56.49, + "scale_error_pct": -0.92 } }, "tape_calibration_success_pct": 99.5, @@ -2961,30 +2967,30 @@ "pairs": 228000, "failed_pct": 1.03, "median_in": 9.53, - "p90_in": 27.72, - "scale_error_pct": -8.89 + "p90_in": 27.73, + "scale_error_pct": -8.9 }, "3-10m": { "pairs": 225783, - "failed_pct": 1.26, + "failed_pct": 1.27, "median_in": 26.35, - "p90_in": 68.81, + "p90_in": 68.84, "scale_error_pct": -10.38 } }, "one_known_distance": { "1-3m": { "pairs": 5700000, - "failed_pct": 1.55, - "median_in": 6.9, - "p90_in": 41.46, - "scale_error_pct": 5.68 + "failed_pct": 1.5, + "median_in": 6.88, + "p90_in": 41.35, + "scale_error_pct": 5.69 }, "3-10m": { "pairs": 5644575, - "failed_pct": 1.79, - "median_in": 17.47, - "p90_in": 77.24, + "failed_pct": 1.74, + "median_in": 17.4, + "p90_in": 76.66, "scale_error_pct": 3.9 } }, @@ -3015,15 +3021,15 @@ "pairs": 2100000, "failed_pct": 0.0, "median_in": 2.13, - "p90_in": 10.88, - "scale_error_pct": 0.06 + "p90_in": 10.86, + "scale_error_pct": 0.04 }, "3-10m": { "pairs": 1875000, "failed_pct": 0.0, - "median_in": 4.53, - "p90_in": 19.67, - "scale_error_pct": -1.52 + "median_in": 4.56, + "p90_in": 19.93, + "scale_error_pct": -1.58 } }, "tape_calibration_success_pct": 100.0, @@ -3034,32 +3040,32 @@ "1-3m": { "pairs": 36000, "failed_pct": 0.0, - "median_in": 3.46, - "p90_in": 9.31, - "scale_error_pct": 0.2 + "median_in": 3.49, + "p90_in": 9.26, + "scale_error_pct": 0.04 }, "3-10m": { "pairs": 33000, "failed_pct": 0.0, - "median_in": 4.82, - "p90_in": 12.97, - "scale_error_pct": -2.47 + "median_in": 5.16, + "p90_in": 12.91, + "scale_error_pct": -2.71 } }, "one_known_distance": { "1-3m": { "pairs": 900000, "failed_pct": 0.0, - "median_in": 4.11, - "p90_in": 11.85, - "scale_error_pct": 2.02 + "median_in": 4.1, + "p90_in": 11.83, + "scale_error_pct": 1.7 }, "3-10m": { "pairs": 825000, "failed_pct": 0.0, - "median_in": 6.62, - "p90_in": 18.67, - "scale_error_pct": -1.9 + "median_in": 6.7, + "p90_in": 19.07, + "scale_error_pct": -2.17 } }, "tape_calibration_success_pct": 100.0, @@ -3070,15 +3076,15 @@ "1-3m": { "pairs": 36000, "failed_pct": 0.0, - "median_in": 3.82, - "p90_in": 39.43, - "scale_error_pct": 1.22 + "median_in": 3.81, + "p90_in": 39.07, + "scale_error_pct": 1.21 }, "3-10m": { "pairs": 33000, "failed_pct": 0.0, "median_in": 7.96, - "p90_in": 19.69, + "p90_in": 19.77, "scale_error_pct": -2.91 } }, @@ -3086,16 +3092,16 @@ "1-3m": { "pairs": 900000, "failed_pct": 0.0, - "median_in": 4.57, - "p90_in": 38.42, - "scale_error_pct": 3.13 + "median_in": 4.52, + "p90_in": 38.08, + "scale_error_pct": 2.89 }, "3-10m": { "pairs": 825000, "failed_pct": 0.0, - "median_in": 8.8, - "p90_in": 29.05, - "scale_error_pct": -1.24 + "median_in": 8.93, + "p90_in": 29.72, + "scale_error_pct": -1.4 } }, "tape_calibration_success_pct": 100.0, @@ -3106,6 +3112,12 @@ } }, "electro": { + "vertical_cohort": { + "vertical_points": 813742, + "vertical_points_with_camera_up": 815147, + "points_changing_cohort": 5291, + "up_change_deg": 5.4 + }, "groups": { "all points": { "oracle": { @@ -3113,330 +3125,330 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.0, - "median_in": 0.67, - "p90_in": 2.35, - "scale_error_pct": -0.61 + "median_in": 0.59, + "p90_in": 2.05, + "scale_error_pct": -0.56 }, "3-10m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.0, - "median_in": 1.4, - "p90_in": 5.24, - "scale_error_pct": -0.58 + "median_in": 1.24, + "p90_in": 4.52, + "scale_error_pct": -0.55 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.0, - "median_in": 0.74, - "p90_in": 2.82, - "scale_error_pct": -0.02 + "median_in": 0.66, + "p90_in": 2.27, + "scale_error_pct": -0.0 }, "3-10m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.0, - "median_in": 1.63, - "p90_in": 6.59, - "scale_error_pct": -0.02 + "median_in": 1.41, + "p90_in": 5.09, + "scale_error_pct": -0.0 } }, "tape_calibration_success_pct": 100.0, - "groups": 8 + "groups": 7 }, "vertical interior": { "none": { "1-3m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.0, - "median_in": 0.28, - "p90_in": 1.44, - "scale_error_pct": -0.31 + "median_in": 0.23, + "p90_in": 1.0, + "scale_error_pct": -0.27 }, "3-10m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.0, - "median_in": 0.8, - "p90_in": 4.54, - "scale_error_pct": -0.37 + "median_in": 0.68, + "p90_in": 3.15, + "scale_error_pct": -0.33 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.0, - "median_in": 0.53, - "p90_in": 2.46, - "scale_error_pct": 0.11 + "median_in": 0.47, + "p90_in": 2.02, + "scale_error_pct": 0.17 }, "3-10m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.0, - "median_in": 1.33, - "p90_in": 6.19, - "scale_error_pct": 0.08 + "median_in": 1.15, + "p90_in": 4.92, + "scale_error_pct": 0.13 } }, "tape_calibration_success_pct": 100.0, - "groups": 8 + "groups": 7 }, "edges": { "none": { "1-3m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.0, - "median_in": 0.39, - "p90_in": 1.54, - "scale_error_pct": -0.45 + "median_in": 0.4, + "p90_in": 1.56, + "scale_error_pct": -0.44 }, "3-10m": { - "pairs": 23816, + "pairs": 20816, "failed_pct": 0.0, - "median_in": 0.94, - "p90_in": 4.21, - "scale_error_pct": -0.58 + "median_in": 0.89, + "p90_in": 3.71, + "scale_error_pct": -0.56 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.0, - "median_in": 0.67, - "p90_in": 2.6, - "scale_error_pct": 0.06 + "median_in": 0.63, + "p90_in": 2.35, + "scale_error_pct": 0.07 }, "3-10m": { - "pairs": 595400, + "pairs": 520400, "failed_pct": 0.0, - "median_in": 1.5, - "p90_in": 5.56, - "scale_error_pct": -0.01 + "median_in": 1.34, + "p90_in": 4.57, + "scale_error_pct": 0.02 } }, "tape_calibration_success_pct": 100.0, - "groups": 8 + "groups": 7 } }, "2": { "surface interior": { "none": { "1-3m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.0, - "median_in": 0.68, - "p90_in": 2.28, - "scale_error_pct": -0.61 + "median_in": 0.6, + "p90_in": 2.03, + "scale_error_pct": -0.56 }, "3-10m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.0, - "median_in": 1.45, - "p90_in": 4.95, - "scale_error_pct": -0.58 + "median_in": 1.3, + "p90_in": 4.27, + "scale_error_pct": -0.56 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.0, - "median_in": 0.74, - "p90_in": 2.89, - "scale_error_pct": -0.01 + "median_in": 0.66, + "p90_in": 2.41, + "scale_error_pct": 0.01 }, "3-10m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.0, - "median_in": 1.61, - "p90_in": 6.64, - "scale_error_pct": -0.01 + "median_in": 1.39, + "p90_in": 5.28, + "scale_error_pct": 0.02 } }, "tape_calibration_success_pct": 100.0, - "groups": 8 + "groups": 7 }, "vertical interior": { "none": { "1-3m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.0, - "median_in": 0.3, - "p90_in": 1.23, - "scale_error_pct": -0.35 + "median_in": 0.25, + "p90_in": 0.93, + "scale_error_pct": -0.3 }, "3-10m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.0, - "median_in": 0.94, - "p90_in": 4.18, - "scale_error_pct": -0.44 + "median_in": 0.8, + "p90_in": 2.63, + "scale_error_pct": -0.38 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.0, - "median_in": 0.56, - "p90_in": 2.48, - "scale_error_pct": 0.14 + "median_in": 0.5, + "p90_in": 2.09, + "scale_error_pct": 0.23 }, "3-10m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.0, - "median_in": 1.43, - "p90_in": 6.27, - "scale_error_pct": 0.11 + "median_in": 1.25, + "p90_in": 5.16, + "scale_error_pct": 0.18 } }, "tape_calibration_success_pct": 100.0, - "groups": 8 + "groups": 7 }, "edges": { "none": { "1-3m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.0, - "median_in": 0.43, - "p90_in": 1.85, - "scale_error_pct": -0.5 + "median_in": 0.44, + "p90_in": 1.94, + "scale_error_pct": -0.51 }, "3-10m": { - "pairs": 23816, + "pairs": 20816, "failed_pct": 0.0, - "median_in": 1.04, - "p90_in": 3.88, - "scale_error_pct": -0.62 + "median_in": 1.0, + "p90_in": 2.94, + "scale_error_pct": -0.6 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.0, - "median_in": 0.72, - "p90_in": 2.73, - "scale_error_pct": 0.07 + "median_in": 0.69, + "p90_in": 2.58, + "scale_error_pct": 0.09 }, "3-10m": { - "pairs": 595400, + "pairs": 520400, "failed_pct": 0.0, - "median_in": 1.56, - "p90_in": 5.91, - "scale_error_pct": -0.0 + "median_in": 1.42, + "p90_in": 4.87, + "scale_error_pct": 0.04 } }, "tape_calibration_success_pct": 100.0, - "groups": 8 + "groups": 7 } }, "4": { "surface interior": { "none": { "1-3m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.0, - "median_in": 0.67, - "p90_in": 2.18, - "scale_error_pct": -0.59 + "median_in": 0.6, + "p90_in": 1.94, + "scale_error_pct": -0.55 }, "3-10m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.0, - "median_in": 1.45, - "p90_in": 4.9, - "scale_error_pct": -0.59 + "median_in": 1.28, + "p90_in": 4.25, + "scale_error_pct": -0.56 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.0, - "median_in": 0.74, - "p90_in": 2.82, - "scale_error_pct": -0.01 + "median_in": 0.67, + "p90_in": 2.39, + "scale_error_pct": 0.02 }, "3-10m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.0, - "median_in": 1.59, - "p90_in": 6.67, - "scale_error_pct": -0.02 + "median_in": 1.39, + "p90_in": 5.4, + "scale_error_pct": 0.0 } }, "tape_calibration_success_pct": 100.0, - "groups": 8 + "groups": 7 }, "vertical interior": { "none": { "1-3m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.0, - "median_in": 0.3, - "p90_in": 1.29, - "scale_error_pct": -0.35 + "median_in": 0.25, + "p90_in": 0.95, + "scale_error_pct": -0.3 }, "3-10m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.0, - "median_in": 0.86, - "p90_in": 4.44, - "scale_error_pct": -0.44 + "median_in": 0.73, + "p90_in": 2.97, + "scale_error_pct": -0.37 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.0, - "median_in": 0.61, - "p90_in": 2.41, - "scale_error_pct": 0.11 + "median_in": 0.55, + "p90_in": 2.09, + "scale_error_pct": 0.19 }, "3-10m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.0, - "median_in": 1.63, - "p90_in": 6.22, - "scale_error_pct": 0.08 + "median_in": 1.43, + "p90_in": 5.26, + "scale_error_pct": 0.15 } }, "tape_calibration_success_pct": 100.0, - "groups": 8 + "groups": 7 }, "edges": { "none": { "1-3m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.0, - "median_in": 0.41, - "p90_in": 1.83, - "scale_error_pct": -0.5 + "median_in": 0.44, + "p90_in": 1.93, + "scale_error_pct": -0.53 }, "3-10m": { - "pairs": 23816, + "pairs": 20816, "failed_pct": 0.0, - "median_in": 1.01, - "p90_in": 3.79, + "median_in": 0.98, + "p90_in": 3.13, "scale_error_pct": -0.61 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.0, - "median_in": 0.76, - "p90_in": 2.62, - "scale_error_pct": 0.06 + "median_in": 0.75, + "p90_in": 2.53, + "scale_error_pct": 0.07 }, "3-10m": { - "pairs": 595400, + "pairs": 520400, "failed_pct": 0.0, - "median_in": 1.69, - "p90_in": 5.82, - "scale_error_pct": -0.04 + "median_in": 1.57, + "p90_in": 4.89, + "scale_error_pct": 0.01 } }, "tape_calibration_success_pct": 100.0, - "groups": 8 + "groups": 7 } }, "8": { @@ -3462,15 +3474,15 @@ "pairs": 525000, "failed_pct": 0.0, "median_in": 0.71, - "p90_in": 2.36, - "scale_error_pct": 0.01 + "p90_in": 2.37, + "scale_error_pct": 0.02 }, "3-10m": { "pairs": 525000, "failed_pct": 0.0, - "median_in": 1.42, - "p90_in": 5.07, - "scale_error_pct": -0.03 + "median_in": 1.44, + "p90_in": 5.1, + "scale_error_pct": -0.02 } }, "tape_calibration_success_pct": 100.0, @@ -3481,16 +3493,16 @@ "1-3m": { "pairs": 21000, "failed_pct": 0.0, - "median_in": 0.3, + "median_in": 0.31, "p90_in": 1.03, - "scale_error_pct": -0.35 + "scale_error_pct": -0.36 }, "3-10m": { "pairs": 21000, "failed_pct": 0.0, "median_in": 0.85, - "p90_in": 3.09, - "scale_error_pct": -0.43 + "p90_in": 3.05, + "scale_error_pct": -0.42 } }, "one_known_distance": { @@ -3499,14 +3511,14 @@ "failed_pct": 0.0, "median_in": 0.59, "p90_in": 2.09, - "scale_error_pct": 0.2 + "scale_error_pct": 0.21 }, "3-10m": { "pairs": 525000, "failed_pct": 0.0, - "median_in": 1.51, - "p90_in": 5.21, - "scale_error_pct": 0.15 + "median_in": 1.53, + "p90_in": 5.22, + "scale_error_pct": 0.17 } }, "tape_calibration_success_pct": 100.0, @@ -3534,7 +3546,7 @@ "pairs": 525000, "failed_pct": 0.0, "median_in": 0.76, - "p90_in": 2.55, + "p90_in": 2.54, "scale_error_pct": 0.09 }, "3-10m": { @@ -3542,7 +3554,7 @@ "failed_pct": 0.0, "median_in": 1.52, "p90_in": 4.67, - "scale_error_pct": 0.01 + "scale_error_pct": 0.02 } }, "tape_calibration_success_pct": 100.0, @@ -3555,330 +3567,330 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.07, - "median_in": 7.63, - "p90_in": 17.94, - "scale_error_pct": 4.09 + "pairs": 21000, + "failed_pct": 0.03, + "median_in": 7.04, + "p90_in": 14.46, + "scale_error_pct": 4.92 }, "3-10m": { - "pairs": 24000, - "failed_pct": 0.12, - "median_in": 21.93, - "p90_in": 49.22, - "scale_error_pct": 5.08 + "pairs": 21000, + "failed_pct": 0.11, + "median_in": 20.44, + "p90_in": 34.39, + "scale_error_pct": 6.79 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 0.07, - "median_in": 2.87, - "p90_in": 13.23, - "scale_error_pct": -0.1 + "pairs": 525000, + "failed_pct": 0.03, + "median_in": 2.49, + "p90_in": 8.94, + "scale_error_pct": -0.12 }, "3-10m": { - "pairs": 600000, - "failed_pct": 0.12, - "median_in": 6.86, - "p90_in": 37.27, - "scale_error_pct": -0.08 + "pairs": 525000, + "failed_pct": 0.11, + "median_in": 5.82, + "p90_in": 22.48, + "scale_error_pct": 0.06 } }, "tape_calibration_success_pct": 100.0, - "groups": 8 + "groups": 7 }, "vertical interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.1, - "median_in": 7.28, - "p90_in": 17.88, - "scale_error_pct": 4.29 + "pairs": 21000, + "failed_pct": 0.08, + "median_in": 6.68, + "p90_in": 14.1, + "scale_error_pct": 5.21 }, "3-10m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.2, - "median_in": 22.76, - "p90_in": 56.19, - "scale_error_pct": 7.19 + "median_in": 20.74, + "p90_in": 35.52, + "scale_error_pct": 7.96 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 0.1, - "median_in": 2.54, - "p90_in": 12.37, - "scale_error_pct": 0.03 + "pairs": 525000, + "failed_pct": 0.08, + "median_in": 2.2, + "p90_in": 8.21, + "scale_error_pct": 0.04 }, "3-10m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.2, - "median_in": 7.08, - "p90_in": 38.08, - "scale_error_pct": 0.32 + "median_in": 5.9, + "p90_in": 22.76, + "scale_error_pct": 0.54 } }, "tape_calibration_success_pct": 100.0, - "groups": 8 + "groups": 7 }, "edges": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.31, - "median_in": 8.79, - "p90_in": 33.05, - "scale_error_pct": 6.58 + "pairs": 21000, + "failed_pct": 0.32, + "median_in": 7.84, + "p90_in": 19.46, + "scale_error_pct": 5.36 }, "3-10m": { - "pairs": 23816, - "failed_pct": 0.17, - "median_in": 19.01, - "p90_in": 54.82, - "scale_error_pct": 5.92 + "pairs": 20816, + "failed_pct": 0.19, + "median_in": 17.41, + "p90_in": 34.52, + "scale_error_pct": 6.21 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 0.31, - "median_in": 4.1, - "p90_in": 36.99, - "scale_error_pct": 0.28 + "pairs": 525000, + "failed_pct": 0.32, + "median_in": 3.48, + "p90_in": 15.01, + "scale_error_pct": -0.64 }, "3-10m": { - "pairs": 595400, - "failed_pct": 0.17, - "median_in": 6.96, - "p90_in": 45.46, - "scale_error_pct": -0.1 - } - }, - "tape_calibration_success_pct": 100.0, - "groups": 8 + "pairs": 520400, + "failed_pct": 0.19, + "median_in": 5.92, + "p90_in": 22.49, + "scale_error_pct": -0.28 + } + }, + "tape_calibration_success_pct": 100.0, + "groups": 7 } }, "2": { "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.06, - "median_in": 6.26, - "p90_in": 16.61, - "scale_error_pct": 2.19 + "pairs": 21000, + "failed_pct": 0.03, + "median_in": 5.6, + "p90_in": 14.03, + "scale_error_pct": 3.04 }, "3-10m": { - "pairs": 24000, - "failed_pct": 0.11, - "median_in": 17.46, - "p90_in": 42.18, - "scale_error_pct": 3.38 + "pairs": 21000, + "failed_pct": 0.1, + "median_in": 16.04, + "p90_in": 30.88, + "scale_error_pct": 4.4 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 0.06, - "median_in": 3.16, - "p90_in": 13.99, - "scale_error_pct": -0.16 + "pairs": 525000, + "failed_pct": 0.03, + "median_in": 2.75, + "p90_in": 10.07, + "scale_error_pct": -0.17 }, "3-10m": { - "pairs": 600000, - "failed_pct": 0.11, - "median_in": 7.0, - "p90_in": 37.37, - "scale_error_pct": -0.26 + "pairs": 525000, + "failed_pct": 0.1, + "median_in": 6.0, + "p90_in": 22.79, + "scale_error_pct": -0.08 } }, "tape_calibration_success_pct": 100.0, - "groups": 8 + "groups": 7 }, "vertical interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.08, - "median_in": 6.08, - "p90_in": 16.25, - "scale_error_pct": 1.74 + "pairs": 21000, + "failed_pct": 0.07, + "median_in": 5.38, + "p90_in": 13.56, + "scale_error_pct": 3.15 }, "3-10m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.16, - "median_in": 17.57, - "p90_in": 45.45, - "scale_error_pct": 4.73 + "median_in": 16.19, + "p90_in": 30.75, + "scale_error_pct": 5.66 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 0.08, - "median_in": 2.79, - "p90_in": 13.44, - "scale_error_pct": -0.3 + "pairs": 525000, + "failed_pct": 0.07, + "median_in": 2.42, + "p90_in": 9.45, + "scale_error_pct": -0.32 }, "3-10m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.16, - "median_in": 6.97, - "p90_in": 38.54, - "scale_error_pct": -0.03 + "median_in": 5.96, + "p90_in": 23.23, + "scale_error_pct": 0.2 } }, "tape_calibration_success_pct": 100.0, - "groups": 8 + "groups": 7 }, "edges": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.31, - "median_in": 7.64, - "p90_in": 30.56, - "scale_error_pct": 5.59 + "pairs": 21000, + "failed_pct": 0.32, + "median_in": 6.98, + "p90_in": 18.98, + "scale_error_pct": 4.59 }, "3-10m": { - "pairs": 23816, - "failed_pct": 0.17, - "median_in": 15.33, - "p90_in": 44.69, - "scale_error_pct": 4.31 + "pairs": 20816, + "failed_pct": 0.19, + "median_in": 13.98, + "p90_in": 27.74, + "scale_error_pct": 4.84 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 0.31, - "median_in": 4.33, - "p90_in": 32.65, - "scale_error_pct": 0.63 + "pairs": 525000, + "failed_pct": 0.32, + "median_in": 3.6, + "p90_in": 16.4, + "scale_error_pct": -0.14 }, "3-10m": { - "pairs": 595400, - "failed_pct": 0.17, - "median_in": 7.05, - "p90_in": 46.43, - "scale_error_pct": -0.06 + "pairs": 520400, + "failed_pct": 0.19, + "median_in": 5.88, + "p90_in": 24.38, + "scale_error_pct": -0.14 } }, "tape_calibration_success_pct": 100.0, - "groups": 8 + "groups": 7 } }, "4": { "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.01, - "median_in": 5.91, - "p90_in": 16.79, - "scale_error_pct": 2.54 + "pairs": 21000, + "failed_pct": 0.0, + "median_in": 5.29, + "p90_in": 13.44, + "scale_error_pct": 3.76 }, "3-10m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.03, - "median_in": 16.5, - "p90_in": 49.78, - "scale_error_pct": 3.69 + "median_in": 14.79, + "p90_in": 29.02, + "scale_error_pct": 4.61 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 1.01, - "median_in": 3.16, - "p90_in": 15.15, - "scale_error_pct": -0.19 + "pairs": 525000, + "failed_pct": 1.14, + "median_in": 2.75, + "p90_in": 10.74, + "scale_error_pct": -0.22 }, "3-10m": { - "pairs": 600000, - "failed_pct": 1.03, - "median_in": 6.87, - "p90_in": 39.4, - "scale_error_pct": -0.0 + "pairs": 525000, + "failed_pct": 1.17, + "median_in": 5.89, + "p90_in": 25.0, + "scale_error_pct": 0.13 } }, - "tape_calibration_success_pct": 99.0, - "groups": 8 + "tape_calibration_success_pct": 98.9, + "groups": 7 }, "vertical interior": { "none": { "1-3m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.02, - "median_in": 5.59, - "p90_in": 16.16, - "scale_error_pct": 1.26 + "median_in": 4.92, + "p90_in": 12.44, + "scale_error_pct": 3.27 }, "3-10m": { - "pairs": 24000, - "failed_pct": 0.05, - "median_in": 16.49, - "p90_in": 52.43, - "scale_error_pct": 4.17 + "pairs": 21000, + "failed_pct": 0.04, + "median_in": 14.94, + "p90_in": 29.77, + "scale_error_pct": 5.31 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 1.02, - "median_in": 2.83, - "p90_in": 14.72, - "scale_error_pct": -0.3 + "pairs": 525000, + "failed_pct": 1.17, + "median_in": 2.41, + "p90_in": 10.25, + "scale_error_pct": -0.32 }, "3-10m": { - "pairs": 600000, - "failed_pct": 1.05, - "median_in": 6.93, - "p90_in": 41.08, - "scale_error_pct": 0.03 + "pairs": 525000, + "failed_pct": 1.18, + "median_in": 5.91, + "p90_in": 28.82, + "scale_error_pct": 0.18 } }, - "tape_calibration_success_pct": 99.0, - "groups": 8 + "tape_calibration_success_pct": 98.9, + "groups": 7 }, "edges": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.11, - "median_in": 6.42, - "p90_in": 21.34, - "scale_error_pct": 4.83 + "pairs": 21000, + "failed_pct": 0.13, + "median_in": 6.05, + "p90_in": 17.21, + "scale_error_pct": 4.6 }, "3-10m": { - "pairs": 23816, - "failed_pct": 0.06, - "median_in": 13.29, - "p90_in": 37.59, - "scale_error_pct": 4.73 + "pairs": 20816, + "failed_pct": 0.07, + "median_in": 12.3, + "p90_in": 26.69, + "scale_error_pct": 5.16 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 1.11, - "median_in": 4.65, - "p90_in": 27.06, - "scale_error_pct": 0.65 + "pairs": 525000, + "failed_pct": 1.27, + "median_in": 3.86, + "p90_in": 15.9, + "scale_error_pct": -0.17 }, "3-10m": { - "pairs": 595400, - "failed_pct": 1.07, - "median_in": 7.65, - "p90_in": 47.82, - "scale_error_pct": 0.82 + "pairs": 520400, + "failed_pct": 1.23, + "median_in": 6.48, + "p90_in": 25.53, + "scale_error_pct": 0.61 } }, - "tape_calibration_success_pct": 99.0, - "groups": 8 + "tape_calibration_success_pct": 98.9, + "groups": 7 } }, "8": { @@ -3903,16 +3915,16 @@ "1-3m": { "pairs": 525000, "failed_pct": 0.57, - "median_in": 3.52, - "p90_in": 13.2, - "scale_error_pct": -0.33 + "median_in": 3.51, + "p90_in": 13.12, + "scale_error_pct": -0.3 }, "3-10m": { "pairs": 525000, "failed_pct": 0.59, - "median_in": 7.07, - "p90_in": 29.8, - "scale_error_pct": -0.14 + "median_in": 7.04, + "p90_in": 29.71, + "scale_error_pct": -0.11 } }, "tape_calibration_success_pct": 99.4, @@ -3922,33 +3934,33 @@ "none": { "1-3m": { "pairs": 21000, - "failed_pct": 0.0, - "median_in": 4.85, - "p90_in": 12.18, - "scale_error_pct": -0.52 + "failed_pct": 0.01, + "median_in": 4.82, + "p90_in": 12.26, + "scale_error_pct": -0.56 }, "3-10m": { "pairs": 21000, - "failed_pct": 0.0, - "median_in": 13.26, - "p90_in": 24.26, - "scale_error_pct": 2.9 + "failed_pct": 0.01, + "median_in": 13.2, + "p90_in": 24.18, + "scale_error_pct": 2.91 } }, "one_known_distance": { "1-3m": { "pairs": 525000, - "failed_pct": 0.57, - "median_in": 3.29, - "p90_in": 13.45, - "scale_error_pct": -1.05 + "failed_pct": 0.59, + "median_in": 3.27, + "p90_in": 13.55, + "scale_error_pct": -1.04 }, "3-10m": { "pairs": 525000, - "failed_pct": 0.57, - "median_in": 7.82, - "p90_in": 30.05, - "scale_error_pct": -0.24 + "failed_pct": 0.58, + "median_in": 7.78, + "p90_in": 29.83, + "scale_error_pct": -0.23 } }, "tape_calibration_success_pct": 99.4, @@ -3961,7 +3973,7 @@ "failed_pct": 0.0, "median_in": 6.48, "p90_in": 15.62, - "scale_error_pct": 3.76 + "scale_error_pct": 3.75 }, "3-10m": { "pairs": 20816, @@ -3975,16 +3987,16 @@ "1-3m": { "pairs": 525000, "failed_pct": 0.57, - "median_in": 4.76, - "p90_in": 16.26, - "scale_error_pct": 0.13 + "median_in": 4.74, + "p90_in": 16.27, + "scale_error_pct": 0.19 }, "3-10m": { "pairs": 520400, "failed_pct": 0.59, - "median_in": 7.3, - "p90_in": 27.99, - "scale_error_pct": 0.81 + "median_in": 7.27, + "p90_in": 28.01, + "scale_error_pct": 0.86 } }, "tape_calibration_success_pct": 99.4, @@ -3997,330 +4009,330 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.36, - "median_in": 7.6, - "p90_in": 18.98, - "scale_error_pct": -7.89 + "pairs": 21000, + "failed_pct": 0.27, + "median_in": 6.77, + "p90_in": 17.99, + "scale_error_pct": -6.72 }, "3-10m": { - "pairs": 24000, - "failed_pct": 0.64, - "median_in": 22.01, - "p90_in": 50.63, - "scale_error_pct": -8.35 + "pairs": 21000, + "failed_pct": 0.65, + "median_in": 20.07, + "p90_in": 46.87, + "scale_error_pct": -7.19 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 1.35, - "median_in": 3.73, - "p90_in": 20.55, - "scale_error_pct": 0.04 + "pairs": 525000, + "failed_pct": 1.41, + "median_in": 3.45, + "p90_in": 17.69, + "scale_error_pct": 0.05 }, "3-10m": { - "pairs": 600000, - "failed_pct": 1.63, - "median_in": 9.72, - "p90_in": 48.66, - "scale_error_pct": -0.74 + "pairs": 525000, + "failed_pct": 1.78, + "median_in": 8.92, + "p90_in": 40.7, + "scale_error_pct": -0.86 } }, - "tape_calibration_success_pct": 99.0, - "groups": 8 + "tape_calibration_success_pct": 98.9, + "groups": 7 }, "vertical interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.62, - "median_in": 7.32, - "p90_in": 19.16, - "scale_error_pct": -9.7 + "pairs": 21000, + "failed_pct": 0.5, + "median_in": 6.33, + "p90_in": 18.19, + "scale_error_pct": -8.09 }, "3-10m": { - "pairs": 24000, - "failed_pct": 0.97, - "median_in": 26.24, - "p90_in": 59.43, - "scale_error_pct": -11.52 + "pairs": 21000, + "failed_pct": 1.11, + "median_in": 23.59, + "p90_in": 58.75, + "scale_error_pct": -10.23 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 1.61, - "median_in": 3.38, - "p90_in": 19.19, - "scale_error_pct": -0.87 + "pairs": 525000, + "failed_pct": 1.64, + "median_in": 3.14, + "p90_in": 16.96, + "scale_error_pct": -0.84 }, "3-10m": { - "pairs": 600000, - "failed_pct": 1.94, - "median_in": 11.38, - "p90_in": 53.88, - "scale_error_pct": -2.48 + "pairs": 525000, + "failed_pct": 2.23, + "median_in": 10.71, + "p90_in": 50.22, + "scale_error_pct": -2.78 } }, - "tape_calibration_success_pct": 99.0, - "groups": 8 + "tape_calibration_success_pct": 98.9, + "groups": 7 }, "edges": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 1.71, - "median_in": 9.47, - "p90_in": 37.53, - "scale_error_pct": -0.93 + "pairs": 21000, + "failed_pct": 1.69, + "median_in": 8.34, + "p90_in": 30.3, + "scale_error_pct": -2.72 }, "3-10m": { - "pairs": 23816, - "failed_pct": 0.94, - "median_in": 22.95, - "p90_in": 113.43, - "scale_error_pct": -2.1 + "pairs": 20816, + "failed_pct": 0.8, + "median_in": 20.77, + "p90_in": 48.51, + "scale_error_pct": -1.93 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 2.7, - "median_in": 7.56, - "p90_in": 57.88, - "scale_error_pct": 5.44 + "pairs": 525000, + "failed_pct": 2.82, + "median_in": 6.06, + "p90_in": 40.45, + "scale_error_pct": 3.87 }, "3-10m": { - "pairs": 595400, - "failed_pct": 1.94, - "median_in": 15.37, - "p90_in": 148.02, - "scale_error_pct": 3.41 + "pairs": 520400, + "failed_pct": 1.95, + "median_in": 12.68, + "p90_in": 60.89, + "scale_error_pct": 3.39 } }, - "tape_calibration_success_pct": 99.0, - "groups": 8 + "tape_calibration_success_pct": 98.9, + "groups": 7 } }, "2": { "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.25, - "median_in": 7.65, - "p90_in": 19.63, - "scale_error_pct": -8.05 + "pairs": 21000, + "failed_pct": 0.21, + "median_in": 6.62, + "p90_in": 17.72, + "scale_error_pct": -6.62 }, "3-10m": { - "pairs": 24000, - "failed_pct": 0.56, - "median_in": 21.61, - "p90_in": 55.88, - "scale_error_pct": -8.49 + "pairs": 21000, + "failed_pct": 0.57, + "median_in": 19.04, + "p90_in": 49.65, + "scale_error_pct": -6.62 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 0.75, - "median_in": 3.95, - "p90_in": 18.27, - "scale_error_pct": 0.16 + "pairs": 525000, + "failed_pct": 0.79, + "median_in": 3.7, + "p90_in": 15.86, + "scale_error_pct": 0.11 }, "3-10m": { - "pairs": 600000, - "failed_pct": 1.05, - "median_in": 9.56, - "p90_in": 41.28, - "scale_error_pct": -0.32 + "pairs": 525000, + "failed_pct": 1.13, + "median_in": 8.7, + "p90_in": 35.61, + "scale_error_pct": -0.54 } }, - "tape_calibration_success_pct": 99.5, - "groups": 8 + "tape_calibration_success_pct": 99.4, + "groups": 7 }, "vertical interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.38, - "median_in": 7.28, - "p90_in": 20.26, - "scale_error_pct": -10.12 + "pairs": 21000, + "failed_pct": 0.37, + "median_in": 6.24, + "p90_in": 18.5, + "scale_error_pct": -7.96 }, "3-10m": { - "pairs": 24000, - "failed_pct": 0.82, - "median_in": 28.03, - "p90_in": 65.28, - "scale_error_pct": -13.29 + "pairs": 21000, + "failed_pct": 0.89, + "median_in": 24.06, + "p90_in": 62.4, + "scale_error_pct": -12.22 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 0.88, - "median_in": 3.56, - "p90_in": 17.34, - "scale_error_pct": -0.94 + "pairs": 525000, + "failed_pct": 0.94, + "median_in": 3.34, + "p90_in": 15.87, + "scale_error_pct": -1.01 }, "3-10m": { - "pairs": 600000, - "failed_pct": 1.3, - "median_in": 11.32, - "p90_in": 51.12, - "scale_error_pct": -2.27 + "pairs": 525000, + "failed_pct": 1.44, + "median_in": 10.63, + "p90_in": 49.11, + "scale_error_pct": -2.73 } }, - "tape_calibration_success_pct": 99.5, - "groups": 8 + "tape_calibration_success_pct": 99.4, + "groups": 7 }, "edges": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 1.34, - "median_in": 9.51, - "p90_in": 43.13, - "scale_error_pct": -0.73 + "pairs": 21000, + "failed_pct": 1.38, + "median_in": 8.58, + "p90_in": 42.46, + "scale_error_pct": -2.28 }, "3-10m": { - "pairs": 23816, - "failed_pct": 0.57, - "median_in": 24.27, - "p90_in": 96.74, - "scale_error_pct": -1.28 + "pairs": 20816, + "failed_pct": 0.61, + "median_in": 22.06, + "p90_in": 60.88, + "scale_error_pct": -0.88 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 1.84, - "median_in": 8.12, - "p90_in": 62.49, - "scale_error_pct": 6.44 + "pairs": 525000, + "failed_pct": 1.95, + "median_in": 6.67, + "p90_in": 49.67, + "scale_error_pct": 4.84 }, "3-10m": { - "pairs": 595400, - "failed_pct": 1.07, - "median_in": 16.73, - "p90_in": 128.58, - "scale_error_pct": 4.35 + "pairs": 520400, + "failed_pct": 1.19, + "median_in": 14.21, + "p90_in": 64.95, + "scale_error_pct": 4.3 } }, - "tape_calibration_success_pct": 99.5, - "groups": 8 + "tape_calibration_success_pct": 99.4, + "groups": 7 } }, "4": { "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.1, - "median_in": 7.18, - "p90_in": 20.51, - "scale_error_pct": -7.53 + "pairs": 21000, + "failed_pct": 0.09, + "median_in": 6.12, + "p90_in": 16.46, + "scale_error_pct": -6.24 }, "3-10m": { - "pairs": 24000, - "failed_pct": 0.28, - "median_in": 20.96, - "p90_in": 59.26, - "scale_error_pct": -8.46 + "pairs": 21000, + "failed_pct": 0.29, + "median_in": 17.98, + "p90_in": 46.8, + "scale_error_pct": -6.28 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 1.1, - "median_in": 4.28, - "p90_in": 21.81, - "scale_error_pct": 0.06 + "pairs": 525000, + "failed_pct": 1.23, + "median_in": 3.91, + "p90_in": 17.43, + "scale_error_pct": -0.01 }, "3-10m": { - "pairs": 600000, - "failed_pct": 1.28, - "median_in": 9.95, - "p90_in": 51.41, - "scale_error_pct": -0.36 + "pairs": 525000, + "failed_pct": 1.43, + "median_in": 8.9, + "p90_in": 41.21, + "scale_error_pct": -0.41 } }, - "tape_calibration_success_pct": 99.0, - "groups": 8 + "tape_calibration_success_pct": 98.9, + "groups": 7 }, "vertical interior": { "none": { "1-3m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.15, - "median_in": 7.02, - "p90_in": 20.47, - "scale_error_pct": -9.57 + "median_in": 6.01, + "p90_in": 16.37, + "scale_error_pct": -7.59 }, "3-10m": { - "pairs": 24000, - "failed_pct": 0.4, - "median_in": 26.57, - "p90_in": 69.41, - "scale_error_pct": -13.5 + "pairs": 21000, + "failed_pct": 0.44, + "median_in": 22.23, + "p90_in": 58.8, + "scale_error_pct": -11.81 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 1.15, - "median_in": 3.73, - "p90_in": 20.88, - "scale_error_pct": -1.12 + "pairs": 525000, + "failed_pct": 1.3, + "median_in": 3.41, + "p90_in": 17.12, + "scale_error_pct": -1.21 }, "3-10m": { - "pairs": 600000, - "failed_pct": 1.4, - "median_in": 11.0, - "p90_in": 62.68, - "scale_error_pct": -2.34 + "pairs": 525000, + "failed_pct": 1.58, + "median_in": 9.92, + "p90_in": 54.11, + "scale_error_pct": -2.5 } }, - "tape_calibration_success_pct": 99.0, - "groups": 8 + "tape_calibration_success_pct": 98.9, + "groups": 7 }, "edges": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.63, - "median_in": 7.99, - "p90_in": 32.24, - "scale_error_pct": -0.99 + "pairs": 21000, + "failed_pct": 0.68, + "median_in": 7.51, + "p90_in": 28.37, + "scale_error_pct": -2.15 }, "3-10m": { - "pairs": 23816, - "failed_pct": 0.28, - "median_in": 20.47, - "p90_in": 78.27, - "scale_error_pct": -2.04 + "pairs": 20816, + "failed_pct": 0.31, + "median_in": 18.5, + "p90_in": 54.91, + "scale_error_pct": -1.64 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 1.63, - "median_in": 7.64, - "p90_in": 51.99, - "scale_error_pct": 5.6 + "pairs": 525000, + "failed_pct": 1.82, + "median_in": 6.35, + "p90_in": 37.97, + "scale_error_pct": 3.88 }, "3-10m": { - "pairs": 595400, - "failed_pct": 1.29, - "median_in": 16.25, - "p90_in": 112.97, - "scale_error_pct": 3.41 + "pairs": 520400, + "failed_pct": 1.46, + "median_in": 13.26, + "p90_in": 58.06, + "scale_error_pct": 3.4 } }, - "tape_calibration_success_pct": 99.0, - "groups": 8 + "tape_calibration_success_pct": 98.9, + "groups": 7 } }, "8": { @@ -4345,16 +4357,16 @@ "1-3m": { "pairs": 525000, "failed_pct": 0.61, - "median_in": 3.9, - "p90_in": 19.02, - "scale_error_pct": -0.02 + "median_in": 3.92, + "p90_in": 19.03, + "scale_error_pct": -0.03 }, "3-10m": { "pairs": 525000, "failed_pct": 0.66, "median_in": 8.77, - "p90_in": 45.61, - "scale_error_pct": -0.26 + "p90_in": 45.62, + "scale_error_pct": -0.27 } }, "tape_calibration_success_pct": 99.4, @@ -4364,33 +4376,33 @@ "none": { "1-3m": { "pairs": 21000, - "failed_pct": 0.01, - "median_in": 7.26, - "p90_in": 18.65, - "scale_error_pct": -9.39 + "failed_pct": 0.04, + "median_in": 7.18, + "p90_in": 18.64, + "scale_error_pct": -9.25 }, "3-10m": { "pairs": 21000, - "failed_pct": 0.04, - "median_in": 25.45, - "p90_in": 59.67, - "scale_error_pct": -12.81 + "failed_pct": 0.05, + "median_in": 25.42, + "p90_in": 59.03, + "scale_error_pct": -12.78 } }, "one_known_distance": { "1-3m": { "pairs": 525000, - "failed_pct": 0.59, - "median_in": 3.72, - "p90_in": 20.43, + "failed_pct": 0.61, + "median_in": 3.71, + "p90_in": 20.28, "scale_error_pct": -1.64 }, "3-10m": { "pairs": 525000, - "failed_pct": 0.61, - "median_in": 11.28, - "p90_in": 54.94, - "scale_error_pct": -3.08 + "failed_pct": 0.62, + "median_in": 11.24, + "p90_in": 54.76, + "scale_error_pct": -3.02 } }, "tape_calibration_success_pct": 99.4, @@ -4419,13 +4431,13 @@ "failed_pct": 0.72, "median_in": 6.17, "p90_in": 41.49, - "scale_error_pct": 3.86 + "scale_error_pct": 3.84 }, "3-10m": { "pairs": 520400, "failed_pct": 0.68, - "median_in": 13.84, - "p90_in": 65.41, + "median_in": 13.85, + "p90_in": 65.44, "scale_error_pct": 3.25 } }, @@ -4441,330 +4453,330 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.0, - "median_in": 0.71, - "p90_in": 1.61, - "scale_error_pct": -0.93 + "median_in": 0.67, + "p90_in": 1.56, + "scale_error_pct": -0.87 }, "3-10m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.0, - "median_in": 1.31, - "p90_in": 2.38, - "scale_error_pct": -0.8 + "median_in": 1.22, + "p90_in": 2.28, + "scale_error_pct": -0.72 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.0, - "median_in": 0.54, - "p90_in": 1.57, - "scale_error_pct": -0.02 + "median_in": 0.53, + "p90_in": 1.56, + "scale_error_pct": -0.03 }, "3-10m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.0, - "median_in": 0.93, - "p90_in": 2.76, - "scale_error_pct": -0.02 + "median_in": 0.94, + "p90_in": 2.8, + "scale_error_pct": -0.03 } }, "tape_calibration_success_pct": 100.0, - "groups": 8 + "groups": 7 }, "vertical interior": { "none": { "1-3m": { - "pairs": 21000, + "pairs": 18000, "failed_pct": 0.0, - "median_in": 0.13, - "p90_in": 0.95, - "scale_error_pct": -0.17 + "median_in": 0.1, + "p90_in": 0.53, + "scale_error_pct": -0.13 }, "3-10m": { - "pairs": 21000, + "pairs": 18000, "failed_pct": 0.0, - "median_in": 0.3, - "p90_in": 1.95, - "scale_error_pct": -0.2 + "median_in": 0.25, + "p90_in": 0.97, + "scale_error_pct": -0.17 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, + "pairs": 450000, "failed_pct": 0.0, - "median_in": 0.39, + "median_in": 0.38, "p90_in": 1.51, - "scale_error_pct": 0.15 + "scale_error_pct": 0.17 }, "3-10m": { - "pairs": 525000, + "pairs": 450000, "failed_pct": 0.0, - "median_in": 0.76, - "p90_in": 2.77, - "scale_error_pct": 0.11 + "median_in": 0.77, + "p90_in": 2.84, + "scale_error_pct": 0.12 } }, "tape_calibration_success_pct": 100.0, - "groups": 7 + "groups": 6 }, "edges": { "none": { "1-3m": { - "pairs": 21000, + "pairs": 18000, "failed_pct": 0.0, - "median_in": 0.3, - "p90_in": 1.1, - "scale_error_pct": -0.36 + "median_in": 0.29, + "p90_in": 1.07, + "scale_error_pct": -0.34 }, "3-10m": { - "pairs": 20786, + "pairs": 17786, "failed_pct": 0.0, "median_in": 0.49, - "p90_in": 1.66, + "p90_in": 1.6, "scale_error_pct": -0.35 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, + "pairs": 450000, "failed_pct": 0.0, "median_in": 0.52, - "p90_in": 1.68, - "scale_error_pct": 0.12 + "p90_in": 1.67, + "scale_error_pct": 0.04 }, "3-10m": { - "pairs": 519650, + "pairs": 444650, "failed_pct": 0.0, - "median_in": 1.03, - "p90_in": 2.83, - "scale_error_pct": 0.11 + "median_in": 1.01, + "p90_in": 2.74, + "scale_error_pct": 0.03 } }, "tape_calibration_success_pct": 100.0, - "groups": 7 + "groups": 6 } }, "2": { "surface interior": { "none": { "1-3m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.0, - "median_in": 0.65, - "p90_in": 1.51, - "scale_error_pct": -0.81 + "median_in": 0.62, + "p90_in": 1.46, + "scale_error_pct": -0.75 }, "3-10m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.0, - "median_in": 1.28, - "p90_in": 2.34, - "scale_error_pct": -0.77 + "median_in": 1.23, + "p90_in": 2.24, + "scale_error_pct": -0.72 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.0, "median_in": 0.49, - "p90_in": 1.57, - "scale_error_pct": -0.02 + "p90_in": 1.6, + "scale_error_pct": -0.04 }, "3-10m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.0, - "median_in": 0.92, - "p90_in": 2.82, + "median_in": 0.93, + "p90_in": 2.95, "scale_error_pct": -0.04 } }, "tape_calibration_success_pct": 100.0, - "groups": 8 + "groups": 7 }, "vertical interior": { "none": { "1-3m": { - "pairs": 21000, + "pairs": 18000, "failed_pct": 0.0, - "median_in": 0.15, - "p90_in": 0.85, - "scale_error_pct": -0.18 + "median_in": 0.12, + "p90_in": 0.51, + "scale_error_pct": -0.16 }, "3-10m": { - "pairs": 21000, + "pairs": 18000, "failed_pct": 0.0, - "median_in": 0.32, - "p90_in": 1.82, - "scale_error_pct": -0.21 + "median_in": 0.27, + "p90_in": 0.91, + "scale_error_pct": -0.18 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, + "pairs": 450000, "failed_pct": 0.0, "median_in": 0.41, - "p90_in": 1.52, - "scale_error_pct": 0.14 + "p90_in": 1.58, + "scale_error_pct": 0.15 }, "3-10m": { - "pairs": 525000, + "pairs": 450000, "failed_pct": 0.0, - "median_in": 0.81, - "p90_in": 2.79, - "scale_error_pct": 0.09 + "median_in": 0.86, + "p90_in": 3.03, + "scale_error_pct": 0.12 } }, "tape_calibration_success_pct": 100.0, - "groups": 7 + "groups": 6 }, "edges": { "none": { "1-3m": { - "pairs": 21000, + "pairs": 18000, "failed_pct": 0.0, "median_in": 0.31, - "p90_in": 1.1, - "scale_error_pct": -0.38 + "p90_in": 1.11, + "scale_error_pct": -0.37 }, "3-10m": { - "pairs": 20786, + "pairs": 17786, "failed_pct": 0.0, - "median_in": 0.48, - "p90_in": 1.62, - "scale_error_pct": -0.34 + "median_in": 0.49, + "p90_in": 1.57, + "scale_error_pct": -0.35 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, + "pairs": 450000, "failed_pct": 0.0, - "median_in": 0.51, - "p90_in": 1.71, - "scale_error_pct": 0.12 + "median_in": 0.54, + "p90_in": 1.82, + "scale_error_pct": 0.03 }, "3-10m": { - "pairs": 519650, + "pairs": 444650, "failed_pct": 0.0, - "median_in": 1.01, - "p90_in": 2.81, - "scale_error_pct": 0.12 + "median_in": 1.06, + "p90_in": 2.9, + "scale_error_pct": 0.04 } }, "tape_calibration_success_pct": 100.0, - "groups": 7 + "groups": 6 } }, "4": { "surface interior": { "none": { "1-3m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.0, - "median_in": 0.57, - "p90_in": 1.29, - "scale_error_pct": -0.68 + "median_in": 0.55, + "p90_in": 1.25, + "scale_error_pct": -0.64 }, "3-10m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.0, - "median_in": 1.13, - "p90_in": 2.23, - "scale_error_pct": -0.67 + "median_in": 1.09, + "p90_in": 2.1, + "scale_error_pct": -0.63 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.0, - "median_in": 0.43, - "p90_in": 1.62, + "median_in": 0.45, + "p90_in": 1.68, "scale_error_pct": -0.02 }, "3-10m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.0, - "median_in": 0.8, - "p90_in": 3.12, - "scale_error_pct": -0.05 + "median_in": 0.81, + "p90_in": 3.33, + "scale_error_pct": -0.03 } }, "tape_calibration_success_pct": 100.0, - "groups": 8 + "groups": 7 }, "vertical interior": { "none": { "1-3m": { - "pairs": 21000, + "pairs": 18000, "failed_pct": 0.0, - "median_in": 0.17, - "p90_in": 0.77, - "scale_error_pct": -0.21 + "median_in": 0.14, + "p90_in": 0.54, + "scale_error_pct": -0.18 }, "3-10m": { - "pairs": 21000, + "pairs": 18000, "failed_pct": 0.0, - "median_in": 0.36, - "p90_in": 1.64, - "scale_error_pct": -0.25 + "median_in": 0.28, + "p90_in": 0.92, + "scale_error_pct": -0.17 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, + "pairs": 450000, "failed_pct": 0.0, - "median_in": 0.4, - "p90_in": 1.62, - "scale_error_pct": 0.09 + "median_in": 0.44, + "p90_in": 1.69, + "scale_error_pct": 0.12 }, "3-10m": { - "pairs": 525000, + "pairs": 450000, "failed_pct": 0.0, - "median_in": 0.81, - "p90_in": 2.99, - "scale_error_pct": 0.06 + "median_in": 0.92, + "p90_in": 3.29, + "scale_error_pct": 0.1 } }, "tape_calibration_success_pct": 100.0, - "groups": 7 + "groups": 6 }, "edges": { "none": { "1-3m": { - "pairs": 21000, + "pairs": 18000, "failed_pct": 0.0, "median_in": 0.31, - "p90_in": 1.05, - "scale_error_pct": -0.4 + "p90_in": 1.09, + "scale_error_pct": -0.39 }, "3-10m": { - "pairs": 20786, + "pairs": 17786, "failed_pct": 0.0, - "median_in": 0.56, - "p90_in": 1.63, - "scale_error_pct": -0.41 + "median_in": 0.57, + "p90_in": 1.59, + "scale_error_pct": -0.43 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, + "pairs": 450000, "failed_pct": 0.0, - "median_in": 0.49, - "p90_in": 1.77, - "scale_error_pct": 0.1 + "median_in": 0.56, + "p90_in": 1.93, + "scale_error_pct": 0.04 }, "3-10m": { - "pairs": 519650, + "pairs": 444650, "failed_pct": 0.0, - "median_in": 0.96, - "p90_in": 3.01, - "scale_error_pct": 0.08 + "median_in": 1.08, + "p90_in": 3.17, + "scale_error_pct": 0.03 } }, "tape_calibration_success_pct": 100.0, - "groups": 7 + "groups": 6 } }, "8": { @@ -4790,14 +4802,14 @@ "pairs": 525000, "failed_pct": 0.0, "median_in": 0.45, - "p90_in": 1.74, - "scale_error_pct": -0.06 + "p90_in": 1.82, + "scale_error_pct": -0.07 }, "3-10m": { "pairs": 525000, "failed_pct": 0.0, - "median_in": 0.79, - "p90_in": 3.62, + "median_in": 0.8, + "p90_in": 3.75, "scale_error_pct": -0.08 } }, @@ -4810,14 +4822,14 @@ "pairs": 18000, "failed_pct": 0.0, "median_in": 0.15, - "p90_in": 0.53, + "p90_in": 0.52, "scale_error_pct": -0.18 }, "3-10m": { "pairs": 18000, "failed_pct": 0.0, "median_in": 0.28, - "p90_in": 0.86, + "p90_in": 0.87, "scale_error_pct": -0.18 } }, @@ -4826,15 +4838,15 @@ "pairs": 450000, "failed_pct": 0.0, "median_in": 0.4, - "p90_in": 1.66, + "p90_in": 1.74, "scale_error_pct": 0.07 }, "3-10m": { "pairs": 450000, "failed_pct": 0.0, - "median_in": 0.85, - "p90_in": 3.2, - "scale_error_pct": 0.08 + "median_in": 0.86, + "p90_in": 3.37, + "scale_error_pct": 0.07 } }, "tape_calibration_success_pct": 100.0, @@ -4853,7 +4865,7 @@ "pairs": 17786, "failed_pct": 0.0, "median_in": 0.53, - "p90_in": 1.55, + "p90_in": 1.56, "scale_error_pct": -0.39 } }, @@ -4861,15 +4873,15 @@ "1-3m": { "pairs": 450000, "failed_pct": 0.0, - "median_in": 0.51, - "p90_in": 1.89, + "median_in": 0.52, + "p90_in": 2.0, "scale_error_pct": 0.01 }, "3-10m": { "pairs": 444650, "failed_pct": 0.0, - "median_in": 1.01, - "p90_in": 3.22, + "median_in": 1.02, + "p90_in": 3.44, "scale_error_pct": 0.01 } }, @@ -4883,330 +4895,330 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.7, - "median_in": 6.88, - "p90_in": 14.89, - "scale_error_pct": 4.39 + "pairs": 21000, + "failed_pct": 0.07, + "median_in": 6.8, + "p90_in": 13.81, + "scale_error_pct": 4.16 }, "3-10m": { - "pairs": 24000, - "failed_pct": 1.83, - "median_in": 15.5, - "p90_in": 30.4, - "scale_error_pct": 4.77 + "pairs": 21000, + "failed_pct": 0.17, + "median_in": 15.24, + "p90_in": 28.56, + "scale_error_pct": 4.38 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 0.7, - "median_in": 1.78, - "p90_in": 8.65, - "scale_error_pct": -0.15 + "pairs": 525000, + "failed_pct": 0.07, + "median_in": 1.44, + "p90_in": 5.22, + "scale_error_pct": -0.11 }, "3-10m": { - "pairs": 600000, - "failed_pct": 1.83, - "median_in": 3.5, - "p90_in": 17.52, - "scale_error_pct": -0.23 + "pairs": 525000, + "failed_pct": 0.17, + "median_in": 2.82, + "p90_in": 10.1, + "scale_error_pct": -0.3 } }, "tape_calibration_success_pct": 100.0, - "groups": 8 + "groups": 7 }, "vertical interior": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 1.07, - "median_in": 5.39, - "p90_in": 14.82, - "scale_error_pct": 3.53 + "pairs": 18000, + "failed_pct": 0.91, + "median_in": 5.17, + "p90_in": 14.28, + "scale_error_pct": 3.36 }, "3-10m": { - "pairs": 21000, - "failed_pct": 2.72, - "median_in": 13.51, - "p90_in": 33.47, - "scale_error_pct": 4.51 + "pairs": 18000, + "failed_pct": 0.99, + "median_in": 13.47, + "p90_in": 33.02, + "scale_error_pct": 4.22 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 1.07, - "median_in": 1.82, - "p90_in": 9.91, - "scale_error_pct": -0.25 + "pairs": 450000, + "failed_pct": 0.91, + "median_in": 1.39, + "p90_in": 5.31, + "scale_error_pct": -0.12 }, "3-10m": { - "pairs": 525000, - "failed_pct": 2.72, - "median_in": 4.07, - "p90_in": 29.1, - "scale_error_pct": 0.05 + "pairs": 450000, + "failed_pct": 0.99, + "median_in": 3.42, + "p90_in": 13.94, + "scale_error_pct": 0.11 } }, "tape_calibration_success_pct": 100.0, - "groups": 7 + "groups": 6 }, "edges": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 1.01, - "median_in": 7.83, - "p90_in": 35.53, - "scale_error_pct": 3.6 + "pairs": 18000, + "failed_pct": 1.13, + "median_in": 6.57, + "p90_in": 16.53, + "scale_error_pct": 3.05 }, "3-10m": { - "pairs": 20786, - "failed_pct": 0.58, - "median_in": 15.09, - "p90_in": 68.66, - "scale_error_pct": 4.67 + "pairs": 17786, + "failed_pct": 0.68, + "median_in": 12.18, + "p90_in": 23.37, + "scale_error_pct": 3.76 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 1.01, - "median_in": 2.82, - "p90_in": 26.09, - "scale_error_pct": 0.06 + "pairs": 450000, + "failed_pct": 1.13, + "median_in": 2.25, + "p90_in": 9.94, + "scale_error_pct": -0.44 }, "3-10m": { - "pairs": 519650, - "failed_pct": 0.58, - "median_in": 4.49, - "p90_in": 52.4, - "scale_error_pct": -0.07 + "pairs": 444650, + "failed_pct": 0.68, + "median_in": 3.68, + "p90_in": 13.45, + "scale_error_pct": -0.46 } }, "tape_calibration_success_pct": 100.0, - "groups": 7 + "groups": 6 } }, "2": { "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.49, - "median_in": 6.81, - "p90_in": 14.99, - "scale_error_pct": 2.42 + "pairs": 21000, + "failed_pct": 0.06, + "median_in": 6.62, + "p90_in": 13.92, + "scale_error_pct": 1.97 }, "3-10m": { - "pairs": 24000, - "failed_pct": 1.68, - "median_in": 14.9, - "p90_in": 28.73, - "scale_error_pct": 3.75 + "pairs": 21000, + "failed_pct": 0.17, + "median_in": 14.62, + "p90_in": 27.78, + "scale_error_pct": 2.59 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 0.49, - "median_in": 2.3, - "p90_in": 11.34, - "scale_error_pct": -0.01 + "pairs": 525000, + "failed_pct": 0.63, + "median_in": 1.99, + "p90_in": 8.32, + "scale_error_pct": -0.07 }, "3-10m": { - "pairs": 600000, - "failed_pct": 1.68, - "median_in": 3.96, - "p90_in": 21.1, - "scale_error_pct": 0.04 + "pairs": 525000, + "failed_pct": 0.74, + "median_in": 3.47, + "p90_in": 15.55, + "scale_error_pct": -0.04 } }, - "tape_calibration_success_pct": 100.0, - "groups": 8 + "tape_calibration_success_pct": 99.4, + "groups": 7 }, "vertical interior": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 0.99, - "median_in": 5.21, - "p90_in": 15.32, - "scale_error_pct": -0.05 + "pairs": 18000, + "failed_pct": 0.91, + "median_in": 4.94, + "p90_in": 13.76, + "scale_error_pct": -1.31 }, "3-10m": { - "pairs": 21000, - "failed_pct": 2.6, - "median_in": 13.27, - "p90_in": 31.63, - "scale_error_pct": 2.53 + "pairs": 18000, + "failed_pct": 0.96, + "median_in": 12.92, + "p90_in": 31.39, + "scale_error_pct": 1.87 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 0.99, - "median_in": 2.4, - "p90_in": 12.78, - "scale_error_pct": -0.26 + "pairs": 450000, + "failed_pct": 1.58, + "median_in": 2.19, + "p90_in": 8.57, + "scale_error_pct": -0.35 }, "3-10m": { - "pairs": 525000, - "failed_pct": 2.6, - "median_in": 5.11, - "p90_in": 32.19, - "scale_error_pct": 0.16 + "pairs": 450000, + "failed_pct": 1.63, + "median_in": 4.53, + "p90_in": 21.92, + "scale_error_pct": 0.13 } }, - "tape_calibration_success_pct": 100.0, - "groups": 7 + "tape_calibration_success_pct": 99.3, + "groups": 6 }, "edges": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 1.0, - "median_in": 8.47, - "p90_in": 33.83, - "scale_error_pct": 1.22 + "pairs": 18000, + "failed_pct": 1.13, + "median_in": 7.15, + "p90_in": 17.3, + "scale_error_pct": 0.37 }, "3-10m": { - "pairs": 20786, - "failed_pct": 0.58, - "median_in": 15.12, - "p90_in": 49.92, - "scale_error_pct": 2.38 + "pairs": 17786, + "failed_pct": 0.68, + "median_in": 11.97, + "p90_in": 23.56, + "scale_error_pct": 1.4 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 1.0, - "median_in": 3.85, - "p90_in": 27.29, - "scale_error_pct": 0.75 + "pairs": 450000, + "failed_pct": 1.8, + "median_in": 3.22, + "p90_in": 14.58, + "scale_error_pct": -0.17 }, "3-10m": { - "pairs": 519650, - "failed_pct": 0.58, - "median_in": 5.53, - "p90_in": 38.06, - "scale_error_pct": 0.14 + "pairs": 444650, + "failed_pct": 1.35, + "median_in": 4.73, + "p90_in": 18.8, + "scale_error_pct": -0.77 } }, - "tape_calibration_success_pct": 100.0, - "groups": 7 + "tape_calibration_success_pct": 99.3, + "groups": 6 } }, "4": { "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.22, - "median_in": 6.14, - "p90_in": 14.4, - "scale_error_pct": 2.54 + "pairs": 21000, + "failed_pct": 0.01, + "median_in": 5.71, + "p90_in": 13.0, + "scale_error_pct": 2.93 }, "3-10m": { - "pairs": 24000, - "failed_pct": 1.16, - "median_in": 13.88, - "p90_in": 25.94, + "pairs": 21000, + "failed_pct": 0.07, + "median_in": 13.52, + "p90_in": 24.33, "scale_error_pct": 4.38 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 0.72, - "median_in": 2.47, - "p90_in": 15.68, - "scale_error_pct": 0.19 + "pairs": 525000, + "failed_pct": 1.15, + "median_in": 2.2, + "p90_in": 9.98, + "scale_error_pct": 0.04 }, "3-10m": { - "pairs": 600000, - "failed_pct": 1.66, - "median_in": 4.37, - "p90_in": 33.08, - "scale_error_pct": 0.56 + "pairs": 525000, + "failed_pct": 1.21, + "median_in": 3.89, + "p90_in": 18.93, + "scale_error_pct": 0.37 } }, - "tape_calibration_success_pct": 99.5, - "groups": 8 + "tape_calibration_success_pct": 98.9, + "groups": 7 }, "vertical interior": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 0.45, - "median_in": 5.6, - "p90_in": 14.98, - "scale_error_pct": -0.13 + "pairs": 18000, + "failed_pct": 0.42, + "median_in": 4.98, + "p90_in": 12.64, + "scale_error_pct": 0.09 }, "3-10m": { - "pairs": 21000, - "failed_pct": 2.03, - "median_in": 13.66, - "p90_in": 30.36, - "scale_error_pct": 4.29 + "pairs": 18000, + "failed_pct": 0.48, + "median_in": 13.06, + "p90_in": 28.85, + "scale_error_pct": 4.22 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 1.02, - "median_in": 2.37, - "p90_in": 19.75, - "scale_error_pct": -0.05 + "pairs": 450000, + "failed_pct": 1.75, + "median_in": 2.09, + "p90_in": 10.96, + "scale_error_pct": -0.2 }, "3-10m": { - "pairs": 525000, - "failed_pct": 2.6, - "median_in": 5.26, - "p90_in": 52.6, - "scale_error_pct": 1.1 + "pairs": 450000, + "failed_pct": 1.82, + "median_in": 4.43, + "p90_in": 30.13, + "scale_error_pct": 0.76 } }, - "tape_calibration_success_pct": 99.4, - "groups": 7 + "tape_calibration_success_pct": 98.7, + "groups": 6 }, "edges": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 0.44, - "median_in": 7.47, - "p90_in": 20.55, - "scale_error_pct": 2.81 + "pairs": 18000, + "failed_pct": 0.51, + "median_in": 6.93, + "p90_in": 16.04, + "scale_error_pct": 2.41 }, "3-10m": { - "pairs": 20786, - "failed_pct": 0.3, - "median_in": 13.34, - "p90_in": 34.63, - "scale_error_pct": 3.31 + "pairs": 17786, + "failed_pct": 0.35, + "median_in": 11.39, + "p90_in": 21.06, + "scale_error_pct": 2.98 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 1.01, - "median_in": 4.15, - "p90_in": 26.44, - "scale_error_pct": 0.8 + "pairs": 450000, + "failed_pct": 1.84, + "median_in": 3.51, + "p90_in": 16.48, + "scale_error_pct": 0.1 }, "3-10m": { - "pairs": 519650, - "failed_pct": 0.88, - "median_in": 6.44, - "p90_in": 50.01, - "scale_error_pct": 1.03 + "pairs": 444650, + "failed_pct": 1.7, + "median_in": 5.22, + "p90_in": 24.68, + "scale_error_pct": 0.35 } }, - "tape_calibration_success_pct": 99.4, - "groups": 7 + "tape_calibration_success_pct": 98.7, + "groups": 6 } }, "8": { @@ -5230,56 +5242,56 @@ "one_known_distance": { "1-3m": { "pairs": 525000, - "failed_pct": 0.57, - "median_in": 2.78, - "p90_in": 10.79, - "scale_error_pct": 0.11 + "failed_pct": 2.29, + "median_in": 2.84, + "p90_in": 11.94, + "scale_error_pct": -0.03 }, "3-10m": { "pairs": 525000, - "failed_pct": 0.59, - "median_in": 4.27, - "p90_in": 21.13, - "scale_error_pct": 0.18 + "failed_pct": 2.3, + "median_in": 4.47, + "p90_in": 24.28, + "scale_error_pct": 0.13 } }, - "tape_calibration_success_pct": 99.4, + "tape_calibration_success_pct": 97.7, "groups": 7 }, "vertical interior": { "none": { "1-3m": { "pairs": 18000, - "failed_pct": 0.11, - "median_in": 4.85, - "p90_in": 12.45, - "scale_error_pct": -1.26 + "failed_pct": 0.12, + "median_in": 4.82, + "p90_in": 12.53, + "scale_error_pct": -1.29 }, "3-10m": { "pairs": 18000, - "failed_pct": 0.23, - "median_in": 11.55, - "p90_in": 25.82, - "scale_error_pct": 2.73 + "failed_pct": 0.21, + "median_in": 11.84, + "p90_in": 26.04, + "scale_error_pct": 2.71 } }, "one_known_distance": { "1-3m": { "pairs": 450000, - "failed_pct": 0.77, - "median_in": 2.77, - "p90_in": 10.03, - "scale_error_pct": 0.16 + "failed_pct": 2.79, + "median_in": 2.89, + "p90_in": 11.42, + "scale_error_pct": -0.06 }, "3-10m": { "pairs": 450000, - "failed_pct": 0.9, - "median_in": 5.47, - "p90_in": 25.64, - "scale_error_pct": 1.4 + "failed_pct": 2.88, + "median_in": 5.59, + "p90_in": 27.63, + "scale_error_pct": 1.23 } }, - "tape_calibration_success_pct": 99.3, + "tape_calibration_success_pct": 97.3, "groups": 6 }, "edges": { @@ -5287,35 +5299,35 @@ "1-3m": { "pairs": 18000, "failed_pct": 0.08, - "median_in": 6.37, - "p90_in": 14.58, - "scale_error_pct": 1.25 + "median_in": 6.43, + "p90_in": 14.55, + "scale_error_pct": 1.33 }, "3-10m": { "pairs": 17786, "failed_pct": 0.16, - "median_in": 7.78, + "median_in": 7.81, "p90_in": 18.54, - "scale_error_pct": -0.32 + "scale_error_pct": -0.31 } }, "one_known_distance": { "1-3m": { "pairs": 450000, - "failed_pct": 0.74, + "failed_pct": 2.74, "median_in": 4.03, - "p90_in": 15.29, - "scale_error_pct": 1.79 + "p90_in": 16.86, + "scale_error_pct": 1.42 }, "3-10m": { "pairs": 444650, - "failed_pct": 0.84, - "median_in": 6.14, - "p90_in": 25.18, - "scale_error_pct": 0.42 + "failed_pct": 2.86, + "median_in": 6.2, + "p90_in": 27.58, + "scale_error_pct": 0.11 } }, - "tape_calibration_success_pct": 99.3, + "tape_calibration_success_pct": 97.3, "groups": 6 } } @@ -5325,330 +5337,330 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 2.36, - "median_in": 6.53, - "p90_in": 19.28, - "scale_error_pct": -5.9 + "pairs": 21000, + "failed_pct": 0.34, + "median_in": 5.68, + "p90_in": 16.88, + "scale_error_pct": -5.62 }, "3-10m": { - "pairs": 24000, - "failed_pct": 4.59, - "median_in": 14.92, - "p90_in": 42.72, - "scale_error_pct": -7.6 + "pairs": 21000, + "failed_pct": 0.95, + "median_in": 13.21, + "p90_in": 36.84, + "scale_error_pct": -7.54 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 2.36, - "median_in": 3.34, - "p90_in": 24.77, - "scale_error_pct": -0.16 + "pairs": 525000, + "failed_pct": 0.34, + "median_in": 2.78, + "p90_in": 16.16, + "scale_error_pct": 0.01 }, "3-10m": { - "pairs": 600000, - "failed_pct": 4.59, - "median_in": 7.4, - "p90_in": 52.22, - "scale_error_pct": -1.07 + "pairs": 525000, + "failed_pct": 0.95, + "median_in": 5.83, + "p90_in": 29.5, + "scale_error_pct": -1.0 } }, "tape_calibration_success_pct": 100.0, - "groups": 8 + "groups": 7 }, "vertical interior": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 4.99, - "median_in": 6.38, - "p90_in": 27.82, - "scale_error_pct": -6.78 + "pairs": 18000, + "failed_pct": 3.8, + "median_in": 5.1, + "p90_in": 19.71, + "scale_error_pct": -6.37 }, "3-10m": { - "pairs": 21000, - "failed_pct": 6.39, - "median_in": 15.4, - "p90_in": 52.92, - "scale_error_pct": -7.44 + "pairs": 18000, + "failed_pct": 3.51, + "median_in": 13.74, + "p90_in": 45.14, + "scale_error_pct": -6.94 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 4.99, - "median_in": 2.52, - "p90_in": 34.67, - "scale_error_pct": -0.05 + "pairs": 450000, + "failed_pct": 3.8, + "median_in": 2.05, + "p90_in": 19.3, + "scale_error_pct": 0.2 }, "3-10m": { - "pairs": 525000, - "failed_pct": 6.39, - "median_in": 6.83, - "p90_in": 118.05, - "scale_error_pct": 0.02 + "pairs": 450000, + "failed_pct": 3.51, + "median_in": 5.44, + "p90_in": 44.97, + "scale_error_pct": 0.17 } }, "tape_calibration_success_pct": 100.0, - "groups": 7 + "groups": 6 }, "edges": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 3.9, - "median_in": 9.15, - "p90_in": 40.85, - "scale_error_pct": -1.42 + "pairs": 18000, + "failed_pct": 4.32, + "median_in": 8.0, + "p90_in": 26.64, + "scale_error_pct": -3.09 }, "3-10m": { - "pairs": 20786, - "failed_pct": 2.46, - "median_in": 19.8, - "p90_in": 142.77, - "scale_error_pct": -1.65 + "pairs": 17786, + "failed_pct": 2.86, + "median_in": 12.46, + "p90_in": 41.1, + "scale_error_pct": -2.35 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 3.9, - "median_in": 5.71, - "p90_in": 66.86, - "scale_error_pct": 3.98 + "pairs": 450000, + "failed_pct": 4.32, + "median_in": 4.62, + "p90_in": 36.73, + "scale_error_pct": 3.05 }, "3-10m": { - "pairs": 519650, - "failed_pct": 2.46, - "median_in": 10.79, - "p90_in": 171.59, - "scale_error_pct": 3.68 + "pairs": 444650, + "failed_pct": 2.86, + "median_in": 8.62, + "p90_in": 50.1, + "scale_error_pct": 2.72 } }, "tape_calibration_success_pct": 100.0, - "groups": 7 + "groups": 6 } }, "2": { "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 1.28, - "median_in": 5.71, - "p90_in": 17.72, - "scale_error_pct": -5.58 + "pairs": 21000, + "failed_pct": 0.32, + "median_in": 4.97, + "p90_in": 15.17, + "scale_error_pct": -5.23 }, "3-10m": { - "pairs": 24000, - "failed_pct": 3.58, - "median_in": 13.55, - "p90_in": 38.16, - "scale_error_pct": -7.37 + "pairs": 21000, + "failed_pct": 0.94, + "median_in": 12.35, + "p90_in": 35.07, + "scale_error_pct": -7.33 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 1.28, - "median_in": 3.61, - "p90_in": 21.63, - "scale_error_pct": 0.07 + "pairs": 525000, + "failed_pct": 0.89, + "median_in": 3.31, + "p90_in": 15.91, + "scale_error_pct": -0.08 }, "3-10m": { - "pairs": 600000, - "failed_pct": 3.58, - "median_in": 7.14, - "p90_in": 44.43, - "scale_error_pct": -0.39 + "pairs": 525000, + "failed_pct": 1.51, + "median_in": 6.35, + "p90_in": 31.0, + "scale_error_pct": -0.81 } }, - "tape_calibration_success_pct": 100.0, - "groups": 8 + "tape_calibration_success_pct": 99.4, + "groups": 7 }, "vertical interior": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 3.43, - "median_in": 5.84, - "p90_in": 20.74, - "scale_error_pct": -6.87 + "pairs": 18000, + "failed_pct": 3.75, + "median_in": 4.96, + "p90_in": 19.45, + "scale_error_pct": -6.48 }, "3-10m": { - "pairs": 21000, - "failed_pct": 4.98, - "median_in": 13.79, - "p90_in": 41.85, - "scale_error_pct": -7.34 + "pairs": 18000, + "failed_pct": 3.43, + "median_in": 12.93, + "p90_in": 41.53, + "scale_error_pct": -6.93 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 3.43, - "median_in": 2.84, - "p90_in": 27.74, + "pairs": 450000, + "failed_pct": 4.41, + "median_in": 2.61, + "p90_in": 21.99, "scale_error_pct": -0.24 }, "3-10m": { - "pairs": 525000, - "failed_pct": 4.98, - "median_in": 7.66, - "p90_in": 76.97, - "scale_error_pct": 0.25 + "pairs": 450000, + "failed_pct": 4.1, + "median_in": 6.61, + "p90_in": 54.6, + "scale_error_pct": 0.1 } }, - "tape_calibration_success_pct": 100.0, - "groups": 7 + "tape_calibration_success_pct": 99.3, + "groups": 6 }, "edges": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 3.84, - "median_in": 8.33, - "p90_in": 37.07, - "scale_error_pct": -1.85 + "pairs": 18000, + "failed_pct": 4.32, + "median_in": 7.47, + "p90_in": 25.37, + "scale_error_pct": -3.03 }, "3-10m": { - "pairs": 20786, - "failed_pct": 2.46, - "median_in": 18.8, - "p90_in": 103.09, - "scale_error_pct": -1.69 + "pairs": 17786, + "failed_pct": 2.86, + "median_in": 10.97, + "p90_in": 38.22, + "scale_error_pct": -2.39 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 3.84, - "median_in": 6.2, - "p90_in": 55.15, - "scale_error_pct": 4.59 + "pairs": 450000, + "failed_pct": 4.98, + "median_in": 4.97, + "p90_in": 38.79, + "scale_error_pct": 3.35 }, "3-10m": { - "pairs": 519650, - "failed_pct": 2.46, - "median_in": 11.27, - "p90_in": 132.07, - "scale_error_pct": 3.93 + "pairs": 444650, + "failed_pct": 3.53, + "median_in": 8.89, + "p90_in": 53.82, + "scale_error_pct": 2.56 } }, - "tape_calibration_success_pct": 100.0, - "groups": 7 + "tape_calibration_success_pct": 99.3, + "groups": 6 } }, "4": { "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.78, - "median_in": 5.6, - "p90_in": 18.1, - "scale_error_pct": -5.66 + "pairs": 21000, + "failed_pct": 0.18, + "median_in": 4.99, + "p90_in": 14.81, + "scale_error_pct": -5.32 }, "3-10m": { - "pairs": 24000, - "failed_pct": 2.98, - "median_in": 13.14, - "p90_in": 36.05, - "scale_error_pct": -7.01 + "pairs": 21000, + "failed_pct": 0.7, + "median_in": 12.1, + "p90_in": 33.64, + "scale_error_pct": -6.87 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 1.77, - "median_in": 3.99, - "p90_in": 25.19, - "scale_error_pct": 0.25 + "pairs": 525000, + "failed_pct": 3.04, + "median_in": 3.61, + "p90_in": 19.89, + "scale_error_pct": 0.09 }, "3-10m": { - "pairs": 600000, - "failed_pct": 3.98, - "median_in": 7.76, - "p90_in": 51.39, - "scale_error_pct": -0.17 + "pairs": 525000, + "failed_pct": 3.56, + "median_in": 6.99, + "p90_in": 38.47, + "scale_error_pct": -0.56 } }, - "tape_calibration_success_pct": 99.0, - "groups": 8 + "tape_calibration_success_pct": 97.1, + "groups": 7 }, "vertical interior": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 2.41, - "median_in": 5.64, - "p90_in": 20.81, - "scale_error_pct": -6.49 + "pairs": 18000, + "failed_pct": 2.69, + "median_in": 4.81, + "p90_in": 18.47, + "scale_error_pct": -5.65 }, "3-10m": { - "pairs": 21000, - "failed_pct": 4.04, - "median_in": 12.69, - "p90_in": 38.37, - "scale_error_pct": -6.5 + "pairs": 18000, + "failed_pct": 2.62, + "median_in": 11.7, + "p90_in": 38.06, + "scale_error_pct": -5.48 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 2.98, - "median_in": 3.27, - "p90_in": 30.94, - "scale_error_pct": 0.06 + "pairs": 450000, + "failed_pct": 5.36, + "median_in": 2.88, + "p90_in": 26.68, + "scale_error_pct": -0.05 }, "3-10m": { - "pairs": 525000, - "failed_pct": 4.6, - "median_in": 8.17, - "p90_in": 76.77, - "scale_error_pct": 0.42 + "pairs": 450000, + "failed_pct": 5.27, + "median_in": 7.24, + "p90_in": 70.92, + "scale_error_pct": 0.17 } }, - "tape_calibration_success_pct": 99.4, - "groups": 7 + "tape_calibration_success_pct": 97.3, + "groups": 6 }, "edges": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 1.96, - "median_in": 7.32, - "p90_in": 23.66, - "scale_error_pct": -1.72 + "pairs": 18000, + "failed_pct": 2.26, + "median_in": 6.68, + "p90_in": 19.37, + "scale_error_pct": -2.61 }, "3-10m": { - "pairs": 20786, - "failed_pct": 1.59, - "median_in": 14.85, - "p90_in": 74.39, - "scale_error_pct": -2.01 + "pairs": 17786, + "failed_pct": 1.86, + "median_in": 9.9, + "p90_in": 34.64, + "scale_error_pct": -2.75 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 2.51, - "median_in": 6.01, - "p90_in": 44.59, - "scale_error_pct": 4.3 + "pairs": 450000, + "failed_pct": 4.91, + "median_in": 5.13, + "p90_in": 34.13, + "scale_error_pct": 3.24 }, "3-10m": { - "pairs": 519650, - "failed_pct": 2.17, - "median_in": 10.7, - "p90_in": 109.42, - "scale_error_pct": 3.46 + "pairs": 444650, + "failed_pct": 4.55, + "median_in": 8.82, + "p90_in": 62.5, + "scale_error_pct": 2.24 } }, - "tape_calibration_success_pct": 99.4, - "groups": 7 + "tape_calibration_success_pct": 97.3, + "groups": 6 } }, "8": { @@ -5672,56 +5684,56 @@ "one_known_distance": { "1-3m": { "pairs": 525000, - "failed_pct": 4.63, - "median_in": 3.57, - "p90_in": 20.15, - "scale_error_pct": 0.52 + "failed_pct": 5.2, + "median_in": 3.61, + "p90_in": 21.45, + "scale_error_pct": 0.51 }, "3-10m": { "pairs": 525000, - "failed_pct": 4.8, - "median_in": 6.46, - "p90_in": 38.85, - "scale_error_pct": 0.05 + "failed_pct": 5.38, + "median_in": 6.55, + "p90_in": 41.47, + "scale_error_pct": 0.02 } }, - "tape_calibration_success_pct": 95.4, + "tape_calibration_success_pct": 94.9, "groups": 7 }, "vertical interior": { "none": { "1-3m": { "pairs": 18000, - "failed_pct": 0.44, - "median_in": 4.85, - "p90_in": 17.73, - "scale_error_pct": -5.78 + "failed_pct": 0.49, + "median_in": 4.82, + "p90_in": 17.6, + "scale_error_pct": -5.77 }, "3-10m": { "pairs": 18000, - "failed_pct": 0.66, - "median_in": 10.36, - "p90_in": 36.45, - "scale_error_pct": -5.75 + "failed_pct": 0.73, + "median_in": 10.76, + "p90_in": 36.58, + "scale_error_pct": -5.61 } }, "one_known_distance": { "1-3m": { "pairs": 450000, - "failed_pct": 3.1, - "median_in": 2.96, - "p90_in": 20.58, - "scale_error_pct": 0.18 + "failed_pct": 3.82, + "median_in": 3.03, + "p90_in": 21.97, + "scale_error_pct": 0.21 }, "3-10m": { "pairs": 450000, - "failed_pct": 3.32, - "median_in": 6.82, - "p90_in": 38.05, - "scale_error_pct": 0.38 + "failed_pct": 4.05, + "median_in": 7.15, + "p90_in": 44.67, + "scale_error_pct": 0.48 } }, - "tape_calibration_success_pct": 97.3, + "tape_calibration_success_pct": 96.7, "groups": 6 }, "edges": { @@ -5729,35 +5741,35 @@ "1-3m": { "pairs": 18000, "failed_pct": 1.08, - "median_in": 6.08, - "p90_in": 18.96, - "scale_error_pct": -2.93 + "median_in": 6.06, + "p90_in": 18.93, + "scale_error_pct": -2.88 }, "3-10m": { "pairs": 17786, "failed_pct": 1.07, - "median_in": 8.43, - "p90_in": 33.23, - "scale_error_pct": -3.35 + "median_in": 8.57, + "p90_in": 33.36, + "scale_error_pct": -3.34 } }, "one_known_distance": { "1-3m": { "pairs": 450000, - "failed_pct": 3.73, - "median_in": 5.0, - "p90_in": 26.84, - "scale_error_pct": 3.56 + "failed_pct": 4.4, + "median_in": 5.06, + "p90_in": 28.19, + "scale_error_pct": 3.59 }, "3-10m": { "pairs": 444650, - "failed_pct": 3.76, - "median_in": 8.44, - "p90_in": 47.01, - "scale_error_pct": 2.49 + "failed_pct": 4.44, + "median_in": 8.57, + "p90_in": 50.97, + "scale_error_pct": 2.5 } }, - "tape_calibration_success_pct": 97.3, + "tape_calibration_success_pct": 96.7, "groups": 6 } } @@ -5788,16 +5800,16 @@ "1-3m": { "pairs": 3375000, "failed_pct": 0.29, - "median_in": 2.46, - "p90_in": 9.72, - "scale_error_pct": -0.05 + "median_in": 2.47, + "p90_in": 9.69, + "scale_error_pct": -0.07 }, "3-10m": { "pairs": 3375000, "failed_pct": 0.35, - "median_in": 5.86, - "p90_in": 25.35, - "scale_error_pct": -0.19 + "median_in": 5.87, + "p90_in": 25.27, + "scale_error_pct": -0.21 } }, "tape_calibration_success_pct": 99.8, @@ -5808,32 +5820,32 @@ "1-3m": { "pairs": 135000, "failed_pct": 0.18, - "median_in": 6.55, - "p90_in": 19.19, - "scale_error_pct": 0.33 + "median_in": 6.56, + "p90_in": 19.2, + "scale_error_pct": 0.34 }, "3-10m": { - "pairs": 133158, - "failed_pct": 0.33, - "median_in": 17.8, - "p90_in": 56.07, - "scale_error_pct": -0.12 + "pairs": 133047, + "failed_pct": 0.3, + "median_in": 17.82, + "p90_in": 56.14, + "scale_error_pct": -0.16 } }, "one_known_distance": { "1-3m": { "pairs": 3375000, - "failed_pct": 0.36, + "failed_pct": 0.35, "median_in": 2.47, - "p90_in": 9.67, - "scale_error_pct": -0.0 + "p90_in": 9.62, + "scale_error_pct": -0.01 }, "3-10m": { - "pairs": 3328950, - "failed_pct": 0.51, + "pairs": 3326175, + "failed_pct": 0.48, "median_in": 5.83, - "p90_in": 25.36, - "scale_error_pct": -0.05 + "p90_in": 25.19, + "scale_error_pct": -0.08 } }, "tape_calibration_success_pct": 99.8, @@ -5845,14 +5857,14 @@ "pairs": 135000, "failed_pct": 0.64, "median_in": 8.01, - "p90_in": 24.92, + "p90_in": 24.93, "scale_error_pct": 4.29 }, "3-10m": { "pairs": 132648, "failed_pct": 0.3, "median_in": 17.1, - "p90_in": 63.42, + "p90_in": 63.41, "scale_error_pct": 2.68 } }, @@ -5861,15 +5873,15 @@ "pairs": 3375000, "failed_pct": 0.81, "median_in": 4.44, - "p90_in": 25.05, + "p90_in": 25.14, "scale_error_pct": 1.11 }, "3-10m": { "pairs": 3316200, "failed_pct": 0.48, - "median_in": 7.23, + "median_in": 7.22, "p90_in": 37.39, - "scale_error_pct": 0.53 + "scale_error_pct": 0.52 } }, "tape_calibration_success_pct": 99.8, @@ -5897,56 +5909,56 @@ "one_known_distance": { "1-3m": { "pairs": 3375000, - "failed_pct": 3.28, - "median_in": 1.58, - "p90_in": 9.57, - "scale_error_pct": -0.03 + "failed_pct": 2.96, + "median_in": 1.56, + "p90_in": 9.26, + "scale_error_pct": -0.05 }, "3-10m": { "pairs": 3375000, - "failed_pct": 4.01, - "median_in": 3.0, - "p90_in": 17.8, - "scale_error_pct": -0.01 + "failed_pct": 3.72, + "median_in": 2.97, + "p90_in": 16.87, + "scale_error_pct": -0.04 } }, - "tape_calibration_success_pct": 97.9, + "tape_calibration_success_pct": 98.2, "photos": 45 }, "vertical interior": { "none": { "1-3m": { "pairs": 102000, - "failed_pct": 3.4, - "median_in": 7.12, - "p90_in": 24.74, - "scale_error_pct": 1.01 + "failed_pct": 2.78, + "median_in": 7.1, + "p90_in": 23.9, + "scale_error_pct": 1.15 }, "3-10m": { - "pairs": 99955, - "failed_pct": 7.75, - "median_in": 15.62, - "p90_in": 64.2, - "scale_error_pct": 1.57 + "pairs": 99972, + "failed_pct": 6.98, + "median_in": 15.64, + "p90_in": 53.97, + "scale_error_pct": 1.85 } }, "one_known_distance": { "1-3m": { "pairs": 2550000, - "failed_pct": 5.38, - "median_in": 1.8, - "p90_in": 15.11, - "scale_error_pct": -0.0 + "failed_pct": 4.55, + "median_in": 1.75, + "p90_in": 13.61, + "scale_error_pct": -0.02 }, "3-10m": { - "pairs": 2498875, - "failed_pct": 9.09, - "median_in": 3.63, - "p90_in": 58.16, - "scale_error_pct": 0.27 + "pairs": 2499300, + "failed_pct": 8.26, + "median_in": 3.59, + "p90_in": 44.67, + "scale_error_pct": 0.26 } }, - "tape_calibration_success_pct": 97.2, + "tape_calibration_success_pct": 97.6, "photos": 34 }, "edges": { @@ -5955,34 +5967,34 @@ "pairs": 102000, "failed_pct": 0.97, "median_in": 8.34, - "p90_in": 31.78, - "scale_error_pct": 3.6 + "p90_in": 31.69, + "scale_error_pct": 3.61 }, "3-10m": { "pairs": 99802, - "failed_pct": 0.71, + "failed_pct": 0.7, "median_in": 14.99, - "p90_in": 52.99, + "p90_in": 53.24, "scale_error_pct": 4.15 } }, "one_known_distance": { "1-3m": { "pairs": 2550000, - "failed_pct": 3.74, - "median_in": 3.67, - "p90_in": 26.88, - "scale_error_pct": 0.61 + "failed_pct": 3.27, + "median_in": 3.63, + "p90_in": 25.3, + "scale_error_pct": 0.6 }, "3-10m": { "pairs": 2495050, - "failed_pct": 3.58, - "median_in": 5.82, - "p90_in": 36.78, - "scale_error_pct": 0.85 + "failed_pct": 3.09, + "median_in": 5.79, + "p90_in": 33.76, + "scale_error_pct": 0.82 } }, - "tape_calibration_success_pct": 97.2, + "tape_calibration_success_pct": 97.6, "photos": 34 } } @@ -6011,14 +6023,14 @@ "pairs": 3375000, "failed_pct": 1.09, "median_in": 3.44, - "p90_in": 15.3, + "p90_in": 15.18, "scale_error_pct": -0.07 }, "3-10m": { "pairs": 3375000, "failed_pct": 1.41, "median_in": 8.63, - "p90_in": 38.01, + "p90_in": 37.72, "scale_error_pct": -0.69 } }, @@ -6029,33 +6041,33 @@ "none": { "1-3m": { "pairs": 135000, - "failed_pct": 0.83, - "median_in": 8.08, - "p90_in": 26.71, - "scale_error_pct": -10.0 + "failed_pct": 0.82, + "median_in": 8.07, + "p90_in": 26.62, + "scale_error_pct": -9.98 }, "3-10m": { - "pairs": 133158, - "failed_pct": 1.45, + "pairs": 133047, + "failed_pct": 1.46, "median_in": 22.6, - "p90_in": 78.45, - "scale_error_pct": -11.46 + "p90_in": 78.55, + "scale_error_pct": -11.47 } }, "one_known_distance": { "1-3m": { "pairs": 3375000, - "failed_pct": 1.44, - "median_in": 3.47, - "p90_in": 15.35, + "failed_pct": 1.43, + "median_in": 3.46, + "p90_in": 15.21, "scale_error_pct": -0.68 }, "3-10m": { - "pairs": 3328950, - "failed_pct": 2.05, - "median_in": 8.88, - "p90_in": 41.36, - "scale_error_pct": -1.07 + "pairs": 3326175, + "failed_pct": 2.07, + "median_in": 8.86, + "p90_in": 41.17, + "scale_error_pct": -1.04 } }, "tape_calibration_success_pct": 99.4, @@ -6074,7 +6086,7 @@ "pairs": 132648, "failed_pct": 1.54, "median_in": 16.94, - "p90_in": 93.0, + "p90_in": 92.99, "scale_error_pct": -4.39 } }, @@ -6082,15 +6094,15 @@ "1-3m": { "pairs": 3375000, "failed_pct": 3.23, - "median_in": 8.74, - "p90_in": 61.86, - "scale_error_pct": 6.66 + "median_in": 8.75, + "p90_in": 61.85, + "scale_error_pct": 6.69 }, "3-10m": { "pairs": 3316200, "failed_pct": 2.13, - "median_in": 13.65, - "p90_in": 95.38, + "median_in": 13.68, + "p90_in": 95.21, "scale_error_pct": 3.6 } }, @@ -6119,92 +6131,92 @@ "one_known_distance": { "1-3m": { "pairs": 3375000, - "failed_pct": 5.89, + "failed_pct": 5.72, "median_in": 2.91, - "p90_in": 25.31, - "scale_error_pct": -0.12 + "p90_in": 24.39, + "scale_error_pct": -0.1 }, "3-10m": { "pairs": 3375000, - "failed_pct": 7.29, - "median_in": 6.09, - "p90_in": 62.64, - "scale_error_pct": -0.71 + "failed_pct": 7.1, + "median_in": 6.05, + "p90_in": 57.92, + "scale_error_pct": -0.69 } }, - "tape_calibration_success_pct": 96.6, + "tape_calibration_success_pct": 97.0, "photos": 45 }, "vertical interior": { "none": { "1-3m": { "pairs": 102000, - "failed_pct": 7.74, + "failed_pct": 7.3, "median_in": 6.23, - "p90_in": 35.3, + "p90_in": 33.25, "scale_error_pct": -7.17 }, "3-10m": { - "pairs": 99955, - "failed_pct": 12.84, - "median_in": 14.58, - "p90_in": Infinity, - "scale_error_pct": -7.31 + "pairs": 99972, + "failed_pct": 12.36, + "median_in": 14.66, + "p90_in": "failed", + "scale_error_pct": -7.3 } }, "one_known_distance": { "1-3m": { "pairs": 2550000, - "failed_pct": 9.84, - "median_in": 2.93, - "p90_in": 171.29, - "scale_error_pct": 0.06 + "failed_pct": 9.56, + "median_in": 2.95, + "p90_in": 74.45, + "scale_error_pct": 0.13 }, "3-10m": { - "pairs": 2498875, - "failed_pct": 14.35, - "median_in": 6.65, - "p90_in": Infinity, - "scale_error_pct": 0.22 + "pairs": 2499300, + "failed_pct": 13.98, + "median_in": 6.76, + "p90_in": "failed", + "scale_error_pct": 0.29 } }, - "tape_calibration_success_pct": 95.5, + "tape_calibration_success_pct": 96.0, "photos": 34 }, "edges": { "none": { "1-3m": { "pairs": 102000, - "failed_pct": 4.27, + "failed_pct": 4.28, "median_in": 6.25, - "p90_in": 34.96, - "scale_error_pct": -1.23 + "p90_in": 34.87, + "scale_error_pct": -1.25 }, "3-10m": { "pairs": 99802, - "failed_pct": 3.11, - "median_in": 9.51, - "p90_in": 73.11, + "failed_pct": 3.12, + "median_in": 9.52, + "p90_in": 73.73, "scale_error_pct": -0.15 } }, "one_known_distance": { "1-3m": { "pairs": 2550000, - "failed_pct": 8.24, - "median_in": 7.1, - "p90_in": 117.42, + "failed_pct": 7.8, + "median_in": 7.09, + "p90_in": 99.4, "scale_error_pct": 5.52 }, "3-10m": { "pairs": 2495050, - "failed_pct": 7.46, - "median_in": 12.41, - "p90_in": 185.2, - "scale_error_pct": 5.94 + "failed_pct": 7.01, + "median_in": 12.49, + "p90_in": 166.74, + "scale_error_pct": 5.96 } }, - "tape_calibration_success_pct": 95.5, + "tape_calibration_success_pct": 96.0, "photos": 34 } } diff --git a/experiments/evals/results/eth3d_recon.md b/experiments/evals/results/eth3d_recon.md index 9e1dd4a24..010cca170 100644 --- a/experiments/evals/results/eth3d_recon.md +++ b/experiments/evals/results/eth3d_recon.md @@ -6,65 +6,65 @@ | Method | Views | Model scale: 1-3 m | 3-10 m | Scale error: 1-3 m | 3-10 m | One taped distance: 1-3 m | 3-10 m | Tape calibrated | | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| MoGe-2, one photo (every photo) | 1 | 6.3 / 13.2 | 21.9 / 52.5 | -6.6% | -7.9% | 2.9 / 12.3 | 9.2 / 45.4 | 100% | -| Depth Anything 3 metric, one photo (every photo) | 1 | 10.7 / 20.1 | 36.6 / 78.0 | -13.8% | -15.1% | 3.8 / 14.8 | 12.0 / 56.5 | 100% | -| scan rendered as depth (evaluation floor) | 1 | 0.2 / 1.6 | 0.6 / 2.9 | -0.2% | -0.2% | 0.3 / 2.7 | 0.7 / 6.2 | 100% | -| scan rendered as depth (evaluation floor) | 2 | 0.2 / 1.6 | 0.6 / 3.0 | -0.2% | -0.2% | 0.3 / 2.8 | 0.7 / 6.2 | 100% | -| scan rendered as depth (evaluation floor) | 4 | 0.2 / 1.6 | 0.6 / 2.8 | -0.2% | -0.2% | 0.3 / 2.9 | 0.7 / 6.6 | 100% | -| scan rendered as depth (evaluation floor) | 8 | 0.2 / 1.6 | 0.7 / 2.8 | -0.2% | -0.2% | 0.3 / 2.9 | 0.8 / 6.7 | 100% | -| MoGe-2 per photo + true poses | 1 | 5.6 / 11.6 | 20.2 / 47.3 | -5.6% | -7.0% | 3.0 / 12.1 | 10.3 / 44.8 | 100% | -| MoGe-2 per photo + true poses | 2 | 5.9 / 12.0 | 20.4 / 47.7 | -6.2% | -7.3% | 3.0 / 12.6 | 9.8 / 43.6 | 100% | -| MoGe-2 per photo + true poses | 4 | 5.7 / 11.6 | 19.2 / 46.9 | -5.9% | -7.0% | 3.3 / 12.3 | 10.8 / 42.1 | 100% | -| MoGe-2 per photo + true poses | 8 | 5.9 / 11.8 | 19.9 / 46.1 | -6.0% | -7.2% | 3.5 / 14.4 | 11.0 / 49.0 | 98% | -| Depth Anything 3 metric per photo + true poses | 1 | 11.4 / 20.9 | 39.2 / 81.5 | -14.8% | -16.3% | 3.6 / 13.9 | 12.1 / 54.7 | 100% | -| Depth Anything 3 metric per photo + true poses | 2 | 11.3 / 21.2 | 38.1 / 82.7 | -14.3% | -15.9% | 4.0 / 16.9 | 13.3 / 59.2 | 100% | -| Depth Anything 3 metric per photo + true poses | 4 | 11.0 / 20.6 | 36.6 / 77.7 | -14.0% | -15.4% | 4.1 / 15.8 | 13.5 / 55.5 | 98% | -| Depth Anything 3 metric per photo + true poses | 8 | 10.7 / 19.9 | 36.0 / 77.0 | -13.2% | -15.1% | 4.2 / 15.4 | 13.7 / 56.0 | 98% | +| MoGe-2, one photo (every photo) | 1 | 6.3 / 13.2 | 21.9 / 52.5 | -6.6% | -7.9% | 2.9 / 12.2 | 9.2 / 45.1 | 100% | +| Depth Anything 3 metric, one photo (every photo) | 1 | 10.7 / 20.1 | 36.6 / 78.0 | -13.8% | -15.1% | 3.8 / 14.6 | 11.9 / 56.1 | 100% | +| scan rendered as depth (evaluation floor) | 1 | 0.2 / 1.6 | 0.6 / 2.9 | -0.2% | -0.2% | 0.3 / 2.6 | 0.7 / 6.0 | 100% | +| scan rendered as depth (evaluation floor) | 2 | 0.2 / 1.6 | 0.6 / 3.0 | -0.2% | -0.2% | 0.3 / 2.7 | 0.8 / 6.0 | 100% | +| scan rendered as depth (evaluation floor) | 4 | 0.2 / 1.6 | 0.6 / 2.8 | -0.2% | -0.2% | 0.3 / 2.8 | 0.8 / 6.3 | 100% | +| scan rendered as depth (evaluation floor) | 8 | 0.2 / 1.6 | 0.7 / 2.8 | -0.2% | -0.2% | 0.3 / 2.8 | 0.8 / 6.4 | 100% | +| MoGe-2 per photo + true poses | 1 | 5.6 / 11.6 | 20.2 / 47.3 | -5.6% | -7.0% | 3.0 / 11.9 | 10.1 / 44.2 | 100% | +| MoGe-2 per photo + true poses | 2 | 5.9 / 12.0 | 20.4 / 47.7 | -6.2% | -7.3% | 3.0 / 12.5 | 9.6 / 43.3 | 100% | +| MoGe-2 per photo + true poses | 4 | 5.7 / 11.6 | 19.2 / 46.9 | -5.9% | -7.0% | 3.3 / 12.3 | 10.6 / 41.9 | 100% | +| MoGe-2 per photo + true poses | 8 | 5.9 / 11.8 | 19.9 / 46.1 | -6.0% | -7.2% | 3.5 / 14.4 | 10.9 / 48.8 | 98% | +| Depth Anything 3 metric per photo + true poses | 1 | 11.4 / 20.9 | 39.2 / 81.5 | -14.8% | -16.3% | 3.5 / 13.7 | 11.6 / 54.8 | 100% | +| Depth Anything 3 metric per photo + true poses | 2 | 11.3 / 21.2 | 38.1 / 82.7 | -14.3% | -15.9% | 3.9 / 16.9 | 12.8 / 59.3 | 100% | +| Depth Anything 3 metric per photo + true poses | 4 | 11.0 / 20.6 | 36.6 / 77.7 | -14.0% | -15.4% | 4.0 / 15.7 | 13.1 / 55.5 | 98% | +| Depth Anything 3 metric per photo + true poses | 8 | 10.7 / 19.9 | 36.0 / 77.0 | -13.2% | -15.1% | 4.2 / 15.3 | 13.4 / 55.8 | 98% | ## facade, all points, vertical interior | Method | Views | Model scale: 1-3 m | 3-10 m | Scale error: 1-3 m | 3-10 m | One taped distance: 1-3 m | 3-10 m | Tape calibrated | | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| MoGe-2, one photo (every photo) | 1 | 6.3 / 12.9 | 21.6 / 54.1 | -7.0% | -8.1% | 2.8 / 11.3 | 8.9 / 46.1 | 100% | -| Depth Anything 3 metric, one photo (every photo) | 1 | 10.8 / 19.9 | 37.0 / 78.1 | -14.1% | -15.5% | 3.6 / 13.7 | 11.5 / 56.8 | 100% | -| scan rendered as depth (evaluation floor) | 1 | 0.2 / 0.9 | 0.6 / 2.0 | -0.2% | -0.2% | 0.2 / 2.2 | 0.5 / 5.6 | 100% | -| scan rendered as depth (evaluation floor) | 2 | 0.2 / 1.0 | 0.6 / 2.0 | -0.2% | -0.2% | 0.2 / 2.3 | 0.5 / 5.7 | 100% | -| scan rendered as depth (evaluation floor) | 4 | 0.2 / 1.0 | 0.6 / 1.9 | -0.2% | -0.2% | 0.2 / 2.5 | 0.5 / 6.1 | 100% | -| scan rendered as depth (evaluation floor) | 8 | 0.2 / 1.0 | 0.6 / 1.9 | -0.2% | -0.2% | 0.2 / 2.6 | 0.6 / 6.1 | 100% | -| MoGe-2 per photo + true poses | 1 | 5.5 / 10.9 | 19.3 / 46.2 | -5.9% | -7.1% | 2.8 / 10.9 | 9.7 / 43.9 | 100% | -| MoGe-2 per photo + true poses | 2 | 5.9 / 11.2 | 19.6 / 46.7 | -6.5% | -7.4% | 2.8 / 11.3 | 9.2 / 42.7 | 100% | -| MoGe-2 per photo + true poses | 4 | 5.6 / 10.9 | 18.3 / 46.5 | -6.2% | -7.0% | 3.1 / 11.1 | 10.3 / 41.0 | 100% | -| MoGe-2 per photo + true poses | 8 | 5.9 / 11.2 | 19.1 / 46.5 | -6.3% | -7.3% | 3.2 / 13.0 | 10.6 / 48.5 | 98% | -| Depth Anything 3 metric per photo + true poses | 1 | 11.7 / 20.7 | 39.2 / 79.3 | -15.3% | -17.0% | 3.4 / 12.5 | 11.3 / 51.2 | 100% | -| Depth Anything 3 metric per photo + true poses | 2 | 11.6 / 21.1 | 38.4 / 81.0 | -14.9% | -16.6% | 3.7 / 15.3 | 12.5 / 56.1 | 100% | -| Depth Anything 3 metric per photo + true poses | 4 | 11.2 / 20.4 | 37.1 / 77.7 | -14.6% | -16.2% | 3.9 / 14.4 | 12.9 / 53.9 | 98% | -| Depth Anything 3 metric per photo + true poses | 8 | 10.8 / 19.7 | 36.2 / 77.1 | -13.8% | -15.6% | 4.0 / 14.2 | 13.1 / 54.8 | 98% | +| MoGe-2, one photo (every photo) | 1 | 6.3 / 12.9 | 21.6 / 54.0 | -7.0% | -8.2% | 2.7 / 11.2 | 8.9 / 45.9 | 100% | +| Depth Anything 3 metric, one photo (every photo) | 1 | 10.8 / 19.8 | 37.1 / 78.1 | -14.1% | -15.6% | 3.5 / 13.5 | 11.4 / 56.5 | 100% | +| scan rendered as depth (evaluation floor) | 1 | 0.2 / 1.0 | 0.6 / 2.0 | -0.2% | -0.2% | 0.2 / 2.1 | 0.5 / 5.4 | 100% | +| scan rendered as depth (evaluation floor) | 2 | 0.2 / 1.0 | 0.6 / 2.0 | -0.2% | -0.2% | 0.2 / 2.2 | 0.5 / 5.4 | 100% | +| scan rendered as depth (evaluation floor) | 4 | 0.2 / 1.0 | 0.6 / 1.8 | -0.2% | -0.2% | 0.2 / 2.5 | 0.6 / 5.8 | 100% | +| scan rendered as depth (evaluation floor) | 8 | 0.2 / 1.0 | 0.6 / 1.9 | -0.2% | -0.2% | 0.2 / 2.5 | 0.6 / 5.8 | 100% | +| MoGe-2 per photo + true poses | 1 | 5.6 / 11.0 | 19.4 / 46.3 | -5.9% | -7.1% | 2.7 / 10.7 | 9.6 / 43.2 | 100% | +| MoGe-2 per photo + true poses | 2 | 5.9 / 11.3 | 19.6 / 46.7 | -6.5% | -7.4% | 2.7 / 11.1 | 9.1 / 42.5 | 100% | +| MoGe-2 per photo + true poses | 4 | 5.6 / 11.0 | 18.5 / 46.9 | -6.2% | -7.1% | 3.1 / 11.0 | 10.2 / 40.9 | 100% | +| MoGe-2 per photo + true poses | 8 | 5.8 / 11.3 | 19.2 / 46.5 | -6.3% | -7.4% | 3.2 / 13.0 | 10.5 / 48.1 | 98% | +| Depth Anything 3 metric per photo + true poses | 1 | 11.7 / 20.6 | 39.5 / 79.3 | -15.3% | -17.0% | 3.3 / 12.0 | 10.7 / 51.1 | 100% | +| Depth Anything 3 metric per photo + true poses | 2 | 11.6 / 21.0 | 38.6 / 81.0 | -14.8% | -16.6% | 3.6 / 15.3 | 12.0 / 56.3 | 100% | +| Depth Anything 3 metric per photo + true poses | 4 | 11.2 / 20.5 | 37.2 / 77.4 | -14.6% | -16.2% | 3.8 / 14.3 | 12.5 / 54.2 | 98% | +| Depth Anything 3 metric per photo + true poses | 8 | 10.9 / 19.8 | 36.5 / 77.1 | -13.9% | -15.7% | 4.0 / 14.1 | 12.8 / 54.6 | 98% | ## facade, all points, edges | Method | Views | Model scale: 1-3 m | 3-10 m | Scale error: 1-3 m | 3-10 m | One taped distance: 1-3 m | 3-10 m | Tape calibrated | | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| MoGe-2, one photo (every photo) | 1 | 5.7 / 20.8 | 15.8 / 48.4 | -2.9% | -4.7% | 4.4 / 23.8 | 11.7 / 50.7 | 100% | -| Depth Anything 3 metric, one photo (every photo) | 1 | 9.5 / 27.7 | 26.4 / 68.8 | -8.9% | -10.4% | 6.9 / 41.5 | 17.5 / 77.2 | 100% | -| scan rendered as depth (evaluation floor) | 1 | 0.5 / 3.4 | 1.0 / 5.4 | -0.3% | -0.2% | 0.6 / 4.2 | 1.4 / 7.7 | 100% | -| scan rendered as depth (evaluation floor) | 2 | 0.5 / 3.6 | 1.0 / 5.7 | -0.3% | -0.2% | 0.7 / 4.3 | 1.4 / 7.8 | 100% | -| scan rendered as depth (evaluation floor) | 4 | 0.5 / 3.5 | 1.0 / 5.3 | -0.2% | -0.2% | 0.6 / 4.2 | 1.4 / 7.9 | 100% | -| scan rendered as depth (evaluation floor) | 8 | 0.5 / 3.5 | 1.0 / 5.3 | -0.3% | -0.2% | 0.7 / 4.3 | 1.4 / 8.2 | 100% | +| MoGe-2, one photo (every photo) | 1 | 5.7 / 20.8 | 15.8 / 48.4 | -2.9% | -4.7% | 4.5 / 23.8 | 11.7 / 50.5 | 100% | +| Depth Anything 3 metric, one photo (every photo) | 1 | 9.5 / 27.7 | 26.4 / 68.8 | -8.9% | -10.4% | 6.9 / 41.4 | 17.4 / 76.7 | 100% | +| scan rendered as depth (evaluation floor) | 1 | 0.5 / 3.4 | 1.0 / 5.4 | -0.3% | -0.2% | 0.6 / 4.0 | 1.4 / 7.5 | 100% | +| scan rendered as depth (evaluation floor) | 2 | 0.5 / 3.6 | 1.0 / 5.7 | -0.3% | -0.2% | 0.7 / 4.2 | 1.4 / 7.6 | 100% | +| scan rendered as depth (evaluation floor) | 4 | 0.5 / 3.5 | 1.0 / 5.3 | -0.2% | -0.2% | 0.6 / 4.1 | 1.4 / 7.7 | 100% | +| scan rendered as depth (evaluation floor) | 8 | 0.5 / 3.5 | 1.0 / 5.3 | -0.3% | -0.2% | 0.7 / 4.2 | 1.4 / 8.0 | 100% | | MoGe-2 per photo + true poses | 1 | 5.3 / 25.3 | 15.1 / 51.1 | -1.8% | -3.2% | 4.9 / 28.2 | 14.0 / 55.4 | 100% | -| MoGe-2 per photo + true poses | 2 | 5.5 / 26.1 | 16.0 / 48.5 | -2.3% | -4.0% | 5.0 / 30.5 | 13.8 / 55.8 | 100% | -| MoGe-2 per photo + true poses | 4 | 5.1 / 24.6 | 14.8 / 45.4 | -2.3% | -3.9% | 5.2 / 28.5 | 13.7 / 52.7 | 100% | -| MoGe-2 per photo + true poses | 8 | 5.3 / 28.2 | 15.3 / 53.2 | -2.4% | -4.0% | 5.7 / 36.0 | 14.7 / 65.2 | 98% | -| Depth Anything 3 metric per photo + true poses | 1 | 9.8 / 29.2 | 30.4 / 71.9 | -8.5% | -11.6% | 6.8 / 42.7 | 19.5 / 82.0 | 100% | -| Depth Anything 3 metric per photo + true poses | 2 | 9.2 / 29.1 | 29.4 / 70.8 | -8.1% | -10.8% | 8.0 / 45.5 | 21.9 / 87.7 | 100% | -| Depth Anything 3 metric per photo + true poses | 4 | 9.0 / 26.5 | 27.8 / 67.2 | -7.9% | -10.3% | 7.6 / 44.5 | 20.6 / 83.5 | 98% | -| Depth Anything 3 metric per photo + true poses | 8 | 9.3 / 28.5 | 28.2 / 69.6 | -8.4% | -10.3% | 7.4 / 43.7 | 19.7 / 78.5 | 98% | +| MoGe-2 per photo + true poses | 2 | 5.5 / 26.1 | 16.0 / 48.5 | -2.3% | -4.0% | 5.0 / 30.6 | 13.8 / 55.9 | 100% | +| MoGe-2 per photo + true poses | 4 | 5.1 / 24.6 | 14.8 / 45.4 | -2.3% | -3.9% | 5.2 / 28.6 | 13.8 / 52.8 | 100% | +| MoGe-2 per photo + true poses | 8 | 5.3 / 28.2 | 15.3 / 53.2 | -2.4% | -4.0% | 5.7 / 36.1 | 14.8 / 65.2 | 98% | +| Depth Anything 3 metric per photo + true poses | 1 | 9.8 / 29.2 | 30.4 / 71.9 | -8.5% | -11.6% | 6.7 / 42.7 | 19.1 / 81.8 | 100% | +| Depth Anything 3 metric per photo + true poses | 2 | 9.2 / 29.1 | 29.4 / 70.8 | -8.1% | -10.8% | 7.8 / 45.5 | 21.6 / 87.5 | 100% | +| Depth Anything 3 metric per photo + true poses | 4 | 9.0 / 26.5 | 27.8 / 67.2 | -7.9% | -10.3% | 7.5 / 44.4 | 20.4 / 83.2 | 98% | +| Depth Anything 3 metric per photo + true poses | 8 | 9.3 / 28.5 | 28.2 / 69.6 | -8.4% | -10.3% | 7.3 / 43.7 | 19.6 / 78.2 | 98% | ## facade, within 6 m, surface interior | Method | Views | Model scale: 1-3 m | 3-10 m | Scale error: 1-3 m | 3-10 m | One taped distance: 1-3 m | 3-10 m | Tape calibrated | | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| MoGe-2, one photo (every photo) | 1 | 7.7 / 20.0 | 16.9 / 31.6 | +10.3% | +11.7% | 0.9 / 6.5 | 2.2 / 12.1 | 100% | -| Depth Anything 3 metric, one photo (every photo) | 1 | 6.5 / 18.2 | 16.4 / 36.5 | -8.2% | -10.9% | 2.1 / 10.9 | 4.5 / 19.7 | 100% | +| MoGe-2, one photo (every photo) | 1 | 7.7 / 20.0 | 16.9 / 31.6 | +10.3% | +11.7% | 0.9 / 6.5 | 2.2 / 12.3 | 100% | +| Depth Anything 3 metric, one photo (every photo) | 1 | 6.5 / 18.2 | 16.4 / 36.5 | -8.2% | -10.9% | 2.1 / 10.9 | 4.6 / 19.9 | 100% | | scan rendered as depth (evaluation floor) | 1 | 0.8 / 1.4 | 1.6 / 2.1 | -1.3% | -1.1% | 0.2 / 1.6 | 0.5 / 2.5 | 100% | | scan rendered as depth (evaluation floor) | 2 | 0.8 / 1.4 | 1.6 / 2.1 | -1.3% | -1.2% | 0.3 / 1.7 | 0.5 / 2.6 | 100% | | scan rendered as depth (evaluation floor) | 4 | 0.8 / 1.4 | 1.6 / 2.1 | -1.3% | -1.1% | 0.2 / 1.7 | 0.5 / 2.6 | 100% | @@ -82,27 +82,27 @@ | Method | Views | Model scale: 1-3 m | 3-10 m | Scale error: 1-3 m | 3-10 m | One taped distance: 1-3 m | 3-10 m | Tape calibrated | | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| MoGe-2, one photo (every photo) | 1 | 14.9 / 25.4 | 26.1 / 37.0 | +20.9% | +17.5% | 2.7 / 8.2 | 4.8 / 13.0 | 100% | -| Depth Anything 3 metric, one photo (every photo) | 1 | 3.5 / 9.3 | 4.8 / 13.0 | +0.2% | -2.5% | 4.1 / 11.8 | 6.6 / 18.7 | 100% | -| scan rendered as depth (evaluation floor) | 1 | 0.0 / 0.4 | 0.1 / 0.6 | -0.0% | -0.1% | 0.5 / 2.0 | 1.0 / 4.0 | 100% | -| scan rendered as depth (evaluation floor) | 2 | 0.0 / 0.4 | 0.1 / 0.6 | -0.0% | -0.1% | 0.5 / 2.1 | 1.0 / 4.0 | 100% | +| MoGe-2, one photo (every photo) | 1 | 14.8 / 25.3 | 25.9 / 37.2 | +20.7% | +17.4% | 2.7 / 8.2 | 4.8 / 13.2 | 100% | +| Depth Anything 3 metric, one photo (every photo) | 1 | 3.5 / 9.3 | 5.2 / 12.9 | +0.0% | -2.7% | 4.1 / 11.8 | 6.7 / 19.1 | 100% | +| scan rendered as depth (evaluation floor) | 1 | 0.0 / 0.3 | 0.1 / 0.5 | -0.0% | -0.1% | 0.5 / 2.1 | 1.0 / 4.0 | 100% | +| scan rendered as depth (evaluation floor) | 2 | 0.0 / 0.4 | 0.1 / 0.5 | -0.1% | -0.1% | 0.5 / 2.1 | 1.0 / 4.0 | 100% | | scan rendered as depth (evaluation floor) | 4 | 0.1 / 0.3 | 0.1 / 0.6 | -0.1% | -0.1% | 0.5 / 2.2 | 1.0 / 4.1 | 100% | -| scan rendered as depth (evaluation floor) | 8 | 0.1 / 0.3 | 0.1 / 0.6 | -0.1% | -0.1% | 0.4 / 2.1 | 0.9 / 4.1 | 100% | -| MoGe-2 per photo + true poses | 1 | 16.3 / 25.6 | 26.6 / 32.9 | +21.4% | +18.0% | 3.3 / 8.8 | 4.7 / 11.2 | 100% | -| MoGe-2 per photo + true poses | 2 | 15.8 / 24.2 | 26.1 / 32.3 | +20.7% | +17.7% | 3.4 / 8.7 | 4.9 / 12.1 | 100% | -| MoGe-2 per photo + true poses | 4 | 15.8 / 23.3 | 25.4 / 31.1 | +20.4% | +17.3% | 3.4 / 8.8 | 5.0 / 13.1 | 100% | -| MoGe-2 per photo + true poses | 8 | 15.4 / 22.6 | 25.3 / 31.0 | +20.3% | +17.3% | 5.0 / 13.5 | 6.6 / 28.4 | 100% | -| Depth Anything 3 metric per photo + true poses | 1 | 1.5 / 4.5 | 1.8 / 4.6 | +0.3% | -0.9% | 2.8 / 8.1 | 4.9 / 14.6 | 100% | -| Depth Anything 3 metric per photo + true poses | 2 | 2.2 / 6.7 | 2.5 / 7.3 | +1.1% | -1.3% | 4.1 / 11.1 | 6.3 / 20.3 | 100% | -| Depth Anything 3 metric per photo + true poses | 4 | 2.5 / 7.0 | 2.4 / 7.0 | +1.6% | -0.8% | 4.3 / 11.8 | 6.6 / 21.3 | 100% | -| Depth Anything 3 metric per photo + true poses | 8 | 2.4 / 5.7 | 4.0 / 7.9 | -0.6% | -2.6% | 3.7 / 11.2 | 5.3 / 24.3 | 100% | +| scan rendered as depth (evaluation floor) | 8 | 0.1 / 0.3 | 0.1 / 0.6 | -0.1% | -0.1% | 0.5 / 2.1 | 0.9 / 4.1 | 100% | +| MoGe-2 per photo + true poses | 1 | 16.2 / 25.3 | 26.2 / 32.8 | +21.0% | +17.7% | 3.2 / 8.6 | 4.7 / 11.3 | 100% | +| MoGe-2 per photo + true poses | 2 | 15.7 / 24.0 | 25.8 / 32.2 | +20.4% | +17.4% | 3.2 / 8.4 | 4.8 / 12.2 | 100% | +| MoGe-2 per photo + true poses | 4 | 15.5 / 23.2 | 25.2 / 31.0 | +20.2% | +17.0% | 3.3 / 8.6 | 5.0 / 13.3 | 100% | +| MoGe-2 per photo + true poses | 8 | 15.2 / 22.5 | 25.1 / 30.5 | +20.2% | +17.1% | 4.9 / 13.5 | 6.6 / 28.8 | 100% | +| Depth Anything 3 metric per photo + true poses | 1 | 1.5 / 4.3 | 1.9 / 4.7 | +0.1% | -1.0% | 2.8 / 8.0 | 4.9 / 14.5 | 100% | +| Depth Anything 3 metric per photo + true poses | 2 | 2.1 / 6.3 | 2.7 / 7.5 | +0.7% | -1.4% | 4.0 / 10.9 | 6.4 / 20.5 | 100% | +| Depth Anything 3 metric per photo + true poses | 4 | 2.3 / 6.9 | 2.5 / 7.0 | +1.2% | -1.0% | 4.3 / 11.6 | 6.6 / 21.4 | 100% | +| Depth Anything 3 metric per photo + true poses | 8 | 2.5 / 5.6 | 4.2 / 8.2 | -0.9% | -2.8% | 3.8 / 11.2 | 5.3 / 24.6 | 100% | ## facade, within 6 m, edges | Method | Views | Model scale: 1-3 m | 3-10 m | Scale error: 1-3 m | 3-10 m | One taped distance: 1-3 m | 3-10 m | Tape calibrated | | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| MoGe-2, one photo (every photo) | 1 | 11.2 / 24.3 | 30.2 / 43.7 | +19.4% | +16.7% | 3.0 / 9.3 | 5.8 / 17.6 | 100% | -| Depth Anything 3 metric, one photo (every photo) | 1 | 3.8 / 39.4 | 8.0 / 19.7 | +1.2% | -2.9% | 4.6 / 38.4 | 8.8 / 29.1 | 100% | +| MoGe-2, one photo (every photo) | 1 | 11.2 / 24.2 | 30.2 / 43.7 | +19.4% | +16.7% | 2.9 / 9.1 | 5.9 / 18.1 | 100% | +| Depth Anything 3 metric, one photo (every photo) | 1 | 3.8 / 39.1 | 8.0 / 19.8 | +1.2% | -2.9% | 4.5 / 38.1 | 8.9 / 29.7 | 100% | | scan rendered as depth (evaluation floor) | 1 | 0.6 / 1.4 | 1.4 / 3.2 | -0.5% | -0.7% | 0.7 / 1.8 | 1.6 / 4.5 | 100% | | scan rendered as depth (evaluation floor) | 2 | 0.6 / 1.4 | 1.4 / 3.2 | -0.4% | -0.7% | 0.7 / 1.9 | 1.7 / 4.7 | 100% | | scan rendered as depth (evaluation floor) | 4 | 0.6 / 1.4 | 1.6 / 3.3 | -0.5% | -0.8% | 0.7 / 1.9 | 1.8 / 4.7 | 100% | @@ -120,112 +120,112 @@ | Method | Views | Model scale: 1-3 m | 3-10 m | Scale error: 1-3 m | 3-10 m | One taped distance: 1-3 m | 3-10 m | Tape calibrated | | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| MoGe-2, one photo (every photo) | 1 | 6.8 / 19.1 | 18.4 / 52.6 | +0.9% | +0.6% | 2.5 / 9.7 | 5.9 / 25.4 | 100% | -| Depth Anything 3 metric, one photo (every photo) | 1 | 7.5 / 24.9 | 21.5 / 72.6 | -8.6% | -9.7% | 3.4 / 15.3 | 8.6 / 38.0 | 99% | -| scan rendered as depth (evaluation floor) | 1 | 0.7 / 2.4 | 1.4 / 5.2 | -0.6% | -0.6% | 0.7 / 2.8 | 1.6 / 6.6 | 100% | -| scan rendered as depth (evaluation floor) | 2 | 0.7 / 2.3 | 1.4 / 5.0 | -0.6% | -0.6% | 0.7 / 2.9 | 1.6 / 6.6 | 100% | -| scan rendered as depth (evaluation floor) | 4 | 0.7 / 2.2 | 1.4 / 4.9 | -0.6% | -0.6% | 0.7 / 2.8 | 1.6 / 6.7 | 100% | +| MoGe-2, one photo (every photo) | 1 | 6.8 / 19.1 | 18.4 / 52.6 | +0.9% | +0.6% | 2.5 / 9.7 | 5.9 / 25.3 | 100% | +| Depth Anything 3 metric, one photo (every photo) | 1 | 7.5 / 24.9 | 21.5 / 72.6 | -8.6% | -9.7% | 3.4 / 15.2 | 8.6 / 37.7 | 99% | +| scan rendered as depth (evaluation floor) | 1 | 0.6 / 2.0 | 1.2 / 4.5 | -0.6% | -0.6% | 0.7 / 2.3 | 1.4 / 5.1 | 100% | +| scan rendered as depth (evaluation floor) | 2 | 0.6 / 2.0 | 1.3 / 4.3 | -0.6% | -0.6% | 0.7 / 2.4 | 1.4 / 5.3 | 100% | +| scan rendered as depth (evaluation floor) | 4 | 0.6 / 1.9 | 1.3 / 4.2 | -0.6% | -0.6% | 0.7 / 2.4 | 1.4 / 5.4 | 100% | | scan rendered as depth (evaluation floor) | 8 | 0.7 / 1.9 | 1.4 / 4.1 | -0.6% | -0.6% | 0.7 / 2.4 | 1.4 / 5.1 | 100% | -| MoGe-2 per photo + true poses | 1 | 7.6 / 17.9 | 21.9 / 49.2 | +4.1% | +5.1% | 2.9 / 13.2 | 6.9 / 37.3 | 100% | -| MoGe-2 per photo + true poses | 2 | 6.3 / 16.6 | 17.5 / 42.2 | +2.2% | +3.4% | 3.2 / 14.0 | 7.0 / 37.4 | 100% | -| MoGe-2 per photo + true poses | 4 | 5.9 / 16.8 | 16.5 / 49.8 | +2.5% | +3.7% | 3.2 / 15.2 | 6.9 / 39.4 | 99% | -| MoGe-2 per photo + true poses | 8 | 5.0 / 12.7 | 13.6 / 25.7 | +1.1% | +1.9% | 3.5 / 13.2 | 7.1 / 29.8 | 99% | -| Depth Anything 3 metric per photo + true poses | 1 | 7.6 / 19.0 | 22.0 / 50.6 | -7.9% | -8.3% | 3.7 / 20.6 | 9.7 / 48.7 | 99% | -| Depth Anything 3 metric per photo + true poses | 2 | 7.7 / 19.6 | 21.6 / 55.9 | -8.1% | -8.5% | 4.0 / 18.3 | 9.6 / 41.3 | 100% | -| Depth Anything 3 metric per photo + true poses | 4 | 7.2 / 20.5 | 21.0 / 59.3 | -7.5% | -8.5% | 4.3 / 21.8 | 9.9 / 51.4 | 99% | +| MoGe-2 per photo + true poses | 1 | 7.0 / 14.5 | 20.4 / 34.4 | +4.9% | +6.8% | 2.5 / 8.9 | 5.8 / 22.5 | 100% | +| MoGe-2 per photo + true poses | 2 | 5.6 / 14.0 | 16.0 / 30.9 | +3.0% | +4.4% | 2.8 / 10.1 | 6.0 / 22.8 | 100% | +| MoGe-2 per photo + true poses | 4 | 5.3 / 13.4 | 14.8 / 29.0 | +3.8% | +4.6% | 2.8 / 10.7 | 5.9 / 25.0 | 99% | +| MoGe-2 per photo + true poses | 8 | 5.0 / 12.7 | 13.6 / 25.7 | +1.1% | +1.9% | 3.5 / 13.1 | 7.0 / 29.7 | 99% | +| Depth Anything 3 metric per photo + true poses | 1 | 6.8 / 18.0 | 20.1 / 46.9 | -6.7% | -7.2% | 3.5 / 17.7 | 8.9 / 40.7 | 99% | +| Depth Anything 3 metric per photo + true poses | 2 | 6.6 / 17.7 | 19.0 / 49.6 | -6.6% | -6.6% | 3.7 / 15.9 | 8.7 / 35.6 | 99% | +| Depth Anything 3 metric per photo + true poses | 4 | 6.1 / 16.5 | 18.0 / 46.8 | -6.2% | -6.3% | 3.9 / 17.4 | 8.9 / 41.2 | 99% | | Depth Anything 3 metric per photo + true poses | 8 | 6.7 / 17.9 | 19.8 / 50.9 | -7.0% | -8.0% | 3.9 / 19.0 | 8.8 / 45.6 | 99% | ## electro, all points, vertical interior | Method | Views | Model scale: 1-3 m | 3-10 m | Scale error: 1-3 m | 3-10 m | One taped distance: 1-3 m | 3-10 m | Tape calibrated | | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| MoGe-2, one photo (every photo) | 1 | 6.5 / 19.2 | 17.8 / 56.1 | +0.3% | -0.1% | 2.5 / 9.7 | 5.8 / 25.4 | 100% | -| Depth Anything 3 metric, one photo (every photo) | 1 | 8.1 / 26.7 | 22.6 / 78.5 | -10.0% | -11.5% | 3.5 / 15.3 | 8.9 / 41.4 | 99% | -| scan rendered as depth (evaluation floor) | 1 | 0.3 / 1.4 | 0.8 / 4.5 | -0.3% | -0.4% | 0.5 / 2.5 | 1.3 / 6.2 | 100% | -| scan rendered as depth (evaluation floor) | 2 | 0.3 / 1.2 | 0.9 / 4.2 | -0.3% | -0.4% | 0.6 / 2.5 | 1.4 / 6.3 | 100% | -| scan rendered as depth (evaluation floor) | 4 | 0.3 / 1.3 | 0.9 / 4.4 | -0.3% | -0.4% | 0.6 / 2.4 | 1.6 / 6.2 | 100% | -| scan rendered as depth (evaluation floor) | 8 | 0.3 / 1.0 | 0.8 / 3.1 | -0.3% | -0.4% | 0.6 / 2.1 | 1.5 / 5.2 | 100% | -| MoGe-2 per photo + true poses | 1 | 7.3 / 17.9 | 22.8 / 56.2 | +4.3% | +7.2% | 2.5 / 12.4 | 7.1 / 38.1 | 100% | -| MoGe-2 per photo + true poses | 2 | 6.1 / 16.2 | 17.6 / 45.5 | +1.7% | +4.7% | 2.8 / 13.4 | 7.0 / 38.5 | 100% | -| MoGe-2 per photo + true poses | 4 | 5.6 / 16.2 | 16.5 / 52.4 | +1.3% | +4.2% | 2.8 / 14.7 | 6.9 / 41.1 | 99% | -| MoGe-2 per photo + true poses | 8 | 4.8 / 12.2 | 13.3 / 24.3 | -0.5% | +2.9% | 3.3 / 13.4 | 7.8 / 30.1 | 99% | -| Depth Anything 3 metric per photo + true poses | 1 | 7.3 / 19.2 | 26.2 / 59.4 | -9.7% | -11.5% | 3.4 / 19.2 | 11.4 / 53.9 | 99% | -| Depth Anything 3 metric per photo + true poses | 2 | 7.3 / 20.3 | 28.0 / 65.3 | -10.1% | -13.3% | 3.6 / 17.3 | 11.3 / 51.1 | 100% | -| Depth Anything 3 metric per photo + true poses | 4 | 7.0 / 20.5 | 26.6 / 69.4 | -9.6% | -13.5% | 3.7 / 20.9 | 11.0 / 62.7 | 99% | -| Depth Anything 3 metric per photo + true poses | 8 | 7.3 / 18.6 | 25.4 / 59.7 | -9.4% | -12.8% | 3.7 / 20.4 | 11.3 / 54.9 | 99% | +| MoGe-2, one photo (every photo) | 1 | 6.6 / 19.2 | 17.8 / 56.1 | +0.3% | -0.2% | 2.5 / 9.6 | 5.8 / 25.2 | 100% | +| Depth Anything 3 metric, one photo (every photo) | 1 | 8.1 / 26.6 | 22.6 / 78.5 | -10.0% | -11.5% | 3.5 / 15.2 | 8.9 / 41.2 | 99% | +| scan rendered as depth (evaluation floor) | 1 | 0.2 / 1.0 | 0.7 / 3.1 | -0.3% | -0.3% | 0.5 / 2.0 | 1.1 / 4.9 | 100% | +| scan rendered as depth (evaluation floor) | 2 | 0.2 / 0.9 | 0.8 / 2.6 | -0.3% | -0.4% | 0.5 / 2.1 | 1.2 / 5.2 | 100% | +| scan rendered as depth (evaluation floor) | 4 | 0.2 / 0.9 | 0.7 / 3.0 | -0.3% | -0.4% | 0.6 / 2.1 | 1.4 / 5.3 | 100% | +| scan rendered as depth (evaluation floor) | 8 | 0.3 / 1.0 | 0.8 / 3.0 | -0.4% | -0.4% | 0.6 / 2.1 | 1.5 / 5.2 | 100% | +| MoGe-2 per photo + true poses | 1 | 6.7 / 14.1 | 20.7 / 35.5 | +5.2% | +8.0% | 2.2 / 8.2 | 5.9 / 22.8 | 100% | +| MoGe-2 per photo + true poses | 2 | 5.4 / 13.6 | 16.2 / 30.8 | +3.1% | +5.7% | 2.4 / 9.4 | 6.0 / 23.2 | 100% | +| MoGe-2 per photo + true poses | 4 | 4.9 / 12.4 | 14.9 / 29.8 | +3.3% | +5.3% | 2.4 / 10.2 | 5.9 / 28.8 | 99% | +| MoGe-2 per photo + true poses | 8 | 4.8 / 12.3 | 13.2 / 24.2 | -0.6% | +2.9% | 3.3 / 13.6 | 7.8 / 29.8 | 99% | +| Depth Anything 3 metric per photo + true poses | 1 | 6.3 / 18.2 | 23.6 / 58.8 | -8.1% | -10.2% | 3.1 / 17.0 | 10.7 / 50.2 | 99% | +| Depth Anything 3 metric per photo + true poses | 2 | 6.2 / 18.5 | 24.1 / 62.4 | -8.0% | -12.2% | 3.3 / 15.9 | 10.6 / 49.1 | 99% | +| Depth Anything 3 metric per photo + true poses | 4 | 6.0 / 16.4 | 22.2 / 58.8 | -7.6% | -11.8% | 3.4 / 17.1 | 9.9 / 54.1 | 99% | +| Depth Anything 3 metric per photo + true poses | 8 | 7.2 / 18.6 | 25.4 / 59.0 | -9.2% | -12.8% | 3.7 / 20.3 | 11.2 / 54.8 | 99% | ## electro, all points, edges | Method | Views | Model scale: 1-3 m | 3-10 m | Scale error: 1-3 m | 3-10 m | One taped distance: 1-3 m | 3-10 m | Tape calibrated | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | MoGe-2, one photo (every photo) | 1 | 8.0 / 24.9 | 17.1 / 63.4 | +4.3% | +2.7% | 4.4 / 25.1 | 7.2 / 37.4 | 100% | -| Depth Anything 3 metric, one photo (every photo) | 1 | 7.3 / 39.4 | 16.9 / 93.0 | -2.2% | -4.4% | 8.7 / 61.9 | 13.7 / 95.4 | 99% | -| scan rendered as depth (evaluation floor) | 1 | 0.4 / 1.5 | 0.9 / 4.2 | -0.5% | -0.6% | 0.7 / 2.6 | 1.5 / 5.6 | 100% | -| scan rendered as depth (evaluation floor) | 2 | 0.4 / 1.9 | 1.0 / 3.9 | -0.5% | -0.6% | 0.7 / 2.7 | 1.6 / 5.9 | 100% | -| scan rendered as depth (evaluation floor) | 4 | 0.4 / 1.8 | 1.0 / 3.8 | -0.5% | -0.6% | 0.8 / 2.6 | 1.7 / 5.8 | 100% | +| Depth Anything 3 metric, one photo (every photo) | 1 | 7.3 / 39.4 | 16.9 / 93.0 | -2.2% | -4.4% | 8.8 / 61.9 | 13.7 / 95.2 | 99% | +| scan rendered as depth (evaluation floor) | 1 | 0.4 / 1.6 | 0.9 / 3.7 | -0.4% | -0.6% | 0.6 / 2.4 | 1.3 / 4.6 | 100% | +| scan rendered as depth (evaluation floor) | 2 | 0.4 / 1.9 | 1.0 / 2.9 | -0.5% | -0.6% | 0.7 / 2.6 | 1.4 / 4.9 | 100% | +| scan rendered as depth (evaluation floor) | 4 | 0.4 / 1.9 | 1.0 / 3.1 | -0.5% | -0.6% | 0.8 / 2.5 | 1.6 / 4.9 | 100% | | scan rendered as depth (evaluation floor) | 8 | 0.5 / 1.9 | 1.1 / 3.3 | -0.6% | -0.7% | 0.8 / 2.5 | 1.5 / 4.7 | 100% | -| MoGe-2 per photo + true poses | 1 | 8.8 / 33.0 | 19.0 / 54.8 | +6.6% | +5.9% | 4.1 / 37.0 | 7.0 / 45.5 | 100% | -| MoGe-2 per photo + true poses | 2 | 7.6 / 30.6 | 15.3 / 44.7 | +5.6% | +4.3% | 4.3 / 32.6 | 7.0 / 46.4 | 100% | -| MoGe-2 per photo + true poses | 4 | 6.4 / 21.3 | 13.3 / 37.6 | +4.8% | +4.7% | 4.7 / 27.1 | 7.7 / 47.8 | 99% | -| MoGe-2 per photo + true poses | 8 | 6.5 / 15.6 | 11.1 / 24.1 | +3.8% | +3.8% | 4.8 / 16.3 | 7.3 / 28.0 | 99% | -| Depth Anything 3 metric per photo + true poses | 1 | 9.5 / 37.5 | 22.9 / 113.4 | -0.9% | -2.1% | 7.6 / 57.9 | 15.4 / 148.0 | 99% | -| Depth Anything 3 metric per photo + true poses | 2 | 9.5 / 43.1 | 24.3 / 96.7 | -0.7% | -1.3% | 8.1 / 62.5 | 16.7 / 128.6 | 100% | -| Depth Anything 3 metric per photo + true poses | 4 | 8.0 / 32.2 | 20.5 / 78.3 | -1.0% | -2.0% | 7.6 / 52.0 | 16.2 / 113.0 | 99% | +| MoGe-2 per photo + true poses | 1 | 7.8 / 19.5 | 17.4 / 34.5 | +5.4% | +6.2% | 3.5 / 15.0 | 5.9 / 22.5 | 100% | +| MoGe-2 per photo + true poses | 2 | 7.0 / 19.0 | 14.0 / 27.7 | +4.6% | +4.8% | 3.6 / 16.4 | 5.9 / 24.4 | 100% | +| MoGe-2 per photo + true poses | 4 | 6.0 / 17.2 | 12.3 / 26.7 | +4.6% | +5.2% | 3.9 / 15.9 | 6.5 / 25.5 | 99% | +| MoGe-2 per photo + true poses | 8 | 6.5 / 15.6 | 11.1 / 24.1 | +3.8% | +3.8% | 4.7 / 16.3 | 7.3 / 28.0 | 99% | +| Depth Anything 3 metric per photo + true poses | 1 | 8.3 / 30.3 | 20.8 / 48.5 | -2.7% | -1.9% | 6.1 / 40.5 | 12.7 / 60.9 | 99% | +| Depth Anything 3 metric per photo + true poses | 2 | 8.6 / 42.5 | 22.1 / 60.9 | -2.3% | -0.9% | 6.7 / 49.7 | 14.2 / 65.0 | 99% | +| Depth Anything 3 metric per photo + true poses | 4 | 7.5 / 28.4 | 18.5 / 54.9 | -2.1% | -1.6% | 6.3 / 38.0 | 13.3 / 58.1 | 99% | | Depth Anything 3 metric per photo + true poses | 8 | 7.3 / 28.6 | 18.0 / 58.5 | -2.9% | -2.4% | 6.2 / 41.5 | 13.8 / 65.4 | 99% | ## electro, within 6 m, surface interior | Method | Views | Model scale: 1-3 m | 3-10 m | Scale error: 1-3 m | 3-10 m | One taped distance: 1-3 m | 3-10 m | Tape calibrated | | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| MoGe-2, one photo (every photo) | 1 | 6.8 / 19.5 | 15.1 / 41.8 | +3.5% | +4.3% | 1.6 / 9.6 | 3.0 / 17.8 | 98% | -| Depth Anything 3 metric, one photo (every photo) | 1 | 6.0 / 20.2 | 13.9 / 43.4 | -6.6% | -7.4% | 2.9 / 25.3 | 6.1 / 62.6 | 97% | -| scan rendered as depth (evaluation floor) | 1 | 0.7 / 1.6 | 1.3 / 2.4 | -0.9% | -0.8% | 0.5 / 1.6 | 0.9 / 2.8 | 100% | -| scan rendered as depth (evaluation floor) | 2 | 0.7 / 1.5 | 1.3 / 2.3 | -0.8% | -0.8% | 0.5 / 1.6 | 0.9 / 2.8 | 100% | -| scan rendered as depth (evaluation floor) | 4 | 0.6 / 1.3 | 1.1 / 2.2 | -0.7% | -0.7% | 0.4 / 1.6 | 0.8 / 3.1 | 100% | -| scan rendered as depth (evaluation floor) | 8 | 0.5 / 1.2 | 1.0 / 2.0 | -0.6% | -0.6% | 0.5 / 1.7 | 0.8 / 3.6 | 100% | -| MoGe-2 per photo + true poses | 1 | 6.9 / 14.9 | 15.5 / 30.4 | +4.4% | +4.8% | 1.8 / 8.7 | 3.5 / 17.5 | 100% | -| MoGe-2 per photo + true poses | 2 | 6.8 / 15.0 | 14.9 / 28.7 | +2.4% | +3.8% | 2.3 / 11.3 | 4.0 / 21.1 | 100% | -| MoGe-2 per photo + true poses | 4 | 6.1 / 14.4 | 13.9 / 25.9 | +2.5% | +4.4% | 2.5 / 15.7 | 4.4 / 33.1 | 100% | -| MoGe-2 per photo + true poses | 8 | 5.4 / 12.5 | 12.4 / 22.6 | +0.7% | +3.0% | 2.8 / 10.8 | 4.3 / 21.1 | 99% | -| Depth Anything 3 metric per photo + true poses | 1 | 6.5 / 19.3 | 14.9 / 42.7 | -5.9% | -7.6% | 3.3 / 24.8 | 7.4 / 52.2 | 100% | -| Depth Anything 3 metric per photo + true poses | 2 | 5.7 / 17.7 | 13.6 / 38.2 | -5.6% | -7.4% | 3.6 / 21.6 | 7.1 / 44.4 | 100% | -| Depth Anything 3 metric per photo + true poses | 4 | 5.6 / 18.1 | 13.1 / 36.0 | -5.7% | -7.0% | 4.0 / 25.2 | 7.8 / 51.4 | 99% | -| Depth Anything 3 metric per photo + true poses | 8 | 5.1 / 14.7 | 11.9 / 33.1 | -5.6% | -6.8% | 3.6 / 20.1 | 6.5 / 38.9 | 95% | +| MoGe-2, one photo (every photo) | 1 | 6.8 / 19.5 | 15.1 / 41.8 | +3.5% | +4.3% | 1.6 / 9.3 | 3.0 / 16.9 | 98% | +| Depth Anything 3 metric, one photo (every photo) | 1 | 6.0 / 20.2 | 13.9 / 43.4 | -6.6% | -7.4% | 2.9 / 24.4 | 6.0 / 57.9 | 97% | +| scan rendered as depth (evaluation floor) | 1 | 0.7 / 1.6 | 1.2 / 2.3 | -0.9% | -0.7% | 0.5 / 1.6 | 0.9 / 2.8 | 100% | +| scan rendered as depth (evaluation floor) | 2 | 0.6 / 1.5 | 1.2 / 2.2 | -0.8% | -0.7% | 0.5 / 1.6 | 0.9 / 3.0 | 100% | +| scan rendered as depth (evaluation floor) | 4 | 0.6 / 1.2 | 1.1 / 2.1 | -0.6% | -0.6% | 0.5 / 1.7 | 0.8 / 3.3 | 100% | +| scan rendered as depth (evaluation floor) | 8 | 0.5 / 1.2 | 1.0 / 2.0 | -0.6% | -0.6% | 0.5 / 1.8 | 0.8 / 3.8 | 100% | +| MoGe-2 per photo + true poses | 1 | 6.8 / 13.8 | 15.2 / 28.6 | +4.2% | +4.4% | 1.4 / 5.2 | 2.8 / 10.1 | 100% | +| MoGe-2 per photo + true poses | 2 | 6.6 / 13.9 | 14.6 / 27.8 | +2.0% | +2.6% | 2.0 / 8.3 | 3.5 / 15.6 | 99% | +| MoGe-2 per photo + true poses | 4 | 5.7 / 13.0 | 13.5 / 24.3 | +2.9% | +4.4% | 2.2 / 10.0 | 3.9 / 18.9 | 99% | +| MoGe-2 per photo + true poses | 8 | 5.4 / 12.5 | 12.4 / 22.6 | +0.7% | +3.0% | 2.8 / 11.9 | 4.5 / 24.3 | 98% | +| Depth Anything 3 metric per photo + true poses | 1 | 5.7 / 16.9 | 13.2 / 36.8 | -5.6% | -7.5% | 2.8 / 16.2 | 5.8 / 29.5 | 100% | +| Depth Anything 3 metric per photo + true poses | 2 | 5.0 / 15.2 | 12.3 / 35.1 | -5.2% | -7.3% | 3.3 / 15.9 | 6.3 / 31.0 | 99% | +| Depth Anything 3 metric per photo + true poses | 4 | 5.0 / 14.8 | 12.1 / 33.6 | -5.3% | -6.9% | 3.6 / 19.9 | 7.0 / 38.5 | 97% | +| Depth Anything 3 metric per photo + true poses | 8 | 5.1 / 14.7 | 11.9 / 33.1 | -5.6% | -6.8% | 3.6 / 21.4 | 6.5 / 41.5 | 95% | ## electro, within 6 m, vertical interior | Method | Views | Model scale: 1-3 m | 3-10 m | Scale error: 1-3 m | 3-10 m | One taped distance: 1-3 m | 3-10 m | Tape calibrated | | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| MoGe-2, one photo (every photo) | 1 | 7.1 / 24.7 | 15.6 / 64.2 | +1.0% | +1.6% | 1.8 / 15.1 | 3.6 / 58.2 | 97% | -| Depth Anything 3 metric, one photo (every photo) | 1 | 6.2 / 35.3 | 14.6 / fails (13% failed) | -7.2% | -7.3% | 2.9 / 171.3 | 6.7 / fails (14% failed) | 96% | -| scan rendered as depth (evaluation floor) | 1 | 0.1 / 0.9 | 0.3 / 1.9 | -0.2% | -0.2% | 0.4 / 1.5 | 0.8 / 2.8 | 100% | -| scan rendered as depth (evaluation floor) | 2 | 0.1 / 0.8 | 0.3 / 1.8 | -0.2% | -0.2% | 0.4 / 1.5 | 0.8 / 2.8 | 100% | -| scan rendered as depth (evaluation floor) | 4 | 0.2 / 0.8 | 0.4 / 1.6 | -0.2% | -0.2% | 0.4 / 1.6 | 0.8 / 3.0 | 100% | -| scan rendered as depth (evaluation floor) | 8 | 0.1 / 0.5 | 0.3 / 0.9 | -0.2% | -0.2% | 0.4 / 1.7 | 0.8 / 3.2 | 100% | -| MoGe-2 per photo + true poses | 1 | 5.4 / 14.8 | 13.5 / 33.5 | +3.5% | +4.5% | 1.8 / 9.9 | 4.1 / 29.1 | 100% | -| MoGe-2 per photo + true poses | 2 | 5.2 / 15.3 | 13.3 / 31.6 | -0.1% | +2.5% | 2.4 / 12.8 | 5.1 / 32.2 | 100% | -| MoGe-2 per photo + true poses | 4 | 5.6 / 15.0 | 13.7 / 30.4 | -0.1% | +4.3% | 2.4 / 19.8 | 5.3 / 52.6 | 99% | -| MoGe-2 per photo + true poses | 8 | 4.8 / 12.4 | 11.6 / 25.8 | -1.3% | +2.7% | 2.8 / 10.0 | 5.5 / 25.6 | 99% | -| Depth Anything 3 metric per photo + true poses | 1 | 6.4 / 27.8 | 15.4 / 52.9 | -6.8% | -7.4% | 2.5 / 34.7 | 6.8 / 118.0 | 100% | -| Depth Anything 3 metric per photo + true poses | 2 | 5.8 / 20.7 | 13.8 / 41.9 | -6.9% | -7.3% | 2.8 / 27.7 | 7.7 / 77.0 | 100% | -| Depth Anything 3 metric per photo + true poses | 4 | 5.6 / 20.8 | 12.7 / 38.4 | -6.5% | -6.5% | 3.3 / 30.9 | 8.2 / 76.8 | 99% | -| Depth Anything 3 metric per photo + true poses | 8 | 4.8 / 17.7 | 10.4 / 36.5 | -5.8% | -5.8% | 3.0 / 20.6 | 6.8 / 38.0 | 97% | +| MoGe-2, one photo (every photo) | 1 | 7.1 / 23.9 | 15.6 / 54.0 | +1.1% | +1.9% | 1.8 / 13.6 | 3.6 / 44.7 | 98% | +| Depth Anything 3 metric, one photo (every photo) | 1 | 6.2 / 33.2 | 14.7 / fails (12% failed) | -7.2% | -7.3% | 3.0 / 74.5 | 6.8 / fails (14% failed) | 96% | +| scan rendered as depth (evaluation floor) | 1 | 0.1 / 0.5 | 0.2 / 1.0 | -0.1% | -0.2% | 0.4 / 1.5 | 0.8 / 2.8 | 100% | +| scan rendered as depth (evaluation floor) | 2 | 0.1 / 0.5 | 0.3 / 0.9 | -0.2% | -0.2% | 0.4 / 1.6 | 0.9 / 3.0 | 100% | +| scan rendered as depth (evaluation floor) | 4 | 0.1 / 0.5 | 0.3 / 0.9 | -0.2% | -0.2% | 0.4 / 1.7 | 0.9 / 3.3 | 100% | +| scan rendered as depth (evaluation floor) | 8 | 0.1 / 0.5 | 0.3 / 0.9 | -0.2% | -0.2% | 0.4 / 1.7 | 0.9 / 3.4 | 100% | +| MoGe-2 per photo + true poses | 1 | 5.2 / 14.3 | 13.5 / 33.0 | +3.4% | +4.2% | 1.4 / 5.3 | 3.4 / 13.9 | 100% | +| MoGe-2 per photo + true poses | 2 | 4.9 / 13.8 | 12.9 / 31.4 | -1.3% | +1.9% | 2.2 / 8.6 | 4.5 / 21.9 | 99% | +| MoGe-2 per photo + true poses | 4 | 5.0 / 12.6 | 13.1 / 28.9 | +0.1% | +4.2% | 2.1 / 11.0 | 4.4 / 30.1 | 99% | +| MoGe-2 per photo + true poses | 8 | 4.8 / 12.5 | 11.8 / 26.0 | -1.3% | +2.7% | 2.9 / 11.4 | 5.6 / 27.6 | 97% | +| Depth Anything 3 metric per photo + true poses | 1 | 5.1 / 19.7 | 13.7 / 45.1 | -6.4% | -6.9% | 2.0 / 19.3 | 5.4 / 45.0 | 100% | +| Depth Anything 3 metric per photo + true poses | 2 | 5.0 / 19.4 | 12.9 / 41.5 | -6.5% | -6.9% | 2.6 / 22.0 | 6.6 / 54.6 | 99% | +| Depth Anything 3 metric per photo + true poses | 4 | 4.8 / 18.5 | 11.7 / 38.1 | -5.7% | -5.5% | 2.9 / 26.7 | 7.2 / 70.9 | 97% | +| Depth Anything 3 metric per photo + true poses | 8 | 4.8 / 17.6 | 10.8 / 36.6 | -5.8% | -5.6% | 3.0 / 22.0 | 7.2 / 44.7 | 97% | ## electro, within 6 m, edges | Method | Views | Model scale: 1-3 m | 3-10 m | Scale error: 1-3 m | 3-10 m | One taped distance: 1-3 m | 3-10 m | Tape calibrated | | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| MoGe-2, one photo (every photo) | 1 | 8.3 / 31.8 | 15.0 / 53.0 | +3.6% | +4.2% | 3.7 / 26.9 | 5.8 / 36.8 | 97% | -| Depth Anything 3 metric, one photo (every photo) | 1 | 6.2 / 35.0 | 9.5 / 73.1 | -1.2% | -0.1% | 7.1 / 117.4 | 12.4 / 185.2 | 96% | -| scan rendered as depth (evaluation floor) | 1 | 0.3 / 1.1 | 0.5 / 1.7 | -0.4% | -0.3% | 0.5 / 1.7 | 1.0 / 2.8 | 100% | -| scan rendered as depth (evaluation floor) | 2 | 0.3 / 1.1 | 0.5 / 1.6 | -0.4% | -0.3% | 0.5 / 1.7 | 1.0 / 2.8 | 100% | -| scan rendered as depth (evaluation floor) | 4 | 0.3 / 1.1 | 0.6 / 1.6 | -0.4% | -0.4% | 0.5 / 1.8 | 1.0 / 3.0 | 100% | -| scan rendered as depth (evaluation floor) | 8 | 0.3 / 1.0 | 0.5 / 1.6 | -0.4% | -0.4% | 0.5 / 1.9 | 1.0 / 3.2 | 100% | -| MoGe-2 per photo + true poses | 1 | 7.8 / 35.5 | 15.1 / 68.7 | +3.6% | +4.7% | 2.8 / 26.1 | 4.5 / 52.4 | 100% | -| MoGe-2 per photo + true poses | 2 | 8.5 / 33.8 | 15.1 / 49.9 | +1.2% | +2.4% | 3.9 / 27.3 | 5.5 / 38.1 | 100% | -| MoGe-2 per photo + true poses | 4 | 7.5 / 20.6 | 13.3 / 34.6 | +2.8% | +3.3% | 4.2 / 26.4 | 6.4 / 50.0 | 99% | -| MoGe-2 per photo + true poses | 8 | 6.4 / 14.6 | 7.8 / 18.5 | +1.2% | -0.3% | 4.0 / 15.3 | 6.1 / 25.2 | 99% | -| Depth Anything 3 metric per photo + true poses | 1 | 9.2 / 40.9 | 19.8 / 142.8 | -1.4% | -1.6% | 5.7 / 66.9 | 10.8 / 171.6 | 100% | -| Depth Anything 3 metric per photo + true poses | 2 | 8.3 / 37.1 | 18.8 / 103.1 | -1.9% | -1.7% | 6.2 / 55.1 | 11.3 / 132.1 | 100% | -| Depth Anything 3 metric per photo + true poses | 4 | 7.3 / 23.7 | 14.8 / 74.4 | -1.7% | -2.0% | 6.0 / 44.6 | 10.7 / 109.4 | 99% | -| Depth Anything 3 metric per photo + true poses | 8 | 6.1 / 19.0 | 8.4 / 33.2 | -2.9% | -3.4% | 5.0 / 26.8 | 8.4 / 47.0 | 97% | +| MoGe-2, one photo (every photo) | 1 | 8.3 / 31.7 | 15.0 / 53.2 | +3.6% | +4.2% | 3.6 / 25.3 | 5.8 / 33.8 | 98% | +| Depth Anything 3 metric, one photo (every photo) | 1 | 6.2 / 34.9 | 9.5 / 73.7 | -1.2% | -0.1% | 7.1 / 99.4 | 12.5 / 166.7 | 96% | +| scan rendered as depth (evaluation floor) | 1 | 0.3 / 1.1 | 0.5 / 1.6 | -0.3% | -0.3% | 0.5 / 1.7 | 1.0 / 2.7 | 100% | +| scan rendered as depth (evaluation floor) | 2 | 0.3 / 1.1 | 0.5 / 1.6 | -0.4% | -0.3% | 0.5 / 1.8 | 1.1 / 2.9 | 100% | +| scan rendered as depth (evaluation floor) | 4 | 0.3 / 1.1 | 0.6 / 1.6 | -0.4% | -0.4% | 0.6 / 1.9 | 1.1 / 3.2 | 100% | +| scan rendered as depth (evaluation floor) | 8 | 0.3 / 1.0 | 0.5 / 1.6 | -0.4% | -0.4% | 0.5 / 2.0 | 1.0 / 3.4 | 100% | +| MoGe-2 per photo + true poses | 1 | 6.6 / 16.5 | 12.2 / 23.4 | +3.0% | +3.8% | 2.2 / 9.9 | 3.7 / 13.4 | 100% | +| MoGe-2 per photo + true poses | 2 | 7.2 / 17.3 | 12.0 / 23.6 | +0.4% | +1.4% | 3.2 / 14.6 | 4.7 / 18.8 | 99% | +| MoGe-2 per photo + true poses | 4 | 6.9 / 16.0 | 11.4 / 21.1 | +2.4% | +3.0% | 3.5 / 16.5 | 5.2 / 24.7 | 99% | +| MoGe-2 per photo + true poses | 8 | 6.4 / 14.6 | 7.8 / 18.5 | +1.3% | -0.3% | 4.0 / 16.9 | 6.2 / 27.6 | 97% | +| Depth Anything 3 metric per photo + true poses | 1 | 8.0 / 26.6 | 12.5 / 41.1 | -3.1% | -2.4% | 4.6 / 36.7 | 8.6 / 50.1 | 100% | +| Depth Anything 3 metric per photo + true poses | 2 | 7.5 / 25.4 | 11.0 / 38.2 | -3.0% | -2.4% | 5.0 / 38.8 | 8.9 / 53.8 | 99% | +| Depth Anything 3 metric per photo + true poses | 4 | 6.7 / 19.4 | 9.9 / 34.6 | -2.6% | -2.8% | 5.1 / 34.1 | 8.8 / 62.5 | 97% | +| Depth Anything 3 metric per photo + true poses | 8 | 6.1 / 18.9 | 8.6 / 33.4 | -2.9% | -3.3% | 5.1 / 28.2 | 8.6 / 51.0 | 97% | diff --git a/experiments/evals/results/eth3d_visibility_sensitivity.md b/experiments/evals/results/eth3d_visibility_sensitivity.md index c28e947c5..68c68ea2a 100644 --- a/experiments/evals/results/eth3d_visibility_sensitivity.md +++ b/experiments/evals/results/eth3d_visibility_sensitivity.md @@ -22,21 +22,21 @@ Surface interior, within 6 m. One taped distance, median / p90 inches, and the m | facade | MoGe-2 per photo + true poses | 2 | 8% | 0.7 / 6.1 | 1.3 / 10.4 | +9.8% | | facade | Depth Anything 3 metric per photo + true poses | 1 | 8% | 0.9 / 4.7 | 1.4 / 6.8 | -8.4% | | facade | Depth Anything 3 metric per photo + true poses | 2 | 8% | 1.1 / 6.9 | 1.6 / 10.8 | -9.1% | -| electro | scan rendered as depth (evaluation floor) | 1 | 2% | 0.2 / 0.9 | 0.4 / 1.6 | -0.5% | -| electro | scan rendered as depth (evaluation floor) | 2 | 2% | 0.2 / 0.9 | 0.4 / 1.5 | -0.4% | -| electro | MoGe-2 per photo + true poses | 1 | 2% | 1.4 / 9.7 | 3.2 / 21.2 | +4.8% | -| electro | MoGe-2 per photo + true poses | 2 | 2% | 1.9 / 11.0 | 3.5 / 22.8 | +3.3% | -| electro | Depth Anything 3 metric per photo + true poses | 1 | 2% | 2.4 / 29.7 | 5.2 / 118.5 | -8.6% | -| electro | Depth Anything 3 metric per photo + true poses | 2 | 2% | 2.6 / 25.5 | 4.8 / 77.3 | -8.3% | -| electro | scan rendered as depth (evaluation floor) | 1 | 4% | 0.5 / 1.6 | 0.9 / 2.8 | -0.8% | +| electro | scan rendered as depth (evaluation floor) | 1 | 2% | 0.3 / 0.9 | 0.4 / 1.6 | -0.5% | +| electro | scan rendered as depth (evaluation floor) | 2 | 2% | 0.2 / 0.9 | 0.4 / 1.4 | -0.4% | +| electro | MoGe-2 per photo + true poses | 1 | 2% | 1.5 / 10.1 | 3.2 / 21.8 | +4.8% | +| electro | MoGe-2 per photo + true poses | 2 | 2% | 1.8 / 10.9 | 3.5 / 23.2 | +3.3% | +| electro | Depth Anything 3 metric per photo + true poses | 1 | 2% | 2.5 / 31.7 | 5.2 / 119.5 | -8.6% | +| electro | Depth Anything 3 metric per photo + true poses | 2 | 2% | 2.6 / 26.4 | 4.8 / 79.3 | -8.3% | +| electro | scan rendered as depth (evaluation floor) | 1 | 4% | 0.5 / 1.5 | 0.9 / 2.7 | -0.8% | | electro | scan rendered as depth (evaluation floor) | 2 | 4% | 0.5 / 1.6 | 0.9 / 2.8 | -0.8% | -| electro | MoGe-2 per photo + true poses | 1 | 4% | 1.8 / 8.7 | 3.5 / 17.5 | +4.8% | -| electro | MoGe-2 per photo + true poses | 2 | 4% | 2.3 / 11.3 | 4.0 / 21.1 | +3.8% | -| electro | Depth Anything 3 metric per photo + true poses | 1 | 4% | 3.3 / 24.8 | 7.4 / 52.2 | -7.6% | -| electro | Depth Anything 3 metric per photo + true poses | 2 | 4% | 3.6 / 21.6 | 7.1 / 44.4 | -7.4% | -| electro | scan rendered as depth (evaluation floor) | 1 | 8% | 0.6 / 1.7 | 1.0 / 2.9 | -0.8% | -| electro | scan rendered as depth (evaluation floor) | 2 | 8% | 0.5 / 1.8 | 1.0 / 3.0 | -0.8% | -| electro | MoGe-2 per photo + true poses | 1 | 8% | 1.8 / 10.0 | 3.3 / 19.7 | +4.8% | -| electro | MoGe-2 per photo + true poses | 2 | 8% | 2.4 / 13.9 | 3.9 / 26.2 | +3.6% | -| electro | Depth Anything 3 metric per photo + true poses | 1 | 8% | 3.3 / 29.7 | 7.0 / 69.1 | -7.5% | -| electro | Depth Anything 3 metric per photo + true poses | 2 | 8% | 3.6 / 25.8 | 6.8 / 57.2 | -7.3% | +| electro | MoGe-2 per photo + true poses | 1 | 4% | 1.7 / 8.3 | 3.4 / 16.6 | +4.8% | +| electro | MoGe-2 per photo + true poses | 2 | 4% | 2.4 / 11.8 | 4.1 / 22.0 | +3.8% | +| electro | Depth Anything 3 metric per photo + true poses | 1 | 4% | 3.4 / 24.8 | 7.1 / 52.2 | -7.6% | +| electro | Depth Anything 3 metric per photo + true poses | 2 | 4% | 3.9 / 22.4 | 7.6 / 46.7 | -7.4% | +| electro | scan rendered as depth (evaluation floor) | 1 | 8% | 0.5 / 1.6 | 1.0 / 2.7 | -0.8% | +| electro | scan rendered as depth (evaluation floor) | 2 | 8% | 0.5 / 1.7 | 0.9 / 2.9 | -0.8% | +| electro | MoGe-2 per photo + true poses | 1 | 8% | 1.8 / 9.7 | 3.4 / 18.8 | +4.8% | +| electro | MoGe-2 per photo + true poses | 2 | 8% | 2.3 / 12.7 | 3.8 / 23.7 | +3.6% | +| electro | Depth Anything 3 metric per photo + true poses | 1 | 8% | 3.2 / 28.3 | 7.0 / 62.6 | -7.5% | +| electro | Depth Anything 3 metric per photo + true poses | 2 | 8% | 3.5 / 23.9 | 6.3 / 50.2 | -7.3% | diff --git a/experiments/evals/results/frames.json b/experiments/evals/results/frames.json index f06a9e326..0d89c8d85 100644 --- a/experiments/evals/results/frames.json +++ b/experiments/evals/results/frames.json @@ -7,15 +7,15 @@ "pairs": 9075000, "failed_pct": 0.4, "median_in": 2.73, - "p90_in": 11.45, - "scale_error_pct": -0.0 + "p90_in": 11.36, + "scale_error_pct": 0.0 }, "3-10m": { "pairs": 9075000, "failed_pct": 0.49, - "median_in": 7.62, - "p90_in": 38.97, - "scale_error_pct": -0.51 + "median_in": 7.6, + "p90_in": 38.7, + "scale_error_pct": -0.52 } }, "wall within 6 m": { @@ -25,15 +25,15 @@ "pairs": 1125000, "failed_pct": 0.39, "median_in": 1.94, - "p90_in": 6.86, + "p90_in": 6.84, "scale_error_pct": -0.09 }, "3-10m": { "pairs": 1125000, "failed_pct": 0.5, "median_in": 4.29, - "p90_in": 14.82, - "scale_error_pct": -0.04 + "p90_in": 14.79, + "scale_error_pct": -0.05 } }, "wall within 8 m": { @@ -42,16 +42,16 @@ "1-3m": { "pairs": 2025000, "failed_pct": 0.22, - "median_in": 2.1, - "p90_in": 7.38, - "scale_error_pct": -0.14 + "median_in": 2.11, + "p90_in": 7.36, + "scale_error_pct": -0.15 }, "3-10m": { "pairs": 2025000, "failed_pct": 0.28, - "median_in": 4.8, - "p90_in": 17.03, - "scale_error_pct": -0.22 + "median_in": 4.81, + "p90_in": 17.0, + "scale_error_pct": -0.23 } }, "wall within 12 m": { @@ -61,15 +61,15 @@ "pairs": 3150000, "failed_pct": 0.19, "median_in": 2.29, - "p90_in": 8.09, - "scale_error_pct": -0.11 + "p90_in": 8.08, + "scale_error_pct": -0.12 }, "3-10m": { "pairs": 3150000, "failed_pct": 0.25, - "median_in": 5.41, - "p90_in": 20.42, - "scale_error_pct": -0.19 + "median_in": 5.42, + "p90_in": 20.38, + "scale_error_pct": -0.21 } }, "wall seen within 40 degrees of head-on": { @@ -78,16 +78,16 @@ "1-3m": { "pairs": 3900000, "failed_pct": 0.28, - "median_in": 2.31, - "p90_in": 9.3, - "scale_error_pct": 0.06 + "median_in": 2.3, + "p90_in": 9.1, + "scale_error_pct": 0.04 }, "3-10m": { "pairs": 3900000, "failed_pct": 0.36, - "median_in": 7.07, - "p90_in": 35.2, - "scale_error_pct": -0.63 + "median_in": 7.05, + "p90_in": 34.49, + "scale_error_pct": -0.65 } }, "wall seen more than 50 degrees off head-on": { @@ -96,16 +96,16 @@ "1-3m": { "pairs": 3000000, "failed_pct": 0.54, - "median_in": 2.83, - "p90_in": 11.67, - "scale_error_pct": -0.08 + "median_in": 2.84, + "p90_in": 11.65, + "scale_error_pct": -0.09 }, "3-10m": { "pairs": 3000000, "failed_pct": 0.64, "median_in": 7.12, - "p90_in": 32.78, - "scale_error_pct": -0.25 + "p90_in": 32.75, + "scale_error_pct": -0.27 } } }, @@ -141,16 +141,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.1, - "median_in": 1.31, - "p90_in": 5.58, - "scale_error_pct": 0.21 + "median_in": 1.28, + "p90_in": 5.57, + "scale_error_pct": 0.19 }, "3-10m": { "pairs": 75000, "failed_pct": 0.13, - "median_in": 4.27, - "p90_in": 24.88, - "scale_error_pct": -0.18 + "median_in": 4.18, + "p90_in": 24.94, + "scale_error_pct": -0.19 } } }, @@ -163,16 +163,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.07, - "median_in": 1.79, - "p90_in": 5.09, - "scale_error_pct": 0.41 + "median_in": 1.76, + "p90_in": 5.06, + "scale_error_pct": 0.42 }, "3-10m": { "pairs": 75000, "failed_pct": 0.2, - "median_in": 5.05, - "p90_in": 14.17, - "scale_error_pct": 0.16 + "median_in": 4.91, + "p90_in": 13.99, + "scale_error_pct": 0.17 } } }, @@ -185,16 +185,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.1, - "median_in": 1.48, - "p90_in": 4.26, + "median_in": 1.43, + "p90_in": 4.24, "scale_error_pct": 0.08 }, "3-10m": { "pairs": 75000, "failed_pct": 0.13, - "median_in": 3.99, - "p90_in": 11.39, - "scale_error_pct": -0.0 + "median_in": 3.8, + "p90_in": 11.23, + "scale_error_pct": 0.04 } } }, @@ -229,16 +229,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 2.45, - "p90_in": 6.45, - "scale_error_pct": -0.87 + "median_in": 2.36, + "p90_in": 6.23, + "scale_error_pct": -0.75 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 6.94, - "p90_in": 18.2, - "scale_error_pct": -1.57 + "median_in": 6.67, + "p90_in": 17.21, + "scale_error_pct": -1.47 } } }, @@ -273,16 +273,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.03, - "median_in": 1.47, - "p90_in": 4.24, - "scale_error_pct": 0.43 + "median_in": 1.46, + "p90_in": 4.23, + "scale_error_pct": 0.41 }, "3-10m": { "pairs": 75000, "failed_pct": 0.07, - "median_in": 4.12, - "p90_in": 11.91, - "scale_error_pct": 0.3 + "median_in": 4.09, + "p90_in": 11.87, + "scale_error_pct": 0.29 } } }, @@ -317,16 +317,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.1, - "median_in": 1.94, - "p90_in": 5.4, - "scale_error_pct": -0.56 + "median_in": 1.99, + "p90_in": 5.51, + "scale_error_pct": -0.4 }, "3-10m": { "pairs": 75000, "failed_pct": 0.3, - "median_in": 6.34, - "p90_in": 19.18, - "scale_error_pct": -1.33 + "median_in": 6.5, + "p90_in": 19.26, + "scale_error_pct": -1.21 } } }, @@ -361,16 +361,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.13, - "median_in": 2.99, - "p90_in": 8.91, - "scale_error_pct": -0.38 + "median_in": 2.79, + "p90_in": 7.87, + "scale_error_pct": 0.15 }, "3-10m": { "pairs": 75000, "failed_pct": 0.23, - "median_in": 10.04, - "p90_in": 29.62, - "scale_error_pct": -1.19 + "median_in": 9.28, + "p90_in": 26.51, + "scale_error_pct": -0.49 } } }, @@ -383,16 +383,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.2, - "median_in": 3.11, - "p90_in": 8.66, + "median_in": 2.77, + "p90_in": 7.53, "scale_error_pct": -0.1 }, "3-10m": { "pairs": 75000, "failed_pct": 0.4, - "median_in": 11.12, - "p90_in": 33.73, - "scale_error_pct": -0.6 + "median_in": 9.73, + "p90_in": 29.93, + "scale_error_pct": -0.75 } } }, @@ -405,16 +405,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.2, - "median_in": 2.71, - "p90_in": 7.17, - "scale_error_pct": 0.42 + "median_in": 2.78, + "p90_in": 7.36, + "scale_error_pct": 1.62 }, "3-10m": { "pairs": 75000, "failed_pct": 0.43, - "median_in": 9.67, - "p90_in": 29.81, - "scale_error_pct": -0.35 + "median_in": 9.66, + "p90_in": 28.14, + "scale_error_pct": 0.58 } } }, @@ -449,16 +449,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.17, - "median_in": 5.26, - "p90_in": 19.79, - "scale_error_pct": -1.44 + "median_in": 4.88, + "p90_in": 19.71, + "scale_error_pct": -1.66 }, "3-10m": { "pairs": 75000, "failed_pct": 0.27, - "median_in": 20.6, - "p90_in": 75.26, - "scale_error_pct": -6.07 + "median_in": 19.73, + "p90_in": 75.06, + "scale_error_pct": -5.96 } } }, @@ -493,16 +493,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.33, - "median_in": 5.1, - "p90_in": 18.32, - "scale_error_pct": 0.36 + "median_in": 3.74, + "p90_in": 11.95, + "scale_error_pct": 0.02 }, "3-10m": { "pairs": 75000, "failed_pct": 0.7, - "median_in": 19.65, - "p90_in": 71.73, - "scale_error_pct": -0.39 + "median_in": 14.05, + "p90_in": 58.74, + "scale_error_pct": -0.86 } } }, @@ -515,16 +515,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.33, - "median_in": 3.4, - "p90_in": 15.09, - "scale_error_pct": 0.49 + "median_in": 3.39, + "p90_in": 14.99, + "scale_error_pct": 0.17 }, "3-10m": { "pairs": 75000, "failed_pct": 0.47, - "median_in": 12.21, - "p90_in": 51.18, - "scale_error_pct": -1.08 + "median_in": 12.12, + "p90_in": 51.6, + "scale_error_pct": -1.21 } } }, @@ -537,16 +537,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.33, - "median_in": 3.09, - "p90_in": 12.33, - "scale_error_pct": 0.87 + "median_in": 2.89, + "p90_in": 11.03, + "scale_error_pct": 0.5 }, "3-10m": { "pairs": 75000, "failed_pct": 0.37, - "median_in": 10.32, - "p90_in": 41.18, - "scale_error_pct": -0.38 + "median_in": 9.68, + "p90_in": 40.26, + "scale_error_pct": -0.48 } } }, @@ -559,16 +559,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.47, - "median_in": 2.58, - "p90_in": 8.14, - "scale_error_pct": -0.55 + "median_in": 2.53, + "p90_in": 7.81, + "scale_error_pct": -0.61 }, "3-10m": { "pairs": 75000, "failed_pct": 0.57, - "median_in": 8.69, - "p90_in": 39.71, - "scale_error_pct": -1.59 + "median_in": 8.51, + "p90_in": 39.91, + "scale_error_pct": -1.64 } } }, @@ -581,16 +581,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.2, - "median_in": 3.48, - "p90_in": 10.15, - "scale_error_pct": 1.13 + "median_in": 3.54, + "p90_in": 10.19, + "scale_error_pct": 0.97 }, "3-10m": { "pairs": 75000, "failed_pct": 0.47, - "median_in": 11.8, - "p90_in": 40.31, - "scale_error_pct": -0.51 + "median_in": 12.05, + "p90_in": 40.64, + "scale_error_pct": -0.65 } } }, @@ -603,16 +603,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.5, - "median_in": 2.93, - "p90_in": 9.41, - "scale_error_pct": -1.35 + "median_in": 3.22, + "p90_in": 10.17, + "scale_error_pct": -1.77 }, "3-10m": { "pairs": 75000, "failed_pct": 0.37, - "median_in": 10.9, - "p90_in": 42.65, - "scale_error_pct": -3.03 + "median_in": 12.19, + "p90_in": 44.39, + "scale_error_pct": -3.68 } } }, @@ -625,16 +625,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.33, - "median_in": 3.86, - "p90_in": 15.2, - "scale_error_pct": -1.14 + "median_in": 3.68, + "p90_in": 15.05, + "scale_error_pct": -1.3 }, "3-10m": { "pairs": 75000, "failed_pct": 0.4, - "median_in": 13.96, - "p90_in": 58.96, - "scale_error_pct": -2.67 + "median_in": 13.36, + "p90_in": 59.28, + "scale_error_pct": -2.83 } } }, @@ -647,16 +647,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.23, - "median_in": 4.56, - "p90_in": 13.12, - "scale_error_pct": 0.95 + "median_in": 4.74, + "p90_in": 13.32, + "scale_error_pct": 1.18 }, "3-10m": { "pairs": 75000, "failed_pct": 0.4, - "median_in": 14.8, - "p90_in": 44.47, - "scale_error_pct": -0.49 + "median_in": 15.28, + "p90_in": 44.91, + "scale_error_pct": -0.25 } } }, @@ -669,16 +669,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.37, - "median_in": 4.41, - "p90_in": 12.86, - "scale_error_pct": 0.82 + "median_in": 4.45, + "p90_in": 12.84, + "scale_error_pct": 1.12 }, "3-10m": { "pairs": 75000, "failed_pct": 0.33, - "median_in": 14.73, - "p90_in": 46.91, - "scale_error_pct": -0.2 + "median_in": 14.93, + "p90_in": 46.75, + "scale_error_pct": -0.05 } } }, @@ -691,16 +691,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.27, - "median_in": 5.24, - "p90_in": 13.59, - "scale_error_pct": 1.13 + "median_in": 5.11, + "p90_in": 13.1, + "scale_error_pct": 0.94 }, "3-10m": { "pairs": 75000, "failed_pct": 0.27, - "median_in": 18.2, - "p90_in": 50.65, - "scale_error_pct": -0.16 + "median_in": 17.85, + "p90_in": 49.0, + "scale_error_pct": -0.3 } } }, @@ -713,16 +713,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.2, - "median_in": 6.74, - "p90_in": 17.45, - "scale_error_pct": 1.16 + "median_in": 6.8, + "p90_in": 17.46, + "scale_error_pct": 1.46 }, "3-10m": { "pairs": 75000, "failed_pct": 0.3, - "median_in": 21.31, + "median_in": 21.47, "p90_in": 57.99, - "scale_error_pct": 0.22 + "scale_error_pct": 0.36 } } }, @@ -757,16 +757,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 4.88, - "p90_in": 13.52, - "scale_error_pct": 1.89 + "median_in": 5.03, + "p90_in": 13.84, + "scale_error_pct": 2.34 }, "3-10m": { "pairs": 75000, "failed_pct": 0.13, - "median_in": 16.66, - "p90_in": 47.82, - "scale_error_pct": 0.6 + "median_in": 17.08, + "p90_in": 48.05, + "scale_error_pct": 0.99 } } }, @@ -779,16 +779,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.17, - "median_in": 4.32, - "p90_in": 12.45, - "scale_error_pct": 0.56 + "median_in": 4.14, + "p90_in": 12.09, + "scale_error_pct": 0.99 }, "3-10m": { "pairs": 75000, "failed_pct": 0.3, - "median_in": 13.91, - "p90_in": 38.48, - "scale_error_pct": -0.22 + "median_in": 13.37, + "p90_in": 37.03, + "scale_error_pct": 0.19 } } }, @@ -801,16 +801,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.23, - "median_in": 3.59, - "p90_in": 9.92, - "scale_error_pct": -1.04 + "median_in": 3.49, + "p90_in": 9.66, + "scale_error_pct": -0.82 }, "3-10m": { "pairs": 75000, "failed_pct": 0.5, - "median_in": 11.84, - "p90_in": 30.33, - "scale_error_pct": -1.88 + "median_in": 11.55, + "p90_in": 29.49, + "scale_error_pct": -1.65 } } }, @@ -824,15 +824,15 @@ "pairs": 75000, "failed_pct": 0.3, "median_in": 3.59, - "p90_in": 10.02, - "scale_error_pct": -2.07 + "p90_in": 10.18, + "scale_error_pct": -1.48 }, "3-10m": { "pairs": 75000, "failed_pct": 0.5, - "median_in": 11.65, - "p90_in": 30.9, - "scale_error_pct": -2.94 + "median_in": 11.51, + "p90_in": 31.27, + "scale_error_pct": -2.39 } } }, @@ -845,16 +845,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 3.67, - "p90_in": 10.58, - "scale_error_pct": -0.92 + "median_in": 3.48, + "p90_in": 10.52, + "scale_error_pct": -0.82 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 10.54, - "p90_in": 31.28, - "scale_error_pct": -1.37 + "median_in": 9.85, + "p90_in": 30.77, + "scale_error_pct": -1.22 } } }, @@ -869,14 +869,14 @@ "failed_pct": 0.13, "median_in": 2.17, "p90_in": 8.24, - "scale_error_pct": 0.37 + "scale_error_pct": 0.35 }, "3-10m": { "pairs": 75000, "failed_pct": 0.23, - "median_in": 5.71, + "median_in": 5.7, "p90_in": 18.2, - "scale_error_pct": 0.29 + "scale_error_pct": 0.28 } } }, @@ -889,16 +889,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.03, - "median_in": 2.28, - "p90_in": 6.73, - "scale_error_pct": 0.42 + "median_in": 2.39, + "p90_in": 6.86, + "scale_error_pct": 0.28 }, "3-10m": { "pairs": 75000, "failed_pct": 0.13, - "median_in": 6.1, - "p90_in": 17.94, - "scale_error_pct": 0.39 + "median_in": 6.49, + "p90_in": 18.49, + "scale_error_pct": 0.26 } } }, @@ -933,16 +933,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.03, - "median_in": 1.28, - "p90_in": 4.66, - "scale_error_pct": -0.16 + "median_in": 1.4, + "p90_in": 5.98, + "scale_error_pct": -0.34 }, "3-10m": { "pairs": 75000, "failed_pct": 0.03, - "median_in": 3.97, - "p90_in": 17.5, - "scale_error_pct": -0.61 + "median_in": 4.35, + "p90_in": 21.73, + "scale_error_pct": -0.84 } } }, @@ -955,16 +955,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.07, - "median_in": 1.31, - "p90_in": 3.83, - "scale_error_pct": -0.18 + "median_in": 1.39, + "p90_in": 4.13, + "scale_error_pct": -0.35 }, "3-10m": { "pairs": 75000, "failed_pct": 0.03, - "median_in": 4.41, - "p90_in": 17.95, - "scale_error_pct": -0.99 + "median_in": 4.73, + "p90_in": 19.3, + "scale_error_pct": -1.2 } } }, @@ -1131,16 +1131,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.13, - "median_in": 2.16, - "p90_in": 5.93, - "scale_error_pct": -0.03 + "median_in": 2.33, + "p90_in": 6.29, + "scale_error_pct": 0.53 }, "3-10m": { "pairs": 75000, "failed_pct": 0.37, - "median_in": 5.91, - "p90_in": 16.94, - "scale_error_pct": -0.36 + "median_in": 6.56, + "p90_in": 17.56, + "scale_error_pct": 0.12 } } }, @@ -1153,16 +1153,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 1.91, + "median_in": 1.9, "p90_in": 5.69, - "scale_error_pct": 0.45 + "scale_error_pct": 0.39 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 4.71, - "p90_in": 14.25, - "scale_error_pct": 0.22 + "median_in": 4.65, + "p90_in": 14.24, + "scale_error_pct": 0.17 } } }, @@ -1219,16 +1219,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 2.43, - "p90_in": 7.49, - "scale_error_pct": 0.28 + "median_in": 2.37, + "p90_in": 7.44, + "scale_error_pct": 0.11 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 6.3, - "p90_in": 18.53, - "scale_error_pct": 0.19 + "median_in": 6.02, + "p90_in": 18.41, + "scale_error_pct": 0.01 } } }, @@ -1241,16 +1241,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.73, - "median_in": 5.12, - "p90_in": 13.81, - "scale_error_pct": -0.09 + "median_in": 5.11, + "p90_in": 13.8, + "scale_error_pct": -0.07 }, "3-10m": { "pairs": 75000, "failed_pct": 0.7, - "median_in": 14.61, - "p90_in": 41.25, - "scale_error_pct": -0.21 + "median_in": 14.58, + "p90_in": 41.24, + "scale_error_pct": -0.2 } } }, @@ -1285,16 +1285,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 4.13, - "median_in": 6.6, - "p90_in": 23.22, - "scale_error_pct": -1.14 + "median_in": 7.07, + "p90_in": 23.97, + "scale_error_pct": -0.76 }, "3-10m": { "pairs": 75000, "failed_pct": 4.29, - "median_in": 21.19, - "p90_in": 84.12, - "scale_error_pct": -1.91 + "median_in": 22.78, + "p90_in": 85.9, + "scale_error_pct": -1.66 } } }, @@ -1351,16 +1351,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 5.17, - "p90_in": 16.68, - "scale_error_pct": 0.32 + "median_in": 4.89, + "p90_in": 13.51, + "scale_error_pct": 0.26 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 21.63, - "p90_in": 79.9, - "scale_error_pct": -1.08 + "median_in": 20.24, + "p90_in": 73.61, + "scale_error_pct": -1.62 } } }, @@ -1439,16 +1439,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 3.03, - "p90_in": 15.64, - "scale_error_pct": 0.98 + "median_in": 3.01, + "p90_in": 16.61, + "scale_error_pct": 0.97 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 9.56, - "p90_in": 105.21, - "scale_error_pct": 0.16 + "median_in": 9.52, + "p90_in": 104.78, + "scale_error_pct": 0.12 } } }, @@ -1461,16 +1461,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 2.86, - "p90_in": 11.64, - "scale_error_pct": -0.0 + "median_in": 2.7, + "p90_in": 11.59, + "scale_error_pct": -0.27 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 8.64, - "p90_in": 91.35, - "scale_error_pct": -1.02 + "median_in": 8.05, + "p90_in": 91.93, + "scale_error_pct": -1.23 } } }, @@ -1483,16 +1483,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 4.03, - "p90_in": 17.94, - "scale_error_pct": 0.7 + "median_in": 4.04, + "p90_in": 17.93, + "scale_error_pct": 0.77 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 17.68, - "p90_in": 90.0, - "scale_error_pct": -0.17 + "median_in": 17.64, + "p90_in": 89.9, + "scale_error_pct": -0.1 } } }, @@ -1505,16 +1505,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 5.15, - "p90_in": 13.7, - "scale_error_pct": -0.35 + "median_in": 5.31, + "p90_in": 13.76, + "scale_error_pct": -0.08 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 19.1, - "p90_in": 72.58, - "scale_error_pct": -1.68 + "median_in": 19.77, + "p90_in": 71.38, + "scale_error_pct": -1.54 } } }, @@ -1527,16 +1527,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.13, - "median_in": 4.65, - "p90_in": 12.76, - "scale_error_pct": 0.66 + "median_in": 4.41, + "p90_in": 12.22, + "scale_error_pct": 0.64 }, "3-10m": { "pairs": 75000, "failed_pct": 0.2, - "median_in": 16.15, - "p90_in": 53.04, - "scale_error_pct": -1.65 + "median_in": 15.28, + "p90_in": 52.72, + "scale_error_pct": -1.44 } } }, @@ -1571,16 +1571,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.3, - "median_in": 2.84, - "p90_in": 9.25, - "scale_error_pct": 0.26 + "median_in": 3.13, + "p90_in": 12.36, + "scale_error_pct": 0.18 }, "3-10m": { "pairs": 75000, "failed_pct": 0.67, - "median_in": 9.86, - "p90_in": 41.58, - "scale_error_pct": -1.07 + "median_in": 10.68, + "p90_in": 45.75, + "scale_error_pct": -1.18 } } }, @@ -1593,16 +1593,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.33, - "median_in": 3.26, - "p90_in": 12.89, - "scale_error_pct": 0.74 + "median_in": 3.27, + "p90_in": 12.67, + "scale_error_pct": 0.36 }, "3-10m": { "pairs": 75000, "failed_pct": 0.23, - "median_in": 11.35, - "p90_in": 43.41, - "scale_error_pct": -0.33 + "median_in": 11.46, + "p90_in": 44.24, + "scale_error_pct": -0.56 } } }, @@ -1615,16 +1615,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 4.13, - "median_in": 5.18, - "p90_in": 23.03, - "scale_error_pct": 1.66 + "median_in": 5.22, + "p90_in": 22.91, + "scale_error_pct": 1.37 }, "3-10m": { "pairs": 75000, "failed_pct": 4.35, - "median_in": 17.42, - "p90_in": 69.25, - "scale_error_pct": -0.02 + "median_in": 17.46, + "p90_in": 69.19, + "scale_error_pct": -0.42 } } }, @@ -1637,16 +1637,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.1, - "median_in": 6.58, - "p90_in": 18.1, - "scale_error_pct": -0.24 + "median_in": 6.78, + "p90_in": 20.38, + "scale_error_pct": 0.83 }, "3-10m": { "pairs": 75000, "failed_pct": 0.33, - "median_in": 20.91, - "p90_in": 61.76, - "scale_error_pct": -2.05 + "median_in": 21.51, + "p90_in": 69.43, + "scale_error_pct": -0.86 } } }, @@ -1659,16 +1659,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.17, - "median_in": 6.86, - "p90_in": 17.57, - "scale_error_pct": 0.2 + "median_in": 6.84, + "p90_in": 17.37, + "scale_error_pct": 0.27 }, "3-10m": { "pairs": 75000, "failed_pct": 0.57, - "median_in": 20.66, - "p90_in": 56.75, - "scale_error_pct": -0.09 + "median_in": 21.05, + "p90_in": 56.04, + "scale_error_pct": -0.01 } } }, @@ -1681,16 +1681,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 4.22, - "median_in": 6.53, + "median_in": 6.51, "p90_in": 16.87, - "scale_error_pct": 0.55 + "scale_error_pct": 0.8 }, "3-10m": { "pairs": 75000, "failed_pct": 4.7, - "median_in": 20.18, - "p90_in": 48.17, - "scale_error_pct": -0.38 + "median_in": 19.99, + "p90_in": 47.8, + "scale_error_pct": -0.03 } } }, @@ -1703,16 +1703,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 1.37, - "p90_in": 4.65, - "scale_error_pct": 0.12 + "median_in": 1.31, + "p90_in": 4.1, + "scale_error_pct": -0.03 }, "3-10m": { "pairs": 75000, "failed_pct": 0.07, - "median_in": 4.04, - "p90_in": 17.29, - "scale_error_pct": -0.27 + "median_in": 3.9, + "p90_in": 15.38, + "scale_error_pct": -0.48 } } }, @@ -1791,16 +1791,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 2.8, - "p90_in": 9.46, - "scale_error_pct": -0.79 + "median_in": 2.76, + "p90_in": 9.44, + "scale_error_pct": -0.9 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 6.64, - "p90_in": 23.39, - "scale_error_pct": 0.02 + "median_in": 6.45, + "p90_in": 23.36, + "scale_error_pct": -0.11 } } }, @@ -1901,16 +1901,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 1.44, - "p90_in": 5.56, - "scale_error_pct": -0.22 + "median_in": 1.49, + "p90_in": 5.57, + "scale_error_pct": -0.17 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 3.47, - "p90_in": 12.75, - "scale_error_pct": -0.14 + "median_in": 3.59, + "p90_in": 12.86, + "scale_error_pct": -0.1 } } }, @@ -1945,16 +1945,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.73, - "median_in": 4.0, - "p90_in": 13.46, - "scale_error_pct": -0.65 + "median_in": 3.95, + "p90_in": 13.42, + "scale_error_pct": -0.69 }, "3-10m": { "pairs": 75000, "failed_pct": 0.57, - "median_in": 11.0, - "p90_in": 38.6, - "scale_error_pct": -0.93 + "median_in": 10.93, + "p90_in": 38.46, + "scale_error_pct": -1.01 } } }, @@ -2121,16 +2121,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 1.41, - "p90_in": 5.41, - "scale_error_pct": -0.21 + "median_in": 1.5, + "p90_in": 5.47, + "scale_error_pct": -0.26 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 2.89, - "p90_in": 11.69, - "scale_error_pct": -0.11 + "median_in": 3.11, + "p90_in": 11.92, + "scale_error_pct": -0.14 } } }, @@ -2143,16 +2143,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 2.03, - "p90_in": 7.36, - "scale_error_pct": 0.56 + "median_in": 2.01, + "p90_in": 7.37, + "scale_error_pct": 0.47 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 4.58, + "median_in": 4.48, "p90_in": 17.25, - "scale_error_pct": 1.26 + "scale_error_pct": 1.17 } } }, @@ -2165,16 +2165,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 1.07, - "p90_in": 4.06, - "scale_error_pct": 0.14 + "median_in": 1.08, + "p90_in": 4.07, + "scale_error_pct": 0.09 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 1.74, + "median_in": 1.76, "p90_in": 7.79, - "scale_error_pct": 0.48 + "scale_error_pct": 0.43 } } }, @@ -2209,16 +2209,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.03, - "median_in": 3.15, - "p90_in": 12.34, - "scale_error_pct": -0.11 + "median_in": 3.12, + "p90_in": 11.84, + "scale_error_pct": -0.06 }, "3-10m": { "pairs": 75000, "failed_pct": 0.1, - "median_in": 8.83, - "p90_in": 32.97, - "scale_error_pct": -0.77 + "median_in": 8.77, + "p90_in": 31.31, + "scale_error_pct": -0.74 } } }, @@ -2231,16 +2231,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 1.61, + "median_in": 1.65, "p90_in": 5.02, - "scale_error_pct": 0.14 + "scale_error_pct": 0.06 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 3.9, - "p90_in": 12.61, - "scale_error_pct": 0.15 + "median_in": 4.04, + "p90_in": 12.63, + "scale_error_pct": 0.04 } } }, @@ -2408,15 +2408,15 @@ "pairs": 75000, "failed_pct": 0.6, "median_in": 3.91, - "p90_in": 11.18, - "scale_error_pct": 0.25 + "p90_in": 11.27, + "scale_error_pct": -0.12 }, "3-10m": { "pairs": 75000, "failed_pct": 0.8, - "median_in": 11.51, - "p90_in": 34.45, - "scale_error_pct": -0.62 + "median_in": 11.16, + "p90_in": 36.13, + "scale_error_pct": -0.96 } } }, @@ -2451,16 +2451,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.17, - "median_in": 10.72, - "p90_in": 53.41, - "scale_error_pct": 8.54 + "median_in": 10.68, + "p90_in": 53.39, + "scale_error_pct": 8.48 }, "3-10m": { "pairs": 75000, "failed_pct": 0.53, - "median_in": 33.27, - "p90_in": 117.69, - "scale_error_pct": -0.9 + "median_in": 33.25, + "p90_in": 117.57, + "scale_error_pct": -0.95 } } }, @@ -2473,16 +2473,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.37, - "median_in": 15.52, - "p90_in": 63.39, - "scale_error_pct": -0.13 + "median_in": 14.52, + "p90_in": 61.92, + "scale_error_pct": 0.39 }, "3-10m": { "pairs": 75000, "failed_pct": 0.17, - "median_in": 44.66, - "p90_in": 140.55, - "scale_error_pct": -7.02 + "median_in": 42.33, + "p90_in": 126.4, + "scale_error_pct": -6.4 } } }, @@ -2649,16 +2649,16 @@ "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 4.01, - "p90_in": 10.55, - "scale_error_pct": -1.23 + "median_in": 4.0, + "p90_in": 10.58, + "scale_error_pct": -1.5 }, "3-10m": { "pairs": 75000, "failed_pct": 0.23, - "median_in": 10.83, - "p90_in": 26.94, - "scale_error_pct": -2.03 + "median_in": 10.69, + "p90_in": 27.21, + "scale_error_pct": -2.29 } } }, diff --git a/experiments/evals/results/frames.md b/experiments/evals/results/frames.md index cced792d1..073a9533a 100644 --- a/experiments/evals/results/frames.md +++ b/experiments/evals/results/frames.md @@ -4,11 +4,11 @@ MoGe-2, one photo at a time, scaled by one taped 1 to 3 m distance; both ETH3D s | Keep | Photos kept | 1-3 m spans | 3-10 m spans | | --- | --- | --- | --- | -| every photo | 121 of 121 | 2.7 / 11.4 | 7.6 / 39.0 | -| wall within 6 m | 15 of 121 | 1.9 / 6.9 | 4.3 / 14.8 | +| every photo | 121 of 121 | 2.7 / 11.4 | 7.6 / 38.7 | +| wall within 6 m | 15 of 121 | 1.9 / 6.8 | 4.3 / 14.8 | | wall within 8 m | 27 of 121 | 2.1 / 7.4 | 4.8 / 17.0 | | wall within 12 m | 42 of 121 | 2.3 / 8.1 | 5.4 / 20.4 | -| wall seen within 40 degrees of head-on | 52 of 121 | 2.3 / 9.3 | 7.1 / 35.2 | +| wall seen within 40 degrees of head-on | 52 of 121 | 2.3 / 9.1 | 7.0 / 34.5 | | wall seen more than 50 degrees off head-on | 40 of 121 | 2.8 / 11.7 | 7.1 / 32.8 | Photo geometry: median wall distance 15.4 m (range 4.0 to 40.3); median incidence 42 degrees (range 26 to 75). diff --git a/experiments/evals/results/map3d.json b/experiments/evals/results/map3d.json index b4349ef31..78626271e 100644 --- a/experiments/evals/results/map3d.json +++ b/experiments/evals/results/map3d.json @@ -8,7 +8,7 @@ "false_observed_share": 0.0, "missed_ft": 6.889763779527559, "seen_two_view_ft": 17.979002624671917, - "claimed_samples_seen_only_where_scan_is_empty": 0.0 + "claimed_samples_unknown_for_lack_of_scan": 0.0 }, "ground": { "claimed_ft": 0.0, @@ -16,7 +16,7 @@ "false_observed_share": 0.0, "missed_ft": 18.11023622047244, "seen_two_view_ft": 18.11023622047244, - "claimed_samples_seen_only_where_scan_is_empty": 0.0 + "claimed_samples_unknown_for_lack_of_scan": 0.0 } }, "bands_5m_truth": { @@ -24,9 +24,9 @@ "claimed_ft": 11.089238845144356, "false_observed_ft": 0.0, "false_observed_share": 0.0, - "missed_ft": 6.102362204724409, - "seen_two_view_ft": 16.076115485564305, - "claimed_samples_seen_only_where_scan_is_empty": 0.0 + "missed_ft": 4.6587926509186355, + "seen_two_view_ft": 14.63254593175853, + "claimed_samples_unknown_for_lack_of_scan": 0.0 }, "ground": { "claimed_ft": 0.0, @@ -34,7 +34,7 @@ "false_observed_share": 0.0, "missed_ft": 16.46981627296588, "seen_two_view_ft": 16.46981627296588, - "claimed_samples_seen_only_where_scan_is_empty": 0.0 + "claimed_samples_unknown_for_lack_of_scan": 0.0 } }, "wall_over_all_claims": { @@ -47,18 +47,18 @@ "false_observed_share": 0.0, "missed_ft": 8.858267716535433, "seen_two_view_ft": 25.196850393700785, - "claimed_samples_seen_only_where_scan_is_empty": 0.0, + "claimed_samples_unknown_for_lack_of_scan": 0.0, "missed_on_pilaster_faces_ft": 5.51 }, "facing_3cm_truth": { - "claimed_ft": 0.52, - "false_observed_ft": 0.52, - "passes": 0.0, + "claimed_ft": 0.5, + "false_observed_ft": 0.5, + "passes": 1.0, "spans_with_unseen_ft": [ [ 8.5, 9.0, - 0.52 + 0.5 ] ] }, @@ -74,14 +74,14 @@ "wall_claimed_total_ft": 19.5, "wall_claimed_on_stretch_ft": 11.09, "facing": { - "claimed_ft": 0.52, - "false_observed_ft": 0.52, - "passes": 0.0, + "claimed_ft": 0.5, + "false_observed_ft": 0.5, + "passes": 1.0, "spans_with_unseen_ft": [ [ 8.5, 9.0, - 0.52 + 0.5 ] ] }, diff --git a/experiments/evals/results/map3d.md b/experiments/evals/results/map3d.md index 0aa3c5614..08fce6d47 100644 --- a/experiments/evals/results/map3d.md +++ b/experiments/evals/results/map3d.md @@ -1,18 +1,18 @@ # The app's 3D map against laser visibility (generated by `make map3d`) -HouseScanKit Map3D at 66cdcba (t3/ios-map3d), unmodified, LiDAR path: one 256 x 192 depth frame per ETH3D electro photo, rendered from the laser scan. Coverage read along section 7's 19.3 ft wall. Lengths in feet along the wall; false-observed passes at 0.5 ft or less. Ideal depth: exact, dense, no sensor noise, so this is an upper bound on what real LiDAR gives. Definitions: README section 7c. +HouseScanKit Map3D at 66cdcba (t3/ios-map3d), unmodified, LiDAR path: one 256 x 192 depth frame per ETH3D electro photo, rendered from the laser scan. Coverage read along section 7's 19.3 ft wall. Lengths in feet along the wall; false-observed passes at 0.5 ft or less. Ideal depth: exact, dense, no sensor noise, so this is an upper bound on what real LiDAR gives. Definitions: METHODS.md section 7c. | Band | Claimed | False-observed (share) | Pass | Missed | Seen from 2 positions | | --- | --- | --- | --- | --- | --- | | wall, CoverageMap (section 7) | 19.3 | 1.1 (6%) | no | 0.0 | 18.0 | | wall, Map3D | 11.1 | 0.0 (0%) | yes | 6.9 | 18.0 | | ground, Map3D | 0.0 | 0.0 (0%) | untested | 18.1 | 18.1 | -| wall, Map3D, 5 m truth | 11.1 | 0.0 (0%) | yes | 6.1 | 16.1 | +| wall, Map3D, 5 m truth | 11.1 | 0.0 (0%) | yes | 4.7 | 14.6 | | ground, Map3D, 5 m truth | 0.0 | 0.0 (0%) | untested | 16.5 | 16.5 | | wall, Map3D, truth over all its claims (s -9.66 to 21.0 ft) | 19.4 | 0.0 (0%) | yes | 8.9, of it 5.5 on pilaster faces | 25.2 | | ground, Map3D, every reach at its own depth | 0.0 | 0.0 | untested | n/a | n/a | | facing, Map3D, 3 cm floor on the truth's tolerance | 0.5 | 0.5 | see note | n/a | n/a | -| facing, Map3D | 0.5 | 0.5 | no | n/a | n/a | +| facing, Map3D | 0.5 | 0.5 | yes | n/a | n/a | | overhead, Map3D | 0.0 | 0.0 | untested | n/a | n/a | Map3D's ground reaches (ft): [0.0]. Ground counts as claimed where the reach is at least 1.2 m (3.94 ft), the depth CoverageMap claims. diff --git a/experiments/evals/results/modern_arkit.md b/experiments/evals/results/modern_arkit.md index 3ebceb6d0..e83a4e412 100644 --- a/experiments/evals/results/modern_arkit.md +++ b/experiments/evals/results/modern_arkit.md @@ -11,9 +11,9 @@ MARViN, iPhone 14 Pro Max, ARKit 6, outdoor walks, against the dataset's COLMAP Walks within 2% of the ground truth's scale: 30 of 35. -## Walk-to-walk spread of ARKit's scale within a site +## Walk-to-walk spread of the ARKit / reference scale ratio within a site -Standard deviation over each site's walks within 10% of the reference, with a 95% bootstrap interval over walks. A reference scaled to ARKit would carry one scale per site, which shifts every walk of that site alike, so this spread survives it: a single walk's scale error is at least this large, unless the reference's own scale varies walk to walk in step with ARKit's. +Standard deviation over each site's walks within 10% of the reference, with a 95% bootstrap interval over walks. It measures how much ARKit and the reference disagree from walk to walk, not ARKit's error alone: a perfect ARKit paired with a reference whose local scale varies would give the same spread. A single site scale for the reference cannot remove its local reconstruction error, and no independent bound on that error exists here. | Site | Walks used | Spread (1 SD) | 95% interval | | --- | --- | --- | --- | @@ -45,7 +45,7 @@ Standard deviation over each site's walks within 10% of the reference, with a 95 ## Can GPS check the ground truth's scale? -GPS / truth over each walk: median 0.936, range 0.155 to 1.065 (35 walks). On the 11 walks where GPS fits within 1.5 m, median 0.978, range 0.923 to 1.065. GPS noise of 1 to 9 m over 25 to 160 m walks cannot pin the scale to 2%. +GPS / truth over each walk: median 0.936, range 0.155 to 1.065 (35 walks). On the 11 walks where GPS fits within 1.5 m, median 0.978, range 0.923 to 1.065. GPS noise of 1 to 9 m over 45 to 255 m walks cannot pin the scale to 2%. ## Files (Google Drive folder https://drive.google.com/drive/folders/18vPfKB4jlDwtFzytMijYOpmeqU1A9-Ie) diff --git a/experiments/evals/results/pose_priors.json b/experiments/evals/results/pose_priors.json index 91f240155..446a0440a 100644 --- a/experiments/evals/results/pose_priors.json +++ b/experiments/evals/results/pose_priors.json @@ -16,27 +16,27 @@ "pairs": 24000, "failed_pct": 10.8, "median_in": 39.42, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": -14.58 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 14.26, - "median_in": 6.22, - "p90_in": Infinity, - "scale_error_pct": -0.24 + "failed_pct": 13.35, + "median_in": 6.17, + "p90_in": "failed", + "scale_error_pct": -0.15 }, "3-10m": { "pairs": 600000, - "failed_pct": 16.5, - "median_in": 19.68, - "p90_in": Infinity, - "scale_error_pct": -1.51 + "failed_pct": 15.61, + "median_in": 19.51, + "p90_in": "failed", + "scale_error_pct": -1.43 } }, - "tape_calibration_success_pct": 93.5, + "tape_calibration_success_pct": 94.5, "groups": 8, "noise_draws": 1, "interval_95": { @@ -47,7 +47,7 @@ ], "p90_in": [ 22.53, - Infinity + "failed" ] }, "3-10m": { @@ -57,7 +57,7 @@ ], "p90_in": [ 93.26, - Infinity + "failed" ] } } @@ -66,57 +66,57 @@ "none": { "1-3m": { "pairs": 24000, - "failed_pct": 4.97, - "median_in": 11.14, - "p90_in": 24.8, - "scale_error_pct": -13.76 + "failed_pct": 5.32, + "median_in": 11.11, + "p90_in": 25.13, + "scale_error_pct": -13.7 }, "3-10m": { "pairs": 24000, - "failed_pct": 6.83, - "median_in": 37.34, - "p90_in": 95.49, + "failed_pct": 6.74, + "median_in": 37.56, + "p90_in": 94.4, "scale_error_pct": -15.01 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 11.08, - "median_in": 5.52, - "p90_in": Infinity, - "scale_error_pct": -0.57 + "failed_pct": 10.49, + "median_in": 5.49, + "p90_in": "failed", + "scale_error_pct": -0.43 }, "3-10m": { "pairs": 600000, - "failed_pct": 12.83, - "median_in": 17.6, - "p90_in": Infinity, - "scale_error_pct": -1.61 + "failed_pct": 11.84, + "median_in": 17.49, + "p90_in": "failed", + "scale_error_pct": -1.52 } }, - "tape_calibration_success_pct": 93.5, + "tape_calibration_success_pct": 94.5, "groups": 8, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 8.47, - 14.61 + 8.48, + 14.54 ], "p90_in": [ - 19.25, - 28.75 + 19.31, + 29.73 ] }, "3-10m": { "median_in": [ - 29.47, - 47.5 + 29.56, + 47.82 ], "p90_in": [ - 73.01, - 128.55 + 73.4, + 128.4 ] } } @@ -125,57 +125,57 @@ "none": { "1-3m": { "pairs": 24000, - "failed_pct": 84.82, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": -10.63 + "failed_pct": 84.83, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": -10.64 }, "3-10m": { "pairs": 24000, - "failed_pct": 91.11, - "median_in": Infinity, - "p90_in": Infinity, + "failed_pct": 91.1, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -12.44 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 85.76, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 5.36 + "failed_pct": 85.64, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 5.52 }, "3-10m": { "pairs": 600000, - "failed_pct": 91.77, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 3.73 + "failed_pct": 91.67, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 3.9 } }, - "tape_calibration_success_pct": 93.5, + "tape_calibration_success_pct": 94.5, "groups": 8, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -202,20 +202,20 @@ "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 9.89, - "median_in": 5.56, - "p90_in": 62.2, - "scale_error_pct": -0.24 + "failed_pct": 9.42, + "median_in": 5.6, + "p90_in": 40.72, + "scale_error_pct": -0.13 }, "3-10m": { "pairs": 600000, - "failed_pct": 11.13, - "median_in": 16.79, - "p90_in": Infinity, - "scale_error_pct": -1.52 + "failed_pct": 10.67, + "median_in": 16.9, + "p90_in": "failed", + "scale_error_pct": -1.42 } }, - "tape_calibration_success_pct": 95.0, + "tape_calibration_success_pct": 95.5, "groups": 8, "noise_draws": 1, "interval_95": { @@ -245,57 +245,57 @@ "none": { "1-3m": { "pairs": 24000, - "failed_pct": 2.47, - "median_in": 10.71, - "p90_in": 22.16, - "scale_error_pct": -13.12 + "failed_pct": 2.65, + "median_in": 10.68, + "p90_in": 22.3, + "scale_error_pct": -13.1 }, "3-10m": { "pairs": 24000, - "failed_pct": 3.58, - "median_in": 35.22, - "p90_in": 77.92, - "scale_error_pct": -14.32 + "failed_pct": 3.41, + "median_in": 35.32, + "p90_in": 77.24, + "scale_error_pct": -14.33 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 7.31, - "median_in": 5.05, - "p90_in": 22.81, - "scale_error_pct": -0.58 + "failed_pct": 7.01, + "median_in": 5.09, + "p90_in": 22.3, + "scale_error_pct": -0.4 }, "3-10m": { "pairs": 600000, - "failed_pct": 8.35, - "median_in": 15.31, - "p90_in": 83.08, - "scale_error_pct": -1.62 + "failed_pct": 7.72, + "median_in": 15.39, + "p90_in": 76.01, + "scale_error_pct": -1.51 } }, - "tape_calibration_success_pct": 95.0, + "tape_calibration_success_pct": 95.5, "groups": 8, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 8.45, - 13.88 + 8.46, + 13.75 ], "p90_in": [ - 17.85, - 25.12 + 18.06, + 25.32 ] }, "3-10m": { "median_in": [ - 29.08, - 44.58 + 29.34, + 44.67 ], "p90_in": [ - 60.25, - 95.08 + 60.3, + 93.29 ] } } @@ -304,57 +304,57 @@ "none": { "1-3m": { "pairs": 24000, - "failed_pct": 80.44, - "median_in": Infinity, - "p90_in": Infinity, + "failed_pct": 80.45, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -9.07 }, "3-10m": { "pairs": 24000, "failed_pct": 86.72, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": -13.31 + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": -13.32 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 81.29, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 5.74 + "failed_pct": 81.15, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 5.88 }, "3-10m": { "pairs": 600000, - "failed_pct": 87.4, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 2.26 + "failed_pct": 87.29, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 2.39 } }, - "tape_calibration_success_pct": 95.0, + "tape_calibration_success_pct": 95.5, "groups": 8, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -376,27 +376,27 @@ "pairs": 24000, "failed_pct": 10.82, "median_in": 41.43, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": -15.72 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 13.8, - "median_in": 5.7, - "p90_in": Infinity, - "scale_error_pct": -0.27 + "failed_pct": 13.35, + "median_in": 5.66, + "p90_in": "failed", + "scale_error_pct": -0.15 }, "3-10m": { "pairs": 600000, - "failed_pct": 16.11, - "median_in": 18.08, - "p90_in": Infinity, - "scale_error_pct": -1.53 + "failed_pct": 15.69, + "median_in": 17.92, + "p90_in": "failed", + "scale_error_pct": -1.4 } }, - "tape_calibration_success_pct": 94.0, + "tape_calibration_success_pct": 94.5, "groups": 8, "noise_draws": 1, "interval_95": { @@ -407,7 +407,7 @@ ], "p90_in": [ 24.21, - Infinity + "failed" ] }, "3-10m": { @@ -417,7 +417,7 @@ ], "p90_in": [ 95.85, - Infinity + "failed" ] } } @@ -426,57 +426,57 @@ "none": { "1-3m": { "pairs": 24000, - "failed_pct": 5.09, - "median_in": 11.81, - "p90_in": 25.66, - "scale_error_pct": -14.65 + "failed_pct": 5.49, + "median_in": 11.83, + "p90_in": 25.94, + "scale_error_pct": -14.64 }, "3-10m": { "pairs": 24000, - "failed_pct": 7.02, - "median_in": 39.21, - "p90_in": 96.81, - "scale_error_pct": -16.53 + "failed_pct": 6.97, + "median_in": 39.58, + "p90_in": 96.48, + "scale_error_pct": -16.55 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 10.75, - "median_in": 5.1, - "p90_in": Infinity, - "scale_error_pct": -0.59 + "failed_pct": 10.69, + "median_in": 5.08, + "p90_in": "failed", + "scale_error_pct": -0.42 }, "3-10m": { "pairs": 600000, - "failed_pct": 12.56, - "median_in": 16.04, - "p90_in": Infinity, - "scale_error_pct": -1.63 + "failed_pct": 12.09, + "median_in": 15.96, + "p90_in": "failed", + "scale_error_pct": -1.49 } }, - "tape_calibration_success_pct": 94.0, + "tape_calibration_success_pct": 94.5, "groups": 8, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 8.59, - 16.36 + 8.51, + 16.33 ], "p90_in": [ - 21.86, - 28.93 + 22.0, + 29.12 ] }, "3-10m": { "median_in": [ - 29.76, - 51.45 + 29.91, + 51.65 ], "p90_in": [ - 75.97, - 129.64 + 76.62, + 128.68 ] } } @@ -485,57 +485,57 @@ "none": { "1-3m": { "pairs": 24000, - "failed_pct": 85.3, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": -12.66 + "failed_pct": 85.31, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": -12.68 }, "3-10m": { "pairs": 24000, "failed_pct": 91.55, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": -13.39 + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": -13.41 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 86.22, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 5.56 + "failed_pct": 86.09, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 5.73 }, "3-10m": { "pairs": 600000, - "failed_pct": 92.15, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 3.19 + "failed_pct": 92.05, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 3.38 } }, - "tape_calibration_success_pct": 94.0, + "tape_calibration_success_pct": 94.5, "groups": 8, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -562,20 +562,20 @@ "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 10.53, - "median_in": 5.14, - "p90_in": Infinity, - "scale_error_pct": -0.41 + "failed_pct": 10.06, + "median_in": 5.15, + "p90_in": "failed", + "scale_error_pct": -0.32 }, "3-10m": { "pairs": 600000, - "failed_pct": 11.56, - "median_in": 15.5, - "p90_in": Infinity, - "scale_error_pct": -1.51 + "failed_pct": 11.1, + "median_in": 15.53, + "p90_in": "failed", + "scale_error_pct": -1.42 } }, - "tape_calibration_success_pct": 94.5, + "tape_calibration_success_pct": 95.0, "groups": 8, "noise_draws": 1, "interval_95": { @@ -605,57 +605,57 @@ "none": { "1-3m": { "pairs": 24000, - "failed_pct": 2.55, - "median_in": 9.49, - "p90_in": 18.99, - "scale_error_pct": -11.71 + "failed_pct": 2.8, + "median_in": 9.47, + "p90_in": 19.03, + "scale_error_pct": -11.66 }, "3-10m": { "pairs": 24000, - "failed_pct": 3.63, - "median_in": 30.23, - "p90_in": 64.63, + "failed_pct": 3.45, + "median_in": 30.3, + "p90_in": 64.21, "scale_error_pct": -12.2 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 7.89, - "median_in": 4.67, - "p90_in": 22.59, - "scale_error_pct": -0.69 + "failed_pct": 7.66, + "median_in": 4.71, + "p90_in": 22.05, + "scale_error_pct": -0.53 }, "3-10m": { "pairs": 600000, - "failed_pct": 8.89, - "median_in": 14.0, - "p90_in": 90.46, - "scale_error_pct": -1.56 + "failed_pct": 8.25, + "median_in": 14.04, + "p90_in": 80.71, + "scale_error_pct": -1.45 } }, - "tape_calibration_success_pct": 94.5, + "tape_calibration_success_pct": 95.0, "groups": 8, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 7.58, - 11.75 + 7.56, + 11.8 ], "p90_in": [ - 15.76, - 20.64 + 16.03, + 20.75 ] }, "3-10m": { "median_in": [ - 26.06, - 35.94 + 25.96, + 36.18 ], "p90_in": [ - 51.46, - 89.89 + 51.75, + 89.43 ] } } @@ -664,57 +664,57 @@ "none": { "1-3m": { "pairs": 24000, - "failed_pct": 81.08, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": -9.93 + "failed_pct": 81.09, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": -9.94 }, "3-10m": { "pairs": 24000, - "failed_pct": 87.2, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": -11.58 + "failed_pct": 87.21, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": -11.59 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 81.88, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 3.48 + "failed_pct": 81.72, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 3.62 }, "3-10m": { "pairs": 600000, - "failed_pct": 87.88, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 0.84 + "failed_pct": 87.77, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 1.0 } }, - "tape_calibration_success_pct": 94.5, + "tape_calibration_success_pct": 95.0, "groups": 8, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -736,27 +736,27 @@ "pairs": 24000, "failed_pct": 10.81, "median_in": 41.41, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": -15.73 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 14.31, - "median_in": 5.86, - "p90_in": Infinity, - "scale_error_pct": -0.2 + "failed_pct": 13.86, + "median_in": 5.8, + "p90_in": "failed", + "scale_error_pct": -0.07 }, "3-10m": { "pairs": 600000, - "failed_pct": 16.55, - "median_in": 18.69, - "p90_in": Infinity, - "scale_error_pct": -1.45 + "failed_pct": 16.12, + "median_in": 18.48, + "p90_in": "failed", + "scale_error_pct": -1.32 } }, - "tape_calibration_success_pct": 93.5, + "tape_calibration_success_pct": 94.0, "groups": 8, "noise_draws": 1, "interval_95": { @@ -767,7 +767,7 @@ ], "p90_in": [ 24.38, - Infinity + "failed" ] }, "3-10m": { @@ -777,7 +777,7 @@ ], "p90_in": [ 96.67, - Infinity + "failed" ] } } @@ -786,57 +786,57 @@ "none": { "1-3m": { "pairs": 24000, - "failed_pct": 5.09, + "failed_pct": 5.49, "median_in": 11.85, - "p90_in": 25.77, - "scale_error_pct": -14.68 + "p90_in": 26.03, + "scale_error_pct": -14.62 }, "3-10m": { "pairs": 24000, - "failed_pct": 7.03, - "median_in": 39.2, - "p90_in": 97.06, - "scale_error_pct": -16.67 + "failed_pct": 6.95, + "median_in": 39.54, + "p90_in": 96.68, + "scale_error_pct": -16.74 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 11.24, - "median_in": 5.24, - "p90_in": Infinity, - "scale_error_pct": -0.5 + "failed_pct": 11.18, + "median_in": 5.19, + "p90_in": "failed", + "scale_error_pct": -0.32 }, "3-10m": { "pairs": 600000, - "failed_pct": 13.04, - "median_in": 16.65, - "p90_in": Infinity, - "scale_error_pct": -1.55 + "failed_pct": 12.55, + "median_in": 16.53, + "p90_in": "failed", + "scale_error_pct": -1.41 } }, - "tape_calibration_success_pct": 93.5, + "tape_calibration_success_pct": 94.0, "groups": 8, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 8.42, - 16.4 + 8.36, + 16.38 ], "p90_in": [ - 21.91, - 28.91 + 22.09, + 30.04 ] }, "3-10m": { "median_in": [ - 29.38, - 52.34 + 29.52, + 52.25 ], "p90_in": [ - 77.68, - 130.32 + 77.83, + 129.05 ] } } @@ -845,57 +845,57 @@ "none": { "1-3m": { "pairs": 24000, - "failed_pct": 85.52, - "median_in": Infinity, - "p90_in": Infinity, + "failed_pct": 85.53, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -13.12 }, "3-10m": { "pairs": 24000, - "failed_pct": 91.52, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": -13.41 + "failed_pct": 91.51, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": -13.42 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 86.43, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 4.45 + "failed_pct": 86.3, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 4.59 }, "3-10m": { "pairs": 600000, - "failed_pct": 92.13, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 3.3 + "failed_pct": 92.03, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 3.49 } }, - "tape_calibration_success_pct": 93.5, + "tape_calibration_success_pct": 94.0, "groups": 8, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -922,20 +922,20 @@ "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 10.71, - "median_in": 5.12, - "p90_in": Infinity, - "scale_error_pct": -0.32 + "failed_pct": 10.24, + "median_in": 5.14, + "p90_in": "failed", + "scale_error_pct": -0.24 }, "3-10m": { "pairs": 600000, - "failed_pct": 11.75, - "median_in": 15.29, - "p90_in": Infinity, - "scale_error_pct": -1.29 + "failed_pct": 11.28, + "median_in": 15.32, + "p90_in": "failed", + "scale_error_pct": -1.22 } }, - "tape_calibration_success_pct": 94.5, + "tape_calibration_success_pct": 95.0, "groups": 8, "noise_draws": 1, "interval_95": { @@ -965,57 +965,57 @@ "none": { "1-3m": { "pairs": 24000, - "failed_pct": 2.8, - "median_in": 10.45, - "p90_in": 19.99, - "scale_error_pct": -13.08 + "failed_pct": 2.97, + "median_in": 10.48, + "p90_in": 20.03, + "scale_error_pct": -13.12 }, "3-10m": { "pairs": 24000, - "failed_pct": 3.88, + "failed_pct": 3.73, "median_in": 33.58, - "p90_in": 66.49, - "scale_error_pct": -13.49 + "p90_in": 66.08, + "scale_error_pct": -13.44 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 8.11, - "median_in": 4.69, - "p90_in": 22.66, - "scale_error_pct": -0.57 + "failed_pct": 7.81, + "median_in": 4.71, + "p90_in": 21.79, + "scale_error_pct": -0.44 }, "3-10m": { "pairs": 600000, - "failed_pct": 9.12, + "failed_pct": 8.51, "median_in": 13.89, - "p90_in": 92.71, - "scale_error_pct": -1.34 + "p90_in": 81.34, + "scale_error_pct": -1.24 } }, - "tape_calibration_success_pct": 94.5, + "tape_calibration_success_pct": 95.0, "groups": 8, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 8.71, - 12.82 + 8.7, + 12.78 ], "p90_in": [ - 17.16, - 21.23 + 17.44, + 21.16 ] }, "3-10m": { "median_in": [ - 29.76, - 38.57 + 29.73, + 38.53 ], "p90_in": [ - 56.03, - 92.73 + 55.92, + 91.51 ] } } @@ -1024,57 +1024,57 @@ "none": { "1-3m": { "pairs": 24000, - "failed_pct": 81.64, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": -10.87 + "failed_pct": 81.65, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": -10.89 }, "3-10m": { "pairs": 24000, "failed_pct": 87.88, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": -12.19 + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": -12.2 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 82.47, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 3.59 + "failed_pct": 82.32, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 3.72 }, "3-10m": { "pairs": 600000, - "failed_pct": 88.55, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 0.94 + "failed_pct": 88.45, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 1.09 } }, - "tape_calibration_success_pct": 94.5, + "tape_calibration_success_pct": 95.0, "groups": 8, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -1096,27 +1096,27 @@ "pairs": 24000, "failed_pct": 10.84, "median_in": 41.84, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": -15.87 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 13.81, - "median_in": 5.71, - "p90_in": Infinity, - "scale_error_pct": -0.27 + "failed_pct": 13.36, + "median_in": 5.68, + "p90_in": "failed", + "scale_error_pct": -0.15 }, "3-10m": { "pairs": 600000, - "failed_pct": 16.12, - "median_in": 18.04, - "p90_in": Infinity, - "scale_error_pct": -1.51 + "failed_pct": 15.7, + "median_in": 17.9, + "p90_in": "failed", + "scale_error_pct": -1.39 } }, - "tape_calibration_success_pct": 94.0, + "tape_calibration_success_pct": 94.5, "groups": 8, "noise_draws": 1, "interval_95": { @@ -1127,7 +1127,7 @@ ], "p90_in": [ 24.36, - Infinity + "failed" ] }, "3-10m": { @@ -1137,7 +1137,7 @@ ], "p90_in": [ 96.29, - Infinity + "failed" ] } } @@ -1146,57 +1146,57 @@ "none": { "1-3m": { "pairs": 24000, - "failed_pct": 5.08, - "median_in": 11.92, - "p90_in": 25.81, - "scale_error_pct": -14.81 + "failed_pct": 5.51, + "median_in": 11.96, + "p90_in": 26.07, + "scale_error_pct": -14.78 }, "3-10m": { "pairs": 24000, - "failed_pct": 7.03, - "median_in": 39.57, - "p90_in": 96.9, - "scale_error_pct": -16.7 + "failed_pct": 6.96, + "median_in": 39.93, + "p90_in": 96.8, + "scale_error_pct": -16.71 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 10.74, - "median_in": 5.11, - "p90_in": Infinity, - "scale_error_pct": -0.59 + "failed_pct": 10.71, + "median_in": 5.09, + "p90_in": "failed", + "scale_error_pct": -0.42 }, "3-10m": { "pairs": 600000, - "failed_pct": 12.57, - "median_in": 16.02, - "p90_in": Infinity, - "scale_error_pct": -1.63 + "failed_pct": 12.08, + "median_in": 15.95, + "p90_in": "failed", + "scale_error_pct": -1.49 } }, - "tape_calibration_success_pct": 94.0, + "tape_calibration_success_pct": 94.5, "groups": 8, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 8.73, - 16.43 + 8.67, + 16.41 ], "p90_in": [ - 21.93, - 29.05 + 22.07, + 29.27 ] }, "3-10m": { "median_in": [ - 30.21, - 51.96 + 30.33, + 52.14 ], "p90_in": [ - 76.43, - 130.79 + 76.95, + 129.42 ] } } @@ -1206,56 +1206,56 @@ "1-3m": { "pairs": 24000, "failed_pct": 85.33, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": -12.95 + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": -12.97 }, "3-10m": { "pairs": 24000, - "failed_pct": 91.55, - "median_in": Infinity, - "p90_in": Infinity, + "failed_pct": 91.54, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -13.45 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 86.24, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 5.76 + "failed_pct": 86.11, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 5.92 }, "3-10m": { "pairs": 600000, - "failed_pct": 92.14, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 3.15 + "failed_pct": 92.05, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 3.36 } }, - "tape_calibration_success_pct": 94.0, + "tape_calibration_success_pct": 94.5, "groups": 8, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -1282,20 +1282,20 @@ "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 10.57, - "median_in": 5.1, - "p90_in": Infinity, - "scale_error_pct": -0.41 + "failed_pct": 10.11, + "median_in": 5.13, + "p90_in": "failed", + "scale_error_pct": -0.34 }, "3-10m": { "pairs": 600000, - "failed_pct": 11.62, - "median_in": 15.4, - "p90_in": Infinity, - "scale_error_pct": -1.48 + "failed_pct": 11.15, + "median_in": 15.47, + "p90_in": "failed", + "scale_error_pct": -1.42 } }, - "tape_calibration_success_pct": 94.5, + "tape_calibration_success_pct": 95.0, "groups": 8, "noise_draws": 1, "interval_95": { @@ -1325,57 +1325,57 @@ "none": { "1-3m": { "pairs": 24000, - "failed_pct": 2.59, - "median_in": 9.8, - "p90_in": 19.39, - "scale_error_pct": -12.09 + "failed_pct": 2.82, + "median_in": 9.81, + "p90_in": 19.41, + "scale_error_pct": -12.01 }, "3-10m": { "pairs": 24000, - "failed_pct": 3.64, - "median_in": 31.38, - "p90_in": 66.09, - "scale_error_pct": -12.61 + "failed_pct": 3.51, + "median_in": 31.46, + "p90_in": 65.72, + "scale_error_pct": -12.59 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 7.93, - "median_in": 4.64, - "p90_in": 22.48, - "scale_error_pct": -0.67 + "failed_pct": 7.68, + "median_in": 4.68, + "p90_in": 21.91, + "scale_error_pct": -0.55 }, "3-10m": { "pairs": 600000, - "failed_pct": 8.91, - "median_in": 13.9, - "p90_in": 90.0, - "scale_error_pct": -1.53 + "failed_pct": 8.31, + "median_in": 13.98, + "p90_in": 80.74, + "scale_error_pct": -1.43 } }, - "tape_calibration_success_pct": 94.5, + "tape_calibration_success_pct": 95.0, "groups": 8, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ 7.94, - 12.11 + 12.09 ], "p90_in": [ - 16.1, - 21.12 + 16.47, + 21.21 ] }, "3-10m": { "median_in": [ - 27.1, - 36.99 + 26.82, + 37.23 ], "p90_in": [ - 52.47, - 91.15 + 52.94, + 90.73 ] } } @@ -1384,57 +1384,57 @@ "none": { "1-3m": { "pairs": 24000, - "failed_pct": 81.34, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": -10.64 + "failed_pct": 81.35, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": -10.66 }, "3-10m": { "pairs": 24000, - "failed_pct": 87.5, - "median_in": Infinity, - "p90_in": Infinity, + "failed_pct": 87.51, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -12.26 } }, "one_known_distance": { "1-3m": { "pairs": 600000, - "failed_pct": 82.13, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 3.39 + "failed_pct": 81.98, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 3.53 }, "3-10m": { "pairs": 600000, - "failed_pct": 88.16, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 0.81 + "failed_pct": 88.05, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 0.96 } }, - "tape_calibration_success_pct": 94.5, + "tape_calibration_success_pct": 95.0, "groups": 8, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -1464,16 +1464,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 0.68, - "median_in": 3.04, - "p90_in": 12.62, - "scale_error_pct": -0.17 + "median_in": 3.0, + "p90_in": 12.53, + "scale_error_pct": -0.05 }, "3-10m": { "pairs": 600000, "failed_pct": 0.72, - "median_in": 9.82, - "p90_in": 43.65, - "scale_error_pct": -1.23 + "median_in": 9.64, + "p90_in": 43.29, + "scale_error_pct": -1.08 } }, "tape_calibration_success_pct": 99.5, @@ -1507,32 +1507,32 @@ "1-3m": { "pairs": 24000, "failed_pct": 0.0, - "median_in": 5.9, - "p90_in": 11.25, - "scale_error_pct": -6.47 + "median_in": 5.87, + "p90_in": 11.34, + "scale_error_pct": -6.46 }, "3-10m": { "pairs": 24000, - "failed_pct": 0.01, - "median_in": 19.57, - "p90_in": 46.74, - "scale_error_pct": -7.38 + "failed_pct": 0.0, + "median_in": 19.59, + "p90_in": 46.71, + "scale_error_pct": -7.44 } }, "one_known_distance": { "1-3m": { "pairs": 600000, "failed_pct": 0.5, - "median_in": 2.78, - "p90_in": 11.26, - "scale_error_pct": -0.44 + "median_in": 2.73, + "p90_in": 11.1, + "scale_error_pct": -0.32 }, "3-10m": { "pairs": 600000, - "failed_pct": 0.51, - "median_in": 9.21, - "p90_in": 42.66, - "scale_error_pct": -1.24 + "failed_pct": 0.5, + "median_in": 9.09, + "p90_in": 42.47, + "scale_error_pct": -1.16 } }, "tape_calibration_success_pct": 99.5, @@ -1542,21 +1542,21 @@ "1-3m": { "median_in": [ 5.0, - 6.85 + 6.88 ], "p90_in": [ - 9.56, - 13.43 + 9.57, + 13.41 ] }, "3-10m": { "median_in": [ - 15.3, - 22.46 + 15.51, + 22.65 ], "p90_in": [ - 35.56, - 58.01 + 35.54, + 57.99 ] } } @@ -1567,7 +1567,7 @@ "pairs": 24000, "failed_pct": 0.0, "median_in": 5.51, - "p90_in": 26.11, + "p90_in": 26.1, "scale_error_pct": -2.26 }, "3-10m": { @@ -1575,23 +1575,23 @@ "failed_pct": 0.03, "median_in": 16.03, "p90_in": 48.48, - "scale_error_pct": -3.98 + "scale_error_pct": -3.99 } }, "one_known_distance": { "1-3m": { "pairs": 600000, "failed_pct": 0.5, - "median_in": 5.01, - "p90_in": 30.48, - "scale_error_pct": 3.92 + "median_in": 4.98, + "p90_in": 30.56, + "scale_error_pct": 4.01 }, "3-10m": { "pairs": 600000, "failed_pct": 0.53, - "median_in": 13.78, - "p90_in": 55.8, - "scale_error_pct": 2.18 + "median_in": 13.76, + "p90_in": 55.88, + "scale_error_pct": 2.29 } }, "tape_calibration_success_pct": 99.5, @@ -1643,16 +1643,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 0.65, - "median_in": 3.34, - "p90_in": 12.34, - "scale_error_pct": -0.32 + "median_in": 3.31, + "p90_in": 12.31, + "scale_error_pct": -0.19 }, "3-10m": { "pairs": 600000, "failed_pct": 0.68, - "median_in": 10.79, - "p90_in": 42.09, - "scale_error_pct": -1.62 + "median_in": 10.64, + "p90_in": 41.92, + "scale_error_pct": -1.5 } }, "tape_calibration_success_pct": 99.5, @@ -1687,31 +1687,31 @@ "pairs": 24000, "failed_pct": 0.0, "median_in": 5.61, - "p90_in": 10.92, - "scale_error_pct": -6.16 + "p90_in": 11.04, + "scale_error_pct": -6.17 }, "3-10m": { "pairs": 24000, - "failed_pct": 0.01, - "median_in": 18.28, - "p90_in": 46.46, - "scale_error_pct": -7.0 + "failed_pct": 0.0, + "median_in": 18.5, + "p90_in": 46.86, + "scale_error_pct": -7.09 } }, "one_known_distance": { "1-3m": { "pairs": 600000, "failed_pct": 0.5, - "median_in": 3.08, - "p90_in": 11.1, - "scale_error_pct": -0.62 + "median_in": 3.06, + "p90_in": 11.04, + "scale_error_pct": -0.5 }, "3-10m": { "pairs": 600000, - "failed_pct": 0.51, - "median_in": 10.29, - "p90_in": 40.99, - "scale_error_pct": -1.66 + "failed_pct": 0.5, + "median_in": 10.16, + "p90_in": 40.9, + "scale_error_pct": -1.62 } }, "tape_calibration_success_pct": 99.5, @@ -1720,22 +1720,22 @@ "interval_95": { "1-3m": { "median_in": [ - 4.91, + 4.97, 6.36 ], "p90_in": [ - 9.52, - 12.91 + 9.53, + 12.97 ] }, "3-10m": { "median_in": [ - 15.01, - 21.29 + 15.19, + 21.45 ], "p90_in": [ - 35.96, - 56.11 + 36.12, + 56.08 ] } } @@ -1745,14 +1745,14 @@ "1-3m": { "pairs": 24000, "failed_pct": 0.0, - "median_in": 5.09, - "p90_in": 24.59, + "median_in": 5.1, + "p90_in": 24.58, "scale_error_pct": -2.27 }, "3-10m": { "pairs": 24000, "failed_pct": 0.02, - "median_in": 14.79, + "median_in": 14.78, "p90_in": 45.42, "scale_error_pct": -3.87 } @@ -1762,15 +1762,15 @@ "pairs": 600000, "failed_pct": 0.5, "median_in": 5.18, - "p90_in": 28.49, - "scale_error_pct": 4.06 + "p90_in": 28.56, + "scale_error_pct": 4.17 }, "3-10m": { "pairs": 600000, "failed_pct": 0.52, - "median_in": 13.74, - "p90_in": 52.73, - "scale_error_pct": 2.02 + "median_in": 13.75, + "p90_in": 52.82, + "scale_error_pct": 2.14 } }, "tape_calibration_success_pct": 99.5, @@ -1780,7 +1780,7 @@ "1-3m": { "median_in": [ 3.93, - 7.23 + 7.24 ], "p90_in": [ 14.67, @@ -1793,7 +1793,7 @@ 18.19 ], "p90_in": [ - 34.15, + 34.16, 55.39 ] } @@ -1822,16 +1822,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 2.13, - "median_in": 3.49, - "p90_in": 14.43, - "scale_error_pct": 0.01 + "median_in": 3.47, + "p90_in": 14.39, + "scale_error_pct": 0.13 }, "3-10m": { "pairs": 600000, "failed_pct": 2.15, - "median_in": 11.03, - "p90_in": 49.0, - "scale_error_pct": -1.36 + "median_in": 10.94, + "p90_in": 48.76, + "scale_error_pct": -1.22 } }, "tape_calibration_success_pct": 98.0, @@ -1865,32 +1865,32 @@ "1-3m": { "pairs": 24000, "failed_pct": 0.0, - "median_in": 5.86, - "p90_in": 11.24, - "scale_error_pct": -6.32 + "median_in": 5.84, + "p90_in": 11.27, + "scale_error_pct": -6.31 }, "3-10m": { "pairs": 24000, - "failed_pct": 0.01, - "median_in": 19.06, + "failed_pct": 0.0, + "median_in": 19.23, "p90_in": 46.46, - "scale_error_pct": -7.3 + "scale_error_pct": -7.36 } }, "one_known_distance": { "1-3m": { "pairs": 600000, "failed_pct": 2.0, - "median_in": 3.21, - "p90_in": 13.02, - "scale_error_pct": -0.31 + "median_in": 3.2, + "p90_in": 12.97, + "scale_error_pct": -0.21 }, "3-10m": { "pairs": 600000, - "failed_pct": 2.01, - "median_in": 10.57, - "p90_in": 48.53, - "scale_error_pct": -1.39 + "failed_pct": 2.0, + "median_in": 10.48, + "p90_in": 48.11, + "scale_error_pct": -1.34 } }, "tape_calibration_success_pct": 98.0, @@ -1899,22 +1899,22 @@ "interval_95": { "1-3m": { "median_in": [ - 5.26, + 5.23, 6.49 ], "p90_in": [ - 10.31, - 12.0 + 10.37, + 12.13 ] }, "3-10m": { "median_in": [ - 15.71, - 21.66 + 15.88, + 21.67 ], "p90_in": [ - 38.09, - 58.95 + 38.26, + 58.64 ] } } @@ -1940,16 +1940,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 2.0, - "median_in": 5.67, - "p90_in": 35.99, - "scale_error_pct": 4.52 + "median_in": 5.69, + "p90_in": 36.07, + "scale_error_pct": 4.66 }, "3-10m": { "pairs": 600000, "failed_pct": 2.02, - "median_in": 14.67, - "p90_in": 65.25, - "scale_error_pct": 2.26 + "median_in": 14.76, + "p90_in": 65.21, + "scale_error_pct": 2.41 } }, "tape_calibration_success_pct": 98.0, @@ -1959,10 +1959,10 @@ "1-3m": { "median_in": [ 3.94, - 7.98 + 7.99 ], "p90_in": [ - 14.88, + 14.89, 43.75 ] }, @@ -1972,7 +1972,7 @@ 19.3 ], "p90_in": [ - 35.06, + 35.07, 73.66 ] } @@ -2003,16 +2003,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 0.68, - "median_in": 3.01, - "p90_in": 12.53, - "scale_error_pct": -0.22 + "median_in": 2.96, + "p90_in": 12.23, + "scale_error_pct": -0.1 }, "3-10m": { "pairs": 600000, "failed_pct": 0.72, - "median_in": 10.18, - "p90_in": 43.53, - "scale_error_pct": -1.25 + "median_in": 9.97, + "p90_in": 42.67, + "scale_error_pct": -1.1 } }, "tape_calibration_success_pct": 99.5, @@ -2046,32 +2046,32 @@ "1-3m": { "pairs": 24000, "failed_pct": 0.0, - "median_in": 2.1, - "p90_in": 7.33, - "scale_error_pct": -1.58 + "median_in": 2.12, + "p90_in": 7.29, + "scale_error_pct": -1.6 }, "3-10m": { "pairs": 24000, - "failed_pct": 0.01, - "median_in": 7.47, - "p90_in": 35.51, - "scale_error_pct": -2.36 + "failed_pct": 0.0, + "median_in": 7.84, + "p90_in": 35.76, + "scale_error_pct": -2.52 } }, "one_known_distance": { "1-3m": { "pairs": 600000, "failed_pct": 0.5, - "median_in": 2.75, - "p90_in": 11.3, - "scale_error_pct": -0.45 + "median_in": 2.69, + "p90_in": 10.93, + "scale_error_pct": -0.35 }, "3-10m": { "pairs": 600000, - "failed_pct": 0.51, - "median_in": 9.59, - "p90_in": 42.74, - "scale_error_pct": -1.26 + "failed_pct": 0.5, + "median_in": 9.44, + "p90_in": 42.06, + "scale_error_pct": -1.19 } }, "tape_calibration_success_pct": 99.5, @@ -2081,21 +2081,21 @@ "1-3m": { "median_in": [ 1.33, - 3.38 + 3.48 ], "p90_in": [ - 5.29, - 10.16 + 5.34, + 9.85 ] }, "3-10m": { "median_in": [ - 4.09, - 11.56 + 4.17, + 11.67 ], "p90_in": [ - 21.61, - 56.76 + 21.92, + 56.09 ] } } @@ -2113,7 +2113,7 @@ "pairs": 24000, "failed_pct": 0.03, "median_in": 10.49, - "p90_in": 47.0, + "p90_in": 46.99, "scale_error_pct": 0.93 } }, @@ -2121,16 +2121,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 0.5, - "median_in": 4.97, - "p90_in": 30.06, - "scale_error_pct": 4.03 + "median_in": 4.93, + "p90_in": 30.09, + "scale_error_pct": 4.15 }, "3-10m": { "pairs": 600000, "failed_pct": 0.53, - "median_in": 13.92, - "p90_in": 55.96, - "scale_error_pct": 2.5 + "median_in": 13.91, + "p90_in": 55.67, + "scale_error_pct": 2.62 } }, "tape_calibration_success_pct": 99.5, @@ -2150,11 +2150,11 @@ "3-10m": { "median_in": [ 7.29, - 15.22 + 15.23 ], "p90_in": [ 32.61, - 61.94 + 61.93 ] } } @@ -2182,16 +2182,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 1.15, - "median_in": 3.15, - "p90_in": 12.41, - "scale_error_pct": -0.15 + "median_in": 3.1, + "p90_in": 12.28, + "scale_error_pct": -0.03 }, "3-10m": { "pairs": 600000, "failed_pct": 1.18, - "median_in": 10.69, - "p90_in": 43.25, - "scale_error_pct": -1.31 + "median_in": 10.47, + "p90_in": 42.89, + "scale_error_pct": -1.18 } }, "tape_calibration_success_pct": 99.0, @@ -2225,32 +2225,32 @@ "1-3m": { "pairs": 24000, "failed_pct": 0.0, - "median_in": 1.59, - "p90_in": 6.46, - "scale_error_pct": -0.37 + "median_in": 1.61, + "p90_in": 6.48, + "scale_error_pct": -0.43 }, "3-10m": { "pairs": 24000, - "failed_pct": 0.01, - "median_in": 5.31, - "p90_in": 32.47, - "scale_error_pct": -0.93 + "failed_pct": 0.0, + "median_in": 5.49, + "p90_in": 32.64, + "scale_error_pct": -1.03 } }, "one_known_distance": { "1-3m": { "pairs": 600000, "failed_pct": 1.0, - "median_in": 2.89, - "p90_in": 11.24, - "scale_error_pct": -0.37 + "median_in": 2.86, + "p90_in": 11.07, + "scale_error_pct": -0.29 }, "3-10m": { "pairs": 600000, - "failed_pct": 1.01, - "median_in": 10.13, - "p90_in": 42.54, - "scale_error_pct": -1.32 + "failed_pct": 1.0, + "median_in": 9.97, + "p90_in": 42.34, + "scale_error_pct": -1.27 } }, "tape_calibration_success_pct": 99.0, @@ -2259,22 +2259,22 @@ "interval_95": { "1-3m": { "median_in": [ - 1.12, - 2.21 + 1.14, + 2.26 ], "p90_in": [ 5.05, - 8.27 + 8.26 ] }, "3-10m": { "median_in": [ - 3.51, - 8.01 + 3.63, + 8.12 ], "p90_in": [ - 20.37, - 52.79 + 20.65, + 51.64 ] } } @@ -2285,7 +2285,7 @@ "pairs": 24000, "failed_pct": 0.0, "median_in": 3.5, - "p90_in": 27.12, + "p90_in": 27.1, "scale_error_pct": 3.32 }, "3-10m": { @@ -2300,16 +2300,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 1.0, - "median_in": 4.98, - "p90_in": 29.62, - "scale_error_pct": 3.91 + "median_in": 4.96, + "p90_in": 29.64, + "scale_error_pct": 4.01 }, "3-10m": { "pairs": 600000, "failed_pct": 1.02, - "median_in": 13.74, - "p90_in": 53.62, - "scale_error_pct": 2.2 + "median_in": 13.73, + "p90_in": 53.57, + "scale_error_pct": 2.31 } }, "tape_calibration_success_pct": 99.0, @@ -2319,7 +2319,7 @@ "1-3m": { "median_in": [ 2.53, - 5.33 + 5.34 ], "p90_in": [ 15.76, @@ -2328,12 +2328,12 @@ }, "3-10m": { "median_in": [ - 8.37, - 14.4 + 8.38, + 14.41 ], "p90_in": [ 34.41, - 60.74 + 60.7 ] } } @@ -2361,16 +2361,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 0.63, - "median_in": 2.92, - "p90_in": 12.15, - "scale_error_pct": -0.0 + "median_in": 2.89, + "p90_in": 12.01, + "scale_error_pct": 0.09 }, "3-10m": { "pairs": 600000, "failed_pct": 0.65, - "median_in": 9.8, - "p90_in": 43.35, - "scale_error_pct": -1.2 + "median_in": 9.65, + "p90_in": 43.11, + "scale_error_pct": -1.08 } }, "tape_calibration_success_pct": 99.5, @@ -2404,32 +2404,32 @@ "1-3m": { "pairs": 24000, "failed_pct": 0.0, - "median_in": 1.49, - "p90_in": 6.15, - "scale_error_pct": -0.11 + "median_in": 1.52, + "p90_in": 6.19, + "scale_error_pct": -0.13 }, "3-10m": { "pairs": 24000, - "failed_pct": 0.01, - "median_in": 4.82, - "p90_in": 29.87, - "scale_error_pct": -0.47 + "failed_pct": 0.0, + "median_in": 5.01, + "p90_in": 30.09, + "scale_error_pct": -0.55 } }, "one_known_distance": { "1-3m": { "pairs": 600000, "failed_pct": 0.5, - "median_in": 2.69, - "p90_in": 10.95, - "scale_error_pct": -0.22 + "median_in": 2.67, + "p90_in": 10.83, + "scale_error_pct": -0.15 }, "3-10m": { "pairs": 600000, - "failed_pct": 0.51, - "median_in": 9.24, - "p90_in": 42.66, - "scale_error_pct": -1.19 + "failed_pct": 0.5, + "median_in": 9.13, + "p90_in": 42.43, + "scale_error_pct": -1.14 } }, "tape_calibration_success_pct": 99.5, @@ -2438,22 +2438,22 @@ "interval_95": { "1-3m": { "median_in": [ - 1.13, - 1.9 + 1.15, + 1.93 ], "p90_in": [ - 4.82, - 7.35 + 4.88, + 7.31 ] }, "3-10m": { "median_in": [ - 3.4, - 6.29 + 3.5, + 6.5 ], "p90_in": [ - 18.27, - 53.92 + 18.0, + 53.42 ] } } @@ -2464,14 +2464,14 @@ "pairs": 24000, "failed_pct": 0.0, "median_in": 3.7, - "p90_in": 29.3, + "p90_in": 29.29, "scale_error_pct": 3.66 }, "3-10m": { "pairs": 24000, "failed_pct": 0.02, "median_in": 11.0, - "p90_in": 51.22, + "p90_in": 51.2, "scale_error_pct": 2.43 } }, @@ -2479,16 +2479,16 @@ "1-3m": { "pairs": 600000, "failed_pct": 0.5, - "median_in": 5.12, - "p90_in": 32.95, - "scale_error_pct": 4.12 + "median_in": 5.1, + "p90_in": 32.92, + "scale_error_pct": 4.19 }, "3-10m": { "pairs": 600000, "failed_pct": 0.52, - "median_in": 13.7, - "p90_in": 60.83, - "scale_error_pct": 2.27 + "median_in": 13.77, + "p90_in": 60.52, + "scale_error_pct": 2.37 } }, "tape_calibration_success_pct": 99.5, @@ -2508,11 +2508,11 @@ "3-10m": { "median_in": [ 8.99, - 14.58 + 14.59 ], "p90_in": [ 31.87, - 82.15 + 82.12 ] } } @@ -2542,16 +2542,16 @@ "1-3m": { "pairs": 3000000, "failed_pct": 0.68, - "median_in": 3.1, - "p90_in": 12.65, - "scale_error_pct": -0.22 + "median_in": 3.05, + "p90_in": 12.61, + "scale_error_pct": -0.1 }, "3-10m": { "pairs": 3000000, "failed_pct": 0.72, - "median_in": 10.06, - "p90_in": 43.48, - "scale_error_pct": -1.29 + "median_in": 9.88, + "p90_in": 43.24, + "scale_error_pct": -1.14 } }, "tape_calibration_success_pct": 99.5, @@ -2585,32 +2585,32 @@ "1-3m": { "pairs": 120000, "failed_pct": 0.0, - "median_in": 5.91, - "p90_in": 11.35, + "median_in": 5.89, + "p90_in": 11.43, "scale_error_pct": -6.47 }, "3-10m": { "pairs": 120000, - "failed_pct": 0.01, - "median_in": 19.64, - "p90_in": 46.82, - "scale_error_pct": -7.4 + "failed_pct": 0.0, + "median_in": 19.68, + "p90_in": 46.8, + "scale_error_pct": -7.46 } }, "one_known_distance": { "1-3m": { "pairs": 3000000, "failed_pct": 0.5, - "median_in": 2.83, - "p90_in": 11.29, - "scale_error_pct": -0.47 + "median_in": 2.77, + "p90_in": 11.17, + "scale_error_pct": -0.36 }, "3-10m": { "pairs": 3000000, - "failed_pct": 0.51, - "median_in": 9.45, - "p90_in": 42.56, - "scale_error_pct": -1.3 + "failed_pct": 0.5, + "median_in": 9.32, + "p90_in": 42.49, + "scale_error_pct": -1.22 } }, "tape_calibration_success_pct": 99.5, @@ -2619,22 +2619,22 @@ "interval_95": { "1-3m": { "median_in": [ - 5.04, - 6.9 + 5.05, + 6.93 ], "p90_in": [ - 9.78, - 13.49 + 9.8, + 13.51 ] }, "3-10m": { "median_in": [ - 15.04, - 22.48 + 15.21, + 22.69 ], "p90_in": [ - 35.95, - 58.11 + 36.09, + 58.17 ] } } @@ -2644,8 +2644,8 @@ "1-3m": { "pairs": 120000, "failed_pct": 0.0, - "median_in": 5.52, - "p90_in": 26.24, + "median_in": 5.53, + "p90_in": 26.23, "scale_error_pct": -2.28 }, "3-10m": { @@ -2660,16 +2660,16 @@ "1-3m": { "pairs": 3000000, "failed_pct": 0.5, - "median_in": 5.11, - "p90_in": 30.48, - "scale_error_pct": 4.01 + "median_in": 5.08, + "p90_in": 30.59, + "scale_error_pct": 4.11 }, "3-10m": { "pairs": 3000000, "failed_pct": 0.53, - "median_in": 13.81, - "p90_in": 55.9, - "scale_error_pct": 2.16 + "median_in": 13.78, + "p90_in": 56.12, + "scale_error_pct": 2.28 } }, "tape_calibration_success_pct": 99.5, @@ -2683,7 +2683,7 @@ ], "p90_in": [ 14.99, - 35.22 + 35.2 ] }, "3-10m": { @@ -2692,7 +2692,7 @@ 20.01 ], "p90_in": [ - 35.44, + 35.46, 58.53 ] } @@ -2721,16 +2721,16 @@ "1-3m": { "pairs": 3000000, "failed_pct": 0.25, - "median_in": 3.46, - "p90_in": 12.14, - "scale_error_pct": -0.39 + "median_in": 3.43, + "p90_in": 12.17, + "scale_error_pct": -0.27 }, "3-10m": { "pairs": 3000000, "failed_pct": 0.28, - "median_in": 10.96, - "p90_in": 43.03, - "scale_error_pct": -1.68 + "median_in": 10.8, + "p90_in": 43.04, + "scale_error_pct": -1.54 } }, "tape_calibration_success_pct": 99.9, @@ -2765,31 +2765,31 @@ "pairs": 120000, "failed_pct": 0.0, "median_in": 5.7, - "p90_in": 10.91, + "p90_in": 10.97, "scale_error_pct": -6.35 }, "3-10m": { "pairs": 120000, - "failed_pct": 0.01, - "median_in": 18.54, - "p90_in": 47.98, - "scale_error_pct": -7.26 + "failed_pct": 0.0, + "median_in": 18.72, + "p90_in": 48.28, + "scale_error_pct": -7.37 } }, "one_known_distance": { "1-3m": { "pairs": 3000000, "failed_pct": 0.1, - "median_in": 3.2, - "p90_in": 10.96, - "scale_error_pct": -0.69 + "median_in": 3.18, + "p90_in": 10.95, + "scale_error_pct": -0.58 }, "3-10m": { "pairs": 3000000, - "failed_pct": 0.11, - "median_in": 10.42, - "p90_in": 41.88, - "scale_error_pct": -1.74 + "failed_pct": 0.1, + "median_in": 10.31, + "p90_in": 41.83, + "scale_error_pct": -1.7 } }, "tape_calibration_success_pct": 99.9, @@ -2798,22 +2798,22 @@ "interval_95": { "1-3m": { "median_in": [ - 5.0, - 6.39 + 5.06, + 6.42 ], "p90_in": [ - 9.68, - 12.69 + 9.72, + 12.57 ] }, "3-10m": { "median_in": [ - 15.39, - 21.87 + 15.44, + 22.23 ], "p90_in": [ - 36.44, - 58.48 + 36.56, + 58.26 ] } } @@ -2824,7 +2824,7 @@ "pairs": 120000, "failed_pct": 0.0, "median_in": 5.11, - "p90_in": 25.58, + "p90_in": 25.57, "scale_error_pct": -1.86 }, "3-10m": { @@ -2839,16 +2839,16 @@ "1-3m": { "pairs": 3000000, "failed_pct": 0.1, - "median_in": 5.38, - "p90_in": 29.85, - "scale_error_pct": 4.35 + "median_in": 5.37, + "p90_in": 29.94, + "scale_error_pct": 4.46 }, "3-10m": { "pairs": 3000000, "failed_pct": 0.12, - "median_in": 13.82, - "p90_in": 52.89, - "scale_error_pct": 1.98 + "median_in": 13.83, + "p90_in": 53.15, + "scale_error_pct": 2.08 } }, "tape_calibration_success_pct": 99.9, @@ -2868,10 +2868,10 @@ "3-10m": { "median_in": [ 12.08, - 18.11 + 18.1 ], "p90_in": [ - 34.31, + 34.33, 55.23 ] } @@ -2900,16 +2900,16 @@ "1-3m": { "pairs": 3000000, "failed_pct": 1.33, - "median_in": 3.58, - "p90_in": 14.05, - "scale_error_pct": -0.17 + "median_in": 3.57, + "p90_in": 14.04, + "scale_error_pct": -0.04 }, "3-10m": { "pairs": 3000000, "failed_pct": 1.35, - "median_in": 11.35, - "p90_in": 47.77, - "scale_error_pct": -1.55 + "median_in": 11.27, + "p90_in": 47.67, + "scale_error_pct": -1.41 } }, "tape_calibration_success_pct": 98.8, @@ -2943,32 +2943,32 @@ "1-3m": { "pairs": 120000, "failed_pct": 0.0, - "median_in": 5.96, - "p90_in": 11.2, + "median_in": 5.95, + "p90_in": 11.24, "scale_error_pct": -6.57 }, "3-10m": { "pairs": 120000, - "failed_pct": 0.01, - "median_in": 19.24, - "p90_in": 46.34, - "scale_error_pct": -7.51 + "failed_pct": 0.0, + "median_in": 19.41, + "p90_in": 46.46, + "scale_error_pct": -7.6 } }, "one_known_distance": { "1-3m": { "pairs": 3000000, "failed_pct": 1.2, - "median_in": 3.33, - "p90_in": 12.83, - "scale_error_pct": -0.49 + "median_in": 3.34, + "p90_in": 12.81, + "scale_error_pct": -0.37 }, "3-10m": { "pairs": 3000000, - "failed_pct": 1.21, - "median_in": 10.84, - "p90_in": 47.03, - "scale_error_pct": -1.61 + "failed_pct": 1.2, + "median_in": 10.79, + "p90_in": 46.83, + "scale_error_pct": -1.57 } }, "tape_calibration_success_pct": 98.8, @@ -2977,22 +2977,22 @@ "interval_95": { "1-3m": { "median_in": [ - 5.36, - 6.57 + 5.33, + 6.56 ], "p90_in": [ 10.22, - 12.0 + 12.15 ] }, "3-10m": { "median_in": [ - 15.74, - 21.83 + 15.85, + 21.98 ], "p90_in": [ - 37.94, - 59.7 + 37.92, + 59.51 ] } } @@ -3003,7 +3003,7 @@ "pairs": 120000, "failed_pct": 0.0, "median_in": 5.35, - "p90_in": 27.75, + "p90_in": 27.74, "scale_error_pct": -2.41 }, "3-10m": { @@ -3018,16 +3018,16 @@ "1-3m": { "pairs": 3000000, "failed_pct": 1.2, - "median_in": 5.76, - "p90_in": 34.57, - "scale_error_pct": 4.63 + "median_in": 5.78, + "p90_in": 34.67, + "scale_error_pct": 4.79 }, "3-10m": { "pairs": 3000000, "failed_pct": 1.22, - "median_in": 14.75, - "p90_in": 63.99, - "scale_error_pct": 2.21 + "median_in": 14.85, + "p90_in": 63.96, + "scale_error_pct": 2.37 } }, "tape_calibration_success_pct": 98.8, @@ -3040,7 +3040,7 @@ 7.86 ], "p90_in": [ - 15.26, + 15.24, 44.06 ] }, @@ -3050,7 +3050,7 @@ 19.02 ], "p90_in": [ - 34.85, + 34.84, 77.17 ] } @@ -3081,16 +3081,16 @@ "1-3m": { "pairs": 3000000, "failed_pct": 7.95, - "median_in": 3.42, - "p90_in": 22.66, - "scale_error_pct": -0.28 + "median_in": 3.37, + "p90_in": 22.6, + "scale_error_pct": -0.14 }, "3-10m": { "pairs": 3000000, "failed_pct": 7.99, - "median_in": 11.51, - "p90_in": 86.42, - "scale_error_pct": -1.32 + "median_in": 11.29, + "p90_in": 85.78, + "scale_error_pct": -1.17 } }, "tape_calibration_success_pct": 92.2, @@ -3104,7 +3104,7 @@ ], "p90_in": [ 24.6, - Infinity + "failed" ] }, "3-10m": { @@ -3114,7 +3114,7 @@ ], "p90_in": [ 83.39, - Infinity + "failed" ] } } @@ -3125,31 +3125,31 @@ "pairs": 120000, "failed_pct": 7.5, "median_in": 11.8, - "p90_in": 38.3, + "p90_in": 38.13, "scale_error_pct": -9.67 }, "3-10m": { "pairs": 120000, - "failed_pct": 7.51, - "median_in": 38.59, - "p90_in": 122.94, - "scale_error_pct": -12.28 + "failed_pct": 7.5, + "median_in": 38.86, + "p90_in": 123.29, + "scale_error_pct": -12.33 } }, "one_known_distance": { "1-3m": { "pairs": 3000000, "failed_pct": 7.8, - "median_in": 3.14, - "p90_in": 19.51, - "scale_error_pct": -0.51 + "median_in": 3.08, + "p90_in": 19.31, + "scale_error_pct": -0.41 }, "3-10m": { "pairs": 3000000, - "failed_pct": 7.81, - "median_in": 10.91, - "p90_in": 88.73, - "scale_error_pct": -1.36 + "failed_pct": 7.8, + "median_in": 10.76, + "p90_in": 87.9, + "scale_error_pct": -1.28 } }, "tape_calibration_success_pct": 92.2, @@ -3158,22 +3158,22 @@ "interval_95": { "1-3m": { "median_in": [ - 6.75, - 16.72 + 6.77, + 16.68 ], "p90_in": [ - 24.25, - Infinity + 24.21, + "failed" ] }, "3-10m": { "median_in": [ - 25.21, - 53.41 + 25.37, + 53.46 ], "p90_in": [ - 80.34, - Infinity + 80.62, + "failed" ] } } @@ -3199,16 +3199,16 @@ "1-3m": { "pairs": 3000000, "failed_pct": 7.8, - "median_in": 5.52, - "p90_in": 67.66, - "scale_error_pct": 3.9 + "median_in": 5.49, + "p90_in": 67.75, + "scale_error_pct": 4.02 }, "3-10m": { "pairs": 3000000, "failed_pct": 7.83, - "median_in": 15.07, - "p90_in": 102.94, - "scale_error_pct": 2.46 + "median_in": 15.1, + "p90_in": 102.92, + "scale_error_pct": 2.6 } }, "tape_calibration_success_pct": 92.2, @@ -3218,21 +3218,21 @@ "1-3m": { "median_in": [ 8.57, - 17.02 + 17.01 ], "p90_in": [ 26.76, - Infinity + "failed" ] }, "3-10m": { "median_in": [ 25.04, - 48.36 + 48.37 ], "p90_in": [ 74.14, - Infinity + "failed" ] } } @@ -3260,16 +3260,16 @@ "1-3m": { "pairs": 3000000, "failed_pct": 0.45, - "median_in": 4.21, - "p90_in": 16.48, - "scale_error_pct": -0.66 + "median_in": 4.16, + "p90_in": 16.55, + "scale_error_pct": -0.52 }, "3-10m": { "pairs": 3000000, "failed_pct": 0.48, - "median_in": 13.41, - "p90_in": 52.47, - "scale_error_pct": -2.33 + "median_in": 13.21, + "p90_in": 52.59, + "scale_error_pct": -2.18 } }, "tape_calibration_success_pct": 99.7, @@ -3303,32 +3303,32 @@ "1-3m": { "pairs": 120000, "failed_pct": 0.0, - "median_in": 8.87, - "p90_in": 19.61, - "scale_error_pct": -10.51 + "median_in": 8.85, + "p90_in": 19.52, + "scale_error_pct": -10.48 }, "3-10m": { "pairs": 120000, - "failed_pct": 0.01, - "median_in": 28.33, - "p90_in": 70.93, - "scale_error_pct": -12.36 + "failed_pct": 0.0, + "median_in": 28.58, + "p90_in": 70.76, + "scale_error_pct": -12.45 } }, "one_known_distance": { "1-3m": { "pairs": 3000000, "failed_pct": 0.3, - "median_in": 3.93, - "p90_in": 15.27, - "scale_error_pct": -0.95 + "median_in": 3.88, + "p90_in": 15.33, + "scale_error_pct": -0.83 }, "3-10m": { "pairs": 3000000, - "failed_pct": 0.31, - "median_in": 12.92, - "p90_in": 51.4, - "scale_error_pct": -2.47 + "failed_pct": 0.3, + "median_in": 12.7, + "p90_in": 51.58, + "scale_error_pct": -2.38 } }, "tape_calibration_success_pct": 99.7, @@ -3337,22 +3337,22 @@ "interval_95": { "1-3m": { "median_in": [ - 5.4, - 12.17 + 5.38, + 12.16 ], "p90_in": [ - 15.56, - 22.05 + 15.48, + 21.98 ] }, "3-10m": { "median_in": [ - 16.96, - 37.79 + 17.11, + 37.88 ], "p90_in": [ - 56.21, - 83.3 + 56.31, + 82.92 ] } } @@ -3370,7 +3370,7 @@ "pairs": 120000, "failed_pct": 0.02, "median_in": 24.22, - "p90_in": 62.49, + "p90_in": 62.5, "scale_error_pct": -7.22 } }, @@ -3378,16 +3378,16 @@ "1-3m": { "pairs": 3000000, "failed_pct": 0.3, - "median_in": 6.09, - "p90_in": 32.23, - "scale_error_pct": 4.02 + "median_in": 6.07, + "p90_in": 32.39, + "scale_error_pct": 4.13 }, "3-10m": { "pairs": 3000000, "failed_pct": 0.32, - "median_in": 15.55, - "p90_in": 60.52, - "scale_error_pct": 1.36 + "median_in": 15.56, + "p90_in": 60.87, + "scale_error_pct": 1.51 } }, "tape_calibration_success_pct": 99.7, @@ -3400,7 +3400,7 @@ 9.83 ], "p90_in": [ - 18.29, + 18.28, 28.97 ] }, @@ -3439,16 +3439,16 @@ "1-3m": { "pairs": 3000000, "failed_pct": 0.93, - "median_in": 4.07, - "p90_in": 16.07, - "scale_error_pct": -0.0 + "median_in": 4.04, + "p90_in": 16.1, + "scale_error_pct": 0.12 }, "3-10m": { "pairs": 3000000, "failed_pct": 0.95, - "median_in": 12.64, - "p90_in": 51.04, - "scale_error_pct": -1.52 + "median_in": 12.52, + "p90_in": 51.18, + "scale_error_pct": -1.37 } }, "tape_calibration_success_pct": 99.2, @@ -3482,15 +3482,15 @@ "1-3m": { "pairs": 120000, "failed_pct": 0.0, - "median_in": 6.57, - "p90_in": 17.22, - "scale_error_pct": -7.51 + "median_in": 6.6, + "p90_in": 17.32, + "scale_error_pct": -7.52 }, "3-10m": { "pairs": 120000, - "failed_pct": 0.01, - "median_in": 21.81, - "p90_in": 63.38, + "failed_pct": 0.0, + "median_in": 21.73, + "p90_in": 64.06, "scale_error_pct": -8.78 } }, @@ -3498,16 +3498,16 @@ "1-3m": { "pairs": 3000000, "failed_pct": 0.8, - "median_in": 3.79, - "p90_in": 14.79, - "scale_error_pct": -0.29 + "median_in": 3.78, + "p90_in": 14.9, + "scale_error_pct": -0.18 }, "3-10m": { "pairs": 3000000, - "failed_pct": 0.81, - "median_in": 12.16, - "p90_in": 50.65, - "scale_error_pct": -1.6 + "failed_pct": 0.8, + "median_in": 12.04, + "p90_in": 50.58, + "scale_error_pct": -1.53 } }, "tape_calibration_success_pct": 99.2, @@ -3516,22 +3516,22 @@ "interval_95": { "1-3m": { "median_in": [ - 4.42, - 9.96 + 4.5, + 9.95 ], "p90_in": [ - 11.29, - 20.25 + 11.42, + 20.44 ] }, "3-10m": { "median_in": [ - 14.55, - 33.1 + 14.61, + 33.28 ], "p90_in": [ - 37.22, - 91.36 + 37.16, + 90.27 ] } } @@ -3542,14 +3542,14 @@ "pairs": 120000, "failed_pct": 0.0, "median_in": 6.4, - "p90_in": 25.42, + "p90_in": 25.41, "scale_error_pct": -3.06 }, "3-10m": { "pairs": 120000, "failed_pct": 0.02, "median_in": 18.74, - "p90_in": 68.38, + "p90_in": 68.37, "scale_error_pct": -5.38 } }, @@ -3558,15 +3558,15 @@ "pairs": 3000000, "failed_pct": 0.8, "median_in": 6.39, - "p90_in": 37.61, - "scale_error_pct": 5.01 + "p90_in": 37.71, + "scale_error_pct": 5.13 }, "3-10m": { "pairs": 3000000, "failed_pct": 0.82, - "median_in": 15.95, - "p90_in": 67.86, - "scale_error_pct": 1.9 + "median_in": 15.97, + "p90_in": 68.06, + "scale_error_pct": 2.03 } }, "tape_calibration_success_pct": 99.2, @@ -3576,7 +3576,7 @@ "1-3m": { "median_in": [ 4.77, - 9.0 + 8.99 ], "p90_in": [ 16.57, @@ -3589,7 +3589,7 @@ 28.66 ], "p90_in": [ - 46.97, + 46.95, 90.06 ] } @@ -3620,16 +3620,16 @@ "1-3m": { "pairs": 3000000, "failed_pct": 0.68, - "median_in": 3.09, - "p90_in": 12.64, - "scale_error_pct": -0.19 + "median_in": 3.04, + "p90_in": 12.56, + "scale_error_pct": -0.06 }, "3-10m": { "pairs": 3000000, "failed_pct": 0.72, - "median_in": 9.93, - "p90_in": 43.7, - "scale_error_pct": -1.25 + "median_in": 9.75, + "p90_in": 43.33, + "scale_error_pct": -1.1 } }, "tape_calibration_success_pct": 99.5, @@ -3663,32 +3663,32 @@ "1-3m": { "pairs": 120000, "failed_pct": 0.0, - "median_in": 5.9, - "p90_in": 11.28, - "scale_error_pct": -6.47 + "median_in": 5.87, + "p90_in": 11.37, + "scale_error_pct": -6.46 }, "3-10m": { "pairs": 120000, - "failed_pct": 0.01, - "median_in": 19.57, - "p90_in": 46.8, - "scale_error_pct": -7.39 + "failed_pct": 0.0, + "median_in": 19.61, + "p90_in": 46.77, + "scale_error_pct": -7.44 } }, "one_known_distance": { "1-3m": { "pairs": 3000000, "failed_pct": 0.5, - "median_in": 2.83, - "p90_in": 11.29, - "scale_error_pct": -0.45 + "median_in": 2.77, + "p90_in": 11.13, + "scale_error_pct": -0.34 }, "3-10m": { "pairs": 3000000, - "failed_pct": 0.51, - "median_in": 9.33, - "p90_in": 42.72, - "scale_error_pct": -1.27 + "failed_pct": 0.5, + "median_in": 9.2, + "p90_in": 42.5, + "scale_error_pct": -1.19 } }, "tape_calibration_success_pct": 99.5, @@ -3698,21 +3698,21 @@ "1-3m": { "median_in": [ 5.05, - 6.88 + 6.87 ], "p90_in": [ - 9.72, - 13.39 + 9.76, + 13.41 ] }, "3-10m": { "median_in": [ - 15.14, - 22.47 + 15.24, + 22.69 ], "p90_in": [ - 35.65, - 58.06 + 35.58, + 58.04 ] } } @@ -3724,7 +3724,7 @@ "failed_pct": 0.0, "median_in": 5.57, "p90_in": 26.11, - "scale_error_pct": -2.23 + "scale_error_pct": -2.22 }, "3-10m": { "pairs": 120000, @@ -3738,16 +3738,16 @@ "1-3m": { "pairs": 3000000, "failed_pct": 0.5, - "median_in": 5.05, - "p90_in": 30.47, - "scale_error_pct": 3.93 + "median_in": 5.02, + "p90_in": 30.55, + "scale_error_pct": 4.03 }, "3-10m": { "pairs": 3000000, "failed_pct": 0.53, - "median_in": 13.83, - "p90_in": 55.79, - "scale_error_pct": 2.17 + "median_in": 13.8, + "p90_in": 55.88, + "scale_error_pct": 2.29 } }, "tape_calibration_success_pct": 99.5, @@ -3760,8 +3760,8 @@ 7.68 ], "p90_in": [ - 14.82, - 35.7 + 14.83, + 35.68 ] }, "3-10m": { @@ -3770,7 +3770,7 @@ 20.0 ], "p90_in": [ - 35.34, + 35.35, 58.73 ] } @@ -3799,16 +3799,16 @@ "1-3m": { "pairs": 3000000, "failed_pct": 0.65, - "median_in": 3.36, - "p90_in": 12.38, - "scale_error_pct": -0.36 + "median_in": 3.32, + "p90_in": 12.35, + "scale_error_pct": -0.22 }, "3-10m": { "pairs": 3000000, "failed_pct": 0.68, - "median_in": 10.84, - "p90_in": 42.51, - "scale_error_pct": -1.66 + "median_in": 10.67, + "p90_in": 42.34, + "scale_error_pct": -1.52 } }, "tape_calibration_success_pct": 99.5, @@ -3843,31 +3843,31 @@ "pairs": 120000, "failed_pct": 0.0, "median_in": 5.62, - "p90_in": 10.92, + "p90_in": 11.03, "scale_error_pct": -6.19 }, "3-10m": { "pairs": 120000, - "failed_pct": 0.01, - "median_in": 18.43, - "p90_in": 46.74, - "scale_error_pct": -7.04 + "failed_pct": 0.0, + "median_in": 18.63, + "p90_in": 47.03, + "scale_error_pct": -7.13 } }, "one_known_distance": { "1-3m": { "pairs": 3000000, "failed_pct": 0.5, - "median_in": 3.09, - "p90_in": 11.14, - "scale_error_pct": -0.67 + "median_in": 3.07, + "p90_in": 11.08, + "scale_error_pct": -0.54 }, "3-10m": { "pairs": 3000000, - "failed_pct": 0.51, - "median_in": 10.32, - "p90_in": 41.51, - "scale_error_pct": -1.7 + "failed_pct": 0.5, + "median_in": 10.19, + "p90_in": 41.35, + "scale_error_pct": -1.65 } }, "tape_calibration_success_pct": 99.5, @@ -3876,22 +3876,22 @@ "interval_95": { "1-3m": { "median_in": [ - 4.93, - 6.3 + 4.99, + 6.34 ], "p90_in": [ - 9.61, - 12.92 + 9.68, + 12.94 ] }, "3-10m": { "median_in": [ - 14.92, - 21.51 + 15.09, + 21.77 ], "p90_in": [ - 35.82, - 57.0 + 36.18, + 56.77 ] } } @@ -3903,30 +3903,30 @@ "failed_pct": 0.0, "median_in": 5.11, "p90_in": 24.62, - "scale_error_pct": -2.18 + "scale_error_pct": -2.17 }, "3-10m": { "pairs": 120000, "failed_pct": 0.02, "median_in": 14.82, "p90_in": 45.45, - "scale_error_pct": -3.85 + "scale_error_pct": -3.86 } }, "one_known_distance": { "1-3m": { "pairs": 3000000, "failed_pct": 0.5, - "median_in": 5.22, - "p90_in": 28.55, - "scale_error_pct": 4.07 + "median_in": 5.21, + "p90_in": 28.62, + "scale_error_pct": 4.18 }, "3-10m": { "pairs": 3000000, "failed_pct": 0.52, - "median_in": 13.72, - "p90_in": 52.85, - "scale_error_pct": 1.98 + "median_in": 13.73, + "p90_in": 52.95, + "scale_error_pct": 2.1 } }, "tape_calibration_success_pct": 99.5, @@ -3940,7 +3940,7 @@ ], "p90_in": [ 14.6, - 33.59 + 33.57 ] }, "3-10m": { @@ -3949,7 +3949,7 @@ 18.12 ], "p90_in": [ - 34.26, + 34.28, 55.41 ] } @@ -3978,16 +3978,16 @@ "1-3m": { "pairs": 3000000, "failed_pct": 1.93, - "median_in": 3.44, - "p90_in": 14.17, - "scale_error_pct": -0.05 + "median_in": 3.42, + "p90_in": 14.12, + "scale_error_pct": 0.08 }, "3-10m": { "pairs": 3000000, "failed_pct": 1.95, - "median_in": 10.92, - "p90_in": 48.16, - "scale_error_pct": -1.4 + "median_in": 10.82, + "p90_in": 47.94, + "scale_error_pct": -1.26 } }, "tape_calibration_success_pct": 98.2, @@ -4021,16 +4021,16 @@ "1-3m": { "pairs": 120000, "failed_pct": 0.0, - "median_in": 5.87, - "p90_in": 11.2, - "scale_error_pct": -6.4 + "median_in": 5.85, + "p90_in": 11.22, + "scale_error_pct": -6.39 }, "3-10m": { "pairs": 120000, - "failed_pct": 0.01, - "median_in": 19.11, - "p90_in": 46.28, - "scale_error_pct": -7.36 + "failed_pct": 0.0, + "median_in": 19.28, + "p90_in": 46.4, + "scale_error_pct": -7.43 } }, "one_known_distance": { @@ -4038,15 +4038,15 @@ "pairs": 3000000, "failed_pct": 1.8, "median_in": 3.16, - "p90_in": 12.82, - "scale_error_pct": -0.36 + "p90_in": 12.75, + "scale_error_pct": -0.26 }, "3-10m": { "pairs": 3000000, - "failed_pct": 1.81, - "median_in": 10.43, - "p90_in": 47.65, - "scale_error_pct": -1.44 + "failed_pct": 1.8, + "median_in": 10.34, + "p90_in": 47.24, + "scale_error_pct": -1.39 } }, "tape_calibration_success_pct": 98.2, @@ -4055,22 +4055,22 @@ "interval_95": { "1-3m": { "median_in": [ - 5.27, - 6.48 + 5.24, + 6.47 ], "p90_in": [ - 10.2, - 11.97 + 10.22, + 12.07 ] }, "3-10m": { "median_in": [ - 15.56, - 21.68 + 15.77, + 21.75 ], "p90_in": [ - 37.96, - 59.69 + 37.93, + 59.5 ] } } @@ -4081,7 +4081,7 @@ "pairs": 120000, "failed_pct": 0.0, "median_in": 5.27, - "p90_in": 28.03, + "p90_in": 28.02, "scale_error_pct": -2.38 }, "3-10m": { @@ -4096,16 +4096,16 @@ "1-3m": { "pairs": 3000000, "failed_pct": 1.8, - "median_in": 5.67, - "p90_in": 35.56, - "scale_error_pct": 4.59 + "median_in": 5.68, + "p90_in": 35.66, + "scale_error_pct": 4.73 }, "3-10m": { "pairs": 3000000, "failed_pct": 1.82, - "median_in": 14.6, - "p90_in": 64.72, - "scale_error_pct": 2.25 + "median_in": 14.68, + "p90_in": 64.65, + "scale_error_pct": 2.41 } }, "tape_calibration_success_pct": 98.2, @@ -4115,7 +4115,7 @@ "1-3m": { "median_in": [ 3.82, - 7.87 + 7.88 ], "p90_in": [ 15.44, @@ -4159,16 +4159,16 @@ "1-3m": { "pairs": 3000000, "failed_pct": 3.18, - "median_in": 3.12, - "p90_in": 14.61, - "scale_error_pct": -0.25 + "median_in": 3.08, + "p90_in": 14.37, + "scale_error_pct": -0.13 }, "3-10m": { "pairs": 3000000, "failed_pct": 3.21, - "median_in": 10.58, - "p90_in": 50.84, - "scale_error_pct": -1.26 + "median_in": 10.37, + "p90_in": 49.98, + "scale_error_pct": -1.11 } }, "tape_calibration_success_pct": 97.0, @@ -4182,7 +4182,7 @@ ], "p90_in": [ 11.67, - Infinity + "failed" ] }, "3-10m": { @@ -4192,7 +4192,7 @@ ], "p90_in": [ 40.21, - Infinity + "failed" ] } } @@ -4202,32 +4202,32 @@ "1-3m": { "pairs": 120000, "failed_pct": 2.5, - "median_in": 5.01, - "p90_in": 14.93, - "scale_error_pct": -3.62 + "median_in": 5.03, + "p90_in": 14.81, + "scale_error_pct": -3.65 }, "3-10m": { "pairs": 120000, - "failed_pct": 2.51, - "median_in": 16.37, - "p90_in": 54.99, - "scale_error_pct": -4.96 + "failed_pct": 2.5, + "median_in": 16.57, + "p90_in": 55.48, + "scale_error_pct": -5.1 } }, "one_known_distance": { "1-3m": { "pairs": 3000000, "failed_pct": 3.0, - "median_in": 2.85, - "p90_in": 13.2, - "scale_error_pct": -0.48 + "median_in": 2.8, + "p90_in": 12.85, + "scale_error_pct": -0.38 }, "3-10m": { "pairs": 3000000, - "failed_pct": 3.01, - "median_in": 9.98, - "p90_in": 50.02, - "scale_error_pct": -1.28 + "failed_pct": 3.0, + "median_in": 9.84, + "p90_in": 49.55, + "scale_error_pct": -1.2 } }, "tape_calibration_success_pct": 97.0, @@ -4237,21 +4237,21 @@ "1-3m": { "median_in": [ 3.74, - 6.74 + 6.75 ], "p90_in": [ - 10.95, - Infinity + 10.99, + "failed" ] }, "3-10m": { "median_in": [ - 12.4, - 22.79 + 12.52, + 22.85 ], "p90_in": [ - 38.2, - Infinity + 38.31, + "failed" ] } } @@ -4262,14 +4262,14 @@ "pairs": 120000, "failed_pct": 2.5, "median_in": 6.24, - "p90_in": 33.95, + "p90_in": 33.94, "scale_error_pct": 1.3 }, "3-10m": { "pairs": 120000, "failed_pct": 2.53, "median_in": 17.27, - "p90_in": 65.28, + "p90_in": 65.27, "scale_error_pct": -1.04 } }, @@ -4277,16 +4277,16 @@ "1-3m": { "pairs": 3000000, "failed_pct": 3.0, - "median_in": 5.18, - "p90_in": 37.1, - "scale_error_pct": 3.98 + "median_in": 5.15, + "p90_in": 37.17, + "scale_error_pct": 4.1 }, "3-10m": { "pairs": 3000000, "failed_pct": 3.03, - "median_in": 14.37, - "p90_in": 65.2, - "scale_error_pct": 2.52 + "median_in": 14.39, + "p90_in": 65.07, + "scale_error_pct": 2.65 } }, "tape_calibration_success_pct": 97.0, @@ -4300,17 +4300,17 @@ ], "p90_in": [ 16.23, - Infinity + "failed" ] }, "3-10m": { "median_in": [ 12.94, - 24.95 + 24.94 ], "p90_in": [ 43.33, - Infinity + "failed" ] } } @@ -4338,16 +4338,16 @@ "1-3m": { "pairs": 3000000, "failed_pct": 0.85, - "median_in": 3.74, - "p90_in": 14.28, - "scale_error_pct": -0.33 + "median_in": 3.69, + "p90_in": 14.32, + "scale_error_pct": -0.2 }, "3-10m": { "pairs": 3000000, "failed_pct": 0.88, - "median_in": 12.29, - "p90_in": 46.8, - "scale_error_pct": -1.69 + "median_in": 12.07, + "p90_in": 46.9, + "scale_error_pct": -1.53 } }, "tape_calibration_success_pct": 99.3, @@ -4381,32 +4381,32 @@ "1-3m": { "pairs": 120000, "failed_pct": 0.0, - "median_in": 2.83, + "median_in": 2.85, "p90_in": 9.26, - "scale_error_pct": -2.46 + "scale_error_pct": -2.51 }, "3-10m": { "pairs": 120000, - "failed_pct": 0.01, - "median_in": 9.5, - "p90_in": 38.89, - "scale_error_pct": -3.23 + "failed_pct": 0.0, + "median_in": 9.59, + "p90_in": 39.19, + "scale_error_pct": -3.28 } }, "one_known_distance": { "1-3m": { "pairs": 3000000, "failed_pct": 0.7, - "median_in": 3.46, - "p90_in": 13.1, - "scale_error_pct": -0.58 + "median_in": 3.42, + "p90_in": 13.09, + "scale_error_pct": -0.48 }, "3-10m": { "pairs": 3000000, - "failed_pct": 0.71, - "median_in": 11.78, - "p90_in": 45.87, - "scale_error_pct": -1.73 + "failed_pct": 0.7, + "median_in": 11.58, + "p90_in": 46.05, + "scale_error_pct": -1.65 } }, "tape_calibration_success_pct": 99.3, @@ -4415,22 +4415,22 @@ "interval_95": { "1-3m": { "median_in": [ - 2.38, - 3.44 + 2.41, + 3.47 ], "p90_in": [ - 7.75, - 10.98 + 7.81, + 10.96 ] }, "3-10m": { "median_in": [ - 7.22, - 12.32 + 7.32, + 12.39 ], "p90_in": [ - 28.33, - 59.4 + 28.34, + 58.61 ] } } @@ -4441,14 +4441,14 @@ "pairs": 120000, "failed_pct": 0.0, "median_in": 4.12, - "p90_in": 26.78, + "p90_in": 26.77, "scale_error_pct": 2.06 }, "3-10m": { "pairs": 120000, "failed_pct": 0.02, "median_in": 11.28, - "p90_in": 47.82, + "p90_in": 47.8, "scale_error_pct": 0.25 } }, @@ -4456,16 +4456,16 @@ "1-3m": { "pairs": 3000000, "failed_pct": 0.7, - "median_in": 5.66, - "p90_in": 30.31, - "scale_error_pct": 4.12 + "median_in": 5.63, + "p90_in": 30.55, + "scale_error_pct": 4.23 }, "3-10m": { "pairs": 3000000, "failed_pct": 0.72, "median_in": 14.81, - "p90_in": 55.9, - "scale_error_pct": 1.99 + "p90_in": 56.33, + "scale_error_pct": 2.12 } }, "tape_calibration_success_pct": 99.3, @@ -4485,11 +4485,11 @@ "3-10m": { "median_in": [ 8.3, - 14.81 + 14.82 ], "p90_in": [ - 34.61, - 62.6 + 34.64, + 62.56 ] } } @@ -4517,16 +4517,16 @@ "1-3m": { "pairs": 3000000, "failed_pct": 0.93, - "median_in": 3.32, - "p90_in": 13.16, - "scale_error_pct": -0.06 + "median_in": 3.28, + "p90_in": 13.05, + "scale_error_pct": 0.06 }, "3-10m": { "pairs": 3000000, "failed_pct": 0.95, - "median_in": 10.71, - "p90_in": 45.37, - "scale_error_pct": -1.33 + "median_in": 10.55, + "p90_in": 45.23, + "scale_error_pct": -1.21 } }, "tape_calibration_success_pct": 99.2, @@ -4560,32 +4560,32 @@ "1-3m": { "pairs": 120000, "failed_pct": 0.0, - "median_in": 2.34, - "p90_in": 7.6, - "scale_error_pct": -1.88 + "median_in": 2.37, + "p90_in": 7.66, + "scale_error_pct": -1.91 }, "3-10m": { "pairs": 120000, - "failed_pct": 0.01, - "median_in": 7.48, - "p90_in": 34.71, - "scale_error_pct": -2.64 + "failed_pct": 0.0, + "median_in": 7.63, + "p90_in": 34.9, + "scale_error_pct": -2.75 } }, "one_known_distance": { "1-3m": { "pairs": 3000000, "failed_pct": 0.8, - "median_in": 3.08, - "p90_in": 11.89, - "scale_error_pct": -0.31 + "median_in": 3.05, + "p90_in": 11.81, + "scale_error_pct": -0.24 }, "3-10m": { "pairs": 3000000, - "failed_pct": 0.81, - "median_in": 10.21, - "p90_in": 44.83, - "scale_error_pct": -1.36 + "failed_pct": 0.8, + "median_in": 10.07, + "p90_in": 44.62, + "scale_error_pct": -1.31 } }, "tape_calibration_success_pct": 99.2, @@ -4594,22 +4594,22 @@ "interval_95": { "1-3m": { "median_in": [ - 1.94, - 2.78 + 1.97, + 2.83 ], "p90_in": [ - 6.41, - 8.58 + 6.47, + 8.65 ] }, "3-10m": { "median_in": [ - 5.92, - 9.33 + 6.02, + 9.45 ], "p90_in": [ - 24.13, - 60.27 + 24.03, + 58.74 ] } } @@ -4620,14 +4620,14 @@ "pairs": 120000, "failed_pct": 0.0, "median_in": 3.58, - "p90_in": 27.59, + "p90_in": 27.57, "scale_error_pct": 2.19 }, "3-10m": { "pairs": 120000, "failed_pct": 0.02, "median_in": 9.95, - "p90_in": 49.83, + "p90_in": 49.81, "scale_error_pct": 0.27 } }, @@ -4635,16 +4635,16 @@ "1-3m": { "pairs": 3000000, "failed_pct": 0.8, - "median_in": 5.66, - "p90_in": 34.54, - "scale_error_pct": 4.69 + "median_in": 5.63, + "p90_in": 34.56, + "scale_error_pct": 4.76 }, "3-10m": { "pairs": 3000000, "failed_pct": 0.82, - "median_in": 14.46, - "p90_in": 63.22, - "scale_error_pct": 2.28 + "median_in": 14.49, + "p90_in": 63.04, + "scale_error_pct": 2.39 } }, "tape_calibration_success_pct": 99.2, @@ -4657,7 +4657,7 @@ 5.55 ], "p90_in": [ - 15.04, + 15.03, 49.59 ] }, @@ -4667,7 +4667,7 @@ 13.27 ], "p90_in": [ - 30.78, + 30.79, 81.52 ] } @@ -4685,14 +4685,14 @@ "pairs": 6000, "failed_pct": 29.77, "median_in": 10.22, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": 6.2 }, "3-10m": { "pairs": 6000, "failed_pct": 44.98, "median_in": 27.66, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": 13.26 } }, @@ -4701,14 +4701,14 @@ "pairs": 150000, "failed_pct": 44.16, "median_in": 4.13, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": -0.11 }, "3-10m": { "pairs": 150000, "failed_pct": 57.81, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -0.66 } }, @@ -4722,18 +4722,18 @@ 17.11 ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ 26.95, - Infinity + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -4742,33 +4742,33 @@ "none": { "1-3m": { "pairs": 3000, - "failed_pct": 6.67, - "median_in": 11.82, - "p90_in": 18.55, - "scale_error_pct": 15.19 + "failed_pct": 7.6, + "median_in": 12.03, + "p90_in": 18.99, + "scale_error_pct": 15.46 }, "3-10m": { "pairs": 3000, - "failed_pct": 8.87, - "median_in": 22.91, - "p90_in": 32.76, - "scale_error_pct": 15.4 + "failed_pct": 9.27, + "median_in": 22.92, + "p90_in": 32.9, + "scale_error_pct": 15.51 } }, "one_known_distance": { "1-3m": { "pairs": 75000, - "failed_pct": 36.53, - "median_in": 4.03, - "p90_in": Infinity, - "scale_error_pct": -1.32 + "failed_pct": 37.17, + "median_in": 4.05, + "p90_in": "failed", + "scale_error_pct": -1.14 }, "3-10m": { "pairs": 75000, - "failed_pct": 38.03, - "median_in": 6.9, - "p90_in": Infinity, - "scale_error_pct": -1.02 + "failed_pct": 38.3, + "median_in": 6.99, + "p90_in": "failed", + "scale_error_pct": -0.93 } }, "tape_calibration_success_pct": 68.0, @@ -4777,22 +4777,22 @@ "interval_95": { "1-3m": { "median_in": [ - 11.82, - 11.82 + 12.03, + 12.03 ], "p90_in": [ - 18.55, - 18.55 + 18.99, + 18.99 ] }, "3-10m": { "median_in": [ - 22.91, - 22.91 + 22.92, + 22.92 ], "p90_in": [ - 32.76, - 32.76 + 32.9, + 32.9 ] } } @@ -4802,15 +4802,15 @@ "1-3m": { "pairs": 3000, "failed_pct": 78.43, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": 15.75 }, "3-10m": { "pairs": 3000, "failed_pct": 77.33, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": 16.09 } }, @@ -4818,15 +4818,15 @@ "1-3m": { "pairs": 75000, "failed_pct": 85.33, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -0.8 }, "3-10m": { "pairs": 75000, "failed_pct": 84.59, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -0.48 } }, @@ -4836,22 +4836,22 @@ "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -4864,14 +4864,14 @@ "pairs": 6000, "failed_pct": 25.88, "median_in": 8.4, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": 4.51 }, "3-10m": { "pairs": 6000, "failed_pct": 41.02, "median_in": 24.74, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": 11.62 } }, @@ -4880,14 +4880,14 @@ "pairs": 150000, "failed_pct": 42.34, "median_in": 4.43, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": -0.41 }, "3-10m": { "pairs": 150000, "failed_pct": 53.79, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -0.95 } }, @@ -4901,18 +4901,18 @@ 14.71 ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ 24.41, - Infinity + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -4921,33 +4921,33 @@ "none": { "1-3m": { "pairs": 3000, - "failed_pct": 2.2, - "median_in": 9.78, - "p90_in": 16.04, - "scale_error_pct": 13.53 + "failed_pct": 3.13, + "median_in": 10.21, + "p90_in": 16.45, + "scale_error_pct": 13.76 }, "3-10m": { "pairs": 3000, - "failed_pct": 4.2, - "median_in": 21.01, - "p90_in": 28.31, - "scale_error_pct": 14.33 + "failed_pct": 4.7, + "median_in": 21.26, + "p90_in": 28.22, + "scale_error_pct": 14.53 } }, "one_known_distance": { "1-3m": { "pairs": 75000, - "failed_pct": 21.76, - "median_in": 3.86, - "p90_in": Infinity, - "scale_error_pct": -2.58 + "failed_pct": 22.51, + "median_in": 3.7, + "p90_in": "failed", + "scale_error_pct": -2.41 }, "3-10m": { "pairs": 75000, - "failed_pct": 23.36, - "median_in": 5.49, - "p90_in": Infinity, - "scale_error_pct": -1.73 + "failed_pct": 23.76, + "median_in": 5.37, + "p90_in": "failed", + "scale_error_pct": -1.61 } }, "tape_calibration_success_pct": 80.0, @@ -4956,22 +4956,22 @@ "interval_95": { "1-3m": { "median_in": [ - 9.78, - 9.78 + 10.21, + 10.21 ], "p90_in": [ - 16.04, - 16.04 + 16.45, + 16.45 ] }, "3-10m": { "median_in": [ - 21.01, - 21.01 + 21.26, + 21.26 ], "p90_in": [ - 28.31, - 28.31 + 28.22, + 28.22 ] } } @@ -4981,15 +4981,15 @@ "1-3m": { "pairs": 3000, "failed_pct": 73.37, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": 14.08 }, "3-10m": { "pairs": 3000, "failed_pct": 71.9, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": 15.5 } }, @@ -4997,15 +4997,15 @@ "1-3m": { "pairs": 75000, "failed_pct": 78.69, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -1.82 }, "3-10m": { "pairs": 75000, "failed_pct": 77.52, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -0.53 } }, @@ -5015,22 +5015,22 @@ "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -5045,14 +5045,14 @@ "pairs": 6000, "failed_pct": 28.95, "median_in": 8.28, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": 3.82 }, "3-10m": { "pairs": 6000, "failed_pct": 39.12, "median_in": 19.98, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": 9.41 } }, @@ -5061,14 +5061,14 @@ "pairs": 150000, "failed_pct": 46.87, "median_in": 5.46, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": -0.27 }, "3-10m": { "pairs": 150000, "failed_pct": 55.13, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -0.89 } }, @@ -5082,8 +5082,8 @@ 13.88 ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { @@ -5092,8 +5092,8 @@ 21.3 ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -5102,33 +5102,33 @@ "none": { "1-3m": { "pairs": 3000, - "failed_pct": 6.23, - "median_in": 9.44, - "p90_in": 15.43, - "scale_error_pct": 12.3 + "failed_pct": 7.63, + "median_in": 9.74, + "p90_in": 16.06, + "scale_error_pct": 12.56 }, "3-10m": { "pairs": 3000, "failed_pct": 8.63, - "median_in": 18.25, - "p90_in": 25.33, - "scale_error_pct": 12.21 + "median_in": 18.32, + "p90_in": 25.1, + "scale_error_pct": 12.34 } }, "one_known_distance": { "1-3m": { "pairs": 75000, - "failed_pct": 36.24, - "median_in": 4.48, - "p90_in": Infinity, - "scale_error_pct": -1.46 + "failed_pct": 37.19, + "median_in": 4.55, + "p90_in": "failed", + "scale_error_pct": -1.26 }, "3-10m": { "pairs": 75000, "failed_pct": 37.87, - "median_in": 7.32, - "p90_in": Infinity, - "scale_error_pct": -1.33 + "median_in": 7.3, + "p90_in": "failed", + "scale_error_pct": -1.23 } }, "tape_calibration_success_pct": 68.0, @@ -5137,22 +5137,22 @@ "interval_95": { "1-3m": { "median_in": [ - 9.44, - 9.44 + 9.74, + 9.74 ], "p90_in": [ - 15.43, - 15.43 + 16.06, + 16.06 ] }, "3-10m": { "median_in": [ - 18.25, - 18.25 + 18.32, + 18.32 ], "p90_in": [ - 25.33, - 25.33 + 25.1, + 25.1 ] } } @@ -5162,15 +5162,15 @@ "1-3m": { "pairs": 3000, "failed_pct": 78.3, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": 12.18 }, "3-10m": { "pairs": 3000, "failed_pct": 77.47, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": 12.72 } }, @@ -5178,15 +5178,15 @@ "1-3m": { "pairs": 75000, "failed_pct": 85.24, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -1.5 }, "3-10m": { "pairs": 75000, "failed_pct": 84.68, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -0.92 } }, @@ -5196,22 +5196,22 @@ "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -5224,14 +5224,14 @@ "pairs": 6000, "failed_pct": 24.75, "median_in": 3.37, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": 1.87 }, "3-10m": { "pairs": 6000, "failed_pct": 37.88, "median_in": 9.33, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": 3.25 } }, @@ -5240,14 +5240,14 @@ "pairs": 150000, "failed_pct": 39.55, "median_in": 3.15, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": -0.16 }, "3-10m": { "pairs": 150000, "failed_pct": 50.61, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -0.62 } }, @@ -5261,8 +5261,8 @@ 6.42 ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { @@ -5271,8 +5271,8 @@ 9.77 ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -5281,33 +5281,33 @@ "none": { "1-3m": { "pairs": 3000, - "failed_pct": 2.37, - "median_in": 4.45, - "p90_in": 7.69, - "scale_error_pct": 5.91 + "failed_pct": 3.4, + "median_in": 4.58, + "p90_in": 7.83, + "scale_error_pct": 6.04 }, "3-10m": { "pairs": 3000, - "failed_pct": 4.3, - "median_in": 8.77, - "p90_in": 12.24, - "scale_error_pct": 5.88 + "failed_pct": 4.9, + "median_in": 8.81, + "p90_in": 12.22, + "scale_error_pct": 5.9 } }, "one_known_distance": { "1-3m": { "pairs": 75000, - "failed_pct": 25.8, - "median_in": 2.96, - "p90_in": Infinity, - "scale_error_pct": -0.51 + "failed_pct": 26.58, + "median_in": 2.99, + "p90_in": "failed", + "scale_error_pct": -0.38 }, "3-10m": { "pairs": 75000, - "failed_pct": 27.27, - "median_in": 4.67, - "p90_in": Infinity, - "scale_error_pct": -0.26 + "failed_pct": 27.72, + "median_in": 4.65, + "p90_in": "failed", + "scale_error_pct": -0.21 } }, "tape_calibration_success_pct": 76.0, @@ -5316,22 +5316,22 @@ "interval_95": { "1-3m": { "median_in": [ - 4.45, - 4.45 + 4.58, + 4.58 ], "p90_in": [ - 7.69, - 7.69 + 7.83, + 7.83 ] }, "3-10m": { "median_in": [ - 8.77, - 8.77 + 8.81, + 8.81 ], "p90_in": [ - 12.24, - 12.24 + 12.22, + 12.22 ] } } @@ -5341,15 +5341,15 @@ "1-3m": { "pairs": 3000, "failed_pct": 73.47, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": 5.98 }, "3-10m": { "pairs": 3000, "failed_pct": 71.3, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": 6.68 } }, @@ -5357,15 +5357,15 @@ "1-3m": { "pairs": 75000, "failed_pct": 79.83, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -0.44 }, "3-10m": { "pairs": 75000, "failed_pct": 78.19, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": 0.63 } }, @@ -5375,22 +5375,22 @@ "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -5405,14 +5405,14 @@ "pairs": 6000, "failed_pct": 28.15, "median_in": 7.68, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": 2.39 }, "3-10m": { "pairs": 6000, "failed_pct": 38.63, "median_in": 19.1, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": 8.84 } }, @@ -5421,14 +5421,14 @@ "pairs": 150000, "failed_pct": 44.54, "median_in": 4.42, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": -0.18 }, "3-10m": { "pairs": 150000, "failed_pct": 53.51, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -0.82 } }, @@ -5442,8 +5442,8 @@ 13.65 ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { @@ -5452,8 +5452,8 @@ 20.72 ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -5462,33 +5462,33 @@ "none": { "1-3m": { "pairs": 3000, - "failed_pct": 6.33, - "median_in": 9.29, - "p90_in": 14.96, - "scale_error_pct": 11.96 + "failed_pct": 7.7, + "median_in": 9.55, + "p90_in": 15.56, + "scale_error_pct": 12.21 }, "3-10m": { "pairs": 3000, - "failed_pct": 9.03, - "median_in": 17.69, - "p90_in": 24.5, - "scale_error_pct": 11.82 + "failed_pct": 8.67, + "median_in": 17.76, + "p90_in": 24.26, + "scale_error_pct": 11.95 } }, "one_known_distance": { "1-3m": { "pairs": 75000, - "failed_pct": 36.31, - "median_in": 4.43, - "p90_in": Infinity, - "scale_error_pct": -1.27 + "failed_pct": 37.24, + "median_in": 4.51, + "p90_in": "failed", + "scale_error_pct": -1.09 }, "3-10m": { "pairs": 75000, - "failed_pct": 38.14, - "median_in": 7.34, - "p90_in": Infinity, - "scale_error_pct": -1.29 + "failed_pct": 37.89, + "median_in": 7.26, + "p90_in": "failed", + "scale_error_pct": -1.18 } }, "tape_calibration_success_pct": 68.0, @@ -5497,22 +5497,22 @@ "interval_95": { "1-3m": { "median_in": [ - 9.29, - 9.29 + 9.55, + 9.55 ], "p90_in": [ - 14.96, - 14.96 + 15.56, + 15.56 ] }, "3-10m": { "median_in": [ - 17.69, - 17.69 + 17.76, + 17.76 ], "p90_in": [ - 24.5, - 24.5 + 24.26, + 24.26 ] } } @@ -5522,15 +5522,15 @@ "1-3m": { "pairs": 3000, "failed_pct": 78.3, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": 11.12 }, "3-10m": { "pairs": 3000, "failed_pct": 77.47, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": 12.05 } }, @@ -5538,15 +5538,15 @@ "1-3m": { "pairs": 75000, "failed_pct": 85.24, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -1.97 }, "3-10m": { "pairs": 75000, "failed_pct": 84.68, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -1.01 } }, @@ -5556,22 +5556,22 @@ "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -5584,14 +5584,14 @@ "pairs": 6000, "failed_pct": 23.77, "median_in": 3.88, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": 1.35 }, "3-10m": { "pairs": 6000, "failed_pct": 37.88, "median_in": 12.85, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": 5.15 } }, @@ -5600,14 +5600,14 @@ "pairs": 150000, "failed_pct": 42.06, "median_in": 4.18, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": -0.22 }, "3-10m": { "pairs": 150000, "failed_pct": 52.79, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -0.65 } }, @@ -5621,8 +5621,8 @@ 7.65 ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { @@ -5631,8 +5631,8 @@ 13.25 ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -5641,33 +5641,33 @@ "none": { "1-3m": { "pairs": 3000, - "failed_pct": 2.2, - "median_in": 4.96, - "p90_in": 9.54, - "scale_error_pct": 6.73 + "failed_pct": 3.17, + "median_in": 5.17, + "p90_in": 9.75, + "scale_error_pct": 7.01 }, "3-10m": { "pairs": 3000, - "failed_pct": 4.0, - "median_in": 11.53, - "p90_in": 16.08, - "scale_error_pct": 7.7 + "failed_pct": 4.43, + "median_in": 11.55, + "p90_in": 16.02, + "scale_error_pct": 7.72 } }, "one_known_distance": { "1-3m": { "pairs": 75000, - "failed_pct": 25.67, - "median_in": 3.87, - "p90_in": Infinity, - "scale_error_pct": -1.77 + "failed_pct": 26.41, + "median_in": 3.85, + "p90_in": "failed", + "scale_error_pct": -1.58 }, "3-10m": { "pairs": 75000, - "failed_pct": 27.04, - "median_in": 6.14, - "p90_in": Infinity, - "scale_error_pct": -0.67 + "failed_pct": 27.37, + "median_in": 6.09, + "p90_in": "failed", + "scale_error_pct": -0.59 } }, "tape_calibration_success_pct": 76.0, @@ -5676,22 +5676,22 @@ "interval_95": { "1-3m": { "median_in": [ - 4.96, - 4.96 + 5.17, + 5.17 ], "p90_in": [ - 9.54, - 9.54 + 9.75, + 9.75 ] }, "3-10m": { "median_in": [ - 11.53, - 11.53 + 11.55, + 11.55 ], "p90_in": [ - 16.08, - 16.08 + 16.02, + 16.02 ] } } @@ -5701,15 +5701,15 @@ "1-3m": { "pairs": 3000, "failed_pct": 73.2, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": 7.77 }, "3-10m": { "pairs": 3000, "failed_pct": 71.1, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": 8.98 } }, @@ -5717,15 +5717,15 @@ "1-3m": { "pairs": 75000, "failed_pct": 79.63, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -0.58 }, "3-10m": { "pairs": 75000, "failed_pct": 78.04, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": 0.75 } }, @@ -5735,22 +5735,22 @@ "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -5765,14 +5765,14 @@ "pairs": 6000, "failed_pct": 28.15, "median_in": 7.63, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": 3.31 }, "3-10m": { "pairs": 6000, "failed_pct": 38.02, "median_in": 18.75, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": 8.51 } }, @@ -5781,14 +5781,14 @@ "pairs": 150000, "failed_pct": 47.86, "median_in": 6.25, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": -0.29 }, "3-10m": { "pairs": 150000, "failed_pct": 55.44, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -0.95 } }, @@ -5802,8 +5802,8 @@ 13.45 ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { @@ -5812,8 +5812,8 @@ 20.46 ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -5822,33 +5822,33 @@ "none": { "1-3m": { "pairs": 3000, - "failed_pct": 6.23, - "median_in": 9.06, - "p90_in": 14.89, - "scale_error_pct": 11.81 + "failed_pct": 7.67, + "median_in": 9.31, + "p90_in": 15.57, + "scale_error_pct": 12.06 }, "3-10m": { "pairs": 3000, - "failed_pct": 9.0, - "median_in": 17.5, - "p90_in": 24.35, - "scale_error_pct": 11.66 + "failed_pct": 8.63, + "median_in": 17.53, + "p90_in": 23.87, + "scale_error_pct": 11.78 } }, "one_known_distance": { "1-3m": { "pairs": 75000, - "failed_pct": 36.24, - "median_in": 4.55, - "p90_in": Infinity, - "scale_error_pct": -1.52 + "failed_pct": 37.21, + "median_in": 4.61, + "p90_in": "failed", + "scale_error_pct": -1.33 }, "3-10m": { "pairs": 75000, - "failed_pct": 38.12, - "median_in": 7.36, - "p90_in": Infinity, - "scale_error_pct": -1.44 + "failed_pct": 37.87, + "median_in": 7.28, + "p90_in": "failed", + "scale_error_pct": -1.34 } }, "tape_calibration_success_pct": 68.0, @@ -5857,22 +5857,22 @@ "interval_95": { "1-3m": { "median_in": [ - 9.06, - 9.06 + 9.31, + 9.31 ], "p90_in": [ - 14.89, - 14.89 + 15.57, + 15.57 ] }, "3-10m": { "median_in": [ - 17.5, - 17.5 + 17.53, + 17.53 ], "p90_in": [ - 24.35, - 24.35 + 23.87, + 23.87 ] } } @@ -5882,15 +5882,15 @@ "1-3m": { "pairs": 3000, "failed_pct": 78.3, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": 11.43 }, "3-10m": { "pairs": 3000, "failed_pct": 77.47, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": 12.09 } }, @@ -5898,15 +5898,15 @@ "1-3m": { "pairs": 75000, "failed_pct": 85.24, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -1.95 }, "3-10m": { "pairs": 75000, "failed_pct": 84.68, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -1.07 } }, @@ -5916,22 +5916,22 @@ "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -5944,14 +5944,14 @@ "pairs": 6000, "failed_pct": 23.45, "median_in": 2.72, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": 1.59 }, "3-10m": { "pairs": 6000, "failed_pct": 35.58, "median_in": 7.68, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": 2.12 } }, @@ -5960,14 +5960,14 @@ "pairs": 150000, "failed_pct": 38.46, "median_in": 2.98, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": -0.09 }, "3-10m": { "pairs": 150000, "failed_pct": 48.68, "median_in": 12.47, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": -0.59 } }, @@ -5981,8 +5981,8 @@ 5.76 ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { @@ -5991,8 +5991,8 @@ 8.46 ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -6001,33 +6001,33 @@ "none": { "1-3m": { "pairs": 3000, - "failed_pct": 2.37, - "median_in": 3.94, - "p90_in": 7.06, - "scale_error_pct": 5.2 + "failed_pct": 3.4, + "median_in": 4.07, + "p90_in": 7.21, + "scale_error_pct": 5.35 }, "3-10m": { "pairs": 3000, - "failed_pct": 4.3, - "median_in": 7.64, - "p90_in": 11.05, - "scale_error_pct": 5.11 + "failed_pct": 4.9, + "median_in": 7.66, + "p90_in": 11.07, + "scale_error_pct": 5.12 } }, "one_known_distance": { "1-3m": { "pairs": 75000, - "failed_pct": 25.8, - "median_in": 3.01, - "p90_in": Infinity, - "scale_error_pct": -0.51 + "failed_pct": 26.58, + "median_in": 3.03, + "p90_in": "failed", + "scale_error_pct": -0.38 }, "3-10m": { "pairs": 75000, - "failed_pct": 27.27, - "median_in": 4.84, - "p90_in": Infinity, - "scale_error_pct": -0.32 + "failed_pct": 27.72, + "median_in": 4.78, + "p90_in": "failed", + "scale_error_pct": -0.27 } }, "tape_calibration_success_pct": 76.0, @@ -6036,22 +6036,22 @@ "interval_95": { "1-3m": { "median_in": [ - 3.94, - 3.94 + 4.07, + 4.07 ], "p90_in": [ - 7.06, - 7.06 + 7.21, + 7.21 ] }, "3-10m": { "median_in": [ - 7.64, - 7.64 + 7.66, + 7.66 ], "p90_in": [ - 11.05, - 11.05 + 11.07, + 11.07 ] } } @@ -6061,15 +6061,15 @@ "1-3m": { "pairs": 3000, "failed_pct": 73.67, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": 5.01 }, "3-10m": { "pairs": 3000, "failed_pct": 71.47, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": 5.79 } }, @@ -6077,15 +6077,15 @@ "1-3m": { "pairs": 75000, "failed_pct": 79.99, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": -0.65 }, "3-10m": { "pairs": 75000, "failed_pct": 78.31, - "median_in": Infinity, - "p90_in": Infinity, + "median_in": "failed", + "p90_in": "failed", "scale_error_pct": 0.54 } }, @@ -6095,22 +6095,22 @@ "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -6183,32 +6183,32 @@ "1-3m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 15.83, - "p90_in": 24.19, - "scale_error_pct": 20.71 + "median_in": 15.67, + "p90_in": 23.97, + "scale_error_pct": 20.35 }, "3-10m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 26.12, - "p90_in": 32.33, - "scale_error_pct": 17.68 + "median_in": 25.79, + "p90_in": 32.2, + "scale_error_pct": 17.45 } }, "one_known_distance": { "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 3.36, - "p90_in": 8.69, - "scale_error_pct": 2.22 + "median_in": 3.25, + "p90_in": 8.45, + "scale_error_pct": 1.97 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 4.87, - "p90_in": 12.11, - "scale_error_pct": -0.56 + "median_in": 4.83, + "p90_in": 12.18, + "scale_error_pct": -0.73 } }, "tape_calibration_success_pct": 100.0, @@ -6217,22 +6217,22 @@ "interval_95": { "1-3m": { "median_in": [ - 15.83, - 15.83 + 15.67, + 15.67 ], "p90_in": [ - 24.19, - 24.19 + 23.97, + 23.97 ] }, "3-10m": { "median_in": [ - 26.12, - 26.12 + 25.79, + 25.79 ], "p90_in": [ - 32.33, - 32.33 + 32.2, + 32.2 ] } } @@ -6362,32 +6362,32 @@ "1-3m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 15.81, - "p90_in": 23.32, - "scale_error_pct": 20.41 + "median_in": 15.54, + "p90_in": 23.18, + "scale_error_pct": 20.18 }, "3-10m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 25.41, - "p90_in": 31.08, - "scale_error_pct": 17.29 + "median_in": 25.24, + "p90_in": 30.97, + "scale_error_pct": 17.04 } }, "one_known_distance": { "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 3.43, - "p90_in": 8.76, - "scale_error_pct": 2.07 + "median_in": 3.34, + "p90_in": 8.63, + "scale_error_pct": 1.87 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 5.01, - "p90_in": 13.12, - "scale_error_pct": -0.71 + "median_in": 4.96, + "p90_in": 13.29, + "scale_error_pct": -0.86 } }, "tape_calibration_success_pct": 100.0, @@ -6396,22 +6396,22 @@ "interval_95": { "1-3m": { "median_in": [ - 15.81, - 15.81 + 15.54, + 15.54 ], "p90_in": [ - 23.32, - 23.32 + 23.18, + 23.18 ] }, "3-10m": { "median_in": [ - 25.41, - 25.41 + 25.24, + 25.24 ], "p90_in": [ - 31.08, - 31.08 + 30.97, + 30.97 ] } } @@ -6541,32 +6541,32 @@ "1-3m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 15.36, - "p90_in": 22.55, - "scale_error_pct": 20.32 + "median_in": 15.24, + "p90_in": 22.54, + "scale_error_pct": 20.21 }, "3-10m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 25.26, - "p90_in": 30.98, - "scale_error_pct": 17.26 + "median_in": 25.15, + "p90_in": 30.54, + "scale_error_pct": 17.07 } }, "one_known_distance": { "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 5.05, - "p90_in": 13.49, - "scale_error_pct": 2.76 + "median_in": 4.91, + "p90_in": 13.46, + "scale_error_pct": 2.51 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 6.63, - "p90_in": 28.42, - "scale_error_pct": -0.81 + "median_in": 6.57, + "p90_in": 28.83, + "scale_error_pct": -1.01 } }, "tape_calibration_success_pct": 100.0, @@ -6575,22 +6575,22 @@ "interval_95": { "1-3m": { "median_in": [ - 15.36, - 15.36 + 15.24, + 15.24 ], "p90_in": [ - 22.55, - 22.55 + 22.54, + 22.54 ] }, "3-10m": { "median_in": [ - 25.26, - 25.26 + 25.15, + 25.15 ], "p90_in": [ - 30.98, - 30.98 + 30.54, + 30.54 ] } } @@ -6722,32 +6722,32 @@ "1-3m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 3.39, - "p90_in": 9.17, - "scale_error_pct": 4.67 + "median_in": 3.09, + "p90_in": 8.82, + "scale_error_pct": 4.25 }, "3-10m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 3.06, - "p90_in": 7.63, - "scale_error_pct": 1.78 + "median_in": 2.81, + "p90_in": 7.45, + "scale_error_pct": 1.59 } }, "one_known_distance": { "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 3.37, - "p90_in": 8.73, - "scale_error_pct": 2.23 + "median_in": 3.26, + "p90_in": 8.49, + "scale_error_pct": 1.97 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 4.89, - "p90_in": 12.16, - "scale_error_pct": -0.55 + "median_in": 4.85, + "p90_in": 12.25, + "scale_error_pct": -0.73 } }, "tape_calibration_success_pct": 100.0, @@ -6756,22 +6756,22 @@ "interval_95": { "1-3m": { "median_in": [ - 3.39, - 3.39 + 3.09, + 3.09 ], "p90_in": [ - 9.17, - 9.17 + 8.82, + 8.82 ] }, "3-10m": { "median_in": [ - 3.06, - 3.06 + 2.81, + 2.81 ], "p90_in": [ - 7.63, - 7.63 + 7.45, + 7.45 ] } } @@ -6901,32 +6901,32 @@ "1-3m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 8.21, - "p90_in": 14.9, - "scale_error_pct": 10.57 + "median_in": 8.05, + "p90_in": 14.84, + "scale_error_pct": 10.45 }, "3-10m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 11.24, - "p90_in": 15.57, - "scale_error_pct": 7.42 + "median_in": 10.82, + "p90_in": 15.25, + "scale_error_pct": 7.18 } }, "one_known_distance": { "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 3.82, - "p90_in": 10.57, - "scale_error_pct": 3.23 + "median_in": 3.76, + "p90_in": 10.39, + "scale_error_pct": 3.05 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 5.04, - "p90_in": 14.8, - "scale_error_pct": 0.04 + "median_in": 4.96, + "p90_in": 14.93, + "scale_error_pct": -0.15 } }, "tape_calibration_success_pct": 100.0, @@ -6935,22 +6935,22 @@ "interval_95": { "1-3m": { "median_in": [ - 8.21, - 8.21 + 8.05, + 8.05 ], "p90_in": [ - 14.9, - 14.9 + 14.84, + 14.84 ] }, "3-10m": { "median_in": [ - 11.24, - 11.24 + 10.82, + 10.82 ], "p90_in": [ - 15.57, - 15.57 + 15.25, + 15.25 ] } } @@ -7080,32 +7080,32 @@ "1-3m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 7.56, - "p90_in": 12.66, - "scale_error_pct": 9.83 + "median_in": 7.47, + "p90_in": 12.77, + "scale_error_pct": 9.72 }, "3-10m": { "pairs": 3000, "failed_pct": 0.0, - "median_in": 11.16, - "p90_in": 14.79, - "scale_error_pct": 7.68 + "median_in": 10.96, + "p90_in": 14.52, + "scale_error_pct": 7.46 } }, "one_known_distance": { "1-3m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 3.51, - "p90_in": 9.04, - "scale_error_pct": 1.95 + "median_in": 3.45, + "p90_in": 8.99, + "scale_error_pct": 1.72 }, "3-10m": { "pairs": 75000, "failed_pct": 0.0, - "median_in": 5.18, - "p90_in": 15.86, - "scale_error_pct": -1.27 + "median_in": 5.24, + "p90_in": 16.06, + "scale_error_pct": -1.47 } }, "tape_calibration_success_pct": 100.0, @@ -7114,22 +7114,22 @@ "interval_95": { "1-3m": { "median_in": [ - 7.56, - 7.56 + 7.47, + 7.47 ], "p90_in": [ - 12.66, - 12.66 + 12.77, + 12.77 ] }, "3-10m": { "median_in": [ - 11.16, - 11.16 + 10.96, + 10.96 ], "p90_in": [ - 14.79, - 14.79 + 14.52, + 14.52 ] } } @@ -7261,32 +7261,32 @@ "1-3m": { "pairs": 15000, "failed_pct": 0.0, - "median_in": 15.83, - "p90_in": 24.42, - "scale_error_pct": 20.72 + "median_in": 15.7, + "p90_in": 24.07, + "scale_error_pct": 20.29 }, "3-10m": { "pairs": 15000, "failed_pct": 0.0, - "median_in": 26.04, - "p90_in": 32.64, - "scale_error_pct": 17.55 + "median_in": 25.7, + "p90_in": 32.51, + "scale_error_pct": 17.34 } }, "one_known_distance": { "1-3m": { "pairs": 375000, "failed_pct": 0.0, - "median_in": 3.45, - "p90_in": 9.14, - "scale_error_pct": 2.17 + "median_in": 3.35, + "p90_in": 8.9, + "scale_error_pct": 1.91 }, "3-10m": { "pairs": 375000, "failed_pct": 0.0, - "median_in": 4.99, - "p90_in": 12.81, - "scale_error_pct": -0.6 + "median_in": 4.95, + "p90_in": 12.89, + "scale_error_pct": -0.79 } }, "tape_calibration_success_pct": 100.0, @@ -7295,22 +7295,22 @@ "interval_95": { "1-3m": { "median_in": [ - 15.68, - 16.06 + 15.54, + 15.89 ], "p90_in": [ - 23.93, - 25.32 + 23.71, + 24.85 ] }, "3-10m": { "median_in": [ - 25.83, - 26.27 + 25.44, + 25.97 ], "p90_in": [ - 31.93, - 33.6 + 31.9, + 33.4 ] } } @@ -7440,32 +7440,32 @@ "1-3m": { "pairs": 15000, "failed_pct": 0.0, - "median_in": 15.69, - "p90_in": 23.32, - "scale_error_pct": 20.31 + "median_in": 15.45, + "p90_in": 23.02, + "scale_error_pct": 20.14 }, "3-10m": { "pairs": 15000, "failed_pct": 0.0, - "median_in": 24.97, - "p90_in": 31.19, - "scale_error_pct": 16.84 + "median_in": 24.67, + "p90_in": 30.85, + "scale_error_pct": 16.65 } }, "one_known_distance": { "1-3m": { "pairs": 375000, "failed_pct": 0.0, - "median_in": 3.35, - "p90_in": 8.63, - "scale_error_pct": 1.72 + "median_in": 3.27, + "p90_in": 8.53, + "scale_error_pct": 1.56 }, "3-10m": { "pairs": 375000, "failed_pct": 0.0, - "median_in": 4.73, - "p90_in": 13.66, - "scale_error_pct": -0.78 + "median_in": 4.68, + "p90_in": 13.76, + "scale_error_pct": -0.93 } }, "tape_calibration_success_pct": 100.0, @@ -7474,22 +7474,22 @@ "interval_95": { "1-3m": { "median_in": [ - 15.48, - 15.94 + 15.27, + 15.68 ], "p90_in": [ - 22.88, - 23.86 + 22.52, + 23.5 ] }, "3-10m": { "median_in": [ - 24.73, - 25.28 + 24.45, + 24.96 ], "p90_in": [ - 30.74, - 31.51 + 30.41, + 31.24 ] } } @@ -7619,32 +7619,32 @@ "1-3m": { "pairs": 15000, "failed_pct": 0.0, - "median_in": 13.54, - "p90_in": 20.57, - "scale_error_pct": 17.36 + "median_in": 13.5, + "p90_in": 20.85, + "scale_error_pct": 17.48 }, "3-10m": { "pairs": 15000, "failed_pct": 0.0, - "median_in": 23.75, - "p90_in": 30.41, - "scale_error_pct": 16.22 + "median_in": 23.79, + "p90_in": 29.75, + "scale_error_pct": 16.13 } }, "one_known_distance": { "1-3m": { "pairs": 375000, "failed_pct": 0.0, - "median_in": 3.22, - "p90_in": 8.73, - "scale_error_pct": 1.63 + "median_in": 3.25, + "p90_in": 8.78, + "scale_error_pct": 1.51 }, "3-10m": { "pairs": 375000, "failed_pct": 0.0, - "median_in": 5.58, - "p90_in": 15.95, - "scale_error_pct": -0.91 + "median_in": 5.62, + "p90_in": 15.97, + "scale_error_pct": -1.04 } }, "tape_calibration_success_pct": 100.0, @@ -7653,22 +7653,22 @@ "interval_95": { "1-3m": { "median_in": [ - 13.21, - 13.88 + 13.2, + 13.79 ], "p90_in": [ - 20.18, - 20.93 + 20.52, + 21.19 ] }, "3-10m": { "median_in": [ - 23.55, - 23.97 + 23.63, + 24.02 ], "p90_in": [ - 30.23, - 30.64 + 29.53, + 30.05 ] } } @@ -7742,14 +7742,14 @@ "pairs": 30000, "failed_pct": 10.0, "median_in": 12.48, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": 13.61 }, "3-10m": { "pairs": 30000, "failed_pct": 10.0, "median_in": 25.01, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": 13.99 } }, @@ -7758,14 +7758,14 @@ "pairs": 750000, "failed_pct": 10.0, "median_in": 1.27, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": -0.01 }, "3-10m": { "pairs": 750000, "failed_pct": 10.0, "median_in": 2.21, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": 0.12 } }, @@ -7780,7 +7780,7 @@ ], "p90_in": [ 17.69, - Infinity + "failed" ] }, "3-10m": { @@ -7790,7 +7790,7 @@ ], "p90_in": [ 30.35, - Infinity + "failed" ] } } @@ -7800,32 +7800,32 @@ "1-3m": { "pairs": 15000, "failed_pct": 0.0, - "median_in": 16.47, - "p90_in": 62.43, - "scale_error_pct": 22.41 + "median_in": 16.33, + "p90_in": 61.33, + "scale_error_pct": 21.93 }, "3-10m": { "pairs": 15000, "failed_pct": 0.0, - "median_in": 28.08, - "p90_in": 107.92, - "scale_error_pct": 18.96 + "median_in": 27.77, + "p90_in": 107.41, + "scale_error_pct": 18.71 } }, "one_known_distance": { "1-3m": { "pairs": 375000, "failed_pct": 0.0, - "median_in": 3.76, - "p90_in": 10.73, - "scale_error_pct": 2.33 + "median_in": 3.64, + "p90_in": 10.4, + "scale_error_pct": 2.06 }, "3-10m": { "pairs": 375000, "failed_pct": 0.0, - "median_in": 5.39, - "p90_in": 14.8, - "scale_error_pct": -0.45 + "median_in": 5.34, + "p90_in": 14.82, + "scale_error_pct": -0.65 } }, "tape_calibration_success_pct": 100.0, @@ -7834,22 +7834,22 @@ "interval_95": { "1-3m": { "median_in": [ - 3.61, - 44.52 + 3.41, + 44.02 ], "p90_in": [ - 17.05, - 81.47 + 16.85, + 81.07 ] }, "3-10m": { "median_in": [ - 3.51, - 93.2 + 3.29, + 91.49 ], "p90_in": [ - 28.08, - 126.95 + 27.77, + 125.7 ] } } @@ -7979,32 +7979,32 @@ "1-3m": { "pairs": 15000, "failed_pct": 0.0, - "median_in": 4.79, + "median_in": 4.78, "p90_in": 11.19, - "scale_error_pct": 5.84 + "scale_error_pct": 5.88 }, "3-10m": { "pairs": 15000, "failed_pct": 0.0, - "median_in": 7.95, - "p90_in": 14.76, - "scale_error_pct": 4.8 + "median_in": 7.93, + "p90_in": 14.4, + "scale_error_pct": 4.79 } }, "one_known_distance": { "1-3m": { "pairs": 375000, "failed_pct": 0.0, - "median_in": 3.81, - "p90_in": 13.44, - "scale_error_pct": 1.26 + "median_in": 3.73, + "p90_in": 13.15, + "scale_error_pct": 1.25 }, "3-10m": { "pairs": 375000, "failed_pct": 0.0, - "median_in": 5.15, - "p90_in": 20.62, - "scale_error_pct": 0.05 + "median_in": 5.09, + "p90_in": 20.68, + "scale_error_pct": -0.01 } }, "tape_calibration_success_pct": 100.0, @@ -8013,22 +8013,22 @@ "interval_95": { "1-3m": { "median_in": [ - 3.81, - 6.05 + 3.76, + 5.95 ], "p90_in": [ - 8.24, - 17.75 + 8.31, + 16.8 ] }, "3-10m": { "median_in": [ - 6.16, - 9.41 + 6.12, + 9.34 ], "p90_in": [ - 12.12, - 18.69 + 12.14, + 18.13 ] } } @@ -8158,16 +8158,16 @@ "1-3m": { "pairs": 15000, "failed_pct": 0.0, - "median_in": 4.39, - "p90_in": 9.77, - "scale_error_pct": -5.99 + "median_in": 4.51, + "p90_in": 9.74, + "scale_error_pct": -6.18 }, "3-10m": { "pairs": 15000, "failed_pct": 0.0, - "median_in": 12.6, - "p90_in": 19.83, - "scale_error_pct": -8.7 + "median_in": 12.8, + "p90_in": 19.86, + "scale_error_pct": -8.88 } }, "one_known_distance": { @@ -8175,15 +8175,15 @@ "pairs": 375000, "failed_pct": 0.0, "median_in": 3.41, - "p90_in": 9.67, - "scale_error_pct": 1.75 + "p90_in": 9.68, + "scale_error_pct": 1.61 }, "3-10m": { "pairs": 375000, "failed_pct": 0.0, - "median_in": 5.95, - "p90_in": 16.07, - "scale_error_pct": -0.67 + "median_in": 6.0, + "p90_in": 16.05, + "scale_error_pct": -0.82 } }, "tape_calibration_success_pct": 100.0, @@ -8192,22 +8192,22 @@ "interval_95": { "1-3m": { "median_in": [ - 3.27, - 5.69 + 3.36, + 5.76 ], "p90_in": [ - 8.03, - 10.84 + 8.02, + 10.9 ] }, "3-10m": { "median_in": [ - 9.88, - 14.55 + 10.25, + 14.91 ], "p90_in": [ - 17.28, - 21.45 + 17.27, + 21.46 ] } } @@ -8339,32 +8339,32 @@ "1-3m": { "pairs": 15000, "failed_pct": 0.0, - "median_in": 15.81, - "p90_in": 24.16, - "scale_error_pct": 20.71 + "median_in": 15.66, + "p90_in": 23.9, + "scale_error_pct": 20.34 }, "3-10m": { "pairs": 15000, "failed_pct": 0.0, - "median_in": 26.09, - "p90_in": 32.32, - "scale_error_pct": 17.67 + "median_in": 25.78, + "p90_in": 32.21, + "scale_error_pct": 17.45 } }, "one_known_distance": { "1-3m": { "pairs": 375000, "failed_pct": 0.0, - "median_in": 3.33, - "p90_in": 8.64, - "scale_error_pct": 2.21 + "median_in": 3.24, + "p90_in": 8.4, + "scale_error_pct": 1.96 }, "3-10m": { "pairs": 375000, "failed_pct": 0.0, - "median_in": 4.85, - "p90_in": 12.01, - "scale_error_pct": -0.56 + "median_in": 4.81, + "p90_in": 12.08, + "scale_error_pct": -0.74 } }, "tape_calibration_success_pct": 100.0, @@ -8373,22 +8373,22 @@ "interval_95": { "1-3m": { "median_in": [ - 15.77, - 15.84 + 15.62, + 15.68 ], "p90_in": [ - 24.03, - 24.33 + 23.79, + 24.01 ] }, "3-10m": { "median_in": [ - 26.06, - 26.16 + 25.74, + 25.84 ], "p90_in": [ - 32.23, - 32.46 + 32.07, + 32.32 ] } } @@ -8518,32 +8518,32 @@ "1-3m": { "pairs": 15000, "failed_pct": 0.0, - "median_in": 15.77, - "p90_in": 23.28, - "scale_error_pct": 20.4 + "median_in": 15.52, + "p90_in": 23.16, + "scale_error_pct": 20.17 }, "3-10m": { "pairs": 15000, "failed_pct": 0.0, - "median_in": 25.33, - "p90_in": 31.07, - "scale_error_pct": 17.25 + "median_in": 25.16, + "p90_in": 30.88, + "scale_error_pct": 16.97 } }, "one_known_distance": { "1-3m": { "pairs": 375000, "failed_pct": 0.0, - "median_in": 3.35, - "p90_in": 8.53, - "scale_error_pct": 2.02 + "median_in": 3.28, + "p90_in": 8.42, + "scale_error_pct": 1.84 }, "3-10m": { "pairs": 375000, "failed_pct": 0.0, - "median_in": 4.91, - "p90_in": 12.86, - "scale_error_pct": -0.69 + "median_in": 4.88, + "p90_in": 13.0, + "scale_error_pct": -0.84 } }, "tape_calibration_success_pct": 100.0, @@ -8552,22 +8552,22 @@ "interval_95": { "1-3m": { "median_in": [ - 15.73, - 15.82 + 15.48, + 15.56 ], "p90_in": [ - 23.16, - 23.38 + 23.03, + 23.22 ] }, "3-10m": { "median_in": [ - 25.27, - 25.4 + 25.08, + 25.23 ], "p90_in": [ - 30.97, - 31.12 + 30.75, + 30.99 ] } } @@ -8697,32 +8697,32 @@ "1-3m": { "pairs": 15000, "failed_pct": 0.0, - "median_in": 15.13, - "p90_in": 22.26, - "scale_error_pct": 20.04 + "median_in": 14.96, + "p90_in": 22.17, + "scale_error_pct": 19.93 }, "3-10m": { "pairs": 15000, "failed_pct": 0.0, - "median_in": 25.1, - "p90_in": 30.84, - "scale_error_pct": 17.13 + "median_in": 25.0, + "p90_in": 30.27, + "scale_error_pct": 16.94 } }, "one_known_distance": { "1-3m": { "pairs": 375000, "failed_pct": 0.0, - "median_in": 4.78, - "p90_in": 12.65, - "scale_error_pct": 2.65 + "median_in": 4.66, + "p90_in": 12.63, + "scale_error_pct": 2.4 }, "3-10m": { "pairs": 375000, "failed_pct": 0.0, - "median_in": 6.43, - "p90_in": 25.85, - "scale_error_pct": -0.81 + "median_in": 6.39, + "p90_in": 26.23, + "scale_error_pct": -1.02 } }, "tape_calibration_success_pct": 100.0, @@ -8731,22 +8731,22 @@ "interval_95": { "1-3m": { "median_in": [ - 15.09, - 15.16 + 14.92, + 15.0 ], "p90_in": [ - 22.17, - 22.31 + 22.12, + 22.23 ] }, "3-10m": { "median_in": [ - 25.06, - 25.13 + 24.96, + 25.04 ], "p90_in": [ - 30.78, - 30.9 + 30.23, + 30.33 ] } } @@ -8820,14 +8820,14 @@ "pairs": 30000, "failed_pct": 10.0, "median_in": 6.83, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": 3.68 }, "3-10m": { "pairs": 30000, "failed_pct": 10.0, "median_in": 14.93, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": 2.57 } }, @@ -8836,14 +8836,14 @@ "pairs": 750000, "failed_pct": 10.0, "median_in": 1.19, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": -0.0 }, "3-10m": { "pairs": 750000, "failed_pct": 10.0, "median_in": 2.1, - "p90_in": Infinity, + "p90_in": "failed", "scale_error_pct": 0.1 } }, @@ -8858,7 +8858,7 @@ ], "p90_in": [ 12.22, - Infinity + "failed" ] }, "3-10m": { @@ -8868,7 +8868,7 @@ ], "p90_in": [ 19.52, - Infinity + "failed" ] } } @@ -8878,32 +8878,32 @@ "1-3m": { "pairs": 15000, "failed_pct": 0.0, - "median_in": 7.12, - "p90_in": 14.1, - "scale_error_pct": 5.97 + "median_in": 7.08, + "p90_in": 13.75, + "scale_error_pct": 5.73 }, "3-10m": { "pairs": 15000, "failed_pct": 0.0, - "median_in": 13.44, - "p90_in": 20.06, - "scale_error_pct": 4.28 + "median_in": 13.4, + "p90_in": 20.11, + "scale_error_pct": 4.04 } }, "one_known_distance": { "1-3m": { "pairs": 375000, "failed_pct": 0.0, - "median_in": 3.29, - "p90_in": 8.63, - "scale_error_pct": 2.2 + "median_in": 3.19, + "p90_in": 8.39, + "scale_error_pct": 1.96 }, "3-10m": { "pairs": 375000, "failed_pct": 0.0, - "median_in": 4.72, - "p90_in": 11.96, - "scale_error_pct": -0.55 + "median_in": 4.69, + "p90_in": 12.05, + "scale_error_pct": -0.73 } }, "tape_calibration_success_pct": 100.0, @@ -8912,22 +8912,22 @@ "interval_95": { "1-3m": { "median_in": [ - 5.41, - 8.91 + 5.39, + 8.74 ], "p90_in": [ - 11.34, - 16.24 + 11.11, + 15.81 ] }, "3-10m": { "median_in": [ - 9.43, - 15.69 + 9.12, + 15.89 ], "p90_in": [ - 17.48, - 22.36 + 17.19, + 22.22 ] } } @@ -9057,32 +9057,32 @@ "1-3m": { "pairs": 15000, "failed_pct": 0.0, - "median_in": 7.26, - "p90_in": 14.67, - "scale_error_pct": 9.6 + "median_in": 7.14, + "p90_in": 14.3, + "scale_error_pct": 9.43 }, "3-10m": { "pairs": 15000, "failed_pct": 0.0, - "median_in": 9.4, - "p90_in": 16.06, - "scale_error_pct": 6.38 + "median_in": 9.15, + "p90_in": 15.31, + "scale_error_pct": 6.22 } }, "one_known_distance": { "1-3m": { "pairs": 375000, "failed_pct": 0.0, - "median_in": 3.88, - "p90_in": 11.63, - "scale_error_pct": 2.99 + "median_in": 3.81, + "p90_in": 11.33, + "scale_error_pct": 2.82 }, "3-10m": { "pairs": 375000, "failed_pct": 0.0, - "median_in": 5.42, - "p90_in": 15.6, - "scale_error_pct": -0.13 + "median_in": 5.36, + "p90_in": 15.53, + "scale_error_pct": -0.32 } }, "tape_calibration_success_pct": 100.0, @@ -9091,22 +9091,22 @@ "interval_95": { "1-3m": { "median_in": [ - 6.31, - 8.71 + 6.22, + 8.59 ], "p90_in": [ - 11.6, - 18.25 + 11.65, + 17.47 ] }, "3-10m": { "median_in": [ - 7.94, - 11.4 + 7.68, + 11.06 ], "p90_in": [ - 12.18, - 19.31 + 12.01, + 18.71 ] } } @@ -9236,32 +9236,32 @@ "1-3m": { "pairs": 15000, "failed_pct": 0.0, - "median_in": 6.64, - "p90_in": 11.81, - "scale_error_pct": 8.62 + "median_in": 6.54, + "p90_in": 11.8, + "scale_error_pct": 8.52 }, "3-10m": { "pairs": 15000, "failed_pct": 0.0, - "median_in": 9.62, - "p90_in": 13.44, - "scale_error_pct": 6.56 + "median_in": 9.37, + "p90_in": 13.17, + "scale_error_pct": 6.38 } }, "one_known_distance": { "1-3m": { "pairs": 375000, "failed_pct": 0.0, - "median_in": 3.37, - "p90_in": 9.2, - "scale_error_pct": 2.11 + "median_in": 3.32, + "p90_in": 9.11, + "scale_error_pct": 1.93 }, "3-10m": { "pairs": 375000, "failed_pct": 0.0, - "median_in": 4.96, - "p90_in": 15.0, - "scale_error_pct": -0.84 + "median_in": 4.99, + "p90_in": 15.11, + "scale_error_pct": -1.03 } }, "tape_calibration_success_pct": 100.0, @@ -9270,22 +9270,22 @@ "interval_95": { "1-3m": { "median_in": [ - 6.11, - 7.12 + 6.07, + 6.99 ], "p90_in": [ - 10.79, - 13.26 + 10.91, + 12.98 ] }, "3-10m": { "median_in": [ - 8.85, - 10.36 + 8.62, + 10.13 ], "p90_in": [ - 12.44, - 14.09 + 12.26, + 13.93 ] } } @@ -9360,58 +9360,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 19.64, - "median_in": 14.53, - "p90_in": Infinity, - "scale_error_pct": 12.88 + "pairs": 21000, + "failed_pct": 13.28, + "median_in": 13.4, + "p90_in": "failed", + "scale_error_pct": 12.79 }, "3-10m": { - "pairs": 24000, - "failed_pct": 24.37, - "median_in": 37.01, - "p90_in": Infinity, - "scale_error_pct": 12.94 + "pairs": 21000, + "failed_pct": 17.5, + "median_in": 33.94, + "p90_in": "failed", + "scale_error_pct": 13.03 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 32.21, - "median_in": 7.84, - "p90_in": Infinity, - "scale_error_pct": -0.05 + "pairs": 525000, + "failed_pct": 24.37, + "median_in": 6.2, + "p90_in": "failed", + "scale_error_pct": -0.16 }, "3-10m": { - "pairs": 600000, - "failed_pct": 35.82, - "median_in": 22.54, - "p90_in": Infinity, - "scale_error_pct": -0.61 + "pairs": 525000, + "failed_pct": 28.07, + "median_in": 17.18, + "p90_in": "failed", + "scale_error_pct": -0.68 } }, - "tape_calibration_success_pct": 80.5, - "groups": 8, + "tape_calibration_success_pct": 86.9, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 10.56, - 20.58 + 9.65, + 18.11 ], "p90_in": [ - Infinity, - Infinity + 37.55, + "failed" ] }, "3-10m": { "median_in": [ - 28.49, - 48.36 + 26.48, + 40.76 ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -9419,58 +9419,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 14.34, - "median_in": 13.91, - "p90_in": Infinity, - "scale_error_pct": 12.57 + "pairs": 21000, + "failed_pct": 9.0, + "median_in": 13.1, + "p90_in": 36.8, + "scale_error_pct": 12.74 }, "3-10m": { - "pairs": 24000, - "failed_pct": 19.75, - "median_in": 32.6, - "p90_in": Infinity, - "scale_error_pct": 13.25 + "pairs": 21000, + "failed_pct": 13.37, + "median_in": 30.6, + "p90_in": "failed", + "scale_error_pct": 13.53 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 28.59, - "median_in": 6.86, - "p90_in": Infinity, - "scale_error_pct": 0.01 + "pairs": 525000, + "failed_pct": 20.93, + "median_in": 5.45, + "p90_in": "failed", + "scale_error_pct": -0.13 }, "3-10m": { - "pairs": 600000, - "failed_pct": 32.34, - "median_in": 19.63, - "p90_in": Infinity, - "scale_error_pct": -0.34 + "pairs": 525000, + "failed_pct": 24.66, + "median_in": 14.93, + "p90_in": "failed", + "scale_error_pct": -0.41 } }, - "tape_calibration_success_pct": 80.5, - "groups": 8, + "tape_calibration_success_pct": 86.9, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 8.55, - 19.31 + 7.9, + 17.73 ], "p90_in": [ - 28.81, - Infinity + 24.85, + "failed" ] }, "3-10m": { "median_in": [ - 23.47, - 41.11 + 22.39, + 37.24 ], "p90_in": [ - Infinity, - Infinity + 79.3, + "failed" ] } } @@ -9478,58 +9478,58 @@ "edges": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 79.03, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 15.19 + "pairs": 21000, + "failed_pct": 79.58, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": -2.35 }, "3-10m": { - "pairs": 23816, - "failed_pct": 83.78, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 9.98 + "pairs": 20816, + "failed_pct": 84.79, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 2.56 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 83.14, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 3.66 + "pairs": 525000, + "failed_pct": 82.0, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 2.71 }, "3-10m": { - "pairs": 595400, - "failed_pct": 87.41, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 1.65 + "pairs": 520400, + "failed_pct": 86.81, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 0.73 } }, - "tape_calibration_success_pct": 80.5, - "groups": 8, + "tape_calibration_success_pct": 86.9, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -9539,58 +9539,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 13.44, - "median_in": 15.99, - "p90_in": Infinity, - "scale_error_pct": 16.08 + "pairs": 21000, + "failed_pct": 9.41, + "median_in": 15.81, + "p90_in": 41.93, + "scale_error_pct": 16.97 }, "3-10m": { - "pairs": 24000, - "failed_pct": 17.85, - "median_in": 40.09, - "p90_in": Infinity, - "scale_error_pct": 15.73 + "pairs": 21000, + "failed_pct": 13.44, + "median_in": 38.86, + "p90_in": "failed", + "scale_error_pct": 17.02 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 22.35, - "median_in": 6.38, - "p90_in": Infinity, - "scale_error_pct": -0.05 + "pairs": 525000, + "failed_pct": 16.6, + "median_in": 5.23, + "p90_in": "failed", + "scale_error_pct": -0.2 }, "3-10m": { - "pairs": 600000, - "failed_pct": 26.08, - "median_in": 16.88, - "p90_in": Infinity, - "scale_error_pct": -0.59 + "pairs": 525000, + "failed_pct": 20.22, + "median_in": 13.66, + "p90_in": "failed", + "scale_error_pct": -0.65 } }, - "tape_calibration_success_pct": 88.5, - "groups": 8, + "tape_calibration_success_pct": 92.0, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 11.84, - 19.45 + 11.81, + 19.39 ], "p90_in": [ - 34.31, - Infinity + 30.61, + "failed" ] }, "3-10m": { "median_in": [ - 29.75, - 47.75 + 28.73, + 46.66 ], "p90_in": [ - 205.98, - Infinity + 81.41, + "failed" ] } } @@ -9598,58 +9598,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 8.7, - "median_in": 14.96, - "p90_in": 39.79, - "scale_error_pct": 15.39 + "pairs": 21000, + "failed_pct": 5.31, + "median_in": 15.23, + "p90_in": 32.45, + "scale_error_pct": 16.71 }, "3-10m": { - "pairs": 24000, - "failed_pct": 11.15, - "median_in": 33.21, - "p90_in": Infinity, - "scale_error_pct": 14.41 + "pairs": 21000, + "failed_pct": 7.3, + "median_in": 32.04, + "p90_in": 81.8, + "scale_error_pct": 15.18 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 18.34, - "median_in": 5.29, - "p90_in": Infinity, - "scale_error_pct": 0.25 + "pairs": 525000, + "failed_pct": 12.83, + "median_in": 4.32, + "p90_in": "failed", + "scale_error_pct": 0.11 }, "3-10m": { - "pairs": 600000, - "failed_pct": 20.25, - "median_in": 15.24, - "p90_in": Infinity, - "scale_error_pct": -0.36 + "pairs": 525000, + "failed_pct": 14.57, + "median_in": 12.11, + "p90_in": "failed", + "scale_error_pct": -0.39 } }, - "tape_calibration_success_pct": 88.5, - "groups": 8, + "tape_calibration_success_pct": 92.0, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 10.02, - 19.59 + 10.58, + 19.83 ], "p90_in": [ - 25.96, - Infinity + 23.67, + 37.56 ] }, "3-10m": { "median_in": [ - 22.88, - 40.26 + 21.62, + 39.05 ], "p90_in": [ - 62.47, - Infinity + 59.43, + 151.68 ] } } @@ -9657,58 +9657,58 @@ "edges": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 67.18, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 23.26 + "pairs": 21000, + "failed_pct": 68.27, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 18.39 }, "3-10m": { - "pairs": 23816, - "failed_pct": 76.37, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 12.91 + "pairs": 20816, + "failed_pct": 76.89, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 8.65 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 71.05, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 3.9 + "pairs": 525000, + "failed_pct": 70.61, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 2.31 }, "3-10m": { - "pairs": 595400, - "failed_pct": 79.04, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 0.56 + "pairs": 520400, + "failed_pct": 78.53, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": -0.14 } }, - "tape_calibration_success_pct": 88.5, - "groups": 8, + "tape_calibration_success_pct": 92.0, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -9720,58 +9720,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 19.58, - "median_in": 5.66, - "p90_in": Infinity, - "scale_error_pct": 3.09 + "pairs": 21000, + "failed_pct": 13.26, + "median_in": 4.92, + "p90_in": "failed", + "scale_error_pct": 3.03 }, "3-10m": { - "pairs": 24000, - "failed_pct": 24.25, - "median_in": 17.27, - "p90_in": Infinity, - "scale_error_pct": 3.12 + "pairs": 21000, + "failed_pct": 17.25, + "median_in": 15.11, + "p90_in": "failed", + "scale_error_pct": 3.21 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 32.42, - "median_in": 7.47, - "p90_in": Infinity, - "scale_error_pct": 0.03 + "pairs": 525000, + "failed_pct": 24.42, + "median_in": 5.92, + "p90_in": "failed", + "scale_error_pct": -0.1 }, "3-10m": { - "pairs": 600000, - "failed_pct": 35.84, - "median_in": 20.42, - "p90_in": Infinity, - "scale_error_pct": -0.44 + "pairs": 525000, + "failed_pct": 27.9, + "median_in": 15.7, + "p90_in": "failed", + "scale_error_pct": -0.54 } }, - "tape_calibration_success_pct": 80.0, - "groups": 8, + "tape_calibration_success_pct": 86.9, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 3.57, - 11.45 + 3.27, + 9.77 ], "p90_in": [ - Infinity, - Infinity + 22.96, + "failed" ] }, "3-10m": { "median_in": [ - 10.79, - 29.94 + 10.16, + 24.18 ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -9779,58 +9779,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 14.68, - "median_in": 4.18, - "p90_in": Infinity, - "scale_error_pct": 3.18 + "pairs": 21000, + "failed_pct": 8.85, + "median_in": 3.76, + "p90_in": 25.3, + "scale_error_pct": 3.16 }, "3-10m": { - "pairs": 24000, - "failed_pct": 19.57, - "median_in": 13.0, - "p90_in": Infinity, - "scale_error_pct": 3.28 + "pairs": 21000, + "failed_pct": 13.1, + "median_in": 11.55, + "p90_in": "failed", + "scale_error_pct": 3.41 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 28.87, - "median_in": 6.36, - "p90_in": Infinity, - "scale_error_pct": 0.16 + "pairs": 525000, + "failed_pct": 20.86, + "median_in": 5.04, + "p90_in": "failed", + "scale_error_pct": 0.01 }, "3-10m": { - "pairs": 600000, - "failed_pct": 32.35, - "median_in": 17.13, - "p90_in": Infinity, - "scale_error_pct": 0.13 + "pairs": 525000, + "failed_pct": 24.54, + "median_in": 13.32, + "p90_in": "failed", + "scale_error_pct": 0.02 } }, - "tape_calibration_success_pct": 80.0, - "groups": 8, + "tape_calibration_success_pct": 86.9, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 2.83, - 8.53 + 2.65, + 7.24 ], "p90_in": [ - 12.64, - Infinity + 11.53, + "failed" ] }, "3-10m": { "median_in": [ - 7.83, - 22.75 + 6.73, + 19.21 ], "p90_in": [ - Infinity, - Infinity + 56.6, + "failed" ] } } @@ -9838,58 +9838,58 @@ "edges": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 77.53, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 5.39 + "pairs": 21000, + "failed_pct": 78.44, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 4.86 }, "3-10m": { - "pairs": 23816, - "failed_pct": 83.28, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 5.64 + "pairs": 20816, + "failed_pct": 84.39, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 5.23 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 82.6, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 3.78 + "pairs": 525000, + "failed_pct": 81.44, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 2.89 }, "3-10m": { - "pairs": 595400, - "failed_pct": 87.31, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 1.67 + "pairs": 520400, + "failed_pct": 86.61, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 0.96 } }, - "tape_calibration_success_pct": 80.0, - "groups": 8, + "tape_calibration_success_pct": 86.9, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -9899,58 +9899,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 13.35, - "median_in": 5.51, - "p90_in": Infinity, - "scale_error_pct": 3.56 + "pairs": 21000, + "failed_pct": 9.56, + "median_in": 4.74, + "p90_in": 23.5, + "scale_error_pct": 4.06 }, "3-10m": { - "pairs": 24000, - "failed_pct": 17.91, - "median_in": 13.99, - "p90_in": Infinity, - "scale_error_pct": 2.71 + "pairs": 21000, + "failed_pct": 13.52, + "median_in": 12.05, + "p90_in": "failed", + "scale_error_pct": 3.21 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 24.02, - "median_in": 6.19, - "p90_in": Infinity, - "scale_error_pct": -0.21 + "pairs": 525000, + "failed_pct": 18.32, + "median_in": 5.03, + "p90_in": "failed", + "scale_error_pct": -0.3 }, "3-10m": { - "pairs": 600000, - "failed_pct": 27.77, - "median_in": 16.11, - "p90_in": Infinity, - "scale_error_pct": -0.81 + "pairs": 525000, + "failed_pct": 21.86, + "median_in": 12.92, + "p90_in": "failed", + "scale_error_pct": -0.76 } }, - "tape_calibration_success_pct": 86.5, - "groups": 8, + "tape_calibration_success_pct": 90.3, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 3.76, - 8.22 + 3.48, + 6.22 ], "p90_in": [ - 14.43, - Infinity + 13.12, + "failed" ] }, "3-10m": { "median_in": [ - 9.77, - 22.4 + 8.72, + 15.0 ], "p90_in": [ - Infinity, - Infinity + 33.36, + "failed" ] } } @@ -9958,58 +9958,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 8.65, - "median_in": 4.48, - "p90_in": 23.59, - "scale_error_pct": 3.78 + "pairs": 21000, + "failed_pct": 5.32, + "median_in": 3.92, + "p90_in": 11.96, + "scale_error_pct": 4.24 }, "3-10m": { - "pairs": 24000, - "failed_pct": 11.3, - "median_in": 12.43, - "p90_in": Infinity, - "scale_error_pct": 3.3 + "pairs": 21000, + "failed_pct": 7.34, + "median_in": 11.06, + "p90_in": 32.11, + "scale_error_pct": 3.6 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 20.1, - "median_in": 4.81, - "p90_in": Infinity, - "scale_error_pct": 0.17 + "pairs": 525000, + "failed_pct": 14.52, + "median_in": 3.88, + "p90_in": "failed", + "scale_error_pct": 0.1 }, "3-10m": { - "pairs": 600000, - "failed_pct": 22.1, - "median_in": 13.33, - "p90_in": Infinity, - "scale_error_pct": -0.3 + "pairs": 525000, + "failed_pct": 16.28, + "median_in": 10.56, + "p90_in": "failed", + "scale_error_pct": -0.25 } }, - "tape_calibration_success_pct": 86.5, - "groups": 8, + "tape_calibration_success_pct": 90.3, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 3.06, - 7.31 + 2.78, + 5.8 ], "p90_in": [ - 9.45, - Infinity + 8.03, + 15.65 ] }, "3-10m": { "median_in": [ - 7.59, - 18.61 + 6.39, + 13.53 ], "p90_in": [ - 23.18, - Infinity + 22.04, + 50.26 ] } } @@ -10017,58 +10017,58 @@ "edges": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 66.58, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 6.2 + "pairs": 21000, + "failed_pct": 67.75, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 5.32 }, "3-10m": { - "pairs": 23816, - "failed_pct": 74.87, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 4.41 + "pairs": 20816, + "failed_pct": 75.58, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 4.02 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 71.41, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 4.5 + "pairs": 525000, + "failed_pct": 70.89, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 3.23 }, "3-10m": { - "pairs": 595400, - "failed_pct": 78.31, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 1.5 + "pairs": 520400, + "failed_pct": 77.78, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 0.9 } }, - "tape_calibration_success_pct": 86.5, - "groups": 8, + "tape_calibration_success_pct": 90.3, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 504.45, - Infinity + 18.14, + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -10080,58 +10080,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 19.59, - "median_in": 5.92, - "p90_in": Infinity, - "scale_error_pct": -0.74 + "pairs": 21000, + "failed_pct": 13.21, + "median_in": 5.22, + "p90_in": "failed", + "scale_error_pct": -0.71 }, "3-10m": { - "pairs": 24000, - "failed_pct": 24.37, - "median_in": 17.88, - "p90_in": Infinity, - "scale_error_pct": -0.77 + "pairs": 21000, + "failed_pct": 17.43, + "median_in": 14.18, + "p90_in": "failed", + "scale_error_pct": -0.63 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 32.87, - "median_in": 7.74, - "p90_in": Infinity, - "scale_error_pct": 0.01 + "pairs": 525000, + "failed_pct": 24.92, + "median_in": 6.07, + "p90_in": "failed", + "scale_error_pct": -0.12 }, "3-10m": { - "pairs": 600000, - "failed_pct": 36.43, - "median_in": 21.44, - "p90_in": Infinity, - "scale_error_pct": -0.41 + "pairs": 525000, + "failed_pct": 28.58, + "median_in": 16.5, + "p90_in": "failed", + "scale_error_pct": -0.51 } }, - "tape_calibration_success_pct": 79.5, - "groups": 8, + "tape_calibration_success_pct": 86.3, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 3.72, - 11.92 + 3.37, + 9.85 ], "p90_in": [ - Infinity, - Infinity + 29.17, + "failed" ] }, "3-10m": { "median_in": [ - 8.73, - 35.0 + 7.74, + 28.19 ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -10139,58 +10139,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 14.67, - "median_in": 4.53, - "p90_in": Infinity, - "scale_error_pct": -1.05 + "pairs": 21000, + "failed_pct": 8.73, + "median_in": 4.15, + "p90_in": 23.82, + "scale_error_pct": -1.06 }, "3-10m": { - "pairs": 24000, - "failed_pct": 19.78, - "median_in": 13.68, - "p90_in": Infinity, - "scale_error_pct": -0.97 + "pairs": 21000, + "failed_pct": 13.2, + "median_in": 11.2, + "p90_in": "failed", + "scale_error_pct": -0.68 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 29.31, - "median_in": 6.46, - "p90_in": Infinity, - "scale_error_pct": 0.03 + "pairs": 525000, + "failed_pct": 21.28, + "median_in": 5.1, + "p90_in": "failed", + "scale_error_pct": -0.11 }, "3-10m": { - "pairs": 600000, - "failed_pct": 32.98, - "median_in": 17.95, - "p90_in": Infinity, - "scale_error_pct": -0.2 + "pairs": 525000, + "failed_pct": 25.17, + "median_in": 13.78, + "p90_in": "failed", + "scale_error_pct": -0.3 } }, - "tape_calibration_success_pct": 79.5, - "groups": 8, + "tape_calibration_success_pct": 86.3, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 3.06, - 8.04 + 2.83, + 6.8 ], "p90_in": [ - 13.61, - Infinity + 12.01, + "failed" ] }, "3-10m": { "median_in": [ - 6.9, - 28.3 + 6.17, + 23.89 ], "p90_in": [ - Infinity, - Infinity + 58.39, + "failed" ] } } @@ -10198,58 +10198,58 @@ "edges": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 77.14, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 3.89 + "pairs": 21000, + "failed_pct": 78.04, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 3.59 }, "3-10m": { - "pairs": 23816, - "failed_pct": 83.2, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 4.14 + "pairs": 20816, + "failed_pct": 84.28, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 3.78 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 82.63, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 3.35 + "pairs": 525000, + "failed_pct": 81.48, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 2.68 }, "3-10m": { - "pairs": 595400, - "failed_pct": 87.46, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 1.75 + "pairs": 520400, + "failed_pct": 86.77, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 0.98 } }, - "tape_calibration_success_pct": 79.5, - "groups": 8, + "tape_calibration_success_pct": 86.3, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -10259,58 +10259,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 13.14, - "median_in": 6.51, - "p90_in": Infinity, - "scale_error_pct": -2.97 + "pairs": 21000, + "failed_pct": 9.72, + "median_in": 5.47, + "p90_in": 27.38, + "scale_error_pct": -2.33 }, "3-10m": { - "pairs": 24000, - "failed_pct": 17.66, - "median_in": 18.91, - "p90_in": Infinity, - "scale_error_pct": -4.41 + "pairs": 21000, + "failed_pct": 13.54, + "median_in": 16.08, + "p90_in": "failed", + "scale_error_pct": -3.65 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 23.45, - "median_in": 5.97, - "p90_in": Infinity, - "scale_error_pct": -0.21 + "pairs": 525000, + "failed_pct": 17.91, + "median_in": 4.86, + "p90_in": "failed", + "scale_error_pct": -0.28 }, "3-10m": { - "pairs": 600000, - "failed_pct": 27.16, - "median_in": 15.49, - "p90_in": Infinity, - "scale_error_pct": -0.83 + "pairs": 525000, + "failed_pct": 21.34, + "median_in": 12.41, + "p90_in": "failed", + "scale_error_pct": -0.77 } }, - "tape_calibration_success_pct": 87.0, - "groups": 8, + "tape_calibration_success_pct": 90.9, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 3.18, - 10.83 + 2.54, + 9.27 ], "p90_in": [ - 17.17, - Infinity + 14.66, + "failed" ] }, "3-10m": { "median_in": [ - 9.43, - 29.8 + 7.12, + 22.49 ], "p90_in": [ - Infinity, - Infinity + 45.77, + "failed" ] } } @@ -10318,58 +10318,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 8.27, - "median_in": 5.13, - "p90_in": 24.9, - "scale_error_pct": -2.48 + "pairs": 21000, + "failed_pct": 5.3, + "median_in": 4.24, + "p90_in": 17.53, + "scale_error_pct": -1.54 }, "3-10m": { - "pairs": 24000, - "failed_pct": 10.98, - "median_in": 14.49, - "p90_in": Infinity, - "scale_error_pct": -4.15 + "pairs": 21000, + "failed_pct": 7.32, + "median_in": 12.78, + "p90_in": 42.88, + "scale_error_pct": -3.12 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 19.38, - "median_in": 4.77, - "p90_in": Infinity, - "scale_error_pct": 0.13 + "pairs": 525000, + "failed_pct": 13.94, + "median_in": 3.88, + "p90_in": "failed", + "scale_error_pct": 0.09 }, "3-10m": { - "pairs": 600000, - "failed_pct": 21.4, - "median_in": 12.93, - "p90_in": Infinity, - "scale_error_pct": -0.32 + "pairs": 525000, + "failed_pct": 15.7, + "median_in": 10.05, + "p90_in": "failed", + "scale_error_pct": -0.26 } }, - "tape_calibration_success_pct": 87.0, - "groups": 8, + "tape_calibration_success_pct": 90.9, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 1.77, - 9.29 + 1.45, + 7.82 ], "p90_in": [ - 10.17, - Infinity + 9.47, + 23.48 ] }, "3-10m": { "median_in": [ - 6.71, - 25.85 + 5.36, + 19.21 ], "p90_in": [ - 26.37, - Infinity + 23.07, + 53.49 ] } } @@ -10377,58 +10377,58 @@ "edges": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 67.24, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 0.59 + "pairs": 21000, + "failed_pct": 68.53, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": -2.41 }, "3-10m": { - "pairs": 23816, - "failed_pct": 74.99, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": -1.33 + "pairs": 20816, + "failed_pct": 75.82, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": -2.64 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 71.68, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 4.22 + "pairs": 525000, + "failed_pct": 71.22, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 2.89 }, "3-10m": { - "pairs": 595400, - "failed_pct": 78.22, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 1.1 + "pairs": 520400, + "failed_pct": 77.74, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 0.62 } }, - "tape_calibration_success_pct": 87.0, - "groups": 8, + "tape_calibration_success_pct": 90.9, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -10440,58 +10440,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 19.6, - "median_in": 4.99, - "p90_in": Infinity, - "scale_error_pct": 1.32 + "pairs": 21000, + "failed_pct": 13.24, + "median_in": 4.18, + "p90_in": "failed", + "scale_error_pct": 1.19 }, "3-10m": { - "pairs": 24000, - "failed_pct": 24.27, - "median_in": 16.51, - "p90_in": Infinity, + "pairs": 21000, + "failed_pct": 17.28, + "median_in": 13.06, + "p90_in": "failed", "scale_error_pct": 1.13 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 31.84, - "median_in": 7.34, - "p90_in": Infinity, - "scale_error_pct": -0.01 + "pairs": 525000, + "failed_pct": 23.94, + "median_in": 5.8, + "p90_in": "failed", + "scale_error_pct": -0.11 }, "3-10m": { - "pairs": 600000, - "failed_pct": 35.35, - "median_in": 20.15, - "p90_in": Infinity, - "scale_error_pct": -0.46 + "pairs": 525000, + "failed_pct": 27.49, + "median_in": 15.46, + "p90_in": "failed", + "scale_error_pct": -0.54 } }, - "tape_calibration_success_pct": 81.0, - "groups": 8, + "tape_calibration_success_pct": 87.4, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 3.04, - 11.69 + 2.85, + 9.59 ], "p90_in": [ - Infinity, - Infinity + 22.46, + "failed" ] }, "3-10m": { "median_in": [ - 7.58, - 31.09 + 6.98, + 24.56 ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -10499,58 +10499,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 14.72, - "median_in": 3.7, - "p90_in": Infinity, - "scale_error_pct": 1.54 + "pairs": 21000, + "failed_pct": 8.82, + "median_in": 3.2, + "p90_in": 25.77, + "scale_error_pct": 1.39 }, "3-10m": { - "pairs": 24000, - "failed_pct": 19.6, - "median_in": 11.04, - "p90_in": Infinity, - "scale_error_pct": 1.6 + "pairs": 21000, + "failed_pct": 13.02, + "median_in": 8.96, + "p90_in": "failed", + "scale_error_pct": 1.62 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 28.21, - "median_in": 6.23, - "p90_in": Infinity, - "scale_error_pct": 0.08 + "pairs": 525000, + "failed_pct": 20.31, + "median_in": 4.95, + "p90_in": "failed", + "scale_error_pct": -0.04 }, "3-10m": { - "pairs": 600000, - "failed_pct": 31.76, - "median_in": 16.75, - "p90_in": Infinity, - "scale_error_pct": 0.06 + "pairs": 525000, + "failed_pct": 23.97, + "median_in": 12.93, + "p90_in": "failed", + "scale_error_pct": -0.03 } }, - "tape_calibration_success_pct": 81.0, - "groups": 8, + "tape_calibration_success_pct": 87.4, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 2.34, - 8.3 + 2.14, + 6.96 ], "p90_in": [ - 11.68, - Infinity + 11.03, + "failed" ] }, "3-10m": { "median_in": [ - 5.56, - 22.45 + 4.95, + 18.59 ], "p90_in": [ - Infinity, - Infinity + 58.84, + "failed" ] } } @@ -10558,58 +10558,58 @@ "edges": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 77.61, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 3.06 + "pairs": 21000, + "failed_pct": 78.5, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 2.5 }, "3-10m": { - "pairs": 23816, - "failed_pct": 83.42, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 3.3 + "pairs": 20816, + "failed_pct": 84.45, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 2.76 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 82.4, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 3.65 + "pairs": 525000, + "failed_pct": 81.36, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 2.79 }, "3-10m": { - "pairs": 595400, - "failed_pct": 87.18, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 1.46 + "pairs": 520400, + "failed_pct": 86.56, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 0.92 } }, - "tape_calibration_success_pct": 81.0, - "groups": 8, + "tape_calibration_success_pct": 87.4, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -10619,58 +10619,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 13.25, - "median_in": 5.14, - "p90_in": Infinity, - "scale_error_pct": 1.96 + "pairs": 21000, + "failed_pct": 9.52, + "median_in": 4.3, + "p90_in": 21.89, + "scale_error_pct": 2.47 }, "3-10m": { - "pairs": 24000, - "failed_pct": 17.82, - "median_in": 13.3, - "p90_in": Infinity, - "scale_error_pct": 1.26 + "pairs": 21000, + "failed_pct": 13.52, + "median_in": 11.3, + "p90_in": "failed", + "scale_error_pct": 1.78 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 23.95, - "median_in": 6.16, - "p90_in": Infinity, - "scale_error_pct": -0.16 + "pairs": 525000, + "failed_pct": 18.28, + "median_in": 5.02, + "p90_in": "failed", + "scale_error_pct": -0.28 }, "3-10m": { - "pairs": 600000, - "failed_pct": 27.72, - "median_in": 16.01, - "p90_in": Infinity, - "scale_error_pct": -0.79 + "pairs": 525000, + "failed_pct": 21.86, + "median_in": 12.96, + "p90_in": "failed", + "scale_error_pct": -0.78 } }, - "tape_calibration_success_pct": 86.5, - "groups": 8, + "tape_calibration_success_pct": 90.3, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 3.0, - 8.06 + 2.68, + 6.08 ], "p90_in": [ - 14.82, - Infinity + 13.25, + "failed" ] }, "3-10m": { "median_in": [ - 7.84, - 22.87 + 6.78, + 14.72 ], "p90_in": [ - Infinity, - Infinity + 31.12, + "failed" ] } } @@ -10678,58 +10678,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 8.61, - "median_in": 3.77, - "p90_in": 22.69, - "scale_error_pct": 2.21 + "pairs": 21000, + "failed_pct": 5.34, + "median_in": 3.15, + "p90_in": 11.66, + "scale_error_pct": 2.64 }, "3-10m": { - "pairs": 24000, - "failed_pct": 11.25, - "median_in": 11.33, - "p90_in": Infinity, - "scale_error_pct": 1.85 + "pairs": 21000, + "failed_pct": 7.33, + "median_in": 9.74, + "p90_in": 30.92, + "scale_error_pct": 2.25 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 20.08, - "median_in": 4.84, - "p90_in": Infinity, - "scale_error_pct": 0.25 + "pairs": 525000, + "failed_pct": 14.54, + "median_in": 3.92, + "p90_in": "failed", + "scale_error_pct": 0.14 }, "3-10m": { - "pairs": 600000, - "failed_pct": 22.06, - "median_in": 13.37, - "p90_in": Infinity, - "scale_error_pct": -0.28 + "pairs": 525000, + "failed_pct": 16.27, + "median_in": 10.68, + "p90_in": "failed", + "scale_error_pct": -0.26 } }, - "tape_calibration_success_pct": 86.5, - "groups": 8, + "tape_calibration_success_pct": 90.3, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 2.14, - 7.16 + 1.97, + 5.52 ], "p90_in": [ - 8.95, - Infinity + 7.34, + 15.58 ] }, "3-10m": { "median_in": [ - 5.55, - 18.56 + 4.37, + 13.28 ], "p90_in": [ - 21.63, - Infinity + 20.04, + 49.55 ] } } @@ -10737,58 +10737,58 @@ "edges": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 66.74, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 4.73 + "pairs": 21000, + "failed_pct": 67.9, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 3.67 }, "3-10m": { - "pairs": 23816, - "failed_pct": 74.51, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 2.85 + "pairs": 20816, + "failed_pct": 75.31, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 2.44 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 71.55, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 4.77 + "pairs": 525000, + "failed_pct": 71.03, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 3.53 }, "3-10m": { - "pairs": 595400, - "failed_pct": 78.02, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 1.71 + "pairs": 520400, + "failed_pct": 77.54, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 1.06 } }, - "tape_calibration_success_pct": 86.5, - "groups": 8, + "tape_calibration_success_pct": 90.3, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 561.65, - Infinity + 17.6, + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -10800,58 +10800,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.06, - "median_in": 6.26, - "p90_in": 16.61, - "scale_error_pct": 2.19 + "pairs": 21000, + "failed_pct": 0.03, + "median_in": 5.6, + "p90_in": 14.03, + "scale_error_pct": 3.04 }, "3-10m": { - "pairs": 24000, - "failed_pct": 0.11, - "median_in": 17.46, - "p90_in": 42.18, - "scale_error_pct": 3.38 + "pairs": 21000, + "failed_pct": 0.1, + "median_in": 16.04, + "p90_in": 30.88, + "scale_error_pct": 4.4 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 0.06, - "median_in": 3.16, - "p90_in": 13.99, - "scale_error_pct": -0.16 + "pairs": 525000, + "failed_pct": 0.03, + "median_in": 2.75, + "p90_in": 10.07, + "scale_error_pct": -0.17 }, "3-10m": { - "pairs": 600000, - "failed_pct": 0.11, - "median_in": 7.0, - "p90_in": 37.37, - "scale_error_pct": -0.26 + "pairs": 525000, + "failed_pct": 0.1, + "median_in": 6.0, + "p90_in": 22.79, + "scale_error_pct": -0.08 } }, "tape_calibration_success_pct": 100.0, - "groups": 8, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 3.96, - 9.28 + 3.62, + 8.26 ], "p90_in": [ - 11.29, - 21.92 + 10.59, + 16.82 ] }, "3-10m": { "median_in": [ - 12.93, - 24.03 + 12.11, + 21.27 ], "p90_in": [ - 27.11, - 74.76 + 25.45, + 33.52 ] } } @@ -10859,58 +10859,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.08, - "median_in": 6.08, - "p90_in": 16.25, - "scale_error_pct": 1.74 + "pairs": 21000, + "failed_pct": 0.07, + "median_in": 5.38, + "p90_in": 13.56, + "scale_error_pct": 3.15 }, "3-10m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.16, - "median_in": 17.57, - "p90_in": 45.45, - "scale_error_pct": 4.73 + "median_in": 16.19, + "p90_in": 30.75, + "scale_error_pct": 5.66 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 0.08, - "median_in": 2.79, - "p90_in": 13.44, - "scale_error_pct": -0.3 + "pairs": 525000, + "failed_pct": 0.07, + "median_in": 2.42, + "p90_in": 9.45, + "scale_error_pct": -0.32 }, "3-10m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.16, - "median_in": 6.97, - "p90_in": 38.54, - "scale_error_pct": -0.03 + "median_in": 5.96, + "p90_in": 23.23, + "scale_error_pct": 0.2 } }, "tape_calibration_success_pct": 100.0, - "groups": 8, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 3.45, - 9.0 + 2.89, + 8.18 ], "p90_in": [ - 10.65, - 21.24 + 9.45, + 16.7 ] }, "3-10m": { "median_in": [ - 14.21, - 25.0 + 13.51, + 22.1 ], "p90_in": [ - 25.45, - 76.14 + 24.12, + 34.36 ] } } @@ -10918,58 +10918,58 @@ "edges": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.31, - "median_in": 7.64, - "p90_in": 30.56, - "scale_error_pct": 5.59 + "pairs": 21000, + "failed_pct": 0.32, + "median_in": 6.98, + "p90_in": 18.98, + "scale_error_pct": 4.59 }, "3-10m": { - "pairs": 23816, - "failed_pct": 0.17, - "median_in": 15.33, - "p90_in": 44.69, - "scale_error_pct": 4.31 + "pairs": 20816, + "failed_pct": 0.19, + "median_in": 13.98, + "p90_in": 27.74, + "scale_error_pct": 4.84 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 0.31, - "median_in": 4.33, - "p90_in": 32.65, - "scale_error_pct": 0.63 + "pairs": 525000, + "failed_pct": 0.32, + "median_in": 3.6, + "p90_in": 16.4, + "scale_error_pct": -0.14 }, "3-10m": { - "pairs": 595400, - "failed_pct": 0.17, - "median_in": 7.05, - "p90_in": 46.43, - "scale_error_pct": -0.06 + "pairs": 520400, + "failed_pct": 0.19, + "median_in": 5.88, + "p90_in": 24.38, + "scale_error_pct": -0.14 } }, "tape_calibration_success_pct": 100.0, - "groups": 8, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 6.04, - 10.78 + 5.85, + 8.76 ], "p90_in": [ - 15.01, - 47.96 + 12.48, + 25.37 ] }, "3-10m": { "median_in": [ - 11.67, - 21.42 + 10.89, + 19.56 ], "p90_in": [ - 24.05, - 70.59 + 22.44, + 32.61 ] } } @@ -10979,58 +10979,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.01, - "median_in": 5.91, - "p90_in": 16.79, - "scale_error_pct": 2.54 + "pairs": 21000, + "failed_pct": 0.0, + "median_in": 5.29, + "p90_in": 13.44, + "scale_error_pct": 3.76 }, "3-10m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.03, - "median_in": 16.5, - "p90_in": 49.78, - "scale_error_pct": 3.69 + "median_in": 14.79, + "p90_in": 29.02, + "scale_error_pct": 4.61 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 1.01, - "median_in": 3.16, - "p90_in": 15.15, - "scale_error_pct": -0.19 + "pairs": 525000, + "failed_pct": 1.14, + "median_in": 2.75, + "p90_in": 10.74, + "scale_error_pct": -0.22 }, "3-10m": { - "pairs": 600000, - "failed_pct": 1.03, - "median_in": 6.87, - "p90_in": 39.4, - "scale_error_pct": -0.0 + "pairs": 525000, + "failed_pct": 1.17, + "median_in": 5.89, + "p90_in": 25.0, + "scale_error_pct": 0.13 } }, - "tape_calibration_success_pct": 99.0, - "groups": 8, + "tape_calibration_success_pct": 98.9, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 4.26, - 8.91 + 3.89, + 7.48 ], "p90_in": [ - 10.52, - 23.9 + 9.89, + 16.1 ] }, "3-10m": { "median_in": [ - 11.88, - 23.13 + 10.99, + 19.36 ], "p90_in": [ - 25.48, - 81.97 + 23.5, + 33.91 ] } } @@ -11038,58 +11038,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.02, - "median_in": 5.59, - "p90_in": 16.16, - "scale_error_pct": 1.26 + "median_in": 4.92, + "p90_in": 12.44, + "scale_error_pct": 3.27 }, "3-10m": { - "pairs": 24000, - "failed_pct": 0.05, - "median_in": 16.49, - "p90_in": 52.43, - "scale_error_pct": 4.17 + "pairs": 21000, + "failed_pct": 0.04, + "median_in": 14.94, + "p90_in": 29.77, + "scale_error_pct": 5.31 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 1.02, - "median_in": 2.83, - "p90_in": 14.72, - "scale_error_pct": -0.3 + "pairs": 525000, + "failed_pct": 1.17, + "median_in": 2.41, + "p90_in": 10.25, + "scale_error_pct": -0.32 }, "3-10m": { - "pairs": 600000, - "failed_pct": 1.05, - "median_in": 6.93, - "p90_in": 41.08, - "scale_error_pct": 0.03 + "pairs": 525000, + "failed_pct": 1.18, + "median_in": 5.91, + "p90_in": 28.82, + "scale_error_pct": 0.18 } }, - "tape_calibration_success_pct": 99.0, - "groups": 8, + "tape_calibration_success_pct": 98.9, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 3.46, - 8.89 + 2.94, + 7.45 ], "p90_in": [ - 9.51, - 23.0 + 8.44, + 15.29 ] }, "3-10m": { "median_in": [ - 11.7, - 24.11 + 11.15, + 20.15 ], "p90_in": [ - 23.34, - 84.95 + 20.36, + 35.11 ] } } @@ -11097,58 +11097,58 @@ "edges": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.11, - "median_in": 6.42, - "p90_in": 21.34, - "scale_error_pct": 4.83 + "pairs": 21000, + "failed_pct": 0.13, + "median_in": 6.05, + "p90_in": 17.21, + "scale_error_pct": 4.6 }, "3-10m": { - "pairs": 23816, - "failed_pct": 0.06, - "median_in": 13.29, - "p90_in": 37.59, - "scale_error_pct": 4.73 + "pairs": 20816, + "failed_pct": 0.07, + "median_in": 12.3, + "p90_in": 26.69, + "scale_error_pct": 5.16 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 1.11, - "median_in": 4.65, - "p90_in": 27.06, - "scale_error_pct": 0.65 + "pairs": 525000, + "failed_pct": 1.27, + "median_in": 3.86, + "p90_in": 15.9, + "scale_error_pct": -0.17 }, "3-10m": { - "pairs": 595400, - "failed_pct": 1.07, - "median_in": 7.65, - "p90_in": 47.82, - "scale_error_pct": 0.82 + "pairs": 520400, + "failed_pct": 1.23, + "median_in": 6.48, + "p90_in": 25.53, + "scale_error_pct": 0.61 } }, - "tape_calibration_success_pct": 99.0, - "groups": 8, + "tape_calibration_success_pct": 98.9, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 5.26, - 8.57 + 5.11, + 7.87 ], "p90_in": [ - 13.83, - 36.63 + 11.57, + 21.77 ] }, "3-10m": { "median_in": [ - 10.38, - 17.95 + 9.77, + 15.4 ], "p90_in": [ - 21.94, - 66.42 + 18.81, + 31.13 ] } } @@ -11176,16 +11176,16 @@ "1-3m": { "pairs": 525000, "failed_pct": 0.57, - "median_in": 3.52, - "p90_in": 13.2, - "scale_error_pct": -0.33 + "median_in": 3.51, + "p90_in": 13.12, + "scale_error_pct": -0.3 }, "3-10m": { "pairs": 525000, "failed_pct": 0.59, - "median_in": 7.07, - "p90_in": 29.8, - "scale_error_pct": -0.14 + "median_in": 7.04, + "p90_in": 29.71, + "scale_error_pct": -0.11 } }, "tape_calibration_success_pct": 99.4, @@ -11218,33 +11218,33 @@ "none": { "1-3m": { "pairs": 21000, - "failed_pct": 0.0, - "median_in": 4.85, - "p90_in": 12.18, - "scale_error_pct": -0.52 + "failed_pct": 0.01, + "median_in": 4.82, + "p90_in": 12.26, + "scale_error_pct": -0.56 }, "3-10m": { "pairs": 21000, - "failed_pct": 0.0, - "median_in": 13.26, - "p90_in": 24.26, - "scale_error_pct": 2.9 + "failed_pct": 0.01, + "median_in": 13.2, + "p90_in": 24.18, + "scale_error_pct": 2.91 } }, "one_known_distance": { "1-3m": { "pairs": 525000, - "failed_pct": 0.57, - "median_in": 3.29, - "p90_in": 13.45, - "scale_error_pct": -1.05 + "failed_pct": 0.59, + "median_in": 3.27, + "p90_in": 13.55, + "scale_error_pct": -1.04 }, "3-10m": { "pairs": 525000, - "failed_pct": 0.57, - "median_in": 7.82, - "p90_in": 30.05, - "scale_error_pct": -0.24 + "failed_pct": 0.58, + "median_in": 7.78, + "p90_in": 29.83, + "scale_error_pct": -0.23 } }, "tape_calibration_success_pct": 99.4, @@ -11253,22 +11253,22 @@ "interval_95": { "1-3m": { "median_in": [ - 2.9, - 6.81 + 2.89, + 6.86 ], "p90_in": [ - 9.64, - 14.3 + 9.74, + 14.32 ] }, "3-10m": { "median_in": [ - 10.32, - 16.66 + 10.47, + 16.71 ], "p90_in": [ - 19.59, - 28.57 + 19.53, + 28.41 ] } } @@ -11280,7 +11280,7 @@ "failed_pct": 0.0, "median_in": 6.48, "p90_in": 15.62, - "scale_error_pct": 3.76 + "scale_error_pct": 3.75 }, "3-10m": { "pairs": 20816, @@ -11294,16 +11294,16 @@ "1-3m": { "pairs": 525000, "failed_pct": 0.57, - "median_in": 4.76, - "p90_in": 16.26, - "scale_error_pct": 0.13 + "median_in": 4.74, + "p90_in": 16.27, + "scale_error_pct": 0.19 }, "3-10m": { "pairs": 520400, "failed_pct": 0.59, - "median_in": 7.3, - "p90_in": 27.99, - "scale_error_pct": 0.81 + "median_in": 7.27, + "p90_in": 28.01, + "scale_error_pct": 0.86 } }, "tape_calibration_success_pct": 99.4, @@ -11339,58 +11339,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 12.56, - "median_in": 2.08, - "p90_in": Infinity, - "scale_error_pct": 0.03 + "pairs": 21000, + "failed_pct": 14.31, + "median_in": 1.74, + "p90_in": "failed", + "scale_error_pct": 0.13 }, "3-10m": { - "pairs": 24000, - "failed_pct": 12.61, - "median_in": 5.19, - "p90_in": Infinity, - "scale_error_pct": 0.61 + "pairs": 21000, + "failed_pct": 14.39, + "median_in": 4.31, + "p90_in": "failed", + "scale_error_pct": 0.76 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 12.56, - "median_in": 3.45, - "p90_in": Infinity, - "scale_error_pct": -0.04 + "pairs": 525000, + "failed_pct": 14.31, + "median_in": 2.98, + "p90_in": "failed", + "scale_error_pct": -0.05 }, "3-10m": { - "pairs": 600000, - "failed_pct": 12.61, - "median_in": 8.72, - "p90_in": Infinity, - "scale_error_pct": 0.17 + "pairs": 525000, + "failed_pct": 14.39, + "median_in": 7.28, + "p90_in": "failed", + "scale_error_pct": 0.37 } }, - "tape_calibration_success_pct": 87.5, - "groups": 8, + "tape_calibration_success_pct": 85.7, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 1.3, - 5.83 + 1.21, + 4.98 ], "p90_in": [ - 5.11, - Infinity + 4.65, + "failed" ] }, "3-10m": { "median_in": [ - 3.25, - 17.81 + 2.95, + 10.79 ], "p90_in": [ - 10.69, - Infinity + 9.02, + "failed" ] } } @@ -11398,58 +11398,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 12.58, - "median_in": 1.59, - "p90_in": Infinity, - "scale_error_pct": -0.04 - }, - "3-10m": { - "pairs": 24000, - "failed_pct": 12.66, - "median_in": 5.89, - "p90_in": Infinity, - "scale_error_pct": 1.41 + "pairs": 21000, + "failed_pct": 14.35, + "median_in": 1.31, + "p90_in": "failed", + "scale_error_pct": 0.07 + }, + "3-10m": { + "pairs": 21000, + "failed_pct": 14.44, + "median_in": 4.81, + "p90_in": "failed", + "scale_error_pct": 1.63 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 12.58, - "median_in": 3.07, - "p90_in": Infinity, - "scale_error_pct": 0.14 + "pairs": 525000, + "failed_pct": 14.35, + "median_in": 2.61, + "p90_in": "failed", + "scale_error_pct": 0.15 }, "3-10m": { - "pairs": 600000, - "failed_pct": 12.66, - "median_in": 9.18, - "p90_in": Infinity, - "scale_error_pct": 0.92 + "pairs": 525000, + "failed_pct": 14.44, + "median_in": 7.61, + "p90_in": "failed", + "scale_error_pct": 1.26 } }, - "tape_calibration_success_pct": 87.5, - "groups": 8, + "tape_calibration_success_pct": 85.7, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 0.96, - 4.87 + 0.89, + 3.76 ], "p90_in": [ - 3.87, - Infinity + 3.55, + "failed" ] }, "3-10m": { "median_in": [ - 3.22, - 23.93 + 2.73, + 17.18 ], "p90_in": [ - 11.98, - Infinity + 8.55, + "failed" ] } } @@ -11457,58 +11457,58 @@ "edges": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 12.81, - "median_in": 3.27, - "p90_in": Infinity, - "scale_error_pct": 0.27 + "pairs": 21000, + "failed_pct": 14.61, + "median_in": 2.23, + "p90_in": "failed", + "scale_error_pct": -0.08 }, "3-10m": { - "pairs": 23816, - "failed_pct": 11.99, - "median_in": 5.01, - "p90_in": Infinity, - "scale_error_pct": 0.09 + "pairs": 20816, + "failed_pct": 13.72, + "median_in": 3.77, + "p90_in": "failed", + "scale_error_pct": 0.03 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 12.81, - "median_in": 5.17, - "p90_in": Infinity, - "scale_error_pct": 0.64 + "pairs": 525000, + "failed_pct": 14.61, + "median_in": 4.11, + "p90_in": "failed", + "scale_error_pct": -0.16 }, "3-10m": { - "pairs": 595400, - "failed_pct": 11.99, - "median_in": 9.01, - "p90_in": Infinity, - "scale_error_pct": 0.16 + "pairs": 520400, + "failed_pct": 13.72, + "median_in": 7.45, + "p90_in": "failed", + "scale_error_pct": 0.03 } }, - "tape_calibration_success_pct": 87.5, - "groups": 8, + "tape_calibration_success_pct": 85.7, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 1.08, - 24.38 + 0.95, + 12.1 ], "p90_in": [ - 10.97, - Infinity + 8.55, + "failed" ] }, "3-10m": { "median_in": [ - 1.67, - 22.41 + 1.46, + 11.88 ], "p90_in": [ - 12.78, - Infinity + 10.13, + "failed" ] } } @@ -11518,58 +11518,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.01, - "median_in": 1.94, - "p90_in": 9.06, - "scale_error_pct": -0.56 + "pairs": 21000, + "failed_pct": 0.0, + "median_in": 1.66, + "p90_in": 6.87, + "scale_error_pct": -0.35 }, "3-10m": { - "pairs": 24000, + "pairs": 21000, "failed_pct": 0.04, - "median_in": 5.02, - "p90_in": 23.66, - "scale_error_pct": -0.24 + "median_in": 4.28, + "p90_in": 12.29, + "scale_error_pct": 0.07 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 0.01, - "median_in": 3.03, - "p90_in": 13.21, - "scale_error_pct": -0.15 + "pairs": 525000, + "failed_pct": 0.0, + "median_in": 2.65, + "p90_in": 10.1, + "scale_error_pct": -0.18 }, "3-10m": { - "pairs": 600000, + "pairs": 525000, "failed_pct": 0.04, - "median_in": 7.19, - "p90_in": 32.51, - "scale_error_pct": -0.02 + "median_in": 6.26, + "p90_in": 24.12, + "scale_error_pct": 0.13 } }, "tape_calibration_success_pct": 100.0, - "groups": 8, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 1.34, - 3.26 + 1.23, + 2.23 ], "p90_in": [ - 5.25, - 12.8 + 4.68, + 9.01 ] }, "3-10m": { "median_in": [ - 3.43, - 8.26 + 3.06, + 5.72 ], "p90_in": [ - 9.69, - 42.06 + 7.99, + 17.47 ] } } @@ -11577,58 +11577,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.05, - "median_in": 1.67, - "p90_in": 8.69, - "scale_error_pct": -0.62 + "pairs": 21000, + "failed_pct": 0.03, + "median_in": 1.45, + "p90_in": 6.77, + "scale_error_pct": -0.41 }, "3-10m": { - "pairs": 24000, - "failed_pct": 0.05, - "median_in": 6.15, - "p90_in": 28.05, - "scale_error_pct": 0.07 + "pairs": 21000, + "failed_pct": 0.04, + "median_in": 5.35, + "p90_in": 14.16, + "scale_error_pct": 0.51 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 0.05, - "median_in": 2.69, - "p90_in": 12.51, - "scale_error_pct": -0.06 + "pairs": 525000, + "failed_pct": 0.03, + "median_in": 2.31, + "p90_in": 9.46, + "scale_error_pct": -0.11 }, "3-10m": { - "pairs": 600000, - "failed_pct": 0.05, - "median_in": 7.61, - "p90_in": 32.44, - "scale_error_pct": 0.26 + "pairs": 525000, + "failed_pct": 0.04, + "median_in": 6.58, + "p90_in": 24.17, + "scale_error_pct": 0.51 } }, "tape_calibration_success_pct": 100.0, - "groups": 8, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 1.24, - 2.62 + 1.13, + 1.73 ], "p90_in": [ - 4.77, - 11.78 + 3.58, + 8.94 ] }, "3-10m": { "median_in": [ - 3.55, - 10.34 + 2.88, + 8.28 ], "p90_in": [ - 9.79, - 44.88 + 8.4, + 27.49 ] } } @@ -11636,58 +11636,58 @@ "edges": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.11, - "median_in": 3.01, - "p90_in": 22.98, - "scale_error_pct": -0.11 + "pairs": 21000, + "failed_pct": 0.13, + "median_in": 2.16, + "p90_in": 12.86, + "scale_error_pct": -0.39 }, "3-10m": { - "pairs": 23816, - "failed_pct": 0.06, - "median_in": 6.19, - "p90_in": 31.51, - "scale_error_pct": -0.3 + "pairs": 20816, + "failed_pct": 0.07, + "median_in": 5.17, + "p90_in": 15.99, + "scale_error_pct": -0.33 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 0.11, - "median_in": 4.51, - "p90_in": 25.91, - "scale_error_pct": 0.21 + "pairs": 525000, + "failed_pct": 0.13, + "median_in": 3.69, + "p90_in": 15.09, + "scale_error_pct": -0.47 }, "3-10m": { - "pairs": 595400, - "failed_pct": 0.06, - "median_in": 7.5, - "p90_in": 40.56, - "scale_error_pct": -0.1 + "pairs": 520400, + "failed_pct": 0.07, + "median_in": 6.41, + "p90_in": 23.15, + "scale_error_pct": -0.31 } }, "tape_calibration_success_pct": 100.0, - "groups": 8, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 1.02, - 6.71 + 0.85, + 4.03 ], "p90_in": [ - 10.62, - 56.52 + 9.52, + 16.97 ] }, "3-10m": { "median_in": [ - 1.72, - 9.89 + 1.24, + 7.74 ], "p90_in": [ - 11.26, - 62.03 + 10.45, + 21.15 ] } } @@ -11716,14 +11716,14 @@ "pairs": 525000, "failed_pct": 0.57, "median_in": 2.56, - "p90_in": 10.9, + "p90_in": 10.79, "scale_error_pct": -0.29 }, "3-10m": { "pairs": 525000, "failed_pct": 0.59, - "median_in": 5.56, - "p90_in": 24.95, + "median_in": 5.52, + "p90_in": 24.85, "scale_error_pct": -0.03 } }, @@ -11757,33 +11757,33 @@ "none": { "1-3m": { "pairs": 21000, - "failed_pct": 0.0, - "median_in": 1.24, - "p90_in": 6.29, - "scale_error_pct": -0.55 + "failed_pct": 0.01, + "median_in": 1.23, + "p90_in": 6.19, + "scale_error_pct": -0.57 }, "3-10m": { "pairs": 21000, - "failed_pct": 0.0, - "median_in": 3.5, - "p90_in": 14.84, - "scale_error_pct": -0.24 + "failed_pct": 0.01, + "median_in": 3.52, + "p90_in": 14.88, + "scale_error_pct": -0.25 } }, "one_known_distance": { "1-3m": { "pairs": 525000, - "failed_pct": 0.57, - "median_in": 2.22, - "p90_in": 10.59, + "failed_pct": 0.59, + "median_in": 2.2, + "p90_in": 10.46, "scale_error_pct": -0.26 }, "3-10m": { "pairs": 525000, - "failed_pct": 0.57, - "median_in": 5.82, - "p90_in": 25.58, - "scale_error_pct": -0.02 + "failed_pct": 0.58, + "median_in": 5.8, + "p90_in": 25.42, + "scale_error_pct": -0.04 } }, "tape_calibration_success_pct": 99.4, @@ -11793,21 +11793,21 @@ "1-3m": { "median_in": [ 0.92, - 1.9 + 1.88 ], "p90_in": [ - 3.57, - 10.63 + 3.58, + 10.06 ] }, "3-10m": { "median_in": [ - 2.35, - 5.19 + 2.33, + 5.15 ], "p90_in": [ - 6.65, - 25.02 + 6.68, + 25.09 ] } } @@ -11818,7 +11818,7 @@ "pairs": 21000, "failed_pct": 0.0, "median_in": 2.27, - "p90_in": 11.3, + "p90_in": 11.28, "scale_error_pct": -0.89 }, "3-10m": { @@ -11833,16 +11833,16 @@ "1-3m": { "pairs": 525000, "failed_pct": 0.57, - "median_in": 3.5, - "p90_in": 15.05, - "scale_error_pct": -0.4 + "median_in": 3.48, + "p90_in": 15.04, + "scale_error_pct": -0.39 }, "3-10m": { "pairs": 520400, "failed_pct": 0.59, - "median_in": 5.45, - "p90_in": 24.1, - "scale_error_pct": -0.1 + "median_in": 5.42, + "p90_in": 24.06, + "scale_error_pct": -0.09 } }, "tape_calibration_success_pct": 99.4, @@ -11856,7 +11856,7 @@ ], "p90_in": [ 7.9, - 15.47 + 15.41 ] }, "3-10m": { @@ -11878,58 +11878,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 0.06, - "median_in": 6.3, - "p90_in": 16.84, - "scale_error_pct": 1.85 + "pairs": 105000, + "failed_pct": 0.03, + "median_in": 5.66, + "p90_in": 14.21, + "scale_error_pct": 2.8 }, "3-10m": { - "pairs": 120000, - "failed_pct": 0.11, - "median_in": 17.2, - "p90_in": 41.76, - "scale_error_pct": 2.78 + "pairs": 105000, + "failed_pct": 0.1, + "median_in": 15.75, + "p90_in": 30.67, + "scale_error_pct": 4.04 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 0.06, - "median_in": 3.41, - "p90_in": 14.78, - "scale_error_pct": -0.2 + "pairs": 2625000, + "failed_pct": 0.03, + "median_in": 2.98, + "p90_in": 10.94, + "scale_error_pct": -0.22 }, "3-10m": { - "pairs": 3000000, - "failed_pct": 0.11, - "median_in": 7.35, - "p90_in": 39.21, - "scale_error_pct": -0.38 + "pairs": 2625000, + "failed_pct": 0.1, + "median_in": 6.3, + "p90_in": 24.61, + "scale_error_pct": -0.21 } }, "tape_calibration_success_pct": 100.0, - "groups": 8, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 4.07, - 9.28 + 3.81, + 8.39 ], "p90_in": [ - 11.43, - 21.98 + 10.79, + 16.91 ] }, "3-10m": { "median_in": [ - 12.48, - 23.63 + 11.9, + 21.6 ], "p90_in": [ - 26.29, - 75.17 + 25.15, + 33.22 ] } } @@ -11937,58 +11937,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 0.08, - "median_in": 6.11, - "p90_in": 16.53, - "scale_error_pct": 1.92 + "pairs": 105000, + "failed_pct": 0.07, + "median_in": 5.43, + "p90_in": 13.98, + "scale_error_pct": 3.24 }, "3-10m": { - "pairs": 120000, + "pairs": 105000, "failed_pct": 0.16, - "median_in": 17.67, - "p90_in": 45.73, - "scale_error_pct": 4.75 + "median_in": 16.3, + "p90_in": 30.79, + "scale_error_pct": 5.67 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 0.08, - "median_in": 2.95, - "p90_in": 13.76, - "scale_error_pct": -0.12 + "pairs": 2625000, + "failed_pct": 0.07, + "median_in": 2.55, + "p90_in": 10.03, + "scale_error_pct": -0.14 }, "3-10m": { - "pairs": 3000000, + "pairs": 2625000, "failed_pct": 0.16, - "median_in": 7.42, - "p90_in": 41.25, - "scale_error_pct": 0.15 + "median_in": 6.32, + "p90_in": 25.71, + "scale_error_pct": 0.36 } }, "tape_calibration_success_pct": 100.0, - "groups": 8, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 3.49, - 9.01 + 2.99, + 8.34 ], "p90_in": [ - 10.68, - 21.32 + 9.54, + 16.88 ] }, "3-10m": { "median_in": [ - 14.17, - 25.77 + 13.59, + 22.46 ], "p90_in": [ - 25.53, - 76.71 + 24.07, + 35.21 ] } } @@ -11996,58 +11996,58 @@ "edges": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 0.31, - "median_in": 7.66, - "p90_in": 30.58, - "scale_error_pct": 5.59 + "pairs": 105000, + "failed_pct": 0.32, + "median_in": 7.01, + "p90_in": 19.11, + "scale_error_pct": 4.61 }, "3-10m": { - "pairs": 119080, - "failed_pct": 0.17, - "median_in": 15.75, - "p90_in": 44.58, - "scale_error_pct": 4.42 + "pairs": 104080, + "failed_pct": 0.19, + "median_in": 14.29, + "p90_in": 28.01, + "scale_error_pct": 4.95 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 0.31, - "median_in": 4.61, - "p90_in": 34.11, - "scale_error_pct": 0.93 + "pairs": 2625000, + "failed_pct": 0.32, + "median_in": 3.82, + "p90_in": 16.86, + "scale_error_pct": 0.07 }, "3-10m": { - "pairs": 2977000, - "failed_pct": 0.17, - "median_in": 7.37, - "p90_in": 48.73, - "scale_error_pct": 0.1 + "pairs": 2602000, + "failed_pct": 0.19, + "median_in": 6.16, + "p90_in": 25.96, + "scale_error_pct": 0.02 } }, "tape_calibration_success_pct": 100.0, - "groups": 8, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 6.2, - 11.2 + 5.94, + 9.13 ], "p90_in": [ - 15.46, - 48.32 + 13.92, + 25.5 ] }, "3-10m": { "median_in": [ - 11.73, - 21.5 + 11.07, + 19.68 ], "p90_in": [ - 24.07, - 70.44 + 22.67, + 33.05 ] } } @@ -12057,58 +12057,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 0.01, - "median_in": 5.95, - "p90_in": 17.36, - "scale_error_pct": 1.76 + "pairs": 105000, + "failed_pct": 0.0, + "median_in": 5.27, + "p90_in": 14.04, + "scale_error_pct": 3.14 }, "3-10m": { - "pairs": 120000, + "pairs": 105000, "failed_pct": 0.03, - "median_in": 16.23, - "p90_in": 49.63, - "scale_error_pct": 3.09 + "median_in": 14.33, + "p90_in": 29.14, + "scale_error_pct": 4.34 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 0.81, - "median_in": 3.81, - "p90_in": 16.23, - "scale_error_pct": -0.54 + "pairs": 2625000, + "failed_pct": 0.91, + "median_in": 3.37, + "p90_in": 12.28, + "scale_error_pct": -0.59 }, "3-10m": { - "pairs": 3000000, - "failed_pct": 0.83, - "median_in": 7.97, - "p90_in": 40.68, - "scale_error_pct": -0.31 + "pairs": 2625000, + "failed_pct": 0.94, + "median_in": 6.88, + "p90_in": 27.81, + "scale_error_pct": -0.16 } }, - "tape_calibration_success_pct": 99.2, - "groups": 8, + "tape_calibration_success_pct": 99.1, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 4.19, - 9.24 + 3.88, + 8.02 ], "p90_in": [ - 11.11, - 23.87 + 10.63, + 16.63 ] }, "3-10m": { "median_in": [ - 11.17, - 23.85 + 10.49, + 19.95 ], "p90_in": [ - 25.7, - 81.51 + 24.18, + 34.07 ] } } @@ -12116,58 +12116,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 120000, + "pairs": 105000, "failed_pct": 0.02, - "median_in": 5.64, - "p90_in": 16.38, - "scale_error_pct": 1.44 + "median_in": 4.92, + "p90_in": 12.88, + "scale_error_pct": 3.22 }, "3-10m": { - "pairs": 120000, - "failed_pct": 0.05, - "median_in": 16.34, - "p90_in": 52.42, - "scale_error_pct": 4.25 + "pairs": 105000, + "failed_pct": 0.04, + "median_in": 14.75, + "p90_in": 29.86, + "scale_error_pct": 5.33 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 0.82, - "median_in": 3.37, - "p90_in": 15.48, - "scale_error_pct": -0.39 + "pairs": 2625000, + "failed_pct": 0.94, + "median_in": 2.92, + "p90_in": 11.45, + "scale_error_pct": -0.41 }, "3-10m": { - "pairs": 3000000, - "failed_pct": 0.85, - "median_in": 8.12, - "p90_in": 41.71, - "scale_error_pct": 0.03 + "pairs": 2625000, + "failed_pct": 0.95, + "median_in": 6.97, + "p90_in": 31.5, + "scale_error_pct": 0.21 } }, - "tape_calibration_success_pct": 99.2, - "groups": 8, + "tape_calibration_success_pct": 99.1, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 3.44, - 9.13 + 2.96, + 8.09 ], "p90_in": [ - 10.25, - 22.99 + 8.88, + 15.58 ] }, "3-10m": { "median_in": [ - 11.69, - 25.04 + 11.13, + 20.66 ], "p90_in": [ - 24.41, - 84.96 + 21.01, + 35.62 ] } } @@ -12175,58 +12175,58 @@ "edges": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 0.11, - "median_in": 6.6, - "p90_in": 21.39, - "scale_error_pct": 5.0 + "pairs": 105000, + "failed_pct": 0.13, + "median_in": 6.19, + "p90_in": 17.45, + "scale_error_pct": 4.74 }, "3-10m": { - "pairs": 119080, - "failed_pct": 0.06, - "median_in": 13.97, - "p90_in": 38.63, - "scale_error_pct": 5.11 + "pairs": 104080, + "failed_pct": 0.07, + "median_in": 12.73, + "p90_in": 27.51, + "scale_error_pct": 5.51 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 0.91, - "median_in": 5.22, - "p90_in": 27.21, - "scale_error_pct": 0.84 + "pairs": 2625000, + "failed_pct": 1.04, + "median_in": 4.44, + "p90_in": 16.49, + "scale_error_pct": -0.21 }, "3-10m": { - "pairs": 2977000, - "failed_pct": 0.87, - "median_in": 8.88, - "p90_in": 47.08, - "scale_error_pct": 1.07 + "pairs": 2602000, + "failed_pct": 0.99, + "median_in": 7.63, + "p90_in": 27.85, + "scale_error_pct": 0.82 } }, - "tape_calibration_success_pct": 99.2, - "groups": 8, + "tape_calibration_success_pct": 99.1, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 5.34, - 8.86 + 5.19, + 8.23 ], "p90_in": [ - 14.31, - 37.34 + 13.06, + 21.99 ] }, "3-10m": { "median_in": [ - 10.78, - 19.87 + 9.95, + 16.36 ], "p90_in": [ - 22.23, - 66.38 + 18.95, + 32.31 ] } } @@ -12254,16 +12254,16 @@ "1-3m": { "pairs": 2625000, "failed_pct": 0.57, - "median_in": 4.28, - "p90_in": 15.15, - "scale_error_pct": -0.57 + "median_in": 4.27, + "p90_in": 15.08, + "scale_error_pct": -0.53 }, "3-10m": { "pairs": 2625000, "failed_pct": 0.59, - "median_in": 8.48, - "p90_in": 33.29, - "scale_error_pct": -0.49 + "median_in": 8.44, + "p90_in": 33.19, + "scale_error_pct": -0.46 } }, "tape_calibration_success_pct": 99.4, @@ -12296,33 +12296,33 @@ "none": { "1-3m": { "pairs": 105000, - "failed_pct": 0.0, - "median_in": 4.96, - "p90_in": 13.23, - "scale_error_pct": -0.79 + "failed_pct": 0.01, + "median_in": 4.94, + "p90_in": 13.38, + "scale_error_pct": -0.83 }, "3-10m": { "pairs": 105000, - "failed_pct": 0.0, - "median_in": 13.84, - "p90_in": 25.23, - "scale_error_pct": 2.78 + "failed_pct": 0.01, + "median_in": 13.76, + "p90_in": 25.19, + "scale_error_pct": 2.82 } }, "one_known_distance": { "1-3m": { "pairs": 2625000, - "failed_pct": 0.57, - "median_in": 3.83, - "p90_in": 15.25, - "scale_error_pct": -0.81 + "failed_pct": 0.59, + "median_in": 3.82, + "p90_in": 15.38, + "scale_error_pct": -0.77 }, "3-10m": { "pairs": 2625000, - "failed_pct": 0.57, - "median_in": 8.4, - "p90_in": 34.45, - "scale_error_pct": 0.12 + "failed_pct": 0.58, + "median_in": 8.34, + "p90_in": 34.27, + "scale_error_pct": 0.14 } }, "tape_calibration_success_pct": 99.4, @@ -12331,22 +12331,22 @@ "interval_95": { "1-3m": { "median_in": [ - 3.52, - 7.18 + 3.5, + 7.17 ], "p90_in": [ - 10.27, - 15.05 + 10.44, + 15.1 ] }, "3-10m": { "median_in": [ - 10.56, - 17.84 + 10.64, + 17.97 ], "p90_in": [ - 20.96, - 28.76 + 20.88, + 28.67 ] } } @@ -12358,7 +12358,7 @@ "failed_pct": 0.0, "median_in": 6.92, "p90_in": 15.86, - "scale_error_pct": 3.75 + "scale_error_pct": 3.74 }, "3-10m": { "pairs": 104080, @@ -12372,16 +12372,16 @@ "1-3m": { "pairs": 2625000, "failed_pct": 0.57, - "median_in": 4.92, - "p90_in": 17.71, - "scale_error_pct": 0.68 + "median_in": 4.89, + "p90_in": 17.73, + "scale_error_pct": 0.74 }, "3-10m": { "pairs": 2602000, "failed_pct": 0.59, - "median_in": 7.96, - "p90_in": 31.4, - "scale_error_pct": 0.95 + "median_in": 7.93, + "p90_in": 31.44, + "scale_error_pct": 1.0 } }, "tape_calibration_success_pct": 99.4, @@ -12417,58 +12417,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 12.56, - "median_in": 10.82, - "p90_in": Infinity, - "scale_error_pct": -9.73 + "pairs": 105000, + "failed_pct": 14.31, + "median_in": 10.51, + "p90_in": "failed", + "scale_error_pct": -9.96 }, "3-10m": { - "pairs": 120000, - "failed_pct": 12.61, - "median_in": 29.39, - "p90_in": Infinity, - "scale_error_pct": -9.76 + "pairs": 105000, + "failed_pct": 14.39, + "median_in": 28.3, + "p90_in": "failed", + "scale_error_pct": -9.73 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 12.56, - "median_in": 4.14, - "p90_in": Infinity, - "scale_error_pct": -0.09 + "pairs": 2625000, + "failed_pct": 14.31, + "median_in": 3.61, + "p90_in": "failed", + "scale_error_pct": -0.1 }, "3-10m": { - "pairs": 3000000, - "failed_pct": 12.61, - "median_in": 10.35, - "p90_in": Infinity, - "scale_error_pct": 0.11 + "pairs": 2625000, + "failed_pct": 14.39, + "median_in": 8.75, + "p90_in": "failed", + "scale_error_pct": 0.31 } }, - "tape_calibration_success_pct": 87.5, - "groups": 8, + "tape_calibration_success_pct": 85.7, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 7.77, - 17.47 + 7.21, + 21.22 ], "p90_in": [ - 17.29, - Infinity + 15.9, + "failed" ] }, "3-10m": { "median_in": [ - 19.67, - 50.97 + 17.57, + 63.6 ], "p90_in": [ - 45.21, - Infinity + 40.63, + "failed" ] } } @@ -12476,58 +12476,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 12.58, - "median_in": 10.38, - "p90_in": Infinity, - "scale_error_pct": -9.98 + "pairs": 105000, + "failed_pct": 14.35, + "median_in": 10.1, + "p90_in": "failed", + "scale_error_pct": -10.21 }, "3-10m": { - "pairs": 120000, - "failed_pct": 12.66, - "median_in": 24.53, - "p90_in": Infinity, - "scale_error_pct": -7.78 + "pairs": 105000, + "failed_pct": 14.44, + "median_in": 23.01, + "p90_in": "failed", + "scale_error_pct": -7.61 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 12.58, - "median_in": 3.56, - "p90_in": Infinity, + "pairs": 2625000, + "failed_pct": 14.35, + "median_in": 3.06, + "p90_in": "failed", "scale_error_pct": 0.21 }, "3-10m": { - "pairs": 3000000, - "failed_pct": 12.66, - "median_in": 10.12, - "p90_in": Infinity, - "scale_error_pct": 1.04 + "pairs": 2625000, + "failed_pct": 14.44, + "median_in": 8.41, + "p90_in": "failed", + "scale_error_pct": 1.36 } }, - "tape_calibration_success_pct": 87.5, - "groups": 8, + "tape_calibration_success_pct": 85.7, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 7.36, - 16.65 + 6.84, + 21.28 ], "p90_in": [ - 16.05, - Infinity + 14.62, + "failed" ] }, "3-10m": { "median_in": [ - 15.14, - 49.5 + 13.92, + 61.79 ], "p90_in": [ - 40.51, - Infinity + 34.99, + "failed" ] } } @@ -12535,58 +12535,58 @@ "edges": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 12.81, - "median_in": 11.45, - "p90_in": Infinity, - "scale_error_pct": -7.78 + "pairs": 105000, + "failed_pct": 14.61, + "median_in": 10.35, + "p90_in": "failed", + "scale_error_pct": -9.38 }, "3-10m": { - "pairs": 119080, - "failed_pct": 11.99, - "median_in": 24.1, - "p90_in": Infinity, - "scale_error_pct": -9.3 + "pairs": 104080, + "failed_pct": 13.72, + "median_in": 21.74, + "p90_in": "failed", + "scale_error_pct": -9.65 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 12.81, - "median_in": 5.74, - "p90_in": Infinity, - "scale_error_pct": 0.82 + "pairs": 2625000, + "failed_pct": 14.61, + "median_in": 4.5, + "p90_in": "failed", + "scale_error_pct": -0.06 }, "3-10m": { - "pairs": 2977000, - "failed_pct": 11.99, - "median_in": 9.94, - "p90_in": Infinity, - "scale_error_pct": 0.27 + "pairs": 2602000, + "failed_pct": 13.72, + "median_in": 8.14, + "p90_in": "failed", + "scale_error_pct": 0.14 } }, - "tape_calibration_success_pct": 87.5, - "groups": 8, + "tape_calibration_success_pct": 85.7, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 7.99, - 28.97 + 7.44, + 22.22 ], "p90_in": [ - 18.98, - Infinity + 16.26, + "failed" ] }, "3-10m": { "median_in": [ - 16.05, - 53.38 + 14.6, + 57.21 ], "p90_in": [ - 39.43, - Infinity + 31.99, + "failed" ] } } @@ -12596,58 +12596,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 0.01, - "median_in": 9.73, - "p90_in": 27.67, - "scale_error_pct": -9.59 + "pairs": 105000, + "failed_pct": 0.0, + "median_in": 9.6, + "p90_in": 27.39, + "scale_error_pct": -9.32 }, "3-10m": { - "pairs": 120000, + "pairs": 105000, "failed_pct": 0.04, - "median_in": 26.21, - "p90_in": 65.58, - "scale_error_pct": -10.16 + "median_in": 24.6, + "p90_in": 62.61, + "scale_error_pct": -9.39 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 0.01, - "median_in": 4.76, - "p90_in": 21.6, - "scale_error_pct": -0.38 + "pairs": 2625000, + "failed_pct": 0.0, + "median_in": 4.3, + "p90_in": 17.87, + "scale_error_pct": -0.42 }, "3-10m": { - "pairs": 3000000, + "pairs": 2625000, "failed_pct": 0.04, - "median_in": 10.44, - "p90_in": 47.8, - "scale_error_pct": -0.57 + "median_in": 9.22, + "p90_in": 37.35, + "scale_error_pct": -0.37 } }, "tape_calibration_success_pct": 100.0, - "groups": 8, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 6.31, - 14.13 + 5.94, + 15.27 ], "p90_in": [ - 16.47, - 64.4 + 15.91, + 68.78 ] }, "3-10m": { "median_in": [ - 15.81, - 37.1 + 14.29, + 37.11 ], "p90_in": [ - 45.84, - 124.61 + 42.82, + 133.96 ] } } @@ -12655,58 +12655,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 0.05, - "median_in": 9.22, - "p90_in": 26.99, - "scale_error_pct": -9.17 + "pairs": 105000, + "failed_pct": 0.03, + "median_in": 9.04, + "p90_in": 27.14, + "scale_error_pct": -8.89 }, "3-10m": { - "pairs": 120000, - "failed_pct": 0.05, - "median_in": 26.53, - "p90_in": 63.98, - "scale_error_pct": -9.96 + "pairs": 105000, + "failed_pct": 0.04, + "median_in": 24.69, + "p90_in": 59.57, + "scale_error_pct": -8.95 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 0.05, - "median_in": 4.03, - "p90_in": 20.04, - "scale_error_pct": -0.02 + "pairs": 2625000, + "failed_pct": 0.03, + "median_in": 3.56, + "p90_in": 16.04, + "scale_error_pct": -0.05 }, "3-10m": { - "pairs": 3000000, - "failed_pct": 0.05, - "median_in": 10.76, - "p90_in": 49.8, - "scale_error_pct": 0.13 + "pairs": 2625000, + "failed_pct": 0.04, + "median_in": 9.41, + "p90_in": 39.31, + "scale_error_pct": 0.42 } }, "tape_calibration_success_pct": 100.0, - "groups": 8, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 5.52, - 13.95 + 4.95, + 14.78 ], "p90_in": [ - 16.26, - 55.85 + 15.32, + 60.31 ] }, "3-10m": { "median_in": [ - 14.52, - 35.18 + 12.23, + 34.45 ], "p90_in": [ - 42.66, - 114.76 + 40.13, + 126.13 ] } } @@ -12714,58 +12714,58 @@ "edges": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 0.11, - "median_in": 10.68, - "p90_in": 30.59, - "scale_error_pct": -8.3 + "pairs": 105000, + "failed_pct": 0.13, + "median_in": 10.1, + "p90_in": 25.52, + "scale_error_pct": -9.73 }, "3-10m": { - "pairs": 119080, - "failed_pct": 0.06, - "median_in": 23.72, - "p90_in": 66.59, - "scale_error_pct": -9.05 + "pairs": 104080, + "failed_pct": 0.07, + "median_in": 22.03, + "p90_in": 59.98, + "scale_error_pct": -9.45 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 0.11, - "median_in": 5.8, - "p90_in": 32.64, - "scale_error_pct": 0.45 + "pairs": 2625000, + "failed_pct": 0.13, + "median_in": 4.88, + "p90_in": 20.35, + "scale_error_pct": -0.54 }, "3-10m": { - "pairs": 2977000, - "failed_pct": 0.06, - "median_in": 10.42, - "p90_in": 57.64, - "scale_error_pct": 0.26 + "pairs": 2602000, + "failed_pct": 0.07, + "median_in": 8.83, + "p90_in": 35.11, + "scale_error_pct": -0.05 } }, "tape_calibration_success_pct": 100.0, - "groups": 8, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 7.4, - 15.59 + 6.65, + 14.95 ], "p90_in": [ - 20.04, - 63.77 + 17.58, + 44.37 ] }, "3-10m": { "median_in": [ - 12.72, - 35.11 + 11.29, + 34.15 ], "p90_in": [ - 41.51, - 99.58 + 36.51, + 100.97 ] } } @@ -12793,16 +12793,16 @@ "1-3m": { "pairs": 2625000, "failed_pct": 0.57, - "median_in": 4.14, - "p90_in": 15.6, + "median_in": 4.12, + "p90_in": 15.56, "scale_error_pct": -0.34 }, "3-10m": { "pairs": 2625000, "failed_pct": 0.59, - "median_in": 8.56, - "p90_in": 33.87, - "scale_error_pct": -0.14 + "median_in": 8.53, + "p90_in": 33.79, + "scale_error_pct": -0.13 } }, "tape_calibration_success_pct": 99.4, @@ -12835,33 +12835,33 @@ "none": { "1-3m": { "pairs": 105000, - "failed_pct": 0.0, - "median_in": 6.01, - "p90_in": 17.4, + "failed_pct": 0.01, + "median_in": 6.0, + "p90_in": 17.45, "scale_error_pct": -7.24 }, "3-10m": { "pairs": 105000, - "failed_pct": 0.0, - "median_in": 13.16, - "p90_in": 35.01, - "scale_error_pct": -6.53 + "failed_pct": 0.01, + "median_in": 13.18, + "p90_in": 35.15, + "scale_error_pct": -6.56 } }, "one_known_distance": { "1-3m": { "pairs": 2625000, - "failed_pct": 0.57, + "failed_pct": 0.59, "median_in": 3.56, "p90_in": 14.58, "scale_error_pct": -0.02 }, "3-10m": { "pairs": 2625000, - "failed_pct": 0.57, - "median_in": 7.93, - "p90_in": 33.84, - "scale_error_pct": 0.56 + "failed_pct": 0.58, + "median_in": 7.9, + "p90_in": 33.85, + "scale_error_pct": 0.54 } }, "tape_calibration_success_pct": 99.4, @@ -12870,22 +12870,22 @@ "interval_95": { "1-3m": { "median_in": [ - 3.78, - 9.83 + 3.8, + 9.79 ], "p90_in": [ - 10.35, - 21.41 + 10.3, + 21.42 ] }, "3-10m": { "median_in": [ - 9.05, - 22.71 + 9.15, + 22.77 ], "p90_in": [ - 20.81, - 44.89 + 21.07, + 45.1 ] } } @@ -12911,15 +12911,15 @@ "1-3m": { "pairs": 2625000, "failed_pct": 0.57, - "median_in": 4.5, - "p90_in": 18.18, - "scale_error_pct": 0.21 + "median_in": 4.48, + "p90_in": 18.16, + "scale_error_pct": 0.23 }, "3-10m": { "pairs": 2602000, "failed_pct": 0.59, - "median_in": 7.74, - "p90_in": 32.11, + "median_in": 7.7, + "p90_in": 32.07, "scale_error_pct": 0.35 } }, @@ -12933,7 +12933,7 @@ 9.53 ], "p90_in": [ - 11.7, + 11.69, 24.52 ] }, @@ -12956,58 +12956,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 0.06, - "median_in": 6.22, - "p90_in": 16.6, - "scale_error_pct": 2.08 + "pairs": 105000, + "failed_pct": 0.03, + "median_in": 5.56, + "p90_in": 14.04, + "scale_error_pct": 2.99 }, "3-10m": { - "pairs": 120000, - "failed_pct": 0.11, - "median_in": 17.4, - "p90_in": 42.11, - "scale_error_pct": 3.32 + "pairs": 105000, + "failed_pct": 0.1, + "median_in": 15.97, + "p90_in": 30.79, + "scale_error_pct": 4.37 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 0.06, - "median_in": 3.17, - "p90_in": 14.02, - "scale_error_pct": -0.17 + "pairs": 2625000, + "failed_pct": 0.03, + "median_in": 2.75, + "p90_in": 10.08, + "scale_error_pct": -0.18 }, "3-10m": { - "pairs": 3000000, - "failed_pct": 0.11, - "median_in": 7.05, - "p90_in": 37.41, - "scale_error_pct": -0.28 + "pairs": 2625000, + "failed_pct": 0.1, + "median_in": 6.04, + "p90_in": 22.75, + "scale_error_pct": -0.1 } }, "tape_calibration_success_pct": 100.0, - "groups": 8, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 3.92, - 9.29 + 3.63, + 8.42 ], "p90_in": [ - 11.26, - 21.93 + 10.64, + 16.9 ] }, "3-10m": { "median_in": [ - 12.85, - 24.16 + 12.18, + 21.91 ], "p90_in": [ - 26.39, - 75.09 + 25.41, + 33.39 ] } } @@ -13015,58 +13015,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 0.08, - "median_in": 6.06, - "p90_in": 16.26, - "scale_error_pct": 1.78 + "pairs": 105000, + "failed_pct": 0.07, + "median_in": 5.38, + "p90_in": 13.65, + "scale_error_pct": 3.17 }, "3-10m": { - "pairs": 120000, + "pairs": 105000, "failed_pct": 0.16, - "median_in": 17.53, - "p90_in": 45.48, - "scale_error_pct": 4.74 + "median_in": 16.16, + "p90_in": 30.74, + "scale_error_pct": 5.68 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 0.08, - "median_in": 2.8, - "p90_in": 13.37, - "scale_error_pct": -0.28 + "pairs": 2625000, + "failed_pct": 0.07, + "median_in": 2.42, + "p90_in": 9.42, + "scale_error_pct": -0.3 }, "3-10m": { - "pairs": 3000000, + "pairs": 2625000, "failed_pct": 0.16, - "median_in": 7.03, - "p90_in": 38.66, - "scale_error_pct": -0.0 + "median_in": 5.99, + "p90_in": 23.26, + "scale_error_pct": 0.22 } }, "tape_calibration_success_pct": 100.0, - "groups": 8, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 3.44, - 8.98 + 2.97, + 8.31 ], "p90_in": [ - 10.58, - 21.31 + 9.41, + 16.83 ] }, "3-10m": { "median_in": [ - 14.06, - 25.78 + 13.58, + 22.69 ], "p90_in": [ - 25.44, - 76.66 + 24.22, + 34.46 ] } } @@ -13074,58 +13074,58 @@ "edges": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 0.31, - "median_in": 7.63, - "p90_in": 30.51, - "scale_error_pct": 5.53 + "pairs": 105000, + "failed_pct": 0.32, + "median_in": 6.98, + "p90_in": 18.99, + "scale_error_pct": 4.55 }, "3-10m": { - "pairs": 119080, - "failed_pct": 0.17, - "median_in": 15.38, - "p90_in": 44.59, - "scale_error_pct": 4.35 + "pairs": 104080, + "failed_pct": 0.19, + "median_in": 14.06, + "p90_in": 27.72, + "scale_error_pct": 4.85 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 0.31, - "median_in": 4.37, - "p90_in": 32.85, - "scale_error_pct": 0.67 + "pairs": 2625000, + "failed_pct": 0.32, + "median_in": 3.64, + "p90_in": 16.4, + "scale_error_pct": -0.11 }, "3-10m": { - "pairs": 2977000, - "failed_pct": 0.17, - "median_in": 7.09, - "p90_in": 46.58, - "scale_error_pct": -0.04 + "pairs": 2602000, + "failed_pct": 0.19, + "median_in": 5.93, + "p90_in": 24.39, + "scale_error_pct": -0.11 } }, "tape_calibration_success_pct": 100.0, - "groups": 8, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 6.15, - 10.93 + 5.88, + 9.14 ], "p90_in": [ - 15.2, - 48.01 + 13.04, + 25.42 ] }, "3-10m": { "median_in": [ - 11.66, - 21.55 + 11.03, + 19.74 ], "p90_in": [ - 23.93, - 70.31 + 22.24, + 32.63 ] } } @@ -13135,58 +13135,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 0.01, - "median_in": 5.86, - "p90_in": 16.81, - "scale_error_pct": 2.38 + "pairs": 105000, + "failed_pct": 0.0, + "median_in": 5.24, + "p90_in": 13.49, + "scale_error_pct": 3.61 }, "3-10m": { - "pairs": 120000, + "pairs": 105000, "failed_pct": 0.03, - "median_in": 16.41, - "p90_in": 49.69, - "scale_error_pct": 3.52 + "median_in": 14.69, + "p90_in": 28.86, + "scale_error_pct": 4.52 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 1.01, - "median_in": 3.21, - "p90_in": 15.21, - "scale_error_pct": -0.28 + "pairs": 2625000, + "failed_pct": 1.14, + "median_in": 2.8, + "p90_in": 10.84, + "scale_error_pct": -0.31 }, "3-10m": { - "pairs": 3000000, - "failed_pct": 1.03, - "median_in": 6.98, - "p90_in": 39.5, - "scale_error_pct": -0.1 + "pairs": 2625000, + "failed_pct": 1.17, + "median_in": 5.98, + "p90_in": 25.15, + "scale_error_pct": 0.04 } }, - "tape_calibration_success_pct": 99.0, - "groups": 8, + "tape_calibration_success_pct": 98.9, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 4.21, - 8.96 + 3.9, + 7.89 ], "p90_in": [ - 10.59, - 23.88 + 10.14, + 16.27 ] }, "3-10m": { "median_in": [ - 11.64, - 23.36 + 11.0, + 19.76 ], "p90_in": [ - 25.44, - 81.86 + 23.73, + 33.61 ] } } @@ -13194,58 +13194,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 120000, + "pairs": 105000, "failed_pct": 0.02, - "median_in": 5.6, - "p90_in": 16.17, - "scale_error_pct": 1.34 + "median_in": 4.91, + "p90_in": 12.5, + "scale_error_pct": 3.28 }, "3-10m": { - "pairs": 120000, - "failed_pct": 0.05, - "median_in": 16.52, - "p90_in": 52.43, - "scale_error_pct": 4.21 + "pairs": 105000, + "failed_pct": 0.04, + "median_in": 14.94, + "p90_in": 29.77, + "scale_error_pct": 5.31 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 1.02, - "median_in": 2.89, - "p90_in": 14.73, - "scale_error_pct": -0.31 + "pairs": 2625000, + "failed_pct": 1.17, + "median_in": 2.47, + "p90_in": 10.3, + "scale_error_pct": -0.33 }, "3-10m": { - "pairs": 3000000, - "failed_pct": 1.05, - "median_in": 7.07, - "p90_in": 41.18, - "scale_error_pct": 0.02 + "pairs": 2625000, + "failed_pct": 1.18, + "median_in": 6.01, + "p90_in": 28.96, + "scale_error_pct": 0.17 } }, - "tape_calibration_success_pct": 99.0, - "groups": 8, + "tape_calibration_success_pct": 98.9, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 3.35, - 8.95 + 2.93, + 7.97 ], "p90_in": [ - 9.63, - 23.07 + 8.54, + 15.46 ] }, "3-10m": { "median_in": [ - 11.71, - 24.76 + 11.16, + 20.44 ], "p90_in": [ - 23.95, - 84.99 + 20.73, + 35.43 ] } } @@ -13253,58 +13253,58 @@ "edges": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 0.11, - "median_in": 6.43, - "p90_in": 21.37, - "scale_error_pct": 4.83 + "pairs": 105000, + "failed_pct": 0.13, + "median_in": 6.06, + "p90_in": 17.28, + "scale_error_pct": 4.62 }, "3-10m": { - "pairs": 119080, - "failed_pct": 0.06, - "median_in": 13.36, - "p90_in": 37.8, - "scale_error_pct": 4.8 + "pairs": 104080, + "failed_pct": 0.07, + "median_in": 12.35, + "p90_in": 26.73, + "scale_error_pct": 5.18 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 1.11, - "median_in": 4.71, - "p90_in": 26.99, - "scale_error_pct": 0.66 + "pairs": 2625000, + "failed_pct": 1.27, + "median_in": 3.93, + "p90_in": 15.85, + "scale_error_pct": -0.21 }, "3-10m": { - "pairs": 2977000, - "failed_pct": 1.07, - "median_in": 7.88, - "p90_in": 47.73, - "scale_error_pct": 0.84 + "pairs": 2602000, + "failed_pct": 1.23, + "median_in": 6.69, + "p90_in": 25.52, + "scale_error_pct": 0.62 } }, - "tape_calibration_success_pct": 99.0, - "groups": 8, + "tape_calibration_success_pct": 98.9, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 5.25, - 8.63 + 5.12, + 8.03 ], "p90_in": [ - 13.77, - 36.67 + 11.78, + 21.94 ] }, "3-10m": { "median_in": [ - 10.61, - 18.65 + 9.8, + 15.56 ], "p90_in": [ - 21.34, - 66.5 + 18.01, + 31.3 ] } } @@ -13333,15 +13333,15 @@ "pairs": 2625000, "failed_pct": 0.57, "median_in": 3.58, - "p90_in": 13.43, - "scale_error_pct": -0.49 + "p90_in": 13.34, + "scale_error_pct": -0.46 }, "3-10m": { "pairs": 2625000, "failed_pct": 0.59, - "median_in": 7.23, - "p90_in": 29.96, - "scale_error_pct": -0.28 + "median_in": 7.21, + "p90_in": 29.87, + "scale_error_pct": -0.25 } }, "tape_calibration_success_pct": 99.4, @@ -13374,33 +13374,33 @@ "none": { "1-3m": { "pairs": 105000, - "failed_pct": 0.0, - "median_in": 4.88, - "p90_in": 12.33, - "scale_error_pct": -0.65 + "failed_pct": 0.01, + "median_in": 4.85, + "p90_in": 12.39, + "scale_error_pct": -0.67 }, "3-10m": { "pairs": 105000, - "failed_pct": 0.0, - "median_in": 13.39, - "p90_in": 24.41, - "scale_error_pct": 2.87 + "failed_pct": 0.01, + "median_in": 13.32, + "p90_in": 24.33, + "scale_error_pct": 2.89 } }, "one_known_distance": { "1-3m": { "pairs": 2625000, - "failed_pct": 0.57, - "median_in": 3.35, - "p90_in": 13.6, - "scale_error_pct": -1.03 + "failed_pct": 0.59, + "median_in": 3.34, + "p90_in": 13.68, + "scale_error_pct": -1.01 }, "3-10m": { "pairs": 2625000, - "failed_pct": 0.57, - "median_in": 7.73, - "p90_in": 30.53, - "scale_error_pct": -0.16 + "failed_pct": 0.58, + "median_in": 7.68, + "p90_in": 30.35, + "scale_error_pct": -0.15 } }, "tape_calibration_success_pct": 99.4, @@ -13409,22 +13409,22 @@ "interval_95": { "1-3m": { "median_in": [ - 3.08, - 7.07 + 3.07, + 7.05 ], "p90_in": [ - 9.55, - 14.44 + 9.79, + 14.45 ] }, "3-10m": { "median_in": [ - 10.43, + 10.56, 17.18 ], "p90_in": [ - 19.82, - 28.67 + 19.73, + 28.49 ] } } @@ -13450,16 +13450,16 @@ "1-3m": { "pairs": 2625000, "failed_pct": 0.57, - "median_in": 4.72, - "p90_in": 16.39, - "scale_error_pct": 0.23 + "median_in": 4.7, + "p90_in": 16.4, + "scale_error_pct": 0.28 }, "3-10m": { "pairs": 2602000, "failed_pct": 0.59, - "median_in": 7.31, - "p90_in": 28.26, - "scale_error_pct": 0.83 + "median_in": 7.28, + "p90_in": 28.29, + "scale_error_pct": 0.88 } }, "tape_calibration_success_pct": 99.4, @@ -13495,58 +13495,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 12.56, - "median_in": 3.82, - "p90_in": Infinity, - "scale_error_pct": -2.03 + "pairs": 105000, + "failed_pct": 14.31, + "median_in": 3.45, + "p90_in": "failed", + "scale_error_pct": -2.0 }, "3-10m": { - "pairs": 120000, - "failed_pct": 12.61, - "median_in": 10.35, - "p90_in": Infinity, - "scale_error_pct": -1.75 + "pairs": 105000, + "failed_pct": 14.39, + "median_in": 9.04, + "p90_in": "failed", + "scale_error_pct": -1.41 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 12.56, - "median_in": 3.54, - "p90_in": Infinity, - "scale_error_pct": -0.07 + "pairs": 2625000, + "failed_pct": 14.31, + "median_in": 3.07, + "p90_in": "failed", + "scale_error_pct": -0.08 }, "3-10m": { - "pairs": 3000000, - "failed_pct": 12.61, - "median_in": 8.89, - "p90_in": Infinity, - "scale_error_pct": 0.14 + "pairs": 2625000, + "failed_pct": 14.39, + "median_in": 7.46, + "p90_in": "failed", + "scale_error_pct": 0.35 } }, - "tape_calibration_success_pct": 87.5, - "groups": 8, + "tape_calibration_success_pct": 85.7, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 2.36, - 7.71 + 2.16, + 8.34 ], "p90_in": [ - 7.7, - Infinity + 6.94, + "failed" ] }, "3-10m": { "median_in": [ - 5.92, - 24.73 + 5.36, + 24.38 ], "p90_in": [ - 18.76, - Infinity + 15.68, + "failed" ] } } @@ -13554,58 +13554,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 12.58, - "median_in": 3.04, - "p90_in": Infinity, - "scale_error_pct": -2.0 + "pairs": 105000, + "failed_pct": 14.35, + "median_in": 2.7, + "p90_in": "failed", + "scale_error_pct": -1.96 }, "3-10m": { - "pairs": 120000, - "failed_pct": 12.66, - "median_in": 10.34, - "p90_in": Infinity, - "scale_error_pct": -0.5 + "pairs": 105000, + "failed_pct": 14.44, + "median_in": 8.85, + "p90_in": "failed", + "scale_error_pct": -0.13 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 12.58, - "median_in": 3.1, - "p90_in": Infinity, + "pairs": 2625000, + "failed_pct": 14.35, + "median_in": 2.64, + "p90_in": "failed", "scale_error_pct": 0.16 }, "3-10m": { - "pairs": 3000000, - "failed_pct": 12.66, - "median_in": 9.24, - "p90_in": Infinity, - "scale_error_pct": 0.93 + "pairs": 2625000, + "failed_pct": 14.44, + "median_in": 7.67, + "p90_in": "failed", + "scale_error_pct": 1.25 } }, - "tape_calibration_success_pct": 87.5, - "groups": 8, + "tape_calibration_success_pct": 85.7, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 1.9, - 7.01 + 1.77, + 7.42 ], "p90_in": [ - 6.48, - Infinity + 4.92, + "failed" ] }, "3-10m": { "median_in": [ - 4.6, - 26.43 + 4.07, + 25.86 ], "p90_in": [ - 20.19, - Infinity + 15.26, + "failed" ] } } @@ -13613,58 +13613,58 @@ "edges": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 12.81, - "median_in": 4.83, - "p90_in": Infinity, - "scale_error_pct": -0.78 + "pairs": 105000, + "failed_pct": 14.61, + "median_in": 3.92, + "p90_in": "failed", + "scale_error_pct": -1.6 }, "3-10m": { - "pairs": 119080, - "failed_pct": 11.99, - "median_in": 8.46, - "p90_in": Infinity, - "scale_error_pct": -1.28 + "pairs": 104080, + "failed_pct": 13.72, + "median_in": 6.97, + "p90_in": "failed", + "scale_error_pct": -1.51 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 12.81, - "median_in": 5.31, - "p90_in": Infinity, - "scale_error_pct": 0.62 + "pairs": 2625000, + "failed_pct": 14.61, + "median_in": 4.16, + "p90_in": "failed", + "scale_error_pct": -0.2 }, "3-10m": { - "pairs": 2977000, - "failed_pct": 11.99, - "median_in": 9.18, - "p90_in": Infinity, - "scale_error_pct": 0.14 + "pairs": 2602000, + "failed_pct": 13.72, + "median_in": 7.55, + "p90_in": "failed", + "scale_error_pct": 0.02 } }, - "tape_calibration_success_pct": 87.5, - "groups": 8, + "tape_calibration_success_pct": 85.7, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 2.58, - 24.63 + 2.37, + 12.64 ], "p90_in": [ - 11.32, - Infinity + 8.49, + "failed" ] }, "3-10m": { "median_in": [ - 4.6, - 29.92 + 4.27, + 21.01 ], "p90_in": [ - 17.98, - Infinity + 13.51, + "failed" ] } } @@ -13674,58 +13674,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 2.51, - "median_in": 3.14, - "p90_in": 12.52, - "scale_error_pct": -2.72 + "pairs": 105000, + "failed_pct": 2.86, + "median_in": 2.79, + "p90_in": 10.01, + "scale_error_pct": -2.49 }, "3-10m": { - "pairs": 120000, - "failed_pct": 2.54, - "median_in": 6.59, - "p90_in": 32.29, - "scale_error_pct": -2.44 + "pairs": 105000, + "failed_pct": 2.89, + "median_in": 5.65, + "p90_in": 19.85, + "scale_error_pct": -2.08 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 2.51, - "median_in": 3.33, - "p90_in": 16.07, - "scale_error_pct": -0.17 + "pairs": 2625000, + "failed_pct": 2.86, + "median_in": 2.92, + "p90_in": 11.91, + "scale_error_pct": -0.21 }, "3-10m": { - "pairs": 3000000, - "failed_pct": 2.54, - "median_in": 7.59, - "p90_in": 40.43, - "scale_error_pct": -0.07 + "pairs": 2625000, + "failed_pct": 2.89, + "median_in": 6.58, + "p90_in": 28.83, + "scale_error_pct": 0.08 } }, - "tape_calibration_success_pct": 97.5, - "groups": 8, + "tape_calibration_success_pct": 97.1, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 2.19, - 4.88 + 2.02, + 3.98 ], "p90_in": [ - 6.92, - Infinity + 6.19, + "failed" ] }, "3-10m": { "median_in": [ - 4.07, - 11.77 + 3.66, + 8.96 ], "p90_in": [ - 14.04, - Infinity + 11.64, + "failed" ] } } @@ -13733,58 +13733,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 2.54, - "median_in": 2.76, - "p90_in": 12.37, - "scale_error_pct": -2.73 + "pairs": 105000, + "failed_pct": 2.89, + "median_in": 2.43, + "p90_in": 10.4, + "scale_error_pct": -2.52 }, "3-10m": { - "pairs": 120000, - "failed_pct": 2.55, - "median_in": 6.86, - "p90_in": 35.19, - "scale_error_pct": -2.24 + "pairs": 105000, + "failed_pct": 2.9, + "median_in": 5.59, + "p90_in": 21.4, + "scale_error_pct": -1.85 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 2.54, - "median_in": 3.0, - "p90_in": 15.37, - "scale_error_pct": 0.0 + "pairs": 2625000, + "failed_pct": 2.89, + "median_in": 2.57, + "p90_in": 11.27, + "scale_error_pct": -0.04 }, "3-10m": { - "pairs": 3000000, - "failed_pct": 2.55, - "median_in": 7.79, - "p90_in": 40.44, - "scale_error_pct": 0.17 + "pairs": 2625000, + "failed_pct": 2.9, + "median_in": 6.71, + "p90_in": 28.98, + "scale_error_pct": 0.38 } }, - "tape_calibration_success_pct": 97.5, - "groups": 8, + "tape_calibration_success_pct": 97.1, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 1.82, - 4.43 + 1.65, + 3.57 ], "p90_in": [ - 6.39, - Infinity + 5.15, + "failed" ] }, "3-10m": { "median_in": [ - 3.84, - 13.14 + 3.44, + 9.99 ], "p90_in": [ - 13.72, - Infinity + 11.71, + "failed" ] } } @@ -13792,58 +13792,58 @@ "edges": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 2.61, - "median_in": 3.89, - "p90_in": 27.32, - "scale_error_pct": -2.29 + "pairs": 105000, + "failed_pct": 2.99, + "median_in": 3.3, + "p90_in": 15.18, + "scale_error_pct": -2.62 }, "3-10m": { - "pairs": 119080, - "failed_pct": 2.58, - "median_in": 7.45, - "p90_in": 42.55, - "scale_error_pct": -2.43 + "pairs": 104080, + "failed_pct": 2.95, + "median_in": 6.21, + "p90_in": 21.82, + "scale_error_pct": -2.48 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 2.61, - "median_in": 4.79, - "p90_in": 33.99, - "scale_error_pct": 0.2 + "pairs": 2625000, + "failed_pct": 2.99, + "median_in": 3.94, + "p90_in": 17.7, + "scale_error_pct": -0.56 }, "3-10m": { - "pairs": 2977000, - "failed_pct": 2.58, - "median_in": 8.03, - "p90_in": 55.25, - "scale_error_pct": -0.05 + "pairs": 2602000, + "failed_pct": 2.95, + "median_in": 6.82, + "p90_in": 27.71, + "scale_error_pct": -0.3 } }, - "tape_calibration_success_pct": 97.5, - "groups": 8, + "tape_calibration_success_pct": 97.1, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 2.56, - 7.69 + 2.44, + 5.14 ], "p90_in": [ - 11.67, - Infinity + 10.13, + "failed" ] }, "3-10m": { "median_in": [ - 4.53, - 13.33 + 4.29, + 10.0 ], "p90_in": [ - 16.46, - Infinity + 14.63, + "failed" ] } } @@ -13872,15 +13872,15 @@ "pairs": 2625000, "failed_pct": 0.57, "median_in": 2.59, - "p90_in": 9.88, + "p90_in": 9.79, "scale_error_pct": -0.42 }, "3-10m": { "pairs": 2625000, "failed_pct": 0.59, - "median_in": 5.67, - "p90_in": 23.04, - "scale_error_pct": -0.14 + "median_in": 5.65, + "p90_in": 22.86, + "scale_error_pct": -0.15 } }, "tape_calibration_success_pct": 99.4, @@ -13913,33 +13913,33 @@ "none": { "1-3m": { "pairs": 105000, - "failed_pct": 0.0, - "median_in": 2.42, - "p90_in": 7.4, - "scale_error_pct": -2.88 + "failed_pct": 0.01, + "median_in": 2.43, + "p90_in": 7.41, + "scale_error_pct": -2.9 }, "3-10m": { "pairs": 105000, - "failed_pct": 0.0, - "median_in": 5.63, - "p90_in": 14.95, - "scale_error_pct": -2.72 + "failed_pct": 0.01, + "median_in": 5.61, + "p90_in": 15.1, + "scale_error_pct": -2.74 } }, "one_known_distance": { "1-3m": { "pairs": 2625000, - "failed_pct": 0.57, - "median_in": 2.26, - "p90_in": 9.37, - "scale_error_pct": -0.2 + "failed_pct": 0.59, + "median_in": 2.24, + "p90_in": 9.31, + "scale_error_pct": -0.22 }, "3-10m": { "pairs": 2625000, - "failed_pct": 0.57, - "median_in": 5.63, - "p90_in": 22.56, - "scale_error_pct": 0.09 + "failed_pct": 0.58, + "median_in": 5.62, + "p90_in": 22.4, + "scale_error_pct": 0.07 } }, "tape_calibration_success_pct": 99.4, @@ -13948,22 +13948,22 @@ "interval_95": { "1-3m": { "median_in": [ - 1.83, - 3.34 + 1.84, + 3.32 ], "p90_in": [ - 4.8, - 9.74 + 4.85, + 9.7 ] }, "3-10m": { "median_in": [ - 4.37, - 7.24 + 4.34, + 7.21 ], "p90_in": [ - 10.23, - 21.39 + 10.3, + 21.45 ] } } @@ -13989,15 +13989,15 @@ "1-3m": { "pairs": 2625000, "failed_pct": 0.57, - "median_in": 3.51, - "p90_in": 14.49, - "scale_error_pct": -0.33 + "median_in": 3.5, + "p90_in": 14.47, + "scale_error_pct": -0.32 }, "3-10m": { "pairs": 2602000, "failed_pct": 0.59, - "median_in": 5.6, - "p90_in": 23.37, + "median_in": 5.58, + "p90_in": 23.32, "scale_error_pct": -0.15 } }, @@ -14036,58 +14036,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 16.0, - "median_in": 14.69, - "p90_in": Infinity, - "scale_error_pct": 13.86 + "pairs": 21000, + "failed_pct": 9.12, + "median_in": 12.76, + "p90_in": 36.37, + "scale_error_pct": 13.15 }, "3-10m": { - "pairs": 24000, - "failed_pct": 17.89, - "median_in": 29.52, - "p90_in": Infinity, - "scale_error_pct": 14.2 + "pairs": 21000, + "failed_pct": 10.3, + "median_in": 25.89, + "p90_in": "failed", + "scale_error_pct": 13.71 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 26.26, - "median_in": 4.65, - "p90_in": Infinity, - "scale_error_pct": 0.06 + "pairs": 525000, + "failed_pct": 17.43, + "median_in": 3.76, + "p90_in": "failed", + "scale_error_pct": 0.24 }, "3-10m": { - "pairs": 600000, - "failed_pct": 27.58, - "median_in": 10.06, - "p90_in": Infinity, - "scale_error_pct": -0.03 + "pairs": 525000, + "failed_pct": 18.53, + "median_in": 7.72, + "p90_in": "failed", + "scale_error_pct": 0.24 } }, - "tape_calibration_success_pct": 84.0, - "groups": 8, + "tape_calibration_success_pct": 90.3, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 8.96, - 25.21 + 7.98, + 18.19 ], "p90_in": [ - 27.07, - Infinity + 23.43, + "failed" ] }, "3-10m": { "median_in": [ - 18.84, - 45.83 + 16.71, + 34.85 ], "p90_in": [ - 50.93, - Infinity + 46.93, + "failed" ] } } @@ -14095,58 +14095,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 20.81, - "median_in": 15.17, - "p90_in": Infinity, - "scale_error_pct": 11.93 + "pairs": 18000, + "failed_pct": 13.51, + "median_in": 12.61, + "p90_in": "failed", + "scale_error_pct": 10.88 }, "3-10m": { - "pairs": 21000, - "failed_pct": 26.6, - "median_in": 29.91, - "p90_in": Infinity, - "scale_error_pct": 11.45 + "pairs": 18000, + "failed_pct": 20.23, + "median_in": 24.66, + "p90_in": "failed", + "scale_error_pct": 10.09 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 32.04, - "median_in": 5.39, - "p90_in": Infinity, - "scale_error_pct": 0.57 + "pairs": 450000, + "failed_pct": 22.65, + "median_in": 4.48, + "p90_in": "failed", + "scale_error_pct": 0.86 }, "3-10m": { - "pairs": 525000, - "failed_pct": 36.89, - "median_in": 14.0, - "p90_in": Infinity, - "scale_error_pct": 1.27 + "pairs": 450000, + "failed_pct": 29.16, + "median_in": 10.56, + "p90_in": "failed", + "scale_error_pct": 1.39 } }, - "tape_calibration_success_pct": 81.7, - "groups": 7, + "tape_calibration_success_pct": 88.7, + "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 8.38, - 41.5 + 7.64, + 19.44 ], "p90_in": [ - 31.77, - Infinity + 23.71, + "failed" ] }, "3-10m": { "median_in": [ - 18.1, - 126.58 + 15.73, + 45.63 ], "p90_in": [ - Infinity, - Infinity + 49.06, + "failed" ] } } @@ -14154,58 +14154,58 @@ "edges": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 80.74, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 30.08 + "pairs": 18000, + "failed_pct": 82.03, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 26.13 }, "3-10m": { - "pairs": 20786, - "failed_pct": 77.7, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 29.9 + "pairs": 17786, + "failed_pct": 84.11, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 6.46 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 84.26, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 4.31 + "pairs": 450000, + "failed_pct": 83.3, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 4.98 }, "3-10m": { - "pairs": 519650, - "failed_pct": 83.8, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 1.42 + "pairs": 444650, + "failed_pct": 85.48, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 1.47 } }, - "tape_calibration_success_pct": 81.7, - "groups": 7, + "tape_calibration_success_pct": 88.7, + "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -14215,58 +14215,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 9.96, - "median_in": 15.66, - "p90_in": 69.59, - "scale_error_pct": 17.14 + "pairs": 21000, + "failed_pct": 4.94, + "median_in": 14.69, + "p90_in": 31.34, + "scale_error_pct": 16.68 }, "3-10m": { - "pairs": 24000, - "failed_pct": 12.5, - "median_in": 31.5, - "p90_in": Infinity, - "scale_error_pct": 16.77 + "pairs": 21000, + "failed_pct": 5.67, + "median_in": 29.57, + "p90_in": 59.21, + "scale_error_pct": 16.48 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 18.19, - "median_in": 3.41, - "p90_in": Infinity, - "scale_error_pct": 0.22 + "pairs": 525000, + "failed_pct": 11.84, + "median_in": 2.83, + "p90_in": "failed", + "scale_error_pct": 0.33 }, "3-10m": { - "pairs": 600000, - "failed_pct": 20.16, - "median_in": 7.12, - "p90_in": Infinity, - "scale_error_pct": 0.19 + "pairs": 525000, + "failed_pct": 12.61, + "median_in": 5.69, + "p90_in": "failed", + "scale_error_pct": 0.38 } }, - "tape_calibration_success_pct": 89.5, - "groups": 8, + "tape_calibration_success_pct": 92.6, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 11.95, - 21.27 + 11.41, + 18.89 ], "p90_in": [ - 23.09, - Infinity + 20.54, + 37.27 ] }, "3-10m": { "median_in": [ - 26.21, - 42.82 + 25.37, + 37.09 ], "p90_in": [ - 42.74, - Infinity + 38.64, + 68.56 ] } } @@ -14274,58 +14274,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 13.92, - "median_in": 16.83, - "p90_in": Infinity, - "scale_error_pct": 17.21 + "pairs": 18000, + "failed_pct": 8.43, + "median_in": 15.58, + "p90_in": 39.85, + "scale_error_pct": 16.96 }, "3-10m": { - "pairs": 21000, - "failed_pct": 20.26, - "median_in": 29.51, - "p90_in": Infinity, - "scale_error_pct": 16.42 + "pairs": 18000, + "failed_pct": 13.83, + "median_in": 27.77, + "p90_in": "failed", + "scale_error_pct": 16.18 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 23.14, - "median_in": 4.46, - "p90_in": Infinity, - "scale_error_pct": 0.95 + "pairs": 450000, + "failed_pct": 16.35, + "median_in": 3.8, + "p90_in": "failed", + "scale_error_pct": 1.15 }, "3-10m": { - "pairs": 525000, - "failed_pct": 28.53, - "median_in": 9.95, - "p90_in": Infinity, - "scale_error_pct": 0.92 + "pairs": 450000, + "failed_pct": 21.68, + "median_in": 7.96, + "p90_in": "failed", + "scale_error_pct": 0.94 } }, - "tape_calibration_success_pct": 88.0, - "groups": 7, + "tape_calibration_success_pct": 91.3, + "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 12.64, - 24.18 + 11.69, + 20.83 ], "p90_in": [ - 28.59, - Infinity + 20.56, + "failed" ] }, "3-10m": { "median_in": [ - 24.76, - 52.5 + 24.1, + 45.26 ], "p90_in": [ - 59.02, - Infinity + 39.44, + "failed" ] } } @@ -14333,58 +14333,58 @@ "edges": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 68.46, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 29.88 + "pairs": 18000, + "failed_pct": 70.52, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 27.78 }, "3-10m": { - "pairs": 20786, - "failed_pct": 72.42, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 25.63 + "pairs": 17786, + "failed_pct": 79.51, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 11.04 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 71.77, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 4.5 + "pairs": 450000, + "failed_pct": 72.05, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 5.11 }, "3-10m": { - "pairs": 519650, - "failed_pct": 76.92, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 0.36 + "pairs": 444650, + "failed_pct": 81.07, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 0.63 } }, - "tape_calibration_success_pct": 88.0, - "groups": 7, + "tape_calibration_success_pct": 91.3, + "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -14396,58 +14396,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 16.0, - "median_in": 4.4, - "p90_in": Infinity, - "scale_error_pct": 3.5 + "pairs": 21000, + "failed_pct": 8.86, + "median_in": 3.56, + "p90_in": 25.7, + "scale_error_pct": 3.13 }, "3-10m": { - "pairs": 24000, - "failed_pct": 18.39, - "median_in": 9.89, - "p90_in": Infinity, - "scale_error_pct": 3.43 + "pairs": 21000, + "failed_pct": 10.31, + "median_in": 7.93, + "p90_in": "failed", + "scale_error_pct": 3.15 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 25.97, - "median_in": 4.25, - "p90_in": Infinity, - "scale_error_pct": -0.02 + "pairs": 525000, + "failed_pct": 16.79, + "median_in": 3.47, + "p90_in": "failed", + "scale_error_pct": 0.17 }, "3-10m": { - "pairs": 600000, - "failed_pct": 27.59, - "median_in": 9.17, - "p90_in": Infinity, - "scale_error_pct": -0.33 + "pairs": 525000, + "failed_pct": 18.15, + "median_in": 7.04, + "p90_in": "failed", + "scale_error_pct": -0.07 } }, - "tape_calibration_success_pct": 84.0, - "groups": 8, + "tape_calibration_success_pct": 90.9, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 2.19, - 13.71 + 1.88, + 7.24 ], "p90_in": [ - 9.97, - Infinity + 8.28, + "failed" ] }, "3-10m": { "median_in": [ - 4.41, - 30.05 + 3.37, + 15.85 ], "p90_in": [ - 19.0, - Infinity + 15.86, + "failed" ] } } @@ -14455,58 +14455,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 21.02, - "median_in": 4.61, - "p90_in": Infinity, - "scale_error_pct": 3.37 + "pairs": 18000, + "failed_pct": 12.92, + "median_in": 3.41, + "p90_in": "failed", + "scale_error_pct": 3.1 }, "3-10m": { - "pairs": 21000, - "failed_pct": 27.45, - "median_in": 14.61, - "p90_in": Infinity, - "scale_error_pct": 4.15 + "pairs": 18000, + "failed_pct": 19.87, + "median_in": 8.63, + "p90_in": "failed", + "scale_error_pct": 3.97 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 31.77, - "median_in": 4.92, - "p90_in": Infinity, - "scale_error_pct": 0.71 + "pairs": 450000, + "failed_pct": 21.76, + "median_in": 4.08, + "p90_in": "failed", + "scale_error_pct": 0.97 }, "3-10m": { - "pairs": 525000, - "failed_pct": 36.94, - "median_in": 12.34, - "p90_in": Infinity, - "scale_error_pct": 1.49 + "pairs": 450000, + "failed_pct": 28.48, + "median_in": 9.31, + "p90_in": "failed", + "scale_error_pct": 1.53 } }, - "tape_calibration_success_pct": 81.7, - "groups": 7, + "tape_calibration_success_pct": 89.3, + "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 2.04, - 21.07 + 1.78, + 11.39 ], "p90_in": [ - 19.4, - Infinity + 5.5, + "failed" ] }, "3-10m": { "median_in": [ - 4.8, - 92.62 + 3.56, + 28.5 ], "p90_in": [ - Infinity, - Infinity + 37.64, + "failed" ] } } @@ -14514,58 +14514,58 @@ "edges": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 79.66, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 8.31 + "pairs": 18000, + "failed_pct": 81.46, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 5.27 }, "3-10m": { - "pairs": 20786, - "failed_pct": 77.49, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 11.23 + "pairs": 17786, + "failed_pct": 84.11, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 4.85 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 83.67, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 4.53 + "pairs": 450000, + "failed_pct": 82.7, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 5.16 }, "3-10m": { - "pairs": 519650, - "failed_pct": 84.03, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 1.06 + "pairs": 444650, + "failed_pct": 85.41, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 1.58 } }, - "tape_calibration_success_pct": 81.7, - "groups": 7, + "tape_calibration_success_pct": 89.3, + "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -14575,58 +14575,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 10.33, - "median_in": 4.14, - "p90_in": Infinity, - "scale_error_pct": 4.17 + "pairs": 21000, + "failed_pct": 5.38, + "median_in": 3.62, + "p90_in": 12.12, + "scale_error_pct": 3.99 }, "3-10m": { - "pairs": 24000, - "failed_pct": 13.24, - "median_in": 7.86, - "p90_in": Infinity, - "scale_error_pct": 3.5 + "pairs": 21000, + "failed_pct": 6.15, + "median_in": 6.81, + "p90_in": 26.07, + "scale_error_pct": 3.29 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 18.99, - "median_in": 3.64, - "p90_in": Infinity, - "scale_error_pct": 0.14 + "pairs": 525000, + "failed_pct": 12.76, + "median_in": 3.07, + "p90_in": "failed", + "scale_error_pct": 0.25 }, "3-10m": { - "pairs": 600000, - "failed_pct": 21.21, - "median_in": 7.94, - "p90_in": Infinity, - "scale_error_pct": -0.04 + "pairs": 525000, + "failed_pct": 13.56, + "median_in": 6.5, + "p90_in": "failed", + "scale_error_pct": 0.18 } }, - "tape_calibration_success_pct": 89.0, - "groups": 8, + "tape_calibration_success_pct": 92.0, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 2.34, - 6.66 + 2.13, + 5.28 ], "p90_in": [ - 8.29, - Infinity + 7.52, + 19.79 ] }, "3-10m": { "median_in": [ - 4.22, - 12.99 + 3.76, + 10.11 ], "p90_in": [ - 15.19, - Infinity + 14.15, + 42.18 ] } } @@ -14634,58 +14634,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 13.92, - "median_in": 3.9, - "p90_in": Infinity, - "scale_error_pct": 3.78 + "pairs": 18000, + "failed_pct": 8.36, + "median_in": 3.19, + "p90_in": 21.95, + "scale_error_pct": 3.63 }, "3-10m": { - "pairs": 21000, - "failed_pct": 19.92, - "median_in": 7.87, - "p90_in": Infinity, - "scale_error_pct": 3.61 + "pairs": 18000, + "failed_pct": 13.32, + "median_in": 6.15, + "p90_in": "failed", + "scale_error_pct": 3.43 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 23.62, - "median_in": 4.36, - "p90_in": Infinity, - "scale_error_pct": 1.05 + "pairs": 450000, + "failed_pct": 16.87, + "median_in": 3.68, + "p90_in": "failed", + "scale_error_pct": 1.24 }, "3-10m": { - "pairs": 525000, - "failed_pct": 28.65, - "median_in": 9.77, - "p90_in": Infinity, - "scale_error_pct": 1.45 + "pairs": 450000, + "failed_pct": 21.77, + "median_in": 7.99, + "p90_in": "failed", + "scale_error_pct": 1.49 } }, - "tape_calibration_success_pct": 87.4, - "groups": 7, + "tape_calibration_success_pct": 90.7, + "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 2.09, - 10.84 + 1.82, + 9.93 ], "p90_in": [ - 8.57, - Infinity + 4.85, + "failed" ] }, "3-10m": { "median_in": [ - 4.3, - 22.46 + 3.92, + 21.4 ], "p90_in": [ - 20.16, - Infinity + 15.85, + "failed" ] } } @@ -14693,58 +14693,58 @@ "edges": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 68.67, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 9.52 + "pairs": 18000, + "failed_pct": 70.99, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 7.34 }, "3-10m": { - "pairs": 20786, - "failed_pct": 71.4, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 6.21 + "pairs": 17786, + "failed_pct": 78.38, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 4.75 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 72.04, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 6.81 + "pairs": 450000, + "failed_pct": 72.54, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 7.35 }, "3-10m": { - "pairs": 519650, - "failed_pct": 75.96, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 2.87 + "pairs": 444650, + "failed_pct": 79.98, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 2.51 } }, - "tape_calibration_success_pct": 87.4, - "groups": 7, + "tape_calibration_success_pct": 90.7, + "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -14756,58 +14756,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 16.17, - "median_in": 4.76, - "p90_in": Infinity, - "scale_error_pct": -0.46 + "pairs": 21000, + "failed_pct": 8.85, + "median_in": 4.05, + "p90_in": 25.22, + "scale_error_pct": -0.79 }, "3-10m": { - "pairs": 24000, - "failed_pct": 18.61, - "median_in": 11.02, - "p90_in": Infinity, - "scale_error_pct": -0.88 + "pairs": 21000, + "failed_pct": 10.38, + "median_in": 8.07, + "p90_in": "failed", + "scale_error_pct": -1.14 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 26.2, - "median_in": 4.41, - "p90_in": Infinity, - "scale_error_pct": 0.03 + "pairs": 525000, + "failed_pct": 16.78, + "median_in": 3.59, + "p90_in": "failed", + "scale_error_pct": 0.23 }, "3-10m": { - "pairs": 600000, - "failed_pct": 27.83, - "median_in": 9.65, - "p90_in": Infinity, - "scale_error_pct": -0.29 + "pairs": 525000, + "failed_pct": 18.21, + "median_in": 7.35, + "p90_in": "failed", + "scale_error_pct": -0.01 } }, - "tape_calibration_success_pct": 83.5, - "groups": 8, + "tape_calibration_success_pct": 90.9, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 2.85, - 13.68 + 2.57, + 8.13 ], "p90_in": [ - 13.05, - Infinity + 8.09, + "failed" ] }, "3-10m": { "median_in": [ - 5.5, - 29.07 + 4.82, + 22.35 ], "p90_in": [ - 32.14, - Infinity + 19.49, + "failed" ] } } @@ -14815,58 +14815,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 21.2, - "median_in": 6.22, - "p90_in": Infinity, - "scale_error_pct": 1.35 + "pairs": 18000, + "failed_pct": 12.85, + "median_in": 4.79, + "p90_in": "failed", + "scale_error_pct": 0.29 }, "3-10m": { - "pairs": 21000, - "failed_pct": 27.68, - "median_in": 13.63, - "p90_in": Infinity, - "scale_error_pct": 3.17 + "pairs": 18000, + "failed_pct": 19.86, + "median_in": 9.64, + "p90_in": "failed", + "scale_error_pct": 2.74 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 32.0, - "median_in": 4.95, - "p90_in": Infinity, - "scale_error_pct": 0.66 + "pairs": 450000, + "failed_pct": 21.7, + "median_in": 4.09, + "p90_in": "failed", + "scale_error_pct": 0.94 }, "3-10m": { - "pairs": 525000, - "failed_pct": 37.12, - "median_in": 12.72, - "p90_in": Infinity, - "scale_error_pct": 1.46 + "pairs": 450000, + "failed_pct": 28.48, + "median_in": 9.61, + "p90_in": "failed", + "scale_error_pct": 1.5 } }, - "tape_calibration_success_pct": 81.1, - "groups": 7, + "tape_calibration_success_pct": 89.3, + "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 2.96, - 18.77 + 2.46, + 10.67 ], "p90_in": [ - 18.18, - Infinity + 7.48, + "failed" ] }, "3-10m": { "median_in": [ - 3.97, - 456.29 + 3.36, + 26.61 ], "p90_in": [ - Infinity, - Infinity + 34.27, + "failed" ] } } @@ -14874,58 +14874,58 @@ "edges": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 79.31, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 4.9 + "pairs": 18000, + "failed_pct": 81.13, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 4.45 }, "3-10m": { - "pairs": 20786, - "failed_pct": 77.27, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 5.38 + "pairs": 17786, + "failed_pct": 83.8, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 4.3 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 83.6, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 4.77 + "pairs": 450000, + "failed_pct": 82.42, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 5.46 }, "3-10m": { - "pairs": 519650, - "failed_pct": 84.16, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 1.09 + "pairs": 444650, + "failed_pct": 85.13, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 1.51 } }, - "tape_calibration_success_pct": 81.1, - "groups": 7, + "tape_calibration_success_pct": 89.3, + "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -14935,58 +14935,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 10.61, - "median_in": 4.97, - "p90_in": Infinity, - "scale_error_pct": -2.07 + "pairs": 21000, + "failed_pct": 5.62, + "median_in": 4.24, + "p90_in": 18.65, + "scale_error_pct": -2.43 }, "3-10m": { - "pairs": 24000, - "failed_pct": 13.42, - "median_in": 11.45, - "p90_in": Infinity, - "scale_error_pct": -3.12 + "pairs": 21000, + "failed_pct": 6.3, + "median_in": 9.92, + "p90_in": 33.81, + "scale_error_pct": -3.39 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 19.69, - "median_in": 3.68, - "p90_in": Infinity, - "scale_error_pct": 0.2 + "pairs": 525000, + "failed_pct": 13.5, + "median_in": 3.12, + "p90_in": "failed", + "scale_error_pct": 0.37 }, "3-10m": { - "pairs": 600000, - "failed_pct": 21.82, - "median_in": 7.89, - "p90_in": Infinity, - "scale_error_pct": -0.08 + "pairs": 525000, + "failed_pct": 14.22, + "median_in": 6.5, + "p90_in": "failed", + "scale_error_pct": 0.18 } }, - "tape_calibration_success_pct": 88.5, - "groups": 8, + "tape_calibration_success_pct": 91.4, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 2.31, - 9.67 + 1.93, + 8.34 ], "p90_in": [ - 10.58, - Infinity + 8.49, + 24.7 ] }, "3-10m": { "median_in": [ - 5.36, - 19.41 + 4.22, + 16.46 ], "p90_in": [ - 19.94, - Infinity + 16.58, + 44.04 ] } } @@ -14994,58 +14994,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 13.99, - "median_in": 6.38, - "p90_in": Infinity, - "scale_error_pct": -1.5 + "pairs": 18000, + "failed_pct": 8.36, + "median_in": 5.67, + "p90_in": 23.17, + "scale_error_pct": -2.47 }, "3-10m": { - "pairs": 21000, - "failed_pct": 19.77, - "median_in": 12.0, - "p90_in": Infinity, - "scale_error_pct": -2.25 + "pairs": 18000, + "failed_pct": 13.27, + "median_in": 10.48, + "p90_in": "failed", + "scale_error_pct": -2.8 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 24.12, - "median_in": 4.49, - "p90_in": Infinity, - "scale_error_pct": 1.1 + "pairs": 450000, + "failed_pct": 17.39, + "median_in": 3.82, + "p90_in": "failed", + "scale_error_pct": 1.4 }, "3-10m": { - "pairs": 525000, - "failed_pct": 28.91, - "median_in": 9.91, - "p90_in": Infinity, - "scale_error_pct": 1.71 + "pairs": 450000, + "failed_pct": 22.14, + "median_in": 8.29, + "p90_in": "failed", + "scale_error_pct": 1.82 } }, - "tape_calibration_success_pct": 86.9, - "groups": 7, + "tape_calibration_success_pct": 90.0, + "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 3.03, - 11.53 + 2.13, + 10.69 ], "p90_in": [ - 13.53, - Infinity + 8.74, + "failed" ] }, "3-10m": { "median_in": [ - 6.46, - 24.24 + 6.11, + 21.48 ], "p90_in": [ - 28.45, - Infinity + 15.46, + "failed" ] } } @@ -15053,58 +15053,58 @@ "edges": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 68.73, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 8.97 + "pairs": 18000, + "failed_pct": 70.96, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 4.81 }, "3-10m": { - "pairs": 20786, - "failed_pct": 71.29, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 5.03 + "pairs": 17786, + "failed_pct": 78.16, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 1.02 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 72.27, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 7.16 + "pairs": 450000, + "failed_pct": 72.73, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 7.81 }, "3-10m": { - "pairs": 519650, - "failed_pct": 75.86, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 2.81 + "pairs": 444650, + "failed_pct": 79.81, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 2.25 } }, - "tape_calibration_success_pct": 86.9, - "groups": 7, + "tape_calibration_success_pct": 90.0, + "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -15116,58 +15116,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 16.08, - "median_in": 3.98, - "p90_in": Infinity, - "scale_error_pct": 1.68 + "pairs": 21000, + "failed_pct": 8.88, + "median_in": 3.25, + "p90_in": 26.33, + "scale_error_pct": 1.4 }, "3-10m": { - "pairs": 24000, - "failed_pct": 18.41, - "median_in": 8.53, - "p90_in": Infinity, - "scale_error_pct": 1.69 + "pairs": 21000, + "failed_pct": 10.33, + "median_in": 7.04, + "p90_in": "failed", + "scale_error_pct": 1.41 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 26.01, - "median_in": 4.28, - "p90_in": Infinity, - "scale_error_pct": -0.0 + "pairs": 525000, + "failed_pct": 16.8, + "median_in": 3.5, + "p90_in": "failed", + "scale_error_pct": 0.19 }, "3-10m": { - "pairs": 600000, - "failed_pct": 27.61, - "median_in": 9.19, - "p90_in": Infinity, - "scale_error_pct": -0.35 + "pairs": 525000, + "failed_pct": 18.17, + "median_in": 7.08, + "p90_in": "failed", + "scale_error_pct": -0.08 } }, - "tape_calibration_success_pct": 84.0, - "groups": 8, + "tape_calibration_success_pct": 90.9, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 2.38, - 14.09 + 2.13, + 6.8 ], "p90_in": [ - 9.01, - Infinity + 7.2, + "failed" ] }, "3-10m": { "median_in": [ - 4.66, - 31.12 + 4.17, + 16.17 ], "p90_in": [ - 20.54, - Infinity + 13.37, + "failed" ] } } @@ -15175,58 +15175,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 21.15, - "median_in": 4.77, - "p90_in": Infinity, - "scale_error_pct": 1.63 + "pairs": 18000, + "failed_pct": 12.91, + "median_in": 3.64, + "p90_in": "failed", + "scale_error_pct": 1.36 }, "3-10m": { - "pairs": 21000, - "failed_pct": 27.48, - "median_in": 12.11, - "p90_in": Infinity, - "scale_error_pct": 2.72 + "pairs": 18000, + "failed_pct": 19.83, + "median_in": 7.64, + "p90_in": "failed", + "scale_error_pct": 2.49 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 31.83, - "median_in": 4.91, - "p90_in": Infinity, - "scale_error_pct": 0.69 + "pairs": 450000, + "failed_pct": 21.75, + "median_in": 4.07, + "p90_in": "failed", + "scale_error_pct": 0.92 }, "3-10m": { - "pairs": 525000, - "failed_pct": 36.94, - "median_in": 12.31, - "p90_in": Infinity, - "scale_error_pct": 1.44 + "pairs": 450000, + "failed_pct": 28.45, + "median_in": 9.24, + "p90_in": "failed", + "scale_error_pct": 1.47 } }, - "tape_calibration_success_pct": 81.7, - "groups": 7, + "tape_calibration_success_pct": 89.3, + "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 1.7, - 21.57 + 1.4, + 11.57 ], "p90_in": [ - 19.65, - Infinity + 5.78, + "failed" ] }, "3-10m": { "median_in": [ - 4.82, - 109.74 + 4.06, + 29.03 ], "p90_in": [ - Infinity, - Infinity + 38.75, + "failed" ] } } @@ -15234,58 +15234,58 @@ "edges": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 79.79, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 7.46 + "pairs": 18000, + "failed_pct": 81.56, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 2.98 }, "3-10m": { - "pairs": 20786, - "failed_pct": 77.6, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 11.04 + "pairs": 17786, + "failed_pct": 84.21, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 2.35 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 83.76, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 4.5 + "pairs": 450000, + "failed_pct": 82.79, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 5.0 }, "3-10m": { - "pairs": 519650, - "failed_pct": 84.11, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 1.13 + "pairs": 444650, + "failed_pct": 85.5, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 1.44 } }, - "tape_calibration_success_pct": 81.7, - "groups": 7, + "tape_calibration_success_pct": 89.3, + "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -15295,58 +15295,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 10.39, - "median_in": 3.43, - "p90_in": Infinity, - "scale_error_pct": 2.58 + "pairs": 21000, + "failed_pct": 5.43, + "median_in": 2.85, + "p90_in": 11.84, + "scale_error_pct": 2.34 }, "3-10m": { - "pairs": 24000, - "failed_pct": 13.29, - "median_in": 6.78, - "p90_in": Infinity, - "scale_error_pct": 1.93 + "pairs": 21000, + "failed_pct": 6.16, + "median_in": 5.59, + "p90_in": 23.85, + "scale_error_pct": 1.7 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 19.04, - "median_in": 3.63, - "p90_in": Infinity, - "scale_error_pct": 0.16 + "pairs": 525000, + "failed_pct": 12.81, + "median_in": 3.07, + "p90_in": "failed", + "scale_error_pct": 0.26 }, "3-10m": { - "pairs": 600000, - "failed_pct": 21.25, - "median_in": 7.95, - "p90_in": Infinity, - "scale_error_pct": -0.11 + "pairs": 525000, + "failed_pct": 13.58, + "median_in": 6.47, + "p90_in": "failed", + "scale_error_pct": 0.13 } }, - "tape_calibration_success_pct": 89.0, - "groups": 8, + "tape_calibration_success_pct": 92.0, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 1.8, - 6.26 + 1.55, + 4.65 ], "p90_in": [ - 7.93, - Infinity + 6.92, + 18.28 ] }, "3-10m": { "median_in": [ - 2.86, - 12.03 + 2.52, + 8.98 ], "p90_in": [ - 14.37, - Infinity + 12.91, + 37.16 ] } } @@ -15354,58 +15354,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 14.0, - "median_in": 2.96, - "p90_in": Infinity, - "scale_error_pct": 2.11 + "pairs": 18000, + "failed_pct": 8.43, + "median_in": 2.26, + "p90_in": 19.89, + "scale_error_pct": 1.93 }, "3-10m": { - "pairs": 21000, - "failed_pct": 19.99, - "median_in": 6.67, - "p90_in": Infinity, - "scale_error_pct": 2.09 + "pairs": 18000, + "failed_pct": 13.34, + "median_in": 4.04, + "p90_in": "failed", + "scale_error_pct": 1.84 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 23.7, - "median_in": 4.34, - "p90_in": Infinity, - "scale_error_pct": 1.04 + "pairs": 450000, + "failed_pct": 16.95, + "median_in": 3.68, + "p90_in": "failed", + "scale_error_pct": 1.19 }, "3-10m": { - "pairs": 525000, - "failed_pct": 28.69, - "median_in": 9.75, - "p90_in": Infinity, - "scale_error_pct": 1.42 + "pairs": 450000, + "failed_pct": 21.79, + "median_in": 7.99, + "p90_in": "failed", + "scale_error_pct": 1.44 } }, - "tape_calibration_success_pct": 87.4, - "groups": 7, + "tape_calibration_success_pct": 90.7, + "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 1.36, - 9.89 + 1.18, + 9.07 ], "p90_in": [ - 8.47, - Infinity + 3.55, + "failed" ] }, "3-10m": { "median_in": [ - 2.24, - 20.65 + 1.92, + 18.87 ], "p90_in": [ - 19.11, - Infinity + 12.05, + "failed" ] } } @@ -15413,58 +15413,58 @@ "edges": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 68.39, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 8.83 + "pairs": 18000, + "failed_pct": 70.68, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 6.26 }, "3-10m": { - "pairs": 20786, - "failed_pct": 71.12, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 5.0 + "pairs": 17786, + "failed_pct": 78.05, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 2.76 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 71.78, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 7.01 + "pairs": 450000, + "failed_pct": 72.24, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 7.41 }, "3-10m": { - "pairs": 519650, - "failed_pct": 75.68, - "median_in": Infinity, - "p90_in": Infinity, - "scale_error_pct": 2.96 + "pairs": 444650, + "failed_pct": 79.66, + "median_in": "failed", + "p90_in": "failed", + "scale_error_pct": 2.57 } }, - "tape_calibration_success_pct": 87.4, - "groups": 7, + "tape_calibration_success_pct": 90.7, + "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] }, "3-10m": { "median_in": [ - Infinity, - Infinity + "failed", + "failed" ], "p90_in": [ - Infinity, - Infinity + "failed", + "failed" ] } } @@ -15476,58 +15476,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.49, - "median_in": 6.81, - "p90_in": 14.99, - "scale_error_pct": 2.42 + "pairs": 21000, + "failed_pct": 0.06, + "median_in": 6.62, + "p90_in": 13.92, + "scale_error_pct": 1.97 }, "3-10m": { - "pairs": 24000, - "failed_pct": 1.68, - "median_in": 14.9, - "p90_in": 28.73, - "scale_error_pct": 3.75 + "pairs": 21000, + "failed_pct": 0.17, + "median_in": 14.62, + "p90_in": 27.78, + "scale_error_pct": 2.59 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 0.49, - "median_in": 2.3, - "p90_in": 11.34, - "scale_error_pct": -0.01 + "pairs": 525000, + "failed_pct": 0.63, + "median_in": 1.99, + "p90_in": 8.32, + "scale_error_pct": -0.07 }, "3-10m": { - "pairs": 600000, - "failed_pct": 1.68, - "median_in": 3.96, - "p90_in": 21.1, - "scale_error_pct": 0.04 + "pairs": 525000, + "failed_pct": 0.74, + "median_in": 3.47, + "p90_in": 15.55, + "scale_error_pct": -0.04 } }, - "tape_calibration_success_pct": 100.0, - "groups": 8, + "tape_calibration_success_pct": 99.4, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 2.94, - 9.42 + 2.67, + 9.38 ], "p90_in": [ - 11.44, - 19.17 + 10.57, + 16.54 ] }, "3-10m": { "median_in": [ - 5.68, - 19.76 + 6.09, + 19.81 ], "p90_in": [ - 20.31, - 32.34 + 18.55, + 31.52 ] } } @@ -15535,58 +15535,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 0.99, - "median_in": 5.21, - "p90_in": 15.32, - "scale_error_pct": -0.05 + "pairs": 18000, + "failed_pct": 0.91, + "median_in": 4.94, + "p90_in": 13.76, + "scale_error_pct": -1.31 }, "3-10m": { - "pairs": 21000, - "failed_pct": 2.6, - "median_in": 13.27, - "p90_in": 31.63, - "scale_error_pct": 2.53 + "pairs": 18000, + "failed_pct": 0.96, + "median_in": 12.92, + "p90_in": 31.39, + "scale_error_pct": 1.87 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 0.99, - "median_in": 2.4, - "p90_in": 12.78, - "scale_error_pct": -0.26 + "pairs": 450000, + "failed_pct": 1.58, + "median_in": 2.19, + "p90_in": 8.57, + "scale_error_pct": -0.35 }, "3-10m": { - "pairs": 525000, - "failed_pct": 2.6, - "median_in": 5.11, - "p90_in": 32.19, - "scale_error_pct": 0.16 + "pairs": 450000, + "failed_pct": 1.63, + "median_in": 4.53, + "p90_in": 21.92, + "scale_error_pct": 0.13 } }, - "tape_calibration_success_pct": 100.0, - "groups": 7, + "tape_calibration_success_pct": 99.3, + "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 1.97, - 8.39 + 1.85, + 8.75 ], "p90_in": [ - 9.94, - 19.04 + 7.96, + 17.65 ] }, "3-10m": { "median_in": [ - 5.8, - 20.02 + 4.69, + 23.11 ], "p90_in": [ - 20.5, - 40.56 + 18.26, + 38.03 ] } } @@ -15594,58 +15594,58 @@ "edges": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 1.0, - "median_in": 8.47, - "p90_in": 33.83, - "scale_error_pct": 1.22 + "pairs": 18000, + "failed_pct": 1.13, + "median_in": 7.15, + "p90_in": 17.3, + "scale_error_pct": 0.37 }, "3-10m": { - "pairs": 20786, - "failed_pct": 0.58, - "median_in": 15.12, - "p90_in": 49.92, - "scale_error_pct": 2.38 + "pairs": 17786, + "failed_pct": 0.68, + "median_in": 11.97, + "p90_in": 23.56, + "scale_error_pct": 1.4 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 1.0, - "median_in": 3.85, - "p90_in": 27.29, - "scale_error_pct": 0.75 + "pairs": 450000, + "failed_pct": 1.8, + "median_in": 3.22, + "p90_in": 14.58, + "scale_error_pct": -0.17 }, "3-10m": { - "pairs": 519650, - "failed_pct": 0.58, - "median_in": 5.53, - "p90_in": 38.06, - "scale_error_pct": 0.14 + "pairs": 444650, + "failed_pct": 1.35, + "median_in": 4.73, + "p90_in": 18.8, + "scale_error_pct": -0.77 } }, - "tape_calibration_success_pct": 100.0, - "groups": 7, + "tape_calibration_success_pct": 99.3, + "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 3.22, - 13.06 + 2.49, + 11.58 ], "p90_in": [ - 13.81, - 50.64 + 12.44, + 28.1 ] }, "3-10m": { "median_in": [ - 5.14, - 22.74 + 4.43, + 19.29 ], "p90_in": [ - 19.43, - 68.72 + 17.49, + 35.8 ] } } @@ -15655,58 +15655,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.22, - "median_in": 6.14, - "p90_in": 14.4, - "scale_error_pct": 2.54 + "pairs": 21000, + "failed_pct": 0.01, + "median_in": 5.71, + "p90_in": 13.0, + "scale_error_pct": 2.93 }, "3-10m": { - "pairs": 24000, - "failed_pct": 1.16, - "median_in": 13.88, - "p90_in": 25.94, + "pairs": 21000, + "failed_pct": 0.07, + "median_in": 13.52, + "p90_in": 24.33, "scale_error_pct": 4.38 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 0.72, - "median_in": 2.47, - "p90_in": 15.68, - "scale_error_pct": 0.19 + "pairs": 525000, + "failed_pct": 1.15, + "median_in": 2.2, + "p90_in": 9.98, + "scale_error_pct": 0.04 }, "3-10m": { - "pairs": 600000, - "failed_pct": 1.66, - "median_in": 4.37, - "p90_in": 33.08, - "scale_error_pct": 0.56 + "pairs": 525000, + "failed_pct": 1.21, + "median_in": 3.89, + "p90_in": 18.93, + "scale_error_pct": 0.37 } }, - "tape_calibration_success_pct": 99.5, - "groups": 8, + "tape_calibration_success_pct": 98.9, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 3.74, - 8.91 + 3.29, + 8.58 ], "p90_in": [ - 10.45, - 21.0 + 9.2, + 15.77 ] }, "3-10m": { "median_in": [ - 8.39, - 18.13 + 7.7, + 18.05 ], "p90_in": [ - 19.43, - 29.93 + 17.8, + 27.7 ] } } @@ -15714,58 +15714,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 0.45, - "median_in": 5.6, - "p90_in": 14.98, - "scale_error_pct": -0.13 + "pairs": 18000, + "failed_pct": 0.42, + "median_in": 4.98, + "p90_in": 12.64, + "scale_error_pct": 0.09 }, "3-10m": { - "pairs": 21000, - "failed_pct": 2.03, - "median_in": 13.66, - "p90_in": 30.36, - "scale_error_pct": 4.29 + "pairs": 18000, + "failed_pct": 0.48, + "median_in": 13.06, + "p90_in": 28.85, + "scale_error_pct": 4.22 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 1.02, - "median_in": 2.37, - "p90_in": 19.75, - "scale_error_pct": -0.05 + "pairs": 450000, + "failed_pct": 1.75, + "median_in": 2.09, + "p90_in": 10.96, + "scale_error_pct": -0.2 }, "3-10m": { - "pairs": 525000, - "failed_pct": 2.6, - "median_in": 5.26, - "p90_in": 52.6, - "scale_error_pct": 1.1 + "pairs": 450000, + "failed_pct": 1.82, + "median_in": 4.43, + "p90_in": 30.13, + "scale_error_pct": 0.76 } }, - "tape_calibration_success_pct": 99.4, - "groups": 7, + "tape_calibration_success_pct": 98.7, + "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 2.9, - 8.64 + 2.21, + 8.37 ], "p90_in": [ - 9.74, - 23.68 + 7.71, + 16.01 ] }, "3-10m": { "median_in": [ - 8.78, - 19.58 + 8.11, + 19.69 ], "p90_in": [ - 21.48, - 48.44 + 17.69, + 44.98 ] } } @@ -15773,58 +15773,58 @@ "edges": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 0.44, - "median_in": 7.47, - "p90_in": 20.55, - "scale_error_pct": 2.81 + "pairs": 18000, + "failed_pct": 0.51, + "median_in": 6.93, + "p90_in": 16.04, + "scale_error_pct": 2.41 }, "3-10m": { - "pairs": 20786, - "failed_pct": 0.3, - "median_in": 13.34, - "p90_in": 34.63, - "scale_error_pct": 3.31 + "pairs": 17786, + "failed_pct": 0.35, + "median_in": 11.39, + "p90_in": 21.06, + "scale_error_pct": 2.98 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 1.01, - "median_in": 4.15, - "p90_in": 26.44, - "scale_error_pct": 0.8 + "pairs": 450000, + "failed_pct": 1.84, + "median_in": 3.51, + "p90_in": 16.48, + "scale_error_pct": 0.1 }, "3-10m": { - "pairs": 519650, - "failed_pct": 0.88, - "median_in": 6.44, - "p90_in": 50.01, - "scale_error_pct": 1.03 + "pairs": 444650, + "failed_pct": 1.7, + "median_in": 5.22, + "p90_in": 24.68, + "scale_error_pct": 0.35 } }, - "tape_calibration_success_pct": 99.4, - "groups": 7, + "tape_calibration_success_pct": 98.7, + "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 3.88, - 11.27 + 3.61, + 11.12 ], "p90_in": [ - 13.62, - 41.82 + 12.39, + 20.78 ] }, "3-10m": { "median_in": [ - 6.21, - 17.22 + 5.59, + 15.78 ], "p90_in": [ - 18.01, - 58.72 + 16.23, + 30.53 ] } } @@ -15851,20 +15851,20 @@ "one_known_distance": { "1-3m": { "pairs": 525000, - "failed_pct": 0.57, - "median_in": 2.78, - "p90_in": 10.79, - "scale_error_pct": 0.11 + "failed_pct": 2.29, + "median_in": 2.84, + "p90_in": 11.94, + "scale_error_pct": -0.03 }, "3-10m": { "pairs": 525000, - "failed_pct": 0.59, - "median_in": 4.27, - "p90_in": 21.13, - "scale_error_pct": 0.18 + "failed_pct": 2.3, + "median_in": 4.47, + "p90_in": 24.28, + "scale_error_pct": 0.13 } }, - "tape_calibration_success_pct": 99.4, + "tape_calibration_success_pct": 97.7, "groups": 7, "noise_draws": 1, "interval_95": { @@ -15894,56 +15894,56 @@ "none": { "1-3m": { "pairs": 18000, - "failed_pct": 0.11, - "median_in": 4.85, - "p90_in": 12.45, - "scale_error_pct": -1.26 + "failed_pct": 0.12, + "median_in": 4.82, + "p90_in": 12.53, + "scale_error_pct": -1.29 }, "3-10m": { "pairs": 18000, - "failed_pct": 0.23, - "median_in": 11.55, - "p90_in": 25.82, - "scale_error_pct": 2.73 + "failed_pct": 0.21, + "median_in": 11.84, + "p90_in": 26.04, + "scale_error_pct": 2.71 } }, "one_known_distance": { "1-3m": { "pairs": 450000, - "failed_pct": 0.77, - "median_in": 2.77, - "p90_in": 10.03, - "scale_error_pct": 0.16 + "failed_pct": 2.79, + "median_in": 2.89, + "p90_in": 11.42, + "scale_error_pct": -0.06 }, "3-10m": { "pairs": 450000, - "failed_pct": 0.9, - "median_in": 5.47, - "p90_in": 25.64, - "scale_error_pct": 1.4 + "failed_pct": 2.88, + "median_in": 5.59, + "p90_in": 27.63, + "scale_error_pct": 1.23 } }, - "tape_calibration_success_pct": 99.3, + "tape_calibration_success_pct": 97.3, "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 2.72, - 8.29 + 2.74, + 8.23 ], "p90_in": [ - 7.74, - 15.56 + 7.79, + 15.55 ] }, "3-10m": { "median_in": [ - 6.91, - 18.23 + 6.98, + 18.35 ], "p90_in": [ - 15.98, + 16.45, 32.65 ] } @@ -15954,56 +15954,56 @@ "1-3m": { "pairs": 18000, "failed_pct": 0.08, - "median_in": 6.37, - "p90_in": 14.58, - "scale_error_pct": 1.25 + "median_in": 6.43, + "p90_in": 14.55, + "scale_error_pct": 1.33 }, "3-10m": { "pairs": 17786, "failed_pct": 0.16, - "median_in": 7.78, + "median_in": 7.81, "p90_in": 18.54, - "scale_error_pct": -0.32 + "scale_error_pct": -0.31 } }, "one_known_distance": { "1-3m": { "pairs": 450000, - "failed_pct": 0.74, + "failed_pct": 2.74, "median_in": 4.03, - "p90_in": 15.29, - "scale_error_pct": 1.79 + "p90_in": 16.86, + "scale_error_pct": 1.42 }, "3-10m": { "pairs": 444650, - "failed_pct": 0.84, - "median_in": 6.14, - "p90_in": 25.18, - "scale_error_pct": 0.42 + "failed_pct": 2.86, + "median_in": 6.2, + "p90_in": 27.58, + "scale_error_pct": 0.11 } }, - "tape_calibration_success_pct": 99.3, + "tape_calibration_success_pct": 97.3, "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 3.1, + 3.12, 10.07 ], "p90_in": [ - 11.78, + 11.79, 18.99 ] }, "3-10m": { "median_in": [ - 2.83, - 15.04 + 2.84, + 15.05 ], "p90_in": [ - 13.61, - 25.21 + 13.65, + 25.31 ] } } @@ -16015,58 +16015,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 12.99, - "median_in": 2.1, - "p90_in": Infinity, - "scale_error_pct": 0.47 + "pairs": 21000, + "failed_pct": 14.34, + "median_in": 1.73, + "p90_in": "failed", + "scale_error_pct": -0.04 }, "3-10m": { - "pairs": 24000, - "failed_pct": 14.18, - "median_in": 4.13, - "p90_in": Infinity, - "scale_error_pct": 0.38 + "pairs": 21000, + "failed_pct": 14.46, + "median_in": 3.57, + "p90_in": "failed", + "scale_error_pct": -0.27 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 12.99, - "median_in": 2.07, - "p90_in": Infinity, - "scale_error_pct": -0.11 + "pairs": 525000, + "failed_pct": 14.34, + "median_in": 1.59, + "p90_in": "failed", + "scale_error_pct": -0.07 }, "3-10m": { - "pairs": 600000, - "failed_pct": 14.18, - "median_in": 3.94, - "p90_in": Infinity, + "pairs": 525000, + "failed_pct": 14.46, + "median_in": 3.07, + "p90_in": "failed", "scale_error_pct": -0.19 } }, - "tape_calibration_success_pct": 87.5, - "groups": 8, + "tape_calibration_success_pct": 85.7, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 1.24, - 8.51 + 1.13, + 3.44 ], "p90_in": [ - 3.42, - Infinity + 3.23, + "failed" ] }, "3-10m": { "median_in": [ - 2.44, - 28.31 + 2.16, + 6.45 ], "p90_in": [ - 5.48, - Infinity + 4.79, + "failed" ] } } @@ -16074,58 +16074,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 15.27, - "median_in": 1.51, - "p90_in": Infinity, - "scale_error_pct": -0.4 + "pairs": 18000, + "failed_pct": 17.58, + "median_in": 1.26, + "p90_in": "failed", + "scale_error_pct": -0.78 }, "3-10m": { - "pairs": 21000, - "failed_pct": 16.89, - "median_in": 3.13, - "p90_in": Infinity, - "scale_error_pct": 0.88 + "pairs": 18000, + "failed_pct": 17.63, + "median_in": 2.74, + "p90_in": "failed", + "scale_error_pct": 0.32 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 15.27, - "median_in": 2.34, - "p90_in": Infinity, - "scale_error_pct": -0.23 + "pairs": 450000, + "failed_pct": 17.58, + "median_in": 1.77, + "p90_in": "failed", + "scale_error_pct": -0.12 }, "3-10m": { - "pairs": 525000, - "failed_pct": 16.89, - "median_in": 5.04, - "p90_in": Infinity, - "scale_error_pct": 0.23 + "pairs": 450000, + "failed_pct": 17.63, + "median_in": 4.13, + "p90_in": "failed", + "scale_error_pct": 0.43 } }, - "tape_calibration_success_pct": 85.7, - "groups": 7, + "tape_calibration_success_pct": 83.3, + "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 0.92, - 27.35 + 0.79, + "failed" ], "p90_in": [ - 2.3, - Infinity + 2.1, + "failed" ] }, "3-10m": { "median_in": [ - 2.09, - 60.82 + 1.88, + "failed" ], "p90_in": [ - 6.9, - Infinity + 3.53, + "failed" ] } } @@ -16133,58 +16133,58 @@ "edges": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 15.29, - "median_in": 2.96, - "p90_in": Infinity, - "scale_error_pct": -0.02 + "pairs": 18000, + "failed_pct": 17.8, + "median_in": 2.24, + "p90_in": "failed", + "scale_error_pct": -0.64 }, "3-10m": { - "pairs": 20786, - "failed_pct": 13.99, - "median_in": 3.83, - "p90_in": Infinity, - "scale_error_pct": -0.6 + "pairs": 17786, + "failed_pct": 16.34, + "median_in": 3.13, + "p90_in": "failed", + "scale_error_pct": -0.96 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 15.29, - "median_in": 3.79, - "p90_in": Infinity, - "scale_error_pct": 0.33 + "pairs": 450000, + "failed_pct": 17.8, + "median_in": 2.95, + "p90_in": "failed", + "scale_error_pct": -0.13 }, "3-10m": { - "pairs": 519650, - "failed_pct": 13.99, - "median_in": 5.69, - "p90_in": Infinity, - "scale_error_pct": -0.09 + "pairs": 444650, + "failed_pct": 16.34, + "median_in": 4.54, + "p90_in": "failed", + "scale_error_pct": -0.41 } }, - "tape_calibration_success_pct": 85.7, - "groups": 7, + "tape_calibration_success_pct": 83.3, + "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 1.25, - 60.91 + 1.12, + "failed" ], "p90_in": [ - 7.16, - Infinity + 3.84, + "failed" ] }, "3-10m": { "median_in": [ - 2.08, - 85.79 + 1.85, + 32.05 ], "p90_in": [ - 8.42, - Infinity + 4.69, + "failed" ] } } @@ -16194,58 +16194,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 24000, - "failed_pct": 0.23, - "median_in": 1.61, - "p90_in": 8.93, - "scale_error_pct": -0.21 + "pairs": 21000, + "failed_pct": 0.02, + "median_in": 1.38, + "p90_in": 4.46, + "scale_error_pct": -0.48 }, "3-10m": { - "pairs": 24000, - "failed_pct": 1.16, - "median_in": 3.27, - "p90_in": 20.54, - "scale_error_pct": -0.26 + "pairs": 21000, + "failed_pct": 0.08, + "median_in": 2.76, + "p90_in": 9.02, + "scale_error_pct": -0.52 } }, "one_known_distance": { "1-3m": { - "pairs": 600000, - "failed_pct": 0.73, - "median_in": 1.97, - "p90_in": 12.97, - "scale_error_pct": -0.01 + "pairs": 525000, + "failed_pct": 1.16, + "median_in": 1.61, + "p90_in": 7.08, + "scale_error_pct": -0.07 }, "3-10m": { - "pairs": 600000, - "failed_pct": 1.66, - "median_in": 3.77, - "p90_in": 27.23, - "scale_error_pct": -0.11 + "pairs": 525000, + "failed_pct": 1.22, + "median_in": 3.1, + "p90_in": 13.29, + "scale_error_pct": -0.24 } }, - "tape_calibration_success_pct": 99.5, - "groups": 8, + "tape_calibration_success_pct": 98.9, + "groups": 7, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 1.09, - 2.71 + 1.02, + 1.76 ], "p90_in": [ - 3.33, - 34.21 + 3.06, + 6.5 ] }, "3-10m": { "median_in": [ - 1.69, - 6.84 + 1.5, + 4.21 ], "p90_in": [ - 5.16, - 74.59 + 3.98, + 10.87 ] } } @@ -16253,58 +16253,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 0.49, - "median_in": 1.45, - "p90_in": 10.89, - "scale_error_pct": -0.86 + "pairs": 18000, + "failed_pct": 0.43, + "median_in": 1.22, + "p90_in": 3.56, + "scale_error_pct": -1.1 }, "3-10m": { - "pairs": 21000, - "failed_pct": 2.05, - "median_in": 2.74, - "p90_in": 53.14, - "scale_error_pct": -0.18 + "pairs": 18000, + "failed_pct": 0.5, + "median_in": 2.19, + "p90_in": 11.26, + "scale_error_pct": -0.46 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 1.06, - "median_in": 1.91, - "p90_in": 16.24, - "scale_error_pct": -0.17 + "pairs": 450000, + "failed_pct": 1.76, + "median_in": 1.46, + "p90_in": 7.13, + "scale_error_pct": -0.27 }, "3-10m": { - "pairs": 525000, - "failed_pct": 2.62, - "median_in": 4.65, - "p90_in": 43.4, - "scale_error_pct": 0.29 + "pairs": 450000, + "failed_pct": 1.83, + "median_in": 3.69, + "p90_in": 19.77, + "scale_error_pct": 0.15 } }, - "tape_calibration_success_pct": 99.4, - "groups": 7, + "tape_calibration_success_pct": 98.7, + "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 0.94, - 2.61 + 0.79, + 1.73 ], "p90_in": [ - 2.37, - 41.16 + 2.09, + 6.64 ] }, "3-10m": { "median_in": [ - 1.46, - 10.35 + 1.33, + 5.91 ], "p90_in": [ - 5.2, - 79.59 + 3.47, + 29.52 ] } } @@ -16312,58 +16312,58 @@ "edges": { "none": { "1-3m": { - "pairs": 21000, - "failed_pct": 0.44, - "median_in": 2.15, - "p90_in": 24.85, - "scale_error_pct": -0.67 + "pairs": 18000, + "failed_pct": 0.51, + "median_in": 1.73, + "p90_in": 10.74, + "scale_error_pct": -0.96 }, "3-10m": { - "pairs": 20786, - "failed_pct": 0.3, - "median_in": 3.33, - "p90_in": 53.66, - "scale_error_pct": -0.99 + "pairs": 17786, + "failed_pct": 0.35, + "median_in": 2.67, + "p90_in": 11.1, + "scale_error_pct": -1.2 } }, "one_known_distance": { "1-3m": { - "pairs": 525000, - "failed_pct": 1.01, - "median_in": 3.16, - "p90_in": 23.54, - "scale_error_pct": -0.27 + "pairs": 450000, + "failed_pct": 1.84, + "median_in": 2.49, + "p90_in": 12.47, + "scale_error_pct": -0.6 }, "3-10m": { - "pairs": 519650, - "failed_pct": 0.88, - "median_in": 5.08, - "p90_in": 41.0, - "scale_error_pct": -0.33 + "pairs": 444650, + "failed_pct": 1.7, + "median_in": 3.92, + "p90_in": 17.18, + "scale_error_pct": -0.53 } }, - "tape_calibration_success_pct": 99.4, - "groups": 7, + "tape_calibration_success_pct": 98.7, + "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 1.19, - 6.5 + 1.08, + 3.55 ], "p90_in": [ - 5.7, - 67.46 + 3.44, + 14.69 ] }, "3-10m": { "median_in": [ - 1.93, - 10.47 + 1.71, + 5.93 ], "p90_in": [ - 7.58, - 84.11 + 4.36, + 19.73 ] } } @@ -16390,20 +16390,20 @@ "one_known_distance": { "1-3m": { "pairs": 525000, - "failed_pct": 0.57, - "median_in": 1.83, - "p90_in": 7.59, - "scale_error_pct": -0.18 + "failed_pct": 1.14, + "median_in": 1.82, + "p90_in": 7.61, + "scale_error_pct": -0.19 }, "3-10m": { "pairs": 525000, - "failed_pct": 0.59, - "median_in": 3.23, - "p90_in": 14.27, - "scale_error_pct": -0.4 + "failed_pct": 1.16, + "median_in": 3.26, + "p90_in": 14.93, + "scale_error_pct": -0.42 } }, - "tape_calibration_success_pct": 99.4, + "tape_calibration_success_pct": 98.9, "groups": 7, "noise_draws": 1, "interval_95": { @@ -16433,57 +16433,57 @@ "none": { "1-3m": { "pairs": 18000, - "failed_pct": 0.11, - "median_in": 1.08, + "failed_pct": 0.12, + "median_in": 1.06, "p90_in": 2.81, "scale_error_pct": -0.72 }, "3-10m": { "pairs": 18000, - "failed_pct": 0.23, - "median_in": 2.09, - "p90_in": 8.3, - "scale_error_pct": -0.49 + "failed_pct": 0.21, + "median_in": 2.14, + "p90_in": 8.63, + "scale_error_pct": -0.45 } }, "one_known_distance": { "1-3m": { "pairs": 450000, - "failed_pct": 0.77, - "median_in": 1.51, - "p90_in": 6.59, - "scale_error_pct": -0.01 + "failed_pct": 1.46, + "median_in": 1.5, + "p90_in": 6.58, + "scale_error_pct": -0.03 }, "3-10m": { "pairs": 450000, - "failed_pct": 0.9, - "median_in": 3.15, - "p90_in": 16.25, - "scale_error_pct": 0.11 + "failed_pct": 1.54, + "median_in": 3.24, + "p90_in": 16.12, + "scale_error_pct": 0.08 } }, - "tape_calibration_success_pct": 99.3, + "tape_calibration_success_pct": 98.7, "groups": 6, "noise_draws": 1, "interval_95": { "1-3m": { "median_in": [ - 0.83, - 1.43 + 0.82, + 1.42 ], "p90_in": [ 1.97, - 4.37 + 4.59 ] }, "3-10m": { "median_in": [ - 1.52, - 4.02 + 1.53, + 4.15 ], "p90_in": [ - 3.31, - 18.97 + 3.38, + 19.09 ] } } @@ -16493,35 +16493,35 @@ "1-3m": { "pairs": 18000, "failed_pct": 0.08, - "median_in": 1.56, - "p90_in": 8.17, + "median_in": 1.57, + "p90_in": 8.16, "scale_error_pct": -0.82 }, "3-10m": { "pairs": 17786, "failed_pct": 0.16, - "median_in": 2.2, - "p90_in": 12.26, + "median_in": 2.21, + "p90_in": 12.29, "scale_error_pct": -1.06 } }, "one_known_distance": { "1-3m": { "pairs": 450000, - "failed_pct": 0.74, + "failed_pct": 1.41, "median_in": 2.37, - "p90_in": 11.0, - "scale_error_pct": -0.31 + "p90_in": 10.81, + "scale_error_pct": -0.36 }, "3-10m": { "pairs": 444650, - "failed_pct": 0.84, - "median_in": 3.61, - "p90_in": 17.45, - "scale_error_pct": -0.49 + "failed_pct": 1.51, + "median_in": 3.66, + "p90_in": 17.2, + "scale_error_pct": -0.53 } }, - "tape_calibration_success_pct": 99.3, + "tape_calibration_success_pct": 98.7, "groups": 6, "noise_draws": 1, "interval_95": { @@ -16531,18 +16531,18 @@ 2.91 ], "p90_in": [ - 2.83, - 13.04 + 2.85, + 13.01 ] }, "3-10m": { "median_in": [ 1.48, - 5.21 + 5.27 ], "p90_in": [ - 3.34, - 15.72 + 3.4, + 16.04 ] } } @@ -16554,58 +16554,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 0.49, - "median_in": 6.81, - "p90_in": 15.44, - "scale_error_pct": 1.94 + "pairs": 105000, + "failed_pct": 0.06, + "median_in": 6.63, + "p90_in": 14.35, + "scale_error_pct": 1.44 }, "3-10m": { - "pairs": 120000, - "failed_pct": 1.68, - "median_in": 15.07, - "p90_in": 28.65, - "scale_error_pct": 3.16 + "pairs": 105000, + "failed_pct": 0.17, + "median_in": 14.84, + "p90_in": 27.68, + "scale_error_pct": 2.21 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 0.49, - "median_in": 2.71, - "p90_in": 13.13, - "scale_error_pct": -0.19 + "pairs": 2625000, + "failed_pct": 0.06, + "median_in": 2.31, + "p90_in": 9.6, + "scale_error_pct": -0.17 }, "3-10m": { - "pairs": 3000000, - "failed_pct": 1.68, - "median_in": 4.69, - "p90_in": 24.23, + "pairs": 2625000, + "failed_pct": 0.17, + "median_in": 4.16, + "p90_in": 17.32, "scale_error_pct": -0.38 } }, "tape_calibration_success_pct": 100.0, - "groups": 8, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 3.11, - 9.48 + 3.04, + 9.49 ], "p90_in": [ - 11.52, - 19.39 + 11.06, + 16.72 ] }, "3-10m": { "median_in": [ - 7.94, - 19.69 + 7.62, + 20.03 ], "p90_in": [ - 20.21, - 32.34 + 19.27, + 31.5 ] } } @@ -16613,58 +16613,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 105000, - "failed_pct": 0.99, - "median_in": 5.41, - "p90_in": 15.29, - "scale_error_pct": -0.67 + "pairs": 90000, + "failed_pct": 0.91, + "median_in": 5.15, + "p90_in": 13.85, + "scale_error_pct": -1.6 }, "3-10m": { - "pairs": 105000, - "failed_pct": 2.6, - "median_in": 13.61, - "p90_in": 31.85, - "scale_error_pct": 2.35 + "pairs": 90000, + "failed_pct": 0.96, + "median_in": 13.22, + "p90_in": 31.5, + "scale_error_pct": 1.75 } }, "one_known_distance": { "1-3m": { - "pairs": 2625000, - "failed_pct": 0.99, - "median_in": 2.7, - "p90_in": 14.63, - "scale_error_pct": -0.22 + "pairs": 2250000, + "failed_pct": 0.91, + "median_in": 2.24, + "p90_in": 9.89, + "scale_error_pct": -0.2 }, "3-10m": { - "pairs": 2625000, - "failed_pct": 2.6, - "median_in": 5.6, - "p90_in": 37.04, - "scale_error_pct": 0.11 + "pairs": 2250000, + "failed_pct": 0.96, + "median_in": 4.83, + "p90_in": 29.37, + "scale_error_pct": 0.26 } }, "tape_calibration_success_pct": 100.0, - "groups": 7, + "groups": 6, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 2.17, - 8.55 + 1.84, + 8.72 ], "p90_in": [ - 10.29, - 19.07 + 7.51, + 17.27 ] }, "3-10m": { "median_in": [ - 5.78, - 21.85 + 4.21, + 21.64 ], "p90_in": [ - 20.94, - 42.81 + 17.31, + 41.41 ] } } @@ -16672,58 +16672,58 @@ "edges": { "none": { "1-3m": { - "pairs": 105000, - "failed_pct": 1.0, - "median_in": 8.61, - "p90_in": 34.03, - "scale_error_pct": 1.21 + "pairs": 90000, + "failed_pct": 1.13, + "median_in": 7.36, + "p90_in": 17.5, + "scale_error_pct": 0.36 }, "3-10m": { - "pairs": 103930, - "failed_pct": 0.58, - "median_in": 15.43, - "p90_in": 49.41, - "scale_error_pct": 1.58 + "pairs": 88930, + "failed_pct": 0.68, + "median_in": 12.25, + "p90_in": 23.65, + "scale_error_pct": 1.16 } }, "one_known_distance": { "1-3m": { - "pairs": 2625000, - "failed_pct": 1.0, - "median_in": 4.29, - "p90_in": 28.86, - "scale_error_pct": 0.88 + "pairs": 2250000, + "failed_pct": 1.13, + "median_in": 3.45, + "p90_in": 16.68, + "scale_error_pct": 0.18 }, "3-10m": { - "pairs": 2598250, - "failed_pct": 0.58, - "median_in": 6.2, - "p90_in": 40.0, - "scale_error_pct": 0.23 + "pairs": 2223250, + "failed_pct": 0.68, + "median_in": 5.05, + "p90_in": 23.16, + "scale_error_pct": -0.4 } }, "tape_calibration_success_pct": 100.0, - "groups": 7, + "groups": 6, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 3.77, - 13.21 + 2.33, + 11.49 ], "p90_in": [ - 14.36, - 50.74 + 12.98, + 27.7 ] }, "3-10m": { - "median_in": [ - 4.27, - 23.22 + "median_in": [ + 3.15, + 18.96 ], "p90_in": [ - 20.45, - 68.34 + 16.97, + 35.77 ] } } @@ -16733,58 +16733,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 0.22, - "median_in": 6.06, - "p90_in": 15.37, - "scale_error_pct": 0.39 + "pairs": 105000, + "failed_pct": 0.01, + "median_in": 5.64, + "p90_in": 13.48, + "scale_error_pct": 0.85 }, "3-10m": { - "pairs": 120000, - "failed_pct": 1.16, - "median_in": 13.64, - "p90_in": 26.25, - "scale_error_pct": 3.87 + "pairs": 105000, + "failed_pct": 0.07, + "median_in": 13.29, + "p90_in": 24.96, + "scale_error_pct": 3.85 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 0.22, - "median_in": 3.21, - "p90_in": 20.91, - "scale_error_pct": 0.13 + "pairs": 2625000, + "failed_pct": 0.24, + "median_in": 2.82, + "p90_in": 13.63, + "scale_error_pct": -0.14 }, "3-10m": { - "pairs": 3000000, - "failed_pct": 1.16, - "median_in": 5.65, - "p90_in": 42.14, - "scale_error_pct": 0.67 + "pairs": 2625000, + "failed_pct": 0.3, + "median_in": 5.02, + "p90_in": 26.57, + "scale_error_pct": 0.39 } }, - "tape_calibration_success_pct": 100.0, - "groups": 8, + "tape_calibration_success_pct": 99.8, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 3.8, - 9.0 + 3.53, + 8.86 ], "p90_in": [ - 10.23, - 21.62 + 9.57, + 16.76 ] }, "3-10m": { "median_in": [ - 8.44, - 18.62 + 8.37, + 18.71 ], "p90_in": [ - 19.16, - 29.95 + 18.26, + 28.43 ] } } @@ -16792,58 +16792,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 105000, - "failed_pct": 0.45, - "median_in": 5.93, - "p90_in": 15.06, - "scale_error_pct": -0.55 + "pairs": 90000, + "failed_pct": 0.42, + "median_in": 5.35, + "p90_in": 12.78, + "scale_error_pct": -0.28 }, "3-10m": { - "pairs": 105000, - "failed_pct": 2.03, - "median_in": 13.56, - "p90_in": 30.12, - "scale_error_pct": 4.09 + "pairs": 90000, + "failed_pct": 0.48, + "median_in": 12.97, + "p90_in": 28.94, + "scale_error_pct": 4.01 } }, "one_known_distance": { "1-3m": { - "pairs": 2625000, - "failed_pct": 0.45, - "median_in": 2.67, - "p90_in": 23.08, - "scale_error_pct": 0.42 + "pairs": 2250000, + "failed_pct": 0.68, + "median_in": 2.3, + "p90_in": 13.52, + "scale_error_pct": 0.1 }, "3-10m": { - "pairs": 2625000, - "failed_pct": 2.03, - "median_in": 6.14, - "p90_in": 56.81, - "scale_error_pct": 1.6 + "pairs": 2250000, + "failed_pct": 0.75, + "median_in": 5.0, + "p90_in": 36.03, + "scale_error_pct": 1.23 } }, - "tape_calibration_success_pct": 100.0, - "groups": 7, + "tape_calibration_success_pct": 99.7, + "groups": 6, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 3.19, - 8.94 + 2.55, + 8.38 ], "p90_in": [ - 10.19, - 23.12 + 8.0, + 15.98 ] }, "3-10m": { "median_in": [ - 8.55, - 19.85 + 7.39, + 19.35 ], "p90_in": [ - 21.45, - 46.09 + 16.02, + 44.85 ] } } @@ -16851,58 +16851,58 @@ "edges": { "none": { "1-3m": { - "pairs": 105000, - "failed_pct": 0.44, - "median_in": 7.94, - "p90_in": 20.51, - "scale_error_pct": 2.61 + "pairs": 90000, + "failed_pct": 0.51, + "median_in": 7.43, + "p90_in": 16.5, + "scale_error_pct": 2.16 }, "3-10m": { - "pairs": 103930, - "failed_pct": 0.3, - "median_in": 13.36, - "p90_in": 35.72, - "scale_error_pct": 2.88 + "pairs": 88930, + "failed_pct": 0.35, + "median_in": 11.46, + "p90_in": 21.12, + "scale_error_pct": 2.21 } }, "one_known_distance": { "1-3m": { - "pairs": 2625000, - "failed_pct": 0.44, - "median_in": 4.82, - "p90_in": 30.56, - "scale_error_pct": 1.7 + "pairs": 2250000, + "failed_pct": 0.78, + "median_in": 3.87, + "p90_in": 19.56, + "scale_error_pct": 0.8 }, "3-10m": { - "pairs": 2598250, - "failed_pct": 0.3, - "median_in": 7.66, - "p90_in": 51.45, - "scale_error_pct": 2.07 + "pairs": 2223250, + "failed_pct": 0.62, + "median_in": 6.01, + "p90_in": 28.58, + "scale_error_pct": 1.0 } }, - "tape_calibration_success_pct": 100.0, - "groups": 7, + "tape_calibration_success_pct": 99.7, + "groups": 6, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 4.06, - 11.39 + 3.47, + 11.21 ], "p90_in": [ - 14.31, - 42.23 + 13.26, + 21.22 ] }, "3-10m": { "median_in": [ - 5.9, - 17.65 + 4.79, + 15.89 ], "p90_in": [ - 18.06, - 59.44 + 16.03, + 29.97 ] } } @@ -16929,20 +16929,20 @@ "one_known_distance": { "1-3m": { "pairs": 2625000, - "failed_pct": 0.0, - "median_in": 4.29, - "p90_in": 16.7, - "scale_error_pct": 0.01 + "failed_pct": 0.23, + "median_in": 4.35, + "p90_in": 16.72, + "scale_error_pct": -0.25 }, "3-10m": { "pairs": 2625000, - "failed_pct": 0.01, - "median_in": 7.3, - "p90_in": 32.34, - "scale_error_pct": 0.18 + "failed_pct": 0.24, + "median_in": 7.49, + "p90_in": 32.53, + "scale_error_pct": 0.11 } }, - "tape_calibration_success_pct": 100.0, + "tape_calibration_success_pct": 99.8, "groups": 7, "noise_draws": 5, "interval_95": { @@ -16972,57 +16972,57 @@ "none": { "1-3m": { "pairs": 90000, - "failed_pct": 0.11, - "median_in": 5.54, - "p90_in": 12.96, - "scale_error_pct": -2.31 + "failed_pct": 0.12, + "median_in": 5.53, + "p90_in": 12.97, + "scale_error_pct": -2.37 }, "3-10m": { "pairs": 90000, - "failed_pct": 0.23, - "median_in": 11.21, - "p90_in": 26.67, - "scale_error_pct": 2.28 + "failed_pct": 0.21, + "median_in": 11.54, + "p90_in": 26.88, + "scale_error_pct": 2.23 } }, "one_known_distance": { "1-3m": { "pairs": 2250000, - "failed_pct": 0.11, - "median_in": 3.46, - "p90_in": 13.95, - "scale_error_pct": 0.34 + "failed_pct": 0.39, + "median_in": 3.61, + "p90_in": 14.13, + "scale_error_pct": 0.08 }, "3-10m": { "pairs": 2250000, - "failed_pct": 0.23, - "median_in": 6.51, - "p90_in": 34.62, - "scale_error_pct": 1.53 + "failed_pct": 0.48, + "median_in": 6.6, + "p90_in": 32.33, + "scale_error_pct": 1.3 } }, - "tape_calibration_success_pct": 100.0, + "tape_calibration_success_pct": 99.7, "groups": 6, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 3.22, - 8.45 + 3.26, + 8.38 ], "p90_in": [ - 8.32, - 16.6 + 8.29, + 16.69 ] }, "3-10m": { "median_in": [ - 6.59, - 18.38 + 6.64, + 18.59 ], "p90_in": [ - 14.26, - 32.47 + 14.47, + 32.88 ] } } @@ -17032,56 +17032,56 @@ "1-3m": { "pairs": 90000, "failed_pct": 0.08, - "median_in": 6.68, - "p90_in": 14.88, - "scale_error_pct": 0.99 + "median_in": 6.73, + "p90_in": 14.87, + "scale_error_pct": 1.09 }, "3-10m": { "pairs": 88930, "failed_pct": 0.16, "median_in": 8.2, - "p90_in": 18.68, - "scale_error_pct": -0.9 + "p90_in": 18.72, + "scale_error_pct": -0.89 } }, "one_known_distance": { "1-3m": { "pairs": 2250000, - "failed_pct": 0.08, - "median_in": 4.72, - "p90_in": 19.94, - "scale_error_pct": 2.39 + "failed_pct": 0.34, + "median_in": 4.61, + "p90_in": 19.06, + "scale_error_pct": 1.83 }, "3-10m": { "pairs": 2223250, - "failed_pct": 0.16, - "median_in": 7.14, - "p90_in": 31.8, - "scale_error_pct": 0.33 + "failed_pct": 0.43, + "median_in": 7.16, + "p90_in": 30.63, + "scale_error_pct": -0.02 } }, - "tape_calibration_success_pct": 100.0, + "tape_calibration_success_pct": 99.7, "groups": 6, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 3.4, + 3.41, 10.42 ], "p90_in": [ - 11.88, + 11.87, 19.33 ] }, "3-10m": { "median_in": [ 2.98, - 15.27 + 15.28 ], "p90_in": [ - 13.06, - 23.82 + 13.11, + 23.99 ] } } @@ -17093,58 +17093,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 12.99, - "median_in": 10.7, - "p90_in": Infinity, - "scale_error_pct": -8.74 + "pairs": 105000, + "failed_pct": 14.34, + "median_in": 10.07, + "p90_in": "failed", + "scale_error_pct": -9.7 }, "3-10m": { - "pairs": 120000, - "failed_pct": 14.18, - "median_in": 23.7, - "p90_in": Infinity, - "scale_error_pct": -8.78 + "pairs": 105000, + "failed_pct": 14.46, + "median_in": 22.31, + "p90_in": "failed", + "scale_error_pct": -9.49 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 12.99, - "median_in": 2.91, - "p90_in": Infinity, - "scale_error_pct": -0.26 + "pairs": 2625000, + "failed_pct": 14.34, + "median_in": 2.4, + "p90_in": "failed", + "scale_error_pct": -0.08 }, "3-10m": { - "pairs": 3000000, - "failed_pct": 14.18, - "median_in": 5.5, - "p90_in": Infinity, - "scale_error_pct": -0.32 + "pairs": 2625000, + "failed_pct": 14.46, + "median_in": 4.65, + "p90_in": "failed", + "scale_error_pct": -0.18 } }, - "tape_calibration_success_pct": 87.5, - "groups": 8, + "tape_calibration_success_pct": 85.7, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 7.66, - 20.15 + 7.04, + 21.14 ], "p90_in": [ - 17.29, - Infinity + 14.69, + "failed" ] }, "3-10m": { "median_in": [ - 15.55, - 43.2 + 14.07, + 45.88 ], "p90_in": [ - 36.84, - Infinity + 31.96, + "failed" ] } } @@ -17152,58 +17152,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 105000, - "failed_pct": 15.27, - "median_in": 11.22, - "p90_in": Infinity, - "scale_error_pct": -9.17 + "pairs": 90000, + "failed_pct": 17.58, + "median_in": 10.59, + "p90_in": "failed", + "scale_error_pct": -11.61 }, "3-10m": { - "pairs": 105000, - "failed_pct": 16.89, - "median_in": 21.73, - "p90_in": Infinity, - "scale_error_pct": -7.43 + "pairs": 90000, + "failed_pct": 17.63, + "median_in": 19.95, + "p90_in": "failed", + "scale_error_pct": -9.1 } }, "one_known_distance": { "1-3m": { - "pairs": 2625000, - "failed_pct": 15.27, - "median_in": 3.14, - "p90_in": Infinity, - "scale_error_pct": -0.23 + "pairs": 2250000, + "failed_pct": 17.58, + "median_in": 2.47, + "p90_in": "failed", + "scale_error_pct": 0.1 }, "3-10m": { - "pairs": 2625000, - "failed_pct": 16.89, - "median_in": 6.24, - "p90_in": Infinity, - "scale_error_pct": 0.39 + "pairs": 2250000, + "failed_pct": 17.63, + "median_in": 5.3, + "p90_in": "failed", + "scale_error_pct": 0.77 } }, - "tape_calibration_success_pct": 85.7, - "groups": 7, + "tape_calibration_success_pct": 83.3, + "groups": 6, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 8.18, - 30.1 + 7.3, + "failed" ], "p90_in": [ - 18.36, - Infinity + 15.04, + "failed" ] }, "3-10m": { "median_in": [ - 14.15, - 74.63 + 12.71, + "failed" ], "p90_in": [ - 35.16, - Infinity + 26.54, + "failed" ] } } @@ -17211,58 +17211,58 @@ "edges": { "none": { "1-3m": { - "pairs": 105000, - "failed_pct": 15.29, - "median_in": 11.91, - "p90_in": Infinity, - "scale_error_pct": -8.6 + "pairs": 90000, + "failed_pct": 17.8, + "median_in": 10.49, + "p90_in": "failed", + "scale_error_pct": -9.92 }, "3-10m": { - "pairs": 103930, - "failed_pct": 13.99, - "median_in": 22.24, - "p90_in": Infinity, - "scale_error_pct": -9.19 + "pairs": 88930, + "failed_pct": 16.34, + "median_in": 18.63, + "p90_in": "failed", + "scale_error_pct": -10.09 } }, "one_known_distance": { "1-3m": { - "pairs": 2625000, - "failed_pct": 15.29, - "median_in": 4.45, - "p90_in": Infinity, - "scale_error_pct": 0.8 + "pairs": 2250000, + "failed_pct": 17.8, + "median_in": 3.54, + "p90_in": "failed", + "scale_error_pct": 0.43 }, "3-10m": { - "pairs": 2598250, - "failed_pct": 13.99, - "median_in": 6.85, - "p90_in": Infinity, - "scale_error_pct": 0.29 + "pairs": 2223250, + "failed_pct": 16.34, + "median_in": 5.52, + "p90_in": "failed", + "scale_error_pct": -0.05 } }, - "tape_calibration_success_pct": 85.7, - "groups": 7, + "tape_calibration_success_pct": 83.3, + "groups": 6, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 8.28, - 57.06 + 7.38, + "failed" ], "p90_in": [ - 20.34, - Infinity + 16.99, + "failed" ] }, "3-10m": { "median_in": [ - 14.57, - 74.86 + 13.3, + 51.39 ], "p90_in": [ - 36.23, - Infinity + 25.8, + "failed" ] } } @@ -17272,58 +17272,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 0.23, - "median_in": 10.33, - "p90_in": 30.73, - "scale_error_pct": -9.02 + "pairs": 105000, + "failed_pct": 0.02, + "median_in": 10.12, + "p90_in": 26.48, + "scale_error_pct": -10.2 }, "3-10m": { - "pairs": 120000, - "failed_pct": 1.16, - "median_in": 22.06, - "p90_in": 61.94, - "scale_error_pct": -8.82 + "pairs": 105000, + "failed_pct": 0.08, + "median_in": 21.24, + "p90_in": 50.08, + "scale_error_pct": -10.36 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 0.23, - "median_in": 4.47, - "p90_in": 28.51, - "scale_error_pct": -0.1 + "pairs": 2625000, + "failed_pct": 0.02, + "median_in": 3.69, + "p90_in": 20.36, + "scale_error_pct": -0.08 }, "3-10m": { - "pairs": 3000000, - "failed_pct": 1.16, - "median_in": 7.78, - "p90_in": 60.13, - "scale_error_pct": 0.02 + "pairs": 2625000, + "failed_pct": 0.08, + "median_in": 6.32, + "p90_in": 41.79, + "scale_error_pct": -0.13 } }, "tape_calibration_success_pct": 100.0, - "groups": 8, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 6.08, - 15.23 + 5.73, + 15.85 ], "p90_in": [ - 18.04, - 70.29 + 16.38, + 70.77 ] }, "3-10m": { "median_in": [ - 13.28, - 32.49 + 12.06, + 32.48 ], "p90_in": [ - 36.65, - 133.98 + 32.72, + 136.09 ] } } @@ -17331,58 +17331,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 105000, - "failed_pct": 0.49, - "median_in": 11.22, - "p90_in": 34.06, - "scale_error_pct": -10.47 + "pairs": 90000, + "failed_pct": 0.43, + "median_in": 10.96, + "p90_in": 28.98, + "scale_error_pct": -11.72 }, "3-10m": { - "pairs": 105000, - "failed_pct": 2.05, - "median_in": 24.15, - "p90_in": 76.71, - "scale_error_pct": -10.42 + "pairs": 90000, + "failed_pct": 0.5, + "median_in": 22.7, + "p90_in": 54.87, + "scale_error_pct": -11.45 } }, "one_known_distance": { "1-3m": { - "pairs": 2625000, - "failed_pct": 0.49, - "median_in": 3.57, - "p90_in": 30.36, - "scale_error_pct": 0.1 + "pairs": 2250000, + "failed_pct": 0.43, + "median_in": 2.68, + "p90_in": 19.8, + "scale_error_pct": 0.12 }, "3-10m": { - "pairs": 2625000, - "failed_pct": 2.05, - "median_in": 8.81, - "p90_in": 67.22, - "scale_error_pct": 0.92 + "pairs": 2250000, + "failed_pct": 0.5, + "median_in": 6.66, + "p90_in": 47.84, + "scale_error_pct": 0.79 } }, "tape_calibration_success_pct": 100.0, - "groups": 7, + "groups": 6, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 6.61, - 16.71 + 5.97, + 16.28 ], "p90_in": [ - 19.05, - 67.41 + 15.77, + 66.12 ] }, "3-10m": { "median_in": [ - 17.21, - 34.97 + 16.47, + 32.58 ], "p90_in": [ - 37.09, - 143.39 + 32.93, + 139.46 ] } } @@ -17390,58 +17390,58 @@ "edges": { "none": { "1-3m": { - "pairs": 105000, - "failed_pct": 0.44, - "median_in": 11.63, - "p90_in": 33.76, - "scale_error_pct": -10.16 + "pairs": 90000, + "failed_pct": 0.51, + "median_in": 10.89, + "p90_in": 27.28, + "scale_error_pct": -11.9 }, "3-10m": { - "pairs": 103930, - "failed_pct": 0.3, - "median_in": 22.97, - "p90_in": 63.53, - "scale_error_pct": -10.42 + "pairs": 88930, + "failed_pct": 0.35, + "median_in": 20.07, + "p90_in": 45.42, + "scale_error_pct": -11.8 } }, "one_known_distance": { "1-3m": { - "pairs": 2625000, - "failed_pct": 0.44, - "median_in": 5.27, - "p90_in": 34.33, - "scale_error_pct": 0.69 + "pairs": 2250000, + "failed_pct": 0.51, + "median_in": 4.28, + "p90_in": 23.34, + "scale_error_pct": 0.36 }, "3-10m": { - "pairs": 2598250, - "failed_pct": 0.3, - "median_in": 8.56, - "p90_in": 59.53, - "scale_error_pct": 0.15 + "pairs": 2223250, + "failed_pct": 0.35, + "median_in": 6.75, + "p90_in": 39.08, + "scale_error_pct": -0.09 } }, "tape_calibration_success_pct": 100.0, - "groups": 7, + "groups": 6, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 7.93, - 18.18 + 7.08, + 17.18 ], "p90_in": [ - 20.8, - 70.18 + 16.59, + 46.81 ] }, "3-10m": { "median_in": [ - 14.68, - 37.36 + 12.7, + 31.36 ], "p90_in": [ - 32.47, - 100.54 + 27.53, + 99.78 ] } } @@ -17468,20 +17468,20 @@ "one_known_distance": { "1-3m": { "pairs": 2625000, - "failed_pct": 0.34, - "median_in": 4.01, - "p90_in": 18.54, - "scale_error_pct": -0.31 + "failed_pct": 1.03, + "median_in": 4.05, + "p90_in": 19.28, + "scale_error_pct": -0.57 }, "3-10m": { "pairs": 2625000, - "failed_pct": 0.36, - "median_in": 6.82, - "p90_in": 35.84, - "scale_error_pct": -0.51 + "failed_pct": 1.04, + "median_in": 6.99, + "p90_in": 38.3, + "scale_error_pct": -0.61 } }, - "tape_calibration_success_pct": 99.7, + "tape_calibration_success_pct": 99.0, "groups": 7, "noise_draws": 5, "interval_95": { @@ -17511,57 +17511,57 @@ "none": { "1-3m": { "pairs": 90000, - "failed_pct": 0.11, - "median_in": 4.89, - "p90_in": 10.8, - "scale_error_pct": -5.62 + "failed_pct": 0.12, + "median_in": 4.87, + "p90_in": 10.82, + "scale_error_pct": -5.64 }, "3-10m": { "pairs": 90000, - "failed_pct": 0.23, - "median_in": 10.03, - "p90_in": 18.41, - "scale_error_pct": -5.17 + "failed_pct": 0.21, + "median_in": 10.09, + "p90_in": 18.66, + "scale_error_pct": -5.13 } }, "one_known_distance": { "1-3m": { "pairs": 2250000, - "failed_pct": 0.37, - "median_in": 3.12, - "p90_in": 15.2, - "scale_error_pct": 0.06 + "failed_pct": 1.19, + "median_in": 3.18, + "p90_in": 16.06, + "scale_error_pct": -0.23 }, "3-10m": { "pairs": 2250000, - "failed_pct": 0.5, + "failed_pct": 1.28, "median_in": 5.91, - "p90_in": 35.59, - "scale_error_pct": 0.37 + "p90_in": 35.3, + "scale_error_pct": 0.08 } }, - "tape_calibration_success_pct": 99.7, + "tape_calibration_success_pct": 98.9, "groups": 6, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 3.72, - 6.55 + 3.69, + 6.51 ], "p90_in": [ - 8.62, - 13.06 + 8.57, + 13.09 ] }, "3-10m": { "median_in": [ - 7.08, - 12.55 + 7.13, + 12.62 ], "p90_in": [ - 15.18, - 26.86 + 15.31, + 26.99 ] } } @@ -17572,55 +17572,55 @@ "pairs": 90000, "failed_pct": 0.08, "median_in": 5.54, - "p90_in": 13.41, - "scale_error_pct": -5.77 + "p90_in": 13.45, + "scale_error_pct": -5.73 }, "3-10m": { "pairs": 88930, "failed_pct": 0.16, "median_in": 10.53, - "p90_in": 20.48, + "p90_in": 20.49, "scale_error_pct": -6.75 } }, "one_known_distance": { "1-3m": { "pairs": 2250000, - "failed_pct": 0.34, - "median_in": 4.01, - "p90_in": 20.45, - "scale_error_pct": -0.13 + "failed_pct": 1.14, + "median_in": 3.98, + "p90_in": 20.55, + "scale_error_pct": -0.51 }, "3-10m": { "pairs": 2223250, - "failed_pct": 0.43, + "failed_pct": 1.24, "median_in": 6.39, - "p90_in": 31.31, - "scale_error_pct": -0.74 + "p90_in": 31.63, + "scale_error_pct": -1.02 } }, - "tape_calibration_success_pct": 99.7, + "tape_calibration_success_pct": 98.9, "groups": 6, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 3.81, - 7.42 + 3.79, + 7.41 ], "p90_in": [ - 9.13, - 17.12 + 9.14, + 17.14 ] }, "3-10m": { "median_in": [ - 7.75, + 7.74, 14.38 ], "p90_in": [ - 14.26, - 23.75 + 14.3, + 23.76 ] } } @@ -17632,58 +17632,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 0.49, - "median_in": 6.74, - "p90_in": 15.0, - "scale_error_pct": 2.3 + "pairs": 105000, + "failed_pct": 0.06, + "median_in": 6.57, + "p90_in": 13.93, + "scale_error_pct": 1.85 }, "3-10m": { - "pairs": 120000, - "failed_pct": 1.68, - "median_in": 14.85, - "p90_in": 28.68, - "scale_error_pct": 3.67 + "pairs": 105000, + "failed_pct": 0.17, + "median_in": 14.6, + "p90_in": 27.74, + "scale_error_pct": 2.52 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 0.49, - "median_in": 2.25, - "p90_in": 11.54, - "scale_error_pct": -0.04 + "pairs": 2625000, + "failed_pct": 0.63, + "median_in": 1.96, + "p90_in": 8.35, + "scale_error_pct": -0.11 }, "3-10m": { - "pairs": 3000000, - "failed_pct": 1.68, - "median_in": 3.96, - "p90_in": 21.43, - "scale_error_pct": -0.04 + "pairs": 2625000, + "failed_pct": 0.74, + "median_in": 3.46, + "p90_in": 15.55, + "scale_error_pct": -0.11 } }, - "tape_calibration_success_pct": 100.0, - "groups": 8, + "tape_calibration_success_pct": 99.4, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 2.71, - 9.43 + 2.81, + 9.37 ], "p90_in": [ - 11.21, - 19.19 + 10.77, + 16.56 ] }, "3-10m": { "median_in": [ - 5.56, - 19.62 + 6.33, + 19.89 ], "p90_in": [ - 19.93, - 32.38 + 18.97, + 31.6 ] } } @@ -17691,58 +17691,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 105000, - "failed_pct": 0.99, - "median_in": 5.22, - "p90_in": 15.28, - "scale_error_pct": -0.21 + "pairs": 90000, + "failed_pct": 0.91, + "median_in": 4.96, + "p90_in": 13.78, + "scale_error_pct": -1.38 }, "3-10m": { - "pairs": 105000, - "failed_pct": 2.6, - "median_in": 13.28, - "p90_in": 31.67, - "scale_error_pct": 2.54 + "pairs": 90000, + "failed_pct": 0.96, + "median_in": 12.97, + "p90_in": 31.41, + "scale_error_pct": 1.87 } }, "one_known_distance": { "1-3m": { - "pairs": 2625000, - "failed_pct": 0.99, - "median_in": 2.37, - "p90_in": 13.01, - "scale_error_pct": -0.24 + "pairs": 2250000, + "failed_pct": 1.58, + "median_in": 2.16, + "p90_in": 8.58, + "scale_error_pct": -0.36 }, "3-10m": { - "pairs": 2625000, - "failed_pct": 2.6, - "median_in": 5.13, - "p90_in": 32.62, - "scale_error_pct": 0.18 + "pairs": 2250000, + "failed_pct": 1.63, + "median_in": 4.52, + "p90_in": 22.6, + "scale_error_pct": 0.14 } }, - "tape_calibration_success_pct": 100.0, - "groups": 7, + "tape_calibration_success_pct": 99.3, + "groups": 6, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 2.1, - 8.49 + 1.8, + 8.73 ], "p90_in": [ - 10.09, - 19.05 + 7.07, + 17.49 ] }, "3-10m": { "median_in": [ - 5.87, - 21.93 + 4.36, + 21.73 ], "p90_in": [ - 20.88, - 39.85 + 17.31, + 38.61 ] } } @@ -17750,58 +17750,58 @@ "edges": { "none": { "1-3m": { - "pairs": 105000, - "failed_pct": 1.0, - "median_in": 8.53, - "p90_in": 33.76, - "scale_error_pct": 1.25 + "pairs": 90000, + "failed_pct": 1.13, + "median_in": 7.17, + "p90_in": 17.33, + "scale_error_pct": 0.4 }, "3-10m": { - "pairs": 103930, - "failed_pct": 0.58, - "median_in": 15.14, - "p90_in": 49.53, - "scale_error_pct": 2.18 + "pairs": 88930, + "failed_pct": 0.68, + "median_in": 12.03, + "p90_in": 23.56, + "scale_error_pct": 1.45 } }, "one_known_distance": { "1-3m": { - "pairs": 2625000, - "failed_pct": 1.0, - "median_in": 3.89, - "p90_in": 27.33, - "scale_error_pct": 0.85 + "pairs": 2250000, + "failed_pct": 1.8, + "median_in": 3.23, + "p90_in": 14.83, + "scale_error_pct": -0.08 }, "3-10m": { - "pairs": 2598250, - "failed_pct": 0.58, - "median_in": 5.6, - "p90_in": 38.11, - "scale_error_pct": 0.19 + "pairs": 2223250, + "failed_pct": 1.35, + "median_in": 4.73, + "p90_in": 18.94, + "scale_error_pct": -0.71 } }, - "tape_calibration_success_pct": 100.0, - "groups": 7, + "tape_calibration_success_pct": 99.3, + "groups": 6, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 3.41, - 13.24 + 2.25, + 11.49 ], "p90_in": [ - 14.23, - 50.64 + 12.23, + 27.6 ] }, "3-10m": { "median_in": [ - 4.68, - 23.13 + 3.6, + 19.15 ], "p90_in": [ - 20.29, - 68.33 + 15.83, + 35.18 ] } } @@ -17811,58 +17811,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 0.22, - "median_in": 6.09, - "p90_in": 14.47, - "scale_error_pct": 2.04 + "pairs": 105000, + "failed_pct": 0.01, + "median_in": 5.62, + "p90_in": 13.04, + "scale_error_pct": 2.51 }, "3-10m": { - "pairs": 120000, - "failed_pct": 1.16, - "median_in": 13.82, - "p90_in": 25.9, - "scale_error_pct": 4.29 + "pairs": 105000, + "failed_pct": 0.07, + "median_in": 13.45, + "p90_in": 24.29, + "scale_error_pct": 4.28 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 0.62, - "median_in": 2.53, - "p90_in": 16.06, - "scale_error_pct": 0.14 + "pairs": 2625000, + "failed_pct": 1.04, + "median_in": 2.21, + "p90_in": 9.7, + "scale_error_pct": -0.07 }, "3-10m": { - "pairs": 3000000, - "failed_pct": 1.56, - "median_in": 4.48, - "p90_in": 33.38, - "scale_error_pct": 0.51 + "pairs": 2625000, + "failed_pct": 1.1, + "median_in": 3.93, + "p90_in": 18.71, + "scale_error_pct": 0.27 } }, - "tape_calibration_success_pct": 99.6, - "groups": 8, + "tape_calibration_success_pct": 99.0, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 3.62, - 8.96 + 3.48, + 8.79 ], "p90_in": [ - 9.97, - 21.07 + 9.34, + 15.85 ] }, "3-10m": { "median_in": [ - 8.28, - 18.43 + 8.02, + 18.27 ], "p90_in": [ - 19.11, - 30.17 + 18.28, + 27.83 ] } } @@ -17870,58 +17870,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 105000, - "failed_pct": 0.45, - "median_in": 5.62, - "p90_in": 14.97, - "scale_error_pct": -0.24 + "pairs": 90000, + "failed_pct": 0.42, + "median_in": 5.0, + "p90_in": 12.65, + "scale_error_pct": -0.0 }, "3-10m": { - "pairs": 105000, - "failed_pct": 2.03, - "median_in": 13.66, - "p90_in": 30.32, - "scale_error_pct": 4.3 + "pairs": 90000, + "failed_pct": 0.48, + "median_in": 13.05, + "p90_in": 28.82, + "scale_error_pct": 4.23 } }, "one_known_distance": { "1-3m": { - "pairs": 2625000, - "failed_pct": 0.91, - "median_in": 2.37, - "p90_in": 19.5, - "scale_error_pct": 0.09 + "pairs": 2250000, + "failed_pct": 1.62, + "median_in": 2.03, + "p90_in": 9.92, + "scale_error_pct": -0.21 }, "3-10m": { - "pairs": 2625000, - "failed_pct": 2.49, - "median_in": 5.29, - "p90_in": 52.31, - "scale_error_pct": 1.17 + "pairs": 2250000, + "failed_pct": 1.68, + "median_in": 4.32, + "p90_in": 27.4, + "scale_error_pct": 0.87 } }, - "tape_calibration_success_pct": 99.5, - "groups": 7, + "tape_calibration_success_pct": 98.8, + "groups": 6, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 2.94, - 8.86 + 2.45, + 8.36 ], "p90_in": [ - 9.94, - 23.82 + 7.23, + 15.93 ] }, "3-10m": { "median_in": [ - 8.92, - 19.96 + 7.74, + 19.26 ], "p90_in": [ - 21.61, - 46.64 + 15.99, + 45.05 ] } } @@ -17929,58 +17929,58 @@ "edges": { "none": { "1-3m": { - "pairs": 105000, - "failed_pct": 0.44, - "median_in": 7.48, - "p90_in": 20.53, - "scale_error_pct": 2.76 + "pairs": 90000, + "failed_pct": 0.51, + "median_in": 6.95, + "p90_in": 16.07, + "scale_error_pct": 2.37 }, "3-10m": { - "pairs": 103930, - "failed_pct": 0.3, - "median_in": 13.34, - "p90_in": 34.93, - "scale_error_pct": 3.17 + "pairs": 88930, + "failed_pct": 0.35, + "median_in": 11.39, + "p90_in": 21.07, + "scale_error_pct": 2.8 } }, "one_known_distance": { "1-3m": { - "pairs": 2625000, - "failed_pct": 0.9, - "median_in": 4.24, - "p90_in": 26.56, - "scale_error_pct": 1.02 + "pairs": 2250000, + "failed_pct": 1.71, + "median_in": 3.47, + "p90_in": 15.05, + "scale_error_pct": 0.26 }, "3-10m": { - "pairs": 2598250, - "failed_pct": 0.76, - "median_in": 6.53, - "p90_in": 49.41, - "scale_error_pct": 1.21 + "pairs": 2223250, + "failed_pct": 1.56, + "median_in": 5.18, + "p90_in": 21.98, + "scale_error_pct": 0.43 } }, - "tape_calibration_success_pct": 99.5, - "groups": 7, + "tape_calibration_success_pct": 98.8, + "groups": 6, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 4.02, - 11.23 + 3.34, + 11.08 ], "p90_in": [ - 13.99, - 41.33 + 12.18, + 20.81 ] }, "3-10m": { "median_in": [ - 6.23, - 17.33 + 5.25, + 15.86 ], "p90_in": [ - 18.09, - 58.44 + 15.87, + 30.04 ] } } @@ -18007,20 +18007,20 @@ "one_known_distance": { "1-3m": { "pairs": 2625000, - "failed_pct": 0.23, - "median_in": 3.11, - "p90_in": 11.8, - "scale_error_pct": -0.02 + "failed_pct": 1.83, + "median_in": 3.17, + "p90_in": 12.64, + "scale_error_pct": -0.23 }, "3-10m": { "pairs": 2625000, - "failed_pct": 0.24, - "median_in": 5.16, - "p90_in": 22.41, - "scale_error_pct": 0.12 + "failed_pct": 1.84, + "median_in": 5.36, + "p90_in": 24.93, + "scale_error_pct": 0.06 } }, - "tape_calibration_success_pct": 99.8, + "tape_calibration_success_pct": 98.2, "groups": 7, "noise_draws": 5, "interval_95": { @@ -18050,57 +18050,57 @@ "none": { "1-3m": { "pairs": 90000, - "failed_pct": 0.11, - "median_in": 5.12, - "p90_in": 12.5, - "scale_error_pct": -1.59 + "failed_pct": 0.12, + "median_in": 5.1, + "p90_in": 12.57, + "scale_error_pct": -1.65 }, "3-10m": { "pairs": 90000, - "failed_pct": 0.23, - "median_in": 11.4, - "p90_in": 25.92, - "scale_error_pct": 2.6 + "failed_pct": 0.21, + "median_in": 11.69, + "p90_in": 26.14, + "scale_error_pct": 2.58 } }, "one_known_distance": { "1-3m": { "pairs": 2250000, - "failed_pct": 0.37, - "median_in": 2.99, - "p90_in": 10.82, - "scale_error_pct": 0.24 + "failed_pct": 2.26, + "median_in": 3.17, + "p90_in": 11.87, + "scale_error_pct": -0.02 }, "3-10m": { "pairs": 2250000, - "failed_pct": 0.5, - "median_in": 5.75, - "p90_in": 26.88, - "scale_error_pct": 1.46 + "failed_pct": 2.34, + "median_in": 5.83, + "p90_in": 27.79, + "scale_error_pct": 1.25 } }, - "tape_calibration_success_pct": 99.7, + "tape_calibration_success_pct": 97.9, "groups": 6, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 2.9, - 8.3 + 2.94, + 8.22 ], "p90_in": [ - 7.29, - 15.61 + 7.31, + 15.66 ] }, "3-10m": { "median_in": [ - 6.52, - 17.84 + 6.51, + 18.17 ], "p90_in": [ - 14.53, - 32.7 + 14.7, + 32.84 ] } } @@ -18110,56 +18110,56 @@ "1-3m": { "pairs": 90000, "failed_pct": 0.08, - "median_in": 6.46, - "p90_in": 14.58, - "scale_error_pct": 1.04 + "median_in": 6.53, + "p90_in": 14.56, + "scale_error_pct": 1.15 }, "3-10m": { "pairs": 88930, "failed_pct": 0.16, - "median_in": 7.81, - "p90_in": 18.55, + "median_in": 7.85, + "p90_in": 18.56, "scale_error_pct": -0.5 } }, "one_known_distance": { "1-3m": { "pairs": 2250000, - "failed_pct": 0.34, - "median_in": 4.24, - "p90_in": 16.13, - "scale_error_pct": 1.97 + "failed_pct": 2.21, + "median_in": 4.14, + "p90_in": 17.2, + "scale_error_pct": 1.49 }, "3-10m": { "pairs": 2223250, - "failed_pct": 0.43, - "median_in": 6.48, - "p90_in": 25.61, - "scale_error_pct": 0.45 + "failed_pct": 2.32, + "median_in": 6.5, + "p90_in": 26.99, + "scale_error_pct": 0.12 } }, - "tape_calibration_success_pct": 99.7, + "tape_calibration_success_pct": 97.9, "groups": 6, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 2.96, + 2.97, 10.21 ], "p90_in": [ 11.39, - 18.89 + 18.9 ] }, "3-10m": { "median_in": [ - 2.71, - 15.12 + 2.7, + 15.13 ], "p90_in": [ - 12.85, - 23.98 + 12.88, + 24.13 ] } } @@ -18171,58 +18171,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 12.99, - "median_in": 3.45, - "p90_in": Infinity, - "scale_error_pct": -1.2 + "pairs": 105000, + "failed_pct": 14.34, + "median_in": 2.84, + "p90_in": "failed", + "scale_error_pct": -1.81 }, "3-10m": { - "pairs": 120000, - "failed_pct": 14.18, - "median_in": 8.06, - "p90_in": Infinity, - "scale_error_pct": -1.04 + "pairs": 105000, + "failed_pct": 14.46, + "median_in": 6.18, + "p90_in": "failed", + "scale_error_pct": -1.79 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 12.99, - "median_in": 2.17, - "p90_in": Infinity, - "scale_error_pct": -0.13 + "pairs": 2625000, + "failed_pct": 14.34, + "median_in": 1.71, + "p90_in": "failed", + "scale_error_pct": -0.06 }, "3-10m": { - "pairs": 3000000, - "failed_pct": 14.18, - "median_in": 4.15, - "p90_in": Infinity, - "scale_error_pct": -0.21 + "pairs": 2625000, + "failed_pct": 14.46, + "median_in": 3.21, + "p90_in": "failed", + "scale_error_pct": -0.19 } }, - "tape_calibration_success_pct": 87.5, - "groups": 8, + "tape_calibration_success_pct": 85.7, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 1.89, - 13.03 + 1.59, + 8.05 ], "p90_in": [ - 6.9, - Infinity + 4.75, + "failed" ] }, "3-10m": { "median_in": [ - 3.54, - 32.35 + 2.8, + 16.66 ], "p90_in": [ - 14.33, - Infinity + 10.99, + "failed" ] } } @@ -18230,58 +18230,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 105000, - "failed_pct": 15.27, - "median_in": 3.53, - "p90_in": Infinity, - "scale_error_pct": -1.68 + "pairs": 90000, + "failed_pct": 17.58, + "median_in": 2.67, + "p90_in": "failed", + "scale_error_pct": -2.24 }, "3-10m": { - "pairs": 105000, - "failed_pct": 16.89, - "median_in": 10.09, - "p90_in": Infinity, - "scale_error_pct": 0.13 + "pairs": 90000, + "failed_pct": 17.63, + "median_in": 6.42, + "p90_in": "failed", + "scale_error_pct": -0.81 } }, "one_known_distance": { "1-3m": { - "pairs": 2625000, - "failed_pct": 15.27, - "median_in": 2.4, - "p90_in": Infinity, - "scale_error_pct": -0.15 + "pairs": 2250000, + "failed_pct": 17.58, + "median_in": 1.87, + "p90_in": "failed", + "scale_error_pct": 0.04 }, "3-10m": { - "pairs": 2625000, - "failed_pct": 16.89, - "median_in": 5.2, - "p90_in": Infinity, - "scale_error_pct": 0.35 + "pairs": 2250000, + "failed_pct": 17.63, + "median_in": 4.36, + "p90_in": "failed", + "scale_error_pct": 0.58 } }, - "tape_calibration_success_pct": 85.7, - "groups": 7, + "tape_calibration_success_pct": 83.3, + "groups": 6, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 1.75, - 30.21 + 1.51, + "failed" ], "p90_in": [ - 7.32, - Infinity + 3.81, + "failed" ] }, "3-10m": { "median_in": [ - 3.31, - 65.29 + 2.61, + "failed" ], "p90_in": [ - 17.32, - Infinity + 12.11, + "failed" ] } } @@ -18289,58 +18289,58 @@ "edges": { "none": { "1-3m": { - "pairs": 105000, - "failed_pct": 15.29, - "median_in": 5.12, - "p90_in": Infinity, - "scale_error_pct": -1.29 + "pairs": 90000, + "failed_pct": 17.8, + "median_in": 3.96, + "p90_in": "failed", + "scale_error_pct": -2.21 }, "3-10m": { - "pairs": 103930, - "failed_pct": 13.99, - "median_in": 9.76, - "p90_in": Infinity, - "scale_error_pct": -1.34 + "pairs": 88930, + "failed_pct": 16.34, + "median_in": 6.81, + "p90_in": "failed", + "scale_error_pct": -2.05 } }, "one_known_distance": { "1-3m": { - "pairs": 2625000, - "failed_pct": 15.29, - "median_in": 3.78, - "p90_in": Infinity, - "scale_error_pct": 0.39 + "pairs": 2250000, + "failed_pct": 17.8, + "median_in": 3.02, + "p90_in": "failed", + "scale_error_pct": -0.03 }, "3-10m": { - "pairs": 2598250, - "failed_pct": 13.99, - "median_in": 5.72, - "p90_in": Infinity, - "scale_error_pct": -0.01 + "pairs": 2223250, + "failed_pct": 16.34, + "median_in": 4.6, + "p90_in": "failed", + "scale_error_pct": -0.32 } }, - "tape_calibration_success_pct": 85.7, - "groups": 7, + "tape_calibration_success_pct": 83.3, + "groups": 6, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 2.63, - 63.81 + 2.1, + "failed" ], "p90_in": [ - 11.15, - Infinity + 7.13, + "failed" ] }, "3-10m": { "median_in": [ - 3.99, - 89.5 + 3.16, + 32.22 ], "p90_in": [ - 17.47, - Infinity + 12.48, + "failed" ] } } @@ -18350,58 +18350,58 @@ "surface interior": { "none": { "1-3m": { - "pairs": 120000, - "failed_pct": 2.73, - "median_in": 2.84, - "p90_in": 14.58, - "scale_error_pct": -2.34 + "pairs": 105000, + "failed_pct": 2.87, + "median_in": 2.54, + "p90_in": 7.92, + "scale_error_pct": -2.69 }, "3-10m": { - "pairs": 120000, - "failed_pct": 3.66, - "median_in": 5.43, - "p90_in": 35.03, - "scale_error_pct": -2.27 + "pairs": 105000, + "failed_pct": 2.93, + "median_in": 4.74, + "p90_in": 15.84, + "scale_error_pct": -2.63 } }, "one_known_distance": { "1-3m": { - "pairs": 3000000, - "failed_pct": 3.23, - "median_in": 2.49, - "p90_in": 20.52, + "pairs": 2625000, + "failed_pct": 3.67, + "median_in": 2.03, + "p90_in": 9.88, "scale_error_pct": -0.06 }, "3-10m": { - "pairs": 3000000, - "failed_pct": 4.16, - "median_in": 4.61, - "p90_in": 44.74, - "scale_error_pct": -0.02 + "pairs": 2625000, + "failed_pct": 3.73, + "median_in": 3.71, + "p90_in": 20.27, + "scale_error_pct": -0.12 } }, - "tape_calibration_success_pct": 97.0, - "groups": 8, + "tape_calibration_success_pct": 96.3, + "groups": 7, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 1.89, - 4.51 + 1.71, + 3.68 ], "p90_in": [ - 5.4, - Infinity + 4.61, + "failed" ] }, "3-10m": { "median_in": [ - 3.25, - 10.4 + 2.64, + 7.91 ], "p90_in": [ - 9.46, - Infinity + 7.53, + "failed" ] } } @@ -18409,58 +18409,58 @@ "vertical interior": { "none": { "1-3m": { - "pairs": 105000, - "failed_pct": 3.34, - "median_in": 3.11, - "p90_in": 19.36, - "scale_error_pct": -3.01 + "pairs": 90000, + "failed_pct": 3.76, + "median_in": 2.79, + "p90_in": 8.13, + "scale_error_pct": -3.38 }, "3-10m": { - "pairs": 105000, - "failed_pct": 4.9, - "median_in": 5.72, - "p90_in": 69.53, - "scale_error_pct": -2.22 + "pairs": 90000, + "failed_pct": 3.83, + "median_in": 4.86, + "p90_in": 22.7, + "scale_error_pct": -2.63 } }, "one_known_distance": { "1-3m": { - "pairs": 2625000, - "failed_pct": 3.91, - "median_in": 2.34, - "p90_in": 27.16, + "pairs": 2250000, + "failed_pct": 4.69, + "median_in": 1.78, + "p90_in": 10.14, "scale_error_pct": -0.07 }, "3-10m": { - "pairs": 2625000, - "failed_pct": 5.47, - "median_in": 5.19, - "p90_in": 62.63, - "scale_error_pct": 0.45 + "pairs": 2250000, + "failed_pct": 4.76, + "median_in": 4.13, + "p90_in": 27.68, + "scale_error_pct": 0.33 } }, - "tape_calibration_success_pct": 96.6, - "groups": 7, + "tape_calibration_success_pct": 95.7, + "groups": 6, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 2.03, - 4.92 + 1.62, + 4.05 ], "p90_in": [ - 5.55, - Infinity + 4.44, + "failed" ] }, "3-10m": { "median_in": [ - 3.69, - 12.24 + 2.93, + 8.16 ], "p90_in": [ - 11.04, - Infinity + 7.29, + "failed" ] } } @@ -18468,58 +18468,58 @@ "edges": { "none": { "1-3m": { - "pairs": 105000, - "failed_pct": 3.3, - "median_in": 3.69, - "p90_in": 31.64, - "scale_error_pct": -2.86 + "pairs": 90000, + "failed_pct": 3.84, + "median_in": 3.27, + "p90_in": 13.83, + "scale_error_pct": -3.33 }, "3-10m": { - "pairs": 103930, - "failed_pct": 3.18, - "median_in": 6.38, - "p90_in": 59.1, - "scale_error_pct": -2.99 + "pairs": 88930, + "failed_pct": 3.72, + "median_in": 5.58, + "p90_in": 16.66, + "scale_error_pct": -3.32 } }, "one_known_distance": { "1-3m": { - "pairs": 2625000, - "failed_pct": 3.87, - "median_in": 3.46, - "p90_in": 34.3, - "scale_error_pct": -0.13 + "pairs": 2250000, + "failed_pct": 4.78, + "median_in": 2.77, + "p90_in": 16.42, + "scale_error_pct": -0.4 }, "3-10m": { - "pairs": 2598250, - "failed_pct": 3.76, - "median_in": 5.5, - "p90_in": 60.56, - "scale_error_pct": -0.17 + "pairs": 2223250, + "failed_pct": 4.67, + "median_in": 4.29, + "p90_in": 24.52, + "scale_error_pct": -0.36 } }, - "tape_calibration_success_pct": 96.6, - "groups": 7, + "tape_calibration_success_pct": 95.7, + "groups": 6, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 2.84, - 6.97 + 2.66, + 4.75 ], "p90_in": [ - 7.32, - Infinity + 5.42, + "failed" ] }, "3-10m": { "median_in": [ - 4.52, - 15.34 + 4.12, + 7.63 ], "p90_in": [ - 11.02, - Infinity + 8.47, + "failed" ] } } @@ -18546,20 +18546,20 @@ "one_known_distance": { "1-3m": { "pairs": 2625000, - "failed_pct": 0.69, + "failed_pct": 1.14, "median_in": 2.09, - "p90_in": 8.94, - "scale_error_pct": -0.28 + "p90_in": 8.76, + "scale_error_pct": -0.38 }, "3-10m": { "pairs": 2625000, - "failed_pct": 0.7, - "median_in": 3.91, - "p90_in": 17.58, - "scale_error_pct": -0.41 + "failed_pct": 1.16, + "median_in": 3.96, + "p90_in": 18.02, + "scale_error_pct": -0.45 } }, - "tape_calibration_success_pct": 99.3, + "tape_calibration_success_pct": 98.9, "groups": 7, "noise_draws": 5, "interval_95": { @@ -18589,57 +18589,57 @@ "none": { "1-3m": { "pairs": 90000, - "failed_pct": 0.11, + "failed_pct": 0.12, "median_in": 2.31, - "p90_in": 4.97, + "p90_in": 4.99, "scale_error_pct": -2.91 }, "3-10m": { "pairs": 90000, - "failed_pct": 0.23, - "median_in": 4.64, - "p90_in": 12.24, - "scale_error_pct": -2.69 + "failed_pct": 0.21, + "median_in": 4.68, + "p90_in": 12.66, + "scale_error_pct": -2.67 } }, "one_known_distance": { "1-3m": { "pairs": 2250000, - "failed_pct": 0.77, + "failed_pct": 1.32, "median_in": 1.72, - "p90_in": 7.99, - "scale_error_pct": 0.17 + "p90_in": 7.83, + "scale_error_pct": 0.03 }, "3-10m": { "pairs": 2250000, - "failed_pct": 0.9, - "median_in": 3.76, - "p90_in": 19.69, - "scale_error_pct": 0.33 + "failed_pct": 1.41, + "median_in": 3.78, + "p90_in": 19.06, + "scale_error_pct": 0.25 } }, - "tape_calibration_success_pct": 99.3, + "tape_calibration_success_pct": 98.8, "groups": 6, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 1.68, + 1.67, 2.8 ], "p90_in": [ - 3.77, - 6.77 + 3.73, + 6.88 ] }, "3-10m": { "median_in": [ - 3.45, - 6.32 + 3.49, + 6.5 ], "p90_in": [ - 6.34, - 21.25 + 6.41, + 21.73 ] } } @@ -18649,56 +18649,56 @@ "1-3m": { "pairs": 90000, "failed_pct": 0.08, - "median_in": 3.14, - "p90_in": 9.61, - "scale_error_pct": -3.15 + "median_in": 3.15, + "p90_in": 9.65, + "scale_error_pct": -3.14 }, "3-10m": { "pairs": 88930, "failed_pct": 0.16, - "median_in": 5.49, - "p90_in": 14.76, + "median_in": 5.5, + "p90_in": 14.78, "scale_error_pct": -3.52 } }, "one_known_distance": { "1-3m": { "pairs": 2250000, - "failed_pct": 0.74, - "median_in": 2.66, - "p90_in": 13.5, - "scale_error_pct": 0.04 + "failed_pct": 1.28, + "median_in": 2.63, + "p90_in": 12.99, + "scale_error_pct": -0.11 }, "3-10m": { "pairs": 2223250, - "failed_pct": 0.84, - "median_in": 3.95, - "p90_in": 19.47, - "scale_error_pct": -0.5 + "failed_pct": 1.38, + "median_in": 3.92, + "p90_in": 18.5, + "scale_error_pct": -0.6 } }, - "tape_calibration_success_pct": 99.3, + "tape_calibration_success_pct": 98.8, "groups": 6, "noise_draws": 5, "interval_95": { "1-3m": { "median_in": [ - 2.73, - 3.8 + 2.75, + 3.82 ], "p90_in": [ - 4.91, + 4.93, 15.01 ] }, "3-10m": { "median_in": [ - 4.68, - 7.55 + 4.69, + 7.59 ], "p90_in": [ 7.22, - 20.26 + 20.29 ] } } diff --git a/experiments/evals/results/pose_priors.md b/experiments/evals/results/pose_priors.md index 58246f74b..61c7366b1 100644 --- a/experiments/evals/results/pose_priors.md +++ b/experiments/evals/results/pose_priors.md @@ -10,94 +10,94 @@ Model-scale cells: median [95% interval] / p90 [95% interval], inches. The point | Method | Views | Groups x noise draws | Model scale: 1-3 m | 3-10 m | Scale error: 1-3 m | 3-10 m | One taped distance: 1-3 m | 3-10 m | Tape calibrated | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| mapanything@392, images only | 2 | 8 x 1 | 11.4 [8.9, 14.6] / 30.2 [22.5, inf] | 39.4 [31.6, 49.8] / fails [93.3, inf] | -13.3% | -14.6% | 6.2 / fails (14% failed) | 19.7 / fails (16% failed) | 94% | -| mapanything@392, images only | 4 | 8 x 1 | 10.9 [8.8, 13.8] / 24.0 [19.6, 28.8] | 36.9 [30.8, 46.3] / 87.3 [67.8, 108.1] | -12.7% | -14.1% | 5.6 / 62.2 | 16.8 / fails (11% failed) | 95% | -| mapanything@392 + exact poses | 2 | 8 x 1 | 12.1 [9.0, 16.2] / 29.5 [24.2, inf] | 41.4 [31.6, 53.6] / fails [95.8, inf] | -14.2% | -15.7% | 5.7 / fails (14% failed) | 18.1 / fails (16% failed) | 94% | -| mapanything@392 + exact poses | 4 | 8 x 1 | 9.6 [7.8, 11.8] / 20.8 [18.2, 23.0] | 31.9 [27.4, 38.0] / 73.9 [58.9, 105.8] | -11.4% | -12.2% | 5.1 / fails (11% failed) | 15.5 / fails (12% failed) | 94% | -| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 2 | 8 x 1 | 12.1 [8.8, 16.3] / 30.1 [24.4, inf] | 41.4 [31.2, 54.5] / fails [96.7, inf] | -14.1% | -15.7% | 5.9 / fails (14% failed) | 18.7 / fails (17% failed) | 94% | -| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 4 | 8 x 1 | 10.7 [8.9, 12.9] / 21.7 [19.5, 23.9] | 35.3 [31.1, 40.7] / 76.0 [62.5, 107.0] | -12.9% | -13.4% | 5.1 / fails (11% failed) | 15.3 / fails (12% failed) | 94% | -| mapanything@392 + modern_assumed poses | 2 | 8 x 1 | 12.2 [9.2, 16.3] / 29.6 [24.4, inf] | 41.8 [32.0, 53.9] / fails [96.3, inf] | -14.3% | -15.9% | 5.7 / fails (14% failed) | 18.0 / fails (16% failed) | 94% | -| mapanything@392 + modern_assumed poses | 4 | 8 x 1 | 9.9 [8.1, 12.1] / 21.3 [18.7, 23.6] | 33.1 [28.3, 39.2] / 75.4 [59.9, 106.9] | -11.8% | -12.6% | 5.1 / fails (11% failed) | 15.4 / fails (12% failed) | 94% | -| MoGe-2 placed with exact poses | 2 | 8 x 1 | 5.9 [5.0, 7.0] / 12.0 [10.0, 14.1] | 20.4 [17.2, 23.2] / 47.7 [37.9, 56.5] | -6.2% | -7.3% | 3.0 / 12.6 | 9.8 / 43.6 | 100% | -| MoGe-2 placed with exact poses | 4 | 8 x 1 | 5.7 [5.0, 6.5] / 11.6 [10.1, 13.8] | 19.2 [16.5, 21.6] / 46.9 [38.2, 53.6] | -5.9% | -7.0% | 3.3 / 12.3 | 10.8 / 42.1 | 100% | -| MoGe-2 placed with exact poses | 8 | 8 x 1 | 5.9 [5.2, 6.7] / 11.8 [10.7, 13.0] | 19.9 [17.5, 21.9] / 46.1 [39.8, 55.8] | -6.0% | -7.2% | 3.5 / 14.4 | 11.0 / 49.0 | 98% | -| MoGe-2 rescaled by triangulation, exact poses | 2 | 8 x 1 | 2.3 [1.5, 3.5] / 8.3 [6.1, 11.4] | 8.2 [5.1, 12.0] / 36.4 [24.5, 48.2] | -1.4% | -2.4% | 3.0 / 12.5 | 10.2 / 43.5 | 100% | -| MoGe-2 rescaled by triangulation, exact poses | 4 | 8 x 1 | 1.8 [1.3, 2.4] / 7.7 [5.9, 10.4] | 5.9 [4.0, 8.3] / 32.6 [23.5, 44.7] | -0.2% | -1.0% | 3.1 / 12.4 | 10.7 / 43.2 | 99% | -| MoGe-2 rescaled by triangulation, exact poses | 8 | 8 x 1 | 1.7 [1.3, 2.0] / 7.5 [5.8, 9.4] | 5.3 [4.0, 6.7] / 30.3 [20.9, 46.3] | +0.0% | -0.6% | 2.9 / 12.2 | 9.8 / 43.4 | 100% | -| MoGe-2 placed with advio_2018 poses | 2 | 8 x 5 | 6.0 [5.1, 7.1] / 12.1 [10.2, 14.1] | 20.5 [17.1, 23.2] / 47.8 [38.2, 56.3] | -6.2% | -7.3% | 3.1 / 12.7 | 10.1 / 43.5 | 100% | -| MoGe-2 placed with advio_2018 poses | 4 | 8 x 5 | 5.8 [5.1, 6.5] / 11.6 [10.1, 13.4] | 19.4 [16.6, 22.3] / 48.6 [38.4, 56.0] | -6.0% | -7.2% | 3.5 / 12.1 | 11.0 / 43.0 | 100% | -| MoGe-2 placed with advio_2018 poses | 8 | 8 x 5 | 6.0 [5.4, 6.7] / 11.7 [10.6, 12.9] | 20.1 [17.6, 22.2] / 46.0 [39.6, 55.6] | -6.2% | -7.5% | 3.6 / 14.1 | 11.3 / 47.8 | 99% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 2 | 8 x 5 | 11.9 [7.2, 16.6] / 39.6 [24.6, inf] | 39.5 [26.0, 54.5] / 128.7 [83.4, inf] | -9.2% | -11.9% | 3.4 / 22.7 | 11.5 / 86.4 | 92% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 4 | 8 x 5 | 8.7 [5.4, 11.8] / 19.6 [15.6, 22.0] | 28.7 [17.2, 38.7] / 71.4 [57.0, 80.7] | -9.8% | -11.9% | 4.2 / 16.5 | 13.4 / 52.5 | 100% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 8 | 8 x 5 | 6.5 [4.6, 9.6] / 17.3 [11.9, 20.4] | 22.5 [15.3, 33.8] / 64.2 [38.9, 81.3] | -7.1% | -8.7% | 4.1 / 16.1 | 12.6 / 51.0 | 99% | -| MoGe-2 placed with modern_assumed poses | 2 | 8 x 5 | 5.9 [5.1, 7.0] / 12.0 [10.2, 14.1] | 20.4 [17.1, 23.2] / 47.8 [38.0, 56.2] | -6.2% | -7.3% | 3.1 / 12.6 | 9.9 / 43.7 | 100% | -| MoGe-2 placed with modern_assumed poses | 4 | 8 x 5 | 5.7 [5.0, 6.5] / 11.6 [10.1, 13.7] | 19.3 [16.6, 21.9] / 47.0 [38.1, 53.7] | -5.9% | -7.0% | 3.4 / 12.4 | 10.8 / 42.5 | 100% | -| MoGe-2 placed with modern_assumed poses | 8 | 8 x 5 | 5.9 [5.3, 6.7] / 11.7 [10.6, 12.8] | 20.0 [17.5, 22.0] / 46.0 [39.7, 55.5] | -6.1% | -7.3% | 3.4 / 14.2 | 10.9 / 48.2 | 98% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 2 | 8 x 5 | 5.2 [3.9, 6.8] / 15.5 [11.7, inf] | 17.1 [13.1, 23.2] / 56.3 [40.2, inf] | -3.3% | -4.9% | 3.1 / 14.6 | 10.6 / 50.8 | 97% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 4 | 8 x 5 | 3.0 [2.6, 3.6] / 10.4 [8.8, 12.3] | 10.2 [8.2, 12.9] / 39.3 [31.3, 51.1] | -2.2% | -3.2% | 3.7 / 14.3 | 12.3 / 46.8 | 99% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 8 | 8 x 5 | 2.5 [2.1, 2.9] / 8.6 [7.1, 9.8] | 8.2 [6.7, 10.1] / 34.8 [26.3, 49.9] | -1.6% | -2.7% | 3.3 / 13.2 | 10.7 / 45.4 | 99% | +| mapanything@392, images only | 2 | 8 x 1 | 11.4 [8.9, 14.6] / 30.2 [22.5, inf] | 39.4 [31.6, 49.8] / fails [93.3, inf] | -13.3% | -14.6% | 6.2 / fails (13% failed) | 19.5 / fails (16% failed) | 94% | +| mapanything@392, images only | 4 | 8 x 1 | 10.9 [8.8, 13.8] / 24.0 [19.6, 28.8] | 36.9 [30.8, 46.3] / 87.3 [67.8, 108.1] | -12.7% | -14.1% | 5.6 / 40.7 | 16.9 / fails (11% failed) | 96% | +| mapanything@392 + exact poses | 2 | 8 x 1 | 12.1 [9.0, 16.2] / 29.5 [24.2, inf] | 41.4 [31.6, 53.6] / fails [95.8, inf] | -14.2% | -15.7% | 5.7 / fails (13% failed) | 17.9 / fails (16% failed) | 94% | +| mapanything@392 + exact poses | 4 | 8 x 1 | 9.6 [7.8, 11.8] / 20.8 [18.2, 23.0] | 31.9 [27.4, 38.0] / 73.9 [58.9, 105.8] | -11.4% | -12.2% | 5.2 / fails (10% failed) | 15.5 / fails (11% failed) | 95% | +| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 2 | 8 x 1 | 12.1 [8.8, 16.3] / 30.1 [24.4, inf] | 41.4 [31.2, 54.5] / fails [96.7, inf] | -14.1% | -15.7% | 5.8 / fails (14% failed) | 18.5 / fails (16% failed) | 94% | +| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 4 | 8 x 1 | 10.7 [8.9, 12.9] / 21.7 [19.5, 23.9] | 35.3 [31.1, 40.7] / 76.0 [62.5, 107.0] | -12.9% | -13.4% | 5.1 / fails (10% failed) | 15.3 / fails (11% failed) | 95% | +| mapanything@392 + modern_assumed poses | 2 | 8 x 1 | 12.2 [9.2, 16.3] / 29.6 [24.4, inf] | 41.8 [32.0, 53.9] / fails [96.3, inf] | -14.3% | -15.9% | 5.7 / fails (13% failed) | 17.9 / fails (16% failed) | 94% | +| mapanything@392 + modern_assumed poses | 4 | 8 x 1 | 9.9 [8.1, 12.1] / 21.3 [18.7, 23.6] | 33.1 [28.3, 39.2] / 75.4 [59.9, 106.9] | -11.8% | -12.6% | 5.1 / fails (10% failed) | 15.5 / fails (11% failed) | 95% | +| MoGe-2 placed with exact poses | 2 | 8 x 1 | 5.9 [5.0, 7.0] / 12.0 [10.0, 14.1] | 20.4 [17.2, 23.2] / 47.7 [37.9, 56.5] | -6.2% | -7.3% | 3.0 / 12.5 | 9.6 / 43.3 | 100% | +| MoGe-2 placed with exact poses | 4 | 8 x 1 | 5.7 [5.0, 6.5] / 11.6 [10.1, 13.8] | 19.2 [16.5, 21.6] / 46.9 [38.2, 53.6] | -5.9% | -7.0% | 3.3 / 12.3 | 10.6 / 41.9 | 100% | +| MoGe-2 placed with exact poses | 8 | 8 x 1 | 5.9 [5.2, 6.7] / 11.8 [10.7, 13.0] | 19.9 [17.5, 21.9] / 46.1 [39.8, 55.8] | -6.0% | -7.2% | 3.5 / 14.4 | 10.9 / 48.8 | 98% | +| MoGe-2 rescaled by triangulation, exact poses | 2 | 8 x 1 | 2.3 [1.5, 3.5] / 8.3 [6.1, 11.4] | 8.2 [5.1, 12.0] / 36.4 [24.5, 48.2] | -1.4% | -2.4% | 3.0 / 12.2 | 10.0 / 42.7 | 100% | +| MoGe-2 rescaled by triangulation, exact poses | 4 | 8 x 1 | 1.8 [1.3, 2.4] / 7.7 [5.9, 10.4] | 5.9 [4.0, 8.3] / 32.6 [23.5, 44.7] | -0.2% | -1.0% | 3.1 / 12.3 | 10.5 / 42.9 | 99% | +| MoGe-2 rescaled by triangulation, exact poses | 8 | 8 x 1 | 1.7 [1.3, 2.0] / 7.5 [5.8, 9.4] | 5.3 [4.0, 6.7] / 30.3 [20.9, 46.3] | +0.0% | -0.6% | 2.9 / 12.0 | 9.7 / 43.1 | 100% | +| MoGe-2 placed with advio_2018 poses | 2 | 8 x 5 | 6.0 [5.1, 7.1] / 12.1 [10.2, 14.1] | 20.5 [17.1, 23.2] / 47.8 [38.2, 56.3] | -6.2% | -7.3% | 3.0 / 12.6 | 9.9 / 43.2 | 100% | +| MoGe-2 placed with advio_2018 poses | 4 | 8 x 5 | 5.8 [5.1, 6.5] / 11.6 [10.1, 13.4] | 19.4 [16.6, 22.3] / 48.6 [38.4, 56.0] | -6.0% | -7.2% | 3.4 / 12.2 | 10.8 / 43.0 | 100% | +| MoGe-2 placed with advio_2018 poses | 8 | 8 x 5 | 6.0 [5.4, 6.7] / 11.7 [10.6, 12.9] | 20.1 [17.6, 22.2] / 46.0 [39.6, 55.6] | -6.2% | -7.5% | 3.6 / 14.0 | 11.3 / 47.7 | 99% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 2 | 8 x 5 | 11.9 [7.2, 16.6] / 39.6 [24.6, inf] | 39.5 [26.0, 54.5] / 128.7 [83.4, inf] | -9.2% | -11.9% | 3.4 / 22.6 | 11.3 / 85.8 | 92% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 4 | 8 x 5 | 8.7 [5.4, 11.8] / 19.6 [15.6, 22.0] | 28.7 [17.2, 38.7] / 71.4 [57.0, 80.7] | -9.8% | -11.9% | 4.2 / 16.6 | 13.2 / 52.6 | 100% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 8 | 8 x 5 | 6.5 [4.6, 9.6] / 17.3 [11.9, 20.4] | 22.5 [15.3, 33.8] / 64.2 [38.9, 81.3] | -7.1% | -8.7% | 4.0 / 16.1 | 12.5 / 51.2 | 99% | +| MoGe-2 placed with modern_assumed poses | 2 | 8 x 5 | 5.9 [5.1, 7.0] / 12.0 [10.2, 14.1] | 20.4 [17.1, 23.2] / 47.8 [38.0, 56.2] | -6.2% | -7.3% | 3.0 / 12.6 | 9.8 / 43.3 | 100% | +| MoGe-2 placed with modern_assumed poses | 4 | 8 x 5 | 5.7 [5.0, 6.5] / 11.6 [10.1, 13.7] | 19.3 [16.6, 21.9] / 47.0 [38.1, 53.7] | -5.9% | -7.0% | 3.3 / 12.3 | 10.7 / 42.3 | 100% | +| MoGe-2 placed with modern_assumed poses | 8 | 8 x 5 | 5.9 [5.3, 6.7] / 11.7 [10.6, 12.8] | 20.0 [17.5, 22.0] / 46.0 [39.7, 55.5] | -6.1% | -7.3% | 3.4 / 14.1 | 10.8 / 47.9 | 98% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 2 | 8 x 5 | 5.2 [3.9, 6.8] / 15.5 [11.7, inf] | 17.1 [13.1, 23.2] / 56.3 [40.2, inf] | -3.3% | -4.9% | 3.1 / 14.4 | 10.4 / 50.0 | 97% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 4 | 8 x 5 | 3.0 [2.6, 3.6] / 10.4 [8.8, 12.3] | 10.2 [8.2, 12.9] / 39.3 [31.3, 51.1] | -2.2% | -3.2% | 3.7 / 14.3 | 12.1 / 46.9 | 99% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 8 | 8 x 5 | 2.5 [2.1, 2.9] / 8.6 [7.1, 9.8] | 8.2 [6.7, 10.1] / 34.8 [26.3, 49.9] | -1.6% | -2.7% | 3.3 / 13.1 | 10.6 / 45.2 | 99% | ## facade, all points, vertical interior | Method | Views | Groups x noise draws | Model scale: 1-3 m | 3-10 m | Scale error: 1-3 m | 3-10 m | One taped distance: 1-3 m | 3-10 m | Tape calibrated | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| mapanything@392, images only | 2 | 8 x 1 | 11.1 [8.5, 14.6] / 24.8 [19.2, 28.8] | 37.3 [29.5, 47.5] / 95.5 [73.0, 128.6] | -13.8% | -15.0% | 5.5 / fails (11% failed) | 17.6 / fails (13% failed) | 94% | -| mapanything@392, images only | 4 | 8 x 1 | 10.7 [8.4, 13.9] / 22.2 [17.9, 25.1] | 35.2 [29.1, 44.6] / 77.9 [60.2, 95.1] | -13.1% | -14.3% | 5.0 / 22.8 | 15.3 / 83.1 | 95% | -| mapanything@392 + exact poses | 2 | 8 x 1 | 11.8 [8.6, 16.4] / 25.7 [21.9, 28.9] | 39.2 [29.8, 51.5] / 96.8 [76.0, 129.6] | -14.7% | -16.5% | 5.1 / fails (11% failed) | 16.0 / fails (13% failed) | 94% | -| mapanything@392 + exact poses | 4 | 8 x 1 | 9.5 [7.6, 11.8] / 19.0 [15.8, 20.6] | 30.2 [26.1, 35.9] / 64.6 [51.5, 89.9] | -11.7% | -12.2% | 4.7 / 22.6 | 14.0 / 90.5 | 94% | -| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 2 | 8 x 1 | 11.8 [8.4, 16.4] / 25.8 [21.9, 28.9] | 39.2 [29.4, 52.3] / 97.1 [77.7, 130.3] | -14.7% | -16.7% | 5.2 / fails (11% failed) | 16.6 / fails (13% failed) | 94% | -| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 4 | 8 x 1 | 10.4 [8.7, 12.8] / 20.0 [17.2, 21.2] | 33.6 [29.8, 38.6] / 66.5 [56.0, 92.7] | -13.1% | -13.5% | 4.7 / 22.7 | 13.9 / 92.7 | 94% | -| mapanything@392 + modern_assumed poses | 2 | 8 x 1 | 11.9 [8.7, 16.4] / 25.8 [21.9, 29.1] | 39.6 [30.2, 52.0] / 96.9 [76.4, 130.8] | -14.8% | -16.7% | 5.1 / fails (11% failed) | 16.0 / fails (13% failed) | 94% | -| mapanything@392 + modern_assumed poses | 4 | 8 x 1 | 9.8 [7.9, 12.1] / 19.4 [16.1, 21.1] | 31.4 [27.1, 37.0] / 66.1 [52.5, 91.2] | -12.1% | -12.6% | 4.6 / 22.5 | 13.9 / 90.0 | 94% | -| MoGe-2 placed with exact poses | 2 | 8 x 1 | 5.9 [5.0, 6.8] / 11.2 [9.6, 13.4] | 19.6 [15.3, 22.5] / 46.7 [35.6, 58.0] | -6.5% | -7.4% | 2.8 / 11.3 | 9.2 / 42.7 | 100% | -| MoGe-2 placed with exact poses | 4 | 8 x 1 | 5.6 [4.9, 6.4] / 10.9 [9.5, 12.9] | 18.3 [15.0, 21.3] / 46.5 [36.0, 56.1] | -6.2% | -7.0% | 3.1 / 11.1 | 10.3 / 41.0 | 100% | -| MoGe-2 placed with exact poses | 8 | 8 x 1 | 5.9 [5.3, 6.5] / 11.2 [10.3, 12.0] | 19.1 [15.7, 21.7] / 46.5 [38.1, 59.0] | -6.3% | -7.3% | 3.2 / 13.0 | 10.6 / 48.5 | 98% | -| MoGe-2 rescaled by triangulation, exact poses | 2 | 8 x 1 | 2.1 [1.3, 3.4] / 7.3 [5.3, 10.2] | 7.5 [4.1, 11.6] / 35.5 [21.6, 56.8] | -1.6% | -2.4% | 2.8 / 11.3 | 9.6 / 42.7 | 100% | -| MoGe-2 rescaled by triangulation, exact poses | 4 | 8 x 1 | 1.6 [1.1, 2.2] / 6.5 [5.0, 8.3] | 5.3 [3.5, 8.0] / 32.5 [20.4, 52.8] | -0.4% | -0.9% | 2.9 / 11.2 | 10.1 / 42.5 | 99% | -| MoGe-2 rescaled by triangulation, exact poses | 8 | 8 x 1 | 1.5 [1.1, 1.9] / 6.2 [4.8, 7.3] | 4.8 [3.4, 6.3] / 29.9 [18.3, 53.9] | -0.1% | -0.5% | 2.7 / 10.9 | 9.2 / 42.7 | 100% | -| MoGe-2 placed with advio_2018 poses | 2 | 8 x 5 | 5.9 [5.0, 6.9] / 11.3 [9.8, 13.5] | 19.6 [15.0, 22.5] / 46.8 [36.0, 58.1] | -6.5% | -7.4% | 2.8 / 11.3 | 9.4 / 42.6 | 100% | -| MoGe-2 placed with advio_2018 poses | 4 | 8 x 5 | 5.7 [5.0, 6.4] / 10.9 [9.7, 12.7] | 18.5 [15.4, 21.9] / 48.0 [36.4, 58.5] | -6.3% | -7.3% | 3.2 / 11.0 | 10.4 / 41.9 | 100% | -| MoGe-2 placed with advio_2018 poses | 8 | 8 x 5 | 6.0 [5.4, 6.6] / 11.2 [10.2, 12.0] | 19.2 [15.7, 21.8] / 46.3 [37.9, 59.7] | -6.6% | -7.5% | 3.3 / 12.8 | 10.8 / 47.0 | 99% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 2 | 8 x 5 | 11.8 [6.8, 16.7] / 38.3 [24.2, inf] | 38.6 [25.2, 53.4] / 122.9 [80.3, inf] | -9.7% | -12.3% | 3.1 / 19.5 | 10.9 / 88.7 | 92% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 4 | 8 x 5 | 8.9 [5.4, 12.2] / 19.6 [15.6, 22.1] | 28.3 [17.0, 37.8] / 70.9 [56.2, 83.3] | -10.5% | -12.4% | 3.9 / 15.3 | 12.9 / 51.4 | 100% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 8 | 8 x 5 | 6.6 [4.4, 10.0] / 17.2 [11.3, 20.2] | 21.8 [14.6, 33.1] / 63.4 [37.2, 91.4] | -7.5% | -8.8% | 3.8 / 14.8 | 12.2 / 50.6 | 99% | -| MoGe-2 placed with modern_assumed poses | 2 | 8 x 5 | 5.9 [5.0, 6.9] / 11.3 [9.7, 13.4] | 19.6 [15.1, 22.5] / 46.8 [35.6, 58.1] | -6.5% | -7.4% | 2.8 / 11.3 | 9.3 / 42.7 | 100% | -| MoGe-2 placed with modern_assumed poses | 4 | 8 x 5 | 5.6 [4.9, 6.3] / 10.9 [9.6, 12.9] | 18.4 [14.9, 21.5] / 46.7 [35.8, 57.0] | -6.2% | -7.0% | 3.1 / 11.1 | 10.3 / 41.5 | 100% | -| MoGe-2 placed with modern_assumed poses | 8 | 8 x 5 | 5.9 [5.3, 6.5] / 11.2 [10.2, 12.0] | 19.1 [15.6, 21.7] / 46.3 [38.0, 59.7] | -6.4% | -7.4% | 3.2 / 12.8 | 10.4 / 47.6 | 98% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 2 | 8 x 5 | 5.0 [3.7, 6.7] / 14.9 [10.9, inf] | 16.4 [12.4, 22.8] / 55.0 [38.2, inf] | -3.6% | -5.0% | 2.9 / 13.2 | 10.0 / 50.0 | 97% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 4 | 8 x 5 | 2.8 [2.4, 3.4] / 9.3 [7.8, 11.0] | 9.5 [7.2, 12.3] / 38.9 [28.3, 59.4] | -2.5% | -3.2% | 3.5 / 13.1 | 11.8 / 45.9 | 99% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 8 | 8 x 5 | 2.3 [1.9, 2.8] / 7.6 [6.4, 8.6] | 7.5 [5.9, 9.3] / 34.7 [24.1, 60.3] | -1.9% | -2.6% | 3.1 / 11.9 | 10.2 / 44.8 | 99% | +| mapanything@392, images only | 2 | 8 x 1 | 11.1 [8.5, 14.5] / 25.1 [19.3, 29.7] | 37.6 [29.6, 47.8] / 94.4 [73.4, 128.4] | -13.7% | -15.0% | 5.5 / fails (10% failed) | 17.5 / fails (12% failed) | 94% | +| mapanything@392, images only | 4 | 8 x 1 | 10.7 [8.5, 13.8] / 22.3 [18.1, 25.3] | 35.3 [29.3, 44.7] / 77.2 [60.3, 93.3] | -13.1% | -14.3% | 5.1 / 22.3 | 15.4 / 76.0 | 96% | +| mapanything@392 + exact poses | 2 | 8 x 1 | 11.8 [8.5, 16.3] / 25.9 [22.0, 29.1] | 39.6 [29.9, 51.6] / 96.5 [76.6, 128.7] | -14.6% | -16.6% | 5.1 / fails (11% failed) | 16.0 / fails (12% failed) | 94% | +| mapanything@392 + exact poses | 4 | 8 x 1 | 9.5 [7.6, 11.8] / 19.0 [16.0, 20.8] | 30.3 [26.0, 36.2] / 64.2 [51.8, 89.4] | -11.7% | -12.2% | 4.7 / 22.1 | 14.0 / 80.7 | 95% | +| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 2 | 8 x 1 | 11.8 [8.4, 16.4] / 26.0 [22.1, 30.0] | 39.5 [29.5, 52.2] / 96.7 [77.8, 129.1] | -14.6% | -16.7% | 5.2 / fails (11% failed) | 16.5 / fails (13% failed) | 94% | +| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 4 | 8 x 1 | 10.5 [8.7, 12.8] / 20.0 [17.4, 21.2] | 33.6 [29.7, 38.5] / 66.1 [55.9, 91.5] | -13.1% | -13.4% | 4.7 / 21.8 | 13.9 / 81.3 | 95% | +| mapanything@392 + modern_assumed poses | 2 | 8 x 1 | 12.0 [8.7, 16.4] / 26.1 [22.1, 29.3] | 39.9 [30.3, 52.1] / 96.8 [77.0, 129.4] | -14.8% | -16.7% | 5.1 / fails (11% failed) | 15.9 / fails (12% failed) | 94% | +| mapanything@392 + modern_assumed poses | 4 | 8 x 1 | 9.8 [7.9, 12.1] / 19.4 [16.5, 21.2] | 31.5 [26.8, 37.2] / 65.7 [52.9, 90.7] | -12.0% | -12.6% | 4.7 / 21.9 | 14.0 / 80.7 | 95% | +| MoGe-2 placed with exact poses | 2 | 8 x 1 | 5.9 [5.0, 6.9] / 11.3 [9.6, 13.4] | 19.6 [15.5, 22.6] / 46.7 [35.5, 58.0] | -6.5% | -7.4% | 2.7 / 11.1 | 9.1 / 42.5 | 100% | +| MoGe-2 placed with exact poses | 4 | 8 x 1 | 5.6 [5.0, 6.4] / 11.0 [9.5, 13.0] | 18.5 [15.2, 21.4] / 46.9 [36.1, 56.1] | -6.2% | -7.1% | 3.1 / 11.0 | 10.2 / 40.9 | 100% | +| MoGe-2 placed with exact poses | 8 | 8 x 1 | 5.8 [5.2, 6.5] / 11.3 [10.4, 12.1] | 19.2 [15.9, 21.7] / 46.5 [38.3, 58.6] | -6.3% | -7.4% | 3.2 / 13.0 | 10.5 / 48.1 | 98% | +| MoGe-2 rescaled by triangulation, exact poses | 2 | 8 x 1 | 2.1 [1.3, 3.5] / 7.3 [5.3, 9.8] | 7.8 [4.2, 11.7] / 35.8 [21.9, 56.1] | -1.6% | -2.5% | 2.7 / 10.9 | 9.4 / 42.1 | 100% | +| MoGe-2 rescaled by triangulation, exact poses | 4 | 8 x 1 | 1.6 [1.1, 2.3] / 6.5 [5.0, 8.3] | 5.5 [3.6, 8.1] / 32.6 [20.6, 51.6] | -0.4% | -1.0% | 2.9 / 11.1 | 10.0 / 42.3 | 99% | +| MoGe-2 rescaled by triangulation, exact poses | 8 | 8 x 1 | 1.5 [1.1, 1.9] / 6.2 [4.9, 7.3] | 5.0 [3.5, 6.5] / 30.1 [18.0, 53.4] | -0.1% | -0.6% | 2.7 / 10.8 | 9.1 / 42.4 | 100% | +| MoGe-2 placed with advio_2018 poses | 2 | 8 x 5 | 5.9 [5.0, 6.9] / 11.4 [9.8, 13.5] | 19.7 [15.2, 22.7] / 46.8 [36.1, 58.2] | -6.5% | -7.5% | 2.8 / 11.2 | 9.3 / 42.5 | 100% | +| MoGe-2 placed with advio_2018 poses | 4 | 8 x 5 | 5.7 [5.1, 6.4] / 11.0 [9.7, 12.6] | 18.7 [15.4, 22.2] / 48.3 [36.6, 58.3] | -6.3% | -7.4% | 3.2 / 10.9 | 10.3 / 41.8 | 100% | +| MoGe-2 placed with advio_2018 poses | 8 | 8 x 5 | 6.0 [5.3, 6.6] / 11.2 [10.2, 12.2] | 19.4 [15.8, 22.0] / 46.5 [37.9, 59.5] | -6.6% | -7.6% | 3.3 / 12.8 | 10.8 / 46.8 | 99% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 2 | 8 x 5 | 11.8 [6.8, 16.7] / 38.1 [24.2, inf] | 38.9 [25.4, 53.5] / 123.3 [80.6, inf] | -9.7% | -12.3% | 3.1 / 19.3 | 10.8 / 87.9 | 92% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 4 | 8 x 5 | 8.8 [5.4, 12.2] / 19.5 [15.5, 22.0] | 28.6 [17.1, 37.9] / 70.8 [56.3, 82.9] | -10.5% | -12.4% | 3.9 / 15.3 | 12.7 / 51.6 | 100% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 8 | 8 x 5 | 6.6 [4.5, 9.9] / 17.3 [11.4, 20.4] | 21.7 [14.6, 33.3] / 64.1 [37.2, 90.3] | -7.5% | -8.8% | 3.8 / 14.9 | 12.0 / 50.6 | 99% | +| MoGe-2 placed with modern_assumed poses | 2 | 8 x 5 | 5.9 [5.0, 6.9] / 11.4 [9.8, 13.4] | 19.6 [15.2, 22.7] / 46.8 [35.6, 58.0] | -6.5% | -7.4% | 2.8 / 11.1 | 9.2 / 42.5 | 100% | +| MoGe-2 placed with modern_assumed poses | 4 | 8 x 5 | 5.6 [5.0, 6.3] / 11.0 [9.7, 12.9] | 18.6 [15.1, 21.8] / 47.0 [36.2, 56.8] | -6.2% | -7.1% | 3.1 / 11.1 | 10.2 / 41.4 | 100% | +| MoGe-2 placed with modern_assumed poses | 8 | 8 x 5 | 5.8 [5.2, 6.5] / 11.2 [10.2, 12.1] | 19.3 [15.8, 21.8] / 46.4 [37.9, 59.5] | -6.4% | -7.4% | 3.2 / 12.8 | 10.3 / 47.2 | 98% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 2 | 8 x 5 | 5.0 [3.7, 6.8] / 14.8 [11.0, inf] | 16.6 [12.5, 22.9] / 55.5 [38.3, inf] | -3.6% | -5.1% | 2.8 / 12.8 | 9.8 / 49.5 | 97% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 4 | 8 x 5 | 2.9 [2.4, 3.5] / 9.3 [7.8, 11.0] | 9.6 [7.3, 12.4] / 39.2 [28.3, 58.6] | -2.5% | -3.3% | 3.4 / 13.1 | 11.6 / 46.0 | 99% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 8 | 8 x 5 | 2.4 [2.0, 2.8] / 7.7 [6.5, 8.7] | 7.6 [6.0, 9.4] / 34.9 [24.0, 58.7] | -1.9% | -2.8% | 3.0 / 11.8 | 10.1 / 44.6 | 99% | ## facade, all points, edges | Method | Views | Groups x noise draws | Model scale: 1-3 m | 3-10 m | Scale error: 1-3 m | 3-10 m | One taped distance: 1-3 m | 3-10 m | Tape calibrated | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | mapanything@392, images only | 2 | 8 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | -10.6% | -12.4% | inf / fails (86% failed) | inf / fails (92% failed) | 94% | -| mapanything@392, images only | 4 | 8 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | -9.1% | -13.3% | inf / fails (81% failed) | inf / fails (87% failed) | 95% | +| mapanything@392, images only | 4 | 8 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | -9.1% | -13.3% | inf / fails (81% failed) | inf / fails (87% failed) | 96% | | mapanything@392 + exact poses | 2 | 8 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | -12.7% | -13.4% | inf / fails (86% failed) | inf / fails (92% failed) | 94% | -| mapanything@392 + exact poses | 4 | 8 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | -9.9% | -11.6% | inf / fails (82% failed) | inf / fails (88% failed) | 94% | +| mapanything@392 + exact poses | 4 | 8 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | -9.9% | -11.6% | inf / fails (82% failed) | inf / fails (88% failed) | 95% | | mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 2 | 8 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | -13.1% | -13.4% | inf / fails (86% failed) | inf / fails (92% failed) | 94% | -| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 4 | 8 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | -10.9% | -12.2% | inf / fails (82% failed) | inf / fails (89% failed) | 94% | -| mapanything@392 + modern_assumed poses | 2 | 8 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | -12.9% | -13.4% | inf / fails (86% failed) | inf / fails (92% failed) | 94% | -| mapanything@392 + modern_assumed poses | 4 | 8 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | -10.6% | -12.3% | inf / fails (82% failed) | inf / fails (88% failed) | 94% | -| MoGe-2 placed with exact poses | 2 | 8 x 1 | 5.5 [4.3, 7.7] / 26.1 [14.8, 36.7] | 16.0 [12.5, 20.3] / 48.5 [34.9, 59.8] | -2.3% | -4.0% | 5.0 / 30.5 | 13.8 / 55.8 | 100% | -| MoGe-2 placed with exact poses | 4 | 8 x 1 | 5.1 [3.9, 7.2] / 24.6 [14.7, 33.5] | 14.8 [12.0, 18.2] / 45.4 [34.1, 55.4] | -2.3% | -3.9% | 5.2 / 28.5 | 13.7 / 52.7 | 100% | -| MoGe-2 placed with exact poses | 8 | 8 x 1 | 5.3 [3.9, 8.0] / 28.2 [14.9, 43.8] | 15.3 [11.8, 19.3] / 53.2 [35.1, 73.7] | -2.4% | -4.0% | 5.7 / 36.0 | 14.7 / 65.2 | 98% | -| MoGe-2 rescaled by triangulation, exact poses | 2 | 8 x 1 | 3.4 [2.0, 5.3] / 26.9 [14.5, 42.4] | 10.5 [7.3, 15.2] / 47.0 [32.6, 61.9] | +1.9% | +0.9% | 5.0 / 30.1 | 13.9 / 56.0 | 100% | +| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 4 | 8 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | -10.9% | -12.2% | inf / fails (82% failed) | inf / fails (88% failed) | 95% | +| mapanything@392 + modern_assumed poses | 2 | 8 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | -13.0% | -13.4% | inf / fails (86% failed) | inf / fails (92% failed) | 94% | +| mapanything@392 + modern_assumed poses | 4 | 8 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | -10.7% | -12.3% | inf / fails (82% failed) | inf / fails (88% failed) | 95% | +| MoGe-2 placed with exact poses | 2 | 8 x 1 | 5.5 [4.3, 7.7] / 26.1 [14.8, 36.7] | 16.0 [12.5, 20.3] / 48.5 [34.9, 59.8] | -2.3% | -4.0% | 5.0 / 30.6 | 13.8 / 55.9 | 100% | +| MoGe-2 placed with exact poses | 4 | 8 x 1 | 5.1 [3.9, 7.2] / 24.6 [14.7, 33.5] | 14.8 [12.0, 18.2] / 45.4 [34.2, 55.4] | -2.3% | -3.9% | 5.2 / 28.6 | 13.8 / 52.8 | 100% | +| MoGe-2 placed with exact poses | 8 | 8 x 1 | 5.3 [3.9, 8.0] / 28.2 [14.9, 43.8] | 15.3 [11.8, 19.3] / 53.2 [35.1, 73.7] | -2.4% | -4.0% | 5.7 / 36.1 | 14.8 / 65.2 | 98% | +| MoGe-2 rescaled by triangulation, exact poses | 2 | 8 x 1 | 3.4 [2.0, 5.3] / 26.9 [14.5, 42.4] | 10.5 [7.3, 15.2] / 47.0 [32.6, 61.9] | +1.9% | +0.9% | 4.9 / 30.1 | 13.9 / 55.7 | 100% | | MoGe-2 rescaled by triangulation, exact poses | 4 | 8 x 1 | 3.5 [2.5, 5.3] / 27.1 [15.8, 43.2] | 10.7 [8.4, 14.4] / 46.1 [34.4, 60.7] | +3.3% | +2.1% | 5.0 / 29.6 | 13.7 / 53.6 | 99% | -| MoGe-2 rescaled by triangulation, exact poses | 8 | 8 x 1 | 3.7 [2.7, 5.8] / 29.3 [15.4, 52.7] | 11.0 [9.0, 14.6] / 51.2 [31.9, 82.2] | +3.7% | +2.4% | 5.1 / 33.0 | 13.7 / 60.8 | 100% | -| MoGe-2 placed with advio_2018 poses | 2 | 8 x 5 | 5.5 [4.3, 7.7] / 26.2 [15.0, 35.2] | 16.0 [12.9, 20.0] / 48.4 [35.4, 58.5] | -2.3% | -4.0% | 5.1 / 30.5 | 13.8 / 55.9 | 100% | -| MoGe-2 placed with advio_2018 poses | 4 | 8 x 5 | 5.1 [3.9, 7.1] / 25.6 [15.0, 35.4] | 15.0 [12.1, 18.1] / 45.8 [34.3, 55.2] | -1.9% | -3.9% | 5.4 / 29.9 | 13.8 / 52.9 | 100% | -| MoGe-2 placed with advio_2018 poses | 8 | 8 x 5 | 5.3 [4.0, 7.9] / 27.8 [15.3, 44.1] | 15.4 [11.9, 19.0] / 52.7 [34.9, 77.2] | -2.4% | -4.1% | 5.8 / 34.6 | 14.8 / 64.0 | 99% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 2 | 8 x 5 | 12.0 [8.6, 17.0] / 69.2 [26.8, inf] | 35.5 [25.0, 48.4] / 156.5 [74.1, inf] | -4.1% | -7.9% | 5.5 / 67.7 | 15.1 / 102.9 | 92% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 4 | 8 x 5 | 7.5 [5.2, 9.8] / 23.3 [18.3, 29.0] | 24.2 [15.8, 32.4] / 62.5 [48.8, 71.5] | -3.2% | -7.2% | 6.1 / 32.2 | 15.6 / 60.5 | 100% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 8 | 8 x 5 | 6.4 [4.8, 9.0] / 25.4 [16.6, 49.2] | 18.7 [13.4, 28.7] / 68.4 [47.0, 90.1] | -3.1% | -5.4% | 6.4 / 37.6 | 15.9 / 67.9 | 99% | -| MoGe-2 placed with modern_assumed poses | 2 | 8 x 5 | 5.6 [4.3, 7.7] / 26.1 [14.8, 35.7] | 16.0 [12.8, 20.0] / 48.5 [35.3, 58.7] | -2.2% | -4.0% | 5.0 / 30.5 | 13.8 / 55.8 | 100% | -| MoGe-2 placed with modern_assumed poses | 4 | 8 x 5 | 5.1 [3.9, 7.2] / 24.6 [14.6, 33.6] | 14.8 [11.9, 18.1] / 45.5 [34.3, 55.4] | -2.2% | -3.9% | 5.2 / 28.6 | 13.7 / 52.9 | 100% | -| MoGe-2 placed with modern_assumed poses | 8 | 8 x 5 | 5.3 [3.8, 7.9] / 28.0 [15.4, 44.3] | 15.3 [11.6, 19.0] / 53.1 [35.1, 76.3] | -2.4% | -4.0% | 5.7 / 35.6 | 14.6 / 64.7 | 98% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 2 | 8 x 5 | 6.2 [4.7, 8.3] / 34.0 [16.2, inf] | 17.3 [12.9, 24.9] / 65.3 [43.3, inf] | +1.3% | -1.0% | 5.2 / 37.1 | 14.4 / 65.2 | 97% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 4 | 8 x 5 | 4.1 [2.9, 6.2] / 26.8 [15.3, 41.4] | 11.3 [8.3, 14.8] / 47.8 [34.6, 62.6] | +2.1% | +0.2% | 5.7 / 30.3 | 14.8 / 55.9 | 99% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 8 | 8 x 5 | 3.6 [2.6, 5.5] / 27.6 [15.0, 49.6] | 9.9 [7.4, 13.3] / 49.8 [30.8, 81.5] | +2.2% | +0.3% | 5.7 / 34.5 | 14.5 / 63.2 | 99% | +| MoGe-2 rescaled by triangulation, exact poses | 8 | 8 x 1 | 3.7 [2.7, 5.8] / 29.3 [15.4, 52.7] | 11.0 [9.0, 14.6] / 51.2 [31.9, 82.1] | +3.7% | +2.4% | 5.1 / 32.9 | 13.8 / 60.5 | 100% | +| MoGe-2 placed with advio_2018 poses | 2 | 8 x 5 | 5.5 [4.3, 7.7] / 26.2 [15.0, 35.2] | 16.0 [12.9, 20.0] / 48.4 [35.5, 58.5] | -2.3% | -4.0% | 5.1 / 30.6 | 13.8 / 56.1 | 100% | +| MoGe-2 placed with advio_2018 poses | 4 | 8 x 5 | 5.1 [3.9, 7.1] / 25.6 [15.0, 35.4] | 15.0 [12.1, 18.1] / 45.8 [34.3, 55.2] | -1.9% | -3.9% | 5.4 / 29.9 | 13.8 / 53.1 | 100% | +| MoGe-2 placed with advio_2018 poses | 8 | 8 x 5 | 5.3 [4.0, 7.9] / 27.7 [15.2, 44.1] | 15.4 [11.9, 19.0] / 52.7 [34.8, 77.2] | -2.4% | -4.1% | 5.8 / 34.7 | 14.8 / 64.0 | 99% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 2 | 8 x 5 | 12.0 [8.6, 17.0] / 69.2 [26.8, inf] | 35.5 [25.0, 48.4] / 156.5 [74.1, inf] | -4.1% | -7.9% | 5.5 / 67.8 | 15.1 / 102.9 | 92% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 4 | 8 x 5 | 7.5 [5.2, 9.8] / 23.3 [18.3, 29.0] | 24.2 [15.8, 32.4] / 62.5 [48.8, 71.5] | -3.2% | -7.2% | 6.1 / 32.4 | 15.6 / 60.9 | 100% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 8 | 8 x 5 | 6.4 [4.8, 9.0] / 25.4 [16.6, 49.2] | 18.7 [13.4, 28.7] / 68.4 [47.0, 90.1] | -3.1% | -5.4% | 6.4 / 37.7 | 16.0 / 68.1 | 99% | +| MoGe-2 placed with modern_assumed poses | 2 | 8 x 5 | 5.6 [4.3, 7.7] / 26.1 [14.8, 35.7] | 16.0 [12.8, 20.0] / 48.5 [35.4, 58.7] | -2.2% | -4.0% | 5.0 / 30.6 | 13.8 / 55.9 | 100% | +| MoGe-2 placed with modern_assumed poses | 4 | 8 x 5 | 5.1 [3.9, 7.2] / 24.6 [14.6, 33.6] | 14.8 [11.9, 18.1] / 45.5 [34.3, 55.4] | -2.2% | -3.9% | 5.2 / 28.6 | 13.7 / 53.0 | 100% | +| MoGe-2 placed with modern_assumed poses | 8 | 8 x 5 | 5.3 [3.8, 7.9] / 28.0 [15.4, 44.3] | 15.3 [11.6, 19.0] / 53.1 [35.1, 76.3] | -2.4% | -4.0% | 5.7 / 35.7 | 14.7 / 64.7 | 98% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 2 | 8 x 5 | 6.2 [4.7, 8.3] / 33.9 [16.2, inf] | 17.3 [12.9, 24.9] / 65.3 [43.3, inf] | +1.3% | -1.0% | 5.2 / 37.2 | 14.4 / 65.1 | 97% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 4 | 8 x 5 | 4.1 [2.9, 6.2] / 26.8 [15.3, 41.4] | 11.3 [8.3, 14.8] / 47.8 [34.6, 62.6] | +2.1% | +0.2% | 5.6 / 30.6 | 14.8 / 56.3 | 99% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 8 | 8 x 5 | 3.6 [2.6, 5.5] / 27.6 [15.0, 49.6] | 9.9 [7.4, 13.3] / 49.8 [30.8, 81.5] | +2.2% | +0.3% | 5.6 / 34.6 | 14.5 / 63.0 | 99% | ## facade, within 6 m, surface interior @@ -134,32 +134,32 @@ Model-scale cells: median [95% interval] / p90 [95% interval], inches. The point | Method | Views | Groups x noise draws | Model scale: 1-3 m | 3-10 m | Scale error: 1-3 m | 3-10 m | One taped distance: 1-3 m | 3-10 m | Tape calibrated | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| mapanything@392, images only | 2 | 1 x 1 | 11.8 [11.8, 11.8] / 18.6 [18.6, 18.6] | 22.9 [22.9, 22.9] / 32.8 [32.8, 32.8] | +15.2% | +15.4% | 4.0 / fails (37% failed) | 6.9 / fails (38% failed) | 68% | -| mapanything@392, images only | 4 | 1 x 1 | 9.8 [9.8, 9.8] / 16.0 [16.0, 16.0] | 21.0 [21.0, 21.0] / 28.3 [28.3, 28.3] | +13.5% | +14.3% | 3.9 / fails (22% failed) | 5.5 / fails (23% failed) | 80% | -| mapanything@392 + exact poses | 2 | 1 x 1 | 9.4 [9.4, 9.4] / 15.4 [15.4, 15.4] | 18.2 [18.2, 18.2] / 25.3 [25.3, 25.3] | +12.3% | +12.2% | 4.5 / fails (36% failed) | 7.3 / fails (38% failed) | 68% | -| mapanything@392 + exact poses | 4 | 1 x 1 | 4.5 [4.5, 4.5] / 7.7 [7.7, 7.7] | 8.8 [8.8, 8.8] / 12.2 [12.2, 12.2] | +5.9% | +5.9% | 3.0 / fails (26% failed) | 4.7 / fails (27% failed) | 76% | -| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 2 | 1 x 1 | 9.3 [9.3, 9.3] / 15.0 [15.0, 15.0] | 17.7 [17.7, 17.7] / 24.5 [24.5, 24.5] | +12.0% | +11.8% | 4.4 / fails (36% failed) | 7.3 / fails (38% failed) | 68% | -| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 4 | 1 x 1 | 5.0 [5.0, 5.0] / 9.5 [9.5, 9.5] | 11.5 [11.5, 11.5] / 16.1 [16.1, 16.1] | +6.7% | +7.7% | 3.9 / fails (26% failed) | 6.1 / fails (27% failed) | 76% | -| mapanything@392 + modern_assumed poses | 2 | 1 x 1 | 9.1 [9.1, 9.1] / 14.9 [14.9, 14.9] | 17.5 [17.5, 17.5] / 24.4 [24.4, 24.4] | +11.8% | +11.7% | 4.5 / fails (36% failed) | 7.4 / fails (38% failed) | 68% | -| mapanything@392 + modern_assumed poses | 4 | 1 x 1 | 3.9 [3.9, 3.9] / 7.1 [7.1, 7.1] | 7.6 [7.6, 7.6] / 11.1 [11.1, 11.1] | +5.2% | +5.1% | 3.0 / fails (26% failed) | 4.8 / fails (27% failed) | 76% | -| MoGe-2 placed with exact poses | 2 | 1 x 1 | 15.8 [15.8, 15.8] / 24.2 [24.2, 24.2] | 26.1 [26.1, 26.1] / 32.3 [32.3, 32.3] | +20.7% | +17.7% | 3.4 / 8.7 | 4.9 / 12.1 | 100% | -| MoGe-2 placed with exact poses | 4 | 1 x 1 | 15.8 [15.8, 15.8] / 23.3 [23.3, 23.3] | 25.4 [25.4, 25.4] / 31.1 [31.1, 31.1] | +20.4% | +17.3% | 3.4 / 8.8 | 5.0 / 13.1 | 100% | -| MoGe-2 placed with exact poses | 8 | 1 x 1 | 15.4 [15.4, 15.4] / 22.6 [22.6, 22.6] | 25.3 [25.3, 25.3] / 31.0 [31.0, 31.0] | +20.3% | +17.3% | 5.0 / 13.5 | 6.6 / 28.4 | 100% | -| MoGe-2 rescaled by triangulation, exact poses | 2 | 1 x 1 | 3.4 [3.4, 3.4] / 9.2 [9.2, 9.2] | 3.1 [3.1, 3.1] / 7.6 [7.6, 7.6] | +4.7% | +1.8% | 3.4 / 8.7 | 4.9 / 12.2 | 100% | -| MoGe-2 rescaled by triangulation, exact poses | 4 | 1 x 1 | 8.2 [8.2, 8.2] / 14.9 [14.9, 14.9] | 11.2 [11.2, 11.2] / 15.6 [15.6, 15.6] | +10.6% | +7.4% | 3.8 / 10.6 | 5.0 / 14.8 | 100% | -| MoGe-2 rescaled by triangulation, exact poses | 8 | 1 x 1 | 7.6 [7.6, 7.6] / 12.7 [12.7, 12.7] | 11.2 [11.2, 11.2] / 14.8 [14.8, 14.8] | +9.8% | +7.7% | 3.5 / 9.0 | 5.2 / 15.9 | 100% | -| MoGe-2 placed with advio_2018 poses | 2 | 1 x 5 | 15.8 [15.7, 16.1] / 24.4 [23.9, 25.3] | 26.0 [25.8, 26.3] / 32.6 [31.9, 33.6] | +20.7% | +17.6% | 3.5 / 9.1 | 5.0 / 12.8 | 100% | -| MoGe-2 placed with advio_2018 poses | 4 | 1 x 5 | 15.7 [15.5, 15.9] / 23.3 [22.9, 23.9] | 25.0 [24.7, 25.3] / 31.2 [30.7, 31.5] | +20.3% | +16.8% | 3.4 / 8.6 | 4.7 / 13.7 | 100% | -| MoGe-2 placed with advio_2018 poses | 8 | 1 x 5 | 13.5 [13.2, 13.9] / 20.6 [20.2, 20.9] | 23.8 [23.6, 24.0] / 30.4 [30.2, 30.6] | +17.4% | +16.2% | 3.2 / 8.7 | 5.6 / 15.9 | 100% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 2 | 1 x 5 | 16.5 [3.6, 44.5] / 62.4 [17.1, 81.5] | 28.1 [3.5, 93.2] / 107.9 [28.1, 127.0] | +22.4% | +19.0% | 3.8 / 10.7 | 5.4 / 14.8 | 100% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 4 | 1 x 5 | 4.8 [3.8, 6.0] / 11.2 [8.2, 17.8] | 8.0 [6.2, 9.4] / 14.8 [12.1, 18.7] | +5.8% | +4.8% | 3.8 / 13.4 | 5.2 / 20.6 | 100% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 8 | 1 x 5 | 4.4 [3.3, 5.7] / 9.8 [8.0, 10.8] | 12.6 [9.9, 14.6] / 19.8 [17.3, 21.4] | -6.0% | -8.7% | 3.4 / 9.7 | 6.0 / 16.1 | 100% | -| MoGe-2 placed with modern_assumed poses | 2 | 1 x 5 | 15.8 [15.8, 15.8] / 24.2 [24.0, 24.3] | 26.1 [26.1, 26.2] / 32.3 [32.2, 32.5] | +20.7% | +17.7% | 3.3 / 8.6 | 4.8 / 12.0 | 100% | -| MoGe-2 placed with modern_assumed poses | 4 | 1 x 5 | 15.8 [15.7, 15.8] / 23.3 [23.2, 23.4] | 25.3 [25.3, 25.4] / 31.1 [31.0, 31.1] | +20.4% | +17.2% | 3.4 / 8.5 | 4.9 / 12.9 | 100% | -| MoGe-2 placed with modern_assumed poses | 8 | 1 x 5 | 15.1 [15.1, 15.2] / 22.3 [22.2, 22.3] | 25.1 [25.1, 25.1] / 30.8 [30.8, 30.9] | +20.0% | +17.1% | 4.8 / 12.7 | 6.4 / 25.9 | 100% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 2 | 1 x 5 | 7.1 [5.4, 8.9] / 14.1 [11.3, 16.2] | 13.4 [9.4, 15.7] / 20.1 [17.5, 22.4] | +6.0% | +4.3% | 3.3 / 8.6 | 4.7 / 12.0 | 100% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 4 | 1 x 5 | 7.3 [6.3, 8.7] / 14.7 [11.6, 18.2] | 9.4 [7.9, 11.4] / 16.1 [12.2, 19.3] | +9.6% | +6.4% | 3.9 / 11.6 | 5.4 / 15.6 | 100% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 8 | 1 x 5 | 6.6 [6.1, 7.1] / 11.8 [10.8, 13.3] | 9.6 [8.8, 10.4] / 13.4 [12.4, 14.1] | +8.6% | +6.6% | 3.4 / 9.2 | 5.0 / 15.0 | 100% | +| mapanything@392, images only | 2 | 1 x 1 | 12.0 [12.0, 12.0] / 19.0 [19.0, 19.0] | 22.9 [22.9, 22.9] / 32.9 [32.9, 32.9] | +15.5% | +15.5% | 4.0 / fails (37% failed) | 7.0 / fails (38% failed) | 68% | +| mapanything@392, images only | 4 | 1 x 1 | 10.2 [10.2, 10.2] / 16.4 [16.4, 16.4] | 21.3 [21.3, 21.3] / 28.2 [28.2, 28.2] | +13.8% | +14.5% | 3.7 / fails (23% failed) | 5.4 / fails (24% failed) | 80% | +| mapanything@392 + exact poses | 2 | 1 x 1 | 9.7 [9.7, 9.7] / 16.1 [16.1, 16.1] | 18.3 [18.3, 18.3] / 25.1 [25.1, 25.1] | +12.6% | +12.3% | 4.5 / fails (37% failed) | 7.3 / fails (38% failed) | 68% | +| mapanything@392 + exact poses | 4 | 1 x 1 | 4.6 [4.6, 4.6] / 7.8 [7.8, 7.8] | 8.8 [8.8, 8.8] / 12.2 [12.2, 12.2] | +6.0% | +5.9% | 3.0 / fails (27% failed) | 4.7 / fails (28% failed) | 76% | +| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 2 | 1 x 1 | 9.6 [9.6, 9.6] / 15.6 [15.6, 15.6] | 17.8 [17.8, 17.8] / 24.3 [24.3, 24.3] | +12.2% | +11.9% | 4.5 / fails (37% failed) | 7.3 / fails (38% failed) | 68% | +| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 4 | 1 x 1 | 5.2 [5.2, 5.2] / 9.8 [9.8, 9.8] | 11.6 [11.6, 11.6] / 16.0 [16.0, 16.0] | +7.0% | +7.7% | 3.9 / fails (26% failed) | 6.1 / fails (27% failed) | 76% | +| mapanything@392 + modern_assumed poses | 2 | 1 x 1 | 9.3 [9.3, 9.3] / 15.6 [15.6, 15.6] | 17.5 [17.5, 17.5] / 23.9 [23.9, 23.9] | +12.1% | +11.8% | 4.6 / fails (37% failed) | 7.3 / fails (38% failed) | 68% | +| mapanything@392 + modern_assumed poses | 4 | 1 x 1 | 4.1 [4.1, 4.1] / 7.2 [7.2, 7.2] | 7.7 [7.7, 7.7] / 11.1 [11.1, 11.1] | +5.3% | +5.1% | 3.0 / fails (27% failed) | 4.8 / fails (28% failed) | 76% | +| MoGe-2 placed with exact poses | 2 | 1 x 1 | 15.7 [15.7, 15.7] / 24.0 [24.0, 24.0] | 25.8 [25.8, 25.8] / 32.2 [32.2, 32.2] | +20.4% | +17.4% | 3.2 / 8.4 | 4.8 / 12.2 | 100% | +| MoGe-2 placed with exact poses | 4 | 1 x 1 | 15.5 [15.5, 15.5] / 23.2 [23.2, 23.2] | 25.2 [25.2, 25.2] / 31.0 [31.0, 31.0] | +20.2% | +17.0% | 3.3 / 8.6 | 5.0 / 13.3 | 100% | +| MoGe-2 placed with exact poses | 8 | 1 x 1 | 15.2 [15.2, 15.2] / 22.5 [22.5, 22.5] | 25.1 [25.1, 25.1] / 30.5 [30.5, 30.5] | +20.2% | +17.1% | 4.9 / 13.5 | 6.6 / 28.8 | 100% | +| MoGe-2 rescaled by triangulation, exact poses | 2 | 1 x 1 | 3.1 [3.1, 3.1] / 8.8 [8.8, 8.8] | 2.8 [2.8, 2.8] / 7.5 [7.5, 7.5] | +4.2% | +1.6% | 3.3 / 8.5 | 4.8 / 12.2 | 100% | +| MoGe-2 rescaled by triangulation, exact poses | 4 | 1 x 1 | 8.1 [8.1, 8.1] / 14.8 [14.8, 14.8] | 10.8 [10.8, 10.8] / 15.2 [15.2, 15.2] | +10.4% | +7.2% | 3.8 / 10.4 | 5.0 / 14.9 | 100% | +| MoGe-2 rescaled by triangulation, exact poses | 8 | 1 x 1 | 7.5 [7.5, 7.5] / 12.8 [12.8, 12.8] | 11.0 [11.0, 11.0] / 14.5 [14.5, 14.5] | +9.7% | +7.5% | 3.5 / 9.0 | 5.2 / 16.1 | 100% | +| MoGe-2 placed with advio_2018 poses | 2 | 1 x 5 | 15.7 [15.5, 15.9] / 24.1 [23.7, 24.9] | 25.7 [25.4, 26.0] / 32.5 [31.9, 33.4] | +20.3% | +17.3% | 3.4 / 8.9 | 5.0 / 12.9 | 100% | +| MoGe-2 placed with advio_2018 poses | 4 | 1 x 5 | 15.4 [15.3, 15.7] / 23.0 [22.5, 23.5] | 24.7 [24.4, 25.0] / 30.9 [30.4, 31.2] | +20.1% | +16.6% | 3.3 / 8.5 | 4.7 / 13.8 | 100% | +| MoGe-2 placed with advio_2018 poses | 8 | 1 x 5 | 13.5 [13.2, 13.8] / 20.9 [20.5, 21.2] | 23.8 [23.6, 24.0] / 29.8 [29.5, 30.1] | +17.5% | +16.1% | 3.2 / 8.8 | 5.6 / 16.0 | 100% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 2 | 1 x 5 | 16.3 [3.4, 44.0] / 61.3 [16.9, 81.1] | 27.8 [3.3, 91.5] / 107.4 [27.8, 125.7] | +21.9% | +18.7% | 3.6 / 10.4 | 5.3 / 14.8 | 100% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 4 | 1 x 5 | 4.8 [3.8, 6.0] / 11.2 [8.3, 16.8] | 7.9 [6.1, 9.3] / 14.4 [12.1, 18.1] | +5.9% | +4.8% | 3.7 / 13.2 | 5.1 / 20.7 | 100% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 8 | 1 x 5 | 4.5 [3.4, 5.8] / 9.7 [8.0, 10.9] | 12.8 [10.2, 14.9] / 19.9 [17.3, 21.5] | -6.2% | -8.9% | 3.4 / 9.7 | 6.0 / 16.1 | 100% | +| MoGe-2 placed with modern_assumed poses | 2 | 1 x 5 | 15.7 [15.6, 15.7] / 23.9 [23.8, 24.0] | 25.8 [25.7, 25.8] / 32.2 [32.1, 32.3] | +20.3% | +17.4% | 3.2 / 8.4 | 4.8 / 12.1 | 100% | +| MoGe-2 placed with modern_assumed poses | 4 | 1 x 5 | 15.5 [15.5, 15.6] / 23.2 [23.0, 23.2] | 25.2 [25.1, 25.2] / 30.9 [30.8, 31.0] | +20.2% | +17.0% | 3.3 / 8.4 | 4.9 / 13.0 | 100% | +| MoGe-2 placed with modern_assumed poses | 8 | 1 x 5 | 15.0 [14.9, 15.0] / 22.2 [22.1, 22.2] | 25.0 [25.0, 25.0] / 30.3 [30.2, 30.3] | +19.9% | +16.9% | 4.7 / 12.6 | 6.4 / 26.2 | 100% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 2 | 1 x 5 | 7.1 [5.4, 8.7] / 13.8 [11.1, 15.8] | 13.4 [9.1, 15.9] / 20.1 [17.2, 22.2] | +5.7% | +4.0% | 3.2 / 8.4 | 4.7 / 12.1 | 100% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 4 | 1 x 5 | 7.1 [6.2, 8.6] / 14.3 [11.7, 17.5] | 9.2 [7.7, 11.1] / 15.3 [12.0, 18.7] | +9.4% | +6.2% | 3.8 / 11.3 | 5.4 / 15.5 | 100% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 8 | 1 x 5 | 6.5 [6.1, 7.0] / 11.8 [10.9, 13.0] | 9.4 [8.6, 10.1] / 13.2 [12.3, 13.9] | +8.5% | +6.4% | 3.3 / 9.1 | 5.0 / 15.1 | 100% | ## facade, within 6 m, edges @@ -196,184 +196,184 @@ Model-scale cells: median [95% interval] / p90 [95% interval], inches. The point | Method | Views | Groups x noise draws | Model scale: 1-3 m | 3-10 m | Scale error: 1-3 m | 3-10 m | One taped distance: 1-3 m | 3-10 m | Tape calibrated | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| mapanything@392, images only | 2 | 8 x 1 | 14.5 [10.6, 20.6] / fails [inf, inf] | 37.0 [28.5, 48.4] / fails [inf, inf] | +12.9% | +12.9% | 7.8 / fails (32% failed) | 22.5 / fails (36% failed) | 80% | -| mapanything@392, images only | 4 | 8 x 1 | 16.0 [11.8, 19.4] / fails [34.3, inf] | 40.1 [29.8, 47.8] / fails [206.0, inf] | +16.1% | +15.7% | 6.4 / fails (22% failed) | 16.9 / fails (26% failed) | 88% | -| mapanything@392 + exact poses | 2 | 8 x 1 | 5.7 [3.6, 11.4] / fails [inf, inf] | 17.3 [10.8, 29.9] / fails [inf, inf] | +3.1% | +3.1% | 7.5 / fails (32% failed) | 20.4 / fails (36% failed) | 80% | -| mapanything@392 + exact poses | 4 | 8 x 1 | 5.5 [3.8, 8.2] / fails [14.4, inf] | 14.0 [9.8, 22.4] / fails [inf, inf] | +3.6% | +2.7% | 6.2 / fails (24% failed) | 16.1 / fails (28% failed) | 86% | -| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 2 | 8 x 1 | 5.9 [3.7, 11.9] / fails [inf, inf] | 17.9 [8.7, 35.0] / fails [inf, inf] | -0.7% | -0.8% | 7.7 / fails (33% failed) | 21.4 / fails (36% failed) | 80% | -| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 4 | 8 x 1 | 6.5 [3.2, 10.8] / fails [17.2, inf] | 18.9 [9.4, 29.8] / fails [inf, inf] | -3.0% | -4.4% | 6.0 / fails (23% failed) | 15.5 / fails (27% failed) | 87% | -| mapanything@392 + modern_assumed poses | 2 | 8 x 1 | 5.0 [3.0, 11.7] / fails [inf, inf] | 16.5 [7.6, 31.1] / fails [inf, inf] | +1.3% | +1.1% | 7.3 / fails (32% failed) | 20.1 / fails (35% failed) | 81% | -| mapanything@392 + modern_assumed poses | 4 | 8 x 1 | 5.1 [3.0, 8.1] / fails [14.8, inf] | 13.3 [7.8, 22.9] / fails [inf, inf] | +2.0% | +1.3% | 6.2 / fails (24% failed) | 16.0 / fails (28% failed) | 86% | -| MoGe-2 placed with exact poses | 2 | 8 x 1 | 6.3 [4.0, 9.3] / 16.6 [11.3, 21.9] | 17.5 [12.9, 24.0] / 42.2 [27.1, 74.8] | +2.2% | +3.4% | 3.2 / 14.0 | 7.0 / 37.4 | 100% | -| MoGe-2 placed with exact poses | 4 | 8 x 1 | 5.9 [4.3, 8.9] / 16.8 [10.5, 23.9] | 16.5 [11.9, 23.1] / 49.8 [25.5, 82.0] | +2.5% | +3.7% | 3.2 / 15.2 | 6.9 / 39.4 | 99% | -| MoGe-2 placed with exact poses | 8 | 7 x 1 | 5.0 [3.5, 6.8] / 12.7 [10.8, 14.4] | 13.6 [10.0, 16.8] / 25.7 [22.1, 31.4] | +1.1% | +1.9% | 3.5 / 13.2 | 7.1 / 29.8 | 99% | -| MoGe-2 rescaled by triangulation, exact poses | 2 | 8 x 1 | 2.1 [1.3, 5.8] / fails [5.1, inf] | 5.2 [3.2, 17.8] / fails [10.7, inf] | +0.0% | +0.6% | 3.5 / fails (13% failed) | 8.7 / fails (13% failed) | 88% | -| MoGe-2 rescaled by triangulation, exact poses | 4 | 8 x 1 | 1.9 [1.3, 3.3] / 9.1 [5.2, 12.8] | 5.0 [3.4, 8.3] / 23.7 [9.7, 42.1] | -0.6% | -0.2% | 3.0 / 13.2 | 7.2 / 32.5 | 100% | -| MoGe-2 rescaled by triangulation, exact poses | 8 | 7 x 1 | 1.6 [1.2, 2.1] / 6.7 [4.8, 8.6] | 3.4 [2.5, 4.6] / 12.7 [7.2, 21.6] | -0.7% | -0.3% | 2.6 / 10.9 | 5.6 / 24.9 | 99% | -| MoGe-2 placed with advio_2018 poses | 2 | 8 x 5 | 6.3 [4.1, 9.3] / 16.8 [11.4, 22.0] | 17.2 [12.5, 23.6] / 41.8 [26.3, 75.2] | +1.9% | +2.8% | 3.4 / 14.8 | 7.3 / 39.2 | 100% | -| MoGe-2 placed with advio_2018 poses | 4 | 8 x 5 | 6.0 [4.2, 9.2] / 17.4 [11.1, 23.9] | 16.2 [11.2, 23.9] / 49.6 [25.7, 81.5] | +1.8% | +3.1% | 3.8 / 16.2 | 8.0 / 40.7 | 99% | -| MoGe-2 placed with advio_2018 poses | 8 | 7 x 5 | 5.2 [4.0, 7.2] / 13.9 [12.0, 15.2] | 13.8 [11.0, 17.5] / 26.1 [22.6, 31.0] | +0.1% | +0.6% | 4.3 / 15.2 | 8.5 / 33.3 | 99% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 2 | 8 x 5 | 10.8 [7.8, 17.5] / fails [17.3, inf] | 29.4 [19.7, 51.0] / fails [45.2, inf] | -9.7% | -9.8% | 4.1 / fails (13% failed) | 10.3 / fails (13% failed) | 88% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 4 | 8 x 5 | 9.7 [6.3, 14.1] / 27.7 [16.5, 64.4] | 26.2 [15.8, 37.1] / 65.6 [45.8, 124.6] | -9.6% | -10.2% | 4.8 / 21.6 | 10.4 / 47.8 | 100% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 8 | 7 x 5 | 6.3 [4.5, 9.7] / 17.6 [11.5, 21.4] | 14.1 [9.8, 24.5] / 37.1 [24.1, 48.8] | -7.1% | -6.5% | 4.1 / 15.6 | 8.6 / 33.9 | 99% | -| MoGe-2 placed with modern_assumed poses | 2 | 8 x 5 | 6.2 [3.9, 9.3] / 16.6 [11.3, 21.9] | 17.4 [12.8, 24.2] / 42.1 [26.4, 75.1] | +2.1% | +3.3% | 3.2 / 14.0 | 7.0 / 37.4 | 100% | -| MoGe-2 placed with modern_assumed poses | 4 | 8 x 5 | 5.9 [4.2, 9.0] / 16.8 [10.6, 23.9] | 16.4 [11.6, 23.4] / 49.7 [25.4, 81.9] | +2.4% | +3.5% | 3.2 / 15.2 | 7.0 / 39.5 | 99% | -| MoGe-2 placed with modern_assumed poses | 8 | 7 x 5 | 5.1 [3.6, 7.0] / 12.8 [11.0, 14.5] | 13.5 [10.2, 17.0] / 25.7 [22.4, 31.1] | +0.8% | +1.5% | 3.6 / 13.4 | 7.2 / 30.0 | 99% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 2 | 8 x 5 | 3.8 [2.4, 7.7] / fails [7.7, inf] | 10.3 [5.9, 24.7] / fails [18.8, inf] | -2.0% | -1.8% | 3.5 / fails (13% failed) | 8.9 / fails (13% failed) | 88% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 4 | 8 x 5 | 3.1 [2.2, 4.9] / 12.5 [6.9, inf] | 6.6 [4.1, 11.8] / 32.3 [14.0, inf] | -2.7% | -2.4% | 3.3 / 16.1 | 7.6 / 40.4 | 98% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 8 | 7 x 5 | 2.7 [2.3, 3.3] / 7.6 [5.9, 9.2] | 5.9 [4.6, 7.7] / 14.9 [10.6, 20.1] | -2.9% | -2.6% | 2.6 / 9.9 | 5.7 / 23.0 | 99% | +| mapanything@392, images only | 2 | 7 x 1 | 13.4 [9.7, 18.1] / fails [37.5, inf] | 33.9 [26.5, 40.8] / fails [inf, inf] | +12.8% | +13.0% | 6.2 / fails (24% failed) | 17.2 / fails (28% failed) | 87% | +| mapanything@392, images only | 4 | 7 x 1 | 15.8 [11.8, 19.4] / 41.9 [30.6, inf] | 38.9 [28.7, 46.7] / fails [81.4, inf] | +17.0% | +17.0% | 5.2 / fails (17% failed) | 13.7 / fails (20% failed) | 92% | +| mapanything@392 + exact poses | 2 | 7 x 1 | 4.9 [3.3, 9.8] / fails [23.0, inf] | 15.1 [10.2, 24.2] / fails [inf, inf] | +3.0% | +3.2% | 5.9 / fails (24% failed) | 15.7 / fails (28% failed) | 87% | +| mapanything@392 + exact poses | 4 | 7 x 1 | 4.7 [3.5, 6.2] / 23.5 [13.1, inf] | 12.1 [8.7, 15.0] / fails [33.4, inf] | +4.1% | +3.2% | 5.0 / fails (18% failed) | 12.9 / fails (22% failed) | 90% | +| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 2 | 7 x 1 | 5.2 [3.4, 9.8] / fails [29.2, inf] | 14.2 [7.7, 28.2] / fails [inf, inf] | -0.7% | -0.6% | 6.1 / fails (25% failed) | 16.5 / fails (29% failed) | 86% | +| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 4 | 7 x 1 | 5.5 [2.5, 9.3] / 27.4 [14.7, inf] | 16.1 [7.1, 22.5] / fails [45.8, inf] | -2.3% | -3.6% | 4.9 / fails (18% failed) | 12.4 / fails (21% failed) | 91% | +| mapanything@392 + modern_assumed poses | 2 | 7 x 1 | 4.2 [2.9, 9.6] / fails [22.5, inf] | 13.1 [7.0, 24.6] / fails [inf, inf] | +1.2% | +1.1% | 5.8 / fails (24% failed) | 15.5 / fails (27% failed) | 87% | +| mapanything@392 + modern_assumed poses | 4 | 7 x 1 | 4.3 [2.7, 6.1] / 21.9 [13.2, inf] | 11.3 [6.8, 14.7] / fails [31.1, inf] | +2.5% | +1.8% | 5.0 / fails (18% failed) | 13.0 / fails (22% failed) | 90% | +| MoGe-2 placed with exact poses | 2 | 7 x 1 | 5.6 [3.6, 8.3] / 14.0 [10.6, 16.8] | 16.0 [12.1, 21.3] / 30.9 [25.4, 33.5] | +3.0% | +4.4% | 2.8 / 10.1 | 6.0 / 22.8 | 100% | +| MoGe-2 placed with exact poses | 4 | 7 x 1 | 5.3 [3.9, 7.5] / 13.4 [9.9, 16.1] | 14.8 [11.0, 19.4] / 29.0 [23.5, 33.9] | +3.8% | +4.6% | 2.8 / 10.7 | 5.9 / 25.0 | 99% | +| MoGe-2 placed with exact poses | 8 | 7 x 1 | 5.0 [3.5, 6.8] / 12.7 [10.8, 14.4] | 13.6 [10.0, 16.8] / 25.7 [22.1, 31.4] | +1.1% | +1.9% | 3.5 / 13.1 | 7.0 / 29.7 | 99% | +| MoGe-2 rescaled by triangulation, exact poses | 2 | 7 x 1 | 1.7 [1.2, 5.0] / fails [4.7, inf] | 4.3 [3.0, 10.8] / fails [9.0, inf] | +0.1% | +0.8% | 3.0 / fails (14% failed) | 7.3 / fails (14% failed) | 86% | +| MoGe-2 rescaled by triangulation, exact poses | 4 | 7 x 1 | 1.7 [1.2, 2.2] / 6.9 [4.7, 9.0] | 4.3 [3.1, 5.7] / 12.3 [8.0, 17.5] | -0.3% | +0.1% | 2.6 / 10.1 | 6.3 / 24.1 | 100% | +| MoGe-2 rescaled by triangulation, exact poses | 8 | 7 x 1 | 1.6 [1.2, 2.1] / 6.7 [4.8, 8.6] | 3.4 [2.5, 4.6] / 12.7 [7.2, 21.6] | -0.7% | -0.3% | 2.6 / 10.8 | 5.5 / 24.9 | 99% | +| MoGe-2 placed with advio_2018 poses | 2 | 7 x 5 | 5.7 [3.8, 8.4] / 14.2 [10.8, 16.9] | 15.8 [11.9, 21.6] / 30.7 [25.1, 33.2] | +2.8% | +4.0% | 3.0 / 10.9 | 6.3 / 24.6 | 100% | +| MoGe-2 placed with advio_2018 poses | 4 | 7 x 5 | 5.3 [3.9, 8.0] / 14.0 [10.6, 16.6] | 14.3 [10.5, 19.9] / 29.1 [24.2, 34.1] | +3.1% | +4.3% | 3.4 / 12.3 | 6.9 / 27.8 | 99% | +| MoGe-2 placed with advio_2018 poses | 8 | 7 x 5 | 5.2 [4.0, 7.2] / 13.9 [12.0, 15.2] | 13.8 [11.0, 17.5] / 26.1 [22.6, 31.0] | +0.1% | +0.6% | 4.3 / 15.1 | 8.4 / 33.2 | 99% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 2 | 7 x 5 | 10.5 [7.2, 21.2] / fails [15.9, inf] | 28.3 [17.6, 63.6] / fails [40.6, inf] | -10.0% | -9.7% | 3.6 / fails (14% failed) | 8.8 / fails (14% failed) | 86% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 4 | 7 x 5 | 9.6 [5.9, 15.3] / 27.4 [15.9, 68.8] | 24.6 [14.3, 37.1] / 62.6 [42.8, 134.0] | -9.3% | -9.4% | 4.3 / 17.9 | 9.2 / 37.4 | 100% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 8 | 7 x 5 | 6.3 [4.5, 9.7] / 17.6 [11.5, 21.4] | 14.1 [9.8, 24.5] / 37.1 [24.1, 48.8] | -7.1% | -6.5% | 4.1 / 15.6 | 8.5 / 33.8 | 99% | +| MoGe-2 placed with modern_assumed poses | 2 | 7 x 5 | 5.6 [3.6, 8.4] / 14.0 [10.6, 16.9] | 16.0 [12.2, 21.9] / 30.8 [25.4, 33.4] | +3.0% | +4.4% | 2.8 / 10.1 | 6.0 / 22.8 | 100% | +| MoGe-2 placed with modern_assumed poses | 4 | 7 x 5 | 5.2 [3.9, 7.9] / 13.5 [10.1, 16.3] | 14.7 [11.0, 19.8] / 28.9 [23.7, 33.6] | +3.6% | +4.5% | 2.8 / 10.8 | 6.0 / 25.1 | 99% | +| MoGe-2 placed with modern_assumed poses | 8 | 7 x 5 | 5.1 [3.6, 7.0] / 12.8 [11.0, 14.5] | 13.5 [10.2, 17.0] / 25.7 [22.4, 31.1] | +0.8% | +1.5% | 3.6 / 13.3 | 7.2 / 29.9 | 99% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 2 | 7 x 5 | 3.5 [2.2, 8.3] / fails [6.9, inf] | 9.0 [5.4, 24.4] / fails [15.7, inf] | -2.0% | -1.4% | 3.1 / fails (14% failed) | 7.5 / fails (14% failed) | 86% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 4 | 7 x 5 | 2.8 [2.0, 4.0] / 10.0 [6.2, inf] | 5.7 [3.7, 9.0] / 19.9 [11.6, inf] | -2.5% | -2.1% | 2.9 / 11.9 | 6.6 / 28.8 | 97% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 8 | 7 x 5 | 2.7 [2.3, 3.3] / 7.6 [5.9, 9.2] | 5.9 [4.6, 7.7] / 14.9 [10.6, 20.1] | -2.9% | -2.6% | 2.6 / 9.8 | 5.7 / 22.9 | 99% | ## electro, all points, vertical interior | Method | Views | Groups x noise draws | Model scale: 1-3 m | 3-10 m | Scale error: 1-3 m | 3-10 m | One taped distance: 1-3 m | 3-10 m | Tape calibrated | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| mapanything@392, images only | 2 | 8 x 1 | 13.9 [8.6, 19.3] / fails [28.8, inf] | 32.6 [23.5, 41.1] / fails [inf, inf] | +12.6% | +13.2% | 6.9 / fails (29% failed) | 19.6 / fails (32% failed) | 80% | -| mapanything@392, images only | 4 | 8 x 1 | 15.0 [10.0, 19.6] / 39.8 [26.0, inf] | 33.2 [22.9, 40.3] / fails [62.5, inf] | +15.4% | +14.4% | 5.3 / fails (18% failed) | 15.2 / fails (20% failed) | 88% | -| mapanything@392 + exact poses | 2 | 8 x 1 | 4.2 [2.8, 8.5] / fails [12.6, inf] | 13.0 [7.8, 22.8] / fails [inf, inf] | +3.2% | +3.3% | 6.4 / fails (29% failed) | 17.1 / fails (32% failed) | 80% | -| mapanything@392 + exact poses | 4 | 8 x 1 | 4.5 [3.1, 7.3] / 23.6 [9.4, inf] | 12.4 [7.6, 18.6] / fails [23.2, inf] | +3.8% | +3.3% | 4.8 / fails (20% failed) | 13.3 / fails (22% failed) | 86% | -| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 2 | 8 x 1 | 4.5 [3.1, 8.0] / fails [13.6, inf] | 13.7 [6.9, 28.3] / fails [inf, inf] | -1.1% | -1.0% | 6.5 / fails (29% failed) | 17.9 / fails (33% failed) | 80% | -| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 4 | 8 x 1 | 5.1 [1.8, 9.3] / 24.9 [10.2, inf] | 14.5 [6.7, 25.9] / fails [26.4, inf] | -2.5% | -4.2% | 4.8 / fails (19% failed) | 12.9 / fails (21% failed) | 87% | -| mapanything@392 + modern_assumed poses | 2 | 8 x 1 | 3.7 [2.3, 8.3] / fails [11.7, inf] | 11.0 [5.6, 22.4] / fails [inf, inf] | +1.5% | +1.6% | 6.2 / fails (28% failed) | 16.8 / fails (32% failed) | 81% | -| mapanything@392 + modern_assumed poses | 4 | 8 x 1 | 3.8 [2.1, 7.2] / 22.7 [8.9, inf] | 11.3 [5.5, 18.6] / fails [21.6, inf] | +2.2% | +1.9% | 4.8 / fails (20% failed) | 13.4 / fails (22% failed) | 86% | -| MoGe-2 placed with exact poses | 2 | 8 x 1 | 6.1 [3.5, 9.0] / 16.2 [10.7, 21.2] | 17.6 [14.2, 25.0] / 45.5 [25.4, 76.1] | +1.7% | +4.7% | 2.8 / 13.4 | 7.0 / 38.5 | 100% | -| MoGe-2 placed with exact poses | 4 | 8 x 1 | 5.6 [3.5, 8.9] / 16.2 [9.5, 23.0] | 16.5 [11.7, 24.1] / 52.4 [23.3, 85.0] | +1.3% | +4.2% | 2.8 / 14.7 | 6.9 / 41.1 | 99% | -| MoGe-2 placed with exact poses | 8 | 7 x 1 | 4.8 [2.9, 6.8] / 12.2 [9.6, 14.3] | 13.3 [10.3, 16.7] / 24.3 [19.6, 28.6] | -0.5% | +2.9% | 3.3 / 13.4 | 7.8 / 30.1 | 99% | -| MoGe-2 rescaled by triangulation, exact poses | 2 | 8 x 1 | 1.6 [1.0, 4.9] / fails [3.9, inf] | 5.9 [3.2, 23.9] / fails [12.0, inf] | -0.0% | +1.4% | 3.1 / fails (13% failed) | 9.2 / fails (13% failed) | 88% | -| MoGe-2 rescaled by triangulation, exact poses | 4 | 8 x 1 | 1.7 [1.2, 2.6] / 8.7 [4.8, 11.8] | 6.2 [3.5, 10.3] / 28.1 [9.8, 44.9] | -0.6% | +0.1% | 2.7 / 12.5 | 7.6 / 32.4 | 100% | -| MoGe-2 rescaled by triangulation, exact poses | 8 | 7 x 1 | 1.2 [0.9, 1.9] / 6.3 [3.6, 10.6] | 3.5 [2.4, 5.2] / 14.8 [6.7, 25.0] | -0.6% | -0.2% | 2.2 / 10.6 | 5.8 / 25.6 | 99% | -| MoGe-2 placed with advio_2018 poses | 2 | 8 x 5 | 6.1 [3.5, 9.0] / 16.5 [10.7, 21.3] | 17.7 [14.2, 25.8] / 45.7 [25.5, 76.7] | +1.9% | +4.8% | 3.0 / 13.8 | 7.4 / 41.2 | 100% | -| MoGe-2 placed with advio_2018 poses | 4 | 8 x 5 | 5.6 [3.4, 9.1] / 16.4 [10.2, 23.0] | 16.3 [11.7, 25.0] / 52.4 [24.4, 85.0] | +1.4% | +4.2% | 3.4 / 15.5 | 8.1 / 41.7 | 99% | -| MoGe-2 placed with advio_2018 poses | 8 | 7 x 5 | 5.0 [3.5, 7.2] / 13.2 [10.3, 15.1] | 13.8 [10.6, 17.8] / 25.2 [21.0, 28.8] | -0.8% | +2.8% | 3.8 / 15.2 | 8.4 / 34.5 | 99% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 2 | 8 x 5 | 10.4 [7.4, 16.6] / fails [16.1, inf] | 24.5 [15.1, 49.5] / fails [40.5, inf] | -10.0% | -7.8% | 3.6 / fails (13% failed) | 10.1 / fails (13% failed) | 88% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 4 | 8 x 5 | 9.2 [5.5, 13.9] / 27.0 [16.3, 55.9] | 26.5 [14.5, 35.2] / 64.0 [42.7, 114.8] | -9.2% | -10.0% | 4.0 / 20.0 | 10.8 / 49.8 | 100% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 8 | 7 x 5 | 6.0 [3.8, 9.8] / 17.4 [10.3, 21.4] | 13.2 [9.1, 22.7] / 35.0 [20.8, 44.9] | -7.2% | -6.5% | 3.6 / 14.6 | 7.9 / 33.8 | 99% | -| MoGe-2 placed with modern_assumed poses | 2 | 8 x 5 | 6.1 [3.4, 9.0] / 16.3 [10.6, 21.3] | 17.5 [14.1, 25.8] / 45.5 [25.4, 76.7] | +1.8% | +4.7% | 2.8 / 13.4 | 7.0 / 38.7 | 100% | -| MoGe-2 placed with modern_assumed poses | 4 | 8 x 5 | 5.6 [3.4, 8.9] / 16.2 [9.6, 23.1] | 16.5 [11.7, 24.8] / 52.4 [23.9, 85.0] | +1.3% | +4.2% | 2.9 / 14.7 | 7.1 / 41.2 | 99% | -| MoGe-2 placed with modern_assumed poses | 8 | 7 x 5 | 4.9 [3.1, 7.1] / 12.3 [9.6, 14.4] | 13.4 [10.4, 17.2] / 24.4 [19.8, 28.7] | -0.7% | +2.9% | 3.4 / 13.6 | 7.7 / 30.5 | 99% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 2 | 8 x 5 | 3.0 [1.9, 7.0] / fails [6.5, inf] | 10.3 [4.6, 26.4] / fails [20.2, inf] | -2.0% | -0.5% | 3.1 / fails (13% failed) | 9.2 / fails (13% failed) | 88% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 4 | 8 x 5 | 2.8 [1.8, 4.4] / 12.4 [6.4, inf] | 6.9 [3.8, 13.1] / 35.2 [13.7, inf] | -2.7% | -2.2% | 3.0 / 15.4 | 7.8 / 40.4 | 98% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 8 | 7 x 5 | 2.4 [1.8, 3.3] / 7.4 [4.8, 9.7] | 5.6 [4.4, 7.2] / 14.9 [10.2, 21.4] | -2.9% | -2.7% | 2.3 / 9.4 | 5.6 / 22.6 | 99% | +| mapanything@392, images only | 2 | 7 x 1 | 13.1 [7.9, 17.7] / 36.8 [24.9, inf] | 30.6 [22.4, 37.2] / fails [79.3, inf] | +12.7% | +13.5% | 5.5 / fails (21% failed) | 14.9 / fails (25% failed) | 87% | +| mapanything@392, images only | 4 | 7 x 1 | 15.2 [10.6, 19.8] / 32.5 [23.7, 37.6] | 32.0 [21.6, 39.0] / 81.8 [59.4, 151.7] | +16.7% | +15.2% | 4.3 / fails (13% failed) | 12.1 / fails (15% failed) | 92% | +| mapanything@392 + exact poses | 2 | 7 x 1 | 3.8 [2.6, 7.2] / 25.3 [11.5, inf] | 11.6 [6.7, 19.2] / fails [56.6, inf] | +3.2% | +3.4% | 5.0 / fails (21% failed) | 13.3 / fails (25% failed) | 87% | +| mapanything@392 + exact poses | 4 | 7 x 1 | 3.9 [2.8, 5.8] / 12.0 [8.0, 15.7] | 11.1 [6.4, 13.5] / 32.1 [22.0, 50.3] | +4.2% | +3.6% | 3.9 / fails (15% failed) | 10.6 / fails (16% failed) | 90% | +| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 2 | 7 x 1 | 4.2 [2.8, 6.8] / 23.8 [12.0, inf] | 11.2 [6.2, 23.9] / fails [58.4, inf] | -1.1% | -0.7% | 5.1 / fails (21% failed) | 13.8 / fails (25% failed) | 86% | +| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 4 | 7 x 1 | 4.2 [1.4, 7.8] / 17.5 [9.5, 23.5] | 12.8 [5.4, 19.2] / 42.9 [23.1, 53.5] | -1.5% | -3.1% | 3.9 / fails (14% failed) | 10.1 / fails (16% failed) | 91% | +| mapanything@392 + modern_assumed poses | 2 | 7 x 1 | 3.2 [2.1, 7.0] / 25.8 [11.0, inf] | 9.0 [5.0, 18.6] / fails [58.8, inf] | +1.4% | +1.6% | 5.0 / fails (20% failed) | 12.9 / fails (24% failed) | 87% | +| mapanything@392 + modern_assumed poses | 4 | 7 x 1 | 3.1 [2.0, 5.5] / 11.7 [7.3, 15.6] | 9.7 [4.4, 13.3] / 30.9 [20.0, 49.5] | +2.6% | +2.2% | 3.9 / fails (15% failed) | 10.7 / fails (16% failed) | 90% | +| MoGe-2 placed with exact poses | 2 | 7 x 1 | 5.4 [2.9, 8.2] / 13.6 [9.4, 16.7] | 16.2 [13.5, 22.1] / 30.8 [24.1, 34.4] | +3.1% | +5.7% | 2.4 / 9.4 | 6.0 / 23.2 | 100% | +| MoGe-2 placed with exact poses | 4 | 7 x 1 | 4.9 [2.9, 7.5] / 12.4 [8.4, 15.3] | 14.9 [11.2, 20.1] / 29.8 [20.4, 35.1] | +3.3% | +5.3% | 2.4 / 10.2 | 5.9 / 28.8 | 99% | +| MoGe-2 placed with exact poses | 8 | 7 x 1 | 4.8 [2.9, 6.9] / 12.3 [9.7, 14.3] | 13.2 [10.5, 16.7] / 24.2 [19.5, 28.4] | -0.6% | +2.9% | 3.3 / 13.6 | 7.8 / 29.8 | 99% | +| MoGe-2 rescaled by triangulation, exact poses | 2 | 7 x 1 | 1.3 [0.9, 3.8] / fails [3.5, inf] | 4.8 [2.7, 17.2] / fails [8.6, inf] | +0.1% | +1.6% | 2.6 / fails (14% failed) | 7.6 / fails (14% failed) | 86% | +| MoGe-2 rescaled by triangulation, exact poses | 4 | 7 x 1 | 1.4 [1.1, 1.7] / 6.8 [3.6, 8.9] | 5.3 [2.9, 8.3] / 14.2 [8.4, 27.5] | -0.4% | +0.5% | 2.3 / 9.5 | 6.6 / 24.2 | 100% | +| MoGe-2 rescaled by triangulation, exact poses | 8 | 7 x 1 | 1.2 [0.9, 1.9] / 6.2 [3.6, 10.1] | 3.5 [2.3, 5.2] / 14.9 [6.7, 25.1] | -0.6% | -0.2% | 2.2 / 10.5 | 5.8 / 25.4 | 99% | +| MoGe-2 placed with advio_2018 poses | 2 | 7 x 5 | 5.4 [3.0, 8.3] / 14.0 [9.5, 16.9] | 16.3 [13.6, 22.5] / 30.8 [24.1, 35.2] | +3.2% | +5.7% | 2.5 / 10.0 | 6.3 / 25.7 | 100% | +| MoGe-2 placed with advio_2018 poses | 4 | 7 x 5 | 4.9 [3.0, 8.1] / 12.9 [8.9, 15.6] | 14.8 [11.1, 20.7] / 29.9 [21.0, 35.6] | +3.2% | +5.3% | 2.9 / 11.4 | 7.0 / 31.5 | 99% | +| MoGe-2 placed with advio_2018 poses | 8 | 7 x 5 | 4.9 [3.5, 7.2] / 13.4 [10.4, 15.1] | 13.8 [10.6, 18.0] / 25.2 [20.9, 28.7] | -0.8% | +2.8% | 3.8 / 15.4 | 8.3 / 34.3 | 99% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 2 | 7 x 5 | 10.1 [6.8, 21.3] / fails [14.6, inf] | 23.0 [13.9, 61.8] / fails [35.0, inf] | -10.2% | -7.6% | 3.1 / fails (14% failed) | 8.4 / fails (14% failed) | 86% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 4 | 7 x 5 | 9.0 [5.0, 14.8] / 27.1 [15.3, 60.3] | 24.7 [12.2, 34.5] / 59.6 [40.1, 126.1] | -8.9% | -8.9% | 3.6 / 16.0 | 9.4 / 39.3 | 100% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 8 | 7 x 5 | 6.0 [3.8, 9.8] / 17.4 [10.3, 21.4] | 13.2 [9.2, 22.8] / 35.1 [21.1, 45.1] | -7.2% | -6.6% | 3.6 / 14.6 | 7.9 / 33.9 | 99% | +| MoGe-2 placed with modern_assumed poses | 2 | 7 x 5 | 5.4 [3.0, 8.3] / 13.7 [9.4, 16.8] | 16.2 [13.6, 22.7] / 30.7 [24.2, 34.5] | +3.2% | +5.7% | 2.4 / 9.4 | 6.0 / 23.3 | 100% | +| MoGe-2 placed with modern_assumed poses | 4 | 7 x 5 | 4.9 [2.9, 8.0] / 12.5 [8.5, 15.5] | 14.9 [11.2, 20.4] / 29.8 [20.7, 35.4] | +3.3% | +5.3% | 2.5 / 10.3 | 6.0 / 29.0 | 99% | +| MoGe-2 placed with modern_assumed poses | 8 | 7 x 5 | 4.8 [3.1, 7.0] / 12.4 [9.8, 14.4] | 13.3 [10.6, 17.2] / 24.3 [19.7, 28.5] | -0.7% | +2.9% | 3.3 / 13.7 | 7.7 / 30.4 | 99% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 2 | 7 x 5 | 2.7 [1.8, 7.4] / fails [4.9, inf] | 8.8 [4.1, 25.9] / fails [15.3, inf] | -2.0% | -0.1% | 2.6 / fails (14% failed) | 7.7 / fails (14% failed) | 86% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 4 | 7 x 5 | 2.4 [1.6, 3.6] / 10.4 [5.2, inf] | 5.6 [3.4, 10.0] / 21.4 [11.7, inf] | -2.5% | -1.9% | 2.6 / 11.3 | 6.7 / 29.0 | 97% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 8 | 7 x 5 | 2.4 [1.8, 3.3] / 7.4 [4.8, 9.7] | 5.6 [4.3, 7.2] / 15.1 [10.3, 21.4] | -2.9% | -2.7% | 2.2 / 9.3 | 5.6 / 22.4 | 99% | ## electro, all points, edges | Method | Views | Groups x noise draws | Model scale: 1-3 m | 3-10 m | Scale error: 1-3 m | 3-10 m | One taped distance: 1-3 m | 3-10 m | Tape calibrated | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| mapanything@392, images only | 2 | 8 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +15.2% | +10.0% | inf / fails (83% failed) | inf / fails (87% failed) | 80% | -| mapanything@392, images only | 4 | 8 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +23.3% | +12.9% | inf / fails (71% failed) | inf / fails (79% failed) | 88% | -| mapanything@392 + exact poses | 2 | 8 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +5.4% | +5.6% | inf / fails (83% failed) | inf / fails (87% failed) | 80% | -| mapanything@392 + exact poses | 4 | 8 x 1 | fails [504.4, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +6.2% | +4.4% | inf / fails (71% failed) | inf / fails (78% failed) | 86% | -| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 2 | 8 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +3.9% | +4.1% | inf / fails (83% failed) | inf / fails (87% failed) | 80% | -| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 4 | 8 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +0.6% | -1.3% | inf / fails (72% failed) | inf / fails (78% failed) | 87% | -| mapanything@392 + modern_assumed poses | 2 | 8 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +3.1% | +3.3% | inf / fails (82% failed) | inf / fails (87% failed) | 81% | -| mapanything@392 + modern_assumed poses | 4 | 8 x 1 | fails [561.6, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +4.7% | +2.9% | inf / fails (72% failed) | inf / fails (78% failed) | 86% | -| MoGe-2 placed with exact poses | 2 | 8 x 1 | 7.6 [6.0, 10.8] / 30.6 [15.0, 48.0] | 15.3 [11.7, 21.4] / 44.7 [24.1, 70.6] | +5.6% | +4.3% | 4.3 / 32.6 | 7.0 / 46.4 | 100% | -| MoGe-2 placed with exact poses | 4 | 8 x 1 | 6.4 [5.3, 8.6] / 21.3 [13.8, 36.6] | 13.3 [10.4, 17.9] / 37.6 [21.9, 66.4] | +4.8% | +4.7% | 4.7 / 27.1 | 7.7 / 47.8 | 99% | -| MoGe-2 placed with exact poses | 8 | 7 x 1 | 6.5 [5.4, 7.5] / 15.6 [11.2, 18.8] | 11.1 [9.5, 12.5] / 24.1 [16.8, 28.1] | +3.8% | +3.8% | 4.8 / 16.3 | 7.3 / 28.0 | 99% | -| MoGe-2 rescaled by triangulation, exact poses | 2 | 8 x 1 | 3.3 [1.1, 24.4] / fails [11.0, inf] | 5.0 [1.7, 22.4] / fails [12.8, inf] | +0.3% | +0.1% | 5.2 / fails (13% failed) | 9.0 / fails (12% failed) | 88% | -| MoGe-2 rescaled by triangulation, exact poses | 4 | 8 x 1 | 3.0 [1.0, 6.7] / 23.0 [10.6, 56.5] | 6.2 [1.7, 9.9] / 31.5 [11.3, 62.0] | -0.1% | -0.3% | 4.5 / 25.9 | 7.5 / 40.6 | 100% | -| MoGe-2 rescaled by triangulation, exact poses | 8 | 7 x 1 | 2.3 [1.5, 3.2] / 11.3 [7.9, 15.5] | 3.4 [1.7, 5.1] / 13.7 [8.4, 18.9] | -0.9% | -0.6% | 3.5 / 15.1 | 5.5 / 24.1 | 99% | -| MoGe-2 placed with advio_2018 poses | 2 | 8 x 5 | 7.7 [6.2, 11.2] / 30.6 [15.5, 48.3] | 15.8 [11.7, 21.5] / 44.6 [24.1, 70.4] | +5.6% | +4.4% | 4.6 / 34.1 | 7.4 / 48.7 | 100% | -| MoGe-2 placed with advio_2018 poses | 4 | 8 x 5 | 6.6 [5.3, 8.9] / 21.4 [14.3, 37.3] | 14.0 [10.8, 19.9] / 38.6 [22.2, 66.4] | +5.0% | +5.1% | 5.2 / 27.2 | 8.9 / 47.1 | 99% | -| MoGe-2 placed with advio_2018 poses | 8 | 7 x 5 | 6.9 [5.7, 8.1] / 15.9 [12.2, 19.0] | 11.6 [10.2, 14.2] / 25.8 [17.2, 30.4] | +3.8% | +3.7% | 4.9 / 17.7 | 8.0 / 31.4 | 99% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 2 | 8 x 5 | 11.4 [8.0, 29.0] / fails [19.0, inf] | 24.1 [16.1, 53.4] / fails [39.4, inf] | -7.8% | -9.3% | 5.7 / fails (13% failed) | 9.9 / fails (12% failed) | 88% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 4 | 8 x 5 | 10.7 [7.4, 15.6] / 30.6 [20.0, 63.8] | 23.7 [12.7, 35.1] / 66.6 [41.5, 99.6] | -8.3% | -9.1% | 5.8 / 32.6 | 10.4 / 57.6 | 100% | +| mapanything@392, images only | 2 | 7 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | -2.4% | +2.6% | inf / fails (82% failed) | inf / fails (87% failed) | 87% | +| mapanything@392, images only | 4 | 7 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +18.4% | +8.7% | inf / fails (71% failed) | inf / fails (79% failed) | 92% | +| mapanything@392 + exact poses | 2 | 7 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +4.9% | +5.2% | inf / fails (81% failed) | inf / fails (87% failed) | 87% | +| mapanything@392 + exact poses | 4 | 7 x 1 | fails [18.1, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +5.3% | +4.0% | inf / fails (71% failed) | inf / fails (78% failed) | 90% | +| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 2 | 7 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +3.6% | +3.8% | inf / fails (81% failed) | inf / fails (87% failed) | 86% | +| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 4 | 7 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | -2.4% | -2.6% | inf / fails (71% failed) | inf / fails (78% failed) | 91% | +| mapanything@392 + modern_assumed poses | 2 | 7 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +2.5% | +2.8% | inf / fails (81% failed) | inf / fails (87% failed) | 87% | +| mapanything@392 + modern_assumed poses | 4 | 7 x 1 | fails [17.6, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +3.7% | +2.4% | inf / fails (71% failed) | inf / fails (78% failed) | 90% | +| MoGe-2 placed with exact poses | 2 | 7 x 1 | 7.0 [5.8, 8.8] / 19.0 [12.5, 25.4] | 14.0 [10.9, 19.6] / 27.7 [22.4, 32.6] | +4.6% | +4.8% | 3.6 / 16.4 | 5.9 / 24.4 | 100% | +| MoGe-2 placed with exact poses | 4 | 7 x 1 | 6.0 [5.1, 7.9] / 17.2 [11.6, 21.8] | 12.3 [9.8, 15.4] / 26.7 [18.8, 31.1] | +4.6% | +5.2% | 3.9 / 15.9 | 6.5 / 25.5 | 99% | +| MoGe-2 placed with exact poses | 8 | 7 x 1 | 6.5 [5.4, 7.5] / 15.6 [11.2, 18.8] | 11.1 [9.5, 12.5] / 24.1 [16.8, 28.1] | +3.8% | +3.8% | 4.7 / 16.3 | 7.3 / 28.0 | 99% | +| MoGe-2 rescaled by triangulation, exact poses | 2 | 7 x 1 | 2.2 [0.9, 12.1] / fails [8.6, inf] | 3.8 [1.5, 11.9] / fails [10.1, inf] | -0.1% | +0.0% | 4.1 / fails (15% failed) | 7.5 / fails (14% failed) | 86% | +| MoGe-2 rescaled by triangulation, exact poses | 4 | 7 x 1 | 2.2 [0.8, 4.0] / 12.9 [9.5, 17.0] | 5.2 [1.2, 7.7] / 16.0 [10.4, 21.1] | -0.4% | -0.3% | 3.7 / 15.1 | 6.4 / 23.1 | 100% | +| MoGe-2 rescaled by triangulation, exact poses | 8 | 7 x 1 | 2.3 [1.5, 3.2] / 11.3 [7.9, 15.4] | 3.4 [1.7, 5.1] / 13.7 [8.4, 18.9] | -0.9% | -0.6% | 3.5 / 15.0 | 5.4 / 24.1 | 99% | +| MoGe-2 placed with advio_2018 poses | 2 | 7 x 5 | 7.0 [5.9, 9.1] / 19.1 [13.9, 25.5] | 14.3 [11.1, 19.7] / 28.0 [22.7, 33.0] | +4.6% | +5.0% | 3.8 / 16.9 | 6.2 / 26.0 | 100% | +| MoGe-2 placed with advio_2018 poses | 4 | 7 x 5 | 6.2 [5.2, 8.2] / 17.4 [13.1, 22.0] | 12.7 [9.9, 16.4] / 27.5 [18.9, 32.3] | +4.7% | +5.5% | 4.4 / 16.5 | 7.6 / 27.9 | 99% | +| MoGe-2 placed with advio_2018 poses | 8 | 7 x 5 | 6.9 [5.7, 8.1] / 15.9 [12.2, 19.0] | 11.6 [10.2, 14.2] / 25.8 [17.2, 30.4] | +3.7% | +3.7% | 4.9 / 17.7 | 7.9 / 31.4 | 99% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 2 | 7 x 5 | 10.3 [7.4, 22.2] / fails [16.3, inf] | 21.7 [14.6, 57.2] / fails [32.0, inf] | -9.4% | -9.7% | 4.5 / fails (15% failed) | 8.1 / fails (14% failed) | 86% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 4 | 7 x 5 | 10.1 [6.7, 14.9] / 25.5 [17.6, 44.4] | 22.0 [11.3, 34.1] / 60.0 [36.5, 101.0] | -9.7% | -9.4% | 4.9 / 20.4 | 8.8 / 35.1 | 100% | | MoGe-2 rescaled by triangulation, advio_2018 poses | 8 | 7 x 5 | 6.2 [4.2, 9.5] / 19.7 [11.7, 24.5] | 12.0 [8.3, 21.4] / 38.5 [18.7, 58.4] | -5.9% | -6.1% | 4.5 / 18.2 | 7.7 / 32.1 | 99% | -| MoGe-2 placed with modern_assumed poses | 2 | 8 x 5 | 7.6 [6.2, 10.9] / 30.5 [15.2, 48.0] | 15.4 [11.7, 21.6] / 44.6 [23.9, 70.3] | +5.5% | +4.3% | 4.4 / 32.9 | 7.1 / 46.6 | 100% | -| MoGe-2 placed with modern_assumed poses | 4 | 8 x 5 | 6.4 [5.2, 8.6] / 21.4 [13.8, 36.7] | 13.4 [10.6, 18.6] / 37.8 [21.3, 66.5] | +4.8% | +4.8% | 4.7 / 27.0 | 7.9 / 47.7 | 99% | +| MoGe-2 placed with modern_assumed poses | 2 | 7 x 5 | 7.0 [5.9, 9.1] / 19.0 [13.0, 25.4] | 14.1 [11.0, 19.7] / 27.7 [22.2, 32.6] | +4.5% | +4.8% | 3.6 / 16.4 | 5.9 / 24.4 | 100% | +| MoGe-2 placed with modern_assumed poses | 4 | 7 x 5 | 6.1 [5.1, 8.0] / 17.3 [11.8, 21.9] | 12.3 [9.8, 15.6] / 26.7 [18.0, 31.3] | +4.6% | +5.2% | 3.9 / 15.8 | 6.7 / 25.5 | 99% | | MoGe-2 placed with modern_assumed poses | 8 | 7 x 5 | 6.6 [5.5, 7.6] / 15.6 [11.3, 18.9] | 11.2 [9.8, 13.0] / 24.3 [16.6, 28.4] | +3.7% | +3.8% | 4.7 / 16.4 | 7.3 / 28.3 | 99% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 2 | 8 x 5 | 4.8 [2.6, 24.6] / fails [11.3, inf] | 8.5 [4.6, 29.9] / fails [18.0, inf] | -0.8% | -1.3% | 5.3 / fails (13% failed) | 9.2 / fails (12% failed) | 88% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 4 | 8 x 5 | 3.9 [2.6, 7.7] / 27.3 [11.7, inf] | 7.5 [4.5, 13.3] / 42.5 [16.5, inf] | -2.3% | -2.4% | 4.8 / 34.0 | 8.0 / 55.2 | 98% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 8 | 7 x 5 | 3.7 [3.0, 4.7] / 11.6 [8.6, 14.7] | 6.1 [5.0, 8.0] / 17.5 [11.9, 23.3] | -3.1% | -3.0% | 3.5 / 14.5 | 5.6 / 23.4 | 99% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 2 | 7 x 5 | 3.9 [2.4, 12.6] / fails [8.5, inf] | 7.0 [4.3, 21.0] / fails [13.5, inf] | -1.6% | -1.5% | 4.2 / fails (15% failed) | 7.5 / fails (14% failed) | 86% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 4 | 7 x 5 | 3.3 [2.4, 5.1] / 15.2 [10.1, inf] | 6.2 [4.3, 10.0] / 21.8 [14.6, inf] | -2.6% | -2.5% | 3.9 / 17.7 | 6.8 / 27.7 | 97% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 8 | 7 x 5 | 3.7 [3.0, 4.7] / 11.6 [8.6, 14.7] | 6.1 [5.0, 8.0] / 17.5 [11.9, 23.3] | -3.1% | -3.0% | 3.5 / 14.5 | 5.6 / 23.3 | 99% | ## electro, within 6 m, surface interior | Method | Views | Groups x noise draws | Model scale: 1-3 m | 3-10 m | Scale error: 1-3 m | 3-10 m | One taped distance: 1-3 m | 3-10 m | Tape calibrated | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| mapanything@392, images only | 2 | 8 x 1 | 14.7 [9.0, 25.2] / fails [27.1, inf] | 29.5 [18.8, 45.8] / fails [50.9, inf] | +13.9% | +14.2% | 4.7 / fails (26% failed) | 10.1 / fails (28% failed) | 84% | -| mapanything@392, images only | 4 | 8 x 1 | 15.7 [11.9, 21.3] / 69.6 [23.1, inf] | 31.5 [26.2, 42.8] / fails [42.7, inf] | +17.1% | +16.8% | 3.4 / fails (18% failed) | 7.1 / fails (20% failed) | 90% | -| mapanything@392 + exact poses | 2 | 8 x 1 | 4.4 [2.2, 13.7] / fails [10.0, inf] | 9.9 [4.4, 30.1] / fails [19.0, inf] | +3.5% | +3.4% | 4.2 / fails (26% failed) | 9.2 / fails (28% failed) | 84% | -| mapanything@392 + exact poses | 4 | 8 x 1 | 4.1 [2.3, 6.7] / fails [8.3, inf] | 7.9 [4.2, 13.0] / fails [15.2, inf] | +4.2% | +3.5% | 3.6 / fails (19% failed) | 7.9 / fails (21% failed) | 89% | -| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 2 | 8 x 1 | 4.8 [2.9, 13.7] / fails [13.1, inf] | 11.0 [5.5, 29.1] / fails [32.1, inf] | -0.5% | -0.9% | 4.4 / fails (26% failed) | 9.7 / fails (28% failed) | 84% | -| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 4 | 8 x 1 | 5.0 [2.3, 9.7] / fails [10.6, inf] | 11.4 [5.4, 19.4] / fails [19.9, inf] | -2.1% | -3.1% | 3.7 / fails (20% failed) | 7.9 / fails (22% failed) | 88% | -| mapanything@392 + modern_assumed poses | 2 | 8 x 1 | 4.0 [2.4, 14.1] / fails [9.0, inf] | 8.5 [4.7, 31.1] / fails [20.5, inf] | +1.7% | +1.7% | 4.3 / fails (26% failed) | 9.2 / fails (28% failed) | 84% | -| mapanything@392 + modern_assumed poses | 4 | 8 x 1 | 3.4 [1.8, 6.3] / fails [7.9, inf] | 6.8 [2.9, 12.0] / fails [14.4, inf] | +2.6% | +1.9% | 3.6 / fails (19% failed) | 8.0 / fails (21% failed) | 89% | -| MoGe-2 placed with exact poses | 2 | 8 x 1 | 6.8 [2.9, 9.4] / 15.0 [11.4, 19.2] | 14.9 [5.7, 19.8] / 28.7 [20.3, 32.3] | +2.4% | +3.8% | 2.3 / 11.3 | 4.0 / 21.1 | 100% | -| MoGe-2 placed with exact poses | 4 | 8 x 1 | 6.1 [3.7, 8.9] / 14.4 [10.4, 21.0] | 13.9 [8.4, 18.1] / 25.9 [19.4, 29.9] | +2.5% | +4.4% | 2.5 / 15.7 | 4.4 / 33.1 | 100% | -| MoGe-2 placed with exact poses | 8 | 7 x 1 | 5.4 [3.3, 8.2] / 12.5 [8.8, 14.9] | 12.4 [6.8, 17.1] / 22.6 [16.4, 25.8] | +0.7% | +3.0% | 2.8 / 10.8 | 4.3 / 21.1 | 99% | -| MoGe-2 rescaled by triangulation, exact poses | 2 | 8 x 1 | 2.1 [1.2, 8.5] / fails [3.4, inf] | 4.1 [2.4, 28.3] / fails [5.5, inf] | +0.5% | +0.4% | 2.1 / fails (13% failed) | 3.9 / fails (14% failed) | 88% | -| MoGe-2 rescaled by triangulation, exact poses | 4 | 8 x 1 | 1.6 [1.1, 2.7] / 8.9 [3.3, 34.2] | 3.3 [1.7, 6.8] / 20.5 [5.2, 74.6] | -0.2% | -0.3% | 2.0 / 13.0 | 3.8 / 27.2 | 100% | -| MoGe-2 rescaled by triangulation, exact poses | 8 | 7 x 1 | 1.3 [1.0, 1.5] / 4.0 [3.3, 4.8] | 2.2 [1.5, 3.2] / 6.2 [3.9, 8.7] | -0.6% | -0.7% | 1.8 / 7.6 | 3.2 / 14.3 | 99% | -| MoGe-2 placed with advio_2018 poses | 2 | 8 x 5 | 6.8 [3.1, 9.5] / 15.4 [11.5, 19.4] | 15.1 [7.9, 19.7] / 28.6 [20.2, 32.3] | +1.9% | +3.2% | 2.7 / 13.1 | 4.7 / 24.2 | 100% | -| MoGe-2 placed with advio_2018 poses | 4 | 8 x 5 | 6.1 [3.8, 9.0] / 15.4 [10.2, 21.6] | 13.6 [8.4, 18.6] / 26.2 [19.2, 29.9] | +0.4% | +3.9% | 3.2 / 20.9 | 5.7 / 42.1 | 100% | -| MoGe-2 placed with advio_2018 poses | 8 | 7 x 5 | 5.9 [4.2, 8.7] / 14.0 [10.1, 16.8] | 13.0 [8.6, 18.2] / 24.2 [18.1, 27.4] | -1.7% | +1.8% | 4.3 / 16.7 | 7.3 / 32.3 | 100% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 2 | 8 x 5 | 10.7 [7.7, 20.1] / fails [17.3, inf] | 23.7 [15.6, 43.2] / fails [36.8, inf] | -8.7% | -8.8% | 2.9 / fails (13% failed) | 5.5 / fails (14% failed) | 88% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 4 | 8 x 5 | 10.3 [6.1, 15.2] / 30.7 [18.0, 70.3] | 22.1 [13.3, 32.5] / 61.9 [36.6, 134.0] | -9.0% | -8.8% | 4.5 / 28.5 | 7.8 / 60.1 | 100% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 8 | 7 x 5 | 6.2 [4.6, 8.6] / 14.6 [10.6, 18.7] | 13.0 [9.6, 19.6] / 27.9 [18.6, 33.6] | -7.3% | -7.4% | 4.0 / 18.5 | 6.8 / 35.8 | 100% | -| MoGe-2 placed with modern_assumed poses | 2 | 8 x 5 | 6.7 [2.7, 9.4] / 15.0 [11.2, 19.2] | 14.8 [5.6, 19.6] / 28.7 [19.9, 32.4] | +2.3% | +3.7% | 2.2 / 11.5 | 4.0 / 21.4 | 100% | -| MoGe-2 placed with modern_assumed poses | 4 | 8 x 5 | 6.1 [3.6, 9.0] / 14.5 [10.0, 21.1] | 13.8 [8.3, 18.4] / 25.9 [19.1, 30.2] | +2.0% | +4.3% | 2.5 / 16.1 | 4.5 / 33.4 | 100% | -| MoGe-2 placed with modern_assumed poses | 8 | 7 x 5 | 5.4 [3.6, 8.2] / 12.7 [9.0, 15.2] | 12.3 [7.8, 17.3] / 22.8 [16.9, 26.1] | -0.1% | +2.6% | 3.1 / 11.8 | 5.2 / 22.4 | 100% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 2 | 8 x 5 | 3.5 [1.9, 13.0] / fails [6.9, inf] | 8.1 [3.5, 32.4] / fails [14.3, inf] | -1.2% | -1.0% | 2.2 / fails (13% failed) | 4.2 / fails (14% failed) | 88% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 4 | 8 x 5 | 2.8 [1.9, 4.5] / 14.6 [5.4, inf] | 5.4 [3.2, 10.4] / 35.0 [9.5, inf] | -2.3% | -2.3% | 2.5 / 20.5 | 4.6 / 44.7 | 97% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 8 | 7 x 5 | 2.5 [2.1, 2.9] / 5.6 [4.6, 6.7] | 5.0 [3.8, 6.3] / 10.8 [7.4, 14.1] | -3.0% | -3.0% | 2.1 / 8.9 | 3.9 / 17.6 | 99% | +| mapanything@392, images only | 2 | 7 x 1 | 12.8 [8.0, 18.2] / 36.4 [23.4, inf] | 25.9 [16.7, 34.9] / fails [46.9, inf] | +13.2% | +13.7% | 3.8 / fails (17% failed) | 7.7 / fails (19% failed) | 90% | +| mapanything@392, images only | 4 | 7 x 1 | 14.7 [11.4, 18.9] / 31.3 [20.5, 37.3] | 29.6 [25.4, 37.1] / 59.2 [38.6, 68.6] | +16.7% | +16.5% | 2.8 / fails (12% failed) | 5.7 / fails (13% failed) | 93% | +| mapanything@392 + exact poses | 2 | 7 x 1 | 3.6 [1.9, 7.2] / 25.7 [8.3, inf] | 7.9 [3.4, 15.8] / fails [15.9, inf] | +3.1% | +3.1% | 3.5 / fails (17% failed) | 7.0 / fails (18% failed) | 91% | +| mapanything@392 + exact poses | 4 | 7 x 1 | 3.6 [2.1, 5.3] / 12.1 [7.5, 19.8] | 6.8 [3.8, 10.1] / 26.1 [14.2, 42.2] | +4.0% | +3.3% | 3.1 / fails (13% failed) | 6.5 / fails (14% failed) | 92% | +| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 2 | 7 x 1 | 4.0 [2.6, 8.1] / 25.2 [8.1, inf] | 8.1 [4.8, 22.4] / fails [19.5, inf] | -0.8% | -1.1% | 3.6 / fails (17% failed) | 7.3 / fails (18% failed) | 91% | +| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 4 | 7 x 1 | 4.2 [1.9, 8.3] / 18.6 [8.5, 24.7] | 9.9 [4.2, 16.5] / 33.8 [16.6, 44.0] | -2.4% | -3.4% | 3.1 / fails (14% failed) | 6.5 / fails (14% failed) | 91% | +| mapanything@392 + modern_assumed poses | 2 | 7 x 1 | 3.2 [2.1, 6.8] / 26.3 [7.2, inf] | 7.0 [4.2, 16.2] / fails [13.4, inf] | +1.4% | +1.4% | 3.5 / fails (17% failed) | 7.1 / fails (18% failed) | 91% | +| mapanything@392 + modern_assumed poses | 4 | 7 x 1 | 2.9 [1.6, 4.7] / 11.8 [6.9, 18.3] | 5.6 [2.5, 9.0] / 23.9 [12.9, 37.2] | +2.3% | +1.7% | 3.1 / fails (13% failed) | 6.5 / fails (14% failed) | 92% | +| MoGe-2 placed with exact poses | 2 | 7 x 1 | 6.6 [2.7, 9.4] / 13.9 [10.6, 16.5] | 14.6 [6.1, 19.8] / 27.8 [18.6, 31.5] | +2.0% | +2.6% | 2.0 / 8.3 | 3.5 / 15.6 | 99% | +| MoGe-2 placed with exact poses | 4 | 7 x 1 | 5.7 [3.3, 8.6] / 13.0 [9.2, 15.8] | 13.5 [7.7, 18.1] / 24.3 [17.8, 27.7] | +2.9% | +4.4% | 2.2 / 10.0 | 3.9 / 18.9 | 99% | +| MoGe-2 placed with exact poses | 8 | 7 x 1 | 5.4 [3.3, 8.2] / 12.5 [8.8, 14.9] | 12.4 [6.8, 17.1] / 22.6 [16.4, 25.8] | +0.7% | +3.0% | 2.8 / 11.9 | 4.5 / 24.3 | 98% | +| MoGe-2 rescaled by triangulation, exact poses | 2 | 7 x 1 | 1.7 [1.1, 3.4] / fails [3.2, inf] | 3.6 [2.2, 6.5] / fails [4.8, inf] | -0.0% | -0.3% | 1.6 / fails (14% failed) | 3.1 / fails (14% failed) | 86% | +| MoGe-2 rescaled by triangulation, exact poses | 4 | 7 x 1 | 1.4 [1.0, 1.8] / 4.5 [3.1, 6.5] | 2.8 [1.5, 4.2] / 9.0 [4.0, 10.9] | -0.5% | -0.5% | 1.6 / 7.1 | 3.1 / 13.3 | 99% | +| MoGe-2 rescaled by triangulation, exact poses | 8 | 7 x 1 | 1.3 [1.0, 1.5] / 4.0 [3.3, 4.8] | 2.2 [1.5, 3.2] / 6.2 [3.9, 8.7] | -0.6% | -0.7% | 1.8 / 7.6 | 3.3 / 14.9 | 99% | +| MoGe-2 placed with advio_2018 poses | 2 | 7 x 5 | 6.6 [3.0, 9.5] / 14.3 [11.1, 16.7] | 14.8 [7.6, 20.0] / 27.7 [19.3, 31.5] | +1.4% | +2.2% | 2.3 / 9.6 | 4.2 / 17.3 | 100% | +| MoGe-2 placed with advio_2018 poses | 4 | 7 x 5 | 5.6 [3.5, 8.9] / 13.5 [9.6, 16.8] | 13.3 [8.4, 18.7] / 25.0 [18.3, 28.4] | +0.8% | +3.9% | 2.8 / 13.6 | 5.0 / 26.6 | 100% | +| MoGe-2 placed with advio_2018 poses | 8 | 7 x 5 | 5.9 [4.2, 8.7] / 14.0 [10.1, 16.8] | 13.0 [8.6, 18.2] / 24.2 [18.1, 27.4] | -1.7% | +1.8% | 4.3 / 16.7 | 7.5 / 32.5 | 100% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 2 | 7 x 5 | 10.1 [7.0, 21.1] / fails [14.7, inf] | 22.3 [14.1, 45.9] / fails [32.0, inf] | -9.7% | -9.5% | 2.4 / fails (14% failed) | 4.7 / fails (14% failed) | 86% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 4 | 7 x 5 | 10.1 [5.7, 15.8] / 26.5 [16.4, 70.8] | 21.2 [12.1, 32.5] / 50.1 [32.7, 136.1] | -10.2% | -10.4% | 3.7 / 20.4 | 6.3 / 41.8 | 100% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 8 | 7 x 5 | 6.2 [4.6, 8.6] / 14.6 [10.6, 18.7] | 13.0 [9.6, 19.6] / 27.9 [18.6, 33.6] | -7.3% | -7.4% | 4.0 / 19.3 | 7.0 / 38.3 | 99% | +| MoGe-2 placed with modern_assumed poses | 2 | 7 x 5 | 6.6 [2.8, 9.4] / 13.9 [10.8, 16.6] | 14.6 [6.3, 19.9] / 27.7 [19.0, 31.6] | +1.9% | +2.5% | 2.0 / 8.3 | 3.5 / 15.6 | 99% | +| MoGe-2 placed with modern_assumed poses | 4 | 7 x 5 | 5.6 [3.5, 8.8] / 13.0 [9.3, 15.8] | 13.4 [8.0, 18.3] / 24.3 [18.3, 27.8] | +2.5% | +4.3% | 2.2 / 9.7 | 3.9 / 18.7 | 99% | +| MoGe-2 placed with modern_assumed poses | 8 | 7 x 5 | 5.4 [3.6, 8.2] / 12.7 [9.0, 15.2] | 12.3 [7.8, 17.3] / 22.8 [16.9, 26.1] | -0.1% | +2.6% | 3.2 / 12.6 | 5.4 / 24.9 | 98% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 2 | 7 x 5 | 2.8 [1.6, 8.1] / fails [4.8, inf] | 6.2 [2.8, 16.7] / fails [11.0, inf] | -1.8% | -1.8% | 1.7 / fails (14% failed) | 3.2 / fails (14% failed) | 86% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 4 | 7 x 5 | 2.5 [1.7, 3.7] / 7.9 [4.6, inf] | 4.7 [2.6, 7.9] / 15.8 [7.5, inf] | -2.7% | -2.6% | 2.0 / 9.9 | 3.7 / 20.3 | 96% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 8 | 7 x 5 | 2.5 [2.1, 2.9] / 5.6 [4.6, 6.7] | 5.0 [3.8, 6.3] / 10.8 [7.4, 14.1] | -3.0% | -3.0% | 2.1 / 8.8 | 4.0 / 18.0 | 99% | ## electro, within 6 m, vertical interior | Method | Views | Groups x noise draws | Model scale: 1-3 m | 3-10 m | Scale error: 1-3 m | 3-10 m | One taped distance: 1-3 m | 3-10 m | Tape calibrated | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| mapanything@392, images only | 2 | 7 x 1 | 15.2 [8.4, 41.5] / fails [31.8, inf] | 29.9 [18.1, 126.6] / fails [inf, inf] | +11.9% | +11.4% | 5.4 / fails (32% failed) | 14.0 / fails (37% failed) | 82% | -| mapanything@392, images only | 4 | 7 x 1 | 16.8 [12.6, 24.2] / fails [28.6, inf] | 29.5 [24.8, 52.5] / fails [59.0, inf] | +17.2% | +16.4% | 4.5 / fails (23% failed) | 9.9 / fails (29% failed) | 88% | -| mapanything@392 + exact poses | 2 | 7 x 1 | 4.6 [2.0, 21.1] / fails [19.4, inf] | 14.6 [4.8, 92.6] / fails [inf, inf] | +3.4% | +4.2% | 4.9 / fails (32% failed) | 12.3 / fails (37% failed) | 82% | -| mapanything@392 + exact poses | 4 | 7 x 1 | 3.9 [2.1, 10.8] / fails [8.6, inf] | 7.9 [4.3, 22.5] / fails [20.2, inf] | +3.8% | +3.6% | 4.4 / fails (24% failed) | 9.8 / fails (29% failed) | 87% | -| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 2 | 7 x 1 | 6.2 [3.0, 18.8] / fails [18.2, inf] | 13.6 [4.0, 456.3] / fails [inf, inf] | +1.4% | +3.2% | 5.0 / fails (32% failed) | 12.7 / fails (37% failed) | 81% | -| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 4 | 7 x 1 | 6.4 [3.0, 11.5] / fails [13.5, inf] | 12.0 [6.5, 24.2] / fails [28.4, inf] | -1.5% | -2.2% | 4.5 / fails (24% failed) | 9.9 / fails (29% failed) | 87% | -| mapanything@392 + modern_assumed poses | 2 | 7 x 1 | 4.8 [1.7, 21.6] / fails [19.6, inf] | 12.1 [4.8, 109.7] / fails [inf, inf] | +1.6% | +2.7% | 4.9 / fails (32% failed) | 12.3 / fails (37% failed) | 82% | -| mapanything@392 + modern_assumed poses | 4 | 7 x 1 | 3.0 [1.4, 9.9] / fails [8.5, inf] | 6.7 [2.2, 20.6] / fails [19.1, inf] | +2.1% | +2.1% | 4.3 / fails (24% failed) | 9.8 / fails (29% failed) | 87% | -| MoGe-2 placed with exact poses | 2 | 7 x 1 | 5.2 [2.0, 8.4] / 15.3 [9.9, 19.0] | 13.3 [5.8, 20.0] / 31.6 [20.5, 40.6] | -0.1% | +2.5% | 2.4 / 12.8 | 5.1 / 32.2 | 100% | -| MoGe-2 placed with exact poses | 4 | 7 x 1 | 5.6 [2.9, 8.6] / 15.0 [9.7, 23.7] | 13.7 [8.8, 19.6] / 30.4 [21.5, 48.4] | -0.1% | +4.3% | 2.4 / 19.8 | 5.3 / 52.6 | 99% | -| MoGe-2 placed with exact poses | 8 | 6 x 1 | 4.8 [2.7, 8.3] / 12.4 [7.7, 15.6] | 11.6 [6.9, 18.2] / 25.8 [16.0, 32.6] | -1.3% | +2.7% | 2.8 / 10.0 | 5.5 / 25.6 | 99% | -| MoGe-2 rescaled by triangulation, exact poses | 2 | 7 x 1 | 1.5 [0.9, 27.4] / fails [2.3, inf] | 3.1 [2.1, 60.8] / fails [6.9, inf] | -0.4% | +0.9% | 2.3 / fails (15% failed) | 5.0 / fails (17% failed) | 86% | -| MoGe-2 rescaled by triangulation, exact poses | 4 | 7 x 1 | 1.4 [0.9, 2.6] / 10.9 [2.4, 41.2] | 2.7 [1.5, 10.3] / 53.1 [5.2, 79.6] | -0.9% | -0.2% | 1.9 / 16.2 | 4.7 / 43.4 | 99% | -| MoGe-2 rescaled by triangulation, exact poses | 8 | 6 x 1 | 1.1 [0.8, 1.4] / 2.8 [2.0, 4.4] | 2.1 [1.5, 4.0] / 8.3 [3.3, 19.0] | -0.7% | -0.5% | 1.5 / 6.6 | 3.1 / 16.2 | 99% | -| MoGe-2 placed with advio_2018 poses | 2 | 7 x 5 | 5.4 [2.2, 8.6] / 15.3 [10.3, 19.1] | 13.6 [5.8, 21.9] / 31.9 [20.9, 42.8] | -0.7% | +2.4% | 2.7 / 14.6 | 5.6 / 37.0 | 100% | -| MoGe-2 placed with advio_2018 poses | 4 | 7 x 5 | 5.9 [3.2, 8.9] / 15.1 [10.2, 23.1] | 13.6 [8.6, 19.9] / 30.1 [21.4, 46.1] | -0.6% | +4.1% | 2.7 / 23.1 | 6.1 / 56.8 | 100% | -| MoGe-2 placed with advio_2018 poses | 8 | 6 x 5 | 5.5 [3.2, 8.4] / 13.0 [8.3, 16.6] | 11.2 [6.6, 18.4] / 26.7 [14.3, 32.5] | -2.3% | +2.3% | 3.5 / 13.9 | 6.5 / 34.6 | 100% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 2 | 7 x 5 | 11.2 [8.2, 30.1] / fails [18.4, inf] | 21.7 [14.2, 74.6] / fails [35.2, inf] | -9.2% | -7.4% | 3.1 / fails (15% failed) | 6.2 / fails (17% failed) | 86% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 4 | 7 x 5 | 11.2 [6.6, 16.7] / 34.1 [19.1, 67.4] | 24.1 [17.2, 35.0] / 76.7 [37.1, 143.4] | -10.5% | -10.4% | 3.6 / 30.4 | 8.8 / 67.2 | 100% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 8 | 6 x 5 | 4.9 [3.7, 6.5] / 10.8 [8.6, 13.1] | 10.0 [7.1, 12.6] / 18.4 [15.2, 26.9] | -5.6% | -5.2% | 3.1 / 15.2 | 5.9 / 35.6 | 100% | -| MoGe-2 placed with modern_assumed poses | 2 | 7 x 5 | 5.2 [2.1, 8.5] / 15.3 [10.1, 19.1] | 13.3 [5.9, 21.9] / 31.7 [20.9, 39.9] | -0.2% | +2.5% | 2.4 / 13.0 | 5.1 / 32.6 | 100% | -| MoGe-2 placed with modern_assumed poses | 4 | 7 x 5 | 5.6 [2.9, 8.9] / 15.0 [9.9, 23.8] | 13.7 [8.9, 20.0] / 30.3 [21.6, 46.6] | -0.2% | +4.3% | 2.4 / 19.5 | 5.3 / 52.3 | 100% | -| MoGe-2 placed with modern_assumed poses | 8 | 6 x 5 | 5.1 [2.9, 8.3] / 12.5 [7.3, 15.6] | 11.4 [6.5, 17.8] / 25.9 [14.5, 32.7] | -1.6% | +2.6% | 3.0 / 10.8 | 5.8 / 26.9 | 100% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 2 | 7 x 5 | 3.5 [1.8, 30.2] / fails [7.3, inf] | 10.1 [3.3, 65.3] / fails [17.3, inf] | -1.7% | +0.1% | 2.4 / fails (15% failed) | 5.2 / fails (17% failed) | 86% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 4 | 7 x 5 | 3.1 [2.0, 4.9] / 19.4 [5.5, inf] | 5.7 [3.7, 12.2] / 69.5 [11.0, inf] | -3.0% | -2.2% | 2.3 / 27.2 | 5.2 / 62.6 | 97% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 8 | 6 x 5 | 2.3 [1.7, 2.8] / 5.0 [3.8, 6.8] | 4.6 [3.5, 6.3] / 12.2 [6.3, 21.2] | -2.9% | -2.7% | 1.7 / 8.0 | 3.8 / 19.7 | 99% | +| mapanything@392, images only | 2 | 6 x 1 | 12.6 [7.6, 19.4] / fails [23.7, inf] | 24.7 [15.7, 45.6] / fails [49.1, inf] | +10.9% | +10.1% | 4.5 / fails (23% failed) | 10.6 / fails (29% failed) | 89% | +| mapanything@392, images only | 4 | 6 x 1 | 15.6 [11.7, 20.8] / 39.9 [20.6, inf] | 27.8 [24.1, 45.3] / fails [39.4, inf] | +17.0% | +16.2% | 3.8 / fails (16% failed) | 8.0 / fails (22% failed) | 91% | +| mapanything@392 + exact poses | 2 | 6 x 1 | 3.4 [1.8, 11.4] / fails [5.5, inf] | 8.6 [3.6, 28.5] / fails [37.6, inf] | +3.1% | +4.0% | 4.1 / fails (22% failed) | 9.3 / fails (28% failed) | 89% | +| mapanything@392 + exact poses | 4 | 6 x 1 | 3.2 [1.8, 9.9] / 21.9 [4.8, inf] | 6.2 [3.9, 21.4] / fails [15.8, inf] | +3.6% | +3.4% | 3.7 / fails (17% failed) | 8.0 / fails (22% failed) | 91% | +| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 2 | 6 x 1 | 4.8 [2.5, 10.7] / fails [7.5, inf] | 9.6 [3.4, 26.6] / fails [34.3, inf] | +0.3% | +2.7% | 4.1 / fails (22% failed) | 9.6 / fails (28% failed) | 89% | +| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 4 | 6 x 1 | 5.7 [2.1, 10.7] / 23.2 [8.7, inf] | 10.5 [6.1, 21.5] / fails [15.5, inf] | -2.5% | -2.8% | 3.8 / fails (17% failed) | 8.3 / fails (22% failed) | 90% | +| mapanything@392 + modern_assumed poses | 2 | 6 x 1 | 3.6 [1.4, 11.6] / fails [5.8, inf] | 7.6 [4.1, 29.0] / fails [38.8, inf] | +1.4% | +2.5% | 4.1 / fails (22% failed) | 9.2 / fails (28% failed) | 89% | +| mapanything@392 + modern_assumed poses | 4 | 6 x 1 | 2.3 [1.2, 9.1] / 19.9 [3.5, inf] | 4.0 [1.9, 18.9] / fails [12.1, inf] | +1.9% | +1.8% | 3.7 / fails (17% failed) | 8.0 / fails (22% failed) | 91% | +| MoGe-2 placed with exact poses | 2 | 6 x 1 | 4.9 [1.9, 8.8] / 13.8 [8.0, 17.6] | 12.9 [4.7, 23.1] / 31.4 [18.3, 38.0] | -1.3% | +1.9% | 2.2 / 8.6 | 4.5 / 21.9 | 99% | +| MoGe-2 placed with exact poses | 4 | 6 x 1 | 5.0 [2.2, 8.4] / 12.6 [7.7, 16.0] | 13.1 [8.1, 19.7] / 28.9 [17.7, 45.0] | +0.1% | +4.2% | 2.1 / 11.0 | 4.4 / 30.1 | 99% | +| MoGe-2 placed with exact poses | 8 | 6 x 1 | 4.8 [2.7, 8.2] / 12.5 [7.8, 15.6] | 11.8 [7.0, 18.4] / 26.0 [16.4, 32.6] | -1.3% | +2.7% | 2.9 / 11.4 | 5.6 / 27.6 | 97% | +| MoGe-2 rescaled by triangulation, exact poses | 2 | 6 x 1 | 1.3 [0.8, inf] / fails [2.1, inf] | 2.7 [1.9, inf] / fails [3.5, inf] | -0.8% | +0.3% | 1.8 / fails (18% failed) | 4.1 / fails (18% failed) | 83% | +| MoGe-2 rescaled by triangulation, exact poses | 4 | 6 x 1 | 1.2 [0.8, 1.7] / 3.6 [2.1, 6.6] | 2.2 [1.3, 5.9] / 11.3 [3.5, 29.5] | -1.1% | -0.5% | 1.5 / 7.1 | 3.7 / 19.8 | 99% | +| MoGe-2 rescaled by triangulation, exact poses | 8 | 6 x 1 | 1.1 [0.8, 1.4] / 2.8 [2.0, 4.6] | 2.1 [1.5, 4.2] / 8.6 [3.4, 19.1] | -0.7% | -0.5% | 1.5 / 6.6 | 3.2 / 16.1 | 99% | +| MoGe-2 placed with advio_2018 poses | 2 | 6 x 5 | 5.2 [1.8, 8.7] / 13.8 [7.5, 17.3] | 13.2 [4.2, 21.6] / 31.5 [17.3, 41.4] | -1.6% | +1.8% | 2.2 / 9.9 | 4.8 / 29.4 | 100% | +| MoGe-2 placed with advio_2018 poses | 4 | 6 x 5 | 5.3 [2.5, 8.4] / 12.8 [8.0, 16.0] | 13.0 [7.4, 19.4] / 28.9 [16.0, 44.9] | -0.3% | +4.0% | 2.3 / 13.5 | 5.0 / 36.0 | 100% | +| MoGe-2 placed with advio_2018 poses | 8 | 6 x 5 | 5.5 [3.3, 8.4] / 13.0 [8.3, 16.7] | 11.5 [6.6, 18.6] / 26.9 [14.5, 32.9] | -2.4% | +2.2% | 3.6 / 14.1 | 6.6 / 32.3 | 100% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 2 | 6 x 5 | 10.6 [7.3, inf] / fails [15.0, inf] | 19.9 [12.7, inf] / fails [26.5, inf] | -11.6% | -9.1% | 2.5 / fails (18% failed) | 5.3 / fails (18% failed) | 83% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 4 | 6 x 5 | 11.0 [6.0, 16.3] / 29.0 [15.8, 66.1] | 22.7 [16.5, 32.6] / 54.9 [32.9, 139.5] | -11.7% | -11.4% | 2.7 / 19.8 | 6.7 / 47.8 | 100% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 8 | 6 x 5 | 4.9 [3.7, 6.5] / 10.8 [8.6, 13.1] | 10.1 [7.1, 12.6] / 18.7 [15.3, 27.0] | -5.6% | -5.1% | 3.2 / 16.1 | 5.9 / 35.3 | 99% | +| MoGe-2 placed with modern_assumed poses | 2 | 6 x 5 | 5.0 [1.8, 8.7] / 13.8 [7.1, 17.5] | 13.0 [4.4, 21.7] / 31.4 [17.3, 38.6] | -1.4% | +1.9% | 2.2 / 8.6 | 4.5 / 22.6 | 99% | +| MoGe-2 placed with modern_assumed poses | 4 | 6 x 5 | 5.0 [2.5, 8.4] / 12.7 [7.2, 15.9] | 13.1 [7.7, 19.3] / 28.8 [16.0, 45.0] | -0.0% | +4.2% | 2.0 / 9.9 | 4.3 / 27.4 | 99% | +| MoGe-2 placed with modern_assumed poses | 8 | 6 x 5 | 5.1 [2.9, 8.2] / 12.6 [7.3, 15.7] | 11.7 [6.5, 18.2] / 26.1 [14.7, 32.8] | -1.6% | +2.6% | 3.2 / 11.9 | 5.8 / 27.8 | 98% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 2 | 6 x 5 | 2.7 [1.5, inf] / fails [3.8, inf] | 6.4 [2.6, inf] / fails [12.1, inf] | -2.2% | -0.8% | 1.9 / fails (18% failed) | 4.4 / fails (18% failed) | 83% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 4 | 6 x 5 | 2.8 [1.6, 4.0] / 8.1 [4.4, inf] | 4.9 [2.9, 8.2] / 22.7 [7.3, inf] | -3.4% | -2.6% | 1.8 / 10.1 | 4.1 / 27.7 | 96% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 8 | 6 x 5 | 2.3 [1.7, 2.8] / 5.0 [3.7, 6.9] | 4.7 [3.5, 6.5] / 12.7 [6.4, 21.7] | -2.9% | -2.7% | 1.7 / 7.8 | 3.8 / 19.1 | 99% | ## electro, within 6 m, edges | Method | Views | Groups x noise draws | Model scale: 1-3 m | 3-10 m | Scale error: 1-3 m | 3-10 m | One taped distance: 1-3 m | 3-10 m | Tape calibrated | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| mapanything@392, images only | 2 | 7 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +30.1% | +29.9% | inf / fails (84% failed) | inf / fails (84% failed) | 82% | -| mapanything@392, images only | 4 | 7 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +29.9% | +25.6% | inf / fails (72% failed) | inf / fails (77% failed) | 88% | -| mapanything@392 + exact poses | 2 | 7 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +8.3% | +11.2% | inf / fails (84% failed) | inf / fails (84% failed) | 82% | -| mapanything@392 + exact poses | 4 | 7 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +9.5% | +6.2% | inf / fails (72% failed) | inf / fails (76% failed) | 87% | -| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 2 | 7 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +4.9% | +5.4% | inf / fails (84% failed) | inf / fails (84% failed) | 81% | -| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 4 | 7 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +9.0% | +5.0% | inf / fails (72% failed) | inf / fails (76% failed) | 87% | -| mapanything@392 + modern_assumed poses | 2 | 7 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +7.5% | +11.0% | inf / fails (84% failed) | inf / fails (84% failed) | 82% | -| mapanything@392 + modern_assumed poses | 4 | 7 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +8.8% | +5.0% | inf / fails (72% failed) | inf / fails (76% failed) | 87% | -| MoGe-2 placed with exact poses | 2 | 7 x 1 | 8.5 [3.2, 13.1] / 33.8 [13.8, 50.6] | 15.1 [5.1, 22.7] / 49.9 [19.4, 68.7] | +1.2% | +2.4% | 3.9 / 27.3 | 5.5 / 38.1 | 100% | -| MoGe-2 placed with exact poses | 4 | 7 x 1 | 7.5 [3.9, 11.3] / 20.6 [13.6, 41.8] | 13.3 [6.2, 17.2] / 34.6 [18.0, 58.7] | +2.8% | +3.3% | 4.2 / 26.4 | 6.4 / 50.0 | 99% | -| MoGe-2 placed with exact poses | 8 | 6 x 1 | 6.4 [3.1, 10.1] / 14.6 [11.8, 19.0] | 7.8 [2.8, 15.0] / 18.5 [13.6, 25.2] | +1.2% | -0.3% | 4.0 / 15.3 | 6.1 / 25.2 | 99% | -| MoGe-2 rescaled by triangulation, exact poses | 2 | 7 x 1 | 3.0 [1.2, 60.9] / fails [7.2, inf] | 3.8 [2.1, 85.8] / fails [8.4, inf] | -0.0% | -0.6% | 3.8 / fails (15% failed) | 5.7 / fails (14% failed) | 86% | -| MoGe-2 rescaled by triangulation, exact poses | 4 | 7 x 1 | 2.1 [1.2, 6.5] / 24.9 [5.7, 67.5] | 3.3 [1.9, 10.5] / 53.7 [7.6, 84.1] | -0.7% | -1.0% | 3.2 / 23.5 | 5.1 / 41.0 | 99% | -| MoGe-2 rescaled by triangulation, exact poses | 8 | 6 x 1 | 1.6 [1.1, 2.9] / 8.2 [2.8, 13.0] | 2.2 [1.5, 5.2] / 12.3 [3.3, 15.7] | -0.8% | -1.1% | 2.4 / 11.0 | 3.6 / 17.4 | 99% | -| MoGe-2 placed with advio_2018 poses | 2 | 7 x 5 | 8.6 [3.8, 13.2] / 34.0 [14.4, 50.7] | 15.4 [4.3, 23.2] / 49.4 [20.4, 68.3] | +1.2% | +1.6% | 4.3 / 28.9 | 6.2 / 40.0 | 100% | -| MoGe-2 placed with advio_2018 poses | 4 | 7 x 5 | 7.9 [4.1, 11.4] / 20.5 [14.3, 42.2] | 13.4 [5.9, 17.6] / 35.7 [18.1, 59.4] | +2.6% | +2.9% | 4.8 / 30.6 | 7.7 / 51.5 | 100% | -| MoGe-2 placed with advio_2018 poses | 8 | 6 x 5 | 6.7 [3.4, 10.4] / 14.9 [11.9, 19.3] | 8.2 [3.0, 15.3] / 18.7 [13.1, 23.8] | +1.0% | -0.9% | 4.7 / 19.9 | 7.1 / 31.8 | 100% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 2 | 7 x 5 | 11.9 [8.3, 57.1] / fails [20.3, inf] | 22.2 [14.6, 74.9] / fails [36.2, inf] | -8.6% | -9.2% | 4.5 / fails (15% failed) | 6.8 / fails (14% failed) | 86% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 4 | 7 x 5 | 11.6 [7.9, 18.2] / 33.8 [20.8, 70.2] | 23.0 [14.7, 37.4] / 63.5 [32.5, 100.5] | -10.2% | -10.4% | 5.3 / 34.3 | 8.6 / 59.5 | 100% | -| MoGe-2 rescaled by triangulation, advio_2018 poses | 8 | 6 x 5 | 5.5 [3.8, 7.4] / 13.4 [9.1, 17.1] | 10.5 [7.8, 14.4] / 20.5 [14.3, 23.8] | -5.8% | -6.8% | 4.0 / 20.4 | 6.4 / 31.3 | 100% | -| MoGe-2 placed with modern_assumed poses | 2 | 7 x 5 | 8.5 [3.4, 13.2] / 33.8 [14.2, 50.6] | 15.1 [4.7, 23.1] / 49.5 [20.3, 68.3] | +1.2% | +2.2% | 3.9 / 27.3 | 5.6 / 38.1 | 100% | -| MoGe-2 placed with modern_assumed poses | 4 | 7 x 5 | 7.5 [4.0, 11.2] / 20.5 [14.0, 41.3] | 13.3 [6.2, 17.3] / 34.9 [18.1, 58.4] | +2.8% | +3.2% | 4.2 / 26.6 | 6.5 / 49.4 | 100% | -| MoGe-2 placed with modern_assumed poses | 8 | 6 x 5 | 6.5 [3.0, 10.2] / 14.6 [11.4, 18.9] | 7.8 [2.7, 15.1] / 18.6 [12.8, 24.0] | +1.0% | -0.5% | 4.2 / 16.1 | 6.5 / 25.6 | 100% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 2 | 7 x 5 | 5.1 [2.6, 63.8] / fails [11.2, inf] | 9.8 [4.0, 89.5] / fails [17.5, inf] | -1.3% | -1.3% | 3.8 / fails (15% failed) | 5.7 / fails (14% failed) | 86% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 4 | 7 x 5 | 3.7 [2.8, 7.0] / 31.6 [7.3, inf] | 6.4 [4.5, 15.3] / 59.1 [11.0, inf] | -2.9% | -3.0% | 3.5 / 34.3 | 5.5 / 60.6 | 97% | -| MoGe-2 rescaled by triangulation, modern_assumed poses | 8 | 6 x 5 | 3.1 [2.7, 3.8] / 9.6 [4.9, 15.0] | 5.5 [4.7, 7.5] / 14.8 [7.2, 20.3] | -3.1% | -3.5% | 2.7 / 13.5 | 4.0 / 19.5 | 99% | +| mapanything@392, images only | 2 | 6 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +26.1% | +6.5% | inf / fails (83% failed) | inf / fails (85% failed) | 89% | +| mapanything@392, images only | 4 | 6 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +27.8% | +11.0% | inf / fails (72% failed) | inf / fails (81% failed) | 91% | +| mapanything@392 + exact poses | 2 | 6 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +5.3% | +4.8% | inf / fails (83% failed) | inf / fails (85% failed) | 89% | +| mapanything@392 + exact poses | 4 | 6 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +7.3% | +4.8% | inf / fails (73% failed) | inf / fails (80% failed) | 91% | +| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 2 | 6 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +4.5% | +4.3% | inf / fails (82% failed) | inf / fails (85% failed) | 89% | +| mapanything@392 + advio_2018 poses, earlier ARCore-referenced scales | 4 | 6 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +4.8% | +1.0% | inf / fails (73% failed) | inf / fails (80% failed) | 90% | +| mapanything@392 + modern_assumed poses | 2 | 6 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +3.0% | +2.4% | inf / fails (83% failed) | inf / fails (86% failed) | 89% | +| mapanything@392 + modern_assumed poses | 4 | 6 x 1 | fails [inf, inf] / fails [inf, inf] | fails [inf, inf] / fails [inf, inf] | +6.3% | +2.8% | inf / fails (72% failed) | inf / fails (80% failed) | 91% | +| MoGe-2 placed with exact poses | 2 | 6 x 1 | 7.2 [2.5, 11.6] / 17.3 [12.4, 28.1] | 12.0 [4.4, 19.3] / 23.6 [17.5, 35.8] | +0.4% | +1.4% | 3.2 / 14.6 | 4.7 / 18.8 | 99% | +| MoGe-2 placed with exact poses | 4 | 6 x 1 | 6.9 [3.6, 11.1] / 16.0 [12.4, 20.8] | 11.4 [5.6, 15.8] / 21.1 [16.2, 30.5] | +2.4% | +3.0% | 3.5 / 16.5 | 5.2 / 24.7 | 99% | +| MoGe-2 placed with exact poses | 8 | 6 x 1 | 6.4 [3.1, 10.1] / 14.6 [11.8, 19.0] | 7.8 [2.8, 15.1] / 18.5 [13.7, 25.3] | +1.3% | -0.3% | 4.0 / 16.9 | 6.2 / 27.6 | 97% | +| MoGe-2 rescaled by triangulation, exact poses | 2 | 6 x 1 | 2.2 [1.1, inf] / fails [3.8, inf] | 3.1 [1.9, 32.0] / fails [4.7, inf] | -0.6% | -1.0% | 3.0 / fails (18% failed) | 4.5 / fails (16% failed) | 83% | +| MoGe-2 rescaled by triangulation, exact poses | 4 | 6 x 1 | 1.7 [1.1, 3.5] / 10.7 [3.4, 14.7] | 2.7 [1.7, 5.9] / 11.1 [4.4, 19.7] | -1.0% | -1.2% | 2.5 / 12.5 | 3.9 / 17.2 | 99% | +| MoGe-2 rescaled by triangulation, exact poses | 8 | 6 x 1 | 1.6 [1.1, 2.9] / 8.2 [2.9, 13.0] | 2.2 [1.5, 5.3] / 12.3 [3.4, 16.0] | -0.8% | -1.1% | 2.4 / 10.8 | 3.7 / 17.2 | 99% | +| MoGe-2 placed with advio_2018 poses | 2 | 6 x 5 | 7.4 [2.3, 11.5] / 17.5 [13.0, 27.7] | 12.2 [3.1, 19.0] / 23.6 [17.0, 35.8] | +0.4% | +1.2% | 3.5 / 16.7 | 5.0 / 23.2 | 100% | +| MoGe-2 placed with advio_2018 poses | 4 | 6 x 5 | 7.4 [3.5, 11.2] / 16.5 [13.3, 21.2] | 11.5 [4.8, 15.9] / 21.1 [16.0, 30.0] | +2.2% | +2.2% | 3.9 / 19.6 | 6.0 / 28.6 | 100% | +| MoGe-2 placed with advio_2018 poses | 8 | 6 x 5 | 6.7 [3.4, 10.4] / 14.9 [11.9, 19.3] | 8.2 [3.0, 15.3] / 18.7 [13.1, 24.0] | +1.1% | -0.9% | 4.6 / 19.1 | 7.2 / 30.6 | 100% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 2 | 6 x 5 | 10.5 [7.4, inf] / fails [17.0, inf] | 18.6 [13.3, 51.4] / fails [25.8, inf] | -9.9% | -10.1% | 3.5 / fails (18% failed) | 5.5 / fails (16% failed) | 83% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 4 | 6 x 5 | 10.9 [7.1, 17.2] / 27.3 [16.6, 46.8] | 20.1 [12.7, 31.4] / 45.4 [27.5, 99.8] | -11.9% | -11.8% | 4.3 / 23.3 | 6.8 / 39.1 | 100% | +| MoGe-2 rescaled by triangulation, advio_2018 poses | 8 | 6 x 5 | 5.5 [3.8, 7.4] / 13.4 [9.1, 17.1] | 10.5 [7.7, 14.4] / 20.5 [14.3, 23.8] | -5.7% | -6.8% | 4.0 / 20.6 | 6.4 / 31.6 | 99% | +| MoGe-2 placed with modern_assumed poses | 2 | 6 x 5 | 7.2 [2.2, 11.5] / 17.3 [12.2, 27.6] | 12.0 [3.6, 19.1] / 23.6 [15.8, 35.2] | +0.4% | +1.4% | 3.2 / 14.8 | 4.7 / 18.9 | 99% | +| MoGe-2 placed with modern_assumed poses | 4 | 6 x 5 | 7.0 [3.3, 11.1] / 16.1 [12.2, 20.8] | 11.4 [5.2, 15.9] / 21.1 [15.9, 30.0] | +2.4% | +2.8% | 3.5 / 15.1 | 5.2 / 22.0 | 99% | +| MoGe-2 placed with modern_assumed poses | 8 | 6 x 5 | 6.5 [3.0, 10.2] / 14.6 [11.4, 18.9] | 7.8 [2.7, 15.1] / 18.6 [12.9, 24.1] | +1.1% | -0.5% | 4.1 / 17.2 | 6.5 / 27.0 | 98% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 2 | 6 x 5 | 4.0 [2.1, inf] / fails [7.1, inf] | 6.8 [3.2, 32.2] / fails [12.5, inf] | -2.2% | -2.0% | 3.0 / fails (18% failed) | 4.6 / fails (16% failed) | 83% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 4 | 6 x 5 | 3.3 [2.7, 4.8] / 13.8 [5.4, inf] | 5.6 [4.1, 7.6] / 16.7 [8.5, inf] | -3.3% | -3.3% | 2.8 / 16.4 | 4.3 / 24.5 | 96% | +| MoGe-2 rescaled by triangulation, modern_assumed poses | 8 | 6 x 5 | 3.1 [2.8, 3.8] / 9.7 [4.9, 15.0] | 5.5 [4.7, 7.6] / 14.8 [7.2, 20.3] | -3.1% | -3.5% | 2.6 / 13.0 | 3.9 / 18.5 | 99% | diff --git a/experiments/evals/tests/test_field.py b/experiments/evals/tests/test_field.py index c4198dcb4..85d9ae643 100644 --- a/experiments/evals/tests/test_field.py +++ b/experiments/evals/tests/test_field.py @@ -100,6 +100,13 @@ def _depth(kid, scale): return d +@pytest.fixture(autouse=True) +def _free_space_is_not_under_test(monkeypatch): + # Unpacking needs 3 GB left free; these tests write a few KB and must not depend on how full + # the machine is. test_unpack_needs_free_space_to_remain restores the bound and mocks the disk. + monkeypatch.setattr(field, "MIN_FREE_AFTER_UNPACK", 0) + + @pytest.fixture def case(tmp_path, monkeypatch): monkeypatch.setattr(field, "FIELD_DIR", tmp_path / "field") @@ -119,6 +126,7 @@ def case(tmp_path, monkeypatch): work = tmp_path / "field" / "work" / "synthetic-field" (work / "moge2").mkdir(parents=True) (work / "turns.json").write_text(json.dumps(dict.fromkeys(CAMERAS, 0))) + (work / "capture.json").write_text(json.dumps({"capture": capture})) for kid in CAMERAS: np.savez( work / "moge2" / f"{kid}.npz", @@ -295,7 +303,7 @@ def test_stamp_fills_in_capture_session_and_feet(case, tmp_path): survey["measurements"][1]["value_ft"] = "32 9 3/4" paths["truth"].write_text(json.dumps(survey)) mapping = json.loads(paths["map"].read_text()) - mapping["session"] = "placeholder" + mapping["session"] = field.MAP_SESSION_PLACEHOLDER paths["map"].write_text(json.dumps(mapping)) stamped = field.stamp(archive, paths["truth"], paths["map"], out) survey = json.loads(paths["truth"].read_text()) @@ -307,6 +315,18 @@ def test_stamp_fills_in_capture_session_and_feet(case, tmp_path): assert json.loads(paths["truth"].read_text())["captures"] == [capture] +def test_stamp_refuses_a_map_bound_to_another_session_before_writing(case): + archive, paths, _, out = case + survey_before = paths["truth"].read_text() + mapping = json.loads(paths["map"].read_text()) + mapping["session"] = "some-other-session" + paths["map"].write_text(json.dumps(mapping)) + with pytest.raises(ValueError, match="is the map for session 'some-other-session'"): + field.stamp(archive, paths["truth"], paths["map"], out) + assert paths["truth"].read_text() == survey_before + assert not (out / "inputs" / "map.json").exists() + + @pytest.mark.parametrize("text", ["30 2 5", "30 1/4 2", "1 1/0", "1/2", "30 -2"]) def test_parse_tape_refuses_partial_or_misordered_readings(text): # "30 2 5" would drop the 5; "30 1/4 2" would read the fraction as inches then drop it. @@ -319,3 +339,315 @@ def test_safe_id_refuses_path_components(bad): with pytest.raises(ValueError): field.safe_id(bad, "session id") assert field.safe_id("synthetic-field_1.2", "session id") == "synthetic-field_1.2" + + +def test_session_file_stays_inside_the_session(tmp_path): + (tmp_path / "keyframes").mkdir() + (tmp_path / "keyframes" / "k.jpg").write_bytes(b"x") + assert ( + field.session_file(tmp_path, "keyframes/k.jpg") + == (tmp_path / "keyframes" / "k.jpg").resolve() + ) + for bad in ["/etc/hosts", "../outside.jpg", "keyframes/../../outside.jpg"]: + with pytest.raises(ValueError): + field.session_file(tmp_path, bad) + + +def test_score_refuses_predictions_prepared_for_another_capture(case): + archive, paths, _, out = case + stamp = field.FIELD_DIR / "work" / "synthetic-field" / "capture.json" + stamp.write_text(json.dumps({"capture": "0" * 64})) # same session id, different archive + with pytest.raises(ValueError, match="prepared for another capture"): + field.score(archive, paths["truth"], paths["map"], paths["rules"], out) + + +def test_score_refuses_a_map_for_another_session(case): + archive, paths, _, out = case + mapping = json.loads(paths["map"].read_text()) + mapping["session"] = "some-other-session" + paths["map"].write_text(json.dumps(mapping)) + with pytest.raises(ValueError, match="map is for session"): + field.score(archive, paths["truth"], paths["map"], paths["rules"], out) + + +def test_a_failed_tape_reference_fails_only_the_tape_row(case, monkeypatch): + archive, paths, _, out = case + + def no_scale(*args, **kwargs): + raise ValueError("no model depth at a reference tap") + + monkeypatch.setattr(field, "tape_scale", no_scale) + report = field.score(archive, paths["truth"], paths["map"], paths["rules"], out) + assert "every value failed" in report + tape_row = json.loads((out / "moge2-tape.json").read_text()) + assert all(m["value_ft"] is None for m in tape_row["measurements"]) + native = json.loads((out / "moge2.json").read_text()) + assert any(m["value_ft"] is not None for m in native["measurements"]) + + +def _zip(path, members): + with zipfile.ZipFile(path, "w") as z: + for name, data in members: + z.writestr(name, data) + return path + + +SESSION_JSON = json.dumps({"format": "measure-lab-session", "formatVersion": 2}) + + +@pytest.mark.parametrize( + "name", + ["../session.json", "/session.json", "a/../../session.json", "C:/session.json", "a\\b.jpg"], +) +def test_unpack_refuses_member_paths_that_leave_the_folder(tmp_path, monkeypatch, name): + monkeypatch.setattr(field, "FIELD_DIR", tmp_path / "field") + archive = _zip(tmp_path / "s.zip", [("s/session.json", SESSION_JSON), (name, "x")]) + with pytest.raises(ValueError, match="not a plain relative path"): + field.unpack(archive) + assert not (tmp_path / "session.json").exists() + assert not (tmp_path / "field").exists() # refused before anything was written + + +def test_unpack_refuses_duplicates_and_too_many_or_too_large_members(tmp_path, monkeypatch): + monkeypatch.setattr(field, "FIELD_DIR", tmp_path / "field") + dup = tmp_path / "dup.zip" + # "s/./a.jpg" names the same file as "s/a.jpg" once normalised. + _zip(dup, [("s/session.json", SESSION_JSON), ("s/a.jpg", "1"), ("s/./a.jpg", "2")]) + with pytest.raises(ValueError, match="appears twice"): + field.unpack(dup) + monkeypatch.setattr(field, "MAX_ZIP_MEMBERS", 2) + many = _zip( + tmp_path / "many.zip", [("s/session.json", SESSION_JSON), ("s/a", "1"), ("s/b", "2")] + ) + with pytest.raises(ValueError, match="more than 2"): + field.unpack(many) + monkeypatch.setattr(field, "MAX_ZIP_MEMBERS", 10) + monkeypatch.setattr(field, "MAX_ZIP_BYTES", 100) + big = _zip(tmp_path / "big.zip", [("s/session.json", SESSION_JSON), ("s/a", "0" * 200)]) + with pytest.raises(ValueError, match="more than 100"): + field.unpack(big) + + +def test_unpack_counts_bytes_even_when_a_header_understates_them(tmp_path, monkeypatch): + monkeypatch.setattr(field, "FIELD_DIR", tmp_path / "field") + monkeypatch.setattr(field, "MAX_ZIP_BYTES", 100) + archive = _zip(tmp_path / "s.zip", [("s/session.json", SESSION_JSON), ("s/a", "0" * 200)]) + real = zipfile.ZipFile.infolist + + def understated(self): + infos = real(self) + for info in infos: + info.file_size = 1 # as a crafted header would claim + return infos + + monkeypatch.setattr(zipfile.ZipFile, "infolist", understated) + with pytest.raises((ValueError, zipfile.BadZipFile)): + field.unpack(archive) + assert not list((tmp_path / "field").glob("*/s/a")) # nothing left half-written + + +def test_unpack_needs_free_space_to_remain(tmp_path, monkeypatch): + from collections import namedtuple + + monkeypatch.setattr(field, "FIELD_DIR", tmp_path / "field") + monkeypatch.setattr(field, "MIN_FREE_AFTER_UNPACK", 3 * 1024**3) + Usage = namedtuple("Usage", "total used free") + monkeypatch.setattr(field.shutil, "disk_usage", lambda p: Usage(0, 0, 1024**3)) + archive = _zip(tmp_path / "s.zip", [("s/session.json", SESSION_JSON)]) + with pytest.raises(ValueError, match="less than 3 GB free"): + field.unpack(archive) + + +def test_unpack_extracts_a_plain_session(tmp_path, monkeypatch): + monkeypatch.setattr(field, "FIELD_DIR", tmp_path / "field") + archive = _zip(tmp_path / "s.zip", [("s/session.json", SESSION_JSON), ("s/k/a.jpg", "x")]) + folder, digest = field.unpack(archive) + assert folder.is_relative_to((tmp_path / "field").resolve()) + assert (folder / "k" / "a.jpg").read_text() == "x" + assert field.unpack(archive) == (folder, digest) # a second call reuses the folder + + +def test_unpack_refuses_an_oversized_archive_before_reading_it(tmp_path, monkeypatch): + monkeypatch.setattr(field, "FIELD_DIR", tmp_path / "field") + monkeypatch.setattr(field, "MAX_ZIP_ARCHIVE_BYTES", 100) + archive = _zip(tmp_path / "s.zip", [("s/session.json", SESSION_JSON), ("s/a", "0" * 500)]) + + def no_read(*args, **kwargs): + raise AssertionError("an oversized archive was hashed or opened") + + monkeypatch.setattr(field, "file_sha256", no_read) + monkeypatch.setattr(field.zipfile, "ZipFile", no_read) + with pytest.raises(ValueError, match="more than 100"): + field.unpack(archive) + + +def test_unpack_counts_every_zip_entry_including_skipped_ones(tmp_path, monkeypatch): + monkeypatch.setattr(field, "FIELD_DIR", tmp_path / "field") + monkeypatch.setattr(field, "MAX_ZIP_ENTRIES", 3) + entries = [("s/session.json", SESSION_JSON)] + [(f"__MACOSX/{i}", "") for i in range(3)] + archive = _zip(tmp_path / "s.zip", entries) + with pytest.raises(ValueError, match="4 zip entries, more than 3"): + field.unpack(archive) + + +def test_file_sha256_streams_and_matches_hashlib(tmp_path): + path = tmp_path / "f" + path.write_bytes(b"x" * (3 << 20) + b"tail") # spans several 1 MB reads + assert field.file_sha256(path) == hashlib.sha256(path.read_bytes()).hexdigest() + + +def test_zip_directory_limits_apply_before_zipfile_reads_the_directory(tmp_path, monkeypatch): + monkeypatch.setattr(field, "FIELD_DIR", tmp_path / "field") + entries = [("s/session.json", SESSION_JSON)] + [(f"s/{i:03d}", "") for i in range(20)] + archive = _zip(tmp_path / "s.zip", entries) + + def no_zipfile(*args, **kwargs): + raise AssertionError("ZipFile read the directory before the limits were checked") + + monkeypatch.setattr(field.zipfile, "ZipFile", no_zipfile) + monkeypatch.setattr(field, "MAX_ZIP_ENTRIES", 10) + with pytest.raises(ValueError, match="21 zip entries, more than 10"): + field.unpack(archive) + monkeypatch.setattr(field, "MAX_ZIP_ENTRIES", 100) + monkeypatch.setattr(field, "MAX_ZIP_DIRECTORY_BYTES", 200) + with pytest.raises(ValueError, match="zip directory, more than 200"): + field.unpack(archive) + + +def test_zip_directory_limits_read_zip64_end_records(tmp_path, monkeypatch): + archive = tmp_path / "z64.zip" + # Writing more entries than this limit makes zipfile add a ZIP64 end record, as a very large + # session would, without writing 65,536 files. + monkeypatch.setattr(zipfile, "ZIP_FILECOUNT_LIMIT", 1) + _zip(archive, [("s/session.json", SESSION_JSON), ("s/a", "x"), ("s/b", "y")]) + with archive.open("rb") as f: + assert zipfile._EndRecData(f)[zipfile._ECD_SIGNATURE] == zipfile.stringEndArchive64 + field.check_zip_directory(archive) # a well-formed ZIP64 archive passes + monkeypatch.setattr(field, "MAX_ZIP_ENTRIES", 2) + with pytest.raises(ValueError, match="3 zip entries, more than 2"): + field.check_zip_directory(archive) + + +def test_a_file_that_is_not_a_zip_is_refused_loudly(tmp_path): + path = tmp_path / "s.zip" + path.write_bytes(b"not a zip at all") + with pytest.raises(ValueError, match="not a zip file"): + field.check_zip_directory(path) + + +def test_a_reported_multi_gigabyte_archive_is_refused_from_its_size_alone(tmp_path, monkeypatch): + import os + from pathlib import Path + + monkeypatch.setattr(field, "FIELD_DIR", tmp_path / "field") + archive = _zip(tmp_path / "s.zip", [("s/session.json", SESSION_JSON)]) + real_stat = Path.stat + + def huge(self, *args, **kwargs): + st = real_stat(self, *args, **kwargs) + if self == archive: + fields = list(st) + fields[6] = 5 * 1024**3 # st_size: a 5 GiB file, without writing one + return os.stat_result(fields) + return st + + def no_read(*args, **kwargs): + raise AssertionError("the archive was read or opened") + + monkeypatch.setattr(Path, "stat", huge) + monkeypatch.setattr(field, "check_zip_directory", no_read) + monkeypatch.setattr(field, "file_sha256", no_read) + monkeypatch.setattr(field.zipfile, "ZipFile", no_read) + with pytest.raises(ValueError, match="more than 2147483648"): + field.unpack(archive) + + +def _jpeg_claiming(width: int, height: int) -> bytes: + """A few dozen bytes: SOI, an APP0 segment to skip, then a baseline frame header declaring + `width` x `height`, with no pixel data behind it.""" + app0 = b"\xff\xe0" + (16).to_bytes(2, "big") + b"JFIF\x00" + bytes(9) + sof0 = ( + b"\xff\xc0" + + (17).to_bytes(2, "big") + + b"\x08" + + height.to_bytes(2, "big") + + width.to_bytes(2, "big") + + b"\x03" + + bytes(9) + ) + return b"\xff\xd8" + app0 + sof0 + b"\xff\xd9" + + +def test_image_size_reads_real_and_crafted_headers(tmp_path): + real = tmp_path / "real.jpg" + cv2.imwrite(str(real), np.zeros((48, 64, 3), np.uint8)) + assert field.image_size(real) == (64, 48) + png = tmp_path / "real.png" + cv2.imwrite(str(png), np.zeros((5, 7), np.uint8)) + assert field.image_size(png) == (7, 5) + crafted = tmp_path / "bomb.jpg" + crafted.write_bytes(_jpeg_claiming(65535, 65535)) + assert field.image_size(crafted) == (65535, 65535) + + +def test_an_image_whose_header_claims_too_many_pixels_is_never_decoded(tmp_path, monkeypatch): + bomb = tmp_path / "bomb.jpg" + bomb.write_bytes(_jpeg_claiming(65535, 65535)) # 4.3 gigapixels claimed in 60 bytes + + def no_decode(*args, **kwargs): + raise AssertionError("an oversized image was decoded") + + monkeypatch.setattr(field.cv2, "imread", no_decode) + with pytest.raises(ValueError, match="more than 50000000"): + field.read_session_image(bomb) + other = tmp_path / "x.gif" + other.write_bytes(b"GIF89a" + bytes(20)) + with pytest.raises(ValueError, match="not a JPEG or PNG"): + field.read_session_image(other) + + +def test_a_session_image_within_the_bound_decodes(tmp_path): + path = tmp_path / "k.jpg" + cv2.imwrite(str(path), np.full((30, 40, 3), 200, np.uint8)) + assert field.read_session_image(path).shape == (30, 40, 3) + assert field.read_session_image(path, cv2.IMREAD_GRAYSCALE).shape == (30, 40) + + +def test_session_memory_follows_one_group_not_the_session(): + loads, live_peak = [], 0 + frames = field.GroupFrames(lambda k: loads.append(k) or (k,), lambda k: 1_000_000) + groups = [[f"k{i}", f"k{i + 1}", f"k{i + 2}"] for i in range(50)] # a 52-keyframe walk + for group in groups: + held = frames.hold(group) + assert set(held) == set(group) + live_peak = max(live_peak, len(frames.held)) + assert live_peak == 3 # never more than one group, however long the session + assert len(loads) == 52 # overlapping frames are kept, not decoded again + + +def test_a_group_over_the_pixel_budget_is_refused_before_decoding(monkeypatch): + def no_decode(k): + raise AssertionError("a frame was decoded") + + # Eight 48 MP photos, sized from their headers only: 390 MP against a 200 MP budget. + frames = field.GroupFrames(no_decode, lambda k: 8064 * 6048) + with pytest.raises(ValueError, match="more than 200000000"): + frames.hold([f"k{i}" for i in range(8)]) + small = field.GroupFrames(lambda k: (k,), lambda k: 1920 * 1440) # eight ARKit keyframes + assert len(small.hold([f"k{i}" for i in range(8)])) == 8 + + +def test_stamp_refuses_a_survey_already_bound_to_another_capture(case): + archive, paths, capture, out = case + survey = json.loads(paths["truth"].read_text()) + survey["captures"] = ["0" * 64] # the survey from another walk + survey["measurements"][1]["value_ft"] = "32 9 3/4" + paths["truth"].write_text(json.dumps(survey)) + before = paths["truth"].read_text() + with pytest.raises(ValueError, match=f"already the survey for capture {'0' * 64}"): + field.stamp(archive, paths["truth"], paths["map"], out) + assert paths["truth"].read_text() == before # nothing converted or appended + assert not (out / "inputs" / "map.json").exists() + survey["captures"] = [capture] # its own capture, stamped before: accepted again + paths["truth"].write_text(json.dumps(survey)) + field.stamp(archive, paths["truth"], paths["map"], out) diff --git a/experiments/evals/tests/test_review_fixes.py b/experiments/evals/tests/test_review_fixes.py new file mode 100644 index 000000000..8312f5962 --- /dev/null +++ b/experiments/evals/tests/test_review_fixes.py @@ -0,0 +1,339 @@ +"""Regressions for review findings on PR #12: each case has one right answer.""" + +import json +from pathlib import Path + +import numpy as np +import pytest + +from evals import coverage_options, replay +from evals.coverage import PhotoTruth, photo_truth +from evals.eth3d import View, read_ply_xyz +from evals.map3d import span_columns, within_bar +from evals.recon import comparable_groups + + +def test_fractional_replay_bounds_are_refused(): + with pytest.raises(ValueError, match="whole seconds"): + replay.replay_identity(20, 40.1, 75.1) + assert replay.replay_identity(20, 40, 75)[0] == "advio-20-0040-0075" + + +def test_a_replay_from_other_inputs_is_not_deleted(tmp_path: Path): + folder = tmp_path / "advio-20-0040-0075" + folder.mkdir() + (folder / "session.json").write_text( + json.dumps({"replaySource": {"sequence": 20, "start": 40, "end": 76}}) + ) + with pytest.raises(ValueError, match="not built from"): + replay.clear_replay(folder, {"sequence": 20, "start": 40, "end": 75}) + assert folder.exists() + replay.clear_replay(folder, {"sequence": 20, "start": 40, "end": 76}) + assert not folder.exists() + + +def test_truncated_ply_header_fails(tmp_path: Path): + path = tmp_path / "scan.ply" + path.write_bytes(b"ply\nformat binary_little_endian 1.0\nelement vertex 3\n") + with pytest.raises(ValueError, match="end_header"): + read_ply_xyz(path) + + +def _view() -> View: + K = np.array([[100.0, 0, 49.5], [0, 100.0, 29.5], [0, 0, 1]]) + return View("v", None, 100, 60, K, np.eye(3), np.zeros(3)) + + +def test_an_empty_scan_pixel_is_unknown_not_seen(): + empty = np.full((6, 10), 1e6, np.float32) # no scan point anywhere + t = photo_truth( + _view(), np.eye(3), empty, np.zeros((6, 10), bool), np.array([[0.0, 0, 3]]), True, 6.0 + ) + assert t.framed[0] and t.no_scan[0] and not t.hidden[0] + assert not t.saw[0] + scanned = np.full((6, 10), 3.0, np.float32) + t = photo_truth( + _view(), np.eye(3), scanned, np.zeros((6, 10), bool), np.array([[0.0, 0, 3]]), True, 6.0 + ) + assert t.saw[0] + + +def test_the_depth_test_option_rejects_rows_without_depth(monkeypatch): + class Wall: + pass + + class Setup: # the fields depth_hidden reads + cfg = { # noqa: RUF012 + "rowsPerBand": 3, + "maxDistance": 6.0, + "cellWidth": 0.1524, + "wallBandHeight": 1.98, + "groundBandDepth": 1.2, + } + views = (_view(),) + wall, R = Wall(), np.eye(3) + zbufs, missing = (None,), (None,) + + unknown = PhotoTruth(*(np.array([[x, x]]) for x in (True, True, True, False, True))) + monkeypatch.setattr(coverage_options, "row_points", lambda *a: np.zeros((3, 2, 3))) + monkeypatch.setattr(coverage_options, "photo_truth", lambda *a: unknown) + hidden = coverage_options.depth_hidden( + Setup(), [{"keyframe": "v", "band": "wall", "index": 0, "rows": [0]}], 3, None, 0.1 + ) + assert hidden and hidden[0]["rows"] == [0] + + +@pytest.mark.parametrize("cells", [1, 2, 4]) +def test_span_columns_tile_whole_half_foot_cells(cells): + # 0.1524 m cells leave a partial 2 cm strip at the end (0.0024 m for 1 cell, 0.0048 m for 2); + # the widths must still add up to the span, and every centre must lie inside it. + hi = cells * 0.1524 + cols, w = span_columns(0.0, hi) + assert w.sum() == pytest.approx(hi, abs=1e-12) + assert (w > 0).all() and (w <= 0.02 + 1e-12).all() + assert cols[0] > 0 and cols[-1] < hi + + +def test_span_columns_on_a_foot_keep_the_last_strip(): + cols, w = span_columns(0.0, 0.3048) + assert len(cols) == 16 and w[-1] == pytest.approx(0.0048) + cols, w = span_columns(0.0, 0.30) # a whole number of columns gets no sliver + assert len(cols) == 15 and w[-1] == pytest.approx(0.02) + + +def test_a_claim_of_exactly_the_bar_passes(): + _, w = span_columns(0.0, 0.1524) # one 0.5 ft cell, all of it unseen + assert within_bar(w.sum() / 0.3048) + _, w = span_columns(0.0, 0.1544) + assert not within_bar(w.sum() / 0.3048) + + +def test_comparable_groups_keep_one_seed_population(): + groups = { + "1": [["a"], ["b"], ["c"]], + "2": [["a", "x"], ["b", "x"], ["c", "x"]], + "8": [["a"] * 8, ["b"] * 8], # seed c lacks neighbours for eight photos + } + out = comparable_groups(groups, [1, 2, 8]) + assert {n: [m[0] for m in g] for n, g in out.items()} == { + "1": ["a", "b"], + "2": ["a", "b"], + "8": ["a", "b"], + } + + +def test_fit_key_changes_with_each_image_and_depth_file(tmp_path: Path): + import os + + from evals.pose_priors import fit_inputs + + T = {"a": np.eye(4), "b": np.eye(4)} + K = {"a": np.eye(3), "b": np.eye(3)} + files = {} + for kind in ("image", "depth"): + for m in "ab": + files[kind, m] = tmp_path / f"{kind}-{m}" + files[kind, m].write_bytes(f"{kind} {m}".encode()) + + def key(): + return fit_inputs( + T, K, 2.0, {m: files["image", m] for m in "ab"}, {m: files["depth", m] for m in "ab"} + ) + + base = key() + assert key() == base + seen = {base} + for kind in ("image", "depth"): + path = files[kind, "b"] + path.write_bytes(path.read_bytes() + b" changed") + # A new size is enough on its own, but move the clock too, as a rewrite would. + st = path.stat() + os.utime(path, ns=(st.st_atime_ns, st.st_mtime_ns + 1_000_000)) + changed = key() + assert changed not in seen # each file changing alone gives a new key + seen.add(changed) + + +def _strict_loads(text: str): + def refuse(token): + raise ValueError(f"{token} is not JSON") + + return json.loads(text, parse_constant=refuse) + + +def test_failed_percentiles_serialise_as_strict_json(): + from evals.pairs import results_json, summarize + + errors = np.r_[np.full(8, 0.01), np.full(2, np.inf)] # 20% failed: the p90 is a failure + row = summarize(errors) + assert row["p90_in"] == float("inf") + doc = _strict_loads(results_json({"row": row, "empty": float("nan"), "ok": [1.5]})) + assert doc["row"]["p90_in"] == "failed" + assert doc["row"]["failed_pct"] == 20.0 + assert doc["empty"] is None and doc["ok"] == [1.5] + with pytest.raises(ValueError, match="-inf"): + results_json({"x": float("-inf")}) + + +@pytest.mark.parametrize( + "path", sorted((Path(__file__).parents[1] / "results").glob("*.json")), ids=lambda p: p.name +) +def test_committed_results_are_strict_json(path: Path): + _strict_loads(path.read_text()) + + +def test_marvin_scores_only_pinned_walks_with_pinned_content(tmp_path: Path): + import hashlib + + from evals import modern_arkit + + files = {"bar/seq2/ARkitPose.txt": b"two", "bar/seq10/ARkitPose.txt": b"ten"} + for rel, data in files.items(): + (tmp_path / rel).parent.mkdir(parents=True, exist_ok=True) + (tmp_path / rel).write_bytes(data) + stale = tmp_path / "bar/seq3/ARkitPose.txt" # on disk but not pinned: never read + stale.parent.mkdir(parents=True) + stale.write_bytes(b"stale") + pinned = {rel: hashlib.sha256(data).hexdigest() for rel, data in files.items()} + assert modern_arkit.pinned_walks(pinned, "bar") == ["seq2", "seq10"] + modern_arkit.verify(pinned, tmp_path) + (tmp_path / "bar/seq10/ARkitPose.txt").write_bytes(b"edited") + with pytest.raises(RuntimeError, match="missing or differ"): + modern_arkit.verify(pinned, tmp_path) + (tmp_path / "bar/seq10/ARkitPose.txt").unlink() + with pytest.raises(RuntimeError, match="missing or differ"): + modern_arkit.verify(pinned, tmp_path) + + +def test_marvin_manifest_covers_every_scored_scene(): + from evals import modern_arkit + + pinned = modern_arkit.manifest() + for scene in modern_arkit.SCENES: + assert {f"{scene}/train.txt", f"{scene}/test.txt"} <= set(pinned) + assert modern_arkit.pinned_walks(pinned, scene) + + +def _archive(tmp_path: Path, members=("d/a.csv", "d/sub/")): + from evals.datasets import Archive + + return Archive("https://example.org/x.zip", 1, "f" * 64, tmp_path, "d/a.csv", members) + + +def test_an_interrupted_unpack_is_not_complete(tmp_path: Path): + from evals import datasets + + archive = _archive(tmp_path) + (tmp_path / "d").mkdir() + (tmp_path / "d/a.csv").write_text("a") # the marker alone, as an interrupted unpack leaves it + assert not datasets.is_complete(archive) + with pytest.raises(SystemExit, match="d/sub/"): + datasets.write_record(archive, ["d/a.csv"]) + assert not datasets.is_complete(archive) + + +def test_a_recorded_unpack_is_complete_until_a_file_goes_missing_or_changes_size(tmp_path: Path): + from evals import datasets + + archive = _archive(tmp_path) + (tmp_path / "d/sub").mkdir(parents=True) + (tmp_path / "d/a.csv").write_text("a") + (tmp_path / "d/sub/b.csv").write_text("bb") + datasets.write_record(archive, ["d/", "d/a.csv", "d/sub/", "d/sub/b.csv"]) + assert datasets.is_complete(archive) + other = datasets.Archive(archive.url, 1, "e" * 64, tmp_path, archive.marker, archive.members) + assert not datasets.is_complete(other) # a record for another archive hash does not count + (tmp_path / "d/sub/b.csv").write_text("b") # truncated + assert not datasets.is_complete(archive) + (tmp_path / "d/sub/b.csv").unlink() + assert not datasets.is_complete(archive) + + +def _run(root: Path, gid: str, poses_sha256=None): + (root / gid).mkdir(parents=True) + doc = {"members": [gid]} + if poses_sha256 is not None: + doc["poses_sha256"] = poses_sha256 + (root / gid / "run.json").write_text(json.dumps(doc)) + + +def test_mapanything_rows_are_labelled_by_the_poses_they_recorded(tmp_path: Path): + import hashlib + + from evals.pose_priors import HISTORICAL_MAPANYTHING, mapanything_label, uses_historical_rows + + poses = tmp_path / "advio_2018.json" + poses.write_text('{"g": 1}') + current = hashlib.sha256(poses.read_bytes()).hexdigest() + # The published runs record no pose hash: they keep the historical label and its note. + historical = tmp_path / "historical" + _run(historical, "n2-a") + label = mapanything_label(historical, "advio_2018", poses) + assert label == HISTORICAL_MAPANYTHING["advio_2018"] + assert uses_historical_rows({"electro": {"all": {f"mapanything@392 + {label}": {}}}}) + # A fresh run on the current poses is labelled by them, with no historical note. + fresh = tmp_path / "fresh" + _run(fresh, "n2-a", current) + _run(fresh, "n4-a", current) + label = mapanything_label(fresh, "advio_2018", poses) + assert label == "advio_2018 poses" + assert not uses_historical_rows({"electro": {"all": {f"mapanything@392 + {label}": {}}}}) + # Recorded runs on other poses, or a mix, are refused rather than mislabelled. + mixed = tmp_path / "mixed" + _run(mixed, "n2-a", current) + _run(mixed, "n4-a") + with pytest.raises(SystemExit, match="rerun"): + mapanything_label(mixed, "advio_2018", poses) + other = tmp_path / "other" + _run(other, "n2-a", "0" * 64) + with pytest.raises(SystemExit, match="rerun"): + mapanything_label(other, "advio_2018", poses) + + +def test_an_option_that_claims_no_wall_is_untested_not_passed(): + from evals.coverage_options import _row, markdown + + bands = { + "wall": {"claimed_ft": 0.0, "false_observed_ft": 0.0, "missed_ft": 18.0}, + "ground": {"claimed_ft": 0.0, "false_observed_ft": 0.0}, + } + nothing = _row("b = 0.25 m, theta = 75 deg", bands, lidar=False) + assert nothing["passes"] is None + claimed = {**bands, "wall": {"claimed_ft": 19.3, "false_observed_ft": 1.1, "missed_ft": 0.0}} + assert _row("the app today", claimed, lidar=False)["passes"] is False + res = { + "scene": "s", + "wall_ft": 19.3, + "grid": [], + "options": [{**nothing, "option": "claims nothing"}], + "relief": [], + } + assert "| claims nothing | no | 0.0 | untested | untested | 18.0 | 0.0 |" in markdown(res) + + +def test_a_partial_multi_view_run_is_refused_naming_the_missing_groups(): + from evals.recon import PartialRun, require_complete + + require_complete("electro", "m", "4", [], 7) # complete + require_complete("electro", "m", "8", [f"n8-{i}" for i in range(7)], 7) # never run: omitted + with pytest.raises(PartialRun, match=r"no output for 2 of 7 4-photo groups \(n4-a, n4-b\)"): + require_complete("electro", "m", "4", ["n4-a", "n4-b"], 7) + + +def test_score_groups_refuses_a_run_missing_one_group(monkeypatch): + from types import SimpleNamespace + + from evals import recon + + scene = SimpleNamespace( + name="electro", groups={"1": [["a"], ["b"]], "2": [["a", "b"], ["b", "a"]]} + ) + + monkeypatch.setattr(recon, "EvalSet", lambda scene, seed, max_range: seed) + monkeypatch.setattr(recon, "_evaluate_into", lambda *args: None) + + def factory(gid, members): + return None if gid == "n2-b" else object() # the run stopped before group n2-b + + with pytest.raises(recon.PartialRun, match="n2-b"): + recon.score_groups(scene, {"model": factory}, sizes=(1, 2))