fix(optics): FOV gate width follows lens confidence, plus lens self-heal (ADR 0029)

[brickbots]

Some rev4 PiFinders shipped with a 12 mm lens and no camera_lens in their config. 2.6.2's derived FOV gate falls back to the imx462's default_lens_key of 16mm, derives 10.40°, and gates [8.84, 11.96] — against frames that are actually 13.51° wide. Those units stopped solving on update having changed nothing, and the symptom presents as an exposure problem rather than a configuration one.

The pre-0027 window was the fixed 12.0 ± 4.0 = [8.0, 16.0], which happened to span the imx462 at both 12 mm and 16 mm. That is the whole reason these units worked on 2.6.1 and not on 2.6.2.

Note

#625 (lens-menu-restart) has merged. This branch was stacked on it; it is now rebased onto main with #625's commit dropped, and no longer carries it. The rebase was clean and the resulting tree is bit-identical to the pre-rebase branch, so nothing in this PR's content changed.

The fix

A fallback is not a claim, and it should never have bought a claim's confidence. Gate width now follows how the lens was obtained:

the lens is…	gate
stated — config names a lens we recognise	±15% of that lens's field of view, exactly as ADR 0027 had it
assumed — nobody has said	the union of every lens that sensor has shipped with

sensor	shipped lenses	assumed gate	stated-16mm gate
imx462 / imx290	12 mm (13.51°), 16 mm (10.40°)	12.19 ± 3.35 = [8.84, 15.53]	[8.84, 11.96]
imx296	12 mm (17.78°), 16 mm (13.71°)	16.05 ± 4.39 = [11.65, 20.44]	[11.65, 15.77]
hq	25 mm (10.33°)	[8.78, 11.88] — unchanged	n/a

The imx462's assumed gate lands within a whisker of the pre-0027 constants it replaces, which is the behaviour these units are known to work under.

Then self-heal closes the assumption. A successful solve reports the fitted FOV and the sensor is already known, so what is left is the lens. Three consecutive identifications agreeing to within 5% promote the assumption to a statement, and the gate narrows onto it. It:

• never overwrites a stated lens — the user's claim stays authoritative, so ADR 0027's "a mis-stated lens means no solves" is untouched, and the write happens at most once per device;
• writes nothing when the measurement matches no shipped lens — third-party glass keeps the wide gate, which is honest;
• skips replayed solves — a recorded session measures whatever device recorded it;
• can never take the integrator down — a read-only filesystem or a dropped manager connection costs the promotion, not the pointing.

Why a union rather than dropping the hint

Measured against the real solver and real frames before committing to this: widening costs nothing on speed. ±15%, ±30%, ±50% and the pre-0027 fixed window all solve in 1.1–1.4 ms, and even no hint at all costs only +0.3 ms; Prob was bit-identical across every width.

But the gate does buy mis-solve rejection. With injected noise, wide and hintless gates returned confident false solves at 20–23° that match_threshold did not reject. The assumed gate still excludes everything above 15.53°, so it keeps that protection — omitting fov_estimate would give it up. (Single-shot noise trials: direction established, magnitude not.)

And the cost is wildly asymmetric. A good frame outside the gate burns the entire solve_timeout and returns nothing, every frame, forever. Against a wide gate's ~0.3 ms, an over-tight gate is the expensive failure by three orders of magnitude.

Rejected

• A rev4 → 12 mm config migration. Rev4 reports as imx462 and so do some v3 units, so the sensor is not the predicate. The board is probeable, but board revision is not the fact we need — which lens was in the box is, and not all rev4s shipped with the 12 mm. It would fix most units by breaking the rest, and break them in exactly the way this PR exists to fix.
• camera_lens in default_config.json. Config.get_option consults the default file before resolve_lens ever sees a None, so hq would derive 17.12° instead of its true 10.33°.
• Raising FOV_GATE_MARGIN globally. Spends 0027's entire benefit on every correctly-configured user to cover a case they are not in.

Full reasoning and the numbers: docs/adr/0029-fov-gate-width-follows-lens-confidence.md.

The hq must not move

The hq only ever shipped one lens, so it has no union to take. That case returns early rather than falling through the union arithmetic — which re-derives the same window through different float operations and lands 1 ULP off the value that has been shipping. test_the_hq_gate_is_bit_identical_to_what_shipped asserts the exact floats, not approx.

Verification

• 1,285 unit + smoke tests pass, 0 failures. Ruff lint + format clean, mypy clean over 151 files.
• 115 new test functions across test_optics.py (63) and the new test_lens_self_heal.py (19), plus the existing suites.
• Sphinx builds with zero warnings (-n -q), and the new prose carries no em-dashes or semicolons.
• Self-heal verified end to end on the running app (--camera debug), which exercises the write path through the integrator wiring: Lens self-heal: 3 solves fitted 10.20 deg, which is the 25mm (10.33 deg derived). The dev config was backed up and restored to its original checksum afterwards.

Warning

The debug camera declares hq, which has one shipped lens, so its assumed gate is identical to its stated one. The debug run cannot exercise the widening at all — the imx462 12mm-vs-16mm case, which is the entire point of the change, is only reachable in unit tests or on real hardware. A green debug run is not evidence the fix works on an affected unit.

Known limitation, not fixed here

After self-heal writes camera_lens, the UI process's Config still holds the dict it loaded at boot. Any later set_option from the menu rewrites config.json from that stale dict and drops the self-healed key. This is self-correcting — the device re-heals after three solves on the next boot, and shared_state carries the correct lens for the whole session, so solving is never affected. It is also pre-existing: camera_interface.py already writes config from a non-UI process with the same exposure. Worth a follow-up issue rather than a wider change to Config in this PR.

Docs

• release_notes/2.6.2.md — the fix, and two corrections it forces: the release no longer claims "no configuration file is rewritten", and the "device deliberately does not recover" sharp edge is restated (self-heal writes into an absence, never over a statement). A new sharp edge covers the post-self-heal physical lens swap.
• docs/source/troubleshooting.rst (Docs: "changed lens → no solves" troubleshooting, and BOM's FOV advice is sensor-specific #613) — the "changed lens → no solves" path, narrower now that self-heal covers the never-stated case.
• docs/source/BOM.rst (Docs: "changed lens → no solves" troubleshooting, and BOM's FOV advice is sensor-specific #613) — "something fast with a 10deg FOV" was imx462-specific and contradicted the parts list two lines above it.
• ADR 0029's imx296 row: [11.66, …] → [11.65, …], a rounding slip. 11.65 is what the release notes already quoted.

Closes #613. Related: #610, #611, #612.

🤖 Generated with Claude Code

[brickbots]

Reviewed this branch. Verified the PR's own claims independently: 1,285 unit + smoke tests pass, ruff check/format clean, mypy clean on the changed modules, and sphinx -n -q builds with zero warnings. The stated/assumed split and the union arithmetic both check out against the ADR table, and the hq early-return with the bit-identical float assertion is the right call.

One defect found and pushed as 5514a39 (now 1,286 passing):

The Lens menu showed the wrong lens after self-heal, and confirming it was unrecoverable.

get_camera_lens resolved the sensor half from shared_state but the lens half from config_object. The UI process loads its Config at boot and reloads only on an explicit reload_config, so after self-heal writes from the integrator the menu still reported the assumed lens. Confirmed against this branch:

after self-heal, on disk : 12mm
Lens menu displays       : 16mm   (shared_state says 12mm)

Not only cosmetic: text_menu writes the highlighted entry on select, so a user confirming that stale 16mm on a healed 12mm unit states the wrong lens, tightens the gate to [8.84, 11.96] against 13.51° frames, and stops solving permanently — and self-heal can't recover it, since it only writes into an absence. The troubleshooting.rst added by this PR sends a user with no solves to exactly that screen.

The fix reads shared_state.camera_lens() instead. Shared state is seeded from config at construction and republished by both set_camera_lens and self-heal, and nothing else writes camera_lens (server.py never touches it), so it is strictly the more current view. Added test_shows_the_self_healed_lens_not_this_process_stale_config, which fails on the old code with ['16mm'] != ['12mm'].

Filed #626 for the general form of the staleness — dump_config rewrites the whole file from a per-process dict, so a cross-process write is dropped by the next UI write. Worth reading before merge for one reason: it means "it writes at most once per device" is not accurate as currently written in ADR 0029 and release_notes/2.6.2.md. A device with an unstated lens re-heals every boot. Harmless (shared state carries the right lens all session) but the claim should be softened. server.py already has the mechanism to fix it properly — ui_queue.put("reload_config") — the integrator just isn't passed ui_queue.

Smaller points, not pushed:

• Self-heal will write a lens identified against a fallback camera profile. resolve_camera_profile silently falls back to imx296 for an unknown sensor, and test_an_unrecognised_sensor_does_not_raise asserts that as desired. If the sensor isn't really an imx296 the identified lens is wrong, and the write is unrecoverable — which contradicts the ADR's own "a wrong write is worse than no write". Near-unreachable in practice (camera_interface sets camera_type before the first capture), but the guard is one line: skip when camera_type not in CAMERA_PROFILES.
• The imx296 assumed gate reaches 20.44°, inside the 20–23° band where the ADR reports wide gates returned confident false solves. The "still excludes everything above 15.53°" argument — and test_the_assumed_gate_still_rejects_a_wild_field_of_view — are imx462-specific. Worth a sentence in the ADR, or an imx296 bound in the test.
• except (OSError, AttributeError, ValueError) is narrower than the docstring's promise: a dropped manager connection usually surfaces as EOFError, which isn't an OSError. And catching AttributeError will silently disable self-heal on a rename rather than failing loudly. A plain except Exception with the same disable-and-warn matches the stated intent and swallows nothing more.
• No test pins LENS_IDENTIFY_TOLERANCE < FOV_GATE_MARGIN, which is what guarantees a mis-identification still solves inside the gate the write produces. Load-bearing, one line.
• LensSelfHeal.__init__ is untyped, so mypy skips its body (integrator.py:116: note: bodies of untyped functions are not checked). _write(..., profile) likewise.
• docs/source/menu_map.rst adds an em-dash ("what is fitted — and naming the wrong one"), which the published-docs house style replaces; the PR body says the new prose carries none. ADR/CONTEXT em-dashes are fine. BOM.rst's addition also ends without a full stop and writes 12mm where the release notes use 12 mm.

Nothing else blocking from me.