feat(catalogs): add the Perek-Kohoutek planetary nebula catalog - #617
Open
mrosseel wants to merge 414 commits into
Open
feat(catalogs): add the Perek-Kohoutek planetary nebula catalog#617mrosseel wants to merge 414 commits into
mrosseel wants to merge 414 commits into
Conversation
…cceed) Brickbots/PiFinder runs the workflow on pull_request:synchronize from PR brickbots#379 (mrosseel:nixos -> brickbots:main). With ATTIC_TOKEN now also set on brickbots, build-emulated's 'Push to Attic' step succeeds — the last failing step. But stamp-build then tries to checkout the PR head ref (mrosseel:nixos) and 'git push' a pifinder-build.json commit there, which can't work from brickbots' Actions runner (no write access to the fork). The PR run therefore failed at the stamp step even after attic was wired correctly. Gating stamp-build on github.event_name == 'push' keeps the canonical stamp on the mrosseel:nixos push run (where it works) and skips it on brickbots PR runs (which only need to verify the build). Net effect: both repos' CI runs in parallel without stomping — - Both build and push the same closure to cache.pifinder.eu (attic FastCDC-dedups, so the second push is a no-op), - Only mrosseel stamps pifinder-build.json, - build-migration-tarball already gates on github.ref == refs/heads/nixos so it only runs on mrosseel push. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sync the fork with upstream up to 0e0ec03 (39 commits since the last sync at e4b623a): typed positioning model (PointingEstimate / ImuSample / AlignedResult), polar alignment, comet vectorization, single-instance lock, resolution-flexible UI, i18n fonts. Conflict resolution (18 files): - Adopt upstream's typed positioning model across solver, integrator, state, imu_pi, status, main, base, console, object_details, fonts, menu_structure, auto_exposure, camera_interface. - Keep NixOS layers: software.py (store-path upgrade UI), utils.py (build_json, writable comet_file, robust pifinder_dir), sys_utils.py. - plot.py: restore top-level `import pandas` (upstream added module-level uses; the fork had made it lazy) and drop the redundant lazy imports. - solver.py taken verbatim from upstream; the cedar-detect dev-spawn is proposed upstream separately (brickbots#478). - Drop fork-deleted nox/pip tooling (requirements.txt, version.txt). deps: add xlrd to python-packages.nix (pyerfa already present). Verification: ruff clean; 479 unit tests pass. Remaining failures are not from this merge -- 6 test_software + 4 test_t9_search pre-exist on origin/nixos; 5 test_comets fail on skyfield 1.53 (upstream vectorization, has a runtime fallback). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Skyfield's propagate() lays a batched Kepler orbit out as (3, #orbits, #times) but sets output_shape = (3,) + t1.shape, so a batched orbit propagated to a single scalar time raises "cannot reshape array of size 3N into shape (3,)" on skyfield >= 1.46. (The fork uses nixpkgs' skyfield 1.53; upstream pins 1.45, which tolerated it.) Give every comet the same target time as an (N, 1) column so output_shape matches the (3, N, 1) result, then squeeze the time axis. Verified against the per-comet path (0 AU difference) and the existing tests/test_comets.py oracle (7/7 pass). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…che) The upstream merge auto-merged catalogs.py entirely to the fork's side, silently dropping upstream's T9 search (brickbots#464: KEYPAD_DIGIT_TO_CHARS, search_by_t9) and the catalog disk cache; test_t9_search (an upstream test) failed as a result. Take upstream's catalogs.py wholesale, matching the upstream main.py and menu_structure the merge already adopted. Trade- off: drops the fork's priority-fast-path / background-loader in favour of upstream's cache + loader (the agreed "take upstream catalogs" choice). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
MIN_NIXOS_VERSION was intentionally bumped 2.5.0 -> 3.0.0 (d705057, prep for 3.0), but test_software still asserted 2.5.x/2.6.x as qualifying. Shift the qualifying mock releases and _meets_min_version cases into the 3.x line so the suite matches the current minimum; below-min (2.4.0) and draft cases are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
main.py created (and server.py switched) the pifinder_logconf.json symlink relative to the cwd, which on NixOS is the read-only /nix/store -> the service crash-looped at startup (OSError: Read-only file system). Follow the same split config.json already uses: the logconf presets stay read-only in the source tree (utils.pifinder_dir/python/logconf_*.json) and the active selection is persisted as a bare filename in the writable data dir (PiFinder_data/log_config), resolved via utils.active_logconf_path(). Storing the name (not a store-path symlink) keeps the choice valid across upgrades (which GC old store paths) and reboots. No NixOS workaround needed; also removes the write-to-source-tree antipattern on Raspberry Pi OS (upstreamable). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The upstream merge kept nixos's outer tetra3_dir (python/PiFinder/tetra3)
while taking upstream's solver.py, which built the DB path as
tetra3_dir/data/default_database.npz. But the submodule nests the package
at tetra3/tetra3, so the DB is actually at tetra3/tetra3/data -> the solver
crashed at startup with FileNotFoundError.
Load it by its canonical name, Tetra3("default_database"); tetra3 resolves
the bundled DB from its own package data dir regardless of the inner/outer
layout. Validated on-device (loads from python/tetra3/data).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
release.yml now installs Nix + Attic like build.yml and pushes the release closure to a dedicated, never-GC'd `pifinder-release` cache. Devices and the migration first-boot trust it ahead of the dev `pifinder` cache and cache.nixos.org. Removes the last Cachix usage from active config. Docs (RELEASE.md, ADR 0004, NIXOS_STATUS.md) document the dev-vs-retained-release two-cache split and the per-cache retention caveat. The pifinder-release trusted-public-key is a placeholder until the cache is bootstrapped server-side. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 6 .direnv/ nix-direnv cache files were tracked but regenerate on every direnv reload, so rebases baked divergent copies into the nixos stack. Gitignore + untrack stops that churn going forward. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…erivations Replace the 524-line nixos/pkgs/python-packages.nix (26 manually packaged PyPI deps, each with hand-chased hashes and build patches) with a uv-managed workspace realized into the Nix store via uv2nix. Changes: * Deps declared in python/pyproject.toml, pinned in python/uv.lock (117 pkgs) * nixos/pkgs/uv-python.nix builds the runtime/dev virtualenvs; the 5 native packages (python-libinput, python-prctl, python-pam, dbus-python, pygobject) keep their build patches as uv2nix overrides * flake.nix: add pyproject-nix/uv2nix/pyproject-build-systems inputs, thread via specialArgs, devShell uses the uv2nix devEnv * libcamera Python bindings stay a Nix overlay (not on PyPI) All four nixosConfigurations + the devShell evaluate; the aarch64 build is to be validated by CI. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…cache DeterminateSystems/magic-nix-cache-action now returns HTTP 418 and the GitHub Actions cache rate-limits it (Twirp ResourceExhausted, "rate limit exceeded"), so `nix develop` cannot fetch the dev environment and every lint/test/type-check job fails before ruff/pytest/mypy even run — which all testable PRs inherit. Mirror build.yml/release.yml and substitute from the self-hosted Attic cache cache.pifinder.eu (ADR 0004) instead, falling back to cache.nixos.org when ATTIC_TOKEN is unavailable (e.g. fork PRs) so the job never hard-fails. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…s_utils failures Changes: * networking.nix: pifinder-wifi-fallback service+timer brings up PiFinder-AP when no client connects within 45s (or when AP is forced via the UI), so a device with an unreachable saved network stays reachable and AP survives reboot. * sys_utils.py: persist chosen WiFi mode to PiFinder_data/wifi_mode for the fallback service to restore. * software.py: show a clear 'No internet - check WiFi' screen instead of an empty channel list when GitHub is unreachable. * utils.py: log why get_sys_utils() falls back to the no-op fake (previously swallowed) so a failed NM/dbus/pam import is diagnosable. * remove dead Raspberry Pi OS scripts switch-ap.sh/switch-cli.sh (dhcpcd/hostapd; inert on NixOS). Needs on-device validation of the AP fallback timing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When the Pi5 self-hosted runner is slow or unavailable, the fallback now uses GitHub's free hosted arm64 runner (public repos) and builds natively: no QEMU binfmt, no --system aarch64-linux cross flag, and extra-platforms dropped. Timeout cut 360 -> 60 min. Job id kept as build-emulated so downstream needs: references are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…t, evdev
These C-extension packages build from sdist in the uv2nix devEnv, but their
overrides did not declare a build backend. When the packages are not cached
(notably the x86_64 lint/test/type-check runners) the sdist build fails and
takes the whole devEnv down with it:
- dbus-python, pygobject: "No module named 'mesonpy'" -> add the meson-python
build system via resolveBuildSystem (alongside the existing pkg-config + C
libs).
- evdev: "No module named 'setuptools'" -> add the setuptools build system
(evdev had no override at all).
Mirrors the existing python-libinput / python-prctl overrides, which already
use resolveBuildSystem { setuptools = []; }.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ders, patches) Surfaced by the first real build. In uv-python.nix: * adafruit-blinka: ignore missing libgpiod.so.2 (vendored non-Pi SoC helpers) * dbus-python/pygobject/pycairo: add ninja for the meson-python build * pygobject: add pycairo for py3cairo.h * evdev: add linuxHeaders and repoint setup.py's header search off /usr/include * rpi-gpio/sh/spidev/pidng: provide the setuptools build backend * python-pam: patch the installed wheel module in postInstall (no src/ at patch time) pyproject.toml: empty [tool.setuptools] package set so the virtual root builds an empty wheel instead of tripping setuptools flat-layout discovery. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…arch64) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(a) Configure cache.pifinder.eu/pifinder as a public substituter (public key, no token) in build-emulated and the lint/test/type-check jobs, so fork-PR runs (brickbots) pull from the cache instead of building from source. Gate attic login + push on ATTIC_TOKEN so tokenless runs are pull-only and don't fail at push. (b) Add always() to stamp-build so a skipped build-emulated (when build-native succeeds) no longer skips the stamp — green builds now update pifinder-build.json. Also add nixos/brickbots-attic-setup.md: how to give brickbots a push token. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- delete by NM connection UUID, not by id == ssid. The old match silently no-ops on any entry whose connection id differs from its SSID (e.g. the corrupt "0x20" entry whose SSID held byte-array text), so such networks could never be deleted from the UI. - add: persist via add_connection(save_to_disk=True) first, then make a best-effort activate. Saving no longer depends on being able to activate right now, which failed (and saved nothing) when wlan0 was busy in AP mode or the network was out of range. - get_wifi_networks: re-query NetworkManager live instead of returning a stale cache, so changes made outside the process (AP/CLI switch, repairs) show up without an app restart. - decode SSIDs with errors="replace" so a single undecodable SSID can't crash the whole network list. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
NetworkManager (like other security-sensitive plugin loaders) refuses to load any plugin file not owned by root. A store with non-root paths — uid 1000 baked into the migration tarball, or imported by single-user nix — makes NM drop its wifi device plugin entirely: wlan0 shows as "unmanaged", WIFI-HW as "missing", and no wifi client connection ever comes up. Add an idempotent boot oneshot (ordered before NetworkManager) that normalises /nix/store and the nix db dir back to root, remounting the store read-write only when a repair is actually needed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The uv2nix env builds RPi.GPIO 0.7.1 from PyPI. Its C module init calls get_rpi_info(), which reads the board revision from /proc/device-tree/system/linux,revision or the /proc/cpuinfo "Revision" line. On a NixOS Pi 4 (arm64, mainline device tree) neither is present, so the import raises "This module can only be run on a Raspberry Pi!" — and since adafruit-blinka imports it via `board`, PiFinder crash-loops at startup. Add a postPatch that makes get_rpi_info() fall back to the always-present /proc/device-tree/model string, synthesising a Pi 4 Model B revision when it sees "Raspberry Pi". Detection then succeeds and the app starts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e / read-only) /nix/store is a read-only bind mount of the same device as /. The repair oneshot remounted it with "remount,rw"/"remount,ro", which operates on the shared superblock, so restoring ro flipped / (and /nix/var) read-only too — breaking the next pifinder-upgrade (nix build could not write temproots). Carry "bind" on both remounts so only this mount's per-mount ro flag changes and the rootfs superblock is left alone. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
wifi_mode() returned a value detected once in __init__. When the radio falls back to AP after startup, NetworkManager shows PiFinder-AP active but the UI still said "Client" (and local_ip() returned the wrong address). Detect the mode live from NetworkManager on each call, mirroring get_wifi_networks(). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…owing them The upgrade piped `nix build` through a gawk progress filter that dropped nix's stderr, so a failed upgrade only reported "failed" with no cause. Tee the build output to /run/pifinder/upgrade-nix.log and, on failure, dump its tail to the journal so the actual error is visible. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The placeholder "pifinder-release:REPLACE_WITH_PIFINDER_RELEASE_PUBKEY=" isn't valid base64, so nix aborts every operation with "invalid character in Base64 string" — bricking upgrades on the device. Remove it from both services.nix and migration.nix; the device trusts the working `pifinder` cache. The real release key is added only once that cache is provisioned. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…weep instrumentation)
…mat stragglers mypy: the B-V cache globals are Optional, so assignments cannot narrow them across statements; parse into locals and use those for savez/len. format: 4 files the CI format session was silently reformatting on every run (timeentry, three tests).
Telemetry captured only IMU and solve streams — blind to the radiometric side. A third event type records the camera-side radiometer sample at ~1 Hz: t (capture epoch, s), exp (driver-reported exposure, s), bg (background median, ADU), mad (median absolute deviation, ADU), grad (quadrant gradient, ADU), seq (frame sequence). Dedupes on sequence; replay players skip unknown event types, so old builds read new files. A full-night radio stream answers what sweep snapshots cannot: how auto-exposure actually behaves over a session (lever-arm availability for the black-level fit), a continuous sky-steadiness timeline, and real-night input for replaying pedestal estimators.
The marking menu down slot is the one readable at the eyepiece without moving your head — it goes to SWEEP, the everyday reference-meter action. SQM Correct is deleted rather than kept: a magnitude-additive session knob silently absorbs ADU-space, brightness-dependent errors (tonight: a stuck pedestal), masking exactly the fault class it appears to fix, and staling whenever the layers beneath it change. Its legitimate output — the unit-vs-reference-meter difference — is already recorded as evidence by every sweep run with a reference reading. The feature never shipped upstream (fork-only), so now is the cheapest moment to remove it. Removed: UISQMCorrect screen, menu entry and corrected indicator, _apply_sqm_correct and both solver call sites, the sqm_correct_delta shared state, and all test references. CONTEXT.md records the decision.
python/result and python/result-lib are nix build output symlinks that were committed before .gitignore's result/result-* rule existed, so they stayed tracked and kept getting re-snapshotted into unrelated commits. Untrack them; the existing ignore rule keeps them out from now on. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PXDxJkU5CzAegv3FcMEwEq
The NixOS migration UI flow (UIMigrationConfirm/UIMigrationProgress) was removed from ui/software.py in 'refactor(software): delete the NixOS migration UI flow', but a later sqm-fix merge reintroduced the test's import and spec builders for those deleted classes, breaking test collection. Drop the stale references. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PXDxJkU5CzAegv3FcMEwEq
test_radec_entry exercised an older dependency-injection API (BlinkingCursor time_provider; CoordinateConverter/CoordinateEntryLogic calc_utils_provider; LayoutConfig no-arg) that radec_entry dropped for module-level calc_utils/time. Patch calc_utils/time at the module level and pass a display stub to LayoutConfig so the suite matches the implementation. 6 stale tests were failing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PXDxJkU5CzAegv3FcMEwEq
Surface the ten shielded optical-black rows the IMX290/462 sensor already
transmits and use their trimmed mean as the complete per-frame pedestal for
radiometric SQM. This makes IMX462 SQM zero-touch: no lens cap, dark frames,
or calibration wizard from the user.
Kernel: a second sensor source pad (MEDIA_BUS_FMT_SENSOR_DATA) routes the OB
rows to the receiver as a separate metadata stream without altering the
1920x1080 image matrix.
libcamera: the imx290 cam-helper unpacks the shielded pixels, trims the outer
5% each side and averages the central 90% (preserving sub-ADU resolution on
the 16-bit SensorBlackLevels scale), publishing {l,l,l,l+1}. The +1 marker
lets Python tell a measured value from a static tuning tuple.
Python: camera_pi reads SensorBlackLevels for imx290/imx462 and passes the
native-ADU value as the radiometer sample's optical_black_pedestal. A valid OB
value is the complete pedestal; calibrated then profile pedestals remain the
fallback when OB is missing or unmarked.
Validated on mr2 (IMX462, gain 30) by a same-frame cupboard test: active-green
vs OB dark-accumulation slopes agree to well under 1 ADU/s, so OB is a valid
complete pedestal. Kernel is built via nixos-hardware; pifinder-fast/
pifinder-kernel-cross give a fast x86_64 cross build to seed the binary cache
so CI substitutes the kernel instead of compiling it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015EWb2JET2JE1t4ya3HZPNL
Add the OB pedestal to the SQM architecture note and glossary, and record the decision in ADR-0023. The justification is the same-frame cupboard test on mr2's IMX462 (active-green vs OB slopes agree to <1 ADU/s), scoped to IMX462; IMX296/IMX477 OB remain open. Defines 'optical black' as a glossary term and sets the pedestal precedence (valid OB > calibrated > profile bias). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015EWb2JET2JE1t4ya3HZPNL
python/.venv, mypy/pytest/ruff caches and __pycache__ directories are not needed at runtime and inflated the source output to ~935 MiB, making Attic cache pushes absurdly slow. Removing them drops it to ~19 MiB (cache push ~10-15 s, device upgrade fetch ~30-43 MiB). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015EWb2JET2JE1t4ya3HZPNL
…stal Gate BlackLevelTracker creation behind SQM_BLACK_LEVEL_TRACKER_ENABLED (default False) so the radiometer pedestal is OB when present, else the static profile bias_offset — no tracker in between. Lets us observe the optical-black pedestal's effect in isolation during rollout/validation. Reversible: flip the flag (or revert) to restore the tracker fallback. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015EWb2JET2JE1t4ya3HZPNL
All aperture/annulus/wing radii are defined for the ~1.0-scale Bayer-green photometry images (imx462 green: 490px vs the 512 solve image). The mono imx296 does photometry on its full-res 1088px frame (scale 2.125), where the fixed radii sit inside the star profile: the r=5 aperture holds only 83-94% of a star's flux (measured growth curves, 2026-07-18 sweeps), every local background annulus lands on the PSF itself, and the wing estimator's sky ring eats the wings so it measures f~1 and stays permanently inert. Convert all radii from solve-image pixels to the photometry image's pitch (_scaled_photometry_radii) and give WingEstimator a set_scale() that rescales its patch geometry and clears the rolling window on change. Both the production path (solver.update_sqm) and the calibration wizard's SQM preview use the shared helper. Effect by sensor: - imx462/imx290: radii round to (5, 10, 17) - essentially unchanged. - imx296: radii become (11, 21, 38); on the 2026-07-18 reference sweeps this moves the median error vs a hand-held meter from -0.20 to +0.00 and lets the estimator track real focus drift. - hq: also gets larger radii; its sqm_band_offset (like imx296's) should be refit against reference sweeps after this change. Validated: 121 tests pass across test_sqm, test_solver_sqm, test_sqm_calibration, test_sweep_frame_record, test_solver_shmem (9 new tests cover the scaling).
The sensor's OB clamp pins raw black to a target that moves with sensor state: on the 2026-07-18 imx296 reference sweeps the delivered level was 55.9-56.3 ADU while the device's wizard calibration (bias 58) and the profile constant (60) both over-subtracted. At dark-site signal levels a 2-4 ADU over-subtraction produces a strong SQM-vs-exposure slope (-0.9..-1.5 mag/decade) and kills short exposures outright (background - pedestal <= 0, 'background unresolved'). The BlackLevelTracker already measures the in-session intercept but was suppressed whenever a wizard calibration existed - precisely backwards: a stored bias is one moment's measurement of a moving level. Flip the priority in both SQM paths (radiometric and stellar): a confidently fitted tracked intercept wins for bias; the wizard's dark-current rate remains authoritative and is added on top (the intercept fit cannot separate dark from sky). The tracker's own stderr / deviation-band / lease gates bound how far it can stray, and an accepted fit >2 ADU from the static bias now logs at info. Replayed against the 2026-07-18 imx296 sweeps (radiometric feed, wizard anchor 58): tracker conditions on all four (stderr 0.26-0.32), fits 55.9-56.3, and the recomputed SQM exposure slopes collapse from -0.9..-1.5 to -0.01..+0.52 with the residual offset a tight constant (sd 0.04) for a later band-offset refit. Also: the published black_level_tracked flag now reflects the leased pedestal actually used, not the raw last fit. Validated: 136 tests pass across the SQM suites (4 updated/new).
Rich's 2026-07-18 calibration reports (imx296 + imx462, wizard run interleaved with the reference sweeps) show two capture defects: 1. The first ~3 frames of every stage report actual_exposure_us at the PREVIOUS exposure (req=1us, actual=999999us) yet pass the frame gate, because _capture_and_wait checks the exposure_time field, which echoes the committed setting rather than what the sensor delivered. Stale full-length frames land in the bias/dark stacks. 2. After the exposure change lands, the sensor's optical-black clamp re-settles to a new level over ~3 more frames (imx296: 56 -> 59 ADU). Frames in that window measure neither the old nor the new black level. (On the imx296 the settled clamp is genuinely exposure-dependent: ~56 ADU at 1s vs ~59 ADU at the 29us minimum - a min-exposure bias is only valid for its own regime, which is why the tracked in-session pedestal takes priority in operation.) Together these explain the wizard bias of 58 on a sensor whose 1s clamp level is 56: a median over stale-1s frames at 56, settling frames, and settled-29us frames at 59. Fix: _capture_and_wait gates on actual_exposure_us (exposure_time only as fallback), re-requests a capture when a stale frame arrives, and uses an exposure-proportional tolerance; bias and dark stages discard CLAMP_SETTLE_DISCARD_FRAMES frames after every exposure change (the dark ramp changes exposure per step). Adds ~30-40s to a wizard run. Evidence: dumps/logs/rich-imx296/sqm_calibration_report_imx296_*.json. Validated: 141 tests pass across the SQM suites (5 new).
feat(sqm): per-frame optical-black pedestal for IMX290/IMX462 (zero-touch)
Telemetry menu under Settings > Telemetry: - Record: inline On/Off toggle (no submenu), same pattern as Test Mode. - Sections: one multi-select checklist with inline checkmarks (like the Catalogs filter) for what to record — IMU, SQM, Solves, Targets and Images. Toggling applies live to an in-progress recording. - Max Size: session cap (250 MB / 500 MB / 1 GB / 2 GB / Unlimited). - Load: unchanged. Images ships OFF by default, preserving the old telemetry_images=false: one 512x512 PNG is written per solve, so at ~1 solve/s it costs roughly 300 MB/hour against a few MB/hour for every other section combined. Session size cap. Frames are written by the camera process, so the recorder measures the session directory from its flush loop rather than counting its own writes. On reaching the cap, frame capture is suspended (the one unbounded consumer) while the event log keeps running, so a capped session stays useful instead of going dark. Default 1 GB. Also logs the SQM *ingredients*, not just the end products, so a session stays recomputable if the SQM/airglow maths changes in code later: - radio event gains the raw per-frame inputs — red/blue Bayer backgrounds, optical-black pedestal, photometry image size (green bg and exposure were already there). - the session header snapshots the FULL camera profile constants plus the airglow calibration and camera_type — the constants those ingredients were produced under. The derived published 'sqm' and applied 'floor' are still logged, but explicitly as a what-the-device-showed audit trail, not ground truth. The airglow import is optional: the model is part of the SQM stack only on branches that carry it (deepchart has the radiometer but not airglow), and there a session simply records no airglow constants.
pull_request_target defaults actions/checkout to the repository's default
branch. That branch carries this workflow so the event fires, but not
necessarily .github/scripts/, so update-manifest died with
bash: .github/scripts/publish_manifest.sh: No such file or directory
after a successful 19-minute build -- the image was pushed to Attic but no
device could ever see it, because the manifest was never written.
Check out the PR's base branch instead: it is the branch this workflow file
itself came from, so the scripts are always beside it, and it stays
maintainer-controlled. Checking out the PR head here is not an option -- this
job holds contents:write and the Attic token.
load_sharpless computed j_dec_deg from b1950_to_j2000() and then passed the unconverted dec_deg to NewCatalogObject, so only right ascension was precessed. All 313 Sharpless objects sat 0.26 to 0.37 degrees off in declination, which is exactly the 1950-to-2000 shift at those coordinates. Found by cross-checking against the Perek-Kohoutek catalogue, which lists several of the same nebulae with independent J2000 positions.
abell.tsv carried the ngc6742 alias on row 51. NGC 6742 matches row 50 to 0.1 arcmin and is 66 degrees from row 51, and the Strasbourg-ESO catalogue independently gives A 50, not A 51, as NGC 6742. The misplaced alias bound the Abell 51 listing onto NGC 6742's sky object. Row 47 had lost its declination sign. Abell 47 is at -00 13 51 per SIMBAD and Kohoutek 2001. Its constellation, Serpens, spans the celestial equator, so nothing else caught it. The two remaining aliases were audited positionally: IC 972, NGC 7076 and IC 1454 all agree within 0.12 arcmin.
Adds catalog code PK: the 1510 galactic planetary nebulae of Kohoutek's
2001 revision of Perek-Kohoutek 1967, from VizieR IV/24. Positions come
from SIMBAD or IV/24 table4, sizes and cross-identifications from V/84.
Sources are vendored under astro_data/perek_kohoutek/ with PROVENANCE.md
and a fetch script; decisions are in docs/adr/0024.
No precession runs here. Every position used is already J2000, and each
refined position must agree with the catalogue's own coarse position
within 5 arcmin, which rejects a four-degree typo in IV/24 table4.
The sequence is the printed catalogue's 1-1510 running number, so "PK 743"
is a list position, not a designation; the real designation is stored as a
name in both spellings. No PK source carries a magnitude, so entries are
built with an empty one: the 211 that resolve to an existing NGC, IC,
Messier, Abell or Sharpless object inherit its magnitude, and the rest fall
out of any magnitude filter via UNKNOWN_MAG.
Alias matching had to be fixed first. ObjectFinder resolved aliases through
ui_utils.normalize(), which strips spaces and hyphens, so any compound
numeric part collapsed into a wrong sequence number. Feeding it this
catalogue's name column produced 147 matches, 145 of them false - 188 of
those names are Minkowski planetary nebulae ("M 1-92"), not Messier
objects, and "H 3-29" is Haro, not Herschel 329. catalog_import_utils now
carries CATALOG_CODE_ALIASES and parse_designation(), which require an
explicitly allowlisted prefix and a bare integer sequence, so a compound
designation never reaches hyphen-stripping.
That fix also makes an existing intent work: post_processing lists "Cr 42"
among M45's aka names so the Pleiades resolve to one object, but Cr had
never been mapped to Col, so the database shipped two Pleiades. It also
removes two false links in SAC Multistars.
Verified against the published 1510-row list transcribed on astronomy.com
and in a widely circulated spreadsheet: all 1510 PK designations and all
1510 other-designations match exactly. That check also found the
spreadsheet drops a declination sign at row 551.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds catalog code PK: the 1510 galactic planetary nebulae of Kohoutek's 2001 revision of Perek-Kohoutek 1967.
Decisions and evidence are in
docs/adr/0024-perek-kohoutek-catalog.md; data provenance is inastro_data/perek_kohoutek/PROVENANCE.md.Source
VizieR IV/24 (Kohoutek 2001), the direct successor to CGPN 1967.
table2supplies the 1510 rows,table4arcsecond positions; V/84 (Acker+ 1992) supplies sizes and cross-identifications; SIMBAD supplies positions and cross-IDs. All vendored underastro_data/perek_kohoutek/with afetch_sources.shthat regenerates the snapshot.Two secondary transcriptions circulate among amateurs — an astronomy.com PDF and a spreadsheet linked from a Google Sites page. Both are reprints of the same Kohoutek 2001 data with the PN G designation, flags and notes stripped and en-dashes substituted for hyphens, so neither is imported. They were used as an independent check instead: all 1510 PK designations and all 1510 other-designations match
table2.datexactly. The check also found the spreadsheet drops a declination sign at row 551, putting that object 88 degrees out.Epoch
No precession runs in the loader. Every position used is already J2000, and each refined position must agree with the catalogue's own coarse position within 5 arcmin. That guard rejected a four-degree typo in
table4(the equinox-2000 row for Vy 1-4 reads-02 26where its five sibling rows and SIMBAD read-06 26). Final split: 1399 SIMBAD, 111table4. Median SIMBAD-to-table4agreement is 0.04 arcmin, so no B1950 leaked in.Sequence and magnitude
The Sequence is the printed catalogue's 1-1510 running number, not the PK designation, which is not an integer. So
PK 743is a position in the list; the real designation is stored as a name in both spellings (PK 036+17.1andPK 036+17 1) and is searchable. Encoding the designation as an integer was rejected: it yields 8-digit sequences, unreadable on a 128px display and impractical by keypad.No PK source carries a magnitude, so entries are built with an empty one. The 211 entries that resolve to an existing NGC, IC, Messier, Abell or Sharpless object inherit its magnitude and size; the rest fall out of any magnitude filter via
UNKNOWN_MAG. 197 entries end up with a real magnitude, 1037 with a size.Alias matching had to be fixed first
ObjectFinderresolved aliases throughui_utils.normalize(), which strips spaces and hyphens, so any compound numeric part collapsed into a plausible but wrong sequence number. Checked against the shipped database, feeding it this catalogue's name column produced 147 matches, 145 of them false:normalize()M 1-1,M 2-9,M 3-1(Minkowski)m11,m29,m31H 1-1,H 3-29(Haro)h11,h329NGC 650-1(M76)ngc6501Sh 2-176sh2176188 of the 1510 names begin
M, and every one is Minkowski, not Messier. Left alone, 145 planetary nebulae would have silently displayed Messier and Herschel objects' positions and magnitudes.catalog_import_utilsnow carriesCATALOG_CODE_ALIASESandparse_designation(), which require an explicitly allowlisted prefix and a bare integer sequence, so a compound designation never reaches hyphen-stripping. This is in the shared util rather than the loader because the defect affects every catalog.Two knock-on effects, both improvements:
post_processinghas always listed"Cr 42"among M45's aka names so the Pleiades resolve to one object, butCrwas never mapped toCol— the database shipped two separate Pleiades objects. Now one, listed as bothM 45andCol 42.Three data errors this exposed
PK carries independent positions, so every cross-link also checks the catalog it links to. Of 211 links, 6 disagreed by more than 5 arcmin, each tracing to a real defect. All are fixed here, in separate commits so they can be split out:
load_sharplessnever precessed declination — it computedj_dec_degthen passed the unconverteddec_deg. All 313 Sharpless objects sat 0.26 to 0.37 degrees off, exactly the 1950-to-2000 shift.abell.tsvrow 47 lost its declination sign. Abell 47 is at-00 13 51. Serpens spans the equator, so nothing else caught it.abell.tsvhad thengc6742alias on row 51. NGC 6742 matches row 50 to 0.1 arcmin and is 66 degrees from row 51; V/84 independently gives A 50 as NGC 6742.After the fixes, 2 of 211 links still disagree — Sh2 176 by 6 arcmin and Sh2 216 by 17. Both are large diffuse nebulae (Sh2 216 spans ~1.6 degrees) where catalogues place the centre differently. Not errors.
The remaining
abell.tsvaliases were audited positionally: IC 972, NGC 6742, NGC 7076 and IC 1454 all agree within 0.12 arcmin.Verification
ruff check,ruff format --check,mypyall clean.tests/test_designation_parsing.pycovers the alias regression;test_catalog_data.pygains PK counts, PK/NGC/Abell/Sharpless linkage assertions, Minkowski-is-not-Messier assertions, and a Pleiades-is-one-object assertion.CatalogBuilder: PK loads 1510 objects,PK 2shares its sky object withNGC 40,PK 18(M 1-1) is correctly not Messier 11, and text search finds bothNGC 7009and036+17.Not verified on device. This branch has no
cedar-detect-serverbinary inbin/, so the app could not be launched here; verification was done at theCatalogslayer. Worth an on-device pass of Objects > DSO... and Filters > Catalogs.Note on the menu label
The entry is labelled "PK Planetary". "Perek-Kohoutek" is 14 characters against an established 12-character maximum on the 128px display. Easy to change if you prefer something else.
🤖 Generated with Claude Code