diff --git a/AGENTS.md b/AGENTS.md index d91d3b8..cd2df8a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -38,8 +38,8 @@ removals. A JOSS paper + citation remain a post-2.0 follow-up. | `backend.py` | Optional NumPy/CuPy array and RNG boundary, explicit host conversion, and backend convolution. NumPy is the reference/default. Owns the stack's `device` vocabulary (`"cpu"`, `"gpu"`, `"gpu:N"`, `"auto"`; parsed by `_parse_device`, resolved by `get_backend`) and `precision` vocabulary (`resolve_precision`: `"single"`/`"double"`, aliases `"float32"`/`"float64"`). A GPU `ArrayBackend` carries a `device_id`; `activate()` makes it current. Kept local rather than `aocore.Backend`: getframes draws per-frame noise from a device-native (cuRAND) stream, which `aocore.Backend.random` (host-side `Generator`) does not provide. | | `_cuda.py` | Private fused CuPy `ElementwiseKernel`s for the launch-bound GPU hot path: the photo + total expectation (`fused_expectation`, called from `simulate_frame`) and the whole readout (`fused_readout`, called from `digitize`). Each mirrors the separate CuPy operations it replaces op for op (same order, working precision, Python scalars rounded to it, `--fmad=false`), so seeded GPU frames are bit-identical to the unfused path, which remains the fallback for any operand it cannot take exactly. Imported only on the GPU backend; CuPy is imported lazily. | | `frame.py` | `Frame` container: a NumPy array (ADU) plus metadata; array-like; optional FITS export. | -| `camera.py` | `Camera`, the main user-facing object. Orchestrates config + scene + noise into `Frame`s. Holds the RNG and high-level methods (`dark_frame`, `dark_series`, reset-correlated `nondestructive_series`, `correlated_double_sample`, `expose`, `observe`, `*_series`, `master_*`). Reset-correlated readout has one core, the private `_ramp_reads`, which walks a ramp on an arbitrary per-read interval pattern; `nondestructive_series` drives it uniformly and `correlated_double_sample` drives it as pedestal-then-signal. Put new ramp-readout modes there rather than in a second loop — the interval-scaled bias/settling/avalanche terms are easy to get subtly wrong twice. Every public method that touches device arrays is decorated with `backend._on_device`, which runs it (each step, for generator methods) with the camera's CUDA device current; decorate new ones too, or `device="gpu:N"` silently allocates on the wrong card. | -| `calibrate.py` | Master-frame builders (`combine`) and `calibrate` reduction — the raw → reduced → truth loop (phase 1.1). | +| `camera.py` | `Camera`, the main user-facing object. Orchestrates config + scene + noise into `Frame`s. Holds the RNG and high-level methods (`dark_frame`, `dark_series`, reset-correlated `nondestructive_series`, `correlated_double_sample`, `expose`, `observe`, `*_series`, `master_*`, plus `calibration` and the input-referred `effective_read_noise_e`). Reset-correlated readout has one core, the private `_ramp_reads`, which walks a ramp on an arbitrary per-read interval pattern; `nondestructive_series` drives it uniformly and `correlated_double_sample` drives it as pedestal-then-signal. Put new ramp-readout modes there rather than in a second loop — the interval-scaled bias/settling/avalanche terms are easy to get subtly wrong twice. Every public method that touches device arrays is decorated with `backend._on_device`, which runs it (each step, for generator methods) with the camera's CUDA device current; decorate new ones too, or `device="gpu:N"` silently allocates on the wrong card. | +| `calibrate.py` | Master-frame builders (`combine`), `calibrate` reduction — the raw → reduced → truth loop (phase 1.1) — and `Calibration`, the ADU → input-referred electrons → photons inverse of the mean signal chain (mean master dark, `gain_e_per_adu / em_gain`, nonlinearity inverse, optional flat, scalar/curve QE) used for WFS frames. It works on NumPy and CuPy frames; keep its inverse in step with the forward model in `noise.py`. | | `observation.py` | `Observation` / `ObservationTruth` / `Pointing`: the time-series driver, jitter/drift/dither, per-frame truth (phase 1.2). | | `spectral.py` | Opt-in spectral mode: `QE`, `SED` (relative or absolute via `from_flux_density`), `Spectrum`, `SpectralBandpass`, effective-QE folding, transmission-product helpers (`product`, `from_file`/`from_product`), optional `astropy.units` coercion. | | `scene/` | The scene/optics layer: `Scene`, `Source` hierarchy (`PointSource`, `ExtendedSource`, `UniformIllumination`, `Catalog`; point/extended sources accept a `flux_sed` absolute SED), PSFs (`GaussianPSF`/`MoffatPSF`/`AiryPSF`/`ArrayPSF`/`EllipticalGaussianPSF`), `Telescope` (+ `Vignetting`/`RadialDistortion`), `Bandpass` (Vega `johnson` / AB `ab` ugriz·Gaia·2MASS + `Extinction`) in `photometry.py`, `Thermal` graybody background in `thermal.py`, `WCSInfo`, `LightCurve`. Renders a photon-rate map; no randomness. | @@ -211,7 +211,7 @@ make the smallest change that fits the existing patterns; and if a change spans layers, respect the one-way data flow. When something is genuinely done and verified, say so plainly; when a step was skipped or a test fails, say *that*. -## Performance notes (GPU hot path) +## Performance notes (hot path) - Small GPU frames are launch-bound (each CuPy call costs 10–20 µs of host time on a slow core), so per-frame work counts CuPy calls, not FLOPs. Large frames, and the @@ -229,6 +229,15 @@ verified, say so plainly; when a step was skipped or a test fails, say *that*. `backend.asarray` for a value that may be a scalar, and pass scalar distribution parameters through the `_CuPyGenerator` wrapper (it reuses a cached device copy), so a GPU frame does not upload a fresh 0-d array per call. +- **CPU gain stage.** NumPy's `Generator.gamma` returns `Gamma(shape=0)` as exactly + zero *without consuming a draw* but still pays its per-element dispatch, so + `apply_gain_stage` samples only the lit pixels (C order) when they are a minority. + This is bit-identical only because zero-shape elements consume nothing; + `tests/test_wfs_calibration.py::test_gain_stage_matches_full_gamma_draw_bit_for_bit` + guards it against the installed NumPy. Do not extend the trick to Poisson: there a + zero mean also consumes nothing, but every pixel carries dark current. An exact + vectorised replica of NumPy's small-rate Poisson (for CIC) was tried and measured + no faster than `rng.poisson`, so it was not kept. - Identity arithmetic on a whole frame (`+ 0`, `/ 1`, `* 1` by a *Python* scalar) is skipped via `noise._is_scalar_value`; it cannot change any ADU (at most the sign of a zero, which rounding erases). Keep the test to Python scalars: reading a 0-d diff --git a/CHANGELOG.md b/CHANGELOG.md index d5606f9..599592c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,68 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [Unreleased] +## [2.6.0] - 2026-10-10 + +### Added + +- **`Calibration`: ADU to input-referred electrons and incident photons** + (`gf.Calibration`, `Camera.calibration`). `Calibration.from_camera(cam, + exposure, n_darks=64, seed=...)` averages exposure-matched dark frames into a + mean master dark on the camera's device and precision; `to_electrons` applies + `L^-1[(ADU - dark) * gain_e_per_adu / em_gain] / flat`, inverting the + configured nonlinearity (closed form or Newton) by default; `to_photons` + divides by the scalar QE `expose` applies, a `quantum_efficiency`, the + `qe_curve` at `wavelength_nm`, or its effective value over a `bandpass`; + `saturated` flags pixels whose charge reached the input well, the output + register or the ADC ceiling. It works on NumPy and CuPy frames, takes an + optional unit-mean flat, and handles binned frames. Tests check that the mean + calibrated flux matches the incident photons to 0.5% for EMCCD, qCMOS, sCMOS, + eAPD, CCD and CMOS presets (issue #10). +- **`Camera.effective_read_noise_e(binning=..., binning_mode=...)`**: the + per-frame read noise referred to the input (electrons collected in the + pixel): output read noise (the quadrature mean of a per-pixel map), kTC/reset + noise and avalanche-scaled noise, divided by the EM/avalanche gain. The + docstring and the new guide give the excess-noise-factor SNR. +- **Guide: [A WFS camera in a closed loop](docs/guides/wfs-closed-loop.md)**: + exposure vs. frame period, ADU calibration and its accuracy, input-referred + noise and the excess noise factor, recommended EMCCD/qCMOS/sCMOS/eAPD/CCD + presets, and keeping small frames cheap. +- `benchmarks/bench_small_frames.py`: seeded 256x256 WFS frames for one preset + per detector class, with before/after comparison and a SHA-256 digest of + seeded output for bit-identity checks; results in + `benchmarks/small-frame-results.md`. + +### Performance + +- **Photon-starved EMCCD and eAPD frames are faster on the CPU, with + bit-identical seeded output.** NumPy returns `Gamma(shape=0)` as exactly zero + without consuming a draw, but still visits every such pixel; the gain stage + now samples only the pixels that hold charge when they are a minority (a + wavefront sensor's dark background), and skips the `* 1` shape scaling at + `F = sqrt(2)`. Same draws, same order, same generator state afterwards; + seeded frames of every preset, in both precisions, are bit-identical to 2.5.0. + On one Neoverse-N1 core (256x256 Shack-Hartmann spots, 1 ms, in-process A/B): + the Gamma draw 1.43 → 0.78 ms; whole frames `andor_ocam2k` 6.57 → 5.95 ms + (1.10x), `leonardo_saphira` 4.79 → 4.29 ms (1.12x), `first_light_imaging_cred_one` + 7.16 → 6.63 ms (1.08x); presets without a gain stage are unchanged. The CPU + chain is otherwise at its sampler floor (about 85% of a frame is NumPy's + Poisson, Gamma and Gaussian draws). An exact vectorised replica of NumPy's + small-rate Poisson for the CIC draw was tried and measured no faster, so it + was not kept. See `benchmarks/small-frame-results.md`. + +### Documented + +- The 1--2% calibrated-flux deficit seen in the dithered Shack-Hartmann + prototype is not a deterministic offset of the signal chain: averaged over + 300 frames the prototype recovers 1.002 (OCAM2K), 0.999 (ORCA-Quest 2) and + 1.001 (generic sCMOS) of the incident flux, while a single 288x288 frame + scatters by 1.2%, 1.0% and 3.3%. The guide lists the deterministic offsets a + pipeline can introduce (median master dark, scalar vs. spectral QE, + nonlinearity, saturation, unflattened fixed patterns) and their remedies. +- The Nüvü HNü 240 / 128 presets cap the *amplified* charge at their 500 / 800 + e- `full_well_e` (no `output_full_well_e`), so at EM gain 1000 a pixel holding + one electron saturates; the guide warns about it (#11). + ## [2.5.0] - 2026-10-07 ### Added diff --git a/README.md b/README.md index 7917f73..64536e1 100644 --- a/README.md +++ b/README.md @@ -127,6 +127,10 @@ for the methodology. - **Calibration & ground truth** — master bias/dark/flat builders and a `calibrate` reduction that closes the **[raw → reduced → truth loop](https://jacotay7.github.io/getframes/guides/calibration/)**. +- **Wavefront-sensor cameras** — `Calibration` turns ADU back into + input-referred electrons and photons, `Camera.effective_read_noise_e` gives + the input-referred noise, and a guide covers + **[a WFS camera in a closed loop](https://jacotay7.github.io/getframes/guides/wfs-closed-loop/)**. - **Observations** — `Observation` drives time series with jitter, drift, dither and persistence, carrying per-frame **[truth](https://jacotay7.github.io/getframes/guides/time-series/)**. diff --git a/benchmarks/bench_small_frames.py b/benchmarks/bench_small_frames.py new file mode 100644 index 0000000..8534a4c --- /dev/null +++ b/benchmarks/bench_small_frames.py @@ -0,0 +1,192 @@ +# SPDX-License-Identifier: MIT +"""Benchmark small, frequent wavefront-sensor frames on the CPU. + +A closed-loop wavefront-sensor study calls :meth:`getframes.Camera.expose` once +per loop step on a small frame, so per-frame latency, not throughput on large +detectors, is what limits it. This script times seeded ``expose`` calls on a +256x256 Shack-Hartmann-like spot pattern (16x16 subapertures of 16x16 pixels, +one Gaussian spot each) for one preset of each WFS detector class, at a +photon-starved and a bright flux, in two call styles: + +* ``default``: ``expose(rate, t, seed=s, include_truth=False)``; +* ``workspace``: the same with a reused :class:`getframes.DetectorWorkspace` and + a caller-owned ``out`` frame. + +It also records a SHA-256 digest of a few seeded frames per configuration, so two +revisions can be compared for bit-identical output (``--compare``). Timings are +process CPU time per frame over interleaved, warmed blocks (the chain is +single-threaded); they are machine dependent and are not part of the test gate. +Each case reports the median and the best (minimum) block; on a shared host, +compare the best blocks, which other load can only inflate. + +Run from the repository root:: + + python benchmarks/bench_small_frames.py --output small-frame-results.json + python benchmarks/bench_small_frames.py --compare before.json after.json +""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import platform +import statistics +import time +from pathlib import Path +from typing import Any + +import numpy as np + +import getframes as gf + +SHAPE = (256, 256) +SUBAPERTURE_PX = 16 +EXPOSURE_S = 1e-3 +#: One preset per detector class a WFS is built from. +PRESETS = ( + ("EMCCD", "andor_ocam2k"), + ("EMCCD", "andor_ixon_ultra_888"), + ("qCMOS", "hamamatsu_orca_quest_2"), + ("sCMOS", "generic_scmos"), + ("eAPD", "first_light_imaging_cred_one"), +) +#: Photons per subaperture per frame: photon-starved and bright. +FLUXES = (100.0, 2000.0) +DIGEST_SEEDS = (0, 1, 2) + + +def spot_pattern(photons_per_subaperture: float) -> np.ndarray[Any, np.dtype[np.float64]]: + """A 256x256 photon map of one Gaussian spot (sigma 1.5 px) per subaperture.""" + coordinates = np.arange(SUBAPERTURE_PX) - (SUBAPERTURE_PX - 1) / 2 + spot = np.exp(-(coordinates[:, None] ** 2 + coordinates[None, :] ** 2) / (2 * 1.5**2)) + spot *= photons_per_subaperture / spot.sum() + reps = (SHAPE[0] // SUBAPERTURE_PX, SHAPE[1] // SUBAPERTURE_PX) + return np.tile(spot, reps) + + +def make_camera(preset: str) -> gf.Camera: + return gf.Camera.from_preset(preset).with_config(resolution=SHAPE, roi=None) + + +def _block_ms(function: Any, calls: int) -> float: + # Process CPU time, not wall time: the chain is single-threaded, and CPU time + # is far less sensitive to other load on a shared host. + start = time.process_time_ns() + for index in range(calls): + function(index) + return (time.process_time_ns() - start) / calls / 1e6 + + +def bench_case(preset: str, flux: float, calls: int, repetitions: int) -> dict[str, Any]: + camera = make_camera(preset) + rate = spot_pattern(flux) / EXPOSURE_S + workspace = gf.DetectorWorkspace() + out = np.empty(SHAPE, dtype=np.uint32) + + def default(seed: int) -> None: + camera.expose(rate, EXPOSURE_S, seed=seed, include_truth=False) + + def reused(seed: int) -> None: + camera.expose( + rate, EXPOSURE_S, seed=seed, include_truth=False, workspace=workspace, out=out + ) + + for seed in range(10): + default(seed) + reused(seed) + samples: dict[str, list[float]] = {"default": [], "workspace": []} + variants = [("default", default), ("workspace", reused)] + for repetition in range(repetitions): + for name, function in variants if repetition % 2 == 0 else variants[::-1]: + samples[name].append(_block_ms(function, calls)) + + digest = hashlib.sha256() + for seed in DIGEST_SEEDS: + frame = camera.expose(rate, EXPOSURE_S, seed=seed) + digest.update(np.ascontiguousarray(frame.data).tobytes()) + assert frame.truth is not None + digest.update(np.ascontiguousarray(frame.truth.mean_electrons).tobytes()) + return { + "preset": preset, + "photons_per_subaperture": flux, + "default_ms": statistics.median(samples["default"]), + "workspace_ms": statistics.median(samples["workspace"]), + "default_best_ms": min(samples["default"]), + "workspace_best_ms": min(samples["workspace"]), + "default_samples_ms": samples["default"], + "workspace_samples_ms": samples["workspace"], + "seeded_digest": digest.hexdigest(), + } + + +def run(calls: int, repetitions: int) -> dict[str, Any]: + results = [] + for sensor, preset in PRESETS: + for flux in FLUXES: + case = bench_case(preset, flux, calls, repetitions) + case["class"] = sensor + results.append(case) + print( + f"{sensor:6s} {preset:30s} {flux:7.0f} ph/subap " + f"default {case['default_ms']:6.2f} ms workspace {case['workspace_ms']:6.2f} ms", + flush=True, + ) + return { + "schema_version": 1, + "getframes": gf.__version__, + "numpy": np.__version__, + "platform": platform.platform(), + "processor": platform.processor() or platform.machine(), + "shape": list(SHAPE), + "exposure_s": EXPOSURE_S, + "calls_per_block": calls, + "repetitions": repetitions, + "results": results, + } + + +def compare(before_path: Path, after_path: Path) -> str: + """A Markdown table of before/after best blocks and whether seeded output matches.""" + before = json.loads(before_path.read_text(encoding="utf-8")) + after = json.loads(after_path.read_text(encoding="utf-8")) + lines = [ + f"Before: getframes {before['getframes']}; after: getframes {after['getframes']} " + f"({after['processor']}, NumPy {after['numpy']}, {after['shape'][0]}x{after['shape'][1]}).", + "", + "| Class | Preset | Photons/subap | Before (ms, best) | After (ms, best) | Speed-up | " + "After, workspace+out (ms, best) | Seeded output |", + "| --- | --- | ---: | ---: | ---: | ---: | ---: | --- |", + ] + old = {(r["preset"], r["photons_per_subaperture"]): r for r in before["results"]} + for row in after["results"]: + ref = old.get((row["preset"], row["photons_per_subaperture"])) + if ref is None: + continue + same = "identical" if ref["seeded_digest"] == row["seeded_digest"] else "CHANGED" + lines.append( + f"| {row['class']} | {row['preset']} | {row['photons_per_subaperture']:.0f} | " + f"{ref['default_best_ms']:.2f} | {row['default_best_ms']:.2f} | " + f"{ref['default_best_ms'] / row['default_best_ms']:.2f}x | " + f"{row['workspace_best_ms']:.2f} | {same} |" + ) + return "\n".join(lines) + "\n" + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--calls", type=int, default=40) + parser.add_argument("--repetitions", type=int, default=7) + parser.add_argument("--output", type=Path) + parser.add_argument("--compare", nargs=2, type=Path, metavar=("BEFORE", "AFTER")) + arguments = parser.parse_args() + if arguments.compare: + print(compare(*arguments.compare), end="") + return + payload = run(arguments.calls, arguments.repetitions) + if arguments.output is not None: + arguments.output.write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8") + + +if __name__ == "__main__": + main() diff --git a/benchmarks/small-frame-results.json b/benchmarks/small-frame-results.json new file mode 100644 index 0000000..b49a9e0 --- /dev/null +++ b/benchmarks/small-frame-results.json @@ -0,0 +1,2118 @@ +{ + "schema_version": 1, + "date": "2026-10-10", + "host": { + "cpu": "Ampere Neoverse-N1 (aarch64), one pinned core (taskset -c 20 / 44)", + "numpy": "2.5.3", + "platform": "Linux-6.17.9-76061709-generic-aarch64-with-glibc2.39" + }, + "workload": { + "shape": [ + 256, + 256 + ], + "pattern": "16x16 subapertures of 16x16 px, one Gaussian spot (sigma 1.5 px) each", + "exposure_s": 0.001, + "seeded": true, + "include_truth": false, + "timer": "process CPU time per frame, blocks of 30 calls" + }, + "before_revision": "2.5.0 (main 1128258)", + "after_revision": "2.6.0 (feat/issue-10-wfs-calibration)", + "cross_process_runs": { + "before": [ + { + "schema_version": 1, + "getframes": "2.5.0", + "numpy": "2.5.3", + "platform": "Linux-6.17.9-76061709-generic-aarch64-with-glibc2.39", + "processor": "aarch64", + "shape": [ + 256, + 256 + ], + "exposure_s": 0.001, + "calls_per_block": 30, + "repetitions": 9, + "results": [ + { + "preset": "andor_ocam2k", + "photons_per_subaperture": 100.0, + "default_ms": 6.615929333333333, + "workspace_ms": 5.826550666666667, + "default_best_ms": 6.609465333333333, + "workspace_best_ms": 5.804096, + "default_samples_ms": [ + 6.609465333333333, + 6.609552, + 6.610694666666667, + 6.614004, + 6.615929333333333, + 6.639630666666667, + 6.632146666666667, + 6.717064, + 6.651636 + ], + "workspace_samples_ms": [ + 5.836932, + 5.812748, + 5.821364, + 5.804096, + 5.826550666666667, + 5.808973333333333, + 5.84588, + 5.853553333333333, + 5.8386 + ], + "seeded_digest": "1712c1ff7389450f37a8dbb58f06a6c34b80e1bc325476a5b9a79de88b0ad36a", + "class": "EMCCD" + }, + { + "preset": "andor_ocam2k", + "photons_per_subaperture": 2000.0, + "default_ms": 7.372577333333333, + "workspace_ms": 6.588757333333333, + "default_best_ms": 7.364164, + "workspace_best_ms": 6.572185333333333, + "default_samples_ms": [ + 7.370608, + 7.374694666666667, + 7.403626666666667, + 7.367029333333333, + 7.369186666666667, + 7.364164, + 7.389101333333333, + 7.372577333333333, + 7.375733333333333 + ], + "workspace_samples_ms": [ + 6.615306666666667, + 6.574377333333333, + 6.572185333333333, + 6.588757333333333, + 6.594228, + 6.577165333333333, + 6.653470666666667, + 6.609945333333333, + 6.588742666666667 + ], + "seeded_digest": "e1f649808cb6de3b0dfb99e09124b9639271816dce307f7ac3b48aee95f37826", + "class": "EMCCD" + }, + { + "preset": "andor_ixon_ultra_888", + "photons_per_subaperture": 100.0, + "default_ms": 5.842421333333333, + "workspace_ms": 5.84852, + "default_best_ms": 5.822017333333333, + "workspace_best_ms": 5.828630666666667, + "default_samples_ms": [ + 5.847826666666667, + 5.843612, + 5.8364053333333334, + 5.844858666666667, + 5.822017333333333, + 5.832390666666667, + 5.830877333333333, + 5.842421333333333, + 5.843886666666667 + ], + "workspace_samples_ms": [ + 5.84852, + 5.843656, + 5.846156, + 5.842566666666667, + 5.860544, + 5.828630666666667, + 5.852502666666667, + 5.849946666666667, + 5.899273333333333 + ], + "seeded_digest": "fbbd124888e3f544a5f03b3e4e54f4f2482d58a243a5bfe56fe456a73e2036f1", + "class": "EMCCD" + }, + { + "preset": "andor_ixon_ultra_888", + "photons_per_subaperture": 2000.0, + "default_ms": 6.608646666666667, + "workspace_ms": 6.589113333333333, + "default_best_ms": 6.589908, + "workspace_best_ms": 6.575272, + "default_samples_ms": [ + 6.609772, + 6.599317333333333, + 6.619436, + 6.589908, + 6.625413333333333, + 6.605418666666667, + 6.6062813333333334, + 6.608646666666667, + 6.608805333333333 + ], + "workspace_samples_ms": [ + 6.5824213333333335, + 6.591257333333333, + 6.5766626666666665, + 6.60924, + 6.577658666666667, + 6.599725333333333, + 6.59126, + 6.589113333333333, + 6.575272 + ], + "seeded_digest": "a7c1805d252f20adc540ea195cad4797db3912a1e6b658a0b1c2682ca1bfa9bd", + "class": "EMCCD" + }, + { + "preset": "hamamatsu_orca_quest_2", + "photons_per_subaperture": 100.0, + "default_ms": 4.051733333333334, + "workspace_ms": 4.033830666666667, + "default_best_ms": 4.04312, + "workspace_best_ms": 4.030321333333333, + "default_samples_ms": [ + 4.051733333333334, + 4.05188, + 4.047348, + 4.051873333333334, + 4.053524, + 4.0488, + 4.04312, + 4.051908, + 4.049608 + ], + "workspace_samples_ms": [ + 4.034929333333333, + 4.030892, + 4.036178666666666, + 4.031504, + 4.037537333333334, + 4.038916, + 4.033570666666667, + 4.030321333333333, + 4.033830666666667 + ], + "seeded_digest": "1cf03881d1d39c2ab5b6e30b82e39892a007d2991fcf09c15b1eb066995dc7a8", + "class": "qCMOS" + }, + { + "preset": "hamamatsu_orca_quest_2", + "photons_per_subaperture": 2000.0, + "default_ms": 4.663429333333333, + "workspace_ms": 4.650585333333333, + "default_best_ms": 4.661558666666667, + "workspace_best_ms": 4.643638666666667, + "default_samples_ms": [ + 4.666753333333333, + 4.662716, + 4.661838666666667, + 4.663429333333333, + 4.662813333333333, + 4.665352, + 4.661558666666667, + 4.667297333333333, + 4.664256 + ], + "workspace_samples_ms": [ + 4.652048, + 4.649618666666667, + 4.656566666666667, + 4.644581333333333, + 4.656330666666667, + 4.643638666666667, + 4.651125333333333, + 4.645750666666667, + 4.650585333333333 + ], + "seeded_digest": "0e6d0e2157d35e1d1f865c07033735984c285dfb5540d4fc2007303f790569d0", + "class": "qCMOS" + }, + { + "preset": "generic_scmos", + "photons_per_subaperture": 100.0, + "default_ms": 4.300109333333333, + "workspace_ms": 4.313113333333333, + "default_best_ms": 4.295012, + "workspace_best_ms": 4.304610666666667, + "default_samples_ms": [ + 4.295012, + 4.300109333333333, + 4.297533333333333, + 4.302616, + 4.300334666666667, + 4.29686, + 4.297498666666667, + 4.303332, + 4.302894666666667 + ], + "workspace_samples_ms": [ + 4.313505333333333, + 4.304610666666667, + 4.313046666666667, + 4.311386666666667, + 4.31672, + 4.313414666666667, + 4.313113333333333, + 4.310670666666667, + 4.314138666666667 + ], + "seeded_digest": "ef998896caa9125df6d1a40f85c9d5e49638295633a8b9060305a0ca814250b9", + "class": "sCMOS" + }, + { + "preset": "generic_scmos", + "photons_per_subaperture": 2000.0, + "default_ms": 4.921825333333333, + "workspace_ms": 4.9328813333333335, + "default_best_ms": 4.914106666666667, + "workspace_best_ms": 4.926917333333333, + "default_samples_ms": [ + 4.921825333333333, + 4.923072, + 4.922116, + 4.929708, + 4.963449333333333, + 4.914106666666667, + 4.919642666666667, + 4.918409333333333, + 4.920436 + ], + "workspace_samples_ms": [ + 4.938830666666667, + 4.931568, + 4.940205333333333, + 4.931381333333333, + 4.940617333333333, + 4.931908, + 4.9328813333333335, + 4.926917333333333, + 4.942517333333333 + ], + "seeded_digest": "463b56f4b72341cdd802f99bec0cb4f32e730cf410eeb680eada605e62dddd54", + "class": "sCMOS" + }, + { + "preset": "first_light_imaging_cred_one", + "photons_per_subaperture": 100.0, + "default_ms": 7.393374666666667, + "workspace_ms": 7.378865333333333, + "default_best_ms": 7.384382666666667, + "workspace_best_ms": 7.375029333333333, + "default_samples_ms": [ + 7.384382666666667, + 7.388909333333333, + 7.3939173333333335, + 7.399049333333333, + 7.388978666666667, + 7.395128, + 7.3989, + 7.393374666666667, + 7.391273333333333 + ], + "workspace_samples_ms": [ + 7.390945333333333, + 7.376338666666667, + 7.385585333333333, + 7.378865333333333, + 7.422828, + 7.376926666666667, + 7.377902666666667, + 7.383341333333333, + 7.375029333333333 + ], + "seeded_digest": "a2f7ae9a4e61a8b024af38cc4d1526ee00390d957a99ad502e04f2f283aa477d", + "class": "eAPD" + }, + { + "preset": "first_light_imaging_cred_one", + "photons_per_subaperture": 2000.0, + "default_ms": 8.048434666666667, + "workspace_ms": 8.045786666666666, + "default_best_ms": 8.041469333333334, + "workspace_best_ms": 8.039514666666667, + "default_samples_ms": [ + 8.041469333333334, + 8.212052, + 8.148821333333332, + 8.044398666666668, + 8.058646666666666, + 8.048254666666667, + 8.048434666666667, + 8.069041333333333, + 8.043757333333334 + ], + "workspace_samples_ms": [ + 8.055090666666667, + 8.040212, + 8.071912, + 8.060202666666667, + 8.044589333333333, + 8.043854666666666, + 8.045786666666666, + 8.064254666666667, + 8.039514666666667 + ], + "seeded_digest": "b07ffc7dbc5e0c502dfeb26401a611f3715ee3823e5e54306fdf6cbf1c3c105b", + "class": "eAPD" + } + ] + }, + { + "schema_version": 1, + "getframes": "2.5.0", + "numpy": "2.5.3", + "platform": "Linux-6.17.9-76061709-generic-aarch64-with-glibc2.39", + "processor": "aarch64", + "shape": [ + 256, + 256 + ], + "exposure_s": 0.001, + "calls_per_block": 30, + "repetitions": 9, + "results": [ + { + "preset": "andor_ocam2k", + "photons_per_subaperture": 100.0, + "default_ms": 6.503658666666667, + "workspace_ms": 5.71714, + "default_best_ms": 6.493264, + "workspace_best_ms": 5.690414666666667, + "default_samples_ms": [ + 6.503658666666667, + 6.509126666666667, + 6.510332, + 6.499210666666667, + 6.509181333333333, + 6.493264, + 6.502746666666667, + 6.501836, + 6.590557333333333 + ], + "workspace_samples_ms": [ + 5.721722666666667, + 5.708834666666667, + 5.726144, + 5.71714, + 5.712317333333333, + 5.690414666666667, + 5.726074666666667, + 5.709846666666667, + 5.74212 + ], + "seeded_digest": "1712c1ff7389450f37a8dbb58f06a6c34b80e1bc325476a5b9a79de88b0ad36a", + "class": "EMCCD" + }, + { + "preset": "andor_ocam2k", + "photons_per_subaperture": 2000.0, + "default_ms": 7.508834666666667, + "workspace_ms": 6.683913333333333, + "default_best_ms": 7.485393333333333, + "workspace_best_ms": 6.662792, + "default_samples_ms": [ + 7.485642666666667, + 7.494745333333333, + 7.508834666666667, + 7.515384, + 7.515585333333333, + 7.490054666666667, + 7.485393333333333, + 7.577572, + 7.543753333333333 + ], + "workspace_samples_ms": [ + 6.696408, + 6.666474666666667, + 6.692708, + 6.669554666666667, + 6.706350666666667, + 6.662792, + 6.68434, + 6.670850666666667, + 6.683913333333333 + ], + "seeded_digest": "e1f649808cb6de3b0dfb99e09124b9639271816dce307f7ac3b48aee95f37826", + "class": "EMCCD" + }, + { + "preset": "andor_ixon_ultra_888", + "photons_per_subaperture": 100.0, + "default_ms": 5.732990666666667, + "workspace_ms": 5.733466666666667, + "default_best_ms": 5.723626666666667, + "workspace_best_ms": 5.720044, + "default_samples_ms": [ + 5.739978666666667, + 5.723626666666667, + 5.732990666666667, + 5.724158666666667, + 5.726322666666667, + 5.728633333333333, + 5.780792, + 5.733392, + 5.754228 + ], + "workspace_samples_ms": [ + 5.7211, + 5.733466666666667, + 5.726266666666667, + 5.735126666666667, + 5.737616, + 5.720044, + 5.754209333333333, + 5.720664, + 5.792268 + ], + "seeded_digest": "fbbd124888e3f544a5f03b3e4e54f4f2482d58a243a5bfe56fe456a73e2036f1", + "class": "EMCCD" + }, + { + "preset": "andor_ixon_ultra_888", + "photons_per_subaperture": 2000.0, + "default_ms": 6.709832, + "workspace_ms": 6.7028826666666665, + "default_best_ms": 6.672721333333333, + "workspace_best_ms": 6.683692, + "default_samples_ms": [ + 6.747277333333333, + 6.734096, + 6.725594666666667, + 6.693434666666667, + 6.716592, + 6.675358666666667, + 6.672721333333333, + 6.677949333333333, + 6.709832 + ], + "workspace_samples_ms": [ + 6.743784, + 6.765090666666667, + 6.771156, + 6.722465333333333, + 6.6941093333333335, + 6.683692, + 6.685496, + 6.693121333333333, + 6.7028826666666665 + ], + "seeded_digest": "a7c1805d252f20adc540ea195cad4797db3912a1e6b658a0b1c2682ca1bfa9bd", + "class": "EMCCD" + }, + { + "preset": "hamamatsu_orca_quest_2", + "photons_per_subaperture": 100.0, + "default_ms": 3.879944, + "workspace_ms": 3.866704, + "default_best_ms": 3.8774906666666666, + "workspace_best_ms": 3.860644, + "default_samples_ms": [ + 3.8868626666666666, + 3.8799146666666666, + 3.8818893333333335, + 3.879944, + 3.8774906666666666, + 3.879484, + 3.8777333333333335, + 3.8821213333333335, + 3.881428 + ], + "workspace_samples_ms": [ + 3.8737306666666664, + 3.8638573333333337, + 3.8741973333333335, + 3.8692053333333334, + 3.8638213333333336, + 3.861988, + 3.866704, + 3.860644, + 3.8688386666666665 + ], + "seeded_digest": "1cf03881d1d39c2ab5b6e30b82e39892a007d2991fcf09c15b1eb066995dc7a8", + "class": "qCMOS" + }, + { + "preset": "hamamatsu_orca_quest_2", + "photons_per_subaperture": 2000.0, + "default_ms": 4.497472, + "workspace_ms": 4.485634666666667, + "default_best_ms": 4.489986666666667, + "workspace_best_ms": 4.482970666666667, + "default_samples_ms": [ + 4.503889333333333, + 4.504270666666667, + 4.489986666666667, + 4.495193333333333, + 4.49652, + 4.497472, + 4.4966186666666665, + 4.498473333333333, + 4.502285333333333 + ], + "workspace_samples_ms": [ + 4.492266666666667, + 4.485504, + 4.487902666666667, + 4.485634666666667, + 4.488264, + 4.482970666666667, + 4.484102666666667, + 4.485244, + 4.488730666666667 + ], + "seeded_digest": "0e6d0e2157d35e1d1f865c07033735984c285dfb5540d4fc2007303f790569d0", + "class": "qCMOS" + }, + { + "preset": "generic_scmos", + "photons_per_subaperture": 100.0, + "default_ms": 4.15708, + "workspace_ms": 4.158226666666667, + "default_best_ms": 4.154192, + "workspace_best_ms": 4.152801333333334, + "default_samples_ms": [ + 4.156632, + 4.154585333333333, + 4.171885333333334, + 4.16448, + 4.163090666666666, + 4.156176, + 4.164176, + 4.154192, + 4.15708 + ], + "workspace_samples_ms": [ + 4.153445333333334, + 4.155096, + 4.16944, + 4.166196, + 4.161081333333334, + 4.158226666666667, + 4.161757333333333, + 4.152801333333334, + 4.158108 + ], + "seeded_digest": "ef998896caa9125df6d1a40f85c9d5e49638295633a8b9060305a0ca814250b9", + "class": "sCMOS" + }, + { + "preset": "generic_scmos", + "photons_per_subaperture": 2000.0, + "default_ms": 4.782804, + "workspace_ms": 4.782308, + "default_best_ms": 4.774609333333333, + "workspace_best_ms": 4.769554666666667, + "default_samples_ms": [ + 4.778953333333333, + 4.776558666666667, + 4.782804, + 4.792422666666667, + 4.783924, + 4.790244, + 4.78756, + 4.778473333333333, + 4.774609333333333 + ], + "workspace_samples_ms": [ + 4.781733333333333, + 4.78178, + 4.782966666666667, + 4.782308, + 4.782669333333333, + 4.781589333333333, + 4.790396, + 4.769554666666667, + 4.808062666666667 + ], + "seeded_digest": "463b56f4b72341cdd802f99bec0cb4f32e730cf410eeb680eada605e62dddd54", + "class": "sCMOS" + }, + { + "preset": "first_light_imaging_cred_one", + "photons_per_subaperture": 100.0, + "default_ms": 7.192026666666667, + "workspace_ms": 7.162134666666667, + "default_best_ms": 7.185874666666667, + "workspace_best_ms": 7.137673333333333, + "default_samples_ms": [ + 7.189873333333333, + 7.188753333333333, + 7.227754666666667, + 7.196601333333333, + 7.197906666666667, + 7.192026666666667, + 7.185874666666667, + 7.191812, + 7.208016 + ], + "workspace_samples_ms": [ + 7.182289333333333, + 7.189838666666667, + 7.145686666666667, + 7.162134666666667, + 7.146958666666667, + 7.137673333333333, + 7.155544, + 7.168869333333333, + 7.173724 + ], + "seeded_digest": "a2f7ae9a4e61a8b024af38cc4d1526ee00390d957a99ad502e04f2f283aa477d", + "class": "eAPD" + }, + { + "preset": "first_light_imaging_cred_one", + "photons_per_subaperture": 2000.0, + "default_ms": 8.391590666666666, + "workspace_ms": 8.361838666666667, + "default_best_ms": 8.369477333333332, + "workspace_best_ms": 8.346936, + "default_samples_ms": [ + 8.414972, + 8.391590666666666, + 8.445548, + 8.376862666666668, + 8.379434666666667, + 8.441904, + 8.388662666666667, + 8.369477333333332, + 8.394961333333335 + ], + "workspace_samples_ms": [ + 8.350528, + 8.384026666666667, + 8.361838666666667, + 8.356468, + 8.363110666666667, + 8.402821333333334, + 8.357544, + 8.400412, + 8.346936 + ], + "seeded_digest": "b07ffc7dbc5e0c502dfeb26401a611f3715ee3823e5e54306fdf6cbf1c3c105b", + "class": "eAPD" + } + ] + }, + { + "schema_version": 1, + "getframes": "2.5.0", + "numpy": "2.5.3", + "platform": "Linux-6.17.9-76061709-generic-aarch64-with-glibc2.39", + "processor": "aarch64", + "shape": [ + 256, + 256 + ], + "exposure_s": 0.001, + "calls_per_block": 30, + "repetitions": 9, + "results": [ + { + "preset": "andor_ocam2k", + "photons_per_subaperture": 100.0, + "default_ms": 6.851937333333333, + "workspace_ms": 6.048213333333333, + "default_best_ms": 6.835146666666667, + "workspace_best_ms": 6.020274666666667, + "default_samples_ms": [ + 6.866328, + 6.892356, + 6.855862666666667, + 6.853557333333333, + 6.851937333333333, + 6.835146666666667, + 6.849772, + 6.840678666666667, + 6.844884 + ], + "workspace_samples_ms": [ + 6.067108, + 6.048652, + 6.040833333333333, + 6.044153333333333, + 6.054321333333333, + 6.023052, + 6.048868, + 6.020274666666667, + 6.048213333333333 + ], + "seeded_digest": "1712c1ff7389450f37a8dbb58f06a6c34b80e1bc325476a5b9a79de88b0ad36a", + "class": "EMCCD" + }, + { + "preset": "andor_ocam2k", + "photons_per_subaperture": 2000.0, + "default_ms": 7.851174666666667, + "workspace_ms": 7.017229333333333, + "default_best_ms": 7.843054666666667, + "workspace_best_ms": 7.006365333333333, + "default_samples_ms": [ + 7.843054666666667, + 7.847694666666667, + 7.847553333333333, + 7.851594666666667, + 7.858054666666667, + 7.851174666666667, + 7.855772, + 7.853192, + 7.847668 + ], + "workspace_samples_ms": [ + 7.03436, + 7.006365333333333, + 7.017229333333333, + 7.007628, + 7.023309333333333, + 7.012412, + 7.026274666666667, + 7.011417333333333, + 7.026496 + ], + "seeded_digest": "e1f649808cb6de3b0dfb99e09124b9639271816dce307f7ac3b48aee95f37826", + "class": "EMCCD" + }, + { + "preset": "andor_ixon_ultra_888", + "photons_per_subaperture": 100.0, + "default_ms": 6.032588, + "workspace_ms": 5.994197333333333, + "default_best_ms": 6.024069333333333, + "workspace_best_ms": 5.987308, + "default_samples_ms": [ + 6.031881333333333, + 6.032588, + 6.038137333333333, + 6.039316, + 6.031286666666667, + 6.024069333333333, + 6.02814, + 6.056876, + 6.062898666666667 + ], + "workspace_samples_ms": [ + 5.987308, + 5.989833333333333, + 5.997914666666667, + 5.988028, + 5.987626666666667, + 5.996932, + 5.997414666666667, + 5.994197333333333, + 6.087725333333333 + ], + "seeded_digest": "fbbd124888e3f544a5f03b3e4e54f4f2482d58a243a5bfe56fe456a73e2036f1", + "class": "EMCCD" + }, + { + "preset": "andor_ixon_ultra_888", + "photons_per_subaperture": 2000.0, + "default_ms": 7.010086666666667, + "workspace_ms": 7.003393333333333, + "default_best_ms": 6.990642666666667, + "workspace_best_ms": 6.980908, + "default_samples_ms": [ + 7.035486666666667, + 7.027397333333333, + 7.046693333333333, + 6.990642666666667, + 7.003142666666667, + 7.016250666666667, + 7.010086666666667, + 6.998418666666667, + 7.006872 + ], + "workspace_samples_ms": [ + 6.999356, + 7.020872, + 7.013176, + 7.015909333333333, + 7.003393333333333, + 7.030041333333333, + 6.984333333333333, + 6.985833333333333, + 6.980908 + ], + "seeded_digest": "a7c1805d252f20adc540ea195cad4797db3912a1e6b658a0b1c2682ca1bfa9bd", + "class": "EMCCD" + }, + { + "preset": "hamamatsu_orca_quest_2", + "photons_per_subaperture": 100.0, + "default_ms": 3.899388, + "workspace_ms": 3.8936413333333335, + "default_best_ms": 3.8919093333333334, + "workspace_best_ms": 3.885364, + "default_samples_ms": [ + 3.899388, + 3.903328, + 3.904056, + 3.8966893333333337, + 3.9009133333333335, + 3.901876, + 3.8977226666666667, + 3.8919093333333334, + 3.8943746666666663 + ], + "workspace_samples_ms": [ + 3.8964693333333336, + 3.8908986666666667, + 3.910868, + 3.9166813333333335, + 3.892416, + 3.8929346666666667, + 3.8936413333333335, + 3.8956093333333337, + 3.885364 + ], + "seeded_digest": "1cf03881d1d39c2ab5b6e30b82e39892a007d2991fcf09c15b1eb066995dc7a8", + "class": "qCMOS" + }, + { + "preset": "hamamatsu_orca_quest_2", + "photons_per_subaperture": 2000.0, + "default_ms": 4.528488, + "workspace_ms": 4.510888, + "default_best_ms": 4.525905333333333, + "workspace_best_ms": 4.506058666666667, + "default_samples_ms": [ + 4.525918666666667, + 4.527606666666667, + 4.532674666666667, + 4.529613333333333, + 4.530861333333333, + 4.528488, + 4.534104, + 4.525905333333333, + 4.528485333333333 + ], + "workspace_samples_ms": [ + 4.511466666666667, + 4.506058666666667, + 4.513970666666667, + 4.510789333333333, + 4.512544, + 4.510888, + 4.514946666666667, + 4.507617333333333, + 4.508634666666667 + ], + "seeded_digest": "0e6d0e2157d35e1d1f865c07033735984c285dfb5540d4fc2007303f790569d0", + "class": "qCMOS" + }, + { + "preset": "generic_scmos", + "photons_per_subaperture": 100.0, + "default_ms": 4.151524, + "workspace_ms": 4.147168, + "default_best_ms": 4.146898666666667, + "workspace_best_ms": 4.142334666666667, + "default_samples_ms": [ + 4.151524, + 4.148445333333333, + 4.146898666666667, + 4.1558546666666665, + 4.149953333333333, + 4.151929333333333, + 4.151869333333334, + 4.157070666666667, + 4.151484 + ], + "workspace_samples_ms": [ + 4.144330666666667, + 4.146181333333334, + 4.147168, + 4.1450933333333335, + 4.147408, + 4.147686666666666, + 4.150204, + 4.158362666666666, + 4.142334666666667 + ], + "seeded_digest": "ef998896caa9125df6d1a40f85c9d5e49638295633a8b9060305a0ca814250b9", + "class": "sCMOS" + }, + { + "preset": "generic_scmos", + "photons_per_subaperture": 2000.0, + "default_ms": 4.795356, + "workspace_ms": 4.792704, + "default_best_ms": 4.791754666666667, + "workspace_best_ms": 4.789384, + "default_samples_ms": [ + 4.794898666666667, + 4.799517333333333, + 4.795356, + 4.798588, + 4.796596, + 4.795073333333333, + 4.791754666666667, + 4.801144, + 4.794741333333333 + ], + "workspace_samples_ms": [ + 4.793777333333333, + 4.792032, + 4.792704, + 4.799004, + 4.796134666666667, + 4.789384, + 4.791953333333333, + 4.790249333333333, + 4.794913333333333 + ], + "seeded_digest": "463b56f4b72341cdd802f99bec0cb4f32e730cf410eeb680eada605e62dddd54", + "class": "sCMOS" + }, + { + "preset": "first_light_imaging_cred_one", + "photons_per_subaperture": 100.0, + "default_ms": 7.34186, + "workspace_ms": 7.312705333333333, + "default_best_ms": 7.336129333333333, + "workspace_best_ms": 7.309441333333333, + "default_samples_ms": [ + 7.352138666666667, + 7.34186, + 7.40412, + 7.344089333333333, + 7.349064, + 7.338712, + 7.336318666666667, + 7.336129333333333, + 7.340684 + ], + "workspace_samples_ms": [ + 7.334209333333333, + 7.312705333333333, + 7.348576, + 7.328677333333333, + 7.325502666666667, + 7.312461333333333, + 7.312490666666667, + 7.309441333333333, + 7.310730666666667 + ], + "seeded_digest": "a2f7ae9a4e61a8b024af38cc4d1526ee00390d957a99ad502e04f2f283aa477d", + "class": "eAPD" + }, + { + "preset": "first_light_imaging_cred_one", + "photons_per_subaperture": 2000.0, + "default_ms": 8.547050666666665, + "workspace_ms": 8.507422666666667, + "default_best_ms": 8.536856, + "workspace_best_ms": 8.501244, + "default_samples_ms": [ + 8.547196, + 8.540116, + 8.547193333333334, + 8.546905333333333, + 8.554873333333333, + 8.536856, + 8.547050666666665, + 8.548081333333334, + 8.545494666666666 + ], + "workspace_samples_ms": [ + 8.509337333333335, + 8.504996, + 8.507422666666667, + 8.514048, + 8.50788, + 8.502144, + 8.506845333333334, + 8.501244, + 8.513486666666665 + ], + "seeded_digest": "b07ffc7dbc5e0c502dfeb26401a611f3715ee3823e5e54306fdf6cbf1c3c105b", + "class": "eAPD" + } + ] + } + ], + "after": [ + { + "schema_version": 1, + "getframes": "2.6.0", + "numpy": "2.5.3", + "platform": "Linux-6.17.9-76061709-generic-aarch64-with-glibc2.39", + "processor": "aarch64", + "shape": [ + 256, + 256 + ], + "exposure_s": 0.001, + "calls_per_block": 30, + "repetitions": 9, + "results": [ + { + "preset": "andor_ocam2k", + "photons_per_subaperture": 100.0, + "default_ms": 5.875957333333333, + "workspace_ms": 5.089613333333333, + "default_best_ms": 5.854708, + "workspace_best_ms": 5.069008, + "default_samples_ms": [ + 5.878001333333333, + 5.893114666666667, + 5.869746666666667, + 5.875449333333333, + 5.875957333333333, + 5.869624, + 5.883485333333333, + 5.854708, + 5.888037333333333 + ], + "workspace_samples_ms": [ + 5.110493333333333, + 5.076633333333333, + 5.108417333333333, + 5.069008, + 5.089613333333333, + 5.081412, + 5.091997333333333, + 5.087652, + 5.090049333333333 + ], + "seeded_digest": "1712c1ff7389450f37a8dbb58f06a6c34b80e1bc325476a5b9a79de88b0ad36a", + "class": "EMCCD" + }, + { + "preset": "andor_ocam2k", + "photons_per_subaperture": 2000.0, + "default_ms": 6.944038666666667, + "workspace_ms": 6.09452, + "default_best_ms": 6.937333333333333, + "workspace_best_ms": 6.08168, + "default_samples_ms": [ + 6.937333333333333, + 6.940517333333333, + 6.951393333333333, + 6.942841333333333, + 6.944038666666667, + 6.943885333333333, + 6.947241333333333, + 6.947826666666667, + 6.9524333333333335 + ], + "workspace_samples_ms": [ + 6.101258666666667, + 6.08168, + 6.107292, + 6.086937333333333, + 6.102308, + 6.084130666666667, + 6.09452, + 6.083078666666667, + 6.100792 + ], + "seeded_digest": "e1f649808cb6de3b0dfb99e09124b9639271816dce307f7ac3b48aee95f37826", + "class": "EMCCD" + }, + { + "preset": "andor_ixon_ultra_888", + "photons_per_subaperture": 100.0, + "default_ms": 5.045970666666667, + "workspace_ms": 5.0444, + "default_best_ms": 5.039430666666667, + "workspace_best_ms": 5.040005333333333, + "default_samples_ms": [ + 5.039430666666667, + 5.042853333333333, + 5.0504, + 5.047777333333333, + 5.045902666666667, + 5.047994666666667, + 5.042085333333333, + 5.048690666666667, + 5.045970666666667 + ], + "workspace_samples_ms": [ + 5.042249333333333, + 5.040005333333333, + 5.046504, + 5.04146, + 5.051016, + 5.041712, + 5.046624, + 5.0444, + 5.0486613333333334 + ], + "seeded_digest": "fbbd124888e3f544a5f03b3e4e54f4f2482d58a243a5bfe56fe456a73e2036f1", + "class": "EMCCD" + }, + { + "preset": "andor_ixon_ultra_888", + "photons_per_subaperture": 2000.0, + "default_ms": 6.067198666666667, + "workspace_ms": 6.060594666666667, + "default_best_ms": 6.062293333333333, + "workspace_best_ms": 6.053577333333333, + "default_samples_ms": [ + 6.06408, + 6.067572, + 6.067198666666667, + 6.0687, + 6.065530666666667, + 6.068053333333333, + 6.064673333333333, + 6.062293333333333, + 6.072118666666667 + ], + "workspace_samples_ms": [ + 6.060594666666667, + 6.054469333333333, + 6.06154, + 6.053577333333333, + 6.062449333333333, + 6.062605333333333, + 6.055270666666667, + 6.05998, + 6.070856 + ], + "seeded_digest": "a7c1805d252f20adc540ea195cad4797db3912a1e6b658a0b1c2682ca1bfa9bd", + "class": "EMCCD" + }, + { + "preset": "hamamatsu_orca_quest_2", + "photons_per_subaperture": 100.0, + "default_ms": 4.062846666666666, + "workspace_ms": 4.056377333333334, + "default_best_ms": 4.052145333333334, + "workspace_best_ms": 4.048442666666666, + "default_samples_ms": [ + 4.062846666666666, + 4.056516, + 4.059996, + 4.062989333333333, + 4.059697333333333, + 4.052145333333334, + 4.064505333333334, + 4.074906666666666, + 4.070882666666667 + ], + "workspace_samples_ms": [ + 4.054593333333333, + 4.052185333333333, + 4.056377333333334, + 4.048442666666666, + 4.058518666666666, + 4.055854666666667, + 4.075948, + 4.06394, + 4.064668 + ], + "seeded_digest": "1cf03881d1d39c2ab5b6e30b82e39892a007d2991fcf09c15b1eb066995dc7a8", + "class": "qCMOS" + }, + { + "preset": "hamamatsu_orca_quest_2", + "photons_per_subaperture": 2000.0, + "default_ms": 4.672822666666667, + "workspace_ms": 4.666968, + "default_best_ms": 4.66676, + "workspace_best_ms": 4.660304, + "default_samples_ms": [ + 4.668242666666667, + 4.683545333333333, + 4.672822666666667, + 4.67066, + 4.6781093333333335, + 4.680898666666667, + 4.670486666666667, + 4.677234666666667, + 4.66676 + ], + "workspace_samples_ms": [ + 4.666136, + 4.660304, + 4.667084, + 4.663585333333333, + 4.672401333333333, + 4.664438666666667, + 4.671516, + 4.674150666666667, + 4.666968 + ], + "seeded_digest": "0e6d0e2157d35e1d1f865c07033735984c285dfb5540d4fc2007303f790569d0", + "class": "qCMOS" + }, + { + "preset": "generic_scmos", + "photons_per_subaperture": 100.0, + "default_ms": 4.316489333333333, + "workspace_ms": 4.317036, + "default_best_ms": 4.313798666666667, + "workspace_best_ms": 4.308366666666667, + "default_samples_ms": [ + 4.315266666666667, + 4.317125333333333, + 4.319930666666667, + 4.313798666666667, + 4.316546666666667, + 4.31646, + 4.318021333333333, + 4.314706666666667, + 4.316489333333333 + ], + "workspace_samples_ms": [ + 4.322644, + 4.308366666666667, + 4.321013333333333, + 4.310328, + 4.319374666666667, + 4.30998, + 4.318153333333333, + 4.309178666666667, + 4.317036 + ], + "seeded_digest": "ef998896caa9125df6d1a40f85c9d5e49638295633a8b9060305a0ca814250b9", + "class": "sCMOS" + }, + { + "preset": "generic_scmos", + "photons_per_subaperture": 2000.0, + "default_ms": 4.942017333333333, + "workspace_ms": 4.936712, + "default_best_ms": 4.936665333333333, + "workspace_best_ms": 4.931709333333333, + "default_samples_ms": [ + 4.94008, + 4.937198666666667, + 4.942017333333333, + 4.944002666666667, + 4.940106666666667, + 4.942656, + 4.942184, + 4.936665333333333, + 4.942344 + ], + "workspace_samples_ms": [ + 4.938102666666667, + 4.936712, + 4.939478666666667, + 4.932714666666667, + 4.940748, + 4.9323, + 4.938449333333333, + 4.931709333333333, + 4.934185333333333 + ], + "seeded_digest": "463b56f4b72341cdd802f99bec0cb4f32e730cf410eeb680eada605e62dddd54", + "class": "sCMOS" + }, + { + "preset": "first_light_imaging_cred_one", + "photons_per_subaperture": 100.0, + "default_ms": 6.919309333333333, + "workspace_ms": 6.910353333333333, + "default_best_ms": 6.894034666666667, + "workspace_best_ms": 6.873621333333333, + "default_samples_ms": [ + 6.969401333333333, + 6.894273333333333, + 6.894034666666667, + 6.907376, + 6.918564, + 6.932921333333333, + 6.923437333333333, + 6.919309333333333, + 6.928216 + ], + "workspace_samples_ms": [ + 6.908712, + 6.882538666666667, + 6.873621333333333, + 6.8798, + 6.914028, + 6.910353333333333, + 6.911874666666667, + 6.931542666666667, + 6.917162666666667 + ], + "seeded_digest": "a2f7ae9a4e61a8b024af38cc4d1526ee00390d957a99ad502e04f2f283aa477d", + "class": "eAPD" + }, + { + "preset": "first_light_imaging_cred_one", + "photons_per_subaperture": 2000.0, + "default_ms": 8.042124, + "workspace_ms": 8.012213333333333, + "default_best_ms": 8.027162666666667, + "workspace_best_ms": 8.006788, + "default_samples_ms": [ + 8.108925333333334, + 8.088550666666666, + 8.101573333333333, + 8.042124, + 8.029769333333332, + 8.0391, + 8.048668, + 8.039681333333332, + 8.027162666666667 + ], + "workspace_samples_ms": [ + 8.059298666666667, + 8.070214666666667, + 8.067170666666668, + 8.122434666666667, + 8.010376, + 8.009437333333333, + 8.006788, + 8.00976, + 8.012213333333333 + ], + "seeded_digest": "b07ffc7dbc5e0c502dfeb26401a611f3715ee3823e5e54306fdf6cbf1c3c105b", + "class": "eAPD" + } + ] + }, + { + "schema_version": 1, + "getframes": "2.6.0", + "numpy": "2.5.3", + "platform": "Linux-6.17.9-76061709-generic-aarch64-with-glibc2.39", + "processor": "aarch64", + "shape": [ + 256, + 256 + ], + "exposure_s": 0.001, + "calls_per_block": 30, + "repetitions": 9, + "results": [ + { + "preset": "andor_ocam2k", + "photons_per_subaperture": 100.0, + "default_ms": 5.899268, + "workspace_ms": 5.091632, + "default_best_ms": 5.876150666666667, + "workspace_best_ms": 5.071581333333333, + "default_samples_ms": [ + 5.876150666666667, + 5.889381333333333, + 5.900961333333333, + 5.902744, + 5.888001333333333, + 5.899268, + 5.881416, + 5.935686666666667, + 5.989366666666667 + ], + "workspace_samples_ms": [ + 5.108094666666667, + 5.076984, + 5.091632, + 5.071581333333333, + 5.094741333333333, + 5.07822, + 5.10038, + 5.07208, + 5.143962666666667 + ], + "seeded_digest": "1712c1ff7389450f37a8dbb58f06a6c34b80e1bc325476a5b9a79de88b0ad36a", + "class": "EMCCD" + }, + { + "preset": "andor_ocam2k", + "photons_per_subaperture": 2000.0, + "default_ms": 6.908548, + "workspace_ms": 6.112342666666667, + "default_best_ms": 6.884417333333333, + "workspace_best_ms": 6.082056, + "default_samples_ms": [ + 6.884417333333333, + 6.897189333333333, + 6.953568, + 6.929869333333333, + 6.885929333333333, + 6.903997333333333, + 6.939068, + 6.931669333333333, + 6.908548 + ], + "workspace_samples_ms": [ + 6.112342666666667, + 6.117176, + 6.090709333333333, + 6.110556, + 6.145774666666667, + 6.082056, + 6.105704, + 6.1157346666666665, + 6.127017333333333 + ], + "seeded_digest": "e1f649808cb6de3b0dfb99e09124b9639271816dce307f7ac3b48aee95f37826", + "class": "EMCCD" + }, + { + "preset": "andor_ixon_ultra_888", + "photons_per_subaperture": 100.0, + "default_ms": 5.087313333333333, + "workspace_ms": 5.088189333333333, + "default_best_ms": 5.070801333333333, + "workspace_best_ms": 5.074753333333333, + "default_samples_ms": [ + 5.086558666666667, + 5.159944, + 5.073778666666667, + 5.098830666666667, + 5.070801333333333, + 5.088012, + 5.092436, + 5.080665333333333, + 5.087313333333333 + ], + "workspace_samples_ms": [ + 5.085392, + 5.172941333333333, + 5.111808, + 5.074753333333333, + 5.099350666666667, + 5.079206666666667, + 5.092092, + 5.088189333333333, + 5.086093333333333 + ], + "seeded_digest": "fbbd124888e3f544a5f03b3e4e54f4f2482d58a243a5bfe56fe456a73e2036f1", + "class": "EMCCD" + }, + { + "preset": "andor_ixon_ultra_888", + "photons_per_subaperture": 2000.0, + "default_ms": 6.107408, + "workspace_ms": 6.076276, + "default_best_ms": 6.100104, + "workspace_best_ms": 6.071854666666667, + "default_samples_ms": [ + 6.109441333333333, + 6.100617333333333, + 6.107273333333333, + 6.103576, + 6.110710666666667, + 6.107764, + 6.107408, + 6.100104, + 6.108314666666667 + ], + "workspace_samples_ms": [ + 6.074530666666667, + 6.075352, + 6.096829333333333, + 6.073209333333333, + 6.077470666666667, + 6.071854666666667, + 6.076582666666667, + 6.083321333333333, + 6.076276 + ], + "seeded_digest": "a7c1805d252f20adc540ea195cad4797db3912a1e6b658a0b1c2682ca1bfa9bd", + "class": "EMCCD" + }, + { + "preset": "hamamatsu_orca_quest_2", + "photons_per_subaperture": 100.0, + "default_ms": 4.040629333333333, + "workspace_ms": 4.033134666666666, + "default_best_ms": 4.0369253333333335, + "workspace_best_ms": 4.026201333333334, + "default_samples_ms": [ + 4.040629333333333, + 4.0399986666666665, + 4.041578666666666, + 4.037248, + 4.041328, + 4.041166666666666, + 4.03872, + 4.044097333333333, + 4.0369253333333335 + ], + "workspace_samples_ms": [ + 4.047062666666666, + 4.031517333333333, + 4.039386666666666, + 4.033134666666666, + 4.042976, + 4.028518666666667, + 4.039576, + 4.026201333333334, + 4.030786666666667 + ], + "seeded_digest": "1cf03881d1d39c2ab5b6e30b82e39892a007d2991fcf09c15b1eb066995dc7a8", + "class": "qCMOS" + }, + { + "preset": "hamamatsu_orca_quest_2", + "photons_per_subaperture": 2000.0, + "default_ms": 4.672128, + "workspace_ms": 4.668113333333333, + "default_best_ms": 4.667069333333333, + "workspace_best_ms": 4.660844, + "default_samples_ms": [ + 4.669049333333333, + 4.676970666666667, + 4.681228, + 4.677313333333333, + 4.672128, + 4.670064, + 4.66812, + 4.667069333333333, + 4.672644 + ], + "workspace_samples_ms": [ + 4.668772, + 4.661590666666667, + 4.66464, + 4.668113333333333, + 4.666353333333333, + 4.670701333333333, + 4.673334666666667, + 4.660844, + 4.669333333333333 + ], + "seeded_digest": "0e6d0e2157d35e1d1f865c07033735984c285dfb5540d4fc2007303f790569d0", + "class": "qCMOS" + }, + { + "preset": "generic_scmos", + "photons_per_subaperture": 100.0, + "default_ms": 4.318957333333333, + "workspace_ms": 4.3181053333333335, + "default_best_ms": 4.310573333333333, + "workspace_best_ms": 4.312201333333333, + "default_samples_ms": [ + 4.310573333333333, + 4.323626666666667, + 4.318957333333333, + 4.366838666666667, + 4.382158666666667, + 4.320749333333333, + 4.317257333333333, + 4.318773333333333, + 4.314710666666667 + ], + "workspace_samples_ms": [ + 4.3181053333333335, + 4.316630666666667, + 4.319530666666667, + 4.312201333333333, + 4.319630666666667, + 4.316796, + 4.324796, + 4.32484, + 4.317757333333333 + ], + "seeded_digest": "ef998896caa9125df6d1a40f85c9d5e49638295633a8b9060305a0ca814250b9", + "class": "sCMOS" + }, + { + "preset": "generic_scmos", + "photons_per_subaperture": 2000.0, + "default_ms": 4.941013333333333, + "workspace_ms": 4.942528, + "default_best_ms": 4.933688, + "workspace_best_ms": 4.935266666666667, + "default_samples_ms": [ + 4.942682666666667, + 4.945497333333333, + 4.941013333333333, + 4.944372, + 4.933688, + 4.943868, + 4.935498666666667, + 4.938021333333333, + 4.939574666666667 + ], + "workspace_samples_ms": [ + 4.944913333333333, + 4.935757333333333, + 4.948830666666667, + 4.940406666666667, + 4.942528, + 4.93912, + 4.943121333333333, + 4.935266666666667, + 4.944846666666667 + ], + "seeded_digest": "463b56f4b72341cdd802f99bec0cb4f32e730cf410eeb680eada605e62dddd54", + "class": "sCMOS" + }, + { + "preset": "first_light_imaging_cred_one", + "photons_per_subaperture": 100.0, + "default_ms": 6.877953333333333, + "workspace_ms": 6.890629333333333, + "default_best_ms": 6.86664, + "workspace_best_ms": 6.881561333333333, + "default_samples_ms": [ + 6.890056, + 6.89234, + 6.912524, + 6.867893333333333, + 6.877953333333333, + 6.879068, + 6.873226666666667, + 6.871218666666667, + 6.86664 + ], + "workspace_samples_ms": [ + 6.890629333333333, + 6.883452, + 6.918774666666667, + 6.898386666666667, + 6.8911706666666666, + 6.895502666666667, + 6.888806666666667, + 6.888530666666667, + 6.881561333333333 + ], + "seeded_digest": "a2f7ae9a4e61a8b024af38cc4d1526ee00390d957a99ad502e04f2f283aa477d", + "class": "eAPD" + }, + { + "preset": "first_light_imaging_cred_one", + "photons_per_subaperture": 2000.0, + "default_ms": 8.006133333333333, + "workspace_ms": 8.022058666666666, + "default_best_ms": 7.993806666666667, + "workspace_best_ms": 7.998997333333333, + "default_samples_ms": [ + 8.001516, + 8.002428, + 8.020185333333332, + 8.004001333333333, + 8.011945333333333, + 8.014489333333334, + 7.993806666666667, + 8.015616, + 8.006133333333333 + ], + "workspace_samples_ms": [ + 8.018296, + 8.022058666666666, + 8.062354666666668, + 8.039532, + 8.020388, + 7.998997333333333, + 8.025022666666667, + 8.021586666666668, + 8.148622666666666 + ], + "seeded_digest": "b07ffc7dbc5e0c502dfeb26401a611f3715ee3823e5e54306fdf6cbf1c3c105b", + "class": "eAPD" + } + ] + }, + { + "schema_version": 1, + "getframes": "2.6.0", + "numpy": "2.5.3", + "platform": "Linux-6.17.9-76061709-generic-aarch64-with-glibc2.39", + "processor": "aarch64", + "shape": [ + 256, + 256 + ], + "exposure_s": 0.001, + "calls_per_block": 30, + "repetitions": 9, + "results": [ + { + "preset": "andor_ocam2k", + "photons_per_subaperture": 100.0, + "default_ms": 5.859732, + "workspace_ms": 5.069958666666667, + "default_best_ms": 5.836029333333333, + "workspace_best_ms": 5.053156, + "default_samples_ms": [ + 5.836029333333333, + 5.85948, + 5.868446666666667, + 5.859732, + 5.853466666666667, + 5.854070666666667, + 5.895164, + 5.926716, + 5.873170666666667 + ], + "workspace_samples_ms": [ + 5.07806, + 5.054112, + 5.069958666666667, + 5.053156, + 5.069629333333333, + 5.054769333333333, + 5.125826666666667, + 5.090264, + 5.082289333333333 + ], + "seeded_digest": "1712c1ff7389450f37a8dbb58f06a6c34b80e1bc325476a5b9a79de88b0ad36a", + "class": "EMCCD" + }, + { + "preset": "andor_ocam2k", + "photons_per_subaperture": 2000.0, + "default_ms": 6.885148, + "workspace_ms": 6.085561333333333, + "default_best_ms": 6.871874666666667, + "workspace_best_ms": 6.06712, + "default_samples_ms": [ + 6.871874666666667, + 6.882353333333333, + 6.929498666666667, + 6.923513333333333, + 6.885148, + 6.88458, + 6.94736, + 6.919684, + 6.883276 + ], + "workspace_samples_ms": [ + 6.085561333333333, + 6.088148, + 6.090757333333333, + 6.06712, + 6.0848786666666665, + 6.089764, + 6.084917333333333, + 6.068705333333333, + 6.096082666666667 + ], + "seeded_digest": "e1f649808cb6de3b0dfb99e09124b9639271816dce307f7ac3b48aee95f37826", + "class": "EMCCD" + }, + { + "preset": "andor_ixon_ultra_888", + "photons_per_subaperture": 100.0, + "default_ms": 5.048406666666667, + "workspace_ms": 5.057957333333333, + "default_best_ms": 5.044786666666667, + "workspace_best_ms": 5.050512, + "default_samples_ms": [ + 5.053094666666667, + 5.048406666666667, + 5.044786666666667, + 5.047476, + 5.047812, + 5.049832, + 5.048989333333333, + 5.050565333333333, + 5.047521333333333 + ], + "workspace_samples_ms": [ + 5.057957333333333, + 5.050512, + 5.056161333333333, + 5.058108, + 5.056793333333333, + 5.060682666666667, + 5.058350666666667, + 5.055752, + 5.059914666666667 + ], + "seeded_digest": "fbbd124888e3f544a5f03b3e4e54f4f2482d58a243a5bfe56fe456a73e2036f1", + "class": "EMCCD" + }, + { + "preset": "andor_ixon_ultra_888", + "photons_per_subaperture": 2000.0, + "default_ms": 6.0692346666666666, + "workspace_ms": 6.080126666666667, + "default_best_ms": 6.065032, + "workspace_best_ms": 6.073564, + "default_samples_ms": [ + 6.074214666666667, + 6.074486666666667, + 6.0692346666666666, + 6.068918666666667, + 6.069258666666667, + 6.070637333333333, + 6.065032, + 6.066761333333333, + 6.068942666666667 + ], + "workspace_samples_ms": [ + 6.080126666666667, + 6.07838, + 6.082052, + 6.073564, + 6.082613333333333, + 6.086921333333333, + 6.080185333333333, + 6.078472, + 6.08008 + ], + "seeded_digest": "a7c1805d252f20adc540ea195cad4797db3912a1e6b658a0b1c2682ca1bfa9bd", + "class": "EMCCD" + }, + { + "preset": "hamamatsu_orca_quest_2", + "photons_per_subaperture": 100.0, + "default_ms": 4.030298666666667, + "workspace_ms": 4.020641333333334, + "default_best_ms": 4.025474666666667, + "workspace_best_ms": 4.0166026666666665, + "default_samples_ms": [ + 4.02844, + 4.034662666666667, + 4.029290666666666, + 4.032276, + 4.032417333333333, + 4.030298666666667, + 4.025474666666667, + 4.029093333333334, + 4.0303786666666666 + ], + "workspace_samples_ms": [ + 4.026112, + 4.0166026666666665, + 4.025630666666666, + 4.023878666666667, + 4.020409333333333, + 4.019109333333334, + 4.020641333333334, + 4.01682, + 4.020786666666667 + ], + "seeded_digest": "1cf03881d1d39c2ab5b6e30b82e39892a007d2991fcf09c15b1eb066995dc7a8", + "class": "qCMOS" + }, + { + "preset": "hamamatsu_orca_quest_2", + "photons_per_subaperture": 2000.0, + "default_ms": 4.650109333333333, + "workspace_ms": 4.642453333333333, + "default_best_ms": 4.645369333333333, + "workspace_best_ms": 4.633158666666667, + "default_samples_ms": [ + 4.645369333333333, + 4.646489333333333, + 4.646270666666667, + 4.688729333333333, + 4.686024, + 4.6493693333333335, + 4.656086666666667, + 4.650109333333333, + 4.662428 + ], + "workspace_samples_ms": [ + 4.637644, + 4.63436, + 4.637042666666667, + 4.65006, + 4.680372, + 4.652329333333333, + 4.658672, + 4.642453333333333, + 4.633158666666667 + ], + "seeded_digest": "0e6d0e2157d35e1d1f865c07033735984c285dfb5540d4fc2007303f790569d0", + "class": "qCMOS" + }, + { + "preset": "generic_scmos", + "photons_per_subaperture": 100.0, + "default_ms": 4.295088, + "workspace_ms": 4.310002666666667, + "default_best_ms": 4.283933333333333, + "workspace_best_ms": 4.287437333333333, + "default_samples_ms": [ + 4.286709333333333, + 4.290589333333333, + 4.303544, + 4.295088, + 4.303358666666667, + 4.283933333333333, + 4.307001333333333, + 4.304248, + 4.284930666666667 + ], + "workspace_samples_ms": [ + 4.310002666666667, + 4.326126666666667, + 4.327218666666667, + 4.291090666666667, + 4.328833333333333, + 4.290956, + 4.3347533333333335, + 4.291986666666667, + 4.287437333333333 + ], + "seeded_digest": "ef998896caa9125df6d1a40f85c9d5e49638295633a8b9060305a0ca814250b9", + "class": "sCMOS" + }, + { + "preset": "generic_scmos", + "photons_per_subaperture": 2000.0, + "default_ms": 4.911070666666667, + "workspace_ms": 4.916312, + "default_best_ms": 4.908973333333333, + "workspace_best_ms": 4.906148, + "default_samples_ms": [ + 4.911018666666667, + 4.911789333333333, + 4.908973333333333, + 4.911070666666667, + 4.922149333333333, + 4.911212, + 4.910494666666667, + 4.916216, + 4.910356 + ], + "workspace_samples_ms": [ + 4.928176, + 4.912162666666667, + 4.921692, + 4.917348, + 4.916312, + 4.910781333333333, + 4.918253333333333, + 4.906148, + 4.913316 + ], + "seeded_digest": "463b56f4b72341cdd802f99bec0cb4f32e730cf410eeb680eada605e62dddd54", + "class": "sCMOS" + }, + { + "preset": "first_light_imaging_cred_one", + "photons_per_subaperture": 100.0, + "default_ms": 6.815244, + "workspace_ms": 6.827389333333333, + "default_best_ms": 6.811014666666667, + "workspace_best_ms": 6.819866666666667, + "default_samples_ms": [ + 6.829054666666667, + 6.814376, + 6.814569333333333, + 6.815244, + 6.811014666666667, + 6.82956, + 6.814930666666667, + 6.824945333333333, + 6.818408 + ], + "workspace_samples_ms": [ + 6.8365306666666665, + 6.825817333333333, + 6.831754666666667, + 6.827389333333333, + 6.839032, + 6.823916, + 6.8263, + 6.819866666666667, + 6.833465333333333 + ], + "seeded_digest": "a2f7ae9a4e61a8b024af38cc4d1526ee00390d957a99ad502e04f2f283aa477d", + "class": "eAPD" + }, + { + "preset": "first_light_imaging_cred_one", + "photons_per_subaperture": 2000.0, + "default_ms": 7.951216, + "workspace_ms": 7.968268, + "default_best_ms": 7.943809333333333, + "workspace_best_ms": 7.958084, + "default_samples_ms": [ + 7.944488, + 7.943809333333333, + 7.951216, + 7.944214666666667, + 7.945570666666667, + 8.049256, + 7.965186666666667, + 7.952957333333333, + 7.959449333333333 + ], + "workspace_samples_ms": [ + 7.968986666666667, + 7.958084, + 7.967110666666667, + 7.964636, + 7.968268, + 7.973502666666667, + 7.980802666666667, + 7.982842666666667, + 7.961750666666667 + ], + "seeded_digest": "b07ffc7dbc5e0c502dfeb26401a611f3715ee3823e5e54306fdf6cbf1c3c105b", + "class": "eAPD" + } + ] + } + ] + }, + "in_process_ab_ms": [ + { + "preset": "andor_ocam2k", + "photons_per_subaperture": 100, + "before_gain_stage_ms": 6.57, + "after_gain_stage_ms": 5.95, + "speedup": 1.104 + }, + { + "preset": "andor_ocam2k", + "photons_per_subaperture": 2000, + "before_gain_stage_ms": 7.36, + "after_gain_stage_ms": 6.91, + "speedup": 1.065 + }, + { + "preset": "first_light_imaging_cred_one", + "photons_per_subaperture": 100, + "before_gain_stage_ms": 7.16, + "after_gain_stage_ms": 6.63, + "speedup": 1.08 + }, + { + "preset": "first_light_imaging_cred_one", + "photons_per_subaperture": 2000, + "before_gain_stage_ms": 8.38, + "after_gain_stage_ms": 8.02, + "speedup": 1.045 + }, + { + "preset": "leonardo_saphira", + "photons_per_subaperture": 100, + "before_gain_stage_ms": 4.79, + "after_gain_stage_ms": 4.29, + "speedup": 1.117 + }, + { + "preset": "leonardo_saphira", + "photons_per_subaperture": 2000, + "before_gain_stage_ms": 6.41, + "after_gain_stage_ms": 6.13, + "speedup": 1.046 + }, + { + "preset": "generic_scmos", + "photons_per_subaperture": 100, + "before_gain_stage_ms": 4.32, + "after_gain_stage_ms": 4.32, + "speedup": 1.0 + }, + { + "preset": "generic_scmos", + "photons_per_subaperture": 2000, + "before_gain_stage_ms": 4.92, + "after_gain_stage_ms": 4.92, + "speedup": 1.0 + } + ], + "in_process_ab_method": "same process, old and new apply_gain_stage swapped in alternately, 15 interleaved blocks of 30 frames, best block", + "gain_stage_only_ms": [ + { + "photons_per_subaperture": 100, + "lit_fraction": 0.14, + "before": 1.426, + "after": 0.784 + }, + { + "photons_per_subaperture": 2000, + "lit_fraction": 0.304, + "before": 1.633, + "after": 1.206 + } + ], + "bit_identity": "seeded frame and truth SHA-256 digests identical for every case; additionally every preset x {double, single} at 48x64 over two fluxes and three seeds (x86, NumPy 2.2.6)" +} diff --git a/benchmarks/small-frame-results.md b/benchmarks/small-frame-results.md new file mode 100644 index 0000000..f1a13cc --- /dev/null +++ b/benchmarks/small-frame-results.md @@ -0,0 +1,85 @@ +# Small WFS frames on the CPU: 2.5.0 vs 2.6.0 + +Issue #10 asked for a faster path for small, frequent wavefront-sensor frames. +Raw data: [`small-frame-results.json`](small-frame-results.json); script: +[`bench_small_frames.py`](bench_small_frames.py). + +**Workload.** 256x256 frames, 1 ms exposure, a Shack-Hartmann spot pattern +(16x16 subapertures of 16x16 px, one Gaussian spot of sigma 1.5 px each) at 100 +(photon-starved) and 2000 photons per subaperture, seeded +`expose(..., include_truth=False)`. Ampere Neoverse-N1, one pinned core, NumPy +2.5.3; process CPU time per frame. + +## Where a frame's time goes + +Profiling `andor_ocam2k` at 100 photons per subaperture (5.8 ms per frame on +this host) puts about 85% of it in NumPy's samplers: the Poisson shot-noise draw +(1.9 ms), the clock-induced-charge Poisson draw (0.95 ms), the Gamma EM-gain +draw (0.76 ms after this change) and the Gaussian read-noise draw (0.79 ms). +Python and array overhead is about 0.4 ms. The draws come from one sequential +stream, and the library guarantees that a config, its inputs and a seed +reproduce the same frame, so the samplers cannot be swapped, threaded or +reordered without changing seeded output. + +## What changed + +NumPy's `Generator.gamma` returns exactly zero for a zero shape without +consuming a draw, but still visits each such pixel. The gain stage now samples +only the pixels that hold charge when they are a minority (a WFS's dark +background), and skips the `* 1` shape scaling at `F = sqrt(2)`. Same draws, +same order, same generator state afterwards: every seeded frame is bit-identical +to 2.5.0 (all presets, both precisions; `tests/test_wfs_calibration.py` asserts +it against a full-array Gamma draw). + +An exact vectorised replica of NumPy's small-rate Poisson sampler (for the CIC +draw) was also written and verified bit-identical, but it measured no faster +than `rng.poisson` on either x86 or Arm, so it was not kept. + +## Results + +Same process, old and new gain stage swapped in alternately, best of 15 +interleaved blocks of 30 frames: + +| Preset | Photons/subap | 2.5.0 (ms) | 2.6.0 (ms) | Speed-up | +| --- | ---: | ---: | ---: | ---: | +| `andor_ocam2k` (EMCCD) | 100 | 6.57 | 5.95 | 1.10x | +| `andor_ocam2k` (EMCCD) | 2000 | 7.36 | 6.91 | 1.07x | +| `first_light_imaging_cred_one` (eAPD) | 100 | 7.16 | 6.63 | 1.08x | +| `first_light_imaging_cred_one` (eAPD) | 2000 | 8.38 | 8.02 | 1.04x | +| `leonardo_saphira` (eAPD) | 100 | 4.79 | 4.29 | 1.12x | +| `leonardo_saphira` (eAPD) | 2000 | 6.41 | 6.13 | 1.05x | +| `generic_scmos` (no gain stage; control) | 100 | 4.32 | 4.32 | 1.00x | +| `generic_scmos` (no gain stage; control) | 2000 | 4.92 | 4.92 | 1.00x | + +The Gamma draw alone, on the OCAM2K frame: 1.43 → 0.78 ms (1.82x) with 14% of +pixels lit, 1.63 → 1.21 ms (1.35x) with 30% lit. + +Separate processes, best block over three interleaved before/after runs; the +last columns add the existing `workspace=`/`out=` reuse. Unchanged paths drift +by about 3% between processes on this host (the qCMOS and sCMOS rows run the +same code before and after): + +| Class | Preset | Photons/subap | 2.5.0 (ms) | 2.6.0 (ms) | Speed-up | 2.6.0 + workspace/out (ms) | vs 2.5.0 default | Seeded output | +| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | --- | +| EMCCD | `andor_ocam2k` | 100 | 6.49 | 5.84 | 1.11x | 5.05 | 1.28x | identical | +| EMCCD | `andor_ocam2k` | 2000 | 7.36 | 6.87 | 1.07x | 6.07 | 1.21x | identical | +| EMCCD | `andor_ixon_ultra_888` | 100 | 5.72 | 5.04 | 1.14x | 5.04 | 1.14x | identical | +| EMCCD | `andor_ixon_ultra_888` | 2000 | 6.59 | 6.06 | 1.09x | 6.05 | 1.09x | identical | +| qCMOS | `hamamatsu_orca_quest_2` | 100 | 3.88 | 4.03 | 0.96x | 4.02 | 0.97x | identical | +| qCMOS | `hamamatsu_orca_quest_2` | 2000 | 4.49 | 4.65 | 0.97x | 4.63 | 0.97x | identical | +| sCMOS | `generic_scmos` | 100 | 4.15 | 4.28 | 0.97x | 4.29 | 0.97x | identical | +| sCMOS | `generic_scmos` | 2000 | 4.77 | 4.91 | 0.97x | 4.91 | 0.97x | identical | +| eAPD | `first_light_imaging_cred_one` | 100 | 7.19 | 6.81 | 1.06x | 6.82 | 1.05x | identical | +| eAPD | `first_light_imaging_cred_one` | 2000 | 8.04 | 7.94 | 1.01x | 7.96 | 1.01x | identical | + +## Reading the result + +* A 256x256 OCAM2K-class frame costs about 5--7 ms of CPU here and about 4 ms on + an Intel i7-10700. The issue's 40 ms per frame covers more than the detector: + on the i7-10700 the prototype's whole noisy-measurement step (diffraction WFS + optics, `expose`, calibration and centroiding, 288x288 pixels) took 14 ms, of + which `expose` is about 5 ms. +* Within the reproducibility guarantee, the CPU chain is now at its sampler + floor. The remaining large lever is the GPU (`device="gpu"`, two fused + kernels; see `device-results*.md`), or a future opt-in sampler that trades + seeded compatibility for speed. diff --git a/docs/guides/wfs-closed-loop.md b/docs/guides/wfs-closed-loop.md new file mode 100644 index 0000000..662fd2b --- /dev/null +++ b/docs/guides/wfs-closed-loop.md @@ -0,0 +1,234 @@ +# A WFS camera in a closed loop + +An adaptive-optics loop reads its wavefront-sensor (WFS) camera once per step: +a small frame, a short exposure, often a photon-starved signal, and thousands of +frames per simulated second. This guide covers what changes when a `getframes` +camera plays that role: how exposure relates to the frame period, how to turn +ADU back into photons with [`Calibration`][getframes.calibrate.Calibration], what +noise to expect, which presets to start from, and how to keep each frame cheap. + +## One frame per loop step + +```python +import numpy as np +import getframes as gf + +cam = gf.Camera.from_preset("andor_ocam2k").with_config(resolution=(256, 256)) +cal = gf.Calibration.from_camera(cam, exposure=1e-3, n_darks=64, seed=0) + +photons = np.full(cam.resolution, 0.5) # your WFS optics: photons/pixel this frame +for step in range(100): + frame = cam.expose(photons / 1e-3, 1e-3, seed=step, include_truth=False) + image = cal.to_photons(frame) # photons/pixel, ready for the centroider + # ... slopes -> reconstructor -> DM -> next step's photons ... +``` + +`expose` takes a photon **rate** (photons/s/pixel), so an optics model that +returns photons per frame divides by the exposure. Seeding each step +(`seed=step`, or seeds derived from your own generator) makes every loop +iteration reproducible on its own. + +## Exposure versus frame period + +`getframes` simulates one integration of length `exposure` followed by one +readout: + +* **Photo-signal, dark current and detector glow scale with `exposure`.** The + photon rate is integrated over it. +* **Read noise, kTC/reset noise and clock-induced charge (CIC) are per frame**, + independent of exposure. At kHz rates CIC usually dominates the EMCCD dark + signal: the OCAM2K preset's 0.0049 e-/pixel/frame CIC is three times its + 1.6 e-/pixel/s dark current integrated over 1 ms. + +The **frame period** is the loop's business. A frame-transfer EMCCD such as the +OCAM2K integrates the next frame while it reads the last, so its exposure is the +whole frame period (1/2067 s at Keck's 2067 fps). A camera with dead time, +or a loop that runs slower than the camera, exposes for less than the period: +pass the integration time as `exposure`, not the period. Readout latency, +rolling-shutter timing and any delay between exposure and DM command are not +part of a single frame; model them in the loop. + +## From ADU to photons + +[`Calibration`][getframes.calibrate.Calibration] inverts the mean signal chain: + +```text +electrons = L^-1[(ADU - D) * g / G] / F (input-referred: collected in the pixel) +photons = electrons / QE +``` + +with `D` the master dark (the per-pixel **mean** of exposure-matched dark +frames: bias pedestal, structured bias, dark current, glow and CIC), `g` the +conversion gain in e-/ADU, `G` the EM or avalanche gain, `L` the detector +nonlinearity (inverted exactly by default), `F` an optional unit-mean flat, and +`QE` the quantum efficiency `expose` applied. It runs on NumPy or CuPy frames +and keeps the camera's working precision. + +```python +cal = gf.Calibration.from_camera(cam, exposure=1e-3, n_darks=64, seed=0) +cal.e_per_adu # input electrons per ADU: 28 / 600 for the OCAM2K +electrons = cal.to_electrons(frame) +photons = cal.to_photons(frame) # / config.quantum_efficiency +photons_700 = cal.to_photons(frame, wavelength_nm=700.0) # / qe_curve(700 nm) +bad = cal.saturated(frame) # pixels whose charge was clipped +``` + +`Camera.calibration(...)` is the same as `Calibration.from_camera(camera, ...)`. +For real data, or a dark built some other way, construct +`gf.Calibration(config, dark, flat=...)` directly. + +### How accurate is it? + +Every stage of the forward chain is linear in expectation (Poisson shot noise, +the Gamma gain stage, zero-mean read noise, round-to-nearest digitisation with +at least half an ADU of read noise), so the **mean** of `to_photons` over many +frames equals the incident photons. The test suite checks this for every WFS +detector class with a 64x64 Shack-Hartmann spot pattern averaged over 150 +frames: + +| Class | Presets tested | Tolerance on the mean calibrated flux | +| --- | --- | --- | +| EMCCD | `andor_ocam2k`, `andor_ixon_ultra_888` | 0.5% | +| qCMOS | `hamamatsu_orca_quest_2` | 0.5% | +| sCMOS | `generic_scmos` | 0.5% | +| eAPD | `generic_eapd`, `leonardo_saphira` | 0.5% | +| eAPD | `first_light_imaging_cred_one`, with a flat and a per-frame background from unlit pixels | 1% | +| CCD / CMOS | `generic_ccd`, `generic_cmos` | 0.5% | + +The C-RED One preset carries a 69 ADU per-frame common-mode readout term, about +6 input electrons on *every* pixel of a frame. It is zero-mean, so it does not +bias the calibration, but it dominates a frame's summed flux; a pipeline +removes it from reference or unlit pixels, as the test does. + +A **single frame** is much noisier than that. Its total flux scatters by the +shot noise times the excess-noise factor plus the read noise summed over every +pixel: for a 288x288 frame with 100 photons per subaperture (about 20,000 +photons in all) the per-frame scatter of the total is 1.2% for the OCAM2K, 1.0% +for the ORCA-Quest 2 and 3.3% for the generic sCMOS, whose 1.6 e- read noise +summed over 83,000 pixels rivals the signal. Averaged over 300 frames the same +prototype recovers 1.002, 0.999 and 1.001 of the incident flux. + +!!! note "Where a 1--2% calibrated-flux offset comes from" + With a mean master dark, the matching scalar QE and no saturation, the + calibrated flux of `expose` frames has no deterministic offset; a 1--2% + deficit on a handful of frames is the per-frame scatter above. Offsets that + *are* deterministic come from the pipeline or the physics, and each has a + remedy: + + * **A median master dark** (`Camera.master_dark`'s default) misses the mean + of a skewed dark: EM-amplified CIC, hot pixels, RTS. The calibrated flux + then reads *high* by the missing mean, about 8% at 0.05 photons/pixel on + the OCAM2K. `Calibration.from_camera` always averages. + * **The QE.** `expose` applies the scalar `quantum_efficiency`; spectral mode + applies the `qe_curve`. They can differ by tens of percent (the + ORCA-Quest 2 scalar 0.85 is its peak; its curve gives 0.50 at 700 nm), so + pass `wavelength_nm=` or `bandpass=` to `to_photons` for spectral frames. + * **Nonlinearity** bends bright pixels low (`generic_scmos`: 0.67% at 2/3 of + full well). `Calibration` inverts it unless `linearize=False`. + * **Saturation.** Charge above the input well, the output-register ceiling + (`output_full_well_e / em_gain`: 467 e- per pixel for the OCAM2K at gain + 600) or the ADC range is lost; `saturated()` flags those pixels. + * **Fixed patterns.** PRNU and gain maps average out over a frame but not + over a few-pixel spot; pass a `flat` (for example + `cam.master_flat(..., bias=cal.dark_adu, method="mean")`). + +## Noise referred to the input + +[`Camera.effective_read_noise_e`][getframes.camera.Camera.effective_read_noise_e] +returns the per-pixel read noise of one ordinary frame in electrons *collected +in the pixel*: the output-amplifier read noise (the quadrature mean of a +per-pixel sCMOS map), kTC/reset noise and any avalanche-scaled term, all divided +by the EM/avalanche gain: + +```python +gf.Camera.from_preset("andor_ocam2k").effective_read_noise_e() # 0.36 e- +``` + +A gain stage also multiplies the *shot* noise of everything it amplifies by the +excess noise factor `F` (`config.gain_excess_noise_factor`): `sqrt(2)` for an +EMCCD at high gain, about 1.2--1.25 for HgCdTe eAPDs. In input electrons the +per-pixel signal-to-noise of a signal `S` on a dark-plus-CIC level `D` is + +```text +SNR = S / sqrt(F^2 (S + D) + sigma_eff^2) +``` + +so an EMCCD buys its sub-electron read noise at the price of halving the +effective QE (`F^2 = 2`) for shot-noise-limited pixels. At equal QE a +gain-free detector with read noise `sigma_1` beats an EMCCD with `sigma_E` once +`S + D > (sigma_1^2 - sigma_E^2) / (F^2 - 1)`: about 0.06 e- per pixel for the +ORCA-Quest 2 (0.44 e-) against the OCAM2K (0.36 e-), about 3 e- for a 1.75 e- +sCMOS. + +## Recommended presets + +| Class | Preset | Format | QE | Effective read noise | F | Notes | +| --- | --- | --- | ---: | ---: | ---: | --- | +| EMCCD | `andor_ocam2k` | 240x240, 24 µm | 0.90 | 0.36 e- | 1.41 | Keck-characterised CCD220 at gain 600, 2 kHz; CIC 0.0049 e-/frame | +| EMCCD | `andor_ixon_ultra_888` | 1024x1024, 13 µm | 0.95 | 0.15 e- | 1.41 | General-purpose EMCCD at gain 300 | +| qCMOS | `hamamatsu_orca_quest_2` | 4096x2304, 4.6 µm | 0.85 | 0.44 e- | 1.00 | Photon-number-resolving read noise; use a small `resolution` (see below) | +| sCMOS | `andor_marana_4_2b_11`, `photometrics_prime_95b` | 11 µm | 0.95 | 1.16, 1.57 e- | 1.00 | Characterised against real dark stacks; per-pixel read noise and RTS | +| eAPD | `first_light_imaging_cred_one` | 256x320, 24 µm | 0.70 | 2.6 e- | 1.21 | HgCdTe infrared WFS; PRNU 5.5% and a gain map (use a flat); 69 ADU common mode | +| eAPD | `leonardo_saphira` | 256x320, 24 µm | 0.70 | 1.16 e- | 1.25 | Avalanche gain 50 | +| CCD | `scimeasure_little_joe_ccd39` | 80x80, 24 µm | 0.905 | 4.5 e- | 1.00 | The classic CCD39 WFS camera | + +Effective read noise is `Camera.effective_read_noise_e()` for the preset as +shipped (for a per-pixel sCMOS map it exceeds the nominal `read_noise_e`, which +is the mean rather than the quadrature mean of the map). + +!!! warning "The Nüvü EMCCD presets saturate at one electron" + `nuvu_hnu_240` and `nuvu_hnu_128_omega` set `full_well_e` to 500 and 800 e- + and no `output_full_well_e`, so that value also caps the *amplified* charge + at the digitiser. At EM gain 1000 a pixel holding a single electron + saturates, and calibrated fluxes come out a few percent of the truth. Until + the presets carry a sourced output-register capacity, set one yourself, e.g. + `with_config(output_full_well_e=...)`, or use `andor_ocam2k`, which models + the same CCD220 with Keck-measured input and output wells. Gains and read modes are preset values; override them with +`with_config(em_gain=..., read_noise_e=...)` to explore a camera's operating +range (the input-referred noise of an EMCCD scales as `read_noise_e / em_gain`). + +## Keeping frames cheap + +A closed-loop study can spend most of its time in `expose`. On the CPU a frame +is bound by NumPy's Poisson, Gamma and Gaussian samplers, so: + +* **Simulate the WFS format, not the whole sensor.** `with_config(resolution=(256, + 256))` builds a 256x256 detector. A detector `roi` is different: it simulates + the full sensor (amplifier geometry, fixed patterns) and then crops, which is + right for sensor studies but costs a full-sensor frame each step. +* **Shrink the config before building the camera** for a large-format preset: + `gf.Camera(gf.load_preset("hamamatsu_orca_quest_2").replace(resolution=(256, + 256)))`. `Camera.from_preset(...).with_config(...)` first builds the full + sensor's fixed-pattern maps (4096x2304 for the ORCA-Quest 2), which costs + seconds and, on a GPU, hundreds of megabytes. +* **`include_truth=False`** skips the noise-free truth arrays. +* **Reuse a [`DetectorWorkspace`][getframes.noise.DetectorWorkspace] and an + `out` frame** (`expose(..., workspace=ws, out=buf)`) to avoid per-frame + allocations: 13% for the OCAM2K frames below, little for the other presets. +* **`device="gpu"`** runs the chain as two fused CUDA kernels; see the + [GPU guide](gpu.md#small-frames-fused-kernels) for frames/s on several GPUs. + +On the CPU a 256x256 frame costs about 4--7 ms (an Intel i7-10700 and an Ampere +Neoverse-N1 core), about 85% of it in NumPy's Poisson, Gamma and Gaussian +samplers. Their draws come from one sequential stream, and a seeded frame must +stay reproducible, so they cannot be swapped or threaded. Since 2.6.0 the Gamma +gain stage samples only the pixels that hold charge, which NumPy's sampler would +otherwise visit one by one to return zero; the draws, their order and the +generator state are unchanged, so seeded frames are bit-identical to 2.5.0. +Measured with `benchmarks/bench_small_frames.py` (1 ms, seeded, +`include_truth=False`, one Neoverse-N1 core): + +| Preset | Photons/subaperture | 2.5.0 (ms) | 2.6.0 (ms) | 2.6.0 + workspace/out (ms) | +| --- | ---: | ---: | ---: | ---: | +| `andor_ocam2k` | 100 | 6.49 | 5.84 | 5.05 | +| `andor_ocam2k` | 2000 | 7.36 | 6.87 | 6.07 | +| `andor_ixon_ultra_888` | 100 | 5.72 | 5.04 | 5.04 | +| `first_light_imaging_cred_one` | 100 | 7.19 | 6.81 | 6.82 | +| `hamamatsu_orca_quest_2` | 100 | 3.88 | 4.03 | 4.02 | +| `generic_scmos` | 100 | 4.15 | 4.28 | 4.29 | + +EMCCD and eAPD frames gain 5--14%; presets without a gain stage run the same +code as before (their differences are run-to-run drift). The full tables, an +in-process A/B and the raw data are in +[`benchmarks/small-frame-results.md`](https://github.com/jacotay7/getframes/blob/main/benchmarks/small-frame-results.md). +For kHz-rate studies that need much more, use the GPU. diff --git a/mkdocs.yml b/mkdocs.yml index 685ed16..38a7538 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -36,6 +36,7 @@ nav: - Radiometry & the infrared: guides/radiometry.md - Scale & datasets: guides/datasets.md - GPU detector execution: guides/gpu.md + - A WFS camera in a closed loop: guides/wfs-closed-loop.md - Camera presets: guides/presets.md - Detector characterisation: guides/characterization.md - Validation: guides/validation.md diff --git a/src/getframes/__about__.py b/src/getframes/__about__.py index b99b9ac..61b2727 100644 --- a/src/getframes/__about__.py +++ b/src/getframes/__about__.py @@ -1,4 +1,4 @@ # SPDX-License-Identifier: MIT """Single source of truth for the package version.""" -__version__ = "2.5.0" +__version__ = "2.6.0" diff --git a/src/getframes/__init__.py b/src/getframes/__init__.py index e6df6b8..bf7c185 100644 --- a/src/getframes/__init__.py +++ b/src/getframes/__init__.py @@ -22,7 +22,7 @@ ramp_photon_transfer, ) from .backend import ArrayBackend, get_array_module, get_backend, resolve_precision, to_numpy -from .calibrate import calibrate, combine +from .calibrate import Calibration, calibrate, combine from .camera import Camera from .config import CameraConfig, SensorType from .frame import Frame, FrameTruth @@ -63,6 +63,7 @@ "ArrayBackend", "ArrayPSF", "Bandpass", + "Calibration", "Camera", "CameraConfig", "Catalog", diff --git a/src/getframes/calibrate.py b/src/getframes/calibrate.py index fe80e10..7451c32 100644 --- a/src/getframes/calibrate.py +++ b/src/getframes/calibrate.py @@ -13,6 +13,11 @@ pedestal, so subtracting it removes bias and dark current together) and ``flat`` is a pedestal-free master flat (see :meth:`getframes.Camera.master_flat`). Pass ``bias`` instead of ``dark`` to subtract only the bias pedestal. + +:class:`Calibration` goes one step further for wavefront-sensor style use: it +turns raw ADU into input-referred electrons and incident photons by inverting the +mean signal chain (mean master dark, conversion and EM/avalanche gain, detector +nonlinearity, optional flat, quantum efficiency), on NumPy or CuPy frames. """ from __future__ import annotations @@ -22,11 +27,17 @@ import numpy as np +from .backend import _array_device, get_array_module, to_numpy from .frame import Frame +from .noise import _input_referred_read_noise_e if TYPE_CHECKING: from numpy.typing import NDArray + from .camera import Camera + from .config import CameraConfig + from .spectral import SED, SpectralBandpass + FrameLike = Frame | NDArray[np.floating[Any] | np.integer[Any]] _COMBINE_METHODS = ("mean", "median", "sigma_clip") @@ -179,4 +190,426 @@ def calibrate( return Frame(data=out, metadata=metadata) -__all__ = ["calibrate", "combine"] +def _input_array(frame: Any) -> Any: + """The pixel array of a :class:`Frame` or array, kept on its own device.""" + return frame.data if isinstance(frame, Frame) else frame + + +class Calibration: + r"""Convert raw frames (ADU) to input-referred electrons and incident photons. + + A wavefront-sensor pipeline needs the photon (or electron) image a camera + saw, not its ADU. :class:`Calibration` inverts the *mean* signal chain of + :func:`getframes.noise.simulate_frame` pixel by pixel:: + + electrons = L^-1[(ADU - D) * g / G] / F + photons = electrons / QE + + ``D`` + The master dark (ADU): the per-pixel **mean** of exposure-matched dark + frames. It carries the bias pedestal, amplifier offsets, structured bias, + dark current, detector glow and clock-induced charge, which are all + additive in expectation, so one subtraction removes them together. + ``g`` + ``config.gain_e_per_adu``, output electrons per ADU. + ``G`` + ``config.em_gain``, the mean multiplication of an EM register or + avalanche stage (``1`` without one). Dividing by it refers the signal to + the input, i.e. to electrons collected in the pixel. + ``L`` + The detector nonlinearity (``config.nonlinearity`` or + ``config.nonlinearity_coeffs``), which bends collected charge before any + gain stage. With ``linearize=True`` (default) it is inverted exactly: in + closed form for the single-parameter model, by Newton iteration for the + polynomial. Signal below zero (read noise on an empty pixel) is left + linear, as in the forward model. + ``F`` + An optional unit-mean flat-field map (PRNU, amplifier-gain differences, + avalanche-gain non-uniformity, vignetting). ``None`` divides by one. + ``QE`` + The quantum efficiency :meth:`getframes.Camera.expose` applied: + ``config.quantum_efficiency`` by default; see :meth:`to_photons` for + frames made in spectral mode. + + **Bias.** Every stage of the forward chain is linear in expectation: Poisson + shot noise and clock-induced charge, the Gamma gain stage (mean ``n G``), + zero-mean Gaussian reset/read noise, and round-to-nearest digitisation + (unbiased once the read noise is at least about half an ADU, true of every + preset). The mean of :meth:`to_photons` over many frames therefore equals + ``photon_rate * exposure`` up to statistics. The deterministic departures + are physical or procedural, not part of the model inverse: + + * **Saturation.** Charge above the input full well, the output-register + ceiling (``output_full_well_e / G``) or the ADC range is lost; see + :meth:`saturated`. + * **A median master dark.** :meth:`getframes.Camera.master_dark` defaults + to a median, which drops the mean of a skewed dark distribution + (EM-amplified clock-induced charge, hot pixels, RTS) and so biases the + calibrated flux *high* by that mean. :meth:`from_camera` always averages. + * **Fixed patterns without a flat.** PRNU and gain maps average out over + many pixels but not over a few-pixel spot; pass ``flat`` to remove them. + * **Scalar vs. spectral QE**, when the frame was made in spectral mode. + * **Dark charge seen through the nonlinearity**: the dark is subtracted + before linearising, an error of order ``nonlinearity * dark / full_well``. + + Parameters + ---------- + config: + The detector configuration the frames come from. + dark: + Master dark in ADU (a :class:`~getframes.frame.Frame` or a NumPy/CuPy + array) shaped like the frames, exposure-matched, mean-combined. Its array + module and floating dtype set where and in which precision the + conversions run (integer darks are promoted to ``float64``). + flat: + Optional flat-field response map shaped like the frames. It is normalised + to unit mean; zero or non-finite pixels (dead) are left unscaled. + linearize: + Invert the configured nonlinearity (default ``True``). + binning: + Binning factor of the frames (and of ``dark``). Binned frames sum + ``binning**2`` native pixels, so the nonlinearity is inverted against a + ``binning**2`` larger well: exact for on-chip binning and for uniform + light under digital binning, approximate for structured light. + exposure_s: + Exposure the dark was taken at (seconds), recorded for provenance. + n_darks: + Number of frames averaged into ``dark``, recorded for provenance. + + Examples + -------- + >>> import getframes as gf + >>> cam = gf.Camera.from_preset("andor_ocam2k").with_config(resolution=(64, 64)) + >>> cal = gf.Calibration.from_camera(cam, exposure=1e-3, n_darks=16, seed=0) + >>> frame = cam.expose(2.0e4, 1e-3, seed=1) + >>> photons = cal.to_photons(frame) # about 20 photons per pixel + """ + + def __init__( + self, + config: CameraConfig, + dark: FrameLike, + *, + flat: FrameLike | None = None, + linearize: bool = True, + binning: int = 1, + exposure_s: float | None = None, + n_darks: int | None = None, + ) -> None: + if binning < 1: + raise ValueError("binning must be a positive integer.") + dark_array = _input_array(dark) + xp = get_array_module(dark_array) + with _array_device(dark_array): + dark_array = xp.asarray(dark_array) + if dark_array.ndim != 2: + raise ValueError("dark must be a 2-D array.") + if not np.issubdtype(dark_array.dtype, np.floating): + dark_array = dark_array.astype(np.float64) + flat_array = None + if flat is not None: + flat_array = xp.asarray(_input_array(flat), dtype=dark_array.dtype) + if flat_array.shape != dark_array.shape: + raise ValueError( + f"flat shape {tuple(flat_array.shape)} does not match dark shape " + f"{tuple(dark_array.shape)}." + ) + good = xp.isfinite(flat_array) & (flat_array > 0) + norm = float(flat_array[good].mean()) if bool(good.any()) else 0.0 + if not norm > 0.0: + raise ValueError("flat has no positive finite pixels; cannot normalise.") + flat_array = xp.where(good, flat_array / norm, 1.0).astype(dark_array.dtype) + self.config = config + self.dark_adu = dark_array + self.flat = flat_array + self.linearize = bool(linearize) + self.binning = int(binning) + self.exposure_s = exposure_s + self.n_darks = n_darks + + @classmethod + def from_camera( + cls, + camera: Camera, + exposure: float, + n_darks: int = 64, + temperature: float | None = None, + *, + seed: int | None = None, + binning: int = 1, + binning_mode: str = "digital", + flat: FrameLike | None = None, + linearize: bool = True, + ) -> Calibration: + """Build a calibration from ``n_darks`` exposure-matched dark frames. + + The master dark is the per-pixel **mean** of ``n_darks`` dark frames from + :meth:`getframes.Camera.expose` at zero photon rate, accumulated in + ``float64`` on the camera's device and stored in its working precision. + Its noise adds ``1/n_darks`` of a frame's dark variance to every + calibrated frame, as a fixed pattern; 64 keeps that a few percent of a + frame's own read noise. + + Parameters + ---------- + camera: + The camera whose frames will be calibrated. + exposure: + Exposure of the frames to calibrate, in seconds (dark current and + glow scale with it; clock-induced charge does not). ``0`` gives a + master bias. + n_darks: + Number of dark frames to average. + temperature: + Sensor temperature in degrees Celsius; defaults to the camera's. + seed: + Seed of the dark series. Each frame gets a distinct derived seed, so + the master dark is reproducible; ``None`` advances the camera's + generator. + binning, binning_mode: + Binning of the frames to calibrate, as passed to + :meth:`getframes.Camera.expose`. + flat, linearize: + As for :class:`Calibration`. + """ + if n_darks < 1: + raise ValueError("n_darks must be >= 1.") + backend = camera._backend + with backend.activate(): + total: Any = None + for frame_seed in camera._series_seeds(seed, n_darks): + frame = camera.expose( + 0.0, + exposure, + temperature, + binning=binning, + binning_mode=binning_mode, + seed=frame_seed, + include_truth=False, + ) + if total is None: + total = backend.xp.zeros(frame.data.shape, dtype=np.float64) + total += frame.data + dark = (total / n_darks).astype(camera._float_dtype) + return cls( + camera.config, + dark, + flat=flat, + linearize=linearize, + binning=binning, + exposure_s=exposure, + n_darks=n_darks, + ) + + @property + def e_per_adu(self) -> float: + """Input-referred electrons per ADU: ``gain_e_per_adu / em_gain``.""" + return self.config.gain_e_per_adu / self._gain_stage + + @property + def _gain_stage(self) -> float: + return self.config.em_gain if self.config.has_gain_stage else 1.0 + + def _maps(self, xp: Any) -> tuple[Any, Any | None]: + """The dark and flat on ``xp`` (copied explicitly when the device differs).""" + if get_array_module(self.dark_adu) is xp: + return self.dark_adu, self.flat + if xp is np: + flat = None if self.flat is None else to_numpy(self.flat) + return to_numpy(self.dark_adu), flat + flat = None if self.flat is None else xp.asarray(self.flat) + return xp.asarray(self.dark_adu), flat + + def _signal_e(self, frame: FrameLike) -> tuple[Any, Any]: + """Dark-subtracted, input-referred electrons before linearising (and ``xp``).""" + data = _input_array(frame) + xp = get_array_module(data) + dark, _ = self._maps(xp) + data = xp.asarray(data) + if tuple(data.shape) != tuple(dark.shape): + raise ValueError( + f"frame shape {tuple(data.shape)} does not match the calibration's " + f"{tuple(dark.shape)}." + ) + signal = data.astype(dark.dtype) + signal -= dark + signal *= dark.dtype.type(self.e_per_adu) + return signal, xp + + def to_electrons(self, frame: FrameLike) -> Any: + """Convert a raw frame (ADU) to input-referred electrons per pixel. + + Applies ``L^-1[(ADU - D) * g / G] / F`` (see the class docstring). The + result is on the frame's device (NumPy or CuPy) in the calibration's + floating precision. Read noise and dark-subtraction noise make empty + pixels scatter about zero, including below it. + """ + data = _input_array(frame) + with _array_device(data): + signal, xp = self._signal_e(data) + if self.linearize: + signal = self._linearize(signal, xp) + _, flat = self._maps(xp) + if flat is not None: + signal /= flat + return signal + + def to_photons( + self, + frame: FrameLike, + *, + quantum_efficiency: float | None = None, + wavelength_nm: float | None = None, + bandpass: SpectralBandpass | None = None, + sed: SED | None = None, + ) -> Any: + """Convert a raw frame (ADU) to incident photons per pixel per exposure. + + :meth:`to_electrons` divided by a quantum efficiency, chosen as: + + * ``quantum_efficiency``, if given; + * else the config's ``qe_curve`` at ``wavelength_nm`` (monochromatic + light), or its effective value over ``bandpass`` for a source ``sed`` + (:func:`getframes.spectral.effective_qe`, flat photon spectrum when + ``sed`` is ``None``); + * else ``config.quantum_efficiency`` --- the scalar + :meth:`getframes.Camera.expose` applies, so frames from ``expose`` + calibrate back to ``photon_rate * exposure`` with no further argument. + + Frames made in spectral mode (:meth:`~getframes.Camera.expose_spectral`, + or :meth:`~getframes.Camera.observe` with a ``qe_curve``) saw the curve, + not the scalar; pass ``wavelength_nm`` or ``bandpass`` for those. The two + can differ by tens of percent (e.g. the ORCA-Quest 2 scalar 0.85 is its + peak; its curve gives 0.50 at 700 nm). + """ + qe = self.quantum_efficiency( + quantum_efficiency=quantum_efficiency, + wavelength_nm=wavelength_nm, + bandpass=bandpass, + sed=sed, + ) + if qe <= 0.0: + raise ValueError("quantum efficiency must be positive to convert to photons.") + electrons = self.to_electrons(frame) + with _array_device(electrons): + electrons /= electrons.dtype.type(qe) + return electrons + + def quantum_efficiency( + self, + *, + quantum_efficiency: float | None = None, + wavelength_nm: float | None = None, + bandpass: SpectralBandpass | None = None, + sed: SED | None = None, + ) -> float: + """The quantum efficiency :meth:`to_photons` divides by (same arguments).""" + if quantum_efficiency is not None: + if wavelength_nm is not None or bandpass is not None: + raise ValueError("give quantum_efficiency, wavelength_nm or bandpass, not several.") + return float(quantum_efficiency) + if wavelength_nm is not None and bandpass is not None: + raise ValueError("give wavelength_nm or bandpass, not both.") + if wavelength_nm is None and bandpass is None: + if sed is not None: + raise ValueError("sed needs a bandpass.") + return float(self.config.quantum_efficiency) + curve = self.config.qe_curve + if curve is None: + raise ValueError( + f"{self.config.name!r} has no qe_curve; pass quantum_efficiency instead." + ) + if wavelength_nm is not None: + return float(curve(wavelength_nm)) + assert bandpass is not None + from .spectral import effective_qe + + return float(effective_qe(curve, bandpass, sed)) + + def saturated(self, frame: FrameLike) -> Any: + """Boolean map of pixels at or near saturation, whose flux is a lower bound. + + A pixel is flagged when its ADU reach the ADC ceiling (``max_adu``, times + ``binning**2`` for binned frames), or when its dark-subtracted, + input-referred charge comes within three effective read-noise RMS of the + charge ceiling: the input full well (bent by the nonlinearity) or the + output-register ceiling ``output_full_well_e / em_gain``, whichever is + lower. Charge beyond either is lost, which is why bright pixels calibrate + low. The read-noise margin makes the flag slightly conservative. + """ + config = self.config + cells = self.binning**2 + data = _input_array(frame) + with _array_device(data): + signal, xp = self._signal_e(data) + gain = self._gain_stage + input_well = float( + np.asarray( + _bend(np.asarray([config.full_well_e * cells]), config, cells), + dtype=np.float64, + )[0] + ) + output_well = ( + config.output_full_well_e + if config.output_full_well_e is not None + else config.full_well_e + ) + ceiling_e = min(input_well, output_well * cells / gain) + margin_e = 3.0 * _input_referred_read_noise_e(config) * self.binning + raw = xp.asarray(data) + return (raw >= config.max_adu * cells) | (signal >= ceiling_e - margin_e) + + def _linearize(self, measured: Any, xp: Any) -> Any: + """Invert the configured nonlinearity on input-referred electrons.""" + config = self.config + well = config.full_well_e * self.binning**2 + if config.nonlinearity_coeffs is not None: + # Newton on q * P(q / well) = m from q = m; P is the response multiplier. + coeffs = config.nonlinearity_coeffs + lit = measured > 0 + q = xp.where(lit, measured, 0.0).astype(measured.dtype) + for _ in range(_NEWTON_STEPS): + u = q / well + value = xp.ones_like(u) + slope = xp.zeros_like(u) + for power, coeff in enumerate(coeffs, start=1): + value += coeff * u**power + slope += power * coeff * u**power + derivative = value + slope + safe = derivative > 0 + q = xp.where(safe, q - (q * value - measured) / xp.where(safe, derivative, 1.0), q) + return xp.where(lit, q, measured).astype(measured.dtype) + if config.nonlinearity <= 0: + return measured + # q (1 - k q) = m => q = 2 m / (1 + sqrt(1 - 4 k m)), stable for small k. + k = config.nonlinearity / well + root = xp.sqrt(xp.clip(1.0 - 4.0 * k * measured, 0.0, None)) + inverse = 2.0 * measured / (1.0 + root) + return xp.where(measured > 0, inverse, measured).astype(measured.dtype) + + def __repr__(self) -> str: + h, w = self.dark_adu.shape + return ( + f"Calibration(camera={self.config.name!r}, shape={h}x{w}, " + f"e_per_adu={self.e_per_adu:.6g}, flat={self.flat is not None}, " + f"linearize={self.linearize})" + ) + + +_NEWTON_STEPS = 12 + + +def _bend(charge_e: Any, config: CameraConfig, cells: int) -> Any: + """The forward nonlinearity on host charge (for the saturation ceiling).""" + well = config.full_well_e * cells + if config.nonlinearity_coeffs is not None: + u = np.clip(charge_e, 0.0, None) / well + factor = np.ones_like(u) + for power, coeff in enumerate(config.nonlinearity_coeffs, start=1): + factor = factor + coeff * u**power + return charge_e * np.clip(factor, 0.0, None) + factor = 1.0 - config.nonlinearity * np.clip(charge_e, 0.0, None) / well + return charge_e * np.clip(factor, 0.0, None) + + +__all__ = ["Calibration", "calibrate", "combine"] diff --git a/src/getframes/camera.py b/src/getframes/camera.py index 8293168..b5549e5 100644 --- a/src/getframes/camera.py +++ b/src/getframes/camera.py @@ -20,6 +20,7 @@ from numpy.typing import NDArray + from .calibrate import Calibration from .noise import DetectorWorkspace, PhotonRate from .scene import Scene from .scene.sources import Source @@ -1512,6 +1513,88 @@ def master_flat( metadata = {**master.metadata, "bias_subtracted": True} return Frame(data=data, metadata=metadata) + @_on_device + def calibration( + self, + exposure: float, + n_darks: int = 64, + temperature: float | None = None, + *, + seed: int | None = None, + binning: int = 1, + binning_mode: str = "digital", + flat: Frame | NDArray[np.floating[Any]] | None = None, + linearize: bool = True, + ) -> Calibration: + """ADU calibration from ``n_darks`` mean-combined dark frames. + + Shorthand for :meth:`getframes.Calibration.from_camera` with this camera; + see there for the arguments. + """ + from .calibrate import Calibration + + return Calibration.from_camera( + self, + exposure, + n_darks, + temperature, + seed=seed, + binning=binning, + binning_mode=binning_mode, + flat=flat, + linearize=linearize, + ) + + @_on_device + def effective_read_noise_e(self, *, binning: int = 1, binning_mode: str = "digital") -> float: + r"""Read noise of one frame referred to the input, in electrons RMS per pixel. + + The Gaussian noise a single ordinary :meth:`expose` adds at the output + amplifier --- read noise, kTC/reset noise and any avalanche-scaled input + noise --- summed in quadrature and divided by the mean EM/avalanche gain + ``G``, i.e. expressed in electrons *collected in the pixel*:: + + sigma_eff = sqrt(sigma_read**2 + sigma_reset**2 + sigma_avalanche**2) / G + + For a per-pixel read-noise map (sCMOS non-uniformity, RTS tail, channel + or edge structure) ``sigma_read`` is the quadrature mean of this sensor's + fixed map over the output pixels, so ``sigma_eff**2`` is the frame-mean + read variance. An EMCCD at gain 600 with 216 e- output noise gives + 0.36 e-. + + This is not the whole noise floor of a gain stage. EM and avalanche + multiplication also multiply the *shot* noise of everything they amplify + (signal, dark current, clock-induced charge) by the excess noise factor + ``F`` (``config.gain_excess_noise_factor``: ``sqrt(2)`` for an EMCCD), so + in input electrons the per-pixel SNR of a signal ``S`` on a dark level + ``D`` is + + .. math:: + + \mathrm{SNR} = \frac{S}{\sqrt{F^2 (S + D) + \sigma_\mathrm{eff}^2}}. + + Parameters + ---------- + binning: + Binning factor of the frames. ``"digital"`` binning sums ``binning**2`` + independent reads, so the noise grows by ``binning``; ``"on_chip"`` + reads each super-pixel once, so it does not. + binning_mode: + ``"digital"`` or ``"on_chip"``, as for :meth:`expose`. + """ + if binning < 1: + raise ValueError("binning must be a positive integer.") + if binning_mode not in ("digital", "on_chip"): + raise ValueError("binning_mode must be 'digital' or 'on_chip'.") + sigma = self._fixed_patterns.read_noise_sigma + if isinstance(sigma, (int, float)): + read_rms = float(sigma) + else: + cropped = self._crop_to_roi(sigma).astype(np.float64) + read_rms = float(np.sqrt(self._backend.scalar(self._backend.xp.mean(cropped**2)))) + per_read = noise._input_referred_read_noise_e(self.config, read_rms) + return per_read * binning if binning_mode == "digital" else per_read + def _metadata( self, frame_type: str, exposure: float, temperature: float, seed: int | None ) -> dict[str, Any]: diff --git a/src/getframes/noise.py b/src/getframes/noise.py index de72b18..7612aea 100644 --- a/src/getframes/noise.py +++ b/src/getframes/noise.py @@ -650,10 +650,61 @@ def apply_gain_stage( f2 = excess_noise_factor**2 alpha = 1.0 / (f2 - 1.0) theta = gain * (f2 - 1.0) + # ``F = sqrt(2)`` gives ``alpha == 1``: the shape is the input itself. + shape = electrons if alpha == 1.0 else electrons * alpha + if ( + isinstance(rng, np.random.Generator) + and isinstance(electrons, np.ndarray) + and electrons.ndim > 0 + ): + # NumPy returns Gamma(shape=0) as exactly 0 *without consuming any draws*, + # so sampling only the lit pixels (in the same C order) yields the same + # values and leaves the generator in the same state, bit for bit. It still + # pays NumPy's per-element broadcasting cost for every zero, which + # dominates photon-starved frames (a wavefront sensor's dark background), + # so gather the lit pixels when they are a minority. + flat = shape.reshape(-1) + lit = flat > 0 + # (Negative input falls through so NumPy still rejects it.) + if 2 * int(np.count_nonzero(lit)) <= flat.size and not bool((flat < 0).any()): + out = np.zeros(flat.size, dtype=electrons.dtype) + out[lit] = rng.gamma(shape=flat[lit], scale=theta) + return out.reshape(electrons.shape) # Both NumPy and CuPy define Gamma(shape=0) as exactly zero. Sampling the # full shape therefore preserves the model while avoiding a mask, gather, # scatter, and (on GPU) a synchronizing ``any`` reduction. - return rng.gamma(shape=electrons * alpha, scale=theta).astype(electrons.dtype, copy=False) + return rng.gamma(shape=shape, scale=theta).astype(electrons.dtype, copy=False) + + +def _input_referred_read_noise_e( + config: CameraConfig, read_noise_rms_e: float | None = None +) -> float: + """Per-read noise of one ordinary frame, referred to the input, in electrons. + + Sums in quadrature the Gaussian terms :func:`digitize` adds at the output + amplifier --- kTC/reset noise, the avalanche-scaled input noise + (``avalanche_input_noise_e`` times ``G * (G / G_ref) ** (p - 1)``) and the read + noise --- and divides by the mean multiplication ``G`` (``em_gain`` when a + gain stage is active, else 1):: + + sigma_in = sqrt(reset_noise_e**2 + sigma_avalanche**2 + read_rms**2) / G + + ``read_noise_rms_e`` is the RMS read noise at the output (for a per-pixel map, + its quadrature mean); it defaults to ``config.read_noise_e``. Shot noise, and + the excess noise factor a gain stage multiplies it by, are not included. + """ + read_rms = config.read_noise_e if read_noise_rms_e is None else read_noise_rms_e + gain = config.em_gain if config.has_gain_stage else 1.0 + avalanche = 0.0 + if config.avalanche_input_noise_e > 0: + avalanche = ( + config.avalanche_input_noise_e + * config.em_gain + * (config.em_gain / config.avalanche_input_noise_reference_gain) + ** (config.avalanche_input_noise_gain_exponent - 1.0) + ) + total = math.sqrt(config.reset_noise_e**2 + avalanche**2 + read_rms**2) + return total / gain def apply_em_gain( diff --git a/tests/test_gpu.py b/tests/test_gpu.py index d004546..6038075 100644 --- a/tests/test_gpu.py +++ b/tests/test_gpu.py @@ -361,3 +361,42 @@ def frames() -> list[gf.Frame]: assert cupy.array_equal( getattr(got.truth, name).view(bits), getattr(want.truth, name).view(bits) ) + + +def test_gpu_calibration_recovers_photons_on_device() -> None: + cupy = _cupy() + camera = gf.Camera.from_preset("andor_ocam2k", device="gpu").with_config(resolution=(64, 64)) + cal = gf.Calibration.from_camera(camera, 1e-3, n_darks=32, seed=1) + assert isinstance(cal.dark_adu, cupy.ndarray) + photons = cupy.full((64, 64), 20.0) + + total = cupy.zeros((64, 64)) + for seed in range(60): + frame = camera.expose(photons / 1e-3, 1e-3, seed=seed, include_truth=False) + image = cal.to_photons(frame) + assert isinstance(image, cupy.ndarray) + total += image + assert float((total / 60).mean()) == pytest.approx(20.0, rel=0.005) + assert isinstance(cal.saturated(frame), cupy.ndarray) + assert camera.effective_read_noise_e() == pytest.approx(216.0 / 600.0) + + +def test_gpu_calibration_matches_cpu_statistically_and_crosses_devices() -> None: + cupy = _cupy() + means = {} + calibrations = {} + gpu_frames = [] + for device in ("cpu", "gpu"): + config = gf.load_preset("generic_scmos").replace(resolution=(64, 64)) + camera = gf.Camera(config, device=device) + calibrations[device] = cal = gf.Calibration.from_camera(camera, 1e-2, n_darks=16, seed=3) + rate = camera._backend.xp.full((64, 64), 5_000.0) + frames = [camera.expose(rate, 1e-2, seed=s, include_truth=False) for s in range(20)] + means[device] = float(np.mean([gf.to_numpy(cal.to_photons(f)).mean() for f in frames])) + if device == "gpu": + gpu_frames = frames + assert means["cpu"] == pytest.approx(50.0, rel=0.005) + assert means["gpu"] == pytest.approx(50.0, rel=0.005) + # A host-built calibration applied to a device frame computes on the device. + crossed = calibrations["cpu"].to_photons(gpu_frames[0]) + assert isinstance(crossed, cupy.ndarray) diff --git a/tests/test_wfs_calibration.py b/tests/test_wfs_calibration.py new file mode 100644 index 0000000..76f9a5a --- /dev/null +++ b/tests/test_wfs_calibration.py @@ -0,0 +1,354 @@ +# SPDX-License-Identifier: MIT +"""Tests for ADU calibration (``Calibration``), effective read noise, and the +bit-identical small-frame fast path of the gain stage.""" + +from __future__ import annotations + +from typing import Any + +import numpy as np +import pytest + +import getframes as gf +from getframes import noise + +EXPOSURE_S = 1e-3 +SHAPE = (64, 64) + +# One preset per detector class, with the documented flux tolerance (fractional +# error of the mean calibrated photon count; see docs/guides/wfs-closed-loop.md). +PRESET_CLASSES = [ + ("EMCCD", "andor_ocam2k"), + # Not the Nuvu presets: their full_well_e (500/800 e-) with no output_full_well_e + # also caps the *amplified* charge, so at EM gain 1000 any pixel holding an + # electron saturates (getframes#11). + ("EMCCD", "andor_ixon_ultra_888"), + ("qCMOS", "hamamatsu_orca_quest_2"), + ("sCMOS", "generic_scmos"), + ("eAPD", "generic_eapd"), + ("eAPD", "leonardo_saphira"), + ("CCD", "generic_ccd"), + ("CMOS", "generic_cmos"), +] +FLUX_TOLERANCE = 0.005 + + +def spot_pattern(photons_per_spot: float, shape: tuple[int, int] = SHAPE) -> Any: + """One Gaussian spot (sigma 1.5 px) per 16x16 subaperture, ``photons_per_spot`` each.""" + coordinates = np.arange(16) - 7.5 + spot = np.exp(-(coordinates[:, None] ** 2 + coordinates[None, :] ** 2) / (2 * 1.5**2)) + spot *= photons_per_spot / spot.sum() + return np.tile(spot, (shape[0] // 16, shape[1] // 16)) + + +def small_camera(preset: str, **kwargs: Any) -> gf.Camera: + return gf.Camera.from_preset(preset, **kwargs).with_config(resolution=SHAPE, roi=None) + + +def mean_photons(camera: gf.Camera, cal: gf.Calibration, photons: Any, n_frames: int) -> Any: + total = np.zeros(SHAPE) + for seed in range(n_frames): + frame = camera.expose(photons / EXPOSURE_S, EXPOSURE_S, seed=seed, include_truth=False) + total += cal.to_photons(frame) + return total / n_frames + + +# -------------------------------------------------------------------------- +# Flux recovery +# -------------------------------------------------------------------------- +@pytest.mark.parametrize(("sensor_class", "preset"), PRESET_CLASSES) +def test_to_photons_recovers_the_incident_photons(sensor_class: str, preset: str) -> None: + camera = small_camera(preset) + cal = gf.Calibration.from_camera(camera, EXPOSURE_S, n_darks=64, seed=1) + photons = spot_pattern(1000.0) + + recovered = mean_photons(camera, cal, photons, n_frames=150) + + assert recovered.sum() / photons.sum() == pytest.approx(1.0, abs=FLUX_TOLERANCE) + # Pixel by pixel too: the spot cores match within their own shot noise. + core = photons > 20.0 + np.testing.assert_allclose(recovered[core].mean(), photons[core].mean(), rtol=0.01) + + +def test_cred_one_recovers_photons_with_a_flat_and_common_mode_removal() -> None: + """The C-RED One needs a flat (5.5% PRNU, avalanche-gain map) and a per-frame + background: its 69 ADU common-mode readout noise is ~6 input e- on every pixel, + which swamps a frame's total flux unless subtracted from unlit pixels.""" + camera = small_camera("first_light_imaging_cred_one") + dark = gf.Calibration.from_camera(camera, EXPOSURE_S, n_darks=64, seed=1).dark_adu + flat = camera.master_flat(2e6, EXPOSURE_S, 64, bias=dark, method="mean", seed=2) + cal = gf.Calibration(camera.config, dark, flat=flat) + photons = spot_pattern(1000.0) + unlit = photons < 1e-6 + + total = np.zeros(SHAPE) + for seed in range(150): + frame = camera.expose(photons / EXPOSURE_S, EXPOSURE_S, seed=seed, include_truth=False) + image = cal.to_photons(frame) + total += image - image[unlit].mean() + recovered = total / 150 + + # Documented 1%: the flat's own noise and IPC mixing a spot's PRNU across + # neighbours leave a few tenths of a percent. + assert recovered.sum() / photons.sum() == pytest.approx(1.0, abs=0.01) + + +def test_flat_corrects_prnu_on_a_few_pixel_spot() -> None: + config = gf.load_preset("generic_cmos").replace( + resolution=SHAPE, roi=None, prnu=0.1, read_noise_e=0.5, bit_depth=20 + ) + camera = gf.Camera(config) + photons = np.zeros(SHAPE) + photons[30:33, 30:33] = 5000.0 + no_flat = gf.Calibration.from_camera(camera, EXPOSURE_S, n_darks=16, seed=1) + flat = camera.master_flat(1e7, EXPOSURE_S, 32, bias=no_flat.dark_adu, method="mean", seed=3) + with_flat = gf.Calibration(config, no_flat.dark_adu, flat=flat) + + raw = mean_photons(camera, no_flat, photons, n_frames=40)[30:33, 30:33] + fixed = mean_photons(camera, with_flat, photons, n_frames=40)[30:33, 30:33] + + # The 9-pixel spot sees the local PRNU; the flat removes it to shot-noise level. + assert np.abs(fixed / 5000.0 - 1.0).max() < 0.01 + assert np.abs(raw / 5000.0 - 1.0).max() > 3 * np.abs(fixed / 5000.0 - 1.0).max() + + +@pytest.mark.parametrize( + "overrides", + [ + {"nonlinearity": 0.02}, + {"nonlinearity": 0.0, "nonlinearity_coeffs": (-0.03, 0.01)}, + ], +) +def test_linearize_removes_the_nonlinearity_bias(overrides: dict[str, Any]) -> None: + config = gf.load_preset("generic_scmos").replace(resolution=SHAPE, roi=None, **overrides) + camera = gf.Camera(config) + photons = np.full(SHAPE, 20_000.0 / config.quantum_efficiency) # 2/3 of full well + linear = gf.Calibration.from_camera(camera, EXPOSURE_S, n_darks=16, seed=1) + bent = gf.Calibration(config, linear.dark_adu, linearize=False) + + frames = [ + camera.expose(photons / EXPOSURE_S, EXPOSURE_S, seed=s, include_truth=False) + for s in range(10) + ] + ratio_linear = np.mean([linear.to_photons(f).mean() for f in frames]) / photons.mean() + ratio_bent = np.mean([bent.to_photons(f).mean() for f in frames]) / photons.mean() + + assert ratio_bent < 0.995 # the uncorrected response reads measurably low + assert ratio_linear == pytest.approx(1.0, abs=5e-4) + + +def test_linearize_leaves_negative_signal_linear() -> None: + config = gf.load_preset("generic_scmos").replace(resolution=(2, 2), roi=None) + cal = gf.Calibration(config, np.full((2, 2), 100.0)) + adu = np.array([[90.0, 100.0], [110.0, 100.0 + 20_000.0 / config.gain_e_per_adu]]) + + electrons = cal.to_electrons(adu) + + assert electrons[0, 0] == pytest.approx(-10.0 * config.gain_e_per_adu) + assert electrons[0, 1] == 0.0 + # The inverse of q (1 - a q / well): q = 20_000 needs 20_000 / (1 - a q / well) in. + q = electrons[1, 1] + assert q * (1 - config.nonlinearity * q / config.full_well_e) == pytest.approx(20_000.0) + + +def test_mean_dark_is_needed_for_emccd_cic() -> None: + """A median master dark drops the EM-amplified CIC mean and reads flux high.""" + camera = small_camera("andor_ocam2k") + mean_cal = gf.Calibration.from_camera(camera, EXPOSURE_S, n_darks=256, seed=1) + median_dark = camera.master_dark(EXPOSURE_S, 256, seed=1, method="median") + median_cal = gf.Calibration(camera.config, median_dark) + photons = np.full(SHAPE, 0.05) # photon-starved: CIC is ~10% of the signal + + mean_ratio = mean_photons(camera, mean_cal, photons, 200).mean() / 0.05 + median_ratio = mean_photons(camera, median_cal, photons, 200).mean() / 0.05 + + assert mean_ratio == pytest.approx(1.0, abs=0.05) + # The median sits between the bias and the mean: about 8% high here. + assert 0.04 < median_ratio - mean_ratio < 0.13 + + +# -------------------------------------------------------------------------- +# Quantum efficiency, saturation, construction +# -------------------------------------------------------------------------- +def test_to_photons_quantum_efficiency_choices() -> None: + camera = small_camera("hamamatsu_orca_quest_2") + cal = gf.Calibration(camera.config, np.zeros(SHAPE)) + frame = np.full(SHAPE, 100.0) + electrons = cal.to_electrons(frame) + + np.testing.assert_allclose(cal.to_photons(frame), electrons / 0.85) + np.testing.assert_allclose(cal.to_photons(frame, quantum_efficiency=0.5), electrons / 0.5) + qe_700 = float(camera.config.qe_curve(700.0)) # type: ignore[misc] + assert qe_700 == pytest.approx(0.50) + np.testing.assert_allclose(cal.to_photons(frame, wavelength_nm=700.0), electrons / qe_700) + band = gf.SpectralBandpass.tophat(700.0, 100.0) + assert cal.quantum_efficiency(bandpass=band) == pytest.approx( + gf.spectral.effective_qe(camera.config.qe_curve, band) # type: ignore[arg-type] + ) + with pytest.raises(ValueError, match="not both"): + cal.quantum_efficiency(wavelength_nm=700.0, bandpass=band) + with pytest.raises(ValueError, match="no qe_curve"): + gf.Calibration(small_camera("generic_scmos").config, np.zeros(SHAPE)).quantum_efficiency( + wavelength_nm=700.0 + ) + + +def test_saturated_flags_pixels_beyond_the_output_ceiling() -> None: + camera = small_camera("andor_ocam2k") + cal = gf.Calibration.from_camera(camera, EXPOSURE_S, n_darks=8, seed=1) + photons = np.zeros(SHAPE) + photons[10, 10] = 2000.0 # 1800 e- x 600 >> 280,000 e- output register + photons[40, 40] = 50.0 + frame = camera.expose(photons / EXPOSURE_S, EXPOSURE_S, seed=3) + + mask = cal.saturated(frame) + + assert mask[10, 10] + assert not mask[40, 40] + assert mask.sum() == 1 + assert cal.to_photons(frame)[10, 10] < 0.3 * 2000.0 # the lost charge + + +def test_from_camera_is_seeded_and_follows_precision() -> None: + camera = small_camera("andor_ocam2k", precision="single") + first = gf.Calibration.from_camera(camera, EXPOSURE_S, n_darks=4, seed=5) + second = gf.Calibration.from_camera(camera, EXPOSURE_S, n_darks=4, seed=5) + + np.testing.assert_array_equal(first.dark_adu, second.dark_adu) + assert first.dark_adu.dtype == np.float32 + frame = camera.expose(1e4, EXPOSURE_S, seed=1) + assert first.to_photons(frame).dtype == np.float32 + assert first.e_per_adu == pytest.approx(28.0 / 600.0) + assert "Andor OCAM2K" in repr(first) + assert camera.calibration(EXPOSURE_S, n_darks=4, seed=5).dark_adu.tolist() == ( + first.dark_adu.tolist() + ) + + +def test_binned_calibration_recovers_photons() -> None: + camera = small_camera("generic_ccd") + cal = gf.Calibration.from_camera( + camera, EXPOSURE_S, n_darks=32, seed=1, binning=2, binning_mode="on_chip" + ) + photons = np.full(SHAPE, 50.0) + total = np.zeros((32, 32)) + for seed in range(50): + frame = camera.expose( + photons / EXPOSURE_S, EXPOSURE_S, binning=2, binning_mode="on_chip", seed=seed + ) + total += cal.to_photons(frame) + assert (total / 50).mean() == pytest.approx(200.0, rel=FLUX_TOLERANCE) + + +def test_calibration_validates_inputs() -> None: + config = small_camera("generic_cmos").config + with pytest.raises(ValueError, match="2-D"): + gf.Calibration(config, np.zeros(4)) + with pytest.raises(ValueError, match="binning"): + gf.Calibration(config, np.zeros(SHAPE), binning=0) + with pytest.raises(ValueError, match="flat shape"): + gf.Calibration(config, np.zeros(SHAPE), flat=np.ones((2, 2))) + with pytest.raises(ValueError, match="no positive"): + gf.Calibration(config, np.zeros(SHAPE), flat=np.zeros(SHAPE)) + cal = gf.Calibration(config, np.zeros(SHAPE)) + with pytest.raises(ValueError, match="does not match"): + cal.to_electrons(np.zeros((2, 2))) + with pytest.raises(ValueError, match="positive"): + cal.to_photons(np.zeros(SHAPE), quantum_efficiency=0.0) + with pytest.raises(ValueError, match="n_darks"): + gf.Calibration.from_camera(small_camera("generic_cmos"), EXPOSURE_S, n_darks=0) + + +def test_flat_is_normalised_and_dead_pixels_are_left_unscaled() -> None: + config = small_camera("generic_cmos").config + flat = np.full(SHAPE, 2.0) + flat[0, 0] = 0.0 + flat[0, 1] = 4.0 + cal = gf.Calibration(config, np.zeros(SHAPE), flat=flat, linearize=False) + + electrons = cal.to_electrons(np.full(SHAPE, 10.0)) + + gain = config.gain_e_per_adu + assert electrons[5, 5] == pytest.approx(10.0 * gain * (2.0 * 4094 + 4.0) / 4095 / 2.0) + assert electrons[0, 0] == pytest.approx(10.0 * gain) + assert electrons[0, 1] == pytest.approx(electrons[5, 5] / 2.0) + + +# -------------------------------------------------------------------------- +# Effective read noise +# -------------------------------------------------------------------------- +def test_effective_read_noise_is_referred_to_the_input() -> None: + ocam = gf.Camera.from_preset("andor_ocam2k") + assert ocam.effective_read_noise_e() == pytest.approx(216.0 / 600.0) + ccd = gf.Camera.from_preset("generic_ccd") + assert ccd.effective_read_noise_e() == pytest.approx(ccd.config.read_noise_e) + assert ccd.effective_read_noise_e(binning=2) == pytest.approx(2 * ccd.config.read_noise_e) + assert ccd.effective_read_noise_e(binning=2, binning_mode="on_chip") == pytest.approx( + ccd.config.read_noise_e + ) + with pytest.raises(ValueError, match="binning_mode"): + ccd.effective_read_noise_e(binning_mode="analog") + + +def test_effective_read_noise_includes_reset_and_avalanche_terms() -> None: + config = gf.load_preset("generic_eapd").replace( + resolution=(16, 16), + reset_noise_e=30.0, + avalanche_input_noise_e=2.0, + avalanche_input_noise_reference_gain=10.0, + avalanche_input_noise_gain_exponent=1.5, + ) + camera = gf.Camera(config) + gain = config.em_gain + avalanche = 2.0 * gain * (gain / 10.0) ** 0.5 + expected = np.sqrt(config.read_noise_e**2 + 30.0**2 + avalanche**2) / gain + assert camera.effective_read_noise_e() == pytest.approx(expected) + + +def test_effective_read_noise_matches_measured_bias_noise() -> None: + """Per-pixel sCMOS read noise: the quadrature mean of the fixed RMS map.""" + camera = small_camera("generic_scmos").with_config(dark_current_e_per_s=0.0, prnu=0.0) + stack = np.stack( + [np.asarray(f.data, dtype=np.float64) for f in camera.dark_series(0.0, 400, seed=4)] + ) + measured = np.sqrt(stack.var(axis=0, ddof=1).mean()) * camera.config.gain_e_per_adu + # Quantisation adds gain**2 / 12 of variance in e-^2. + quantised = np.sqrt(camera.effective_read_noise_e() ** 2 + camera.config.gain_e_per_adu**2 / 12) + assert measured == pytest.approx(quantised, rel=0.03) + assert camera.effective_read_noise_e() > camera.config.read_noise_e # RTS + spread + + +# -------------------------------------------------------------------------- +# Bit-identical fast path of the gain stage +# -------------------------------------------------------------------------- +@pytest.mark.parametrize("excess_noise_factor", [np.sqrt(2.0), 1.25]) +@pytest.mark.parametrize("lit_fraction", [0.0, 0.03, 0.4, 0.6, 1.0]) +@pytest.mark.parametrize("dtype", [np.float64, np.float32]) +def test_gain_stage_matches_full_gamma_draw_bit_for_bit( + excess_noise_factor: float, lit_fraction: float, dtype: Any +) -> None: + source = np.random.default_rng(7) + electrons = (source.poisson(3.0, (48, 40)) * (source.random((48, 40)) < lit_fraction)).astype( + dtype + ) + gain = 600.0 + f2 = excess_noise_factor**2 + + reference_rng = np.random.default_rng(11) + reference = reference_rng.gamma( + shape=electrons * (1.0 / (f2 - 1.0)), scale=gain * (f2 - 1.0) + ).astype(dtype) + rng = np.random.default_rng(11) + result = noise.apply_gain_stage(electrons, gain, excess_noise_factor, rng) + + assert result.dtype == dtype + assert result.shape == electrons.shape + np.testing.assert_array_equal(result, reference) + assert rng.random() == reference_rng.random() # same generator state afterwards + + +def test_gain_stage_still_rejects_negative_input() -> None: + electrons = np.zeros((4, 4)) + electrons[0, 0] = -1.0 + with pytest.raises(ValueError): + noise.apply_gain_stage(electrons, 100.0, np.sqrt(2.0), np.random.default_rng(0))