fix: correct mirrored row weights + round-off-dependent cells in rectangular mapper - #490
Merged
Merged
Conversation
…angular mapper `adaptive_rectangular_mappings_weights_via_interpolation_from` — the bilinear mapper shared by every adaptive rectangular mesh — had two coupled defects. 1. Mirrored row weights. `t_row` is the fractional distance measured FROM `ix_down`, so `ix_up` must carry `t_row` and `ix_down` `1 - t_row`. They were the other way round, mirroring the interpolation in the row direction. The column weights were, and remain, correctly paired. 2. Round-off-dependent cell assignment. `ix_up = ceil(g)` collapses onto `ix_down` wherever `g` is exactly integral. `transform()` ends in `clip(F_q, 0.0, 1.0)`, so saturated points land on exactly integer `g` systematically, not by chance. There the cell degenerated and `t_row` was forced to 0 by the `+ 1e-12` guard; a 1-ULP change in the traced grid moved a point off the plateau and jumped its weight a whole mesh row. (2) is why the eager and jitted likelihoods disagreed by ~1.6e-3 on an otherwise smooth surface, tripping `assert_eager_jit_consistent` in autolens_workspace_test (issue #279). (1) is why (2) could not be fixed without a behaviour change. Both are regressions, not design choices. The mapper was correct when introduced in fd11b17 (2025-06-24); 8f00795 (2025-09-15) forked it for the adaptive meshes and mirrored the columns, and 9b1c91c (2025-09-23) "fixed mappings and weights" fixed the columns and broke the rows. The correct formulation never left the package — it survives verbatim for the uniform mesh at interpolator/rectangular_uniform.py:72-99, which is what this restores. Affected meshes (all inherit `InterpolatorRectangular` from `RectangularRTUAdaptDensity`): RectangularRTUAdaptDensity, RectangularRTUAdaptImage, RectangularBilinearAdaptDensity, RectangularBilinearAdaptImage. Not affected: RectangularUniform, Delaunay, DelaunayNN, KNearestNeighbor, KNNBarycentric. Adds the two regression tests whose absence let this survive eleven months and a "fix" commit: partition of unity plus linear reproduction (the property any consistent mis-pairing satisfies the first but not the second), and continuity of the cell assignment across integer boundaries. Both fail on the previous implementation — linear reproduction by ~1.0 in the row axis, continuity by a jump of 5.999 (a two-row flip) — and pass here. Behaviour change: reconstructions using an adaptive rectangular mesh shift. Measured on autolens_workspace_test/scripts/imaging/jax_likelihood/rectangular.py, log likelihood moves -651692.997799 -> -650470.379097 (+1222.6, a better fit). Downstream `EXPECTED_LOG_*` constants need regenerating; see #279. Validated: pytest test_autoarray/ 1222 passed on Python 3.12 and 3.13. autolens_workspace_test/scripts/interferometer/jax_grad/gradient.py green on both legs, all four variants, eager/jit gap exactly 0.0 with the guard left at rtol=1e-10 (3.13 83s, 3.12 86s against the 300s cap). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016azuS2UbS3mFvfxkDFGsKj
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the bilinear mapper shared by every adaptive rectangular mesh. Found while diagnosing autolens_workspace_test#279 (an eager/jit likelihood divergence three repos away).
The two defects
adaptive_rectangular_mappings_weights_via_interpolation_from(autoarray/inversion/mesh/interpolator/rectangular.py):1. Mirrored row weights.
t_rowis the fractional distance measured fromix_down, soix_upmust carryt_rowandix_downmust carry1 - t_row. They were the other way round. The column weights were, and remain, correctly paired — only the rows were mirrored.2. Round-off-dependent cell assignment.
ix_up = ceil(g)collapses ontoix_downwherevergis exactly integral.transform()ends inclip(F_q, 0.0, 1.0), so saturated points land on exactly integergsystematically, not by chance. There the bracketing cell degenerated andt_rowwas forced to 0 by the+ 1e-12guard; a 1-ULP change in the traced grid moved a point off the plateau and jumped its weight a whole mesh row.(2) is why the eager and jitted likelihoods disagreed by ~1.6e-3 on an otherwise smooth surface. (1) is why (2) could not be fixed without a behaviour change.
Evidence
A correct bilinear scheme must satisfy partition of unity and reproduce the query position (
sum_i w_i * node_i == g). Partition of unity alone is satisfied by any consistent mis-pairing — which is exactly what was happening:fd11b178original8f007957adaptive fork9b1c91cfcurrentInterpolation-accuracy census (the decisive evidence)
Likelihood at a fixed model is a poor discriminator here, and that is much of why this survived: with a pixelized source the inversion solves for the source values, so a mirrored mapping is still a valid linear basis over the same mesh pixels — merely mislabelled about where the flux sits. It can fit the data comparably well while being geometrically wrong.
So this was measured directly instead: interpolate a known function through the real mesh geometry and compare against ground truth, querying where the traced points actually are (an adaptive mesh must not be judged in regions it deliberately leaves coarse). Max error,
RectangularUniform's known-correct mapper as control:The fix is better at every mesh size, for both transforms, on both functions. At n=64 it is ~4.9x more accurate for Bilinear and ~27-31x for RTU. RTU is the largest beneficiary, so there is no compensating flip that made the old behaviour accidentally right for the kernel-CDF meshes.
Control (
RectangularUniform, untouched by this PR) converges cleanly at ~O(h²) — 0.0979 → 0.0217 → 0.0052 → 0.0013 — confirming the harness measures what it claims.Both defects are regressions
The mapper was correct when introduced in
fd11b178(2025-06-24):clip(floor(f), 0, N-2),ix + 1rather thanceil, each corner given its own weight.8f007957(2025-09-15, "adpative stuff implemented") forked it for the adaptive meshes withceiland adelta_up/delta_downform — mirroring the columns, and dropping exactly-integer points entirely (no1e-12guard then, so all four weights were exactly zero).9b1c91cf(2025-09-23, "fixed mappings and weights") then fixed the columns and the dropped-point bug and broke the rows.The correct formulation never left the package. It survives verbatim for the uniform mesh at
autoarray/inversion/mesh/interpolator/rectangular_uniform.py:72-99and scores 0.000000 on both axes today. This PR restores that sibling's scheme rather than inventing one — which is also whyRectangularUniformpasses the eager/jit guard while the adaptive meshes fail.Affected meshes
All inherit
interpolator_cls→InterpolatorRectangularfromRectangularRTUAdaptDensity:RectangularRTUAdaptDensity(kernel-CDF, density-adapted)RectangularRTUAdaptImage(kernel-CDF, image-adapted)RectangularBilinearAdaptDensity(rank-CDF)RectangularBilinearAdaptImage(rank-CDF)Not affected — different interpolators:
RectangularUniform,Delaunay,DelaunayNN,KNearestNeighbor,KNNBarycentric.Regression tests
The bug survived eleven months and a "fix" commit because nothing asserted the interpolation property itself. Two tests added, both of which fail on the previous implementation and pass here:
test__mappings_sizes_weights__reproduces_the_query_position— partition of unity plus linear reproduction. Fails previously by ~1.0 in the row axis.test__mappings_sizes_weights__cell_assignment_is_continuous_at_integers— sweeps the row coordinate across every interior integer boundary and asserts the interpolant has no jumps. Fails previously with a jump of 5.999 (a two-row flip).rectangular_uniform.pypasses both as-is.Behaviour change
Reconstructions using an adaptive rectangular mesh shift, and the sign is not consistent — measured on
autolens_workspace_testat unfitted model points:imaging/jax_likelihood/rectangular.pymoves-651692.998 → -650470.379(+1223) whilerectangular_rtu.pymoves-644121.022 → -652043.028(−7922). That is expected and not a correctness signal, for the reason given in the census section above; the interpolation-accuracy numbers are the metric that discriminates, and they improve in every case.Downstream
assert_allcloseconstants inautolens_workspace_testneed regenerating (all affected scripts confirmed passing on the pre-fix build, so these shifts are attributable to this PR alone).autogalaxy_workspace_testneeds no changes — it asserts jax-vs-numpy parity, and both backends shift together. Merge this first; the workspace PR follows behind the library-first gate.Separate observation (not fixed here)
KERNEL_CDF_DEFAULT_KNOTS = 64is independent ofmesh_pixels, so the knot-table inverse used to place mesh nodes drifts from the exact forward transform as the mesh refines: max|g_roundtrip - ix|measured at 0.0055 (n=16), 0.0238 (n=32), 0.101 (n=64) index units. Harmless at production mesh sizes (well under 64), but worth scalingn_knotswithmesh_pixelsat some point. Out of scope for this PR.Validation
pytest test_autoarray/— 1222 passed on Python 3.12 and 3.13autolens_workspace_test/scripts/interferometer/jax_grad/gradient.py— green on both legs, all four variants, all FD/AD checks, all gradients live. Eager/jit gap exactly 0.0 withassert_eager_jit_consistentleft untouched atrtol=1e-10. 3.13: 83s, 3.12: 86s against the 300s cap.jax/jaxlib0.11.1,jaxnnls1.0.1,numpy2.5.2,scipy1.17.1).Note
AGENTS.md: unit tests are NumPy-only, so the workspace parity scripts above are the real gate for thexppath.Refs PyAutoLabs/autolens_workspace_test#279
🤖 Generated with Claude Code
https://claude.ai/code/session_016azuS2UbS3mFvfxkDFGsKj