Skip to content

fix: correct mirrored row weights + round-off-dependent cells in rectangular mapper - #490

Merged
Jammy2211 merged 1 commit into
mainfrom
feature/rectangular-mapper-bilinear-row-weights
Aug 26, 2026
Merged

fix: correct mirrored row weights + round-off-dependent cells in rectangular mapper#490
Jammy2211 merged 1 commit into
mainfrom
feature/rectangular-mapper-bilinear-row-weights

Conversation

@Jammy2211

@Jammy2211 Jammy2211 commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

Fixes the bilinear mapper shared by every adaptive rectangular mesh. Found while diagnosing autolens_workspace_test#279 (an eager/jit likelihood divergence three repos away).

⚠️ This changes reconstructions. See "Behaviour change" below — downstream constants need regenerating, and this must merge before the workspace PRs that regenerate them.

The two defects

adaptive_rectangular_mappings_weights_via_interpolation_from (autoarray/inversion/mesh/interpolator/rectangular.py):

1. Mirrored row weights. t_row is the fractional distance measured from ix_down, so ix_up must carry t_row and ix_down must carry 1 - t_row. They were the other way round. The column weights were, and remain, correctly paired — only the rows were mirrored.

2. Round-off-dependent cell assignment. ix_up = ceil(g) collapses onto ix_down wherever g is exactly integral. transform() ends in clip(F_q, 0.0, 1.0), so saturated points land on exactly integer g systematically, not by chance. There the bracketing cell degenerated and t_row was forced to 0 by the + 1e-12 guard; a 1-ULP change in the traced grid moved a point off the plateau and jumped its weight a whole mesh row.

(2) is why the eager and jitted likelihoods disagreed by ~1.6e-3 on an otherwise smooth surface. (1) is why (2) could not be fixed without a behaviour change.

Evidence

A correct bilinear scheme must satisfy partition of unity and reproduce the query position (sum_i w_i * node_i == g). Partition of unity alone is satisfied by any consistent mis-pairing — which is exactly what was happening:

version date partition row err col err
fd11b178 original 2025-06-24 1.1e-16 0.000000 0.000000 correct
8f007957 adaptive fork 2025-09-15 1.1e-16 0.000000 0.999884 columns mirrored
9b1c91cf current 2025-09-23 2.2e-16 0.999968 0.000000 rows mirrored
this PR 1.1e-16 0.000000 0.000000 correct

Interpolation-accuracy census (the decisive evidence)

Likelihood at a fixed model is a poor discriminator here, and that is much of why this survived: with a pixelized source the inversion solves for the source values, so a mirrored mapping is still a valid linear basis over the same mesh pixels — merely mislabelled about where the flux sits. It can fit the data comparably well while being geometrically wrong.

So this was measured directly instead: interpolate a known function through the real mesh geometry and compare against ground truth, querying where the traced points actually are (an adaptive mesh must not be judged in regions it deliberately leaves coarse). Max error, RectangularUniform's known-correct mapper as control:

mesh function n=8 n=16 n=32 n=64
Bilinear (rank) before smooth 0.8187 0.4416 0.2444 0.1266
Bilinear (rank) after smooth 0.4357 0.2835 0.0489 0.0260
RTU (kernel) before smooth 0.7635 0.3793 0.1871 0.1068
RTU (kernel) after smooth 0.3608 0.1838 0.0095 0.0034
RTU (kernel) before linear 9.5816 7.2794 1.3158 0.6241
RTU (kernel) after linear 2.9737 1.0731 0.0848 0.0229

The fix is better at every mesh size, for both transforms, on both functions. At n=64 it is ~4.9x more accurate for Bilinear and ~27-31x for RTU. RTU is the largest beneficiary, so there is no compensating flip that made the old behaviour accidentally right for the kernel-CDF meshes.

Control (RectangularUniform, untouched by this PR) converges cleanly at ~O(h²) — 0.0979 → 0.0217 → 0.0052 → 0.0013 — confirming the harness measures what it claims.

Both defects are regressions

The mapper was correct when introduced in fd11b178 (2025-06-24): clip(floor(f), 0, N-2), ix + 1 rather than ceil, each corner given its own weight.

8f007957 (2025-09-15, "adpative stuff implemented") forked it for the adaptive meshes with ceil and a delta_up/delta_down form — mirroring the columns, and dropping exactly-integer points entirely (no 1e-12 guard then, so all four weights were exactly zero). 9b1c91cf (2025-09-23, "fixed mappings and weights") then fixed the columns and the dropped-point bug and broke the rows.

The correct formulation never left the package. It survives verbatim for the uniform mesh at autoarray/inversion/mesh/interpolator/rectangular_uniform.py:72-99 and scores 0.000000 on both axes today. This PR restores that sibling's scheme rather than inventing one — which is also why RectangularUniform passes the eager/jit guard while the adaptive meshes fail.

Affected meshes

All inherit interpolator_clsInterpolatorRectangular from RectangularRTUAdaptDensity:

  • RectangularRTUAdaptDensity (kernel-CDF, density-adapted)
  • RectangularRTUAdaptImage (kernel-CDF, image-adapted)
  • RectangularBilinearAdaptDensity (rank-CDF)
  • RectangularBilinearAdaptImage (rank-CDF)

Not affected — different interpolators: RectangularUniform, Delaunay, DelaunayNN, KNearestNeighbor, KNNBarycentric.

Regression tests

The bug survived eleven months and a "fix" commit because nothing asserted the interpolation property itself. Two tests added, both of which fail on the previous implementation and pass here:

  • test__mappings_sizes_weights__reproduces_the_query_position — partition of unity plus linear reproduction. Fails previously by ~1.0 in the row axis.
  • test__mappings_sizes_weights__cell_assignment_is_continuous_at_integers — sweeps the row coordinate across every interior integer boundary and asserts the interpolant has no jumps. Fails previously with a jump of 5.999 (a two-row flip).

rectangular_uniform.py passes both as-is.

Behaviour change

Reconstructions using an adaptive rectangular mesh shift, and the sign is not consistent — measured on autolens_workspace_test at unfitted model points: imaging/jax_likelihood/rectangular.py moves -651692.998 → -650470.379 (+1223) while rectangular_rtu.py moves -644121.022 → -652043.028 (−7922). That is expected and not a correctness signal, for the reason given in the census section above; the interpolation-accuracy numbers are the metric that discriminates, and they improve in every case.

Downstream assert_allclose constants in autolens_workspace_test need regenerating (all affected scripts confirmed passing on the pre-fix build, so these shifts are attributable to this PR alone). autogalaxy_workspace_test needs no changes — it asserts jax-vs-numpy parity, and both backends shift together. Merge this first; the workspace PR follows behind the library-first gate.

Separate observation (not fixed here)

KERNEL_CDF_DEFAULT_KNOTS = 64 is independent of mesh_pixels, so the knot-table inverse used to place mesh nodes drifts from the exact forward transform as the mesh refines: max |g_roundtrip - ix| measured at 0.0055 (n=16), 0.0238 (n=32), 0.101 (n=64) index units. Harmless at production mesh sizes (well under 64), but worth scaling n_knots with mesh_pixels at some point. Out of scope for this PR.

Validation

  • pytest test_autoarray/1222 passed on Python 3.12 and 3.13
  • autolens_workspace_test/scripts/interferometer/jax_grad/gradient.py — green on both legs, all four variants, all FD/AD checks, all gradients live. Eager/jit gap exactly 0.0 with assert_eager_jit_consistent left untouched at rtol=1e-10. 3.13: 83s, 3.12: 86s against the 300s cap.
  • Stack built from source at the versions of retime run 32741386752 (jax/jaxlib 0.11.1, jaxnnls 1.0.1, numpy 2.5.2, scipy 1.17.1).

Note AGENTS.md: unit tests are NumPy-only, so the workspace parity scripts above are the real gate for the xp path.

Refs PyAutoLabs/autolens_workspace_test#279

🤖 Generated with Claude Code

https://claude.ai/code/session_016azuS2UbS3mFvfxkDFGsKj

…angular mapper

`adaptive_rectangular_mappings_weights_via_interpolation_from` — the bilinear
mapper shared by every adaptive rectangular mesh — had two coupled defects.

1. Mirrored row weights. `t_row` is the fractional distance measured FROM
   `ix_down`, so `ix_up` must carry `t_row` and `ix_down` `1 - t_row`. They were
   the other way round, mirroring the interpolation in the row direction. The
   column weights were, and remain, correctly paired.

2. Round-off-dependent cell assignment. `ix_up = ceil(g)` collapses onto
   `ix_down` wherever `g` is exactly integral. `transform()` ends in
   `clip(F_q, 0.0, 1.0)`, so saturated points land on exactly integer `g`
   systematically, not by chance. There the cell degenerated and `t_row` was
   forced to 0 by the `+ 1e-12` guard; a 1-ULP change in the traced grid moved
   a point off the plateau and jumped its weight a whole mesh row.

(2) is why the eager and jitted likelihoods disagreed by ~1.6e-3 on an
otherwise smooth surface, tripping `assert_eager_jit_consistent` in
autolens_workspace_test (issue #279). (1) is why (2) could not be fixed
without a behaviour change.

Both are regressions, not design choices. The mapper was correct when
introduced in fd11b17 (2025-06-24); 8f00795 (2025-09-15) forked it for the
adaptive meshes and mirrored the columns, and 9b1c91c (2025-09-23) "fixed
mappings and weights" fixed the columns and broke the rows. The correct
formulation never left the package — it survives verbatim for the uniform mesh
at interpolator/rectangular_uniform.py:72-99, which is what this restores.

Affected meshes (all inherit `InterpolatorRectangular` from
`RectangularRTUAdaptDensity`): RectangularRTUAdaptDensity, RectangularRTUAdaptImage,
RectangularBilinearAdaptDensity, RectangularBilinearAdaptImage. Not affected:
RectangularUniform, Delaunay, DelaunayNN, KNearestNeighbor, KNNBarycentric.

Adds the two regression tests whose absence let this survive eleven months and
a "fix" commit: partition of unity plus linear reproduction (the property any
consistent mis-pairing satisfies the first but not the second), and continuity
of the cell assignment across integer boundaries. Both fail on the previous
implementation — linear reproduction by ~1.0 in the row axis, continuity by a
jump of 5.999 (a two-row flip) — and pass here.

Behaviour change: reconstructions using an adaptive rectangular mesh shift.
Measured on autolens_workspace_test/scripts/imaging/jax_likelihood/rectangular.py,
log likelihood moves -651692.997799 -> -650470.379097 (+1222.6, a better fit).
Downstream `EXPECTED_LOG_*` constants need regenerating; see #279.

Validated: pytest test_autoarray/ 1222 passed on Python 3.12 and 3.13.
autolens_workspace_test/scripts/interferometer/jax_grad/gradient.py green on
both legs, all four variants, eager/jit gap exactly 0.0 with the guard left at
rtol=1e-10 (3.13 83s, 3.12 86s against the 300s cap).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016azuS2UbS3mFvfxkDFGsKj
@Jammy2211
Jammy2211 merged commit 158db38 into main Aug 26, 2026
3 checks passed
@Jammy2211
Jammy2211 deleted the feature/rectangular-mapper-bilinear-row-weights branch August 26, 2026 20:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants