Repository navigation
Make the log and spectral TV regularizers boundary-correct - #323
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## dev-backend #323 +/- ##
===============================================
- Coverage 49.29% 49.16% -0.13%
===============================================
Files 58 58
Lines 27776 27637 -139
Branches 4722 4717 -5
===============================================
- Hits 13693 13589 -104
+ Misses 12540 12521 -19
+ Partials 1543 1527 -16 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
achael
left a comment
There was a problem hiding this comment.
looks good. I think edge padding for all tv regularizers is probably strictly better than zero padding. So i'd suggest exploring back-porting that behavior to the standard tv, tv2, ptv regularizers as well, then reverting to having the log, spec reg terms calling them. It's ok if they are not byte-identical to the existing versions as long as they pass the finite diff tests. But let me know if you disagree.
| flux = kwargs['flux'] | ||
| npix = nx * ny | ||
| if np.any(np.invert(mask)): | ||
| imvec = embed(imvec, mask, clipfloor=flux/npix) |
There was a problem hiding this comment.
add a check that this clipfloor value is >=0 (it should be, but just in case)
There was a problem hiding this comment.
clipfloor is now removed. The wrappers uses only active pixels before the shared kernel embeds them. They now reject nonpositive or nonfinite flux
| log_kwargs['flux'] = logflux | ||
| return reg_tv(xp.log(imvec), np.ones(imvec.shape, dtype=bool), **log_kwargs) | ||
| norm = logflux * psize / beam_size if kwargs.get('norm_reg', True) else 1 | ||
| # compute TV of the log image |
There was a problem hiding this comment.
good catch. Would porting this exact behavior over to standard tv,tv2 meaningfully change behavior? In a case where the edge pixels are zero they should be near-identical, and I think this version with edge padding is strictly better for all versions.
So I would consider re-writing standard tv/tv2 and revert to the log versions calling them. It's ok if they are not byte-identical to the existing versions as long as they pass the finite diff tests.
There was a problem hiding this comment.
I have added the boundary rule to TV, TV2, PTV, and PTV2. The new partial mask value and finite difference tests pass
|
@achael I agree, I have changed Ready for re-review! |
achael
left a comment
There was a problem hiding this comment.
looks good! cleaner and hopefully more stable too
Summary
Fixes incorrect boundary handling in the log-TV and spectral-TV regularizers, plus several avoidable NaNs in polarimetric gradients.
The main bug is that
reg_tvlog/reg_tv2logcompute TV onlog(imvec)through code that pads the image with zero.For linear flux, zero means empty sky outside the FOV, which is a sensible boundary condition.
For a log image, however:
so the regularizer was effectively placing a 1 Jy ring just outside images whose edge pixels may be
1e-4to1e-20Jy.That boundary term was not just cosmetic:
tvlogvalue;tvlogweights it visibly worsens reconstruction quality and pushes flux onto the FOV rim.For example:
At the repository's existing e2e weight of 10 the effect is tiny, which likely explains why this survived.
What changed
For the log and spectral TV families, differences that cross either the FOV boundary or an embed-mask boundary are now excluded.
At the grid edge this is implemented with edge padding, so the boundary difference is exactly zero. At mask boundaries, invalid neighbours are replaced with the current pixel before taking each individual difference.
This applies to:
tvlog/tv2logLinear and polarimetric TV variants keep their existing zero boundary condition because zero outside the image is meaningful for those quantities.
reg_tv,reggrad_tv,reg_tv2, andreggrad_tv2are left byte-identical todev-backend.Partial-mask NaNs
tvlogandtv2logalso failed on partial embed masks because masked pixels were filled withepsilon_tv, which defaults to zero, and then passed throughlog().Partial masks are possible at default settings whenever the prior contains exact-zero pixels.
Masked pixels are now given a positive
flux / npixvalue before taking the log.The important point is that the fill no longer affects the result: no retained TV edge crosses into a masked pixel, and the tests explicitly verify fill independence.
Spectral TV
The spectral regularizers had the same boundary problem, effectively imposing
alpha = 0outside the FOV.The effect is smaller than for
tvlog, but the direction is measurable. In a two-frequency test with truealpha = -1, increasing the spectral-TV weight pulled the interior solution toward zero with the old boundary condition:The new boundary keeps the interior spectral index much more stable. Other metrics improve only modestly and not uniformly, so this part is primarily a correctness fix.
Polarimetric gradient cleanup
This PR also removes three algebraically unnecessary divisions that produced
0/0at valid zero-valued pixels:These occur across seven functions, including
chisqgrad_vvis.Previously, several functions produced NaNs or infinities at masked or zero-polarization pixels and relied on a later mask slice to discard them. They are now finite throughout.
For valid nonzero inputs, values agree with the old expressions to floating-point precision.
One deliberate convention remains: an isolated valid pixel with no valid TV edges contributes
sqrt(epsilon_tv), as before. At the defaultepsilon_tv=0this is zero and has zero gradient.