Skip to content

Force bool to float32, add explicit error message - #14

Open
tjgalvin wants to merge 12 commits into
mainfrom
beth-fix
Open

tjgalvin wants to merge 12 commits into
mainfrom
beth-fix

Conversation

@tjgalvin

Copy link
Copy Markdown

Beth, of GASKAP-OH fame, pointed out two issues.

The first was that in some circumstances a bool mask array is passed for fits.writeto, which subsequently raises an error. I don't know how long this one has been around for - I feel like I should have seen it long ago. We may have overlooked something when removing the logic from flint to eye-patch. An easy fix anyway.

The next was around how the gaussian kernels are constructed in the beam erosion stage. It is possible for the scale to be so small that a gaussian kernel can not be created. The sigma width is so small that it is well within a single pixel, which then breaks the coordinate system we evaluate the kernel over. In this case I have added a more informative error as there is really nothing I think we ought to be doing here. This error would only be tripped for a very small scale, like --beam-shape-erode-scales 1, otherwise no change.

@tjgalvin
tjgalvin requested a review from AlecThomson August 17, 2026 12:51
@tjgalvin

Copy link
Copy Markdown
Author

I also

  • added a logger sub-module like our other packages
  • copied over the flint precommit yaml. This one was complaining about an older version of python not being compatible with the type alias we are using

Comment thread eye_patch/masking.py
if pix_sigma < 1:
# linspace can only take integer inputs, and if sigma is too small then this array comes
# out as length zero.
msg = f"{scale=} is too small and an appropriately sized kernel can not be formed. Consider removing it. "

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Consider removing it."

What does "it" mean here? Can we make this message more explicit on what to change?

Comment thread .pre-commit-config.yaml Outdated
Comment on lines +28 to +40
# - repo: https://github.com/pre-commit/pygrep-hooks
# rev: "v1.10.0"
# hooks:
# - id: rst-backticks
# - id: rst-directive-colons
# - id: rst-inline-touching-normal

- repo: https://github.com/rbubley/mirrors-prettier
rev: "v3.8.3"
hooks:
- id: prettier
types_or: [yaml, markdown, html, css, scss, javascript, json]
args: [--prose-wrap=always]
# - repo: https://github.com/pre-commit/mirrors-prettier
# rev: "v4.0.0-alpha.8"
# hooks:
# - id: prettier
# types_or: [yaml, markdown, html, css, scss, javascript, json]
# args: [--prose-wrap=always]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are these removed?

Comment thread .pre-commit-config.yaml Outdated
hooks:
- id: mypy
files: eye_patch|tests
files: flint|tests

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
files: flint|tests
files: eye_patch|tests

Bad copy-paste

Comment thread .pre-commit-config.yaml
@AlecThomson

Copy link
Copy Markdown
Contributor

@tjgalvin - can we revert the pre-commit changes? I'd rather address more explicitly, I think its a numpy / mypy / python version clash.

We're also running to an issue I've hit bunch lately with my CD.yml

The fix is to bump to hynek/build-and-inspect-python-package@v3.0.1 Lemme know if you like me to sort that

Three related failures were making CI and CD red on main and on every
open PR.

CD: the "Distribution build" job failed on `twine check --strict` with
"Invalid distribution metadata: '2.5' is not a valid metadata version".
hatchling emits Metadata-Version 2.5, which only Twine 7 understands.
build-and-inspect-python-package gained Twine 7 in v3.0.0, but v3 also
stopped force-tagging minor releases, so the floating `@v2` tag could
never pick the fix up. Pin to v3.0.1.

CI: the "Format" job failed in mypy while parsing numpy's own stubs:
"Type statement is only supported in Python 3.12 and greater". The
Format job runs on Python 3.14, where numpy resolves to 2.5.x, whose
stubs use PEP 695 `type` statements. mypy was targeting 3.10, so it
refused to parse them. numpy 2.5 itself requires Python >=3.12, so the
target has to be at least 3.12 for its stubs to parse.

Align the stale 3.10 references, which sat below the project's own
`requires-python = ">=3.11"`:
  - mypy python_version 3.10 -> 3.12
  - test matrix 3.10 -> 3.11, so the minimum supported version is
    actually exercised (it previously was not tested at all)
  - drop the 3.10 classifier

Raising the mypy target uncovered a real bug that the 3.10 target had
been masking: `reverse_negative_flood_fill` returns a bool array, which
was passed to `beam_shape_erode` (which expects floats) and then handed
to `fits.writeto`, where astropy rejects it. This is the same failure
Beth reported via #14. Cast the flood-fill mask to float32 at the point
it is created, so both the erosion step and the FITS write get a float
array, and drop the now-redundant `type: ignore[assignment]`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb
Target mypy at the project's actual minimum (3.11) rather than 3.12.

numpy 2.5 requires Python >=3.12 and its stubs use PEP 695 `type`
statements, which mypy refuses to parse while targeting anything below
3.12. Pinning the hook's environment to 3.11 resolves numpy to the
2.4.x line, whose stubs parse cleanly at that target, so the type
check now matches `requires-python = ">=3.11"` instead of silently
skipping over it.

The Format job installs 3.11 alongside 3.x so the hook can find it;
3.x stays last and remains the interpreter pre-commit itself runs on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb
Per review on #14:

Revert .pre-commit-config.yaml to the version on main. The copied flint
config regressed several things here: it pointed mypy at `files: flint|tests`
(the wrong package for this repo, so mypy checked nothing), dropped the
`exclude` for the copier answers file, downgraded blacken-docs to black 24,
commented out the prettier and pygrep hooks, renamed `ruff-check` to the
deprecated `ruff` alias, and dropped the numpy/capn-crunch/pytest-stub
dependencies that mypy needs to resolve this package's own types.

The underlying numpy / mypy / python version clash is addressed explicitly
instead, via the CI fixes merged in from #15: mypy targets 3.11 (the project's
real minimum) with the hook pinned to a 3.11 environment, so numpy resolves to
the 2.4.x stubs that parse at that target.

Annotate CustomFormatter.format, which mypy requires now that it actually
checks this package (`disallow_untyped_defs` is on for eye_patch.*).

The bool mask fix is now applied at the point the flood-fill mask is created
rather than only at `fits.writeto`, which also fixes the type error passing it
into `beam_shape_erode`. The `.astype(np.float32)` at the write is kept: it is
no longer load-bearing for the bool bug, but `beam_shape_erode` still returns
float64 on the multi-scale path, so the cast keeps the mask on disk at float32.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb
The docstring claimed a bool return for no/single scale and int32 otherwise.
Neither held: the single-scale path casts to the input dtype, and the
multi-scale path returns the float64 array it accumulates into. Now that the
mask is float from the point it is created, the bool return is gone entirely.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb

Copy link
Copy Markdown
Contributor

@tjgalvin heads up — I've pushed to beth-fix on Alec's behalf, since the pre-commit revert he asked about is tangled up with the CI fix. Nothing of yours was dropped, and no history was rewritten (it's a merge, so your checkout stays valid). Three things:

1. Reverted .pre-commit-config.yaml to main's version. The copied flint config regressed a few things in this repo — most importantly files: flint|tests, which points mypy at a package that doesn't exist here, so it was checking nothing. It also dropped the exclude for the copier answers, downgraded blacken-docs to black 24, commented out the prettier and pygrep hooks, switched ruff-check to the deprecated ruff alias, and dropped the numpy/capn-crunch/pytest-stub deps mypy needs to resolve this package's types.

2. The version clash is fixed explicitly in #15, which is merged into this branch so it can go green now. You were right that it's a numpy / mypy / python clash — the specifics:

requires-python is >=3.11, but [tool.mypy] python_version was still "3.10", left over from the cookie template. The Format job runs on 3.x (3.14), where numpy resolves to 2.5.x, and numpy 2.5's stubs use PEP 695 type statements that mypy won't parse below 3.12. Hence the error you were seeing.

Rather than raise the target to 3.12, the hook is pinned to language_version: python3.11, so numpy resolves to 2.4.x whose stubs parse at 3.11 — mypy now actually verifies the minimum version we claim to support.

3. Annotated CustomFormatter.format. Once mypy points at eye_patch again, disallow_untyped_defs applies, so it needed record: logging.LogRecord) -> str.

On the bool mask

Your diagnosis was right and it's a real bug — mypy caught it independently the moment it targeted a supported Python:

masking.py:910: error: Argument "mask" to "beam_shape_erode" has incompatible type
ndarray[..., dtype[numpy.bool]]; expected ndarray[..., dtype[floating]]

One refinement: #15 moves the cast to where the flood-fill mask is created rather than at fits.writeto. reverse_negative_flood_fill returns bool, so the bool array was already wrong one call earlier, going into beam_shape_erode. Casting at the source fixes both, and since beam_shape_erode takes its return dtype from its input, it also removes the bool return that function could previously produce.

Your .astype(np.float32) at the write is kept. It's no longer what saves us from the bool, but the multi-scale path of beam_shape_erode returns float64, so it's what keeps the mask float32 on disk.

Your kernel-too-small ValueError is untouched — nice catch, that one would have been baffling to debug from the linspace failure alone.


Generated by Claude Code

@codecov

codecov Bot commented Sep 21, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.88889% with 3 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
eye_patch/masking.py 25.00% 3 Missing ⚠️

📢 Thoughts on this report? Let us know!

codecov flagged 0% patch coverage, which turned out to be pointing at a
real gap rather than a threshold quirk: every existing mask test runs with
`flood_fill=False`, so nothing exercised the branch that produced the bool
array. That is why this shipped.

Covers the path end to end and asserts the written mask is floating point.
Verified it fails without the fix, raising the same KeyError from
astropy's image HDU while writing the mask that Beth hit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb
AlecThomson added a commit that referenced this pull request Sep 22, 2026
* Fix CI and CD: twine metadata, mypy/numpy/python version clash

Three related failures were making CI and CD red on main and on every
open PR.

CD: the "Distribution build" job failed on `twine check --strict` with
"Invalid distribution metadata: '2.5' is not a valid metadata version".
hatchling emits Metadata-Version 2.5, which only Twine 7 understands.
build-and-inspect-python-package gained Twine 7 in v3.0.0, but v3 also
stopped force-tagging minor releases, so the floating `@v2` tag could
never pick the fix up. Pin to v3.0.1.

CI: the "Format" job failed in mypy while parsing numpy's own stubs:
"Type statement is only supported in Python 3.12 and greater". The
Format job runs on Python 3.14, where numpy resolves to 2.5.x, whose
stubs use PEP 695 `type` statements. mypy was targeting 3.10, so it
refused to parse them. numpy 2.5 itself requires Python >=3.12, so the
target has to be at least 3.12 for its stubs to parse.

Align the stale 3.10 references, which sat below the project's own
`requires-python = ">=3.11"`:
  - mypy python_version 3.10 -> 3.12
  - test matrix 3.10 -> 3.11, so the minimum supported version is
    actually exercised (it previously was not tested at all)
  - drop the 3.10 classifier

Raising the mypy target uncovered a real bug that the 3.10 target had
been masking: `reverse_negative_flood_fill` returns a bool array, which
was passed to `beam_shape_erode` (which expects floats) and then handed
to `fits.writeto`, where astropy rejects it. This is the same failure
Beth reported via #14. Cast the flood-fill mask to float32 at the point
it is created, so both the erosion step and the FITS write get a float
array, and drop the now-redundant `type: ignore[assignment]`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb

* Pin the mypy pre-commit hook to Python 3.11

Target mypy at the project's actual minimum (3.11) rather than 3.12.

numpy 2.5 requires Python >=3.12 and its stubs use PEP 695 `type`
statements, which mypy refuses to parse while targeting anything below
3.12. Pinning the hook's environment to 3.11 resolves numpy to the
2.4.x line, whose stubs parse cleanly at that target, so the type
check now matches `requires-python = ">=3.11"` instead of silently
skipping over it.

The Format job installs 3.11 alongside 3.x so the hook can find it;
3.x stays last and remains the interpreter pre-commit itself runs on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb

* Correct beam_shape_erode return-dtype docstring

The docstring claimed a bool return for no/single scale and int32 otherwise.
Neither held: the single-scale path casts to the input dtype, and the
multi-scale path returns the float64 array it accumulates into. Now that the
mask is float from the point it is created, the bool return is gone entirely.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb

* Add regression test for the flood-fill mask dtype

codecov flagged 0% patch coverage, which turned out to be pointing at a
real gap rather than a threshold quirk: every existing mask test runs with
`flood_fill=False`, so nothing exercised the branch that produced the bool
array. That is why this shipped.

Covers the path end to end and asserts the written mask is floating point.
Verified it fails without the fix, raising the same KeyError from
astropy's image HDU while writing the mask that Beth hit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb

* Drop explanatory comments from CI and pre-commit config

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb

* Cover the flood-fill mask reaching beam_shape_erode

codecov was still reporting 0% patch coverage. The remaining uncovered
changed line was the `beam_shape_erode` call itself: no test ran
`create_snr_mask_from_fits` with `beam_shape_erode=True`, so the one place
the bool mask was actually passed into a float-typed parameter, which is
what mypy flagged, went unexercised.

Adds a fixture whose image carries the beam keywords, and a test running
flood fill and beam erosion together through the public entry point.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb

* Type the new tests

The two tests added here were unannotated. mypy did not catch it because
disallow_untyped_defs is only switched on for eye_patch.*, not tests.

The new fixture takes `tmp_path` rather than `tmpdir`, since the latter is
`py.path.local` and the surrounding fixture was converting it to a Path
anyway.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb

---------

Co-authored-by: Claude <noreply@anthropic.com>
main now carries the CI/CD fix (#15), the actions bump (#13) and the
pre-commit hook bumps (#8), so this branch needed bringing up to date.

The only conflict was in tests/test_masking.py, where this branch held an
earlier iteration of the flood-fill tests that landed on main via #15.
Resolved in favour of main's version, which is a strict superset: the same
test, now annotated, plus the fixture and beam-erosion test added there.
Nothing from this branch was lost; the PR never touched tests itself.

What remains here is the original contribution: the logging sub-module,
the float32 cast at fits.writeto, and the kernel-too-small ValueError.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb

Copy link
Copy Markdown
Contributor

Brought main into this branch — #15 (CI/CD fix), #13 (actions bump) and #8 (pre-commit hook bumps) have all landed, so it had gone stale and had a conflict.

One conflict, in tests/test_masking.py. This branch held an earlier iteration of the flood-fill tests that subsequently landed on main via #15. Resolved in favour of main's version, which is a strict superset — the same test, now annotated, plus the fixture and beam-erosion test added there. Nothing from here was lost: the PR never touched the tests itself, so all of that content had come from my earlier merges. A merge commit, so no history rewritten and your checkout stays valid.

The diff is now just your three changes, which is what it should have been all along:

eye_patch/logging.py the logging sub-module
masking.py .astype(np.float32) at the write
masking.py the kernel-too-small ValueError

Worth calling out that the logger change fixes a bug nobody had noticed: masking.py had

logger = logging.getLogger("__name__")

That's the string "__name__", not the __name__ variable — which is why the CI logs read INFO __name__:masking.py:780. Every module sharing that name shared one logger. Your sub-module replaces it properly.

22 tests pass and pre-commit is clean locally; CI is re-running on the merge.


Generated by Claude Code

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants