Conversation
|
I also
|
| if pix_sigma < 1: | ||
| # linspace can only take integer inputs, and if sigma is too small then this array comes | ||
| # out as length zero. | ||
| msg = f"{scale=} is too small and an appropriately sized kernel can not be formed. Consider removing it. " |
There was a problem hiding this comment.
"Consider removing it."
What does "it" mean here? Can we make this message more explicit on what to change?
| # - repo: https://github.com/pre-commit/pygrep-hooks | ||
| # rev: "v1.10.0" | ||
| # hooks: | ||
| # - id: rst-backticks | ||
| # - id: rst-directive-colons | ||
| # - id: rst-inline-touching-normal | ||
|
|
||
| - repo: https://github.com/rbubley/mirrors-prettier | ||
| rev: "v3.8.3" | ||
| hooks: | ||
| - id: prettier | ||
| types_or: [yaml, markdown, html, css, scss, javascript, json] | ||
| args: [--prose-wrap=always] | ||
| # - repo: https://github.com/pre-commit/mirrors-prettier | ||
| # rev: "v4.0.0-alpha.8" | ||
| # hooks: | ||
| # - id: prettier | ||
| # types_or: [yaml, markdown, html, css, scss, javascript, json] | ||
| # args: [--prose-wrap=always] |
There was a problem hiding this comment.
Why are these removed?
| hooks: | ||
| - id: mypy | ||
| files: eye_patch|tests | ||
| files: flint|tests |
There was a problem hiding this comment.
| files: flint|tests | |
| files: eye_patch|tests |
Bad copy-paste
|
@tjgalvin - can we revert the pre-commit changes? I'd rather address more explicitly, I think its a numpy / mypy / python version clash. We're also running to an issue I've hit bunch lately with my CD.yml The fix is to bump to |
Three related failures were making CI and CD red on main and on every
open PR.
CD: the "Distribution build" job failed on `twine check --strict` with
"Invalid distribution metadata: '2.5' is not a valid metadata version".
hatchling emits Metadata-Version 2.5, which only Twine 7 understands.
build-and-inspect-python-package gained Twine 7 in v3.0.0, but v3 also
stopped force-tagging minor releases, so the floating `@v2` tag could
never pick the fix up. Pin to v3.0.1.
CI: the "Format" job failed in mypy while parsing numpy's own stubs:
"Type statement is only supported in Python 3.12 and greater". The
Format job runs on Python 3.14, where numpy resolves to 2.5.x, whose
stubs use PEP 695 `type` statements. mypy was targeting 3.10, so it
refused to parse them. numpy 2.5 itself requires Python >=3.12, so the
target has to be at least 3.12 for its stubs to parse.
Align the stale 3.10 references, which sat below the project's own
`requires-python = ">=3.11"`:
- mypy python_version 3.10 -> 3.12
- test matrix 3.10 -> 3.11, so the minimum supported version is
actually exercised (it previously was not tested at all)
- drop the 3.10 classifier
Raising the mypy target uncovered a real bug that the 3.10 target had
been masking: `reverse_negative_flood_fill` returns a bool array, which
was passed to `beam_shape_erode` (which expects floats) and then handed
to `fits.writeto`, where astropy rejects it. This is the same failure
Beth reported via #14. Cast the flood-fill mask to float32 at the point
it is created, so both the erosion step and the FITS write get a float
array, and drop the now-redundant `type: ignore[assignment]`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb
Target mypy at the project's actual minimum (3.11) rather than 3.12. numpy 2.5 requires Python >=3.12 and its stubs use PEP 695 `type` statements, which mypy refuses to parse while targeting anything below 3.12. Pinning the hook's environment to 3.11 resolves numpy to the 2.4.x line, whose stubs parse cleanly at that target, so the type check now matches `requires-python = ">=3.11"` instead of silently skipping over it. The Format job installs 3.11 alongside 3.x so the hook can find it; 3.x stays last and remains the interpreter pre-commit itself runs on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb
Per review on #14: Revert .pre-commit-config.yaml to the version on main. The copied flint config regressed several things here: it pointed mypy at `files: flint|tests` (the wrong package for this repo, so mypy checked nothing), dropped the `exclude` for the copier answers file, downgraded blacken-docs to black 24, commented out the prettier and pygrep hooks, renamed `ruff-check` to the deprecated `ruff` alias, and dropped the numpy/capn-crunch/pytest-stub dependencies that mypy needs to resolve this package's own types. The underlying numpy / mypy / python version clash is addressed explicitly instead, via the CI fixes merged in from #15: mypy targets 3.11 (the project's real minimum) with the hook pinned to a 3.11 environment, so numpy resolves to the 2.4.x stubs that parse at that target. Annotate CustomFormatter.format, which mypy requires now that it actually checks this package (`disallow_untyped_defs` is on for eye_patch.*). The bool mask fix is now applied at the point the flood-fill mask is created rather than only at `fits.writeto`, which also fixes the type error passing it into `beam_shape_erode`. The `.astype(np.float32)` at the write is kept: it is no longer load-bearing for the bool bug, but `beam_shape_erode` still returns float64 on the multi-scale path, so the cast keeps the mask on disk at float32. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb
The docstring claimed a bool return for no/single scale and int32 otherwise. Neither held: the single-scale path casts to the input dtype, and the multi-scale path returns the float64 array it accumulates into. Now that the mask is float from the point it is created, the bool return is gone entirely. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb
|
@tjgalvin heads up — I've pushed to 1. Reverted 2. The version clash is fixed explicitly in #15, which is merged into this branch so it can go green now. You were right that it's a numpy / mypy / python clash — the specifics:
Rather than raise the target to 3.12, the hook is pinned to 3. Annotated On the bool maskYour diagnosis was right and it's a real bug — mypy caught it independently the moment it targeted a supported Python: One refinement: #15 moves the cast to where the flood-fill mask is created rather than at Your Your kernel-too-small Generated by Claude Code |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
codecov flagged 0% patch coverage, which turned out to be pointing at a real gap rather than a threshold quirk: every existing mask test runs with `flood_fill=False`, so nothing exercised the branch that produced the bool array. That is why this shipped. Covers the path end to end and asserts the written mask is floating point. Verified it fails without the fix, raising the same KeyError from astropy's image HDU while writing the mask that Beth hit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb
* Fix CI and CD: twine metadata, mypy/numpy/python version clash
Three related failures were making CI and CD red on main and on every
open PR.
CD: the "Distribution build" job failed on `twine check --strict` with
"Invalid distribution metadata: '2.5' is not a valid metadata version".
hatchling emits Metadata-Version 2.5, which only Twine 7 understands.
build-and-inspect-python-package gained Twine 7 in v3.0.0, but v3 also
stopped force-tagging minor releases, so the floating `@v2` tag could
never pick the fix up. Pin to v3.0.1.
CI: the "Format" job failed in mypy while parsing numpy's own stubs:
"Type statement is only supported in Python 3.12 and greater". The
Format job runs on Python 3.14, where numpy resolves to 2.5.x, whose
stubs use PEP 695 `type` statements. mypy was targeting 3.10, so it
refused to parse them. numpy 2.5 itself requires Python >=3.12, so the
target has to be at least 3.12 for its stubs to parse.
Align the stale 3.10 references, which sat below the project's own
`requires-python = ">=3.11"`:
- mypy python_version 3.10 -> 3.12
- test matrix 3.10 -> 3.11, so the minimum supported version is
actually exercised (it previously was not tested at all)
- drop the 3.10 classifier
Raising the mypy target uncovered a real bug that the 3.10 target had
been masking: `reverse_negative_flood_fill` returns a bool array, which
was passed to `beam_shape_erode` (which expects floats) and then handed
to `fits.writeto`, where astropy rejects it. This is the same failure
Beth reported via #14. Cast the flood-fill mask to float32 at the point
it is created, so both the erosion step and the FITS write get a float
array, and drop the now-redundant `type: ignore[assignment]`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb
* Pin the mypy pre-commit hook to Python 3.11
Target mypy at the project's actual minimum (3.11) rather than 3.12.
numpy 2.5 requires Python >=3.12 and its stubs use PEP 695 `type`
statements, which mypy refuses to parse while targeting anything below
3.12. Pinning the hook's environment to 3.11 resolves numpy to the
2.4.x line, whose stubs parse cleanly at that target, so the type
check now matches `requires-python = ">=3.11"` instead of silently
skipping over it.
The Format job installs 3.11 alongside 3.x so the hook can find it;
3.x stays last and remains the interpreter pre-commit itself runs on.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb
* Correct beam_shape_erode return-dtype docstring
The docstring claimed a bool return for no/single scale and int32 otherwise.
Neither held: the single-scale path casts to the input dtype, and the
multi-scale path returns the float64 array it accumulates into. Now that the
mask is float from the point it is created, the bool return is gone entirely.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb
* Add regression test for the flood-fill mask dtype
codecov flagged 0% patch coverage, which turned out to be pointing at a
real gap rather than a threshold quirk: every existing mask test runs with
`flood_fill=False`, so nothing exercised the branch that produced the bool
array. That is why this shipped.
Covers the path end to end and asserts the written mask is floating point.
Verified it fails without the fix, raising the same KeyError from
astropy's image HDU while writing the mask that Beth hit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb
* Drop explanatory comments from CI and pre-commit config
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb
* Cover the flood-fill mask reaching beam_shape_erode
codecov was still reporting 0% patch coverage. The remaining uncovered
changed line was the `beam_shape_erode` call itself: no test ran
`create_snr_mask_from_fits` with `beam_shape_erode=True`, so the one place
the bool mask was actually passed into a float-typed parameter, which is
what mypy flagged, went unexercised.
Adds a fixture whose image carries the beam keywords, and a test running
flood fill and beam erosion together through the public entry point.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb
* Type the new tests
The two tests added here were unannotated. mypy did not catch it because
disallow_untyped_defs is only switched on for eye_patch.*, not tests.
The new fixture takes `tmp_path` rather than `tmpdir`, since the latter is
`py.path.local` and the surrounding fixture was converting it to a Path
anyway.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb
---------
Co-authored-by: Claude <noreply@anthropic.com>
main now carries the CI/CD fix (#15), the actions bump (#13) and the pre-commit hook bumps (#8), so this branch needed bringing up to date. The only conflict was in tests/test_masking.py, where this branch held an earlier iteration of the flood-fill tests that landed on main via #15. Resolved in favour of main's version, which is a strict superset: the same test, now annotated, plus the fixture and beam-erosion test added there. Nothing from this branch was lost; the PR never touched tests itself. What remains here is the original contribution: the logging sub-module, the float32 cast at fits.writeto, and the kernel-too-small ValueError. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GTPJweMZavwh86CQ7qH7rb
|
Brought One conflict, in The diff is now just your three changes, which is what it should have been all along:
Worth calling out that the logger change fixes a bug nobody had noticed: logger = logging.getLogger("__name__")That's the string 22 tests pass and pre-commit is clean locally; CI is re-running on the merge. Generated by Claude Code |
Beth, of GASKAP-OH fame, pointed out two issues.
The first was that in some circumstances a
boolmask array is passed forfits.writeto, which subsequently raises an error. I don't know how long this one has been around for - I feel like I should have seen it long ago. We may have overlooked something when removing the logic from flint to eye-patch. An easy fix anyway.The next was around how the gaussian kernels are constructed in the beam erosion stage. It is possible for the scale to be so small that a gaussian kernel can not be created. The sigma width is so small that it is well within a single pixel, which then breaks the coordinate system we evaluate the kernel over. In this case I have added a more informative error as there is really nothing I think we ought to be doing here. This error would only be tripped for a very small scale, like
--beam-shape-erode-scales 1, otherwise no change.