Skip to content

Keep clip_extrema indices in the array namespace - #954

Merged
mwcraig merged 5 commits into
astropy:mainfrom
nomad3:fix-928-namespace-flat-indices
Aug 1, 2026
Merged

Keep clip_extrema indices in the array namespace#954
mwcraig merged 5 commits into
astropy:mainfrom
nomad3:fix-928-namespace-flat-indices

Conversation

@nomad3

@nomad3 nomad3 commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

Summary

  • replace NumPy ravel_multi_index with equivalent C-order arithmetic in the selected array namespace
  • keep coordinate indices on the input array device and use Array API-compatible reshaping/indexing
  • add an asymmetric regression that checks the exact high/low flat indices and non-default-device preservation

Fixes #928

Validation

  • python -m pytest ccdproc -q — 380 passed, 29 skipped
  • python -m pytest ccdproc/tests/test_combiner.py -q — 87 passed with NumPy and 87 passed with Dask
  • JAX Combiner module with 64-bit enabled — 81 passed, 6 established xfails
  • focused regression with array-api-strict on a non-default device — 1 passed
  • Ruff, Black, git diff --check, and git show --check — passed

array-api-strict deliberately excludes integer-array assignment, so the focused strict regression replaces only the final xpx.at update with a test spy. The production update path is exercised by the NumPy, Dask, and JAX runs; strict verifies that the exact C-order indices are computed in the selected namespace and remain on the data device.

Checklist

  • For new contributors: Did you add yourself to the AUTHORS.rst file? (The entry is already present on current main, so it is intentionally not duplicated here.)
  • For documentation changes: Does your commit message include a [skip ci]? (Not a documentation-only change.)
  • For bugfixes: Did you add an entry to the CHANGES.rst file?
  • For bugfixes: Did you add a regression test?
  • For bugfixes: Does the commit message include Fixes #928?
  • Does this PR add, rename, move or remove any existing functions or parameters?

AI assistance disclosure

OpenAI Codex was used for source inspection, test design, implementation, and automated review. I have personally reviewed the current diff and validation evidence and take responsibility for this contribution.

@codecov

codecov Bot commented Jul 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.83%. Comparing base (21041b0) to head (89e90f9).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #954   +/-   ##
=======================================
  Coverage   95.82%   95.83%           
=======================================
  Files           8        8           
  Lines        1629     1631    +2     
=======================================
+ Hits         1561     1563    +2     
  Misses         68       68           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mwcraig

mwcraig commented Jul 26, 2026

Copy link
Copy Markdown
Member

This PR remains a draft pending the contributor's personal review; the contributor will take responsibility for the final contribution before marking it ready.

Can you please go ahead and make this non-draft when you have a chance to look it over?

@mwcraig

mwcraig commented Jul 31, 2026

Copy link
Copy Markdown
Member

@nomad3 -- is this one ready for review?

nomad3 added 3 commits August 1, 2026 12:09
Fixes astropy#928

Signed-off-by: Simon Aguilera <saguilera1608@gmail.com>
Signed-off-by: Simon Aguilera <saguilera1608@gmail.com>
Signed-off-by: Simon Aguilera <saguilera1608@gmail.com>
@nomad3
nomad3 force-pushed the fix-928-namespace-flat-indices branch from aa5beee to 8860623 Compare August 1, 2026 16:11
Signed-off-by: Simon Aguilera <saguilera1608@gmail.com>
@nomad3
nomad3 marked this pull request as ready for review August 1, 2026 16:17
@nomad3

nomad3 commented Aug 1, 2026

Copy link
Copy Markdown
Contributor Author

Yes—thank you for checking. I have personally reviewed the rebased diff and fresh validation, corrected the changelog placement, and marked it ready for review. All completed exact-head CI jobs are green; the remaining Dask+Bottleneck matrix job is still queued.

@mwcraig mwcraig left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This review was drafted by Claude Code and is being posted on behalf of @mwcraig, who reviewed and approved its contents.

Thanks @nomad3 — this is a careful fix and an unusually well-constructed test.

I verified the replacement arithmetic by hand: the accumulation loop over shape[1:] is exactly C-order ravel_multi_index, and the strict=True zip guards against a coordinate/shape mismatch. Two quiet improvements ride along: argsorted[(i, ...)] removes the old hardcoded assumption of a 3D stack (which also violated array-api-strict indexing rules for 3D-image stacks), and creating mg on the data device removes the implicit host-to-device transfer that #928 was about. I also recomputed the test's hardcoded flat indices and expected mask for the 3×2×3 data and they're correct.

One optional suggestion — happy to merge as-is if you'd rather not:

  • The test builds the Combiner via object.__new__ and sets _xp / _data_arr / _data_arr_mask directly, which couples it to private internals for every backend even though only array-api-strict needs the bypass (it can't stack a list of arrays). If you're willing, branching so that non-strict backends construct the Combiner normally (from a list of CCDData) and only strict uses the bypass would make the test more robust against future changes to clip_extrema's internals. Entirely non-blocking.

Signed-off-by: Simon Aguilera <saguilera1608@gmail.com>

@mwcraig mwcraig left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again @nomad3!

@mwcraig
mwcraig merged commit 5e7e21d into astropy:main Aug 1, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Array API: clip_extrema uses numpy ravel_multi_index on index arrays from the array namespace

2 participants