Skip to content

Bound NetCDF chunks for long native-grid records - #5

Merged
yuyi13 merged 1 commit into
mainfrom
codex/bounded-netcdf-20261017
Oct 17, 2026
Merged

yuyi13 merged 1 commit into
mainfrom
codex/bounded-netcdf-20261017

Conversation

@yuyi13

@yuyi13 yuyi13 commented Oct 17, 2026

Copy link
Copy Markdown
Owner

Long native-grid records could use multi-year NetCDF chunks while exporters copied 32-day blocks, causing repeated decompression and compression. Align output chunks with temporal and seasonal row writes, and temporarily cap each source variable's read cache at 512 MiB when copying older chunk layouts. Apply the same policy to final crops, initialization/trimming, paired auxiliaries and seasonal controls while preserving values, missing-data handling and CF metadata.

Validation:

  • Gadi job 181481635: 387 tests, source/wheel builds, installed-wheel demo and the offline paired workflow passed.
  • A full 5,844-day native-grid crop finished in 246 seconds; all 50 variables exactly matched their source subsets and the 167 × 167 grid contract passed. Peak parent memory was 3.38 GiB.
  • Regression checks cover cache restoration, classic NetCDF input, reordered auxiliary dates, seasonal missing values and explicit row/time chunk layouts.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-17T16:47:06.010187Z 3e4d1c2 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@yuyi13
yuyi13 merged commit affc1d9 into main Oct 17, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant