Add cycle-aware NCEP observation windows - #1012
Conversation
cf99578 to
b7ac02d
Compare
|
@aayushg55 I changed my mind 😆 and just make cycle_aware always an option and have it on by default since that represents the observations available in the operational sense. lmk your thoughts |
Greptile SummaryThis PR adds a
|
| Filename | Overview |
|---|---|
| earth2studio/data/utils_ncep.py | Adds cycle_aware parameter to observation_cycle_times and cycle_windows; changes default cycle-selection logic from end-of-window to start-of-window. The tolerance_lower >= timedelta(0) guard in the cycle_aware=False path is a deliberate semantic refinement. |
| earth2studio/data/ufs.py | Routes UFSObsConv and UFSObsSat task planning through the shared cycle_windows utility, matching the pattern already used by GDAS/NNJA. Adds cycle_aware parameter to _UFSObsBase.__init__ and UFSObsSat.__init__. |
| earth2studio/data/gdas.py | Minimal change: threads cycle_aware through _create_tasks to cycle_windows. Logic is correct and follows the new shared pattern. |
| earth2studio/data/nnja.py | Adds cycle_aware to NNJAObsConv and NNJAObsSat; threads it through cycle_windows in _create_tasks. All three call sites updated consistently. |
| test/data/test_utils_ncep.py | Renames and expands the cycle-times test to cover both cycle_aware=True and cycle_aware=False for each existing case. |
| test/data/test_ufs.py | New parametrized test covers both UFSObsConv and UFSObsSat for cycle_aware=True and False with a single requested time. No test exercises multiple requested times sharing the same 6-hour cycle. |
| test/data/test_gdas.py | Cycle-aware assertions added inline to the existing task-creation test. Correct assertions for both aware and unaware modes. |
| test/data/test_nnja.py | New standalone test for NNJAObsConv cycle_aware selection; existing NNJAObsSat cycle-window test updated for new default semantics and extended with explicit cycle_aware=False assertions. |
Reviews (1): Last reviewed commit: "Update changelog for cycle-aware observa..." | Re-trigger Greptile
eeb5325 to
400a5a3
Compare
400a5a3 to
40bd633
Compare
|
Thanks @NickGeneva . I like the idea of the cycle_aware flag. But I think this doesn't fully resolve the issue as is. See some examples of how tolerances map to files under each setting of the flag and the actual desired file. While it is mostly okay, in some cases it reads additional files or still causes some overlap. Maybe a simpler way to do this is the below: Where we separate the file selection (force the requested time to be cycle aligned so that it removes all ambiguity) and the time range of obs returned within that file. |

Summary
Stacked on: #1011
cycle_awareoption to UFS, GDAS, NNJA conventional, and NNJA satellite observation sources.cycle_aware=Trueexcludes future cycle files relative to the request upper tolerance bound;cycle_aware=Falseretains retrospective reads that include the next cycle file when needed.Closes: #967
Validation
uv run ruff check earth2studio/data/utils_ncep.py earth2studio/data/ufs.py earth2studio/data/gdas.py earth2studio/data/nnja.py test/data/test_utils_ncep.py test/data/test_ufs.py test/data/test_gdas.py test/data/test_nnja.pyuv run mypy --ignore-missing-imports earth2studio/data/utils_ncep.py earth2studio/data/ufs.py earth2studio/data/gdas.py earth2studio/data/nnja.pyuv run pytest test/data/test_ufs.py -q -m 'not slow'observation_cycle_timesassertions for cycle-aware and cycle-unaware windowsNote: local GDAS/NNJA test files are dependency-gated in this environment (
data/pybufrkit) and were skipped by pytest.