From a full-repo review (line refs from ecb7fe0).
src/data/transforms/nan_transforms.py:237-324 rules are parameterized for 1440-min single days (all-zero channel = "not worn"; short_sleep_threshold=180 min/day), but forecasting_evaluation/data/online_dataset.py:315/484 applies the transform once to whole multi-month (19, T_total) trajectories:
- the all-zero rule fires only if a channel is zero for the entire study;
- total sleep summed across months is always ≥ 180 min, so the short-sleep rule never fires.
Watch-off-overnight zeros therefore remain as ground truth in the binary sleep channels that AUROC/AUPRC score. Applied identically to all models (so relative comparisons survive) and partially mitigated by the day_remain_mask wear filter — but GT semantics silently diverge from the imputation track and from the transform's documented intent, and wear-pattern differences across demographics can bleed into subgroup metrics.
Suggested fix: apply the transform per-day (reshape to days before the rules), or re-parameterize the rules for trajectory length; add a test with a synthetic watch-off night asserting the zeros become NaN.
From a full-repo review (line refs from
ecb7fe0).src/data/transforms/nan_transforms.py:237-324rules are parameterized for 1440-min single days (all-zero channel = "not worn";short_sleep_threshold=180min/day), butforecasting_evaluation/data/online_dataset.py:315/484applies the transform once to whole multi-month(19, T_total)trajectories:Watch-off-overnight zeros therefore remain as ground truth in the binary sleep channels that AUROC/AUPRC score. Applied identically to all models (so relative comparisons survive) and partially mitigated by the
day_remain_maskwear filter — but GT semantics silently diverge from the imputation track and from the transform's documented intent, and wear-pattern differences across demographics can bleed into subgroup metrics.Suggested fix: apply the transform per-day (reshape to days before the rules), or re-parameterize the rules for trajectory length; add a test with a synthetic watch-off night asserting the zeros become NaN.