Skip to content

Track 3: ZeroToNaNTransform heuristics are inert when applied to full multi-month trajectories #89

Description

@NarayanSchuetz

From a full-repo review (line refs from ecb7fe0).

src/data/transforms/nan_transforms.py:237-324 rules are parameterized for 1440-min single days (all-zero channel = "not worn"; short_sleep_threshold=180 min/day), but forecasting_evaluation/data/online_dataset.py:315/484 applies the transform once to whole multi-month (19, T_total) trajectories:

  • the all-zero rule fires only if a channel is zero for the entire study;
  • total sleep summed across months is always ≥ 180 min, so the short-sleep rule never fires.

Watch-off-overnight zeros therefore remain as ground truth in the binary sleep channels that AUROC/AUPRC score. Applied identically to all models (so relative comparisons survive) and partially mitigated by the day_remain_mask wear filter — but GT semantics silently diverge from the imputation track and from the transform's documented intent, and wear-pattern differences across demographics can bleed into subgroup metrics.

Suggested fix: apply the transform per-day (reshape to days before the rules), or re-parameterize the rules for trajectory length; add a test with a synthetic watch-off night asserting the zeros become NaN.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingmedium priorityMedium priority

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions