Skip to content

Performance footguns: per-call tracemalloc, repeated dataset/parquet reloads, path-keyed caches #99

Description

@NarayanSchuetz

From a full-repo review (line refs from ecb7fe0). Excludes items already tracked in #15 (skill-rank bootstrap scaling) and #16 (HDF5 chunking).

  • forecasting_evaluation/evaluation/evaluator.py:108-113: tracemalloc.start()/stop() wraps every predict call — 2-4x allocation-cost slowdown for a perf_* column with no consumers. Gate behind a flag.
  • Forecasting cache keys hash file paths, not content (data/online_dataset.py:264-286) — regenerating the dataset in place silently reuses stale H5 caches. (Correctness-adjacent; content fingerprint or mtime in the key.)
  • imputation_evaluation/evaluation/pair_aggregator.py:572-588: aggregate_pairs_by_subgroup re-reads all 19 pair parquets per (attr, group) combination.
  • Forecasting metrics pipeline reloads the full dataset + splits 3-4x per run.
  • Track 1 per-task re-materialization: cohort re-bound/drained per (task, split); Linear._features re-pools full history 64x; GRU-D rebuilds its dataset per task.

Metadata

Metadata

Assignees

No one assigned

    Labels

    performancePerformance and scalability improvements

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions