Skip to content

[fix] train on trial-aligned windows for correct R² evaluation - #6

Merged
peterajhgraham merged 1 commit into
mainfrom
claude/trial-aligned-windows-I58vY
May 18, 2026
Merged

peterajhgraham merged 1 commit into
mainfrom
claude/trial-aligned-windows-I58vY

Conversation

@peterajhgraham

Copy link
Copy Markdown
Owner

Summary

  • Replaces NLBDataset (sliding windows over full recording) with TrialAlignedDataset via build_trial_aligned_datasets in scripts/train_benchmark.py
  • Sliding windows sample ~85% rest periods where velocity ≈ 0, producing R² ≈ 0 regardless of model quality; trial-aligned windows match the published NLB evaluation protocol
  • Each sample is now a fixed [-100ms, +500ms] window around move_onset_time, yielding meaningful R² (Wiener filter baseline ~0.49)
  • Replaces --data-root CLI arg with --nwb-path pointing directly to the MC_Maze train+behavior NWB file
  • Removes the now-unnecessary val subsetting (trial datasets are already small — one sample per trial)
  • Renames train_windows/val_windows keys in the JSON output to train_trials/val_trials

Test plan

  • Run python scripts/train_benchmark.py --nwb-path <path-to-nwb> --max-steps 100 and confirm it loads without error
  • Confirm train_trials and val_trials appear in benchmarks/training/results_raw.json
  • Confirm R² after convergence is meaningfully above 0 (expected 0.3–0.6 range)

https://claude.ai/code/session_01AcwBjnbswWKkeyujGsuFzy


Generated by Claude Code

Switch train_benchmark.py from NLBDataset (sliding windows over the full
recording) to TrialAlignedDataset (one window per trial, -100ms to +500ms
around move_onset_time). The sliding-window approach samples ~85% rest
periods, collapsing R² to ~0; trial-aligned windows match the published NLB
evaluation protocol and yield meaningful R² (Wiener baseline ~0.49).

https://claude.ai/code/session_01AcwBjnbswWKkeyujGsuFzy
@peterajhgraham
peterajhgraham marked this pull request as ready for review May 18, 2026 21:28
@peterajhgraham
peterajhgraham merged commit dbd155b into main May 18, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants