You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Follow up final eval JSON types and cache-only model loading #157
At PR #12 head 17e338d, the final bot reviews identified two non-blocking follow-ups.
Keep pass flags boolean in JSON
experiments/evals/evals/map3d.py:169-172 can return numpy.bool_. The default=float JSON fallback serializes it as 1.0 or 0.0. The committed facing rows in results/map3d.json:53-79 contain 1.0 while other pass flags are booleans. The threshold calculation itself is unchanged; the defect is the inconsistent output type.
Return a native bool or use a NumPy-aware serializer. Add a strict type regression and update the saved JSON without changing the measured values.
experiments/evals/models/run_groups.py:73-82 loads and verifies the model before the matching-input cache check at 96-99. A cache-only invocation still needs weights, can fail offline after the checkpoint was removed, and allocates the model without inference. This is unnecessary work, not evidence of uncontrolled memory growth or a wrong result.
Calculate candidate fingerprints from the pinned checkpoint identity first. Load and verify the actual checkpoint when inference is required, preserving cache validation and provenance checks. Test an all-cache-hit invocation with the model loader mocked to fail if called.
Deferred under the manager's normal-use/plausible-failure convergence standard. The separate pose-realization and grouped-publication findings are not included in this issue.
Activity