Skip to content

evals: draw pose-prior noise per seed so photo counts compare on the same pose error #159

Open

Description

@SamGu-NRX

Follow-up from PR #12, review thread #12 (comment).

Problem. group_poses (experiments/evals/evals/ar_poses.py:87-106) draws pose noise for every group from one generator, in group order across all sizes. It also assigns advio_2018 scales by a running group counter (k % 3). So the 2-, 4- and 8-photo groups seeded by the same photo get different noise, and can get different advio_2018 scales. The pose-prior rows for modern_assumed and advio_2018 therefore don't isolate photo count. The exact-pose rows are unaffected because they have no noise, and so are the 8-photo headline rows.

What a fix needs.

  • Draw each seed's pose error and scale once, keyed by setting, draw and seed photo, and apply the same error to that seed's cameras at every group size. For example, derive the noise per camera from a key of seed and camera, not from the running generator.
  • Rerun python -m evals.pose_priors prepare and score. The fit cache recomputes on its own, because its key includes the poses. The last full refit took 11 minutes at 0.98 GB.
  • Update METHODS.md section 3 and any 4-versus-8 comparison, listing old numbers against new.

Constraint. MapAnything's grouped rows used draw-0 poses from the current generator, and its weights are no longer on the machine. A change that alters draw 0 either leaves those rows labelled as run on the earlier poses, or needs MapAnything re-downloaded (about 5 GB) and rerun.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions