Add recent Physics-IQ Verified leaderboard entries

[tuliti]

Adds four I2V entries to the Physics-IQ Verified leaderboard and reorders the I2V rankings accordingly:

• MiniMax H3 (FL2VA): 39.8 ± 0.4
• MiniMax H3 Max: 36.2 ± 0.7
• CogVideoX-5B: 31.8 ± 1.5
• Wan 2.2 5B: 27.7 ± 0.9

[rgeirhos]

Thanks for adding these!

Would you mind linking a "reported here" with details as well? The details can be anywhere (including e.g. in this PR) as long as there's enough level of detail such that others would be able to reproduce the setup.

[tuliti]

Details below for each added model.

MiniMax H3 (FL2VA)

• Organization: MiniMax
• Model reference: MiniMaxAI/MiniMax-H3
• Model/run identifier: MiniMax H3 FL2VA
• Parameters: 33B
• Input type: image-to-video (I2V)
• Benchmark: Physics-IQ Verified
• Score: 39.79 ± 0.35 (table display: 39.8 ± 0.4)
• Runs / seeds: 4 complete runs; seeds 42, 43, 44, 45
  • 42: 39.95
  • 43: 39.95
  • 44: 39.99
  • 45: 39.26
• Reported submetrics:
  • Spatial IoU: 58.90 ± 0.78
  • Spatiotemporal IoU: 22.80 ± 0.55
  • Weighted Spatial IoU: 40.47 ± 0.78
  • MSE-based score: 36.98 ± 0.70
• Prompt setting: BPP Opus (bpp-opus), with Claude Opus 4.8 prompt upsampling
• Generation hardware: 4 × NVIDIA H100
• Average generation time: 98 seconds per video
• Output / evaluation: 1344 × 768, 24 FPS; 124 native frames, terminally trimmed to 120 frames / 5.0 seconds; no cadence conversion
• Evaluation date: 2026-08-24

MiniMax H3 Max

• Organization / provider: MiniMax H3 Max via fal.ai
• Model reference: fal.ai: minimax/h3-max/image-to-video
• Input type: image-to-video (I2V)
• Benchmark: Physics-IQ Verified
• Score: 36.21 ± 0.67 (table display: 36.2 ± 0.7)
• Runs / seeds: 4 complete runs; seeds 42, 43, 44, 45
  • 42: 35.40
  • 43: 36.91
  • 44: 36.57
  • 45: 35.94
• Reported submetric means:
  • Spatial IoU: 54.90
  • Weighted Spatial IoU: 38.19
  • Spatiotemporal IoU: 24.50
  • MSE-based score: 27.23
• Prompt setting: BPP Opus, with fal.ai balanced API prompt expansion applied on top of frozen BPP Opus prompts
• Generation settings: requested 5 seconds at 768P; native output 1344 × 768, 24 FPS, 124 frames; terminally trimmed to 120 frames / 5.0 seconds; safety checker enabled; no interpolation, upsampling, or frame-rate conversion
• Provider runtime: approximately 2.91 seconds provider-reported inference time per video; submission concurrency 20
• Hardware: provider-managed API hardware was not disclosed
• API price: $0.08 per generated second ($0.40 per five-second video)
• Evaluation date: 2026-08-27

CogVideoX-5B

• Organization: Z.ai
• Model reference: zai-org/CogVideoX-5b-I2V
• Parameters: 5B
• Input type: image-to-video (I2V)
• Benchmark: Physics-IQ Verified, Take 1
• Score: 31.75 ± 1.47 (table display: 31.8 ± 1.5)
• Runs / seeds: 4 complete runs; seeds 42, 43, 44, 45
  • 42: 33.08
  • 43: 30.34
  • 44: 32.97
  • 45: 30.61
• Reported submetrics:
  • Spatial IoU: 37.77 ± 4.60
  • Spatiotemporal IoU: 35.45 ± 4.91
  • Weighted Spatial IoU: 21.81 ± 1.89
  • MSE-based score: 31.97 ± 1.71
• Prompt setting: BPP using fixed prepared BPP prompts; no generation-time prompt upsampling
• Generation settings: 720 × 480; 49 native frames at 8 FPS (6.125 seconds); terminally trimmed to 40 frames / 5.0 seconds; 60 inference steps; guidance scale 6.0; no frame interpolation, duplication, or FPS conversion
• Generation hardware: NVIDIA A100; one GPU per shard job
• Measured generation time: 521.59 seconds per video; 114.75 total A100 GPU-hours across 792 videos
• Evaluation date: 2026-08-18

Wan 2.2 5B

• Organization: Alibaba Group
• Model reference: Wan-AI/Wan2.2-TI2V-5B
• Model/run identifier: Wan2.2-TI2V-5B
• Parameters: 5B
• Input type: image-to-video (I2V)
• Benchmark: Physics-IQ Verified
• Score: 27.71 ± 0.91 (table display: 27.7 ± 0.9)
• Reported submetrics:
  • Spatial IoU: 35.89 ± 1.08
  • Spatiotemporal IoU: 26.44 ± 1.24
  • Weighted Spatial IoU: 22.96 ± 1.74
  • MSE-based score: 25.54 ± 0.29
• Prompt setting: BPP
• Generation settings: 1248 × 704, 24 FPS; 121 native frames (about 5.04 seconds), terminally trimmed to 120 frames / 5.0 seconds
• Generation hardware/time reference: 1 × NVIDIA GeForce RTX 4090; official I2V-5B 720P reference benchmark: 524.8 seconds per generation
• Evaluation date: 2026-08-18

[rgeirhos]

Wonderful, thanks Tim. LGTM once the merge conflicts are resolved.

[rgeirhos]

Sorry one more question: what's the difference between Wan 2.2 reported in Physics-IQ Verified vs. this Wan 2.2 5B entry? Is there something we could add to the previous Wan 2.2 entry to make the delta clear, e.g. was this a different model size than 5B?

[tuliti]

Thanks — clarified in the latest commit. The existing 32.2 ± 0.6 entry is now labelled Wan 2.2 14B; the newly added 27.7 ± 0.9 entry is Wan 2.2 5B. They are distinct model-size variants.

[rgeirhos]

LGTM