Add StrucPhysVideo to Physics-IQ Verified leaderboard

[Estrellama]

This PR adds Physics-IQ Verified evaluation results for Awomo-v0.1.

Evaluation setup:

Benchmark: Physics-IQ Verified
Evaluation script: physiq/run_physics_iq.py
Mode: default Verified evaluation
Input type: image-to-video (i2v)
Number of generated videos per run: 198
Video format: 5 seconds, 15 FPS
Number of runs: 4 independent seeds
Reported score: mean ± sample standard deviation across 4 runs
Scores:

run_01: 44.8119
run_02: 44.4944
run_03: 45.4600
run_04: 45.7751
mean ± std: 45.1353 ± 0.5860

[rgeirhos]

Thanks for the PR! Is there a technical report that can be linked for your model? Currently, the only link points to a website in Chinese.

[Estrellama]

Hi, thanks for the review!

You are right that the current link points to a Chinese blog. Would the PR be acceptable if we replace it with an English blog?

Our technical report is still in preparation. Once it is ready, we will update the link to point to the full technical report. Would this be OK for the current submission?

[rgeirhos]

Thanks for the quick response. It'd be great if you could update the PR with a pointer to the technical report once it's out - I'll then make sure to merge it quickly. Unfortunately, a blog post is not ideal (irrespective of the language) since it lacks the level of detail that e.g. a tech report or reproducible github link can provide.

[Estrellama]

Hi @rgeirhos , thank you again for your patience.

Our technical report is now available on arXiv:

StrucPhysVideo: Learning Physical Dynamics from Structured Captions and Robot Actions

Using the same official Physics-IQ Verified evaluation protocol as described in the PR—image-to-video generation, 198 videos per run, 5-second videos at 15 FPS, and four independent runs—our latest results are:

Run	Verified score
run_01	45.44
run_02	45.17
run_03	45.30
run_04	46.17

Physics-IQ Verified: 45.52 ± 0.39
(mean ± sample standard deviation over 4 independent runs)

The technical report describes the model, training data, and evaluation context in detail. Please let us know if any further information or changes to the PR are needed.

Best regards,
Awomo-WM Team

[rgeirhos]

Thanks for the update, exciting, and congrats on the results!

Happy to merge once we have all the required information; would you mind completing these steps? https://github.com/google-deepmind/physics-IQ-benchmark#preparing-your-submission (we recently updated the submission procedure)

[Estrellama]

@sylvesterkaczmarek
Thank you for pointing this out. It has been over a month since we first opened this PR, and both the model name and the reported results have changed since then, as reflected in the comment history on this PR.

Would you recommend opening a new PR with the updated model name and results, or updating the existing PR directly?

[rgeirhos]

@Estrellama please ignore the comment by @sylvesterkaczmarek; they are not a benchmark maintainer nor otherwise involved in Physics-IQ. The comment has been flagged as spam and reported. Please free to keep this PR or open a new one, whichever is most convenient for you.

[Estrellama]

@rgeirhos Thank you for the clarification. We would like to submit the updated entry through the Physics-IQ Verified submission workflow.

• Organization: Awomo
• Descriptive ID: strucphysvideo-ti2v-bpp
• Model: StrucPhysVideo-TI2V
• Input type: I2V
• Prompt setting: BPP with a static-camera constraint and Qwen3.5-27B temporal prompt expansion
• Physics-IQ Verified score: 45.5 ± 0.4 across four runs

We have prepared the submission card, the descriptions file used for generation, and the four generated-video run directories.

Could you please generate a run ID and provide the scoped temporary S3 credentials for this submission? We will update this PR to ensure that the model name and reported results match the submitted materials exactly.

[sten2lu]

Hi @Estrellama,
thanks for preparing your results. Could you email research@anates.ai? I will then reply with the S3 credentials for the submission.