This work introduces Principia, a benchmark that evaluates Newtonian physics through relational consistency between paired objects, and introduces a calibration-independent consistency score that quantifies physical violation directly in image space.
Abstract
Evaluating physical reasoning in video models is difficult because absolute motion measurements depend on frame rate, object scale, and camera calibration, all of which are often ambiguous or unavailable in generated video. We propose a different approach. When two objects in the same scene obey the same physical law, their motions must satisfy predictable relationships, and these relationships hold independent of calibration. We introduce Principia, a benchmark that evaluates Newtonian physics through relational consistency between paired objects. Principia spans eight phenomena - gravity, restitution, friction, rotational inertia, projectile motion, momentum, pendulum, and mass-spring oscillation - across translational, rotational, collisional, and oscillatory dynamics, using real-world scenes recorded under controlled protocols. We also introduce a calibration-independent consistency score that quantifies physical violation directly in image space. Across thousands of generations from six state-of-the-art video generators, no model exceeds 0.42 on Principia despite all scoring around 0.8 on VBench. Vision-language models are evaluated on their ability to detect relational physics violations, with the best model achieving only 67% accuracy and most performing near chance level.
This work introduces PhysicsLENS, a dataset and benchmark for evaluating plausibility of physical properties grounded in robotics, and shows that plausible-looking videos often ignore the stated property, and that stating the property lowers plausibility only slightly and not significantly.
Isaiah Milkey, Som Sagar, Aditya Taparia et al.· 0 citations
RididBench is introduced, a simulator-grounded benchmark that compares a generated continuation with a reference rollout from the same initial frame and motion description, with per-frame masks, depth, 6-DoF trajectories, and contacts available for scoring.
PhysWeep finds a specific, reproducible, previously undocumented failure in image-to-video generators, treating a frozen generator as a black box, recovering the realized parameter from generated pixels, and reporting how often generation is trackable at all.
Physical-condition diversity is largely missing from current benchmarks for robot manipulation. While large-scale simulation benchmarks increasingly incorporate variations in object appearance, scene layout, and visual observations, they typically keep the underlying physical parameters fixed. As a result, important so...
Jia-Qi Zhang, Feng Ye, Ming-Jia Yang et al.· 0 citations
EgoGenEval introduces EgoGenEval, a geometry-grounded, pose-free benchmark designed to evaluate the physical consistency of visual generators under ego-motion, and shows that pairwise supervision does not reliably improve camera-motion grounding and scene-state preservation together.
Yi-Lin Long, Chen-Ming Zhu, Zi-Tang Gou et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.