Humans can infer hidden physical processes from sparse observations, yet current evaluation protocols for Vision Language Models fail to assess whether such physical reasoning is genuinely captured. To address this gap, we introduce Retrospective Physical Process Reasoning, a new evaluation paradigm to reason backward from outcomes under explicit physical constraints. Building on the paradigm, we present RetroHolmes, the first real-world benchmark for Retrospective Physical Process Reasoning, comprising object-centric image pairs annotated with reachability labels and causal step sequences across diverse physical transitions. Using RetroHolmes, we analyze state of the art Vision Language Models and uncover systematic failure modes, including judgment bias in reachability assessment and belief dominance over physical evidence, mirroring sycophancy behavior observed in large language models. We further demonstrate a simple analysis-by-synthesis instantiation with visual simulation as an intermediate step, validating the diagnostic value of RetroHolmes and highlighting the importance of physically grounded intermediate representations for physical reasoning.
Visual fluency in generated video does not imply physical reliability, and a scalar quality score alone is incapable of indicating the obligation a clip violates or the moment it fails. We present VeriPhy, an auditable physical-verification system in which a text-only planner compiles the prompt into typed physical obligations and a statically validated execution plan before any frame is observed. During execution, observations gate and scope only declared calls to frozen low-level experts (e.g., segmentation and tracking, counting, eleven typed physical measurements over the resulting tracks, depth, OCR, and audio-event detection). Each action returns a provenance-carrying evidence record whose payload, when usable, is either a typed measurement or an explicitly tagged learned state. Typed resolvers and fixed composition map usable records to a three-valued state (supported, contradicted, or unknown, surfaced as plausible, implausible, or abstain) with full provenance, so that every verdict is traceable to the evidence that produced it. We anchor evaluation in a 1,500-clip corpus of human-annotated flaw records that localize real generation failures in prompt reference, space, and time. On a 149-clip core carrying 304 such records, VeriPhy accounts for 228, against 164 for a published question-decomposition evaluator given the same clips and the same claims. Recall alone does not separate it from prompting the same backbone monolithically, which reaches 222; what separates them is that each decision retains its evidence record and provenance, making the traces auditable one verdict at a time and usable as the interface through which a critic verdict could be written back into generation.
Wenzhuo Xu, Yu-Chen Zhu, Chongjian Ge et al.· 0 citations
SymboUQ is a symbolic uncertainty quantification framework that estimates final-answer reliability from reasoning traces by distinguishing symbolizability, whether a claim can be represented in the verifier's formal language, from semantic determinacy, whether its execution yields an entailed or contradicted verdict rather than an unknown or not-evaluable outcome.
Dahai Yu, Lin Jiang, Rongchao Xu et al.· 0 citations
: Large Language Models (LLMs) have recently advanced in real-world commonsense reasoning, including understanding everyday object behaviors and inferring their attributes from text. However, they remain limited in reasoning about the real-world consequences of events, such as how object failures, obstructions, or structural changes affect the surrounding environment-especially without visual or sensorimotor input. Existing works like PIQA and NEWTON evaluate narrow sub-skills, such as whether an object action makes sense and whether object properties can be inferred, providing valuable benchmarks for commonsense and physical reasoning but offering limited evaluation of how events alter environmental functionality and downstream conditions. To address this gap, we propose Embodied Semantic Grounding (ESG), a framework that equips LLMs with consequence-aware text representations. ESG learns a consequence-grounded space by aligning event descriptions with affordance maps-structured representations of how an environment can be used or traversed after an event-capturing how structural changes modify environmental functionality. A Flan-T5-XL model is trained with a contrastive alignment objective to encode event descriptions into this space, for coherent prediction of consequences such as collapses, blockages, and environmental changes. Rather than introducing a new language-model architecture, ESG extends affordance-grounding with consequence-level representations of post-event environmental functionality. We evaluate ESG on a unified benchmark comprising PIQA, NEWTON, LIBERO-derived affordance text, and 2400 synthetic scenario-based tasks. Results show that ESG improves performance over baseline language models across commonsense reasoning and consequence-prediction benchmarks. Under structured affordance-map supervision, ESG improves zero-shot accuracy on PIQA and NEWTON and demonstrates improved performance on synthetic consequence-prediction scenarios designed to evaluate post-event environmental reasoning.
Manaswi Kulahara, Khadija Parwez, Faisal Alhwikem et al.· Computers, Materials & C...· 0 citations
Results show that capability failures can manifest as distributed, task-dependent changes in the structure of visible reasoning, and that CoT dynamics agnostic to whether the verbalized trace reflects the model's internal computations can help diagnose and correct failures.
Shashwat Sourav, Aishwarya H. Balwani· 0 citations
Physical understanding and reasoning depend on forming compact and generalizable representations of the world. While modern vision-language models can recognize and explain diverse physical events, they often lack explicit representations of the underlying mechanisms-such as object states, physical parameters, and governing dynamics-needed for reliably reasoning how the world evolves and responds to interventions. In this work, we introduce Code-as-World, a paradigm that represents physical worlds through executable world representations. By expressing physical composition, dynamic evolution, and visual appearance as executable code, Code-as-World provides a compact, quantitatively grounded, and controllable abstraction of the physical world. To construct such representations from multimodal observations, such as natural-language descriptions or real-world videos, we develop an agentic discovery loop inspired by abductive reasoning, where an agent proposes, executes, renders, verifies, and iteratively refines executable world hypotheses. As a concrete application, we use verified executable worlds to provide scalable physical supervision for training vision-language models on quantitative physical reasoning. Experiments show that Code-as-World-VL achieves state-of-the-art performance on QuantiPhy and surpasses leading proprietary models, highlighting the potential of executable world representations as a scalable foundation for physical intelligence.
Han-Yang Wang, Yiyang Cai, Weiliang Chen et al.· 1 citation
A probe corpus of 42 retracted, fraudulent, and pseudoscientific papers is paired with a methodology for eliciting and scoring single-shot model engagement with each paper's framing, indicating an urgent need for guardrail infrastructure for scientific deployment of language models.
V. Rodionov, Shamil Assylbekov· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
