Text2Sim is presented, a simulation-specialized agentic pipeline that converts a text-only request into an executable, editable dynamic case and achieves higher scores than all four state-of-the-art baselines on both metrics.
Abstract
Creating diverse physical simulations remains labor-intensive because assets, layout, physical parameters, motion, control, and rendering must be designed and debugged jointly. We present Text2Sim, a simulation-specialized agentic pipeline that converts a text-only request into an executable, editable dynamic case. Built on Genesis, Text2Sim uses a hierarchical agentic structure that combines a Planner with specialized Writers, asset-generation tools, and an independent Critic. Compact skills (Debug Cards) distilled from graphics demonstrations provide role-specific physical guidance for execution-based repair. We evaluate physical quality, visual quality, and human preference on 42 held-out prompts spanning rigid, articulated, deformable, and cloth phenomena, with a paper-level split between experience construction and evaluation. We design automatic physical and visual scorers to evaluate the quality of the results, and Text2Sim achieves higher scores than all four state-of-the-art baselines on both metrics. In blinded user studies with these baselines, significantly more participants prefer Text2Sim than prefer the baselines, which is consistent with the results from our automatic scorers. The pipeline also supports a broad range of downstream applications; we select dataset construction and extension to multimodal input as two representative examples. We will release the code, the Debug Card library, and a dataset of generated cases, each pairing the text prompt and rendered video with the executable program, assets, physical parameters, controls, and recorded states.
This work presents a training-free multi-agent system that edits existing 3D meshes directly in Blender by emulating the iterative workflow of human artists, and views this work as an exploratory step toward visual-centric agentic geometry editing in professional graphics software.
Bo Pang, Jiaqi Pan, Xiao-Chen Zhang et al.· 0 citations
4DSynth is presented, a controllable procedural system that turns a natural-language description, a blueprint mask, or a single photograph into an editable 4D environment with explicit geometry, animated actors, collision-free trajectories, and physics-ready simulation state.
Ze-Hao Qi, Hao-Chen Luo, Jia-Wang Bian et al.· 0 citations
An Agent-centric Domain-Specific Language (aDSL) and a role-specialized multi-agent system to close the gap between programmatic interfaces and reasoning strengths of LLMs, which favor semantic structure and spatial relations over fragile numeric choices.
Rui-Huan Wang, Si-Tong Wei, Jia-Qi He et al.· 2 citations
Experiments show that Fysiverse-3D-SimReady improves input-view alignment, contact plausibility, and physical stability over existing single-image reconstruction and scene generation baselines, while enabling goal-conditioned physical interactions from a single image.
Lin-Tao Wang, Ming-Yang Sun, Yang Liu et al.· arXiv.org· 1 citation
Building interactive simulators from real-world observations is a promising way to scale embodied data, but current pipelines still rely heavily on manual environment construction and calibration. We study whether frontier foundation models and coding agents can automate this process end to end. We formulate \emph{auto...
Jin-Zhou Tang, Z. Zhang, Jing Yang et al.· 2 citations
DiagGen is presented, an agentic framework that turns a single in-the-wild image into a simulation-ready deformable asset through a generate--simulate--diagnose--refine loop, and shows that diagnostics provides useful repair cues and can moderately improve the quality of generated deformable assets.
Guan-Xiong Chen, Yi-Duo Qu, Qian-Jun Xia et al.· 1 citation