Skip to content

Text2Sim: Agentic Physics-Based Simulation Generation with Distilled Expertise

Sep 2026 · arXiv.org · Vol abs/2609.36593 · 1 citation · 54 references
Computer Science

TL;DR

Text2Sim is presented, a simulation-specialized agentic pipeline that converts a text-only request into an executable, editable dynamic case and achieves higher scores than all four state-of-the-art baselines on both metrics.

Abstract

Creating diverse physical simulations remains labor-intensive because assets, layout, physical parameters, motion, control, and rendering must be designed and debugged jointly. We present Text2Sim, a simulation-specialized agentic pipeline that converts a text-only request into an executable, editable dynamic case. Built on Genesis, Text2Sim uses a hierarchical agentic structure that combines a Planner with specialized Writers, asset-generation tools, and an independent Critic. Compact skills (Debug Cards) distilled from graphics demonstrations provide role-specific physical guidance for execution-based repair. We evaluate physical quality, visual quality, and human preference on 42 held-out prompts spanning rigid, articulated, deformable, and cloth phenomena, with a paper-level split between experience construction and evaluation. We design automatic physical and visual scorers to evaluate the quality of the results, and Text2Sim achieves higher scores than all four state-of-the-art baselines on both metrics. In blinded user studies with these baselines, significantly more participants prefer Text2Sim than prefer the baselines, which is consistent with the results from our automatic scorers. The pipeline also supports a broad range of downstream applications; we select dataset construction and extension to multimodal input as two representative examples. We will release the code, the Debug Card library, and a dataset of generated cases, each pairing the text prompt and rendered video with the executable program, assets, physical parameters, controls, and recorded states.

View source

Similar papers

Preprint Aug 2026

ViSculpt: Visual-Centric Agentic Geometry Editing

This work presents a training-free multi-agent system that edits existing 3D meshes directly in Blender by emulating the iterative workflow of human artists, and views this work as an exploratory step toward visual-centric agentic geometry editing in professional graphics software.

Bo Pang, Jiaqi Pan, Xiao-Chen Zhang et al. · 0 citations
Preprint Aug 2026

4DSynth: Controllable Procedural World Synthesis for Dynamic Embodied Simulation

4DSynth is presented, a controllable procedural system that turns a natural-language description, a blueprint mask, or a single photograph into an editable 4D environment with explicit geometry, animated actors, collision-free trajectories, and physics-ready simulation state.

Ze-Hao Qi, Hao-Chen Luo, Jia-Wang Bian et al. · 0 citations
Preprint Aug 2026

aDSL: Agentic 3D Creation via Joint Agent-Program Design

An Agent-centric Domain-Specific Language (aDSL) and a role-specialized multi-agent system to close the gap between programmatic interfaces and reasoning strengths of LLMs, which favor semantic structure and spatial relations over fragile numeric choices.

Rui-Huan Wang, Si-Tong Wei, Jia-Qi He et al. · 2 citations
Sep 2026

Fysiverse-3D-SimReady Technical Report: Agentic Physical Simulation for Pragmatic 3D World Reconstruction

Experiments show that Fysiverse-3D-SimReady improves input-view alignment, contact plausibility, and physical stability over existing single-image reconstruction and scene generation baselines, while enabling goal-conditioned physical interactions from a single image.

Lin-Tao Wang, Ming-Yang Sun, Yang Liu et al. · 1 citation
Preprint Oct 2026

Video2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos

Building interactive simulators from real-world observations is a promising way to scale embodied data, but current pipelines still rely heavily on manual environment construction and calibration. We study whether frontier foundation models and coding agents can automate this process end to end. We formulate \emph{auto...

Jin-Zhou Tang, Z. Zhang, Jing Yang et al. · 2 citations
#artificial intelligence Preprint Sep 2026

DiagGen: Agentic Generation of Deformable Assets with Sim-based Diagnostics for Robotic Simulation

DiagGen is presented, an agentic framework that turns a single in-the-wild image into a simulation-ready deformable asset through a generate--simulate--diagnose--refine loop, and shows that diagnostics provides useful repair cues and can moderately improve the quality of generated deformable assets.

Guan-Xiong Chen, Yi-Duo Qu, Qian-Jun Xia et al. · 1 citation

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.