Generative and predictive artificial intelligence models are increasingly used to generate geometry and to predict physical fields and scalar quantities in engineering design and simulation. Yet these models are typically evaluated in isolation, on academic datasets at unconstrained scales, with inconsistent metrics and procedures. We present PhysicsBench, a unified benchmark and leaderboard that evaluates generative and predictive models under one standardized procedure. PhysicsBench spans seven generation and prediction tasks across 1D, 2D, and 3D domains and ranks 66 models on nine datasets, comprising industrial-scale CAD/CFD/FEA simulations and public references, expanded into 28 configurations. One procedure and ranking apply to both families, each ranked within its own tasks. Evaluation spans realistic, limited data scales from S to XL rather than the unlimited training sets common in academic benchmarks. A common metric suite captures geometric fidelity with distributional distances, physical-field and scalar accuracy, and engineering-specific field- and shape-validity. BenchRank debiases correlated metrics and ranks by PageRank over a head-to-head dominance graph, so every reported quality metric is also ranked, with computational cost in a separate efficiency view. Across tasks, an architecture's large-scale academic standing weakly predicts its small-data ranking. The top model changes with data scale in six of the seven tasks, and no model leads more than one task. PhysicsBench turns"state-of-the-art"from a self-reported claim into an openly published foundation for model selection.
GAUGE, a real-world-grounded diagnostic benchmark for jointly evaluating how numerical simulators and generative video world models reproduce or deviate from real-world physics, is introduced and no uniformly faithful physics engine is revealed.
Shuai Wang, Yaxin Feng, Xuekun Jiang et al.· 2 citations
Parametric computer-aided design (CAD) modeling is difficult to evaluate with a single metric. Existing CAD benchmarks often emphasize synthetic or CAD-native settings, limited input modalities, or executability and IoUs alone. We introduce RealCADBench, a benchmark for intent-to-program CAD modeling from real industrial design intents. It contains 12,632 tasks from 19 factory-automation categories and spans text descriptions, 2D engineering drawings, real product pictures, and rendered images for both Part and Assembly modeling. We report results on a 1,770-task evaluation slice: 1,745 Part tasks across four input regimes and RCB-Assm25, a 25-task assembly study used in every reported assembly comparison. Each method generates FreeCAD API Python, which a shared runtime executes to export the 3D model. We evaluate the exported model using executability, Solid IoU, Surface IoU, and a rubric-based visual-semantic identity Judge. Among the nine standalone frontier large models evaluated, no model leads all four metrics. Across six frontier-scale large models, executability ranges from 0.565 to 0.812, Solid IoU from 0.2841 to 0.5379, and Surface IoU from 0.112 to 0.217 across the four Part regimes. The highest regime-balanced composite comes from a different model than the leaders on the four component metrics. On RCB-Assm25, Codex with GPT-5.5 improves executability and both IoU metrics over standalone GPT-5.5, but lowers the Judge score by 6.98 percentage points, leaving GPT-5.5 as the Judge leader. We also observe recurring failure modes, most notably missing fine structures, loss of part identity, and incorrect assembly placement. These results show that execution alone is insufficient to characterize realistic CAD modeling and that frontier models and agents differ substantially across executability, IoUs, and visual-semantic identity.
JoyIndustrial VisCAD Team Linxin Cai, Qiu-He Hong, Zhi-Chao Huang et al.· 0 citations
MecEng, a fully automated benchmark that evaluates LLMs on the creation of multibody simulation models from parameterized textual descriptions, indicates rapidly improving, but still error-prone, mechanical engineering awareness of current LLMs.
Johannes Gerstmayr, Sebastian Weyrer, Tobias Möltner et al.· 0 citations
Neural surrogates offer a promising route to accelerating computationally expensive simulations governed by partial differential equations across science and industry. Their practical deployment, however, is limited by unreliable predictions under out-of-distribution (OOD) conditions. We develop a solver-coupled surrogate-Newton framework that uses surrogate predictions as high-quality initial guesses for Newton-Krylov iterations, thereby combining rapid global flow-field prediction with high-accuracy numerical convergence at the terminal stage. On an OOD benchmark comprising geometries sampled from actual transonic airfoil optimization trajectories, the framework lowers the median residual L_2 ratio by over seven orders of magnitude while substantially reducing field and aerodynamic errors. In practical supercritical airfoil optimization, it improves online prediction reliability while achieving a 15.5-fold generation-level speedup over CFD. We further test the framework's extension to three dimensions using a flying-wing dataset. Together, these studies demonstrate the potential of surrogate-Newton coupling to deliver accurate, efficient and scalable steady CFD across industrial workflows.
Ming Lei, Weishao Tang, Yufei Zhang et al.· 0 citations
Modern CFD increasingly places simulation inside workflows for design, inference, optimization, and data-driven modeling, creating pressure to connect physical models, numerical kernels, heterogeneous hardware, differentiation, and learning. Julia offers a distinctive approach: high-level scientific abstractions can be specialized for performance and composed within a common language and compiler ecosystem. This critical survey examines where that model benefits CFD software and where its limits remain. We review representative open-source projects and synthesize application-level evidence on performance, scalability, accelerator portability, automatic differentiation, and software composition. Published results demonstrate credible Julia-native CFD on large distributed CPU systems and multi-GPU platforms, as well as emerging differentiable workflows. Comparisons with C++ performance-portability frameworks, finite-element domain-specific languages, and JAX-based differentiable CFD show that these capabilities are not unique to Julia. Julia's distinction is their integration through shared types, dispatch, and specialization. The evidence is mixed: Julia has progressed beyond proof of concept in several CFD regimes, but still lacks the ecosystem breadth, industrial tooling, and deployment experience of established C/C++/Fortran environments. Its strongest current role is as a platform for developing and testing CFD architectures that connect simulation with downstream analysis.
The integration of physics-based modeling and data-driven prediction is creating new opportunities for predictive design, optimization, and the deployment of digital twins in advanced manufacturing systems. In compliant mechanisms, particularly double-bridge configurations used in precision positioning and surface engineering applications, accurate prediction and optimization of amplification ratio remain challenging due to coupled geometric interactions and nonlinear design trade-offs. This study presents a Physics-Guided Digital-Twin-Ready Framework for the predictive design and multi-objective optimization of double-bridge compliant mechanisms. A physics-consistent dataset comprising 8,000 design samples was generated using Latin Hypercube Sampling, analytical compliance modeling, constraint-based filtering, and response-space stratified sampling. The resulting dataset provides balanced coverage of amplification ratios within the range of 5–50, enabling robust learning across diverse design regimes. Machine-learning models, including Random Forest and Extreme Gradient Boosting (XGBoost), were developed to predict amplification ratio from geometric and material parameters. The models achieved excellent predictive performance, with coefficients of determination (R
2
) exceeding 0.99, mean absolute errors below 0.93, and root mean square errors below 0.65. Uncertainty quantification was incorporated through ensemble variance estimation, yielding prediction intervals with less than 5% relative uncertainty in well-sampled regions. SHAP-based explainability and sensitivity analyses revealed that amplification behavior is primarily governed by geometric parameters, particularly beam lengths and flexure thickness, whereas material stiffness has comparatively lower influence. NSGA-II-based multi-objective optimization identified Pareto-optimal solutions that balance amplification ratio and equivalent stiffness, highlighting the inherent trade-off between displacement amplification and structural rigidity. The developed surrogate models enable rapid design exploration, uncertainty assessment, and optimization, while achieving computational speed-ups of approximately 10
3
–10
7
times compared with finite-element-based evaluation workflows, depending on the evaluation method. The primary contribution of this work is the integration of analytical compliance modeling, physics-consistent dataset generation, uncertainty-aware machine learning, explainable artificial intelligence, and multi-objective optimization within a unified predictive framework. The proposed methodology should be interpreted as a digital-twin-ready surrogate architecture rather than a fully implemented digital twin, as real-time sensing, and online model updating are beyond the scope of the present study. Nevertheless, the framework provides a scalable foundation for future integration with experimental measurements, multi-fidelity datasets, and digital-twin-enabled manufacturing environments.
V. Kolate, P. D. Darade, Suhas P. Deshmukh· Frontiers of Mechanical Engi...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
