Jul 2026· SIGGRAPH Posters· pp. 1-3· 0 citations· 4 references
Computer Science
TL;DR
This study introduces the S2VMO (Sensor-to-Voxel Multi-Agentic Observatory), a framework that reconstructs structural state through finite element modeling (FEM) from four strain sensors by a two-stage neural surrogate that is stored as a graph of spatially indexed voxels.
Abstract
Digital twins for space structures can estimate the physical states of systems with real-time data, yet long-term understanding and consistent analysis remain difficult. Current digital twin systems primarily aggregate data for monitoring and simulation, but they require experts to extract further insight, and most focus on state reconstruction and visualization, offering limited support for interpretation and decision-making. This study introduces the S2VMO (Sensor-to-Voxel Multi-Agentic Observatory), a framework that reconstructs structural state through finite element modeling (FEM) from four strain sensors by a two-stage neural surrogate that is stored as a graph of spatially indexed voxels. The shared voxel world model enables agents to identify anomalies, explain structural behavior, and provide inspection-oriented recommendations through natural-language interaction. This extends digital twins beyond passive monitoring toward explainable structural assessment and decision support for future space habitat operations.
HounsBench is introduced, a computed tomography (CT) centric patient-state benchmark that unifies these three task families with patient-disjoint splits and per-family metrics, and HounsWorld, a 3B multimodal world model that treats volumetric scans and language as observations of the shared state through Joint Understanding-Generation Learning.
Yunhao Bai, Zhongwei Qiu, Guangyu Guo et al.· 0 citations
State-Space Models (SSMs), particularly Mamba, offer linear-time complexity for long-range dependencies, making them attractive for medical imaging with limited annotated data. However, adapting these sequential models to 2D images through unconstrained state evolution causes representational drift, the dynamic hidden state progressively loses fidelity to local image features. We introduce Energy-Mamba, integrating SSM dynamics with physics-informed constraints via a learnable potential energy function that quantifies compatibility between evolving states and static local features. Our Energy-Mamba Block introduces a gradient-based forcing term, computed dynamically via automatic differentiation, that pulls states toward low-energy configurations maintaining local visual fidelity. This formulation mirrors Hamiltonian dynamics: kinetic energy (SSM scan) plus potential energy (our constraint function) govern state trajectories. This architectural prior enables learning implicit constraints for robust, faithful representations, crucial in medical imaging where fine-grained local detail drives accurate diagnosis. Evaluated on four datasets (retinal OCT, chest X-ray, microscopy, abdominal CT), Energy-Mamba achieves state-of-the-art classification performance with significantly fewer parameters, demonstrating that physics-informed grounding can enhance both efficiency and representational quality in medical vision tasks.
Long-term bridge monitoring systems inevitably contain missing or unusable data segments after sensor faults, equipment failures, transmission interruptions, and other non-structural data-quality problems are identified and removed. Traditional reconstruction methods struggle to preserve the non-stationary, multi-scale, and sensor-dependent characteristics of bridge responses. This article proposes a mechanics-prior latent diffusion model for high-fidelity bridge monitoring data reconstruction. The method converts one-dimensional monitoring sequences into image representations, maps them into a regularized latent space through a pre-trained variational autoencoder, and trains a conditional diffusion U-Net to recover missing regions through reverse denoising. Mechanics-prior condition encoding is introduced as a global prior constraint, incorporating frequency-domain energy distribution and quasi-static morphological features tailored to different sensor types. Validation using numerical simulations and real bridge data, including acceleration, Global Positioning System (GPS) displacement, and strain, shows robust reconstruction under both long continuous gaps and fragmented missing scenarios. In representative 40% long-gap cases, the proposed method achieves NRMSE values of 0.0921, 0.0620, and 0.0523 for acceleration, GPS displacement, and strain, respectively, outperforming LSTM, GAN, and cubic spline interpolation baselines.
Inferring physical properties such as mass, stiffness, and elasticity from a single image is essential for simulation and embodied AI, yet most existing approaches rely on multi-view reconstruction or physics-based supervision. We introduce SiPhy, a unified framework for single-image physical property reasoning that aligns 3D-aware visual cues, depth with language-based material knowledge. From one RGB image, SiPhy samples pseudo-voxel points, extracts CLIP features, and grounds them to material candidates proposed by a VLM. A part-based contrastive aggregator enforces region consistency, while a heaviness-aware refinement improves thickness and volume estimation for dense objects. Across ABO-500, MVImgNet-100, and PhysXNet-100, SiPhy achieves state-of-the-art single-image performance, surpassing multi-view reconstruction methods by improving mass MnRE by up to 93% (vs. PUGS), reducing density MAE by 35.5% (vs. NeRF2Physics), and lowering Young's modulus error by 23.5%. We further validate SiPhy on real hand-object interaction datasets, demonstrating its potential as a data annotation engine for physical understanding from single-view imagery.
H. Lê, Joon-Byum Kwon, Elkhan Ismayilzada et al.· arXiv.org· 0 citations
Mechanistic interpretability seeks quantities that models do not expose directly: represented states, component effects, interactions, and responses to interventions. Patching, gradients, Hessian-vector products, and subset interventions provide different measurements under different access assumptions and may target different quantities. We formulate their shared measurement structure as mechanistic tomography: designed measurement for recovering internal mechanisms and intervention effects. For a chosen basis and intervention family, measurements take the form y = Ax + w, where A describes the interventions, x is the target map, and w contains nonlinear response, sampling error, and basis misspecification. This language gives a practical procedure: start with the least costly measurements, test on held-out interventions at the intended scale, calibrate simple mismatch, and expand the measurement family when structured residuals remain. Control provides a demanding validation setting because an estimate that guides an intervention acts as an observer. In a two-HMM model, control error rises with observer error, while target improvement can hide nuisance-state movement. Under forward-only access, sparse aggregate measurements recover a finite-effect map with fewer interventions than coordinate patching. With gradient access, finite probes improve a local attribution map. Lifted measurements and Hessian-vector products recover interactions missed by first-order maps, while Tracr shows that the required family depends on the basis. On GPT-2-small IOI, the Name Mover-Negative Name Mover interaction is the largest held-out predictive term among three tested cross-group pairs. On Qwen-2.5-7B, finite calibration makes an additive refusal-response map adequate, so held-out error does not support pairwise lifting.
Flow-based generative models have emerged as powerful image priors for training-free inverse problem solving, capturing coherent semantics and fine-grained structure. Despite these strengths, existing flow-based inverse solvers primarily focus on the design of individual updates, largely overlooking spatio-temporal information allocation under a fixed number of function evaluations (NFEs). Temporally, insufficient early exploration can trap the flow trajectory in an incorrect semantic basin, whereas excessive allocation of NFEs to early stages leaves little budget for late-stage refinement. Spatially, data consistency provides direct constraints only within observed regions, whereas the recovery of missing regions relies mainly on the generative prior. To address these two issues, we introduce two complementary and training-free components, i.e., Spectrum-Adaptive Scheduling (SAS) and Measurement-Prioritized Attention (MPA). For temporal allocation, SAS distributes the available NFEs over flow time according to the degradation spectrum and logSNR geometry, thus better balancing semantic exploration and detail refinement. For spatial propagation, MPA exploits data-prior conflicts to guide information toward weakly constrained regions, thereby enhancing semantic and structural fidelity. Extensive experiments on standard image inverse problems, e.g., super-resolution, motion deblurring, and inpainting, demonstrate that the proposed components can be integrated into existing flow-based inverse solvers in a plug-and-play manner without retraining or additional flow-model evaluations, and can also significantly improve the restoration quality of existing solvers.
Yijie Cao, Xiangyong Cao, Peibing Liu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
