Self-driving laboratories (SDLs) are laboratory systems that couple experimental planning, automated execution, and measurement feedback in a closed loop. Integrating flexible robot control with the typed application programming interfaces (APIs) and continuous measurements exposed by laboratory instruments remains an execution-interface challenge. To address this challenge, we propose a traceable framework comprising four components: LAM (Lab Action Model), OpenSDL, Safe-SDL, and L3-Bench. LAM is a proposed measurement-conditioned policy architecture that extends vision-language-action paradigms to generate both continuous robot trajectories and typed instrument API calls. OpenSDL is an execution substrate with an initial control implementation, designed to map role-level operations to heterogeneous devices, coordinate execution, and record timestamped causal events. The proposed Safe-SDL runtime gate combines pre-dispatch schema and resource checks with signal temporal logic monitoring during execution, including extended waiting periods. L3-Bench (Level-3 Laboratory Benchmark) specifies an episode-level evaluation protocol covering physical outcomes, timing, reproducibility, safety violations, and fault recovery. Together, these components provide an end-to-end design for linking hybrid action generation, monitored device execution, measurement feedback, and experimental evaluation, addressing coordination and traceability challenges in SDL workflows.
This paper investigates developing real-world perception for active agents, proposes Gibson Environment for this purpose, and showcases a set of perceptual tasks learned therein.
F. Xia, Amir Zamir, Zhi-Yang He et al.· 2018 IEEE/CVF Conference on...· 1,069 citations· ⚡112
RMBench is introduced, a simulation benchmark comprising 9 manipulation tasks that span multiple levels of memory complexity, enabling systematic evaluation of policy memory capabilities and Mem-0, a modular manipulation policy with explicit memory components designed to support controlled ablation studies.
Tian-Xing Chen, Yu-Ran Wang, Ming-Lei Li et al.· arXiv.org· 62 citations· ⚡23
This work introduces StarVLA-$\alpha, a simple yet strong baseline designed to study VLA design choices under controlled conditions that deliberately minimizes architectural and pipeline complexity to reduce experimental confounders and enable systematic analysis.
Jinhui Ye, Ning Gao, Sen-Qiao Yang et al.· arXiv.org· 49 citations· ⚡11
This work characterize models from this family of methods as JEPA-WMs and investigates the technical choices that make algorithms from this class work, and proposes a model that outperforms two established baselines, DINO-WM and V-JEPA-2-AC, in both navigation and manipulation tasks.
Basile Terver, Tsung-Yen Yang, Jean Ponce et al.· Trans. Mach. Learn. Res.· 49 citations· ⚡7
A novel vision-language model (VLM)-based navigation framework that addresses challenges by adaptively retrieving task-relevant cues from a global memory module and integrating them with the agent's egocentric observations and enhances spatial reasoning and decision-making in long-horizon tasks.
The Very Big Video Reasoning (VBVR) Dataset is introduced, an unprecedentedly large-scale resource spanning 200 curated reasoning tasks following a principled taxonomy and over one million video clips, approximately three orders of magnitude larger than existing datasets.
Maijunxian Wang, Rui-Si Wang, Ju-Yi Lin et al.· arXiv.org· 28 citations· ⚡5
Robots are getting smarter, but how can their hardware match that growth? New Microsoft Research findings show that moving AI inference beyond the robot can improve task success, boost efficiency, and support more advanced physical AI workloads. The post Offloaded inference for real-world physical AI robotics appeared first on Microsoft Research.