Results demonstrate that a fully onboard MAS architecture is a viable, cost-efficient alternative to cloud-dependent deployments, with strong potential for real-world transfer.
Abstract
Vision Language Models (VLMs) and Vision Language Action (VLA) models have shown promise in robotic control. Yet, they face significant challenges regarding explainability, generalization, and compute requirements. This paper presents a Multi-Agent System (MAS) architecture that addresses these limitations by deploying specialized agents on onboard hardware - eliminating dependence on external compute. The system controls a multi-purpose autonomous mobile manipulator in a simulated industrial warehouse, fulfilling five task categories: safety inspection, warehouse maintenance, warehouse search, package quality verification, and responding to human requests. Compact VLMs (3-20B parameters) are used throughout, with fine-tuning applied to improve package inspection accuracy. A novel"Megamind"orchestration agent mitigates context retention issues inherent to long-horizon planning with smaller models. The system was validated in a hardware-in-the-loop simulation using an AMD Ryzen(TM) AI mini PC. Results demonstrate that a fully onboard MAS architecture is a viable, cost-efficient alternative to cloud-dependent deployments, with strong potential for real-world transfer. The simulation environment has been released as open source under the Apache 2.0 licence.
Experimental results in various task scenarios show that the proposed framework consistently improves overall task success rates compared with unimodal settings with different LLMs and achieves a higher success rate compared to using only visual or force data.
Robot manipulation is a complex task that requires visual understanding, physical reasoning, planning, and closed-loop control. General-purpose foundation models (FMs) have grown remarkably capable of some of these, especially vision and reasoning. To leverage this for generalist robot policies, current methods typically involve converting existing FMs into vision-language-action (VLA) models by fine-tuning on robot data to output low-level actions. However, VLAs are often orders of magnitude smaller than frontier FMs given the limited data and compute available for fine-tuning, which in turn limits their general capability. Inspired by the growing ability of FMs to operate software through visual interfaces, we ask whether that same competence suffices to control a robot. We present VIA (Visual Interface Agent for robot control), a framework that recasts robot control as an agentic task: an off-the-shelf FM-powered agent drives a manipulator through a browser-based 3D interface by taking screenshots, issuing intuitive commands, observing the outcome, and adjusting. The agent receives no robot-specific fine-tuning and no access to privileged state information: it perceives visual input and acts through a small set of general tools. VIA inherits the agent's general reasoning, closed-loop error recovery, and ability to plan and re-plan from what it observes. It solves a diverse suite of tabletop manipulation tasks zero-shot with both Claude Code and Codex. With the strongest model (Fable 5) it achieves 96.7% success on three LIBERO-Goal tasks and 100% on a long-horizon rainbow assembly task. Performance improves with the scale and strength of the underlying model. These results suggest that frontier agents already possess skills that transfer directly to robot control given the right interface: your coding or computer-use agent is, in a sense, secretly a robot-control agent.
Deployable autonomy remains a key challenge for unmanned aerial vehicles (UAVs) operating in open-ended missions. Large language models (LLMs) and their multimodal variants, which can process visual and other sensory inputs, have introduced new capabilities for semantic perception, task reasoning, and language-conditioned control. However, these capabilities do not by themselves produce flight-ready autonomy. We structure our analysis around a Perception–Cognition–Action (P–C–A) framework. At each layer, we identify the capabilities contributed by LLM-based components and examine how they connect to existing flight modules through input specifications, output representations, architectural coupling patterns, and safety mechanisms. Across the surveyed systems, LLMs extend UAV autonomy beyond fixed perception categories, scripted task plans, and pre-programmed controllers. However, field deployment depends on whether model outputs can be transformed into representations that downstream modules can parse, verify, and safely execute. Without adequate validation, captions, task plans, code, waypoints, and control commands may become failure points that propagate across the P–C–A loop. Our analysis highlights structured output contracts, independent safety barriers, and deterministic fallback mechanisms as key design elements for the reliable integration of LLM capabilities into UAV platforms.
Ting-Quan Xiong, Jianning Zhan, Qiu-Wei Deng et al.· Drones· 0 citations
Capek 0.5 is presented, an embodied vision-language model built around an execution-centric capability taxonomy that improves the large majority of matched benchmark rows over its initialization, retains all four specialized capabilities in one checkpoint with quantified losses, and transfers to closed-loop embodied task execution.
RoboBRIDGE is presented, a modular framework that provides an orchestration layer over five coordinated modules, namely Monitor, Perceptor, Planner, Controller, and Robot Interface, to compose robust robotic agents from off-the-shelf components, including pretrained VLAs.
The strong coupling among guidance laws, control loops, aerodynamics, and mission constraints poses growing challenges to the design of modern tactical missile guidance systems for autonomous flight. Conventional manual tuning and simulation-based trial-and-error result in long iteration cycles, limited reuse of design knowledge, and poor adaptability to changing scenarios. To address these limitations, we propose GSMultiAgent, a multi-agent collaborative cascade framework built atop Hermes Agent, which transforms natural-language mission requirements into optimized guidance system models through structured agent cooperation with feedback-driven iterative refinement. Three innovations are introduced: (1) a three-layer correction pipeline covering syntactic checking, deterministic mathematical verification, and semantic reasoning; (2) a bimodal experience repository supporting similarity-guided retrieval with access-count decay and best-quality retrieval for PPO warm-start initialization; and (3) a self-adjudicating optimizer that autonomously decides between PPO-based systematic parameter search and heuristic LLM-tuning guided by a reflection agent. Across four engagement scenarios, GSMultiAgent consistently attains high feasibility at a small fraction of the simulation budget required by conventional optimizers and single-agent baselines, and its design paths escalate autonomously from parameter tuning to structural law modification as task difficulty increases. Ablation studies confirm that the reflection agent, the optimization agent, and structured memory each contribute essential and complementary gains. These results establish multi-agent coordination with structured memory and self-adjudicating optimization as an effective paradigm for intelligent, reusable guidance system design.
