Skip to content

DuoMind: Enabling Distributed Multi-Robot Coordination with Semantic Communication

Oct 2026 · 1 citation · 50 references
Computer Science

TL;DR

This work introduces DuoMind, a distributed hierarchical framework for multi-robot coordination through semantic communication, and develops RoboPoly, a benchmark comprising long-horizon manipulation tasks that require coordinated, closed-loop execution under distributed control.

Abstract

Vision-language models (VLMs) and vision-language-action models (VLAs) have recently driven rapid progress in general-purpose robots, yet most progress has focused on single-robot settings. Extending these capabilities to multi-robot systems remains challenging because robots must coordinate long-horizon behaviors while maintaining reliable, fine-grained execution. We introduce DuoMind, a distributed hierarchical framework for multi-robot coordination through semantic communication. Each robot uses a VLA-based action model for low-level execution and a VLM-based orchestrator for high-level reasoning and inter-agent coordination. At each planning step, the orchestrator at each robot reasons over the task instruction, local observations, and messages received from other robots. It then generates low-level instructions for the action model and semantic messages for peer robots. This architecture exploits the complementary strengths of pretrained models by combining the semantic reasoning capabilities of VLMs with the precise action-generation capabilities of VLAs. To address the scarcity of benchmarks for multi-robot coordination, we further develop RoboPoly, a benchmark comprising long-horizon manipulation tasks that require coordinated, closed-loop execution under distributed control. Experiments on RoboPoly and RoboTwin demonstrate that DuoMind improves multi-robot task performance, while ablation studies confirm the contributions of hierarchical orchestration and semantic communication. More details are available on our project page.

View source

Similar papers

#robotics Sep 2026

RoboTalk: Learning Multi-Robot Communication and Coordination from Multimodal Demonstrations

RoboTalk is introduced, a synthetic data-generation pipeline and dataset of 7,950 multimodal trajectories spanning 53 mobile-manipulation kitchen tasks for training small VLMs to communicate and coordinate and fine-tuning open-source models on this dataset can reach 77% success on novel held-out tasks.

Dorian Benhamou Goldfajn, Mason Nakamura, Saaduddin Mahmud et al. · 0 citations
#machine learning Preprint Sep 2026

Efficient Vision-Language-Action Management and Serving for Robot Factories

Robion is designed, the first VLA serving and management system for multi-robot, multi-model requests on multi-GPU edge servers that meets SLOs and enables flexible model placements on multi-GPU servers, and integrates an intelligent traffic controller.

Dionysios Adamopoulos, Nattapol Chanpaisit, Basel Fakhri et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Systematic Multi-Agent Vision-and-Language Navigation: Formulation, Benchmark, and Method

Vision-and-Language Navigation (VLN) has largely focused on a single agent following a single instruction, yet many real-world applications require teams of robots to tackle tasks beyond the capabilities of any individual agent. We present Systematic Multi-Agent Vision-and-Language Navigation, providing, to our knowled...

Yun-Zhe Xu, Zhe Liu · 0 citations
Sep 2026

MotorMind: Scaffolding General Vision Language Models for Zero-Shot Robot Manipulation

MotorMind is introduced, a robot manipulation harness that connects VLM-proposed mid-level actions to deterministic robot control and feedback, with asynchronous monitoring and background memory updates, and shows that a general-purpose VLM, when equipped with an appropriate mid-level action representation and asynchro...

Bing-Xuan Li, Si-Qi Song, Yi-Zhuo Wu et al. · 1 citation
Preprint Oct 2026

From VLM Planners to Robotic Agents: Self-Evolving Harnesses for History-Dependent Manipulation

Vision-language-action (VLA) models achieve strong performance across embodied manipulation tasks by unifying visual understanding, language conditioning, and action generation. However, long-horizon tasks can involve non-Markovian dependencies: successful decisions may require retaining task progress and past interact...

Ting-Wen Du, Hao-Yu Guo, Han-Xue Zhang et al. · 1 citation

Cooperative Multi-Agent Vision-Language-Action Models via Reinforced Fine Tuning

A three-stage reinforced fine-tuning (RFT) pipeline for multi-agent VLAs is presented, which freezes the VLA and performs RL in its latent noise space and finds existing online RL for VLAs are less effective for hard multi-agent tasks, which are attributed to noisy co-exploration and unstable updates.

Rui-Xiao Xu, W. Kenny, Zhi-Qian Liu et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.