Skip to content

A multimodal foundation model-enabled agent for human–robot collaboration in construction

Aug 2026 · Construction Robotics · Vol 10 · 0 citations · 38 references

TL;DR

This paper presents a multimodal foundation model-enabled agent that integrates BIM, simulation, and physical execution by enabling a robot to retrieve BIM data, translate it into assembly targets, and generate robot motion trajectories through natural-language interaction.

View source

Similar papers

Conference Aug 2026

Speech-to-Trajectory: An LLM-Enabled Human-Robot Collaborative Framework for Generating New Assembly Tasks From Verbal Guidance

Human-Robot Collaboration has enhanced manufacturing efficiency and productivity by integrating the precision and repeatability of robots with the adaptability of human operators. The growing demand for customized products frequently requires human operators to manually reprogram robot task plans. This process demands...

Jia-Bao Zhao, Hong-Liang Li, Jonghan Lim et al. · 0 citations
Review Sep 2026

Skill Sequence Planning for Collaborative Multi-Robot Construction

Robots have significant potential to automate construction processes. However, their industry adoption remains limited, partly because of the programming effort required to adapt robots to diverse tasks. This paper presents a skill sequence planning method that enables a heterogeneous team of multi-functional robots to...

Xi Wang, Bo Fu, Carol C. Menassa et al. · 0 citations
#machine learning Preprint Oct 2026

RobotWorld: Benchmarking Multimodal Agents for Robot Use Across Diverse Tasks and Embodiments

General-purpose agents increasingly write code, use tools, and complete complex digital tasks, raising the question of how far these capabilities carry into the physical world. To investigate this, we introduce RobotWorld, a challenging simulation testbed for robot use: turning instructions and observations into physic...

Zhiqin Yang, Chen-Xin Li, Xiao-Meng Hu et al. · 2 citations
Open access Sep 2026

A System Architecture Framework for BIM-to-Robot Information Exchange in Construction

The adoption of robotics in construction has progressed more slowly than in many other sectors. Building information models (BIMs) contain information that can support construction robotics, while robots require task-specific project data, such as geometry, location, material, sequence, and operating constraints, that...

Austin D. McClymonds, Somayeh Asadi, Robert M. Leicht · 0 citations
#small language model Conference Open access Sep 2026

A Review of Multimodal Perception and Closed-Loop Task Execution for Human-Robot Collaboration Robots Oriented Toward Intelligent Assembly

This review discusses recent research from the perspective of a complete "perception-decision-execution-feedback" process, and focuses on the main topics operator action and intention recognition, perception of part states, task allocation under human-related constraints, safe trajectory planning, and quality feedback.

Zhuo-Yuan Li · 0 citations
Preprint Sep 2026

DexAgent: An Agentic Human2Sim2Robot Framework for Dexterous Manipulation with Self-Evolving Tool Library

Human videos offer a scalable source of demonstrations for dexterous robot manipulation. However, existing human-to-simulation-to-robot (Human2Sim2Robot) pipelines rely on predefined procedures that struggle to accommodate diverse object properties and interactions, particularly those involving articulated and deformab...

You-Hui Wang, Yun-Zhu Li, Fei-Fei Li et al. · 1 citation

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.