MoTIL: Mode-Triggered Thought-Conditioned Imitation Learning for Predictable Human-Robot Collaboration

Abstract

In physical-contact human-robot collaboration (HRC), human partners need to anticipate not only how a robot will move, but also which subtask the robot is currently executing. This is challenging when a high-level language mission must be translated into continuous robot motions, because end-to-end imitation policies do not explicitly expose how the mission is mapped to moment-to-moment behavior. This paper presents Mode-triggered Thought-conditioned Imitation Learning (MoTIL), which introduces step-wise thoughts as an intermediate representation between language-level task interpretation and real-time action generation. At the beginning of a task, a Vision-Language Model (VLM) generates a plan set from the scene and a natural-language mission, and each plan step is converted into a thought vector. During execution, a mode change model selects the current thought from the history of robot states, and an Action Generator produces continuous control actions conditioned on the selected thought. Additionally, the selected thought is verbalized at mode switches, allowing the robot's current intent to be externally shared with the human partner. In a real-robot collaborative platetransport task with right/left target-position variants, MoTIL achieved an 80% task success rate, compared with 30% for the LSTM baseline without thought conditioning. Further analysis supports the benefit of step-wise thought conditioning, and a user study suggests that verbalizing the selected thought improves perceived predictability and perceived safety.