A vision-guided robotic action generation framework for industrial assembly task

Abstract

Visual perception plays a critical role in industrial assembly systems, where robotic actions are driven by image-based sensing under variable and data-dependent conditions. A key challenge in such systems lies in transforming unstructured visual inference outputs, including object detection and pose estimation results, into structured and executable control parameters that can be reliably grounded in physical execution. In practice, mismatches between perception outputs and downstream control interfaces often lead to execution errors and reduced system robustness in multi-stage assembly processes. To address this challenge, this paper proposes a vision-guided robotic action generation framework that explicitly models the data flow from visual perception to robotic action execution. The framework introduces a structured visual data extraction mechanism that interprets raw visual outputs into type-consistent, constraint-aware, and physically feasible motion parameters, enabling reliable perception-action coupling in industrial assembly systems. By decoupling visual interpretation from low-level control execution, the proposed approach improves modularity and robustness across heterogeneous hardware platforms and low-code industrial orchestration environments. The proposed framework is implemented and evaluated through an end-to-end, data-dependent toy vehicle assembly task involving multiple perception-driven operations. Experimental results demonstrate that the proposed method significantly improves perception-action alignment robustness, achieving higher phase-level execution reliability and an end-to-end assembly success rate of up to 94%, outperforming baseline approaches that lack explicit visual data alignment mechanisms.