An autonomous grasping path planning algorithm for robotic arms that integrates open vocabulary visual perception

Abstract

In the context of the rapid development of intelligent manufacturing and flexible production, the stable grasping of multiple types and forms of targets in open scenarios poses higher requirements for the visual perception and path planning of robotic arms. This paper, in the context of open vocabulary conditions, constructs an algorithm framework for autonomous grasping path planning of robotic arms that integrates visual-language pre-training models and geometric information: The upstream uses an open vocabulary visual perception network to generate regional-level features and text semantic embeddings, and through a deep learning feasibility assessment model, integrates visual semantics and local point cloud geometric information to score and filter candidate grasping postures; On this basis, multiple cost functions including path length, smoothness, collision risk, and semantic consistency are designed, and combined with heuristic search and incremental path evaluation algorithms to complete the selection of grasping paths; At the same time, the distance field-based dangerous zone collision cost and visual feedback-driven local re-planning are introduced to reduce the online collision detection and search overhead. Experiments are conducted on a self-built open vocabulary grasping dataset, with a total of 10,500 RGB-D images and over 75,000 grasping annotations in the training/validation/test sets, and 20 new categories specified only through text descriptions are introduced in the test set; In three types of scenarios - single object, desktop multi-object, and box stacking - the grasping success rates of this method reach 94%, 88%, and 81% respectively, and the average planning time remains within the range of 105 to 140 ms. The study shows that the path planning algorithm integrating open vocabulary visual perception and multi-index cost evaluation can effectively improve the grasping success rate and real-time performance of robotic arms in complex open scenarios, providing a feasible technical path for the engineering implementation of general grasping systems in flexible manufacturing environments.