Machine vision-based grasp detection and grasping system for complex environments

Abstract

This study proposes an integrated machine-vision framework for robotic grasp detection and execution in complex environments. CenterNet is used to locate target objects, while a coordinated grasp-frame branch estimates feasible 2D grasp poses. Camera calibration and hand-eye calibration convert image-space predictions into the robot-base coordinate system, enabling consistent motion commands. The experimental platform combines a Kinova J2S6S300 manipulator, an AstraPro RGB-D camera, ROS, MoveIt, and Gazebo. Its software architecture links perception, coordinate transformation, trajectory planning, collision avoidance, simulation, and physical grasping within a unified workflow. The method is designed to improve target localization, grasp-pose consistency, and operational stability under background clutter, illumination variation, partial occlusion, and differences in object material. Simulation and physical tests are used to evaluate detection accuracy, grasp success, landing-position consistency, and failure modes. The resulting system provides a reproducible technical route for connecting object detection, grasp generation, calibration, and manipulator control, and offers practical value for automated production, logistics, warehousing, and service-robot applications.