A deep learning-based grasp estimation model designed to enable robotic manipulation with articulated objects that incorporates the attention-based semantic and geometric feature fusion (ASGF) module improved the grasp success rate in the evaluated setting.
Abstract
Unlike rigid object grasping, manipulating articulated objects requires grasp poses that ensure both stability and actionability, that is, the ability to interact with an object’s joints. Therefore, we propose a deep learning-based grasp estimation model designed to enable robotic manipulation with articulated objects. The model first utilizes RGB-D images to predict graspable regions in the form of semantic heatmaps, identifying where to grasp. These regions are then combined with geometric features derived from point clouds through an attention-based semantic and geometric feature fusion module, ultimately to regress a manipulable 6-DoF grasp pose. To train and evaluate the proposed model, we constructed a custom dataset composed of RGB-D images and 6-DoF grasp labels collected from real-world articulated objects such as doors. Real-world robotic experiments demonstrated the effectiveness of the proposed model in producing stable and actionable grasps for articulated-object manipulation. Furthermore, ablation studies showed that incorporating the attention-based semantic and geometric feature fusion (ASGF) module improved the grasp success rate in the evaluated setting.
A reinforcement learning-based framework for robotic grasp refinement, integrating keypoint-based object representations with a Deep Q-Network (DQN), is proposed, offering a scalable and adaptable solution for contact-rich manipulation tasks.
Amir Arsalan Nematollahi, Shayan Ahmadi, M. T. Masouleh et al.· 0 citations
Grasping in unstructured environments requires handling objects with widely different mechanical properties, from soft and deformable items to rigid everyday objects. Most existing approaches address these categories separately and often rely on tactile sensing, object-specific models, or specialized grippers. In this paper, we present a unified monocular vision-based grasping framework that targets both soft and rigid objects within a single control pipeline, using only RGB input and a position-controlled gripper. The proposed system combines open-vocabulary object detection, image segmentation, boundary-aware point assignment, real-time point tracking, and monocular depth estimation to recover object motion and geometry from visual observations. A key component of the framework is a language-based stiffness estimation model that infers an object's expected compliance from its semantic description and provides an object-level prior for selecting the grasping strategy before contact. For deformable objects, grasp adaptation is governed by a Procrustes-based dissimilarity measure computed from tracked keypoints, which acts as a visual proxy for deformation. For rigid objects, the gripper width is regulated through the scaling of tracked point distances. We validate the proposed method in real-world pick-and-place experiments on a Franka Emika Research 3 arm using objects with substantially different mechanical properties, including lettuce, fresh mozzarella cheese, croissants, paper towels, and hard plastic bottles. Results demonstrate that the framework achieves stable grasping across both soft and rigid objects using visual feedback alone, highlighting a practical, sensor-efficient, and generalizable approach for food handling and household manipulation.
Robotic grasping of small, low-feature objects requires highly precise grasp pose estimation to ensure reliable in-gripper alignment. Vision-based approaches (e.g., YOLO-derived detectors) can localize the object but often fail to recover the exact grasp position under occlusion or partial views, while tactile-only methods lack global context; moreover, existing work rarely evaluates, in a task-specific and systematic manner, which fusion configurations are most suitable for this setting. Motivated by this gap, this paper investigates in-hand pose estimation of a USB stick that is already held within a robotic gripper; initial grasp detection is outside the scope of this work. A deep-learning regression model based on YOLOv11 as a feature extractor was developed to estimate the grasp position using inputs from an RGB eye-in-hand camera and a tactile sensor. Within the defined experimental setup, the early RGB–tactile fusion model selected via NAS achieved the lowest summed lateral error of 1.02 mm, compared to tactile-only (1.35 mm) and RGB-only (1.31 mm) models trained under identical protocols. These results reflect performance under the dataset’s image-level partitioning and indicate improved lateral localization within the evaluated setup. The study does not claim a universally optimal fusion strategy, but provides a controlled task-specific analysis showing that structured model selection can improve multimodal grasp-pose regression within the evaluated setup.
Adam Michael Altenbuchner, Bsher Karbouj, Fabian Dilly et al.· IEEE Access· 0 citations
Neural-network-based grasp detection has achieved remarkable success in robotic manipulation due to its efficiency and generalization ability. However, detected poses are often not optimized, leading to undesired object motion or collisions during physical execution. This paper proposes a motion-aware refinement framework that minimizes estimated object motion while enforcing collision avoidance. The seven-dimensional pose is decomposed into approach direction, engagement depth, planar projection, and gripper opening width, enabling efficient and interpretable optimization in lower-dimensional subspaces. To evaluate grasp stability beyond conventional success metrics, we introduce the observed success rate (OSR) together with quantitative motion measurements including translation, rotation, and tilt. Real-robot experiments show that, for high-profile objects, the full pipeline improves the measured success rate (MSR) from 93.33% to 100% and OSR from 83.33% to 97.78%. It also reduces the mean translation from <inline-formula> <tex-math notation="LaTeX">$6.099{\,}mm$ </tex-math></inline-formula> to <inline-formula> <tex-math notation="LaTeX">$2.684{\,}mm$ </tex-math></inline-formula>, rotation from 3.732° to 1.344°, and tilt from <inline-formula> <tex-math notation="LaTeX">$4.417{\,}mm$ </tex-math></inline-formula> to <inline-formula> <tex-math notation="LaTeX">$1.313{\,}mm$ </tex-math></inline-formula>, while requiring <inline-formula> <tex-math notation="LaTeX">$0.82\pm 0.42{\,}s$ </tex-math></inline-formula> on average. For low-profile objects that cannot be detected by the baseline point-cloud-based planner, the full pipeline achieves 100% MSR and OSR.
Tian Tan, Redwan Alqasemi, R. Dubey· IEEE Access· 0 citations
Dexterous grasp generation across robot hands is challenging because hands differ in kinematic topology, actuation dimensions, and native command spaces. We introduce GraspGraphNet, a topology-aware grasp generation framework that represents each hand as a URDF-derived kinematic graph and directly generates executable palm poses and joint configurations. GraspGraphNet combines hierarchical object surface encoding, differentiable forward kinematics, and dynamic world-edge message passing to model evolving robot-object interactions. It applies conditional flow matching directly in executable palm-pose and joint-state space, avoiding post-processing optimization, inverse kinematics, and retargeting. Using a shared model trained on Barrett Hand, Allegro Hand, and Shadow Hand, GraspGraphNet achieves an average success rate of 83.48% with 40ms inference time per grasp on a 40-object benchmark. Without retraining, the same model achieves 72.70% success on controlled finger-removal variants, demonstrating robustness to hand-topology variations. These results suggest that graph-structured hand representations can effectively support dexterous grasp generation across robot hands with different kinematic structures. Project: https://lysees.github.io/graspgraphnet-page
Y. Lee, Taeyeop Lee, Hyosup Shin et al.· 0 citations
An iterative closed-loop optimization framework that deeply couples SAM with FoundationPose and designs a multi-dimensional confidence assessment module that integrates both the 2D image domain and the 3D geometric domain to comprehensively evaluate the reliability of the current pose.
Zhiqiang Gao, Mengqi Li, Huihui Bai et al.· Italian National Conference...· 0 citations