Grasp Prediction for Pneumatic Soft Hands via Proprioceptive Sensing

Abstract

Although grasp prediction is well-established for rigid robotic hands, it remains particularly challenging for pneumatic soft hands due to their high compliance, nonlinear actuation, and limited sensing capabilities. To address these challenges, an integrated hardware-algorithm method is developed that leverages proprioceptive signals captured during a prelift interaction phase. A multichannel, IMU-based modular sensing system is embedded in each finger to capture joint orientation and acceleration in real time during grasp attempts. The Soft-hand Fusion Attention Network (SoftFA-Net), a lightweight multimodal model, is designed to fuse pressure, acceleration, and orientation signals, and to adaptively emphasize informative temporal segments, sensing modalities, and feature channels via a hierarchical attention mechanism. Experimental results show that the method achieves 92.9% accuracy (macro-F1 92.0%) on seen objects and yields 88.6% accuracy (macro-F1 89.0%) on unseen objects. It is further deployed in an online grasping task, enabling iterative refinement of the grasping strategy based on prediction feedback. This work provides a practical and extensible solution for pneumatic soft hand grasp prediction, laying the groundwork for future real-time multimodal sensing integration in complex manipulation tasks.