A Bio-Inspired Tactile System for Multi-Fingered Robot Hand Toward Object Recognition

Abstract

Dexterous hands equipped with tactile sensors not only enable flexible object manipulation but also significantly enhance object identification capabilities, particularly in vision-failure scenarios. However, the arrangement of tactile sensors within the dexterous hand leads to challenges such as difficult signal interpretation and significant signal variability, which ultimately hinder object identification accuracy. Therefore, in this work, we propose a bio-inspired tactile recognition system. This system consists of two main components: a distributed Event-driven Tactile Sensor (ET-Skin) and a Spatial-temporal coupling Tactile-Transformer network(ST-Tacformer). The ET-Skin is designed by neuronal triggering mechanisms, featuring flexibility, low-cost and rapid response. It works as pressure receptor with the ability to measure contact pressure values, which is integrated onto a multi-fingered robot hand to perceive object information by grasping operation. In order to organize tactile data into a coherent structure for feature extraction and object recognition, the ST-Tacformer is proposed. It utilizes topology graph to consolidate tactile inputs and learns a robust tactile feature representation with attention mechanisms. The ET-Skin exhibits high positional and force sensing accuracy. The performance and applicability of ST-Tacformer are evaluated on two public datasets and two self-collected datasets. Comparative studies show that our model significantly improves recognition accuracy, achieving improvements of 4.5% and 12.6% on the public datasets and accuracies of 95.67% and 93.06% on our self-collected datasets. Note to Practitioners—Robotic tactile perception is an important research area that can enhance the flexibility, accuracy and stability of robots in tactile manipulation tasks. However, the accuracy of tactile recognition relies on the performance of tactile sensors and the quality of the features extracted, which are limited by conventional deep learning methods. To address this, we propose a bio-inspired tactile perception system. The low-cost, scalable and event-driven characteristics of the proposed sensors can support various grasping tasks across different dexterous hands. We introduce the ST-Tacformer model, which extracts spatio-temporal features from different feature scales. This network resolves the issue of information loss caused by traditional deep learning methods when extracting single features. Experiments on two public datasets and two datasets collected by our proposed sensors show that our method outperforms competing algorithms in object recognition tasks in robotic multi-tactile sensor scenarios. In future research, we will focus on the characteristics of bio-inspired sensors and the stable grasping perception problem of dexterous hands.