Empowering embodied intelligence via a flexible sensing system integrating multi-dimensional tactile information

Abstract

Enabling robots to assess physical suitability through contact, termed here Embodied Understanding, remains an open challenge because current vision-based models lack the tactile feedback necessary to perceive real physical properties. However, mainstream Visual-Language Models suffer from disembodiment. Without tactile feedback, they cannot perceive real physical properties. Simultaneously, traditional supervised learning paradigms face the dilemma of Label Ambiguity, where evaluating physical suitability highly depends on dynamic task contexts, making it difficult to define a unified ground truth for training. To address these challenges, we propose a soft-hard integrated multi-dimensional tactile cognitive system. At the hardware level, we developed a flexible sensing array based on 3D-printed micro-protrusions, achieving efficient decoupling of three-dimensional contact force fields through a mechanical routing mechanism. At the algorithmic level, we adopt In-Context Engineering as the core paradigm, designing calibration-free Physical Proxy Indices (PPI) combined with a Physics-Aware Chain-of-Thought (Phy-CoT) strategy. Without finetuning, this approach uses structured prompts to guide LLMs toward physics-aware inference from tactile signals, without requiring fine-tuning. Experimental results in tool selection tasks demonstrate that the system successfully corrects visual bias caused by disembodiment in mainstream models such as GPT-4o and Gemini-Pro, achieving correct physical decisions in deceptive test cases. Furthermore, experiments confirm Phy-CoT as an enhancement approach that smooths commonsense biases and promotes decision convergence. This study demonstrates that In-Context Learning can help resolve physical semantic ambiguity in tool selection tasks, offering a promising direction for multimodal embodied agents.