Based on Multi-modal Data Fusion for Subconscious Decision Simulation

Abstract

This paper presents a novel approach to simulating subconscious decision-making processes by leveraging multi-modal data fusion. The core idea is to construct a computational model capable of mirroring the complexities of human subconscious decision-making, moving beyond traditional behavioral analysis. We employ a graph neural network (GNN) architecture for robust multi-modal data integration, transforming diverse data streams – including visual, auditory, and tactile information – into a unified representation. This representation is then utilized within a reinforcement learning framework to simulate the subconscious decision-making process, explicitly modeling the interactive effects between different modalities. The resulting model provides a deeper understanding of how individuals make decisions without conscious awareness, offering potential applications in fields such as robotics, human-computer interaction, and cognitive modeling. The key innovation lies in the comprehensive incorporation of multi-modal interactions, providing a more accurate representation of the human subconscious than existing approaches. We define the following key equations to represent the core processes within the model: Let *xi* represent the input vector for modality *i*, where *i* ∈ {V, A, T}, representing Visual, Auditory, and Tactile modalities, respectively. The dimensionality of each *xi* is denoted as *di*. The multi-modal fusion process can be expressed as: * *xfused* = FusionNetwork(*xV*, *xA*, *xT*) Where *xfused* is the fused representation and FusionNetwork is the graph neural network. The reinforcement learning agent's decision-making process is governed by the following equation: * *ai* = argmaxj [Q( *xfused*, *aj* ) + β * R( *xfused*, *aj*)] Where *ai* is the action taken, *Q* is the Q-function estimating the expected reward, *R* is the reward function, and β is a weighting factor. The model's training objective can be formalized as: Minimize Eτ [ Σt=0T γt *R( *xfused*, *at* )] Where τ is a trajectory, *R* is the reward function, γ is the discount factor, and T is the time horizon.