Hierarchical gist representation of events in human brain during naturalistic stimuli.

Abstract

The neural mechanisms underlying the brain's representation of event gist remain poorly understood. In this study, we investigate the neural architecture and computational principles governing the hierarchical representation of event gist in the human brain. Using an integrative approach that combines natural language processing with fMRI data from narrative listening, we extracted semantic event gist vectors from texts and aligned them with neural responses through representational similarity analysis. This analysis revealed that the brain's representation of event gist follows a functional hierarchy. Specifically, brain regions can be categorized into four functional levels based on their temporal preference for narrative information processing: from the Fine-grained level representing sentence-level event gist, to the Medium-Fine and Medium-Coarse levels integrating cross-sentence event gist, ultimately reaching the Coarse level constructing long-range narrative gist. Using lagged inter-subject functional connectivity (lag-ISFC), we further characterized the temporal lag structure across this hierarchical organization, revealing a systematic offset pattern consistent with the temporal hierarchy. To explore the computational principles underlying these hierarchical patterns, we implemented long short-term memory (LSTM) and convolutional neural network (CNN) models to map fine-scale to coarse-scale regional dynamics. The results revealed that LSTM models outperformed CNNs in capturing the hierarchical processing of narrative events across gist-representing brain regions, with deeper LSTM hidden layers corresponding more closely with higher-order cortical activity. Perturbation of LSTM hidden states further revealed that lower-layer representations are computationally upstream of higher-layer ones, and that deep hidden states are specifically required for correspondence with Coarse-grained regions. These findings, obtained within a brain-to-brain modeling framework that characterizes the fine-to-coarse cortical transformation directly, identify sustained recurrent state integration as a candidate computational property of intra-cortical hierarchical transformation during naturalistic narrative processing.