Modeling the Selective and Lossy Nature of Visual Processing Bridges the Discrepancy in Noninvasive Neural Decoding.

Abstract

Noninvasive neural decoding seeks to infer visual stimuli from brain recordings, such as EEG and MEG, thereby providing a computational approach for probing human visual information processing. Current contrastive learning approaches align brain signals with image embeddings from pretrained neural networks. However, these methods often struggle with the intrinsic visual-brain discrepancy arising from systematic representational gaps and noise because brain signals are treated as direct, lossless replicas of visual inputs. To alleviate this mismatch, the proposed framework accounts for the selective and information-compressive characteristics of biological vision. On the visual side, the framework incorporates an adaptive, context-aware visual calibration mechanism, where multilevel blurred representations are dynamically fused under the guidance of structural salience to refine visual features toward the brain's abstraction level. On the brain-signal side, a robust and lightweight neural encoder is used to mitigate inter-channel and temporal variations. Experiments on standard EEG and MEG benchmarks show that the proposed method achieves strong brain-to-image retrieval performance. Our modeling hypothesis, supported by retrieval behavior, suggests that computational representations aligning with salient and structurally meaningful content better match macroscopic neural responses than fine-grained image details. These findings suggest that incorporating the lossy and selective characteristics of human visual processing into computational models is important for advancing noninvasive neural decoding and understanding brain-machine alignment. The source codes are available at SLVP.