Skip to content

Natural Image Autoencoder-Based fMRI Representations for Trait and State Prediction

Sep 2026 · arXiv.org · Vol abs/2609.34167 · 0 citations · 32 references
Computer Science

TL;DR

FReD is introduced, which derives fMRI representations from a frozen Deep Compression AutoEncoder pre-trained exclusively on natural images and pairs them with a task specific readout, making frozen natural-image features as a useful baseline for assessing its added value on current fMRI benchmarks.

Abstract

Foundation models pre-trained on large-scale fMRI datasets have shown strong downstream performance, but at substantial data and computation cost. To investigate how much fMRI-specific pre-training is actually needed for such performance, we introduce FReD, which derives fMRI representations from a frozen Deep Compression AutoEncoder (DCAE) pre-trained exclusively on natural images and pairs them with a task specific readout. For trait prediction, FReD summarizes frame-wise representations by their temporal mean and log-standard deviation and applies linear probing, with late fusion across two normalization schemes. For state prediction, it represents each frame as a single token and models temporal dependencies with a shallow Transformer. Across four resting-state datasets spanning six trait-prediction targets, linear probes on frozen DCAE features generally outperform those on fMRI foundation model representations and remain competitive with fully fine-tuned fMRI foundation models. On three task-fMRI state-prediction tasks, a temporal readout on DCAE features performs comparably to the strongest foundation models evaluated. A Gaussian injection analysis further shows that localized signal changes are recovered more accurately from the frozen DCAE features than from the evaluated foundation-model representations. Together, these results show that strong performance on current fMRI benchmarks is possible without fMRI-specific representation pre-training, making frozen natural-image features as a useful baseline for assessing its added value.

View source

Similar papers

Preprint Aug 2026

MnemoDyn: Learning Resting State Dynamics from 40K FMRI sequences

It is shown that MnemoDyn is compute efficient and generalizes very well across diverse populations and scanning protocols, and when benchmarked against current state-of-the-art transformer-based approaches, MnemoDyn consistently delivers superior reconstruction quality.

Sourav Pal, V. Lương, Hoseok Lee et al. · 1 citation

Flattening the Connectome Spectrum: A Spectral Filter for FC Induces a Pretraining Target for fMRI Encoders

It is shown that KRR is weighted by the eigenvalues of the FC which are miscalibrated for phenotype prediction, and an efficient spectral filter is applied to recalibrate the eigenvalues of each subject's FC matrix, enabling the model to exploit more inter-individual variance.

G. Marraffini, Victoria Shevchenko, Carlo Alberto Barbano et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Neural State Prediction: Obstructing Shortcut Learning in EEG Foundation Models

Neural State Prediction (NSP), a latent-predictive framework that constrains both the prediction target and the available context, is introduced and jointly designing latent targets and their context offers a promising direction for EEG foundation models that learn from distributed signal structure.

Kieren Yu, Zi-Yang Liu, Chang Huang et al. · 0 citations
Open access Aug 2026

Frequency-filtered attention for cross-species fMRI time series prediction under small-sample learning

A general fMRI sequence prediction model, the Frequency-Filtered Attention Transformer (FFAformer), which models low-frequency variations in the frequency domain to capture long-range dependencies and incorporates FC consistency constraints to preserve brain network structure.

Chengcheng Du, Fei-Fei Zhao, Yin-Qian Sun et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.