Skip to content
Preprint

BrainTaskonomy: Learning How to Pretrain and What to Transfer in fMRI Foundation Models

Sep 2026 · 2 citations · 31 references
Computer Science Biology

TL;DR

Findings support organizing fMRI pretraining and adaptation by measured learning relations rather than treating domains and tasks as independent flat sets.

Abstract

fMRI foundation models increasingly aggregate heterogeneous data across brain states, cohorts, and acquisition settings, yet pretraining domains are commonly treated as a flat mixture and downstream tasks are adapted independently. We study whether measured learning relations can organize both stages without modifying the backbone. During pretraining, a lightweight Brain-DiT proxy estimates difficulty and directed facilitation across ten fMRI domains, yielding a priority-guided cumulative domain curriculum combined with high-to-low-noise timestep scheduling and joint consolidation. During adaptation, controlled first- and higher-order transfer across fifteen tasks constructs a directed taskonomy, from which budgeted integer programming (BIP) selects directly supervised source tasks and target-specific routes. The joint priority-domain and high-to-low-timestep curriculum reduces v-NMSE, PSD-NMSE, and FC-MSE by 6.5%, 16.3%, and 10.5%, respectively, relative to uniform sampling over both dimensions, and shows strong downstream performance across six in- and out-of-domain tasks. The taskonomy reveals asymmetric, target-dependent transfer, while exploratory sealed-test evaluation shows larger descriptive gains for BIP policies when higher-order route spaces are available than for matched random controls. Together, these findings support organizing fMRI pretraining and adaptation by measured learning relations rather than treating domains and tasks as independent flat sets.

View source

Similar papers

BrainWideBench: Benchmarking large-scale pretraining and across-animal transfer in multi-region neural recordings

BrainWideBench is presented, a benchmark for evaluating across-animal transfer on multi-region neural recordings, built on the International Brain Laboratory Brainwide Map dataset of neural and behavioral recordings spanning 276 brain regions from 139 mice performing a sensory-guided decision-making task.

Alexandre Andre, S. Mahato, Vinam Arora et al. · 1 citation
#machine learning Preprint Sep 2026

A Scaling Study for fMRI Foundation Models

Overall, the results show that compute alone does not characterize fMRI scaling: performance depends on how pretraining data, model size, and training duration are combined.

Wen-Hao Ye, Xuan-Ye Pan, Jun-Feng Xia et al. · 0 citations
#artificial intelligence Review Sep 2026

A Roadmap for MEG Foundation Models

This perspective lays down the basic concepts needed to understand MEG foundation models and provides a didactic overview of the field's key design choices, including tokenization, sensor- versus source-space representations, sensor-geometry encoding, backbone architectures, self-supervised objectives, and pretraining...

Philipp Thölke, Hamza Abdelhedi, Yorguin-José Mantilla-Ramos et al. · 1 citation
#machine learning Preprint Oct 2026

Two Routes to the Middle: Placement Search and Brain Readouts Converge on Where Continual Learners Should Specialize

LS-B observes the first tasks through a frozen fMRI encoding model of twelve human visual areas and commits task-specific capacity once to the blocks whose readouts vary most across tasks relative to their stable structure, yielding stable, backbone-specific allocations.

Yuan Huang, Zi-Han Chen, Run-Bin Zhang et al. · 0 citations
Sep 2026

Natural Image Autoencoder-Based fMRI Representations for Trait and State Prediction

FReD is introduced, which derives fMRI representations from a frozen Deep Compression AutoEncoder pre-trained exclusively on natural images and pairs them with a task specific readout, making frozen natural-image features as a useful baseline for assessing its added value on current fMRI benchmarks.

Juhyeon Park, Yeonwook Kim, P. Y. Kim et al. · 0 citations

Cross-attention encoding models reveal dynamic spatiotemporal routing across human higher visual cortex

This work uses per-parcel cross-attention to dynamically route features from a self-supervised video model (V-JEPA-2) across both space and time, and finds that joint routing improves predictions of brain responses to held-out videos across higher visual regions, most consistently in lateral and dorsal visual areas ass...

Iishaan Inabathini, Margaret M. Henderson · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.