The Current Generation of Tabular Foundation Models: A Critical Review
Tabular foundation models (TFMs) have moved tabular machine learning from per-dataset training towards amortised in-context inference, fitting a small-to-medium table in a single forward pass without a training run. The 2024–2026 release train, the TabPFN and TabICL lines and challengers such as Mitra, LimiX and Orion, has produced a generation whose architectures, capabilities and limits are documented mainly in preprints, while existing surveys treat these models as a subsection of tabular deep learning or of language-model table understanding. This review is, to our knowledge, the first organised around the current generation. From a corpus of 961 screened records and 98 retained studies, it taxonomises the architectures by pretraining regime, maps the capability space across five axes, isolates the language-model-on-tabular strand for prediction, feature engineering and generation, and summarises openness and deployment. A dedicated critical synthesis then reads the reported capabilities against independent evidence: on the studies reviewed here, tree-based and deep models retain the lead across 142 curated datasets that go beyond the standard independent and identically distributed setting; on 112 datasets, the models attain the highest accuracy but weaker conditional coverage than gradient-boosted trees; and robustness under feature shift, fairness and generation quality remain open. Amortised in-context prediction is thus a working paradigm whose independent evidence has yet to match its benchmark claims.