When Does Depth Help? Understanding Layerwise Inference in Tabular Foundation Models
Transformer-based tabular foundation models (TFMs) achieve state-of-the-art performance on small to medium-sized tabular prediction tasks while processing diverse inputs through a shared stack of layers. Recent layerwise analyses identify strong predictive performance in early TFM representations and substantial overla...
Wen-Wen Gong, Hao-Tian Gu, Zhen Yang et al.
· 0 citations