Hao-Tian Gu

When Does Depth Help? Understanding Layerwise Inference in Tabular Foundation Models

Transformer-based tabular foundation models (TFMs) achieve state-of-the-art performance on small to medium-sized tabular prediction tasks while processing diverse inputs through a shared stack of layers. Recent layerwise analyses identify strong predictive performance in early TFM representations and substantial overla...