Tabular foundation models (TFMs) based on prior-data fitted networks typically use sampled hard query labels, leaving avoidable sampling variability in pretraining. A key observation is that intermediate generator states can reveal label uncertainty beyond a sampled outcome. We propose Generator-state-Informed Supervis...
Zhang-Yang Peng, Jing-Ze Shi, Tong-Yi Zhang et al.· 0 citations
MALA, a fused attention primitive that preserves score access to every legal causal interaction and uses normalized contribution to allocate post-score computation, indicates that allocating post-score computation according to normalized attention contributions can retain the evaluated capabilities of FullAttn while re...
Jing-Ze Shi, Zhang-Yang Peng, Xian-Duo Li et al.· 0 citations
CoWA, a structured attention architecture that distributes access to the causal history across KV heads, shows that full causal coverage can be a collective property of the head ensemble rather than a duplicated property of every head.
Jing-Ze Shi, Zhang-Yang Peng, Xian-Duo Li et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.