We study learning rate (LR) scaling for hybrid architectures combining Transformer and State-Space Model (SSM) blocks, a class adopted by several recent production language models. In particular, we focus on the gap between the theoretical scaling rules derived for SSMs under zero-order-hold (ZOH) discretization at inf...
Jimin Seo, Gyubok Lee, Yeonsik Jo et al.· 0 citations
DCGC is introduced, a Masked Diffusion Model (MDM) framework for global correction that uses an imperfect solution draft from an upstream solver as auxiliary context that improves full test set accuracy by correcting low-consensus upstream outputs, highlighting its utility as a verifier-free global correction module fo...
STEP-KTODER is proposed, a framework for code preference optimization that defines steps as module-level functions in decomposed multi-function programs and assigns binary correctness labels via automatically generated unit tests and shows that execution-based labels are essential.
Idris Nechnech, Sehwan Kim, Jimin Seo et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.