This work introduces Coarse-to-fine Error-aware Dynamic Attention Routing (CEDAR), a coarse-to-fine method that keeps the language model frozen while preserving global coverage and derives an output-error bound governed by within-chunk key/value dispersion and uses it to allocate a variable refinement budget.
This work shows that a state-of-the-art weight-inheritance distiller, Low-Rank Clone (LRC), deploys a full-width student MLP but ties training to a teacher-induced slice, leaving 62.5-81.4% of each deployed matrix's independent linear degrees of freedom unreachable-paid for at inference, never trainable.
Wen-Hui Chen, Zhi-Feng Li, Jie Zhou et al.· 0 citations
MathForm is introduced, an autoformalization framework for constructing verified training data through Mathlib knowledge retrieval and verification-guided iterative refinement, and outperforming multiple specialized 32B autoformalizers.
Lushi Pu, Weiming Zhang, Xinheng Xie et al.· 0 citations
UltraX is proposed, a function-calling refinement framework for large-scale pre-training data that completes the editing function space by introducing insertion in addition to deletion and modification, enabling fine-grained instance-level editing.
Xinlong Zhao, Dongsheng Liu, Hengyu Zhao et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.