Model fusion aims to integrate the capabilities of multi-expert models into a single model. Existing evaluations offer limited coverage of reasoning with long chains of thought and tool use, while strong average performance can mask substantial losses in individual expert capabilities. We introduce FusionEval, a benchm...
Shuo Cai, Yang-Gan Gu, Yuhang Liu et al.· 0 citations
Quantization-aware training (QAT) can recover much of the accuracy lost when large language models are compressed below four bits. Existing re- covery stages, however, are commonly optimized on fixed completions or teacher-generated answers, whereas the deployed quantized model condi- tions on prefixes generated by its...
Wen-Jun Wang, He-Ping Li, Yang-Gan Gu et al.· 0 citations
This paper characterize the interaction between mismatch and the policy-gradient direction, distinguishing locally amplifying from contracting update contributions that mismatch magnitude alone cannot identify and introduces TRIAGE, a direction-aware stabilization method that uses segment-level diagnosis to selectively...
Zhen Li, Shuai Zhang, Yang-Gan Gu et al.· 1 citation
This work introduces InfiMed2, a family of 4B and 27B generalist medical multimodal foundation models built around stage-aware data design, and curates a 55.68B-token corpus that combines broad clinical knowledge with context-rich biomedical visual evidence through source-specific processing.
SMAT (Simple MAT) is introduced, which jointly optimizes expert loss and expected loss at simulated merged parameters generated by sampling scaling coefficients, masks, and additive noise, and improves the mean score across five merging methods.
Yang-Gan Gu, Yuan-Yi Wang, Zhen Li et al.· 0 citations
Findings show that routing drift alone is insufficient evidence of routing failure, and source-informed corrections must be judged by their task-level intervention effects, and proposes Selective Router Repair as a case study, and finds that source-specialist token-likelihood advantages do not reliably identify benefic...
Yuan-Yi Wang, Yang-Gan Gu, Su Lu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.