Model fusion aims to integrate the capabilities of multi-expert models into a single model. Existing evaluations offer limited coverage of reasoning with long chains of thought and tool use, while strong average performance can mask substantial losses in individual expert capabilities. We introduce FusionEval, a benchm...
Shuo Cai, Yang-Gan Gu, Yuhang Liu et al.· 0 citations
We present IronLLM-0.6B, a 654M-parameter language model designed for efficient on-device inference. IronLLM-0.6B combines a hybrid attention architecture with X-MTP, a lightweight shared-KV multi-token prediction design that eliminates per-depth KV-cache replay and employs a lightweight verification head for rollback-...
Chang-Di Yang, Feng-Quan Jiao, Hao-Chih Lin et al.· arXiv.org· 0 citations
Findings show that routing drift alone is insufficient evidence of routing failure, and source-informed corrections must be judged by their task-level intervention effects, and proposes Selective Router Repair as a case study, and finds that source-specialist token-likelihood advantages do not reliably identify benefic...
Yuan-Yi Wang, Yang-Gan Gu, Su Lu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.