The KSA V2 is developed, which not only achieves a more flexible KV cache memory compression but also reduces the computation cost in Full attention layer and finds that the summary tokens can serve two roles: the KV memory and sparse indexer.
OneRec Team Chenglong Chu, Guo-Hong Mu, Guo-Wang Zhang et al.· 0 citations
A series of models are explored that connect item Semantic IDs with natural language in a unified representation space and seek to unlock the potential of natural-language chain-of-thought (CoT) reasoning for recommendation, including OneReason.
The proposed Uncertainty-Calibrated MOPD (UCMOPD), which addresses limitations of standard MOPD through two complementary mechanisms, improves the general-capability average over standard MOPD by $4.48\% and $7.86\% while maintaining vertical-domain performance.
Zi-Yuan Liu, Jiao Ou, Jian Liang et al.· 0 citations
Counteraction-Aware Multi-Teacher On-Policy Distillation is proposed, which addresses issues with decoupled alternating training and gap-based sample selection and outperforms all baselines in general capability recovery while maintaining domain-specific behavior.
Tian-Le Chen, Jiao Ou, Zi-Yuan Liu et al.· arXiv.org· 13 citations· ⚡1
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.