Processing long visual token sequences from high-resolution images makes multi-step reasoning computationally expensive for multimodal Large Language Models (MLLMs). Existing one-shot pruning and aggregation methods compress visual tokens into a fixed context before decoding. However, visual evidence needs can shift as...
Yi-Cheng Xue, Han Wu, Ju-Feng Yang et al.· 0 citations
Large language model search agents are often trained with synthetic questions whose difficulty is increased through larger evidence graphs, additional hops, and longer trajectories. These global properties, however, are only indirect proxies for the local retrieval capabilities required during search. To address this m...
Lin-Zhi Peng, Han-Ting Chen, Heng Chang et al.· 0 citations
This work formulate expert-cache management as a model-side algorithmic problem and proposes cache-aware post-training that jointly adapts the MoE backbone and lightweight auxiliary routers while preserving the native inference-time Top-K rule.
Zhen-He Wu, Ya-Ping Jin, Qing-Hua Xing et al.· 4 citations
These results support depthwise convolution as a lightweight complement to self-attention for modeling short-range token interactions and suggest that the convolution makes repeated token IDs more sensitive to their immediate context.
Yu-Chuan Tian, Yingte Shu, Wei He et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
