With the rapid advancement of large language models (LLMs), multi-agent systems have emerged as a promising alternative to scaling up a single model. Existing approaches ensemble multiple LLMs to improve response quality, but they often rely on static prior knowledge of model capabilities and prompts, and require extensive parameter tuning. Some of the methods also treat each combination of LLMs as a learning objective, which leads to exponential time complexity. In this work, we propose an offline-to-online combinatorial experts selection (CES) framework to address these limitations. CES leverages offline evaluation to warm-start model capability estimation and employs online learning to adapt to capability shifts and correct offline inaccuracies. By integrating model features and input semantic representations into a combinatorial multi-armed bandit formulation, CES captures the interaction between prompts and LLMs without introducing complex auxiliary structures such as knowledge graphs. Modeling each LLM as a base arm with answer quality represented by a linear function of model and prompt features, CES achieves polynomial time complexity while excellently balancing performance and cost. Our experiments, conducted on popular LLM evaluation datasets such as AlpacaEval 2.0, show CES's effectiveness, laying the groundwork for future extensions.
Jinkun Xu, Minghan Wang, Zhiyong Wang et al.· Proceedings of the 32nd ACM...· 0 citations
On-policy distillation (OPD) provides dense teacher supervision on student-generated trajectories, but standard reverse-KL training can assign insufficient probability to other plausible continuations. Teacher entropy alone does not reveal whether uncertainty is concentrated among a few plausible next tokens or dispersed over a long probability tail, nor whether the student already represents those candidates well. Moreover, local teacher probabilities may not predict downstream success. We introduce Sparse Probing and Outcome-calibrated Targets OPD (SPOT), which addresses two coupled decisions, where to probe and what to distill, through an acquisition--exploration--exploitation procedure. During acquisition, a position-level score combines normalized teacher entropy, the probability mass captured by a small top-$k$ candidate set, and student--teacher mismatch to allocate a limited probing budget. During exploration, SPOT evaluates teacher-proposed candidates through verifier-scored student continuations. During exploitation, these outcomes produce a closed-form, KL-regularized target that favors candidates with better downstream outcomes while remaining anchored to the teacher distribution. Extensive experiments across multiple student models and reasoning benchmarks demonstrate the effectiveness of SPOT in improving reasoning performance while balancing solution quality and coverage.
Zikun Qu, Min Zhang, Ming-Ze Kong et al.· 3 citations
The EXPonential-weight algorithm for prompt Optimization} (EXPO) is proposed to automatically optimize the task description and meta-instruction in the meta-prompt for LLM-based agents and is extended to additionally optimize the exemplars (i.e., history of interactions) in the meta-prompt to further enhance the performance, hence introducing the EXPO-ES algorithm.
Ming-Ze Kong, Zhiyong Wang, Yao Shu et al.· arXiv.org· 7 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
