This paper considers multilingual pretraining as a cooperative game in which each language acts as a player that jointly contributes to pretraining, gaining the resulting reduction in test loss as the payoff and proposes a game-theoretic multilingual scaling law called ShapleyLaw.
It is shown that a lightweight LLM-only DPO update on tiny single-object-pair synthetic data mitigates the bias, lifting four-way robust accuracy by up to 100 points on synthetic data, and by 68.1 points on broader evaluation datasets WhatsUp, SpatialMQA-Direct, and VSR.
A novel Multi-Margin Collaborative Debiasing (MMCD) framework is proposed, which adaptively integrates frequency-aware, confidence-aware, and difficulty-aware angular margins with a dynamic, difficulty-aware contrastive learning mechanism to reshape decision boundaries under biased training conditions.
Huanjia Zhu, Shuyuan Zheng, Yishu Liu et al.· Neural Information Processin...· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.