InvPT applies semantic-preserving transformations to the corpus and combines masked language modeling with multi-positive supervised contrastive learning that treats all augmentations of the same source function as positives, mixing self-contrast pairs with invariant-contrast pairs for positives of varying difficulty.
Yifeng He, Yundi Xu, Christopher Castro Gaw Gonzalo et al.· 0 citations
Speculative decoding accelerates rollout generation, which dominates the cost of reinforcement learning (RL) post-training. Online co-training can further increase the draft's accuracy, yielding greater speedups. However, scaling this approach to co-training on large models with long contexts poses two obstacles: (1) b...
This work introduces Farseer, a novel and refined scaling law offering enhanced predictive accuracy across scales, and provides new insights into optimal compute allocation, better reflecting the nuanced demands of modern LLM training.
Houyi Li, Wen-Zheng Zheng, Qiufeng Wang et al.· Neural Information Processin...· 4 citations· ⚡1
A novel Continual Audio-Visual Segmentation (CAVS) task, aiming to continuously segment new classes guided by audio, and a Collision-based Multi-modal Rehearsal (CMR) framework, designed to address challenges of multi-modal semantic drift and co-occurrence confusion.
Yuyang Hong, Qi Yang, Tao Zhang et al.· arXiv.org· 3 citations· ⚡1
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
