Parallel Inconsistency Guidance (PIG), a new paradigm that uses a trained auxiliary model to directly estimate the inconsistency between parallel and sequential generation from a single model forward pass, is proposed.
Quentin Fruytier, Anton Xue, Sanjay Shakkottai et al.· 1 citation
Output-layer gradients suffice for effective data selection, yet require only the cheaper forward pass, and a drop-in wrapper for selection methods that reduces the feature-extraction FLOP cost by $9.7\times$ for SFT and $3.0\times$ for RL benchmarks, while tracking full-gradient performance on downstream tasks.
Lyuxin David Zhang, Eric Wong, Surbhi Goel et al.· 1 citation
This work investigates whether hybrid backbones can become effective DLMs by adapting Qwen3.5 at 0.8B, 2B, 4B, and 9B scales, and finds that they can be efficient starting points for adaptation.
Anton Xue, Litu Rout, Aditya Akella et al.· arXiv.org· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.