Following complex instructions with multiple explicit constraints remains a fundamental challenge for large language models (LLMs). Existing alignment methods, such as DPO, optimize holistic reward signals that often underemphasize strict satisfaction of individual constraints, particularly under out-of-distribution or multi-constraint settings. In this paper, we propose STAIF, a stage-wise optimization framework that decouples the alignment of subjective (soft) constraints from the optimization of objectively verifiable (hard) constraints. Stage 1 applies preference optimization with multiple negative samples to sharpen sensitivity to soft constraints, while Stage 2 applies Reinforcement Learning with Verifiable Rewards (RLVR) to enforce strict compliance with hard constraints. To support this method, we construct STAINSTRUCT, a high-quality bilingual (English, Chinese) dataset of approximately 31,000 complex multi-constraint instructions. Extensive analyses validate the design of STAIF and show state-of-the-art performance on representative benchmarks against strong baselines, as well as genuine generalization.
This work proposes Cross-Relational Preference Learning (CRPL), a novel framework for constructing preference data that explicitly models inter-instruction relationships through two key techniques: Cross-Relationship Perturbation and Cross-Region Pair Sampling, which enables the generation of more diverse preference da...
Run-Sheng Li, Kai Sun, Bin Shi et al.· Proceedings of the Thirty-Fi...· 0 citations
This work proposes VPO, a negative gradient constraint method for human non-preference samples based on V -usable information, which can alleviate the squeezing effect of DPO, enhance alignment with the generation objective, and maintain the model’s ability to distinguish between preference and non-preference samples.
Zecheng Wang, Chunshan Li, Yupeng Zhang et al.· Neural Information Processin...· 1 citation
UNSPECIFIC is a novel framework that synthesizes constraints common to two similar reference articles to reduce copy-pasting, selectively hardens only trivially satisfied constraints to balance difficulty and naturalness, and evaluates satisfaction on both the generated article and its summary to penalize superficial i...
J. Sharma, Balpreet Kaur, Jeremiah Hong et al.· 0 citations
This paper proposes LoRE, a novel two-stage training framework for e-commerce search relevance, which outperforms GPT-5 by 29.1% in Macro-F1 and achieving a relative 27% online gain, offering a vital reference for industrial domain-specific post-training post-training.
Chenji Lu, Zhuo Chen, Hui Zhao et al.· Annual Meeting of the Associ...· 0 citations
We present LuckyStar 111B, a 111B-parameter hybrid reasoning model developed through a collaboration between Cohere and LG CNS for Korean-English enterprise agents under practical memory and serving constraints. The model trains from Cohere's fully post-trained Command A model rather than a new pretraining run, and use...
Utsav Garg, Sungjin Hong, Jason J. Jung et al.· arXiv.org· 1 citation
Faithfulness -- how precisely a generated image aligns with its prompt -- is increasingly central to the real-world utility of text-to-image (T2I) models. Existing faithfulness benchmarks, however, rely on simple atomic instructions, on which top-tier systems already achieve near-perfect scores. As T2I models enter cre...
Yuanhao Ban, Tong Xie, Sohyun An et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
