Behavioral consistency across repeated executions provides an informative signal for assessing the reliability of large language model (LLM) agents. However, existing approaches typically summarize consistency with a single task-level score, without identifying where behavioral differences occur or whether actions are...
Ya-Xin Ma, Wen-Bo Zhang, Satya K. Pothapragada et al.· 0 citations
The Optimal Transport-based Robust Off-Policy Evaluation (OTROPE) is proposed, a likelihood-free evaluation that performs distributional correction in a semantic space via optimal transport to align labeled behavior-policy samples with unlabeled target-policy samples, yielding a doubly robust-style evaluation.
Empirically, UOPD is evaluated across a broad range of agentic tasks, including ALFWorld, WebShop, and Search, demonstrating its superior performance over OPD methods and their variants.
Wen-Bo Zhang, Peng-Cheng Xu, Wei-Zhi Du et al.· 3 citations· ⚡1
This work proposes Distilled Preference Probability Policy Optimization (DP3O), an effective and efficient offline alignment algorithm that outperforms state-of-the-art offline methods, achieves performance comparable to iterative DPO, and reduces training time, demonstrating both its effectiveness and efficiency.
Wen-Bo Zhang, Wen-Zhuo Zhou, Heng-Rui Cai et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.