Large language model (LLM) evaluators are increasingly used to assess personalized assistants, yet whether they can reliably judge personalization has rarely been tested. We introduce PersEvalBench, a benchmark testing whether an evaluator can recognize appropriate and inappropriate uses of user history. It contains 40...
Xin-Yi Liu, H. Nayyeri, Emine Yilmaz et al.· 0 citations
Belief-Calibrated Optimization is introduced, a method that writes that belief down as a persistent in-context document and continually revises that document as new candidates are evaluated, and reaches a higher train passrate than a matched control that lacks only the world model.
Yu-Han Chen, Zhi-Hua Tian, Mahavir Dabas et al.· 1 citation
Hear2Act is introduced, a unified evaluation protocol for text and spoken assistants with 480 persona-grounded scenarios, hidden user concerns, and objectively verifiable outcomes that show that prosody matters when lexical evidence is insufficient, and that audio-capable LLMs can recover information from speech but do...
Xin-Yi Liu, H. Nayyeri, Dilek Hakkani-Tur et al.· 3 citations· ⚡1
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.