When retrieved evidence contradicts an agent's prior beliefs, does it revise its answer, acknowledge uncertainty, or persist with an incorrect conclusion? Existing evaluations of agentic systems focus primarily on task success, offering limited insight into how agents handle such conflicts. We propose to evaluate agent...
Kaiser Sun, Bernal Jiménez Gutiérrez, Hong-Jun Liu et al.· 0 citations
An expert human evaluation is conducted, measuring summary preferences based on information satisfaction given a specific person’s background and use case, and finds that both traditional and LLM-based metrics are insufficient measures of information satisfaction and agree poorly with human judgment.
Isabel Cachola, W. Walden, Reno Kriz et al.· Proceedings of the 2026 ACM...· 0 citations
This work proposes A-B-D to infer traits bottom-up from behavioral data (B-data), namely how agents act on their environment and communicate with users, as recorded in existing trajectories, and offers a new lens for understanding AI personality.
Hao-Kai Zhao, Jie Gao, Yunze Xiao et al.· 0 citations
Analysis reveals that scaling model size, adding reasoning effort, expanding to authoritative web sources, and applying medical fine-tuning do not resolve these failure modes, demonstrating that they represent fundamental limitations of the retrieve-then-verify paradigm in open-ended medical settings rather than artifa...
He-Yuan Huang, Ji Dai, Alexandra DeLucia et al.· 0 citations
Integration of artificial intelligence chatbots into healthcare requires rigorous, patient-centered evaluation. This study implements the CREATE TRUST framework-a novel tool evaluating both clinical substance and communication style-to compare responses from healthcare providers (HCPs) and two AI chatbots (GPT-4, Mixtr...
Matthew R. Allen, V. Tiyyala, Armaan S Johal et al.· Journal of health communicat...· 0 citations
GenAI should augment, not replace, clinical judgment and relational care, and responsible use in geriatrics requires transparency, clinician-in-the-loop oversight, validation in older adult populations using age-relevant outcomes, and governance safeguards to protect dignity, safety, and equity.
A. Bharija, Ariba Khan, J. Pavon et al.· Journal of The American Geri...· 1 citation
It is argued that WG is more plausible as an adversarial threat-requiring careful data engineering-rather than as a significant hazard inherent to routine fine-tuning.
Miriam Wanner, M. Dredze, W. Walden· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
