Linguistic Uncertainty Markers for Trust Calibration in AI-Assisted Decision-Making
TL;DR
While participants rated hedged and unhedged AI as equally trustworthy and likely to be correct, they were significantly less likely to follow hedged advice in a binary choice, and how linguistic markers can be used to calibrate user reliance to model certainty is discussed.
Abstract
As artificial intelligence (AI) systems are increasingly deployed for complex decision-making, calibrating user trust to prevent overreliance on overconfident AI remains a critical challenge. This paper investigates linguistic hedging – the use of tentative language to soften claims and indicate limited certainty – to communicate AI uncertainty, and its impact on user reliance. In a two-part investigation into decision-making in the financial domain, a formative study (N=36) explored strategies for eliciting hedged responses in accordance with model confidence. A confirmatory study (N=71) then measured actual behavioral reliance in a financial investment decision task, manipulating both AI confidence and the decision risk. Our findings reveal that while participants rated hedged and unhedged AI as equally trustworthy and likely to be correct, they were significantly less likely to follow hedged advice in a binary choice. We discuss how linguistic markers can be used to calibrate user reliance to model certainty, reducing overreliance while preserving trust in the system.