How Faithful Is the Reasoning of LLM Recommenders? A Counterfactual Audit
TL;DR
This work introduces a counterfactual audit that edits a model’s own reasoning trace using four operators that flip the stated preference, swap in another user’s reasoning, erase it, or paraphrase it as a control, and measures whether the top-1 recommendation changes.
Abstract
Large language model (LLM)-based recommenders increasingly generate reasoning alongside their recommendations [5], implicitly suggesting that the stated reasoning determines the selected item. We investigate whether these explanations exhibit reason-responsiveness or are post-hoc rationalizations. We introduce a counterfactual audit that edits a model’s own reasoning trace using four operators that (1) flip the stated preference, (2) swap in another user’s reasoning, (3) erase it, or (4) paraphrase it as a control, and then measures whether the top-1 recommendation changes. We define the faithfulness gap as the mean top-1 change rate under the three meaning-changing edits (flip, swap, erase) minus the change rate under paraphrasing. We evaluate the same 100 users from the Amazon Video Games dataset using GPT-5 Chat, GPT-5 Reasoning, and DeepSeek-R1. GPT-5 Chat exhibits substantial reason-responsiveness, with a faithfulness gap of 0.57. In contrast, DeepSeek-R1 and GPT-5 Reasoning yield gaps of 0.02 and − 0.11, respectively, providing little evidence of semantic-specific reason-responsiveness in their displayed explanations beyond sensitivity to the control edit. As a follow-up diagnostic, we apply the same audit to DeepSeek-R1’s trace and obtain a gap of 0.30. This result suggests that, for DeepSeek-R1, recommendations are more responsive to interventions on the internal reasoning trace than on the explanation presented to the user.