Are Reasoning Traces Faithful in LLM-Based Code Vulnerability Detection?

Abstract

Reasoning traces generated by large language models (LLMs) provide interpretability by exposing the factors that purportedly support their predictions. Such interpretability is important because model decisions may rely on spurious factors, and it can help practitioners identify such reliance and respond appropriately. However, the usefulness of reasoning traces hinges on their faithfulness, namely whether they reflect the factors that the model actually relies on when producing its prediction. Unfaithful traces may instead serve as plausible narratives that do not correspond to the true decision process. Despite its importance, reasoning-trace faithfulness has not been systematically studied in LLM-based vulnerability detection. We present VulFaith, the first framework for evaluating reasoning-trace faithfulness in this domain. We formalize two complementary criteria: completeness, which requires that all decision-relevant factors sustaining an insecure prediction are exposed, and per-factor faithfulness, which requires that each cited factor is genuinely relied upon by the model. To operationalize these criteria, we design an automated repair-based evaluation scheme that removes cited insecurity factors through code repairs and re-queries the model to test whether the prediction remains sustained. We implement VulFaith and conduct experiments on both repository-level and function-level vulnerability datasets, evaluating ten state-of-the-art open-source models and four commercial LLMs. Our results show that unfaithfulness is widespread, with completeness unfaithfulness reaching up to 66.14% and per-factor unfaithfulness up to 94.00%. We further validate that the unfaithfulness cases identified by VulFaith can support data augmentation for supervised fine-tuning, reducing completeness unfaithfulness by up to 47.89% and per-factor unfaithfulness by up to 59.18%.