Can Annual Report Tone Signal Audit Risk? Evidence from Large Language Model Textual Analysis of Chinese Listed Firms

Abstract

Using a sample of Chinese A-share listed firms from 2021 to 2024, we employ a large language model (LLM) to measure the tone of approximately 20,000 Management Discussion and Analysis (MD&A) sections and examine whether narrative tone signals future audit risk. We document four findings. First, MD&A tone significantly and negatively predicts the probability of receiving a modified audit opinion in the following year; the effect survives controls for the current-year opinion, firm fundamentals, and industry and year fixed effects, and holds within the subsample of firms with clean current-year opinions. Second, the predictive power operates through forward-looking information embedded in tone: firms with more negative tone exhibit significantly lower profitability and a higher likelihood of losses in the subsequent year. Third, in a horse race between the LLM-based measure and a conventional dictionary-based measure built on an established Chinese financial sentiment lexicon, only the LLM tone retains predictive power, indicating that the audit-relevant component of tone resides in contextual meaning that word counts cannot capture. Fourth, an anonymization test addresses look-ahead bias: after stripping all firm identifiers from the text, re-scored tone remains highly correlated with the original measure (Pearson correlation 0.79) and preserves its predictive power, ruling out the concern that the model “remembers” firm outcomes from its training data. In contrast, tone explains neither audit fees nor audit report lag, suggesting that soft information shapes auditors’ risk judgments rather than audit pricing. Our study provides a validation framework for LLM-based textual measures in accounting research and offers practical implications for using narrative disclosure in risk assessment.