Artificial Intelligence-Assisted Chest Radiography: A Prospective Crossover Multi-Reader Study on Diagnostic Performance and Workflow Efficiency.
Abstract
Rationale
AND
Objectives
To evaluate the impact of four commercially available AI solutions for chest radiography on diagnostic performance, workflow efficiency, and clinical decision-making in a real-world setting.
Materials And Methods
In this prospective, monocentric, crossover reader study, five readers (one to six years of experience) assessed 1200 consecutive patients undergoing chest radiography (1861 total radiographs) for pulmonary infiltrates, pleural effusions, mediastinal masses, pneumothorax, and pulmonary nodules under five conditions: without AI and with each of four AI algorithms. A 14-day washout period separated sessions, and case order was randomized. The reference standard was the finalized clinical report, supplemented by confirmatory CT when available. The primary outcome was diagnostic performance. Secondary outcomes included interpretation time, diagnostic confidence (5-point Likert scale), escalation to senior review, and CT recommendations.
Results
AI assistance did not improve diagnostic accuracy. For pleural effusions and pulmonary nodules, accuracy decreased in several reader-tool pairings due to increased false positives. However, workflow benefits were observed: three of five residents reported shorter interpretation times, with median reductions of 6 to 17 s per case (p≤0.031). Four of five readers reported higher diagnostic confidence. AI support reduced senior consultations in selected pairings and, in one case, lowered CT escalation.
Conclusion
Commercial AI tools did not improve diagnostic accuracy and, for some findings, reduced it through increased false positives, while shortening reading time, increasing confidence, and lowering escalation. These results suggest that to avoid automation bias, maintain accuracy, and achieve efficiency gains, AI deployment requires careful local adaptation. Additionally, clinical trials are necessary to address patient outcomes.