A Lesion-Aware Knowledge-Guided Framework for Explainable Diabetic Retinopathy Diagnosis

Abstract

Diabetic Retinopathy (DR) is a leading cause of preventable blindness, requiring accurate and timely diagnosis for effective treatment. While deep learning models achieve high classification accuracy, they often lack clinical interpretability. Conversely, large language models (LLMs) provide strong reasoning capabilities but struggle to process raw visual inputs. In this work, we propose a lesion-aware and knowledge-guided framework for explainable DR diagnosis. The proposed system integrates visual descriptions generated by a vision-language model, structured lesion features extracted using a segmentation model, and clinical knowledge incorporated through a curated knowledge base. These components enable LLM-based reasoning over structured medical representations. We evaluate our approach on an external retinal dataset under a binary screening setting (No DR vs Vision-Threatening DR). Experimental results demonstrate improved classification performance compared to baseline LLM approaches, with reduced prediction bias and more balanced sensitivity and specificity. In addition, the proposed framework produces clinically consistent explanations aligned with established guidelines. Our findings highlight that grounding LLMs with lesion-level features and domain knowledge is essential for reliable and interpretable medical decision-making, offering a promising direction for explainable AI in medical imaging.