Adaptive Edge-Aware Breast Cancer Classification from Mammograms using Explainable Density-Guided Filtering with Transformer-Enhanced Deep Unfolding Network and Hybrid Optimization

Abstract

Breast cancer (BC) remains one of the foremost causes of cancer-related mortality among women globally, and early detection through mammography screening is essential for improving patient survival outcomes. Although recent deep learning frameworks have demonstrated encouraging performance in automated mammogram classification, two critical limitations persist: first, static preprocessing parameters that fail to adapt to heterogeneous breast tissue densities, often resulting in the over-smoothing of diagnostically critical microcalcifications; and second, opaque black-box classification mechanisms that hinder clinical interpretability and limit radiologist trust. To address these challenges, this study proposes a novel framework integrating an Adaptive Density-Guided Filtering (ADGF) module for patient-specific image preprocessing, which dynamically estimates tissue density maps aligned with BI-RADS categories and adjusts filtering parameters accordingly. A deep unfolding network with vision transform enhanced architecture for classification purposes improves upon a traditional convolutional encoder by capturing long-range spatial correlation using a multi-head self-attention mechanism across the entire image of the mammogram. The clinical readability of this approach is produced by combining gradient-weighted class activation mapping (Grad-CAM) to provide pixel level saliency maps that visually illustrate the logic behind each classification.Cascading Residual Graph Convolutional Networks (CRGCN) retain structural relational modeling among spatial tissue regions. An Adaptive Levy Flight Greylag Goose Optimization (ALF-GGO) algorithm fine-tunes the ViT-DUN weight parameters, enhancing convergence stability and generalization across diverse datasets. Evaluation of the proposed framework was conducted on three publicly available mammography benchmarks, MIAS, INbreast, and DDSM; the classification accuracies achieved were 99.12%, 99.08%, and 98.97% respectively, and the overall ROC-AUC score across all three datasets was greater than 0.99. The results indicate that the proposed framework has a high degree of robustness, explanation, and computational efficiency, demonstrating its capabilities for computer-aided diagnosis of multi-class breast cancer.