GAA-DETR: Query-level gaze alignment for end-to-end lesion detection.
TL;DR
Gaze-Aligned-Attention Detection Transformer (GAA-DETR), which integrates gaze through three components: an adaptive gaze kernel for magnification-aware heatmap generation, a query-level alignment strategy for associating detector queries with gaze supervision, and a GAGO loss that remains effective when gaze is available for only part of the training data.
Abstract
Lesion detection is a fundamental task in medical image analysis. However, detectors trained only with bounding box supervision often develop boundary-biased attention and may overlook diagnostically relevant lesion content, which is closely related to false-positive predictions. To take a step toward addressing this limitation, we investigate expert gaze as an additional supervisory signal for lesion detection. We curate and release a large-scale gaze-enabled benchmark spanning three imaging modalities across five subsets, comprising 6148 images in total, with 2648 publicly released images paired with both gaze and lesion annotations. The dataset further records magnification information, allowing the modeling of diagnostic attention under multi-scale examination. Based on this resource, we propose Gaze-Aligned-Attention Detection Transformer (GAA-DETR), which integrates gaze through three components: an adaptive gaze kernel for magnification-aware heatmap generation, a query-level alignment strategy for associating detector queries with gaze supervision, and a GAGO loss that remains effective when gaze is available for only part of the training data. Extensive experiments across multiple datasets and detector families show that GAA-DETR consistently improves detection performance, reduces boundary-biased attention, and enhances interpretability. These results highlight the potential of clinically informed gaze supervision for reliable lesion detection. The dataset and code are available at https://github.com/YanKong0408/GAA-DETR.