Rotation Decoupled DETR: Task-Aware Spatially Disentanglement for End-to-End Oriented Object Detection

Abstract

Although many detection transformer (DETR) variants have been successfully applied to oriented object detection, they largely overlook the spatial misalignment between classification and localization. While some studies have explored spatial disentanglement in end-to-end models, these methods either still suffer from feature misalignment or are specifically designed for horizontal objects, making them unsuitable for direct application to rotated object detection. To address this limitation, we focus on spatially decoupling the localization and classification tasks within an end-to-end oriented object detection framework. Specifically, we introduce a novel design scheme called rotation decoupled DETR (RDDetr), which incorporates three key components: a region-sensitive decoupled oriented query generation (RSDOQG) module, a task-aware mahalanobis self-attention (TAMSA) module, and a deformable rotated dual-align attention (DRDAA) module. Extensive experiments demonstrate that our approach achieves significant performance gains across five remote sensing datasets compared with the baseline model. Furthermore, we identify the distinct distribution patterns of the optimal feature regions for both classification and localization. Ultimately, by spatially decoupling these two tasks within the decoder, our approach effectively resolves the feature misalignment issue, thereby significantly boosting the performance of DETR-based models in rotated object detection.