Edge-Aware Dynamic Convolution and Cross-Modal Relation-Aware Fusion Network for Infrared and Visible Image Fusion

Abstract

Infrared and visible image fusion combines complementary thermal and structural information from the two modalities into a single composite image. Existing methods have two critical limitations: (1) inadequate utilization of visible structural information causes blurred edges, and (2) modality-specific and shared responses are not always separately represented when learning the fusion weights. To address these problems, an edge-aware dynamic convolution and cross-modal relation-aware fusion network is proposed to fully exploit visible structural details and explicitly model complementary information. Specifically, an edge-aware dynamic convolution is designed to extract edge features from visible images for generating dynamic convolution kernels, which adaptively enhance edge details during fusion. Then, a cross-modal relation-aware adaptive fusion module is designed to explicitly compute difference and interaction features of infrared and visible features and predicts adaptive fusion weights to balance modality-specific information. In addition, a complementarity consistency loss is introduced to jointly constrain intensity, edge and complementary information. Experiments on several public datasets demonstrate that the proposed method provides better visual quality and quantitative performance.