A DeepLabv3+ and CBAM based framework for multi-focus image fusion

Abstract

Multi-focus Image Fusion (MFIF) technique aims to produce a full focused image that preserves the details from source images and enhance image fusion performance. A significant problem in MFIF is accurately segmenting the focused region particularly in complicated images with challenging boundaries. This limitation causes a loss of critical image information resulting in inferior fusion quality. The existing semantic segmentation methods such as FCN (Fully Convolutional Network), PSPNet (Pyramid Scene Parsing Network) and UNet frequently have difficulty with this task. To overcome these difficulties, this work presents a framework employing DeepLabv3+ architecture with Convolutional Block Attention Module (CBAM), named DLCMF. The CBAM improves the framework's ability using channel and spatial attention to focus on key characteristics in an image, effectively capturing “what” and “where” to look in an image. This helps preserve edge details and improves feature extraction accuracy which decreases the loss of critical information during the process of fusion. To optimize training, both cross-entropy loss and dice loss functions are employed. The complementary source images are fed to DLCMF network which generates binary pair segmentation maps that are subsequently refined using morphological operations to yield final segmentation maps. Finally, the dot product is computed between the inputs and their corresponding binary map followed by a pixel-wise summation. Experimental findings indicate that DLCMF generates fused images of higher quality compared to those generated by other nine existing state-of-the-art methods.