A Hybrid Curvelet-Deep Learning Framework for Optical and SAR Image Fusion

Abstract

The images offered by Optical and Synthetic Aperture Radar (SAR) provide supplementary data regarding the surface of the Earth. Rich spectral and color information is obtained with optical imagery, while structural and textual information is strong with SAR images in all weather and illumination conditions. Nevertheless, current fusion techniques tend to be ineffective in ensuring spectral fidelity and structural information at the same time and at the lowest levels of noise and distortion. This paper presents a hybrid optical-SAR image fusion framework to resolve this limitation, incorporating fast discrete Curvelet transform and a deep learning model using the VGG19 CNN. Under the proposed scheme, the optical data is initially transformed to the YCbCr colorspace, where the luminance component is combined with the SAR data to save the structural information, and the chrominance data is saved to recreate the color information later. Multiscale decomposition using Curvelets can seize the directional edge data, whereas the CNN creates high-level semantic data. A maximal fusion rule is applied to the combined output of the two branches to form the final fused image. Experiments on the QXS-SAROPT dataset reveal that the suggested hybrid framework achieves better quantitative results, with an entropy value of 7.75, which is evidence of a large amount of information. It also attains balanced Mutual Information values up to 0.341 (optical) and 0.309 (SAR), and has SAM values as low as 0.264. In addition, spectral preservation and structural integration are also better than a single CNN approach and a Curvelet. The novelty of the proposed scheme lies in the integration of transform-domain directional analysis with deep semantic features learning to obtain more informative, visually consistent fused images. The proposed approach can be applied to remote sensing procedures, including environmental monitoring, disaster evaluation, and city analysis.