MAMFuse: Multi-modality Image Fusion with Multiscale Attention Mechanism

Abstract:
Merging multi-modality images aim to integrate valuable information from various source images into the same image for human perception. Infrared and visible images have unique imaging modes, and their complementary information fusion can be widely used in night monitoring, detection, and tracking. However, the existing fusion strategies cannot clearly distinguish between target and background regions in the image. Consequently, we propose a fusion strategy of a multi-scale attention mechanism called MAMFuse. The attention mechanism can capture typical information without losing details and suppress the image noise interference in the fusion stage. We adopt the nested encoder-decoder unsupervised learning structure. Meanwhile, MAMFuse includes an encoder group, a fusion strategy, and a decoder group. The encoder group extracts the original features from the infrared and visible spectrum. And the obtained original aspects are input into a multi-scale attention module to obtain attention features. Then, two attention features are multiplied by elements to obtain the target attention feature. Original features and target attention feature constitute the final reconstruction feature. Finally, the decoder group generates fused images by reconstruction feature. We conduct qualitative and quantitative experiments on the seven fusion methods, which proved the effectiveness of MAMFuse.