Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Sep 24;19(9):e0308885.
doi: 10.1371/journal.pone.0308885. eCollection 2024.

SDAM: A dual attention mechanism for high-quality fusion of infrared and visible images

Jun Hu  1 , Xiaocen Zhu  1 , Kai Niu  1
Affiliations

SDAM: A dual attention mechanism for high-quality fusion of infrared and visible images

Jun Hu et al. PLoS One. .

Abstract

Image fusion of infrared and visible images to obtain high-quality fusion images with prominent infrared targets has important applications in various engineering fields. However, current fusion processes encounter problems such as unclear texture details and imbalanced infrared targets and texture detailed information, which lead to information loss. To address these issues, this paper proposes a method for infrared and visible image fusion based on a specific dual-attention mechanism (SDAM). This method employs an end-to-end network structure, which includes the design of channel attention and spatial attention mechanisms. Through these mechanisms, the method can fully exploit the texture details in the visible images while preserving the salient information in the infrared images. Additionally, an optimized loss function is designed to combine content loss, edge loss, and structure loss to achieve better fusion effects. This approach can fully utilize the texture detailed information of visible images and prominent information in infrared images, while maintaining better brightness and contrast, which improves the visual effect of fusion images. Through conducted ablation experiments and comparative evaluations on public datasets, our research findings demonstrate that the SDAM method exhibits superior performance in both subjective and objective assessments compared to the current state-of-the-art fusion methods.

PubMed Disclaimer

Conflict of interest statement

The authors have declared that no competing interests exist.

Figures

Fig 1
Fig 1. SDAM network structure.
Fig 2
Fig 2. Structure of the channel attention module.
Fig 3
Fig 3. Structure of the spatial attention module.
Fig 4
Fig 4. Partial examples of the dataset.
Fig 5
Fig 5. A qualitative comparison of SDAM with five state-of-the-art fusion methods on six typical pairs of infrared and visible images in the TNO dataset.
From top to bottom, they are: (a) the infrared image, (b) the visible image, (c) the fusion results of GANMcC, (d) the fusion results of IFCNN, (e) fusion results of SEDRFuse.
Fig 6
Fig 6. Qualitative comparison of SDAM and 5 state-of-the-art fusion methods on 6 typical pairs of infrared and visible images.
From top to bottom: (f) fusion results of RFN-Nest, (g) fusion results of STDFusion, (h) fusion results of MrFDDGAN, (i) fusion results of SOSMaskFuse, (j) fusion results of SDAM.
Fig 7
Fig 7. Qualitative comparison of SDAM with 5 state-of-the-art fusion methods on 6 typical infrared and visible image pairs in the M3FD dataset.
From top to bottom: (a) the infrared image, (b) the visible image, (c) the fusion results of GANMcC, (d) the fusion results of IFCNN, (e) fusion results of SEDRFuse.
Fig 8
Fig 8. Qualitative comparison of SDAM and 5 state-of-the-art fusion methods on 6 typical pairs of infrared and visible images.
From top to bottom: (f) fusion results of RFN-Nest, (g) fusion results of STDFusion, (h) fusion results of MrFDDGAN, (i) fusion results of SOSMaskFuse, (j) fusion results of SDAM.
Fig 9
Fig 9. Impact of parameter variations on image fusion performance.
Fig 10
Fig 10. Representative results of ablation experiments under different parameters.
(a) Infrared image, (b) visible image, (c) α = 1, β = 10, γ = 10, (d) α = 10, β = 10, γ = 1, (e) α = 10, β = 1, γ = 10, (f) α = 1, β = 10, γ = 1, (g) α = 10, β = 1, γ = 1, (h) α = 1, β = 1, γ = 10.
Fig 11
Fig 11. Representative results of ablation experiment with and without channel attention.
(a) Infrared image, (b) Visible image, (c) Without channel attention, (d) With channel attention.
Fig 12
Fig 12. Representative results of spatial attention ablation experiment.
(a) Infrared image, (b) Visible image, (c) Without spatial attention, (d) With spatial attention.
See this image and copyright information in PMC

Similar articles

See all similar articles

References

    1. Liu H, Ma M, Wang M, Chen Z, Zhao Y. SCFusion: Infrared and Visible Fusion Based on Salient Compensation. Entropy. 2023;25(7):985. doi: 10.3390/e25070985 - DOI - PMC - PubMed
    1. Han Z, Zhang C, Feng H, Yue M, Quan K. PFFNET: A Fast Progressive Feature Fusion Network for Detecting Drones in Infrared Images. Drones. 2023;7(7):424. doi: 10.3390/drones7070424 - DOI
    1. Xue Y, Zhang J, Lin Z, Li C, Huo B, Zhang Y. SiamCAF: Complementary Attention Fusion-Based Siamese Network for RGBT Tracking. Remote Sensing. 2023;15(13):3252. doi: 10.3390/rs15133252 - DOI
    1. Li L, Lv M, Jia Z, Jin Q, Liu M, Chen L, et al.. An Effective Infrared and Visible Image Fusion Approach via Rolling Guidance Filtering and Gradient Saliency Map. Remote Sensing. 2023;15(10):2486. doi: 10.3390/rs15102486 - DOI
    1. Wang C, Zang Y, Zhou D, Nie R, Mei J. An interactive deep model combined with Retinex for low-light visible and infrared image fusion. Neural Computing and Applications. 2023; p. 1–19.

LinkOut - more resources