Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Jun 25;41(3):544-551.
doi: 10.7507/1001-5515.202305025.

[Skin lesion classification with multi-level fusion of Swin-T and ConvNeXt]

[Article in Chinese]
Zetong Wang  1 , Junhua Zhang  1 , Xiao Wang  1
Affiliations

[Skin lesion classification with multi-level fusion of Swin-T and ConvNeXt]

[Article in Chinese]
Zetong Wang et al. Sheng Wu Yi Xue Gong Cheng Xue Za Zhi. .

Abstract
in English, Chinese

Skin cancer is a significant public health issue, and computer-aided diagnosis technology can effectively alleviate this burden. Accurate identification of skin lesion types is crucial when employing computer-aided diagnosis. This study proposes a multi-level attention cascaded fusion model based on Swin-T and ConvNeXt. It employed hierarchical Swin-T and ConvNeXt to extract global and local features, respectively, and introduced residual channel attention and spatial attention modules for further feature extraction. Multi-level attention mechanisms were utilized to process multi-scale global and local features. To address the problem of shallow features being lost due to their distance from the classifier, a hierarchical inverted residual fusion module was proposed to dynamically adjust the extracted feature information. Balanced sampling strategies and focal loss were employed to tackle the issue of imbalanced categories of skin lesions. Experimental testing on the ISIC2018 and ISIC2019 datasets yielded accuracy, precision, recall, and F1-Score of 96.01%, 93.67%, 92.65%, and 93.11%, respectively, and 92.79%, 91.52%, 88.90%, and 90.15%, respectively. Compared to Swin-T, the proposed method achieved an accuracy improvement of 3.60% and 1.66%, and compared to ConvNeXt, it achieved an accuracy improvement of 2.87% and 3.45%. The experiments demonstrate that the proposed method accurately classifies skin lesion images, providing a new solution for skin cancer diagnosis.

皮肤癌是一个重要的公共卫生问题，计算机辅助诊断技术可以有效地减轻这一负担。在采用计算机辅助诊断时，准确识别皮肤病变类型至关重要。为此，本文提出一种基于Swin-T与ConvNeXt的多级注意力逐级融合模型，采用分层Swin-T与ConvNeXt分别提取全局与局部特征，并提出残差通道注意力与空间注意力模块进一步提取有效特征；利用多级注意力机制对多尺度全局与局部特征进行处理；针对浅层特征因离分类器较远而丢失的问题，采用逐级聚合的思想，提出逐级倒置残差融合模块动态调整提取的特征信息。本文通过均衡采样策略以及焦点损失，解决皮肤病变类别不平衡的问题。在ISIC2018、ISIC2019数据集上进行测试，其准确率、精确率、召回率和F1-Score分别是96.01%、93.67%、92.65%、93.11%与92.79%、91.52%、88.90%、90.15%。与Swin-T相比，准确率分别提升了3.60%和1.66%；与ConvNeXt相比，准确率分别提升了2.87%和3.45%。实验表明，本文提出的方法能够准确分类皮肤病变图像，为皮肤癌的诊断提供了新的解决方案。.

Keywords: ConvNeXt; Hierarchical inverted residual fusion module; Multi-level attention mechanisms; Skin lesion images; Swin-T.

PubMed Disclaimer

Conflict of interest statement

利益冲突声明：本文全体作者均声明不存在利益冲突。

Figures

图 1
图 1
Overall network structure diagram 网络整体结构图
图 2
图 2
Residual spatial attention (Left) and residual channel attention (Right) 残差空间注意力（左）和残差通道注意力（右）
图 3
图 3
Hierarchical inverted residual fusion 逐级倒置残差融合块
图 4
图 4
Heatmap of various layers' feature in HIRF HIRF各层特征热力图
图 5
图 5
Confusion matrix of classification results in ISIC2018 and ISIC2019 datasets ISIC2018、ISIC2019数据集分类结果混淆矩阵
图 6
图 6
Cross-test confusion matrix in ISIC2018 and ISIC2019 datasets ISIC2018、ISIC2019数据集交叉测试混淆矩阵
See this image and copyright information in PMC

Similar articles

See all similar articles

References

    1. Kumari S, Choudhary P K, Shukla R, et al Recent advances in nanotechnology based combination drug therapy for skin cancer. J Biomater Sci Polym Ed. 2022;33(11):1435–1468. doi: 10.1080/09205063.2022.2054399. - DOI - PubMed
    1. Steiner A, Binder M, Schemper M, et al Statistical evaluation of epiluminescence microscopy criteria for melanocytic pigmented skin lesions. J Am Acad Dermatol. 1993;29(4):581–588. doi: 10.1016/0190-9622(93)70225-I. - DOI - PubMed
    1. Celebi M E, Iyatomi H, Schaefer G, et al Lesion border detection in dermoscopy images. Comput Med Imaging Graph. 2009;33(2):148–153. doi: 10.1016/j.compmedimag.2008.11.002. - DOI - PMC - PubMed
    1. Ganster H, Pinz P, Rohrer R, et al Automated melanoma recognition. IEEE Trans Med Imaging. 2001;20(3):233–239. doi: 10.1109/42.918473. - DOI - PubMed
    1. Yu L, Chen H, Dou Q, et al Automated melanoma recognition in dermoscopy images via very deep residual networks. IEEE Trans Med Imaging. 2017;36(4):994–1004. doi: 10.1109/TMI.2016.2642839. - DOI - PubMed

Publication types

MeSH terms

LinkOut - more resources