Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Oct;48(10):5782-5793.
doi: 10.1002/mp.15196. Epub 2021 Sep 14.

Multiclass classification of whole-body scintigraphic images using a self-defined convolutional neural network with attention modules

Qiang Lin  1   2   3 , Chuangui Cao  1   2 , Tongtong Li  1   2 , Yongchun Cao  1   2   3 , Zhengxing Man  1   2   3 , Haijun Wang  4
Affiliations

Multiclass classification of whole-body scintigraphic images using a self-defined convolutional neural network with attention modules

Qiang Lin et al. Med Phys. 2021 Oct.

Abstract

Purpose: A self-defined convolutional neural network is developed to automatically classify whole-body scintigraphic images of concern (i.e., the normal, metastasis, arthritis, and thyroid carcinoma), automatically detecting diseases with whole-body bone scintigraphy.

Methods: A set of parameter transformation operations are first used to augment the original dataset of whole-body bone scintigraphic images. A hybrid attention mechanism including the spatial and channel attention module is then introduced to develop a deep classification network, Dscint, which consists of eight weight layers, one hybrid attention module, two normalization modules, two fully connected layers, and one softmax layer.

Results: Experimental evaluations conducted on a set of whole-body scintigraphic images show that the proposed deep classification network, Dscint, performs well for automated detection of diseases by classifying the images of concerns, achieving the accuracy, precision, recall, specificity, and F-1 score of 0.9801, 0.9795, 0.9791, 0.9933, and 0.9792, respectively, on the test data in the augmented dataset. A comparative analysis of Dscint and several classical deep classification networks (i.e., AlexNet, ResNet, VGGNet, DenseNet, and Inception-v4) reveals that our self-defined network, Dscint, performs best on classifying whole-body scintigraphic images on the same dataset.

Conclusions: The self-defined deep classification network, Dscint, can be utilized to automatically determine whether a whole-body scintigraphic image is either normal or contains diseases of concern. Specifically, better performance of Dscint is obtained on images with lesions that are present in relatively fixed locations like thyroid carcinoma than those with lesions occurring in nonfixed locations of bone tissue.

Keywords: attention mechanism; bone scintigraphy; convolutional neural network; medical image analysis; multiclass classification.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

FIGURE 1
FIGURE 1
Illustration of mirroring, translating, and rotating whole‐body SPECT scintigraphic image. (a) Original posterior image; (b) mirrored image; (c) translated image; and (d) rotated image by 3o to the right direction
FIGURE 2
FIGURE 2
Illustration of labelling a 2D whole‐body SPECT scintigraphic image using the LabelMe‐based annotation system
FIGURE 3
FIGURE 3
Hybrid attention module with the channel and spatial attention in the self‐defined Dscint network
FIGURE 4
FIGURE 4
Illustration of training Dscint on the original (blue) and augmented (orange) datasets. (a) Accuracy curves and (b) loss curves
FIGURE 5
FIGURE 5
Quantitative performance obtained by Dscint on test samples in the augmented dataset with average scores of evaluation metrics for different classes of concerns
FIGURE 6
FIGURE 6
Examples of misclassified whole‐body SPECT scintigraphic images with N = Normal; M = Metastasis; A = Arthritis; and T = Thyroid carcinoma. (a) Correctly classified images and (b) wrongly classified images
FIGURE 7
FIGURE 7
A comparison of evaluation metrics obtained by CNNs‐based classification models on test samples in the augmented dataset. (a) Specificity and (b) F‐1 score
FIGURE 8
FIGURE 8
ROC curves obtained by six models on test samples of the augmented dataset in Table 3
FIGURE 9
FIGURE 9
Confusion matrices obtained by six models on test samples in the augmented dataset
See this image and copyright information in PMC

References

    1. Lukaszewski B, Nazar J, Goch M, et al. Diagnostic methods for detection of bone metastases. Contemp Oncol. 2017;21:98‐103. - PMC - PubMed
    1. Lin Q, Man Z, Cao Y, et al. Classifying functional nuclear images with convolutional neural networks: a survey. IET Image Proc. 2020;14(14):3300‐3313.
    1. Pi Y, Zhao Z, Xiang Y, et al. Automated diagnosis of bone metastasis based on multi‐view bone scans using attention‐augmented deep neural networks. Med Image Anal. 2020;65:101784. - PubMed
    1. Zhao Z, Pi Y, Jiang L, et al. Deep neural network based artificial intelligence assisted diagnosis of bone scintigraphy for cancer bone metastasis. Sci Rep. 2020;10:17046. - PMC - PubMed
    1. Papandrianos N, Papageorgiou E, Anagnostis A, et al. Bone metastasis classification using whole body images from prostate cancer patients based on convolutional neural networks application. PLoS One. 2020;15(8):e0237213. - PMC - PubMed