Artificial intelligence-based analysis of whole-body bone scintigraphy: The quest for the optimal deep learning algorithm and comparison with human observer performance

Abstract

Purpose: Whole-body bone scintigraphy (WBS) is one of the most widely used modalities in diagnosing malignant bone diseases during the early stages. However, the procedure is time-consuming and requires vigour and experience. Moreover, interpretation of WBS scans in the early stages of the disorders might be challenging because the patterns often reflect normal appearance that is prone to subjective interpretation. To simplify the gruelling, subjective, and prone-to-error task of interpreting WBS scans, we developed deep learning (DL) models to automate two major analyses, namely (i) classification of scans into normal and abnormal and (ii) discrimination between malignant and non-neoplastic bone diseases, and compared their performance with human observers.

Materials and methods: After applying our exclusion criteria on 7188 patients from three different centers, 3772 and 2248 patients were enrolled for the first and second analyses, respectively. Data were split into two parts, including training and testing, while a fraction of training data were considered for validation. Ten different CNN models were applied to single- and dual-view input (posterior and anterior views) modes to find the optimal model for each analysis. In addition, three different methods, including squeeze-and-excitation (SE), spatial pyramid pooling (SPP), and attention-augmented (AA), were used to aggregate the features for dual-view input models. Model performance was reported through area under the receiver operating characteristic (ROC) curve (AUC), accuracy, sensitivity, and specificity and was compared with the DeLong test applied to ROC curves. The test dataset was evaluated by three nuclear medicine physicians (NMPs) with different levels of experience to compare the performance of AI and human observers.

Results: DenseNet121_AA (DensNet121, with dual-view input aggregated by AA) and InceptionResNetV2_SPP achieved the highest performance (AUC = 0.72) for the first and second analyses, respectively. Moreover, on average, in the first analysis, Inception V3 and InceptionResNetV2 CNN models and dual-view input with AA aggregating method had superior performance. In addition, in the second analysis, DenseNet121 and InceptionResNetV2 as CNN methods and dual-view input with AA aggregating method achieved the best results. Conversely, the performance of AI models was significantly higher than human observers for the first analysis, whereas their performance was comparable in the second analysis, although the AI model assessed the scans in a drastically lower time.

Conclusion: Using the models designed in this study, a positive step can be taken toward improving and optimizing WBS interpretation. By training DL models with larger and more diverse cohorts, AI could potentially be used to assist physicians in the assessment of WBS images.

Keywords: Artificial intelligence; Bone; Deep learning; Scintigraphy; Whole-body.

Figure 1

Flowchart of inclusion and exclusion criteria.

Figure 2

An instance of normal and pathological cases according to nuclear medicine physicians’ reports.

Figure 3

Workflow of applied deep learning models. Ant: anterior, Post: posterior, SPP: spatial pyramid pooling, SE: squeeze-and-excitation, AA: attention-augmented.

Figure 4

Performance of the various models used in the first analysis in terms of accuracy, AUC, sensitivity, and specificity. Ant: anterior, Post: posterior, SPP: spatial pyramid pooling, SE: squeeze-and-excitation, AA: attention-augmented.

Figure 5

(a-e) ROC curves of the best 5 models for the first analysis. (f) Comparison between the ROC curves achieved by nuclear medicine physicians (NMPs) and the DL model achieving the highest AUC (DenseNet121_AA). Ant: anterior, Post: posterior, SPP: spatial pyramid pooling, SE: squeeze-and-excitation, AA: attention-augmented.

Figure 6

Performance of the various models used in the second analysis regarding the accuracy, AUC, sensitivity, and specificity. Ant: anterior, Post: posterior, SPP: spatial pyramid pooling, SE: squeeze-and-excitation, AA: attention-augmented.

Figure 7

(a-e) ROC curves of the best 5 models for the second analysis. (f) Comparison between the ROC curves achieved by nuclear medicine physicians (NMPs) and the best-performing DL model (InceptionResNetV2_SPP). Ant: anterior, Post: posterior, SPP: spatial pyramid pooling, SE: squeeze-and-excitation, AA: attention-augmented.

Figure 8

Model performance is compared using the DeLong test for the first strategy, which is run on the models’ AUCs. The models on columns and rows were evaluated against each other. Light blue: if the row model outperformed significantly the column model in terms of p-value. Purple: if the comparison between the row model and column model yielded a non-significant p-value. Ant: anterior, Post: posterior, SPP: spatial pyramid pooling, SE: squeeze-and-excitation, AA: attention-augmented.

Figure 9

Model performance is compared using the DeLong test for the second strategy, which is run on the models’ AUCs. The models on columns and rows were evaluated against each other. Light blue: if the row model significantly outperformed the column model in terms of p-value. Purple: if the comparison between the row model and column model yielded a non-significant p-value. Ant: anterior, Post: posterior, SPP: spatial pyramid pooling, SE: squeeze-and-excitation, AA: attention-augmented.