An accurate prediction of the origin for bone metastatic cancer using deep learning on digital pathological images

Abstract

Background: Determining the origin of bone metastatic cancer (OBMC) is of great significance to clinical therapeutics. It is challenging for pathologists to determine the OBMC with limited clinical information and bone biopsy.

Methods: We designed a regional multiple-instance learning algorithm to predict the OBMC based on hematoxylin-eosin (H&E) staining slides alone. We collected 1041 cases from eight different hospitals and labeled 26,431 regions of interest to train the model. The performance of the model was assessed by ten-fold cross validation and external validation. Under the guidance of top3 predictions, we conducted an IHC test on 175 cases of unknown origins to compare the consistency of the results predicted by the model and indicated by the IHC markers. We also applied the model to identify whether there was tumor or not in a region, as well as distinguishing squamous cell carcinoma, adenocarcinoma, and neuroendocrine tumor.

Findings: In the within-cohort, our model achieved a top1-accuracy of 91.35% and a top3-accuracy of 97.75%. In the external cohort, our model displayed a good generalizability with a top3-accuracy of 97.44%. The top1 consistency between the results of the model and the immunohistochemistry markers was 83.90% and the top3 consistency was 94.33%. The model obtained an accuracy of 98.98% to identify whether there was tumor or not and an accuracy of 93.85% to differentiate three types of cancers.

Interpretation: Our model demonstrated good performance to predict the OBMC from routine histology and had great potential for assisting pathologists with determining the OBMC accurately.

Funding: National Science Foundation of China (61875102 and 61975089), Natural Science Foundation of Guangdong province (2021A15-15012379 and 2022A1515 012550), Science and Technology Research Program of Shenzhen City (JCYJ20200109110606054 and WDZC20200821141349001), and Tsinghua University Spring Breeze Fund (2020Z99CFZ023).

Keywords: Bone metastatic cancer; Deep learning; Digital pathology; Origin; Regional multiple-instance learning.

Fig. 1

The work-flow of diagnosing the OBMC. (a) The typical workflow for pathologists to make differential diagnosis of the OBMC. Pathology doctors diagnosed the OBMC based on the pathology of H&E slides, the results of dozens of IHC tests and clinical correlation. (b) The new workflow of applying the method of RMIL to determine the OBMC. Based on the labeled ROI in a WSI, our model provided three most likely OBMCs. Only three IHC stains need to be used to confirm the final origin. (c) The workflow of RMIL.

Fig. 2

The influence of the number of patches in a bag. In order to realize end-to-end training, we randomly selected a fixed number of patches from every bag for model training. To obtain the optimum number, a series of numbers of patches were investigated with associated 95% confidence intervals. n = 2720 ROI.

Fig. 3

Attention heatmaps. (a) Attention heatmaps of the WSIs. (b) Attention heatmaps of the ROI. These ROI can be found in the black marks of their corresponding left WSIs. (c) Metastatic carcinomas from the regions with high attention scores. (a)–(c) the OBMCs from top to bottom are lung, prostate, liver, intestine, kidney, stomach, thyroid and breast.

Fig. 4

The influence of sex on the performance of the model. The top figure demonstrates the AUROCs (95% confidence intervals) of every primary site with or without the sex as an input of the model. The lower row shows the ROC curves of each primary site with or without sex as an input to the model. n = 2720 ROI.

Fig. 5

The performance of the model. (a) The ROI-level confusion matrix of predictions of the model. Given to the imbalance in class distribution, the value of each row in the confusion matrix was divided by the total number of this category. (b) The ROI-level precision of each primary site. (c), The ROI-level recall of each primary site. (a)–(c) n = 2720 ROI. (d) The ROI-level top-k accuracies for the predictions of the OBMCs on the test set (n = 2720 ROI) and the external set (n = 383 ROI). (e) The case-level top-k accuracies for the predictions of the OBMCs on the test set (n = 136 cases) and the external set (n = 43 cases). (f) Micro averaged one-versus-rest ROC curves for the classification of the OBMC, evaluated on the test set (n = 2720 ROI) and the external test set (n = 383 ROI). The micro averaged AUROC was 99.57% (95% CI: 99.54%, 99.60%) on the test and 95.19% (95% CI: 94.81%, 95.57%) on the external test. (g) We calculated the consistency between the predictions of the model and the results indicated by IHC tests on 145 unknown cases (IHC). For the left 33 cases which could not be indicated by IHC, we assessed the agreement between the predictions of the model and the judgements made by pathologists according to the morphologies of H&E slides (DOC). The metrics of agreement include Cohen Kappa score (K), top1 agreement (agr1), and top3 agreement (agr3). (h) The fractions of samples (y axis) that were correctly classified at or above a certain confidence threshold. Due to the limited cases, seven primary sites were included in this analysis (n = 142 cases). (b)–(e), (g) Error bars indicate 95% confidence intervals.

Fig. 6

Top3 Predictions. (a) The fractions of samples (y axis) that were correctly predicted in top3 results at or above a certain confidence threshold. Due to the limited cases (n = 142 cases), seven primary sites were included in this analysis. (b) The frequency of each primary site occurred in tops3 predictions. The test set contained 136 cases and the unknown-test set contained 142 cases. Error bars indicate 95% confidence intervals. (c) The frequency of the combinations of three origins occurred in top3 predictions on the test set (n = 136 cases). (d) The frequency of the combinations of three origins occurred in top3 predictions on the unknown set (n = 142 cases). (e) The frequency of combinations of three origins occurred in top3 predictions when the ground truths were lung, breast, and liver respectively (n = 278 cases). (c)–(e) The x-axis means the combination of three primary sites which was available in supplementary Table S4.

Fig. 7

The performance of the model to predict whether there are metastatic carcinomas in bone tissue. (a) The ROI-level confusion matrix of predictions of the model. The value of each row in the confusion matrix was divided by the total number of this category. (b) The ROI-level sensitivity, specificity, accuracy and f1-score of the model. The positive category refers to the ROI with metastatic carcinomas. Error bars indicate 95% confidence intervals. (c) The ROC curve with an AUROC of 99.90% (95% CI: 99.89%, 99.91%). (a)–(c), n = 2734 ROI.

Fig. 8

The performance of the model to classify adenocarcinoma, squamous, and neuroendocrine carcinoma. (a) The ROI-level confusion matrix of predictions of the model. The value of each row in the confusion matrix was divided by the total number of this category. (b) The ROI-level precision of each category. (c) The ROI-level recall of each category. (a)–(c) “ade” represents adenocarcinoma, “squ” represents squamous cell carcinoma, and “neu” represents neuroendocrine tumor. (b)–(c) Error bars indicate 95% confidence intervals. (d) The ROC curve with an AUROC of 98.83% (95% CI: 98.59%, 99.07%). (a)–(d) n = 1090 ROI.