Clinical-Grade Detection of Microsatellite Instability in Colorectal Tumors by Deep Learning

Abstract

Background & aims: Microsatellite instability (MSI) and mismatch-repair deficiency (dMMR) in colorectal tumors are used to select treatment for patients. Deep learning can detect MSI and dMMR in tumor samples on routine histology slides faster and less expensively than molecular assays. However, clinical application of this technology requires high performance and multisite validation, which have not yet been performed.

Methods: We collected H&E-stained slides and findings from molecular analyses for MSI and dMMR from 8836 colorectal tumors (of all stages) included in the MSIDETECT consortium study, from Germany, the Netherlands, the United Kingdom, and the United States. Specimens with dMMR were identified by immunohistochemistry analyses of tissue microarrays for loss of MLH1, MSH2, MSH6, and/or PMS2. Specimens with MSI were identified by genetic analyses. We trained a deep-learning detector to identify samples with MSI from these slides; performance was assessed by cross-validation (N = 6406 specimens) and validated in an external cohort (n = 771 specimens). Prespecified endpoints were area under the receiver operating characteristic (AUROC) curve and area under the precision-recall curve (AUPRC).

Results: The deep-learning detector identified specimens with dMMR or MSI with a mean AUROC curve of 0.92 (lower bound, 0.91; upper bound, 0.93) and an AUPRC of 0.63 (range, 0.59-0.65), or 67% specificity and 95% sensitivity, in the cross-validation development cohort. In the validation cohort, the classifier identified samples with dMMR with an AUROC of 0.95 (range, 0.92-0.96) without image preprocessing and an AUROC of 0.96 (range, 0.93-0.98) after color normalization.

Conclusions: We developed a deep-learning system that detects colorectal cancer specimens with dMMR or MSI using H&E-stained slides; it detected tissues with dMMR with an AUROC of 0.96 in a large, international validation cohort. This system might be used for high-throughput, low-cost evaluation of colorectal tissue specimens.

Keywords: Lynch syndrome; biomarker; cancer immunotherapy; mutation.

Figure 1:. Deep learning workflow and learning curves.

(A) Histological routine images were collected from four large patient cohorts. All slides were manually quality-checked to ensure presence of tumor tissue (circled in black). (B) Tumor regions were automatically tessellated and a library of millions of non-normalized (native) image tiles was created. (C) The deep learning system was trained on increasing numbers of patients and evaluated on a random subset (n=906 patients). Performance initially increased by adding more patients to the training set, but reached a plateau at approximately 5000 patients. (D) Cross-validated experiment on the full international cohort (comprising TCGA, DACHS, QUASAR and NLCS). Receiver operating characteristic (ROC) with true positive rate (TPR) shown against false positive rate (FPR), area under the ROC curve (AUROC) is shown on top. (E) ROC curve (left) and precision-recall-curve (right) of the same classifier applied to a large external dataset. High test performance was maintained in this dataset and thus, the classifier generalized well beyond the training cohorts. Black line = average performance, shaded area = bootstrapped confidence interval, red line = random model (no skill).

Figure 2:. Cross-validated subgroup analysis for detection of MSI and dMMR in the international cohort (n=6406 patients).

AUC = area under the receiver operating curve as shown in the image, TPR = true positive rate, FPR = false positive rate, WT = wild type, MUT = mutated.

Figure 3:. Prediction map in the external test cohort YCR-BCIP-RESECT.

(A-C) Representative images from the YCR-BCIP-RESECT test cohort labeled with immunohistochemically defined mismatch repair (MMR) status. (D-F) Corresponding deep learning prediction maps. The edge length of each prediction tile is 256 μm. (G-I) Higher magnification of regions highlighted in a-e. True MSI or dMMR patients were strongly and homogeneously predicted to be MSI or dMMR (such as the patient shown in A). True MSS or pMMR patients were overall predicted to be MSS or pMMR (such as the patients in B and C), but a pronounced heterogeneity was observed in necrotic areas, poorly differentiated areas and immune-infiltrated tumor areas at the invasive edge.

Figure 4:. Effect of color normalization on classifier performance.

(A) A representative set of tiles from the MSIDETECT study. (B) The same tiles after color normalization. (C) Classifier performance on an external test set (YCR-BCIP-RESECT, n=771 patients) improves after color-normalizing training and test sets. Experiment #4N is with color normalization, experiment #4 is without color normalization. AUROC: area under the receiver operating curve, TPR: true positive rate, FPR: false positive rate.