Single-cell AI-based detection and prognostic and predictive value of DNA mismatch repair deficiency in colorectal cancer

Abstract

Testing for DNA mismatch repair deficiency (MMRd) is recommended for all colorectal cancers (CRCs). Automating this would enable precision medicine, particularly if providing information on etiology not captured by deep learning (DL) methods. We present AIMMeR, an AI-based method for determination of mismatch repair (MMR) protein expression at a single-cell level in routine pathology samples. AIMMeR shows an area under the receiver-operator curve (AUROC) of 0.98, and specificity of ≥75% at 98% sensitivity against pathologist ground truth in stage II/III in two trial cohorts, with positive predictive value of ≥98% for the commonest pattern of somatic MMRd. Lower agreement with microsatellite instability (MSI) testing (AUROC 0.86) reflects discordance between MMR and MSI PCR rather than AIMMeR misclassification. Analysis of the SCOT trial confirms MMRd prognostic value in oxaliplatin-treated patients; while MMRd does not predict differential benefit of chemotherapy duration, it correlates with difference in relapse by regimen (P_Interaction = 0.04). AIMMeR may help reduce pathologist workload and streamline diagnostics in CRC.

Keywords: AI; colorectal cancer; digital pathology; microsatellite instability; mismatch repair deficiency.

Graphical abstract

Figure 1

Flow (CONSORT) diagram of cases included in this study CONSORT diagrams for (A) SCOT and (B) QUASAR2 trial cases. ∗Additional QC included exclusion of tissue microarray (TMA) cores with <100 epithelial cells per core, cores with negative staining (<20 positive cells) for all four MMR proteins in both epithelium and stroma, and cases uninformative for all four MMR proteins. ^†Details of cases subject to pathologist review are provided in the STAR Methods.

Figure 2

Schematic of study methodology Immunohistochemistry (IHC) for DNA mismatch repair (MMR) proteins MLH1, PMS2, MSH2, and MSH6 was performed on tissue microarray (TMA) cores from 2,352 SCOT and 1,195 QUASAR2 tumors. Scanned IHC images were classified according to type and MMR staining by AIMMeR (see STAR Methods and Results). Following QC, the percentage of epithelial cells positive for expression of individual MMR proteins was calculated for each core, and summary metrics were calculated for each case (Step 3). AIMMeR performance was then benchmarked against a ground-truth set defined by pathologist review of 685 SCOT tumors and 504 QUASAR2 tumors and against MSI status in QUASAR2 cases. Further details are provided in the STAR Methods and the main text. Min, minimum; Max, maximum; AUROC, area under the receiver-operator curve; IRR, inter-rater reliability.

Figure 3

AIMMeR identifies mismatch repair deficiency with high accuracy in SCOT cases (A) Receiver-operator curve (ROC) for AIMMeR classification of MMRd against consensus pathologist ground truth in 685 SCOT cases. 95% confidence intervals were obtained by bootstrap (1,000 resamples). (B) Sensitivity and specificity according to AIMMeR^MIN threshold. Thresholds with sensitivity of 95% (Sens 95), 98% (Sens 98), and maximum Youden index (sensitivity plus specificity) are shown. (C) Confusion matrices showing AIMMeR classification vs. consensus pathologist ground truth at AIMMeR^MIN thresholds shown in (B). (D) Positive predictive value of AIMMeR classification of MMR protein loss for the identical combination of protein loss and for any type of MMR loss. MMR status and protein loss were classified using threshold with maximal Youden index. (E) Reasons for discordance between AI and consensus pathologist calls identified at discrepancy review for both MMR status and protein status (∗limited to cases for which MMR status was concordant). Reasons for discordance between pathologists are provided for comparison. Additional detail is provided in Tables S2–S5 and Figures S2–S7; illustrative cases are shown in Figure S8.

Figure 4

AIMMeR identifies mismatch repair deficiency with high accuracy in QUASAR2 cases (A) Receiver-operator curve (ROC) for AIMMeR classification of MMRd against consensus pathologist ground truth in 381 QUASAR2 cases. 95% confidence intervals were obtained by bootstrap (1,000 resamples). (B) Sensitivity and specificity against consensus pathologist ground truth according to AIMMeR^MIN threshold. Thresholds with sensitivity of 95% (Sens 95), 98% (Sens 98), and maximum Youden index in the SCOT cohort, and the corresponding thresholds determined within the QUASAR2 cohort are shown. (C) ROC for AIMMeR classification of MMR status against MSI PCR ground truth. Main panel shows curve in the 381 cases shown in (A), while inset shows that from analysis of all 965 cases with available MSI status. (D) Confusion matrices showing AIMMeR MMR classification against consensus pathologist ground truth and against MSI PCR ground truth at AIMMeR^MIN thresholds defined in the SCOT cohort and QUASAR2 cohort shown in (B). (E) Positive predictive value of AIMMeR classification of MMR protein loss for the identical combination of protein loss and for any type of MMR loss. MMR status and protein loss were classified using threshold with maximal Youden index in QUASAR2 cohort. (F) Discordance between AIMMeR MMR classification, MSI testing, and consensus pathology MMR status. Confusion matrices of AIMMeR MMR classification against MSI PCR are shown for all 965 cases and the 381 with consensus pathologist review; pie charts show MMR status from consensus pathologist review for the four groups. Pathologist review revealed misclassification of one case in the MMRp, MSI subgroup was due to signal from non-malignant epithelial cells present in the section.

Figure 5

Prognostic and predictive value of combined AIMMeR and pathologist classification of MMRd in the SCOT trial cohort (A) Kaplan-Meier plot showing recurrence-free interval (RFI) for patients according to MMR status. (B) Forest plot showing hazard ratios (HR) with 95% confidence intervals (95% CI) for RFI according to MMR status within clinical and pathological subgroups by multivariable analysis∗. (C) Kaplan-Meier plot showing RFI according tumor sidedness and MMR status. (D) Kaplan-Meier plot showing RFI according to duration of chemotherapy and MMR status. (E) Kaplan-Meier plot showing RFI according to chemotherapy regimen and MMR status. p values in (A, C, D, and E) were obtained by log rank test. Hazard ratios (HRs) in (B) were obtained by multivariable Cox proportional hazards models including prespecified covariables of age, gender, pN stage, pT stage, sidedness, treatment regimen, and treatment duration. Additional detail is provided in Tables S6 and S7; relationship between MMR status and tumor lymphocytic infiltrate and stroma is shown in Figure S9.

Figure 6

Potential role of AIMMeR in classification of MMR status in CRC Plots show outcomes of application of DL methods to H&E-stained slides and of AIMMeR to IHC-stained slides in 100 colorectal cancers with MMRd prevalence of 10%. Current performance of DL methods correctly identifies 49 cases as MMRp, though current false-negative rate of 5%^, (using within-cohort thresholds) means 0.5 MMRd case is misclassified as MMRp. 50 cases require further testing such as IHC with all requiring pathologist review. AIMMeR requires immunostaining of all cases but correctly identifies nearly 80% of cases as MMRp with false-negative rate of ≤2% (using within-cohort threshold) and also identifies 7 MMRd cases with combined MLH1-PMS2 loss with PPV of 98%, potentially allowing reflex testing for BRAF mutation or MLH1 promoter methylation. Pathologist review of cases with possible MMRd is required in 13 cases.