Comparison of Prediction Models for Lynch Syndrome Among Individuals With Colorectal Cancer

Abstract

Background: Recent guidelines recommend the Lynch Syndrome prediction models MMRPredict, MMRPro, and PREMM1,2,6 for the identification of MMR gene mutation carriers. We compared the predictive performance and clinical usefulness of these prediction models to identify mutation carriers.

Methods: Pedigree data from CRC patients in 11 North American, European, and Australian cohorts (6 clinic- and 5 population-based sites) were used to calculate predicted probabilities of pathogenic MLH1, MSH2, or MSH6 gene mutations by each model and gene-specific predictions by MMRPro and PREMM1,2,6. We examined discrimination with area under the receiver operating characteristic curve (AUC), calibration with observed to expected (O/E) ratio, and clinical usefulness using decision curve analysis to select patients for further evaluation. All statistical tests were two-sided.

Results: Mutations were detected in 539 of 2304 (23%) individuals from the clinic-based cohorts (237 MLH1, 251 MSH2, 51 MSH6) and 150 of 3451 (4.4%) individuals from the population-based cohorts (47 MLH1, 71 MSH2, 32 MSH6). Discrimination was similar for clinic- and population-based cohorts: AUCs of 0.76 vs 0.77 for MMRPredict, 0.82 vs 0.85 for MMRPro, and 0.85 vs 0.88 for PREMM1,2,6. For clinic- and population-based cohorts, O/E deviated from 1 for MMRPredict (0.38 and 0.31, respectively) and MMRPro (0.62 and 0.36) but were more satisfactory for PREMM1,2,6 (1.0 and 0.70). MMRPro or PREMM1,2,6 predictions were clinically useful at thresholds of 5% or greater and in particular at greater than 15%.

Conclusions: MMRPro and PREMM1,2,6 can well be used to select CRC patients from genetics clinics or population-based settings for tumor and/or germline testing at a 5% or higher risk. If no MMR deficiency is detected and risk exceeds 15%, we suggest considering additional genetic etiologies for the cause of cancer in the family.

Figure 1.

Receiver operating characteristic curves for MMRPredict, MMRPro, and PREMM_1,2,6 models. A) Receiver operating characteristic curves for discriminating mismatch repair mutation carriers from noncarriers with MMRPredict, MMRPro, and PREMM_1,2,6 in clinic-based cohorts. B) Receiver operating characteristic curves for discriminating mismatch repair mutation carriers from noncarriers with MMRPredict, MMRPro, and PREMM_1,2,6 in population-based cohorts. AUC = area under the curve.

Figure 2.

Calibration plots for MMRPredict, MMRPro, and PREMM_1,2,6 models. A-C) Clinic-based cohort: (A-C) display calibration plots for external validation of (A) MMRpredict, (B) MMRPro, and (C) PREMM_1,2,6 for predicting MMR mutations for individuals in clinic-based settings. D-F) Population-based cohort: (D-F) display calibration plots for external validation of (A) MMRpredict, (B) MMRPro, and (C) PREMM_1,2,6 for predicting MMR mutations for individuals in population-based settings. The x-axis represents predicted probabilities, the y-axis represents the observed proportion of MMR mutations, and the long dashed diagonal line represents the ideal model with perfect prediction. The short dashed line represents the relation between MMR mutations and model-based predictions (according to a loess smoother). The triangles represent observed frequencies by quintiles of predicted probability with corresponding 95% confidence limits (vertical lines). The distribution of predicted probabilities is displayed for individuals with and without a mutation in the lower portion of the figure. ROC = receiver operating characteristic curve.

Figure 3.

Net benefit analyses comparing MMRPredict, MMRPro, and PREMM_1,2,6 to identify mismatch repair mutation carriers at different risk thresholds. A and B) Display of the net benefit curves comparing the three prediction models among the clinic-based cohort. The y-axis measures net benefit, which is calculated by summing the benefits (true positives) and subtracting the harms (false positives), where the latter are weighted by a factor related to the relative harm of a missed mutation carrier compared with the harm of unnecessary genetic testing. A model is considered of clinical value if it has the highest net benefit compared with other models and simple strategies such as performing genetic testing in all patients (dashed black line) or no patients (horizontal black line) across the full range of threshold probabilities at which a patient would undergo genetic testing. For example, the net benefit of using PREMM_1,2,6 or MMRPro to selectively test for mutation carriers exceeds that of testing all at a risk of 5% or higher. A) The net benefit at the 10% threshold is 0.18 for PREMM_1,2,6 vs 0.15 for testing all and 0 for testing none among clinic-based cases. The net benefit of the PREMM_1,2,6 model over testing all is thus 0.03, which means that three individuals would be identified as mutation carriers for every 100 people assessed with PREMM_1,2,6 without an increase in the number of false-positive results. At the 10% threshold, this calculation assumes that we value a true-positive classification worth incurring up to nine false-positives (since 1:9, which is the odds corresponding to a probability of 10%). B) Display of the net benefit curves for the three models among the population-based cohort. The benefit at the 10% threshold is 0.02 for PREMM_1,2,6 vs 0 for testing all and 0 for testing none. This means that two individuals would be identified as mutation carriers for every 100 people assessed with PREMM_1,2,6 without an increase in the number of false positives.