Variability Among Breast Cancer Risk Classification Models When Applied at the Level of the Individual Woman

Abstract

Background: Breast cancer risk models guide screening and chemoprevention decisions, but the extent and effect of variability among models, particularly at the individual level, is uncertain.

Objective: To quantify the accuracy and disagreement between commonly used risk models in categorizing individual women as average vs. high risk for developing invasive breast cancer.

Design: Comparison of three risk prediction models: Breast Cancer Risk Assessment Tool (BCRAT), Breast Cancer Surveillance Consortium (BCSC) model, and International Breast Intervention Study (IBIS) model.

Subjects: Women 40 to 74 years of age presenting for screening mammography at a multisite health system between 2011 and 2015, with 5-year follow-up for cancer outcome.

Main measures: Comparison of model discrimination and calibration at the population level and inter-model agreement for 5-year breast cancer risk at the individual level using two cutoffs (≥ 1.67% and ≥ 3.0%).

Key results: A total of 31,115 women were included. When using the ≥ 1.67% threshold, more than 21% of women were classified as high risk for developing breast cancer in the next 5 years by one model, but average risk by another model. When using the ≥ 3.0% threshold, more than 5% of women had disagreements in risk severity between models. Almost half of the women (46.6%) were classified as high risk by at least one of the three models (e.g., if all three models were applied) for the threshold of ≥ 1.67%, and 11.1% were classified as high risk for ≥ 3.0%. All three models had similar accuracy at the population level.

Conclusions: Breast cancer risk estimates for individual women vary substantially, depending on which risk assessment model is used. The choice of cutoff used to define high risk can lead to adverse effects for screening, preventive care, and quality of life for misidentified individuals. Clinicians need to be aware of the high false-positive and false-negative rates and variation between models when talking with patients.

Keywords: breast cancer; chemoprevention; mammography; risk models; screening.

Figure 1

Pairwise comparisons of an individual woman’s risk of being told she is at “high risk” of a breast cancer diagnosis within 5 years when using the three commonly used risk models (N = 31,115). * Panel a. Dashed lines in the scatter plots indicate a 1.67% cutoff for binary classification of high- vs. average-risk. Panel b. Dashed lines in the scatter plots indicate a 3.0% cutoff for binary classification of high- vs. average-risk. *The sum of the percentages may not add up to 100% due to rounding.

Figure 2

Agreement of breast cancer risk model results for individual women identified as high risk within 5 years on at least one of the breast cancer risk prediction models (N = 31,115). The Venn diagram at the bottom presents data for all women identified as high risk by at least one model, using shaded overlapping regions to represent the Boolean operation. The size of each Venn diagram circle corresponds to the number of women identified as high risk by each risk model.

Figure 3

Receiver operating characteristic curve (ROC curve) illustrating the similar population-level performance of the three models for 5-year breast cancer risk prediction (N = 11,589 with 5 years of follow-up data).