Robust Estimation of Breast Cancer Incidence Risk in Presence of Incomplete or Inaccurate Information

Abstract

Purpose: To evaluate the robustness of multiple machine learning classifiers for breast cancer risk estimation in the presence of incomplete or inaccurate information.

Data and methods: Open data for this study was obtained from the BCSC Data Resource (http://breastscreening.cancer.gov/). We conducted two ablation-type experiments to compare the robustness of different classifiers where we randomly switched known information to missing with a missing probability of pm in one experiment, and randomly corrupted the existing information with a probability of pc in another experiment. We considered three prominent machine-learning classifiers such as Logistic regression (LR), Random Forests (RF) and a custom Neural Network (NN) architecture and compared their degradation of discrimination performance as a function of increasing probability of missing or inaccurate data.

Results: LR, RF and custom NN resulted in an Area Under Curve (AUC) of 0.645, 0.643 and 0.649, respectively, on a test set with 500,000 total observations. When we manipulated the data by varying probabilities pm and pc from 0 to 1, NN resulted in better performance in terms of AUC compared to RF and LR as long as less than half the data was missing/inaccurate (that is, for values of pm < 0.5 and pc < 0.5). However, for missing (pm) or corruption (pc) probabilities above 0.5, LR gave similar performance as the custom NN. RF resulted in overall poorer performance when the data had additional missing or incorrect entries.

Conclusion: In cases where the input information is missing or inaccurate, our experiments show that the proposed custom NN provides reliable risk estimates in medical datasets like BCSC. These results are particularly important in health care applications where not every attribute of the individual participant might be available.<br />.

Keywords: Artificial Neural Networks; Breast cancer risk; Machine Learning; inaccurate data; missing values.

Figure 1

Shows Variation of Mean AUCs with Missing Probability, p_m, for All the Three Classifiers on (a) validation set (b) test set

Figure 2

Shows Variation of Mean AUCs with Probability, p_c, for All the Three Classifiers on (a) validation set (b) test set

Figure 3

Illustration of Two Hypothetical Examples (Female A and Female B, as discussed in the text). For each example, the predicted risk of breast cancer within 1 year of screening is superimposed over the observed proportion of breast cancer diagnoses in the BCSC population as a function of age bracket