Computerized assessment of breast lesion malignancy using DCE-MRI robustness study on two independent clinical datasets from two manufacturers

Abstract

Rationale and objectives: To conduct a preclinical evaluation of the robustness of our computerized system for breast lesion characterization on two breast magnetic resonance imaging (MRI) databases that were acquired using scanners from two different manufacturers.

Materials and methods: Two clinical breast MRI databases were acquired from a Siemens scanner and a GE scanner, which shared similar imaging protocols and retrospectively collected under an institutional review board-approved protocol. In our computerized analysis system, after a breast lesion is identified by the radiologist, the computer performs automatic lesion segmentation and feature extraction and outputs an estimated probability of malignancy. We used a Bayesian neural network with automatic relevance determination for joint feature selection and classification. To evaluate the robustness of our classification system, we first used Database 1 for feature selection and classifier training, and Database 2 to test the trained classifier. Then, we exchanged the two datasets and repeated the process. Area under the receiver operating characteristic curve (AUC) was used as a performance figure of merit in the task of distinguishing between malignant and benign lesions.

Results: We obtained an AUC of 0.85 (approximate 95% confidence interval [CI] 0.79-0.91) for (a) feature selection and classifier training using Database 1 and testing on Database 2; and an AUC of 0.90 (approximate 95% CI 0.84-0.96) for (b) feature selection and classifier training using Database 2 and testing on Database 1. We failed to observe statistical significance for the difference AUC of 0.05 between the two database conditions (P = .24; 95% confidence interval -0.03, 0.1).

Conclusion: These results demonstrate the robustness of our computerized classification system in the task of distinguishing between malignant and benign breast lesions on dynamic contrast-enhanced (DCE) MRI images from two manufacturers. Our study showed the feasibility of developing a computerized classification system that is robust across different scanners.

Figure 1

Distribution of patients over their primary lesion pathology for (a) Database 1 and (b) Database 2.

Figure 1

Distribution of patients over their primary lesion pathology for (a) Database 1 and (b) Database 2.

Figure 2

Schematic diagram of our computerized analysis and interpretation scheme.

Figure 3

The relative importance of computerized image features to the classification task as assessed by the Bayesian neural network model with automatic relevance determination priors using (a) Database 1; and (b) Database 2. The results for the top-ranked 14 features are plotted. See Table 2 for the definition of the features.

Figure 3

The relative importance of computerized image features to the classification task as assessed by the Bayesian neural network model with automatic relevance determination priors using (a) Database 1; and (b) Database 2. The results for the top-ranked 14 features are plotted. See Table 2 for the definition of the features.

Figure 4

The classification performance of the computerized system as a function of the number of top-ranked features.

Figure 5

ROC curves and statistical comparison of the two “independent assessment” conditions: (a) feature selection and classifier training using DB1 and testing the classifier on DB2 (dash); (b) feature selection and classifier training using DB2 and testing the classifier on DB1 (solid).