Improving performance of computer-aided detection scheme by combining results from two machine learning classifiers

Abstract

Rationale and objectives: Global data-based and local instance-based machine-learning methods and classifiers have been widely used to optimize computer-aided detection and diagnosis (CAD) schemes to classify between true-positive and false-positive detections. In this study, the correlation between these two types of classifiers was investigated using a new independent testing data set, and the potential improvement of a CAD scheme's performance by combining the results of the two classifiers in detecting breast masses was assessed.

Materials and methods: The CAD scheme first used image filtering and a multilayer topographic region growth algorithm to detect and segment suspicious mass regions. The scheme then used an image feature-based classifier to classify these regions into true-positive and false-positive regions. Two classifiers were used in this study. One was a global data-based machine-learning classifier, an artificial neural network (ANN), and the other was a local instance-based machine-learning classifier, a k-nearest neighbor (KNN) algorithm. An independent image database including 400 mammographic examinations was used in this study. Of these, 200 were cancer cases and 200 were negative cases. The preoptimized CAD scheme was applied twice to the database using the two different classifiers. The correlation between the two sets of classification results was analyzed. Three sets of CAD performance results using the ANN, KNN, and average detection scores from both classifiers were assessed and compared using the free-response receiver-operating characteristic method.

Results: The results showed that the ANN achieved higher performance than the KNN algorithm, with a normalized area under the performance curve (AUC) of 0.891 versus 0.845. The correlation coefficients between the detection scores generated by the two classifiers were 0.436 and 0.161 for the true-positive and false-positive detections, respectively. The average detection scores of the two classifiers improved CAD performance and reliability by increasing the AUC to 0.912 and reducing the standard error of the estimated AUC by 14.4%. The detection sensitivity was also increased from 75.8% (ANN) and 65.9% (KNN) to 80.3% at a false-positive detection rate of 0.3 per image.

Conclusions: This study demonstrates that two global data-based and local data-based machine-learning classifiers (ANN and KNN) generated low correlated detection results and that combining the detection scores of these two classifiers significantly improved overall CAD performance (P < .01) and reduced standard error in CAD performance assessment.

Figure 1

Two FROC curves of the CAD scheme using ANN classifier.

Figure 2

Distribution between ANN and KNN generated detection scores among 347 detected true-positive mass regions.

Figure 3

Three case-based FROC curves using the ANN, KNN, and average scores.

Figure 4

Three region-based FROC curves using the ANN, KNN, and average scores.

Figure 5

Histogram of KNN-generated scores for all suspicious mass regions detected and cued by the CAD scheme using the ANN classifier.

Figure 6

Histogram of ANN-generated scores for all suspicious mass regions detected and cued by the CAD scheme using the KNN classifier.

Figure 7

Comparison of CAD performance after using the second scores to replace the original scores of the detected mass regions. The curve marked with “Δ ” indicates CAD performance after using KNN generated scores to replace the original ANN generated scores; while curve marked with “O” indicates CAD performance after using ANN generated scores to replace original KNN generated scores. The two smooth curves (without marks) represent sections of the original FROC curves copied from Figure 4 in which the solid curve is generated by CAD using the ANN and the dashed curve is generated by CAD using the KNN.