Assessment of performance and reliability of computer-aided detection scheme using content-based image retrieval approach and limited reference database

Abstract

Content-based image retrieval approach was used in our computer-aided detection (CAD) schemes for breast cancer detection with mammography. In this study, we assessed CAD performance and reliability using a reference database including 1500 positive (breast mass) regions of interest (ROIs) and 1500 normal ROIs. To test the relationship between CAD performance and the similarity level between the queried ROI and the retrieved ROIs, we applied a set of similarity thresholds to the retrieved similar ROIs selected by the CAD schemes for all queried suspicious regions, and used only the ROIs that were above the threshold for assessing CAD performance at each threshold level. Using the leave-one-out testing method, we computed areas under receiver operating characteristic (ROC) curves (A(Z)) to assess CAD performance. The experimental results showed that as threshold increase, (1) less true positive ROIs can be referenced in the database than normal ROIs and (2) the A(Z) value was monotonically increased from 0.854 ± 0.004 to 0.932 ± 0.016. This study suggests that (1) in order to more accurately detect and diagnose subtle masses, a large and diverse database is required, and (2) assessing the reliability of the decision scores based on the similarity measurement is important in application of the CBIR-based CAD schemes when the limited database is used.

Fig. 1

Comparison of histograms of normalized similarity scores between 1500 positive ROIs that depict verified masses (TP ROIs) and 1,500 negative ROIs that depict CAD-cued false-positive masses (FP ROIs). It shows that diversity level of TP ROIs is larger than FP ROIs.

Fig. 2

The change of CBIR scheme performance (A_Z values) using the decision index D₁(q) as the increase of threshold values on the similarity scores of the queried ROIs. It shows a trend of monotonic increase of the A_Z values as the increase of thresholds on the normalized similarity scores using decision index D₁(q).

Fig. 3

The change of CBIR scheme performance (A_Z values) using the decision index D₂(q) as the increase of threshold values on the similarity scores of the queried ROIs. It shows a trend of monotonic increase of the A_Z values as the increase of thresholds on the normalized similarity scores using decision index D₂(q).