Adaptive learning for relevance feedback: application to digital mammography

Abstract

Purpose: With the rapid growing volume of images in medical databases, development of efficient image retrieval systems to retrieve relevant or similar images to a query image has become an active research area. Despite many efforts to improve the performance of techniques for accurate image retrieval, its success in biomedicine thus far has been quite limited. This article presents an adaptive content-based image retrieval (CBIR) system for improving the performance of image retrieval in mammographic databases.

Methods: In this work, the authors propose a new relevance feedback approach based on incremental learning with support vector machine (SVM) regression. Also, the authors present a new local perturbation method to further improve the performance of the proposed relevance feedback system. The approaches enable efficient online learning by adapting the current trained model to changes prompted by the user's relevance feedback, avoiding the burden of retraining the CBIR system. To demonstrate the proposed image retrieval system, the authors used two mammogram data sets: A set of 76 mammograms scored based on geometrical similarity and a larger set of 200 mammograms scored by expert radiologists based on pathological findings.

Results: The experimental results show that the proposed relevance feedback strategy improves the retrieval precision for both data sets while achieving high efficiency compared to offline SVM. For the data set of 200 mammograms, the authors obtained an average precision of 0.48 and an area under the precision-recall curve of 0.79. In addition, using the same database, the authors achieved a high pathology matching rate greater than 80% between the query and the top retrieved images after relevance feedback.

Conclusions: Using mammographic databases, the results demonstrate that the proposed approach is more accurate than the model without using relevance feedback not only in image retrieval but also in pathology matching while maintaining its effectiveness for online relevance feedback applications.

Figure 1

The proposed image retrieval framework with relevance feedback.

Figure 2

Illustration of ε-insensitive support vector machine (ε-SVM) for regression. The support vectors are indicated by filled squares.

Figure 3

Examples of mammogram regions containing clustered microcalcifications (indicated by circles).

Figure 4

MDS plot of scores of six observers. No. 7 indicates the average of the six observers; No. 8 is a random observer.

Figure 5

Precision-recall curves using relevance feedback with 76 mammogram images. RFB stands for relevance feedback.

Figure 6

Precision-recall curves using relevance feedback with different parameter values.

Figure 7

AUPRC histograms using relevance feedback with different parameter values. The “w∕o LP” means “without local perturbation.”

Figure 8

Histograms of AUPRC for 20 000 bootstrap experiments.

Figure 9

Precision-recall curves showing adaptation to individual observer’s scores.

Figure 10

AUPRC histograms using ε_n=0.5 and each observer’s scores instead of averaged scores.

Figure 11

An example to show the effectiveness of relevance feedback. Given a query image, the top three images (top row) retrieved using relevance feedback and retrieval results (bottom row) by the offline trained SVM are shown.

Figure 12

Average matching fraction for the top k retrieved images (k=1, 2, 3, 4, and 5) that actually match the disease condition of the query.