Retrieval of brain MRI with tumor using contrastive loss based similarity on GoogLeNet encodings

Abstract

An image retrieval system for medical images aids in disease diagnosis by providing similar images from the medical database to a query image. In this article, a content-based medical image retrieval (CBMIR) system is proposed for the retrieval of magnetic resonance imaging (MRI) images of the brain with three types of tumors:- meningioma, glioma and pituitary tumors. The proposed system uses GoogLeNet encodings via transfer learning as image features. A Siamese Neural Network (SNN), is designed, to represent the GoogLeNet encodings in a two-dimensional (2-D) feature space. The SNN is trained using the contrastive loss function to learn the class-specific image features. The similarity, between a query image and the database images, is measured by the Euclidean metric in the lower dimensional feature space. The proposed method achieves state-of-the-art performance for the retrieval of MRI images with brain tumors. The evaluation is done on the openly available Figshare dataset and the performance metrics used are mean average precision (mAP) and precision@10.

Keywords: Brain tumor; Medical image retrieval; Metric learning; Siamese networks; Transfer learning.