Design and development of a content-based medical image retrieval system for spine vertebrae irregularity

Abstract

Background: Content-based medical image retrieval (CBMIR) system enables medical practitioners to perform fast diagnosis through quantitative assessment of the visual information of various modalities.

Methods: In this paper, a more robust CBMIR system that deals with both cervical and lumbar vertebrae irregularity is afforded. It comprises three main phases, namely modelling, indexing and retrieval of the vertebrae image. The main tasks in the modelling phase are to improve and enhance the visibility of the x-ray image for better segmentation results using active shape model (ASM). The segmented vertebral fractures are then characterized in the indexing phase using region-based fracture characterization (RB-FC) and contour-based fracture characterization (CB-FC). Upon a query, the characterized features are compared to the query image. Effectiveness of the retrieval phase is determined by its retrieval, thus, we propose an integration of the predictor model based cross validation neural network (PMCVNN) and similarity matching (SM) in this stage. The PMCVNN task is to identify the correct vertebral irregularity class through classification allowing the SM process to be more efficient. Retrieval performance between the proposed and the standard retrieval architectures are then compared using retrieval precision (Pr@M) and average group score (AGS) measures.

Results: Experimental results show that the new integrated retrieval architecture performs better than those of the standard CBMIR architecture with retrieval results of cervical (AGS > 87%) and lumbar (AGS > 82%) datasets.

Conclusions: The proposed CBMIR architecture shows encouraging results with high Pr@M accuracy. As a result, images from the same visualization class are returned for further used by the medical personnel.

Figure 1

Examples of AOs classes of the cervical x-ray images. Samples of Macnab’s classification and their osteophyte severity grading in the cervical vertebrae x-ray images, (a) normal (b) moderate traction (c) severe claw and (d) slight claw-traction.

Figure 2

Examples of AOs classes of the lumbar x-ray images. The four classes of AOs in lumbar vertebrae (a) normal (b) L-LHC (c) U-LHC and (d) B-LHC.

Figure 3

Main phases involved in developing the proposed CBMIR for spine vertebrae irregularity assessment. The region localization (RL) of the cervical and lumbar x-ray images are presented within the red boxes, RL-Cervical (C3-C6) and RL-Lumbar (L1-L5). From left-to-right, the blue coloured diagram signifies the MP steps. The diagram in yellow represents the IP and green represents the PA for the retrieval phase.

Figure 4

Shape boundary representation of the 9-APR and the third order B-spline schemes. The 9-anatomical points are marked by expert radiologist, Figure 4 (a) represents the marked made on (a1) synthetic and (a2) authentic vertebra image. Figure 4 (b) depicts the generated shape boundary using the 3rd order B-SR in which (b1) represents the 9-APR points, (b2) depicts the 27 equally spaced point over the boundaries measured from (b1) and (b3) depicts the resulting boundary of the vertebra using B-SR.

Figure 5

Summary of the ASM training process using the 9-APR.

Figure 6

The Main interface screen shot of the developed CBMIR prototype. The Query Menu, as labelled on the left by the yellow boxes executes tasks of the query platform. The Enrolment Menu labelled using green colour box was designed to execute tasks of the enrolment platform. The Screening and Display menu as labelled comprises an upper and lower sections that serve to display both visual and statistical retrieval results, respectively. The upper panel on the left/right provides additional functions as zooming, refresh, search, close, etc.

Figure 7

Presentation of query platform process via QRY-M. The push-button which is in black is meant for the execution to load the query image (either cervical or lumbar) with the specified fracture AOs represented in two different models 9-APR (M1) and B-SR (M2) as follows: (a, c) cervical AOs database with M1 and M2, respectively and (b, d) lumbar AOs database with M1 and M2, respectively.

Figure 8

Presentation of enrolment platform process via ENR-M. Following a query image selected, search (push-button) for the task to execute the retrieval process and similarity matching (sorting based minimum distance) utilizing the selected computation approach as follows: (i) retrieval via region-based (GW, GLCM, RT and OH), (ii) retrieval via contour-based FDs produced from GSP, CC and CC).

Figure 9

Sample retrieval results of AOs query using 9-APR via SD-M for cervical and lumbar. Top of the panel of the screening and display menu visualized the initial top 8 images from the similarity ranking measure (as indicated in the right lower Table) as follows: top retrieved consequence of sample query, (a) cervical AOs using 9-APR (b) cervical AOs using b-SR (c) lumbar AOs using 9-APR and (d) lumbar AOs using B-SR. A more retrieval images can be visualized retrieval results by clicking the "slider"? button.

Figure 10

ROC plots of the PMCVNN classification results of the DB1, DB2, DB3 and DB4 datasets. Average classification results of the (a) DB1 (b) DB2 (c) DB3 and (d) DB4.

Figure 11

Retrieval results of the Standard Architecture (SA) and Proposed (PA) using GW features of a selected query image. Sample results comprising M =20 retrieved images from DB1 (cervical & 9-APR) dataset of a query image for a) SA b) PA and another set of results from the same query image using DB2 (cervical & B-SR) dataset for c) SA and d) PA.

Figure 12

Retrieval results of the Standard Architecture (SA) and Proposed (PA) using GW features of a selected query image. Sample results comprising M =20 retrieved images from DB1 (lumbar & 9-APR) dataset of a query image for a) SA b) PA and another set of results from the same query image using DB2 (lumbar & B-SR) dataset for c) SA and d) PA.