Medical image databases: a content-based retrieval approach

Abstract

Information contained in medical images differs considerably from that residing in alphanumeric format. The difference can be attributed to four characteristics: (1) the semantics of medical knowledge extractable from images is imprecise; (2) image information contains form and spatial data, which are not expressible in conventional language; (3) a large part of image information is geometric; (4) diagnostic inferences derived from images rest on an incomplete, continuously evolving model of normality. This paper explores the differentiating characteristics of text versus images and their impact on design of a medical image database intended to allow content-based indexing and retrieval. One strategy for implementing medical image databases is presented, which employs object-oriented iconic queries, semantics by association with prototypes, and a generic schema.

Figure 1

A conceptual model of the content understanding—query completion—interaction space, plotting the location of text databases, commercial image browsing databases, and medical image databases. Note the scale designations from low to high. The lowest degrees of each property are located in the lower left hand corner and the highest lie in the farther right top corner. The evolving property of content-based medical image databases is depicted by the first location at its original implementation, point “A,” but evolving over time to point “B.”

Figure 2

Medical images created by diagnostic instruments can result in large digital collections. Microscopic histology images possess unique color signatures and cell textures that might be used as indexing methods for grouping slides that share similar staining techniques. Thus, a database indexing scheme could take into account color hue as an indexing feature. A query structure could therefore be devised that would make it possible for a user to retrieve images that share a common staining technique.

Figure 3

Ultrasound images of large organs (here the liver) appear to be dominated by textures and may lend themselves to indexing schemas that emphasize extracting a global property rather than local features.

Figure 4

Chest x-ray images are projections of many overlapping structures (for example lung tissues). Indexing procedures might be constructed to address textures rather than organ boundaries because of the difficulty of isolating organs without overlap.

Figure 5

A coronal MRI tomographic section of the chest and heart. Tomographic images readily permit non-overlapping, geometrically bounded organs and tissues to be identified as a collection of individual features.

Figure 6

Axial MRI sections of the brain. The left image is normal. The right image shows high-intensity lesions typical of multiple sclerosis. An image database would need to provide an ability to retrieve groups of images whose lesion sizes, shapes or clustering would bear some notion of similarity.

Figure 7

Cardiac ventriculograms illustrating the difficulty of precisely textually defining the term “ventricular aneurysm.” Two exemplary angiograms from different individuals are shown on the top. Below these are boundary drawings pairs (systole and diastole) illustrating contraction patterns of other patients who are candidates for being labeled as having one form of aneurysm or another. Separation into meaningful visual subgroups and the threshold for such classification are subject to considerable debate.

Figure 8

A geometric schema for organizing the arrangement and properties of component features of an image. The left image is a “cardiac four-chamber” MRI section displaying the cardiac chambers. The middle image represents an interactively generated abstract of the main image features. The right image shows the topologic operator (Voronoi diagram) that uniquely creates an indexable mathematical value derived from these segments. This operation takes into account both the explicitly identified objects and implicitly calculated features such as the shape and properties of a wall between features (here, the interventricular septum).

Figure 9

Coronal MRIs of three different individuals exemplifying variations in the shape and configuration of the ascending aorta (arrow). The image on the left is normal while the middle image might be textually described as “tortuous” and the right image might be variably called “ectatic” or “aneurysmal.” In a large collection, shape based on geometry might serve to better index the collection.