Fusion viewer: a new tool for fusion and visualization of multimodal medical data sets

Abstract

A new application, Fusion Viewer, available for free, has been designed and implemented with a modular object-oriented design. The viewer provides both traditional and novel tools to fuse 3D data sets such as CT (computed tomography), MRI (magnetic resonance imaging), PET (positron emission tomography), and SPECT (single photon emission tomography) of the same subject, to create maximum intensity projections (MIP) and to adjust dynamic range. In many situations, it is desirable and advantageous to acquire biomedical images in more than one modality. For example, PET can be used to acquire functional data, whereas MRI can be used to acquire morphological data. In some situations, a side-by-side comparison of the images provides enough information, but in most of the cases it may be necessary to have the exact spatial relationship between the modalities presented to the observer. To accomplish this task, the images need to first be registered and then combined (fused) to create a single image. In this paper, we discuss the options for performing such fusion in the context of multimodal breast imaging. Additionally, a novel spline-based dynamic range technique is presented in detail. It has the advantage of obtaining a high level of contrast in the intensity range of interest without discarding the intensity information outside of this range while maintaining a user interface similar to the standard window/level windowing procedure.

Fig 1

A volume displayed in the three conventional orthogonal views: coronal, axial, and sagittal from top to bottom.

Fig 2

Example of spline-based TTF.

Fig 3

MIP formed from MRI. Bright spots correspond to contrast markers placed on patient’s skin for image registration.

Fig 4

Image created using color mixing fusion technique.

Fig 5

Original MRI image.

Fig 6

Original PET image.

Fig 7

Image created using HSL color space. Hue = PET, saturation = CT, lightness = MRI.

Fig 8

Image created using HSL color space. Hue varies from cyan to green to yellow with the value of PET image; saturation is constant and lightness varies with MRI.

Fig 9

Image created by interlacing PET and MRI volumes. PET image is displayed using a fire color table and MRI with a grayscale color table.

Fig 10

Color MIP from fused PET/MRI.