Real-time snapshot hyperspectral imaging endoscope

Abstract

Hyperspectral imaging has tremendous potential to detect important molecular biomarkers of early cancer based on their unique spectral signatures. Several drawbacks have limited its use for in vivo screening applications: most notably the poor temporal and spatial resolution, high expense, and low optical throughput of existing hyperspectral imagers. We present the development of a new real-time hyperspectral endoscope (called the image mapping spectroscopy endoscope) based on an image mapping technique capable of addressing these challenges. The parallel high throughput nature of this technique enables the device to operate at frame rates of 5.2 frames per second while collecting a (x, y, λ) datacube of 350 × 350 × 48. We have successfully imaged tissue in vivo, resolving a vasculature pattern of the lower lip while simultaneously detecting oxy-hemoglobin.

Figure 1

Image mapping spectroscopy (IMS) concept.

Figure 2

Image mapping spectrometer (IMS) endoscope optical design layout.

Figure 3

Diagram showing the role of the image mapper in the system as it relates to the final datacube size. See text for detail.

Figure 4

Prism/lens array design layouts and performance metrics.

Figure 5

IMS endoscope miniature grin lens: (a) design, (b) and (c) performance metrics, (d) prototype, and (e) imaging result.

Figure 6

Image mapper (a) design, (b) fabrication process, and (c) prototype segment.

Figure 7

Assembled IMS endoscope system prototype (a) full system and (b) with cover. Close up images of (c) image mapper, (d) lens array, and (e) rows of Amici prisms.

Figure 8

Comparison of predicted and measured wavelength to pixel shift for the IMS endoscope system.

Figure 9

Demonstration of how the raw data (a) is remapped one line (b) at a time to form a single monochromatic image (c) within the simultaneously acquired (x, y, λ) datacube. This process is repeated for each wavelength within the datacube.

Figure 10

Depiction of the procedure for correcting small mapping errors in the IMS system. (a) Close up of six subimages in the IMS system. (b) The incorrect remapped image of a Ronchi ruling. (c) The corrected remapped image of a Ronchi ruling.

Figure 11

Raw monochromatic image from the IMS endoscope showing (a) all 24 subimages and (b) a close up of a single subimage. Polychromatic image with (c) all 24 subimages and (d) close up of a single polychromatic subimage. (e) A close up of a single line within a monochromatic subimage shows the individual fibers within the multifiber bundle.

Figure 12

(a) Raw image from the IMS of a USAF resolution target simultaneously acquired. (b)–(e) four out of 48 remapped images for the wavelengths (584, 567, 549, and 532 nm).

Figure 13

Experimental setup for imaging the tissue vascularization of the lower lip of a normal volunteer.

Figure 14

Lower lip vasculature imaging results from a normal volunteer using the IMS endoscope. (a) One of the 29 spectral images acquired (546 nm band). (b) Reference image taken with the color CCD camera. (c) Spectral curves from an area in the image where there is a vein (solid line) and no vein (dashed line).

Figure 15

(a) A more robust IMS endoscope for use in a clinical setting. (b) Miniature imaging end of the IMS endoscope at the end of the Pentax endoscope. (c) Fiber optics of the IMS endoscope inserted into the instrument channel of the Pentax system.

Figure 16

Images of the IMS endoscope taken by the Pentax endoscope's camera: (a) 99% reflectance standard and (b) lower lip of a normal human volunteer.

Figure 17

Datacube acquired using the IMS endoscope of the lower lip of a normal human volunteer. (a) Color composite image, (b) spectrum from vein and no-vein region, and (c) nine out of 45 spectral channel images. Datacubes were acquired at 143 ms using the full 12-bit dynamic range.