Image overlay solution based on threshold detection for a compact near infrared fluorescence goggle system

Abstract

Near infrared (NIR) fluorescence imaging has shown great potential for various clinical procedures, including intraoperative image guidance. However, existing NIR fluorescence imaging systems either have a large footprint or are handheld, which limits their usage in intraoperative applications. We present a compact NIR fluorescence imaging system (NFIS) with an image overlay solution based on threshold detection, which can be easily integrated with a goggle display system for intraoperative guidance. The proposed NFIS achieves compactness, light weight, hands-free operation, high-precision superimposition, and a real-time frame rate. In addition, the miniature and ultra-lightweight light-emitting diode tracking pod is easy to incorporate with NIR fluorescence imaging. Based on experimental evaluation, the proposed NFIS solution has a lower detection limit of 25 nM of indocyanine green at 27 fps and realizes a highly precise image overlay of NIR and visible images of mice in vivo. The overlay error is limited within a 2-mm scale at a 65-cm working distance, which is highly reliable for clinical study and surgical use.

Fig. 1

Dual-spectrum imaging system with a goggle display. (a) The imaging system is composed of two complementary metal oxide semiconductor (CMOS) imaging sensors, long-pass and short-pass optical filters, an FPGA data acquisition and image processing board, and an HDMI goggle system; (b) a custom-built NIR and visible white light-emitting diode (LED) illumination module; and (c) an NIR-visible spectrum LED tracking pod.

Fig. 2

Block diagram of the NIR-visible imaging system. NIR-visible tracking pods are used to compute an average disparity between the two images. This disparity information is applied to the entire image to create a combined image in which the location of fluorescent data is highlighted on the color image data.

Fig. 3

Flowchart of the disparity image processing algorithm using NIR-visible tracking pods. The algorithm is implemented on both FPGA and PC for real-time display of combined NIR-color images to the surgeon.

Fig. 4

Disparity error estimate illustration.

Fig. 5

Disparity error measurement.

Fig. 6

Sensitivity test using ICG-DMSO and LS301-DMSO: (a) logarithmic scale and (b) linear scale.

Fig. 7

Signal-to-background ratio (SBR); green line shows the $\mathrm{SBR}=2$ threshold: (a) logarithmic scale and (b) linear scale.

Fig. 8

Mouse study test result: (a) near infrared (NIR) channel image, (b) visible channel image, (c) predisparity correction image, and (d) corrected image using a threshold detection algorithm.

Fig. 9

Mouse study test result (open skin): (a) NIR channel image, (b) visible channel image, (c) predisparity correction image, and (d) corrected image using a threshold detection algorithm.