Designing a wearable navigation system for image-guided cancer resection surgery

Abstract

A wearable surgical navigation system is developed for intraoperative imaging of surgical margin in cancer resection surgery. The system consists of an excitation light source, a monochromatic CCD camera, a host computer, and a wearable headset unit in either of the following two modes: head-mounted display (HMD) and Google glass. In the HMD mode, a CMOS camera is installed on a personal cinema system to capture the surgical scene in real-time and transmit the image to the host computer through a USB port. In the Google glass mode, a wireless connection is established between the glass and the host computer for image acquisition and data transport tasks. A software program is written in Python to call OpenCV functions for image calibration, co-registration, fusion, and display with augmented reality. The imaging performance of the surgical navigation system is characterized in a tumor simulating phantom. Image-guided surgical resection is demonstrated in an ex vivo tissue model. Surgical margins identified by the wearable navigation system are co-incident with those acquired by a standard small animal imaging system, indicating the technical feasibility for intraoperative surgical margin detection. The proposed surgical navigation system combines the sensitivity and specificity of a fluorescence imaging system and the mobility of a wearable goggle. It can be potentially used by a surgeon to identify the residual tumor foci and reduce the risk of recurrent diseases without interfering with the regular resection procedure.

Figure 1

(a) Schematic diagram of the surgical navigation system in two operation modes: Google glass mode and HMD mode. (b) surgical navigation in the Google glass mode. (c) Surgical navigation in the HMD mode.

Figure 2

Flow chart for navigation system calibration and image co-registration. A: image of the surgical scene acquired by the stationary CCD camera. B: image of the tumor-simulating tissue acquired by the wearable headset. C and D: fiducial markers are identified by FLSBM algorithm. E and F: four vertices of the fiducial marker quadrangle are calculated for C and D respectively. G: a transform matrix is generated from E and F. H: fluorescence image acquired by the stationary CCD camera. I: the transformed fluorescence image after the transform matrix G is applied to H. J: image fusion between the surgical scene image F and the transformed fluorescence image I.

Figure 3

Program flow chart for image acquisition, processing, transport, and display on the Google glass side and the host computer side of the surgical navigation system.

Figure 4

Correction of the co-registration error between the fluorescence image of the tumor margin and the surgical scene image. (a) Photo of agar-agar gel phantom with four embedded tumor simulators. (b) Without correction, the co-registration error between the fluorescence image (green) and surgical scene image is significant. (c) With height correction, the center of the projected tumor simulator set matches the actual position.

Figure 5

Relative comparison of the achievable spatial resolutions in the Google glass and the HMD navigation modes. (a) The test pattern is 71 mm × 36 mm, consisting of horizontal and vertical stripes with thicknesses from 0.5 mm to 4 mm and the separation distances from 0.5 mm to 4 mm. (b) Test pattern image acquired by the Google glass. (c) Test pattern image acquired by the HMD device.

Figure 6

Augmented reality of the surgical margin image. (a) The ex vivo tumor simulating model with tumor simulating material injected at the location of the white arrow. (b–d) Google glass display and the surrounding surgical scene as the glass moves to different locations.

Figure 7

Experimental results that simulate the effect of the surgeon’s motion on the imaging lag of the surgical navigation system. (a): No motion exists between the navigation system and the surgical scene. (b)–(d): The navigation system moves with respect to the surgical scene at a speed of 1 m/min, 2m/min, and 6 m/min, respectively.

Figure 8

Image-guided tumor resection surgery on an ex vivo tumor-simulating model. Top row are images for Google glass guided surgery. Bottom row are images for HMD guided surgery. (A) Photographic images of the surgical field without fluorescence filter. (B) Fluorescence images of the surgical margins before resection. (C) Fluorescence images of the surgical margins and the resected tissue samples after resection. (D) IVIS image of the resected tissue sample and the surgical cavity.