Intraoperative hyperspectral label-free imaging: from system design to first-in-patient translation

Abstract

Despite advances in intraoperative surgical imaging, reliable discrimination of critical tissue during surgery remains challenging. As a result, decisions with potentially life-changing consequences for patients are still based on the surgeon's subjective visual assessment. Hyperspectral imaging (HSI) provides a promising solution for objective intraoperative tissue characterisation, with the advantages of being non-contact, non-ionising and non-invasive. However, while its potential to aid surgical decision-making has been investigated for a range of applications, to date no real-time intraoperative HSI (iHSI) system has been presented that follows critical design considerations to ensure a satisfactory integration into the surgical workflow. By establishing functional and technical requirements of an intraoperative system for surgery, we present an iHSI system design that allows for real-time wide-field HSI and responsive surgical guidance in a highly constrained operating theatre. Two systems exploiting state-of-the-art industrial HSI cameras, respectively using linescan and snapshot imaging technology, were designed and investigated by performing assessments against established design criteria and ex vivo tissue experiments. Finally, we report the use of our real-time iHSI system in a clinical feasibility case study as part of a spinal fusion surgery. Our results demonstrate seamless integration into existing surgical workflows.

Keywords: computer assisted interventions; exoscope; first-in-patient; hyperspectral imaging; medical device; translational research.

Figure 1.

Schematic diagram of our intraoperative hyperspectral imaging (HSI) system illustrated for the example of spine surgery. A snapshot HSI camera system was used for the in-patient clinical feasibility case study as part of a spinal fusion surgery. Video-rate HSI data was acquired during surgery. An example in vivo snapshot hyperspectral mosaic image demonstrating the exposed dura of the spinal cord following laminectomy is provided.

Figure 2.

Illustration of the Imec snapshot mosaic CMV2K-SM5x5-NIR sensor which acquires 25 spectral bands between 665 and 975 nm in a 5 × 5 mosaic. Hyperspectral data is captured in a single shot (‘snapshot mosaic’) by acquiring spatially and spectrally interleaved information following a mosaic array arrangement.

Figure 3.

(a) Two custom heat sinks are mounted on the Photonfocus camera to keep operating temperatures low. (b) Custom adapter plates with 1/4-20 UNC and 3/8-16 UNC threaded holes were created for both Photonfocus and Imec cameras for use with standard tripod systems as described in section 3.4.

Figure 4.

Measured spectrum of Asahi Spectra Xenon light source using the VIS and UV-NIR mirror modules. For the UV-NIR mirror module, the spectrum with and without additional UV filter was measured. Each curve was normalized based on maximum intensity.

Figure 5.

(a) Checkerboard with 48 colour patches used for HSI system validation. (b) Spectrometer setup to acquire reference data. (c) Snapshot mosaic image of 4C patch with five manually placed circular annotations of 10 pixel radius distributed over the patch for spectral analysis. The same annotation steps were performed to evaluate the respective linescan data.

Figure 6.

Comparison of measured reflectance curves between the linescan and snapshot iHSI camera systems and the reference spectrometer for each of the 48 colour patches. For both linescan and snapshot camera, mean and standard deviation of reflectance measurements within the manually segmented regions are shown.

Figure 7.

(a) Tripod setup with mounted intraoperative HSI (iHSI) system using the linescan camera for ex vivo experiments. A quick release plate used with standard tripod systems was used to mount linescan, snapshot and RGB cameras using custom adapter plates such as shown in figure 3(b). (b) Imaging setup during cadaveric veal experiments with orientated camera head for tissue assessment. Labels are provided to reference anatomical locations for tissue analysis shown in figure 9.

Figure 8.

Example sequence of camera system acquisitions during ex vivo imaging to capture HSI data of the spinal cord and rootlets. With fiducials visible across the VIS and NIR spectrum, alignment between linescan (VIS & NIR) and snapshot (NIR) imagery was achieved using affine point-based registration for spectral analysis (figure 9).

Figure 9.

Comparison of reflectance curve measurements between 470 and 740 nm associated with ex vivo tissue sample imaging of eight different anatomical scenes shown in figure 7(b) using the proposed iHSI system with linescan and snapshot cameras. For each camera, both the mean and standard deviation of reflectance measurements within the manually segmented regions are shown. Relative distribution and qualitative behaviour of reflectance values across tissue types between the cameras are well aligned. However, quantitative measurements of tissue reflectances between cameras generally deviate from each other likely due to different white balancing requirements associated with imaging multiple anatomical regions during the experiment.

Figure 10.

(a) Intraoperative HSI (iHSI) setup during spinal fusion surgery. A live display provides real-time visualisation of snapshot HSI data. (b) Example snapshot mosaic images of acquired in vivo HSI imagery.