Intraoperative laparoscopic photoacoustic image guidance system in the da Vinci surgical system

Abstract

This paper describes a framework allowing intraoperative photoacoustic (PA) imaging integrated into minimally invasive surgical systems. PA is an emerging imaging modality that combines the high penetration of ultrasound (US) imaging with high optical contrast. With PA imaging, a surgical robot can provide intraoperative neurovascular guidance to the operating physician, alerting them of the presence of vital substrate anatomy invisible to the naked eye, preventing complications such as hemorrhage and paralysis. Our proposed framework is designed to work with the da Vinci surgical system: real-time PA images produced by the framework are superimposed on the endoscopic video feed with an augmented reality overlay, thus enabling intuitive three-dimensional localization of critical anatomy. To evaluate the accuracy of the proposed framework, we first conducted experimental studies in a phantom with known geometry, which revealed a volumetric reconstruction error of 1.20 ± 0.71 mm. We also conducted an ex vivo study by embedding blood-filled tubes into chicken breast, demonstrating the successful real-time PA-augmented vessel visualization onto the endoscopic view. These results suggest that the proposed framework could provide anatomical and functional feedback to surgeons and it has the potential to be incorporated into robot-assisted minimally invasive surgical procedures.

Fig. 1.

(a) The customized laparoscopic photoacoustic (PA) probe compatible with the da Vinci System. (b) The localization markers on the fiber mount. (c) The imaging tip of the probe with two side-illumination diffusing fibers attached by the mount. (d) and (e) The Computer-Aided Design (CAD) design of the customized laparoscopic PA probe compatible with the da Vinci system and the design dimension.

Fig. 2.

Device overview of the proposed photoacoustic (PA) imaging integrated da Vinci surgical platform.

Fig. 3.

Scanning trajectory and the coordinate frame assignment for the photoacoustic (PA) augmented surgical scene rendering. The endoscopic camera manipulator (ECM) image displays the detected localization markers and the defined ECM frame.

Fig. 4.

Communication architecture of the proposed framework.

Fig. 5.

Ex vivo vessel-mimicking phantom design. (a) shows the buried tube filled with blood and (b) shows the sutured tissue prepared for scanning.

Fig. 6.

The 3D reconstructed wire phantom photoacoustic (PA) image based on the endoscopic frame. (a) shows the maximum intensity projection (MIP) of the reconstructed PA image across the entire depth range. (b) shows the reconstructed ultrasound (US) image with same viewpoint as (a). (c) shows the MIP image from the Y-Z plane with the white line indicating scanning region, and (d) from X-Z plane. (e) shows the top-view photo of the phantom and the red points indicate the marker location for the reconstruction error calculations. (f) shows the marker point location error that occurred during the reconstruction scanning.

Fig. 7.

The ex vivo study result. (a) and (b) show the photoacoustic (PA) images at two scanning locations highlighting the vascular structure overlaid with the corresponding ultrasound (US) images collected at the same location.

Fig. 8.

(a) The real-time photoacoustic-augmented surgical scene displayed to the da Vinci master console. (b) The maximum intensity projection (MIP) of the reconstructed volumetric PA images of vasculature overlaid the da Vinci endoscopic camera manipulator (ECM) image in depth encoding color. The depth is computed with respect to the ECM viewpoint. (see Visualization 1)

Fig. 9.

(a) 3D cone-beam computed tomography (CB-CT) scanning of the ex vivo sample after the study. The yellow dashed line indicates the detected vessels. The green dots label the detected vessel trajectory. (b) Reconstructed 3D vessel trajectory from CB-CT imaging and proposed photoacoustic (PA) scanning. (c) Actual sample picture before scanning overlaid with the maximum intensity projection (MIP) image of the PA scanning.