The Fusion of Robotics and Imaging: A Vision of the Future

Abstract

Medical robotics has evolved significantly over the past decades, with more robotic systems than ever now in hospitals. The use of robots for surgery has already revolutionized the surgical landscape by providing high accuracy, greater surgeon dexterity, 3-dimensional visualization capabilities, and the potential for telesurgery. At the same time, diverse medical imaging modalities such as X-ray, computed tomography, magnetic resonance imaging, and ultrasound can provide meaningful visual information required for pre- and intraoperative planning and guidance, which is particularly important for robotic surgery given the surgeon's limited direct view of the surgical workspace in minimally invasive procedures. This review provides an overview of robotics technology for surgery and image acquisition and describes the main steps involved in the processing of medical imaging and its use. The authors also share their views on what the future of robotics and imaging may look like for surgical applications given the recent advances in artificial intelligence, computer vision, and machine learning.

Keywords: future trends in robotics; image fusion; robotic surgery.

Figure 1

Pipeline of modern image-guided surgery. Left: Surgeons use a variety of imaging techniques preoperatively for surgical planning, such as X-ray, CT, MRI, PET, and others. Right: During surgery, imaging and modern visualization tools are used for guidance and verification. CT: computed tomography; MRI: magnetic resonance imaging; PET: positron emission tomography

Figure 2

Example of a segmented and colorized 3-dimensional model of a patient heart as seen in a virtual reality headset.

Figure 3

Preliminary work enabling visualization of thoracic vasculature using mixed reality to fuse the real patient with 3D reconstruction of the CT-scanned anatomy of interest. 3D: 3-dimensional; CT: computed tomography

Figure 4

Robotic treatment of type II endoleak after endovascular aneurysm repair. (A) Axial view of dynamic CTA showing type II endoleak evidenced by retrograde filling of the excluded aneurysm sac from the inferior mesenteric artery (IMA). (B) Dissection carried down to the anterior surface of the aneurysm sac with the origin and first branch of the IMA is mobilized prior to ligation. (C) Axial view of dynamic CTA showing type II endoleak evidenced by retrograde filling of the excluded aneurysm sac from the lumbar artery, allowing orientation and spinal level to be identified for preoperative planning. (D) Aneurysm sac visualized on the left side of the screen with identification of left-sided posterolateral lumbar artery seen on preoperative dynamic CTA as source for type II endoleak. (E) Da Vinci Xi patient-side cart with 4 arms docked to robotic ports and working instruments. CTA: computed tomography angiogram.