Robust feature tracking on the beating heart for a robotic-guided endoscope

Abstract

Background: Visualization during minimally invasive bypass surgery on the beating heart can be enhanced by using a robotic-guided endoscope and visual servoing from the endoscopic images. In order to achieve these objectives, this work has focused on developing and testing algorithms for accurate, robust and real-time motion tracking of features on the beating heart, using marker-less approaches and an uncalibrated endoscope.

Methods: Lucas-Kanade pyramidal optical flow-based algorithms and speeded-up robust features (SURF)-based methods have been extensively evaluated, using a range of developed metrics, in order to quantify accuracy, robustness and drift under a variety of circumstances. Three sets of experiments are reported: the first set compared the two tracking methods, using a beating-heart phantom and a static endoscope; the second set evaluated the methods when images were taken using a moving robotic-guided endoscope; and finally, the Lucas-Kanade optical flow algorithm was extensively tested in a visual servoing application, using a robotic endoscope.

Results: The combination of a Lucas-Kanade tracking algorithm and a SURF-based feature detection method gave the best performance in terms of accuracy and robustness of tracking, while preserving real-time computation requirements. The optimal parameters consist of a window size of 51 × 51 pixels and an interframe motion threshold of 20 pixels. Feature tracking was successfully integrated into uncalibrated visual servoing or a robotic-guided endoscope.

Conclusions: Robust feature tracking on a beating heart with endoscopic video can be achieved in real-time and may facilitate robotically-assisted, minimally invasive bypass surgery and conventional laparoscopic surgery.