Computer vision in surgery: from potential to clinical value

Abstract

Hundreds of millions of operations are performed worldwide each year, and the rising uptake in minimally invasive surgery has enabled fiber optic cameras and robots to become both important tools to conduct surgery and sensors from which to capture information about surgery. Computer vision (CV), the application of algorithms to analyze and interpret visual data, has become a critical technology through which to study the intraoperative phase of care with the goals of augmenting surgeons' decision-making processes, supporting safer surgery, and expanding access to surgical care. While much work has been performed on potential use cases, there are currently no CV tools widely used for diagnostic or therapeutic applications in surgery. Using laparoscopic cholecystectomy as an example, we reviewed current CV techniques that have been applied to minimally invasive surgery and their clinical applications. Finally, we discuss the challenges and obstacles that remain to be overcome for broader implementation and adoption of CV in surgery.

Fig. 1. Framework for the analysis of endoscopic videos.

Temporal (a) and spatial (b) annotations at different resolutions are used to model tasks at increasingly finer details.

Fig. 2. CV-based real-time assistance in laparoscopic cholecystectomy.

Overviewed CV models could be used to evaluate the difficulty of a case and whether it is fit for a surgical resident (a), to warn surgeons against incising below the appropriate site (b), to guide safe dissection (c), to automatically evaluate safety measures (d), to prevent misapplications of clips (e) and to improve OR staff awareness and readiness.

Fig. 3. Obstacles and possible solutions for CV in surgery.

Behavior and technical/operational obstacles can limit the development and implementation of CV models in surgery. A combination of statutory, behavioral, and operational changes in the regulatory, clinical, and technical environments could result in improvements in the application of CV for surgery. AI artificial intelligence, PSO patient safety organization, CME continuing medical education, OR operating room.