Computer-vision based analysis of the neurosurgical scene - A systematic review

Abstract

Introduction: With increasing use of robotic surgical adjuncts, artificial intelligence and augmented reality in neurosurgery, the automated analysis of digital images and videos acquired over various procedures becomes a subject of increased interest. While several computer vision (CV) methods have been developed and implemented for analyzing surgical scenes, few studies have been dedicated to neurosurgery.

Research question: In this work, we present a systematic literature review focusing on CV methodologies specifically applied to the analysis of neurosurgical procedures based on intra-operative images and videos. Additionally, we provide recommendations for the future developments of CV models in neurosurgery.

Material and methods: We conducted a systematic literature search in multiple databases until January 17, 2023, including Web of Science, PubMed, IEEE Xplore, Embase, and SpringerLink.

Results: We identified 17 studies employing CV algorithms on neurosurgical videos/images. The most common applications of CV were tool and neuroanatomical structure detection or characterization, and to a lesser extent, surgical workflow analysis. Convolutional neural networks (CNN) were the most frequently utilized architecture for CV models (65%), demonstrating superior performances in tool detection and segmentation. In particular, mask recurrent-CNN manifested most robust performance outcomes across different modalities.

Discussion and conclusion: Our systematic review demonstrates that CV models have been reported that can effectively detect and differentiate tools, surgical phases, neuroanatomical structures, as well as critical events in complex neurosurgical scenes with accuracies above 95%. Automated tool recognition contributes to objective characterization and assessment of surgical performance, with potential applications in neurosurgical training and intra-operative safety management.

Keywords: Automated detection; Neuroanatomy; Surgical instruments; Surgical phase recognition; Surgical videos; computer vision.

Fig. 1

Fundamental outcomes of computer vision: A. (top left) illustrates how two micro forceps are recognized through classification of images on a frame-wise level. B. (top right) illustrates the coarse localization of the two micro forceps with bounding boxes through object detection. C. (down left) illustrates the detailed localization and mapping of the two micro forceps (green) by labelling all pixels pertaining to microforceps and background (purple) using semantic segmentation. D. (down right) illustrates the detailed localization of the two micro forceps (green & blue) as separate instances with respect to the background using instance segmentation. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)

Fig. 2

Study selection according to PRISMA guidelines.

Fig. 3

Quality assessment Stacked bar plots displaying the quality of the included studies according to the degree (complete = white; incomplete = light gray; unspecified = dark gray) of reporting data, model architecture, model hyperparameters and performance metrics.

Fig. 4

Model performances per task.