Automatic evaluation of degree of cleanliness in capsule endoscopy based on a novel CNN architecture

Abstract

Capsule endoscopy (CE) is a widely used, minimally invasive alternative to traditional endoscopy that allows visualisation of the entire small intestine. Patient preparation can help to obtain a cleaner intestine and thus better visibility in the resulting videos. However, studies on the most effective preparation method are conflicting due to the absence of objective, automatic cleanliness evaluation methods. In this work, we aim to provide such a method capable of presenting results on an intuitive scale, with a relatively light-weight novel convolutional neural network architecture at its core. We trained our model using 5-fold cross-validation on an extensive data set of over 50,000 image patches, collected from 35 different CE procedures, and compared it with state-of-the-art classification methods. From the patch classification results, we developed a method to automatically estimate pixel-level probabilities and deduce cleanliness evaluation scores through automatically learnt thresholds. We then validated our method in a clinical setting on 30 newly collected CE videos, comparing the resulting scores to those independently assigned by human specialists. We obtained the highest classification accuracy for the proposed method (95.23%), with significantly lower average prediction times than for the second-best method. In the validation of our method, we found acceptable agreement with two human specialists compared to interhuman agreement, showing its validity as an objective evaluation method.

Figure 1

Different types of intestinal content that we aim to detect in our data set, extracted from videos of CE procedures performed with the PillCam SB 3 model, obtained as explained in “Data conditioning” section through the Rapid Reader v8.3 software, http://medtronic.com/covidien/en-us/support/software/gastrointestinal-products/rapid-reader-software-v8-3.html.

Figure 2

The process of our data collection and partitioning. From videos we extracted frame images in which patches were annotated by specialists. We then partitioned the images into five folds (green), while we let the remaining images be the training images in those folds. From these, we extracted the annotated patches into equivalent folds, so that we ended up with the corresponding five sets of training (blue) and test (green) patches.

Figure 3

Visualisation of the CNN architecture proposed for intestinal content classification in this work.

Figure 4

Visual intestinal content detection results of the proposed method, interpolating the probabilities per pixel from the patch probabilities and displaying as a heat map, with original images displayed next to each result. The images in (a–f) are selected images from our validation set showing various types of intestinal content. The images in (g–i) are selected images from the test set of our model showing intestinal content detection results in the presence of pathologies: (g) a polyp; (h) angioectasia and (i) an ulcer with bleeding. The images in (j–l) are characteristic images for which there was a significant difference between the assigned evaluation score and the evaluation scores assigned by both human specialists. Original images were extracted from videos of CE procedures performed with the PillCam SB 3 model, obtained as explained in “Data conditioning” section through the Rapid Reader v8.3 software, http://medtronic.com/covidien/en-us/support/software/gastrointestinal-products/rapid-reader-software-v8-3.html. The overlays with the detection results were created through MATLAB 2018a, http://mathworks.com/products/new_products/release2018a.html.

Figure 5

The agreement values of ${\kappa }_1$ measured over the different folds and over the concatenated results of all folds, each plotted with its corresponding 95% confidence interval. Figure created using MATLAB 2018a, http://mathworks.com/products/new_products/release2018a.html.