Small bowel capsule endoscopy examination and open access database with artificial intelligence: The SEE-artificial intelligence project

Abstract

Objectives: Artificial intelligence (AI) may be practical for image classification of small bowel capsule endoscopy (CE). However, creating a functional AI model is challenging. We attempted to create a dataset and an object detection CE AI model to explore modeling problems to assist in reading small bowel CE.

Methods: We extracted 18,481 images from 523 small bowel CE procedures performed at Kyushu University Hospital from September 2014 to June 2021. We annotated 12,320 images with 23,033 disease lesions, combined them with 6161 normal images as the dataset, and examined the characteristics. Based on the dataset, we created an object detection AI model using YOLO v5 and we tested validation.

Results: We annotated the dataset with 12 types of annotations, and multiple annotation types were observed in the same image. We test validated our AI model with 1396 images, and sensitivity for all 12 types of annotations was about 91%, with 1375 true positives, 659 false positives, and 120 false negatives detected. The highest sensitivity for individual annotations was 97%, and the highest area under the receiver operating characteristic curve was 0.98, but the quality of detection varied depending on the specific annotation.

Conclusions: Object detection AI model in small bowel CE using YOLO v5 may provide effective and easy-to-understand reading assistance. In this SEE-AI project, we open our dataset, the weights of the AI model, and a demonstration to experience our AI. We look forward to further improving the AI model in the future.

Keywords: artificial intelligence; capsule endoscopy; diagnostic imaging; gastrointestinal tract; intestine small.

FIGURE 1

Image extraction procedure of the dataset. We used 523 of 954 collected capsule endoscopy (CE) videos as the dataset. This dataset contains 23,033 annotation labels.

FIGURE 2

Type of annotations in the dataset. Representative images of the annotations and the distribution of annotation label numbers in the dataset; the number of labels per annotation ranged from 54 to 6788. The specific annotation definitions were as follows: Angiodysplasia; areas of erythema with suspected capillary lesions, Erosion; areas of mucosal damage such as erosions, ulcers, and notches, Stenosis; areas of constriction and rigidity, Lymphangiectasis; areas containing lymphatic vessels larger than a point, Lymph follicle; areas containing normal follicles and suspected lymphatic follicles, Submucosal tumor (SMT); areas resembling submucosal tumors, Polyp like; elevated lesions with a base or areas of suspected adenoma, Bleeding; areas of apparent hemorrhage, exclude sites that have become darker and are distant from the source of bleeding, Diverticula; areas of the suspected diverticulum, Redness; areas of redness and edema that may be related to inflammation, Foreign body; foreign objects other than food, Venous; areas with venous structures.

FIGURE 3

Features of annotations in the dataset. (a) Distribution of annotation labels per image and 4083 of 18,481 images have multiple lesions in one image. Approximately 2% of all images contained more than 10 annotation labels. (b) Distribution of annotation label types per image. There are two different abnormalities in 1356 images and three types in 44 images. (c) The number of annotation labels appears in the same image; for example, “erosion” and “redness” both tended to appear more often with other annotation label types.

FIGURE 4

Relationship between Precision, Recall, F1 score, and confidence rate in creating AI model. The point with the highest F1 score had a confidence rate of 0.279.

FIGURE 5

Representative images detected by created artificial intelligence (AI) model. Disease lesions detected by the AI are indicated on a rectangle, along with the confidence rate. The left image detected two different erosions, the middle image detected submucosal tumor (SMT) and erosion at the top, and the right image detected erosion and bleeding (representative detection videos are also included in the dataset).

FIGURE 6

Receiver operating characteristic (ROC) curves for each annotation. The ROC curve is calculated by determining annotations per frame in 1396 test‐validated images. When multiple annotations of the same type are present in the same image, the bounding box with the highest confidence score is utilized in the calculation. The highest area under the ROC curve (AUC) was 0.98 for “venous” and “lymphangiectasis”.