A demonstration of automated visual evaluation of cervical images taken with a smartphone camera

Abstract

We examined whether automated visual evaluation (AVE), a deep learning computer application for cervical cancer screening, can be used on cervix images taken by a contemporary smartphone camera. A large number of cervix images acquired by the commercial MobileODT EVA system were filtered for acceptable visual quality and then 7587 filtered images from 3221 women were annotated by a group of gynecologic oncologists (so the gold standard is an expert impression, not histopathology). We tested and analyzed on multiple random splits of the images using two deep learning, object detection networks. For all the receiver operating characteristics curves, the area under the curve values for the discrimination of the most likely precancer cases from least likely cases (most likely controls) were above 0.90. These results showed that AVE can classify cervix images with confidence scores that are strongly associated with expert evaluations of severity for the same images. The results on a small subset of images that have histopathologic diagnoses further supported the capability of AVE for predicting cervical precancer. We examined the associations of AVE severity score with gynecologic oncologist impression at all regions where we had a sufficient number of cases and controls, and the influence of a woman's age. The method was found generally resilient to regional variation in the appearance of the cervix. This work suggests that using AVE on smartphones could be a useful adjunct to health-worker visual assessment with acetic acid, a cervical cancer screening method commonly used in low- and middle-resource settings.

Keywords: automated visual evaluation; cervical cancer screening; deep learning; smartphone camera.

Figure 1.

Examples of cervigrams used in the published validation of automated visual evaluation, demonstrating the uniform monochromatic and flattened images characteristic of that image capturing method. AVE prob = AVE case probability; Ground truth = histologic ground truth.

Figure 2.

Examples of cervical images from the MobileODT archive, demonstrating the variability of appearance characteristic of smartphone images. Image on the left shows MobileODT’s EVA handheld cerviscope (https://www.mobileodt.com/eva-system)

Figure 3.

Results of test set using Faster R-CNN, for three random 80:20% training/test samplings of highest rated visits (most likely precancer cases) versus least likely rated visits (most likely controls). All the AUC values refer to images not patients. In other words, the predictions of multiple images per patient were not pooled before calculating the AUC. AUC value for experiment 1 is 0.94 (95%CI 0.92–0.96), experiment 2 is 0.92 (95%CI 0.90–0.94), experiment 3 is 0.93 (95%CI 0.92–0.95).

Figure 4.

ROC curve for test set in the biopsy validated dataset (CIN2+ versus CIN1/normal) and the AUC value is 0.87 (95% CI 0.81–0.92)

Figure 5.

The associations of AVE severity score by gynecologic oncologist impression (Women of Low risk, Low-Med risk, Medium risk, Med-High risk and High risk). In the plot, “Nobs” is short for “Number of observations” implying number of visits (women).