Expert Performance in Visual Differentiation of Bacterial and Fungal Keratitis

Abstract

This study quantifies the performance of an international cohort of cornea specialists in image-based differentiation of bacterial and fungal keratitis, identifying significant regional variation and establishing a reference standard for comparison against machine learning models.

Keywords: Bacterial keratitis; Corneal ulcer; Fungal keratitis; Infectious keratitis.

Figure 1:. Comparison of Performance Among Indian and non-Indian Cornea Specialists

A) Empirical receiver operating characteristic (ROC) curves for the ensemble estimated probability of fungal keratitis among all cornea specialists combined (black), cornea specialists practicing in India (blue), and cornea specialists practicing outside India (orange). TPR = true positive rate. FPR = false positive rate. B) The three corneal ulcer images from the testing set which demonstrated the the largest difference in ensemble estimated probability between Indian (blue) and non-Indian (orange) expert image graders. Next to each image is the corresponding distribution of responses among both groups of experts. In all three examples the image was obtained from a case of culture-proven fungal keratitis, and in each case the ensemble prediction of Indian graders was closer to the ground truth than the ensemble prediction among non-Indian graders.