Predictive modeling of proliferative vitreoretinopathy using automated machine learning by ophthalmologists without coding experience

Abstract

We aimed to assess the feasibility of machine learning (ML) algorithm design to predict proliferative vitreoretinopathy (PVR) by ophthalmologists without coding experience using automated ML (AutoML). The study was a retrospective cohort study of 506 eyes who underwent pars plana vitrectomy for rhegmatogenous retinal detachment (RRD) by a single surgeon at a tertiary-care hospital between 2012 and 2019. Two ophthalmologists without coding experience used an interactive application in MATLAB to build and evaluate ML algorithms for the prediction of postoperative PVR using clinical data from the electronic health records. The clinical features associated with postoperative PVR were determined by univariate feature selection. The area under the curve (AUC) for predicting postoperative PVR was better for models that included pre-existing PVR as an input. The quadratic support vector machine (SVM) model built using all selected clinical features had an AUC of 0.90, a sensitivity of 63.0%, and a specificity of 97.8%. An optimized Naïve Bayes algorithm that did not include pre-existing PVR as an input feature had an AUC of 0.81, a sensitivity of 54.3%, and a specificity of 92.4%. In conclusion, the development of ML models for the prediction of PVR by ophthalmologists without coding experience is feasible. Input from a data scientist might still be needed to tackle class imbalance-a common challenge in ML classification using real-world clinical data.

Figure 1

This diagram demonstrates the workflow for predictive modeling of proliferative vitreoretinopathy (PVR) using automated machine learning (AutoML). After data preparation and class balancing using random undersampling (RUS), the two ophthalmologists used the Classification Learner App to design support vector machine (SVM) and Naïve Bayes (NB) algorithms. Details about the feature sets and models are described in the methodology. In parallel to the design by the two ophthalmologists, manually coded algorithms were also prepared as a benchmarking measure.

Figure 2

Receiver operating characteristic (ROC) curves of the discriminative performance of Models 1–4. Models 1 (quadratic Support Vector Machine [SVM]) and 2 (optimized Naïve Bayes [NB]) used Feature Set 1 that included all clinically important features. Models 3 (optimized SVM) and 4 (optimized NB) used Feature Set 2, which did not include pre-existing PVR as an input feature.

Figure 3

Four representative cases are shown highlighting correct classifications and misclassifications by Model 1. Case 1 illustrates a correctly classified case of PVR in an eye with a total rhegmatogenous retinal detachment (RRD) and late presentation prior to surgery. Case 3 shows a common case of simple macula-off RRD with no obvious risk factors for PVR, correctly classified as “No PVR”. In case 2, the algorithm misclassified the case as “PVR” probably due to the long duration of symptoms and the presence of vitreous hemorrhage. In case 4, the algorithm failed to predict PVR despite the presence of a giant retinal tear. The normal intraocular pressure, absence of pre-existing PVR, and macula-on status might have influenced the classifier’s decision.