Characterization of Errors in Retinopathy of Prematurity Diagnosis by Ophthalmologists-in-Training in Middle-Income Countries

Abstract

Purpose: To characterize common errors in the diagnosis of retinopathy of prematurity (ROP) among ophthalmologistsin-training in middle-income countries.

Methods: In this prospective cohort study, 200 ophthalmologists-in-training from programs in Brazil, Mexico, and the Philippines participated. A secure web-based educational system was developed using a repository of more than 2,500 unique image sets of ROP, and a reference standard diagnosis was established by combining the clinical diagnosis and the image-based diagnosis by multiple experts. Twenty web-based cases of wide-field retinal images were presented, and ophthalmologists-in-training were asked to diagnose plus disease, zone, stage, and category for each eye. Trainees' responses were compared to the consensus reference standard diagnosis. Main outcome measures were frequency and types of diagnostic errors were analyzed.

Results: The error rate in the diagnosis of any category of ROP was between 48% and 59% for all countries. The error rate in identifying type 2 or pre-plus disease was 77%, with a tendency for overdiagnosis (27% underdiagnosis vs 50% overdiagnosis; mean difference: 23.4; 95% CI: 12.1 to 34.7; P = .005). Misdiagnosis of treatment-requiring ROP as type 2 ROP was most commonly associated with incorrectly identifying plus disease (plus disease error rate = 18% with correct category diagnosis vs 69% when misdiagnosed; mean difference: 51.0; 95% CI: 49.3 to 52.7; P = .003).

Conclusions: Ophthalmologists-in-training from middle-income countries misdiagnosed ROP more than half of the time. Identification of plus disease was the salient factor leading to incorrect diagnosis. These findings emphasize the need for improved access to ROP education to improve competency in diagnosis among ophthalmologists-in-training in middle-income countries. [J Pediatr Ophthalmol Strabismus. 2023;60(5):344-352.].

Figure 1.

Summary of study methods for retinopathy of prematurity (ROP) competency evaluation. Two hundred ophthalmologists-in-training from middle-income countries were directed to a web-based system. They were prompted with 20 ROP cases and were asked to indicate the presence of plus disease, zone, stage, and category of ROP. The trainees’ performance was assessed in accordance with a consensus reference standard diagnosis that was determined for each case. The rates at which common errors occurred were statistically analyzed.

Figure 2.

Diagnostic error rate of retinopathy of prematurity (ROP) category among ophthalmologists-in-training stratified by post-graduate year. For each web-based case, trainees were prompted to provide the category of ROP (none, mild, type 2 or pre-plus, treatment-requiring). The rates at which the category of ROP was misdiagnosed by ophthalmology trainees were stratified by post-graduate year (PGY-1, PGY-2, PGY-3) and by country (A, B, C).

Figure 3.

Distribution of responses among all ophthalmologists-in-training for each diagnostic category of retinopathy of prematurity (ROP). Trainees were prompted to provide the category of ROP (none, mild, type 2 or pre-plus, treatment-requiring) for each case presented. The correct diagnostic category was determined in accordance with a consensus reference standard diagnosis. The trainees’ responses are represented proportionately for each category of ROP.

Figure 4.

Eye examination with diagnostic discrepancies between the consensus reference standard diagnosis and trainee diagnosis. The patient is a 23-week-old infant with a birth weight of 608 grams and a postmenstrual age of 35 weeks at the time of imaging. The consensus reference standard diagnosis was zone I, stage 3 retinopathy of prematurity (ROP), and plus disease. The trainees most commonly provided a diagnosis of zone II, stage 3 ROP and pre-plus disease.