Assessment of a Tele-education System to Enhance Retinopathy of Prematurity Training by International Ophthalmologists-in-Training in Mexico

Abstract

Purpose: To evaluate a tele-education system developed to improve diagnostic competency in retinopathy of prematurity (ROP) by ophthalmologists-in-training in Mexico.

Design: Prospective, randomized cohort study.

Participants: Fifty-eight ophthalmology residents and fellows from a training program in Mexico consented to participate. Twenty-nine of 58 trainees (50%) were randomized to the educational intervention (pretest, ROP tutorial, ROP educational chapters, and posttest), and 29 of 58 trainees (50%) were randomized to a control group (pretest and posttest only).

Methods: A secure web-based educational system was created using clinical cases (20 pretest, 20 posttest, and 25 training chapter-based) developed from a repository of over 2500 unique image sets of ROP. For each image set used, a reference standard ROP diagnosis was established by combining the clinical diagnosis by indirect ophthalmoscope examination and image-based diagnosis by multiple experts. Trainees were presented with image-based clinical cases of ROP during a pretest, posttest, and training chapters.

Main outcome measures: The accuracy of ROP diagnosis (e.g., plus disease, zone, stage, category) was determined using sensitivity and specificity calculations from the pretest and posttest results of the educational intervention group versus control group. The unweighted kappa statistic was used to analyze the intragrader agreement for ROP diagnosis by the ophthalmologists-in-training during the pretest and posttest for both groups.

Results: Trainees completing the tele-education system had statistically significant improvements (P < 0.01) in the accuracy of ROP diagnosis for plus disease, zone, stage, category, and aggressive posterior ROP (AP-ROP). Compared with the control group, trainees who completed the ROP tele-education system performed better on the posttest for accurately diagnosing plus disease (67% vs. 48%; P = 0.04) and the presence of ROP (96% vs. 91%; P < 0.01). The specificity for diagnosing AP-ROP (94% vs. 78%; P < 0.01), type 2 ROP or worse (92% vs. 84%; P = 0.04), and ROP requiring treatment (89% vs. 79%; P < 0.01) was better for the trainees completing the tele-education system compared with the control group. Intragrader agreement improved for identification of plus disease, zone, stage, and category of ROP after completion of the educational intervention.

Conclusions: A tele-education system for ROP education was effective in improving the diagnostic accuracy of ROP by ophthalmologists-in-training in Mexico. This system has the potential to increase competency in ROP diagnosis and management for ophthalmologists-in-training from middle-income nations.

Figure 2.

Survey results of trainees who completed the retinopathy of prematurity (ROP) tele-education system (N = 24). Survey questions after completion of the ROP tele-education system were rated using a Likert scale of “strongly agree” to “strongly disagree.” The questions included in the survey were (1) “I had an adequate understanding of the diagnosis of ROP before taking the pretest in the ROP Student Module”; (2) “The ROP Student Module was easy to use”; (3) “I learned from the feedback provided at the end of each case”; (4) “I had an adequate understanding of the diagnosis of ROP after completing the ROP Student Module”; (5) “I learn more effectively in a web-based environment compared to a traditional textbook format”; and (6) “I learned more effectively from ROP cases with automatic feedback compared to a traditional textbook format.”