The Global Education Network for Retinopathy of Prematurity (Gen-Rop): Development, Implementation, and Evaluation of A Novel Tele-Education System (An American Ophthalmological Society Thesis)

Abstract

Purpose: To describe the design, implementation, and evaluation of a tele-education system developed to improve diagnostic competency in retinopathy of prematurity (ROP) by ophthalmology residents.

Methods: A secure Web-based tele-education system was developed utilizing a repository of over 2,500 unique image sets of ROP. For each image set used in the system, a reference standard ROP diagnosis was established. Performance by ophthalmology residents (postgraduate years 2 to 4) from the United States and Canada in taking the ROP tele-education program was prospectively evaluated. Residents were presented with image-based clinical cases of ROP during a pretest, posttest, and training chapters. Accuracy and reliability of ROP diagnosis (eg, plus disease, zone, stage, category) were determined using sensitivity, specificity, and the kappa statistic calculations of the results from the pretest and posttest.

Results: Fifty-five ophthalmology residents were provided access to the ROP tele-education program. Thirty-one ophthalmology residents completed the program. When all training levels were analyzed together, a statistically significant increase was observed in sensitivity for the diagnosis of plus disease, zone, stage, category, and aggressive posterior ROP (P<.05). Statistically significant changes in specificity for identification of stage 2 or worse (P=.027) and pre-plus (P=.028) were observed.

Conclusions: A tele-education system for ROP education is effective in improving diagnostic accuracy of ROP by ophthalmology residents. This system may have utility in the setting of both healthcare and medical education reform by creating a validated method to certify telemedicine providers and educate the next generation of ophthalmologists.

FIGURE 1

Design of a clinical case in the retinopathy of prematurity (ROP) tele-education system. Each clinical case consisted of two eyes from one patient. Demographic information was provided to the trainee. Trainees were asked to provide a clinical diagnosis of plus disease (no, pre-plus, plus), zone (I, II, III), ROP (yes, no), stage (1–5), category (none, mild, type 2 ROP or pre-plus, treatment-requiring ROP), and aggressive posterior ROP (yes, no) for each individual eye. Trainees had the option to view OD and OS images simultaneously. For each individual color fundus photograph, a larger-resolution (1024×768) photograph was displayed if the trainee selected that photograph.

FIGURE 2

Educational material within the tutorial on retinopathy of prematurity (ROP) diagnosis. Individual slides were developed by the coauthors for different classifications of ROP diagnosis (plus, zone, stage, category, aggressive posterior ROP) and pertinent management considerations (treatment, follow-up time). Color fundus photographs included in the tutorial were annotated to highlight relevant findings. References to articles on ROP classification criteria and management considerations were provided to the trainees for additional information.

FIGURE 3

Case results after completion of the clinical case within the training chapter section of the retinopathy of prematurity (ROP) tele-education system. Top, Trainees were given immediate feedback on which aspects of their ROP diagnosis (plus, zone, stage, category, aggressive posterior ROP) were correct and incorrect. They were shown both their answer and the correct answer. Bottom, Trainees were able to review all available images from the preceding case in the case results section.

FIGURE 4

Feedback section after viewing case results within the chapter section of the retinopathy of prematurity (ROP) tele-education system. Top, For each aspect of the trainee’s incorrect ROP diagnosis (plus, zone, stage, category, aggressive posterior ROP) from the clinical case, a feedback section was generated from an educational repository of annotated images along with images from the clinical case that was just completed. Images from the case were annotated in this section to outline specific pathology that the trainee should have recognized to make the correct diagnosis. For incorrect answers, the dynamic feedback section also included examples of the incorrect response to help the trainees differentiate similar diagnoses. This feedback section was repeated for each aspect of the incorrect ROP diagnosis in each eye (blue arrow). Bottom, Images within the feedback section could also be selected to generate a larger-resolution image for better display.

FIGURE 5

Trainee options after completing a training chapter section of the retinopathy of prematurity (ROP) tele-education program. After trainees completed the 5 clinical cases within a training chapter, they were able to (1) re-review the cases from current chapter (blue arrow); (2) refer back to the ROP tutorial (yellow arrow and yellow box); (3) proceed to the next chapter (black arrow); or (4) save their results and exit the program.

FIGURE 6

Survey results of trainees who completed the retinopathy of prematurity (ROP) tele-education program (N=17). Survey questions after completion of the ROP tele-education program were rated using a Likert scale of “strongly agree” to “strongly disagree.” The questions included in the survey were: (1) “I had an adequate understanding of the diagnosis of ROP before taking the pretest in the ROP Student Module.” (2) “The ROP Student Module was easy to use.” (3) “I learned from the feedback provided at the end of each case.” (4) “I had an adequate understanding of the diagnosis of ROP after completing the ROP Student Module.” (5) “I learn more effectively in a Web-based environment compared to a traditional textbook format.” and (6) “I learned more effectively from ROP cases with automatic feedback compared to a traditional textbook format.”