Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Aug 1;140(8):791-798.
doi: 10.1001/jamaophthalmol.2022.2135.

External Validation of a Retinopathy of Prematurity Screening Model Using Artificial Intelligence in 3 Low- and Middle-Income Populations

Aaron S Coyner  1 , Minn A Oh  1 , Parag K Shah  2 , Praveer Singh  3   4 , Susan Ostmo  1 , Nita G Valikodath  5 , Emily Cole  5 , Tala Al-Khaled  5 , Sanyam Bajimaya  6 , Sagun K C  7 , Tsengelmaa Chuluunbat  8 , Bayalag Munkhuu  8 , Prema Subramanian  2 , Narendran Venkatapathy  2 , Karyn E Jonas  5 , Joelle A Hallak  5 , R V Paul Chan  5 , Michael F Chiang  9 , Jayashree Kalpathy-Cramer  3   4 , J Peter Campbell  1
Affiliations

External Validation of a Retinopathy of Prematurity Screening Model Using Artificial Intelligence in 3 Low- and Middle-Income Populations

Aaron S Coyner et al. JAMA Ophthalmol. .

Abstract

Importance: Retinopathy of prematurity (ROP) is a leading cause of preventable blindness that disproportionately affects children born in low- and middle-income countries (LMICs). In-person and telemedical screening examinations can reduce this risk but are challenging to implement in LMICs owing to the multitude of at-risk infants and lack of trained ophthalmologists.

Objective: To implement an ROP risk model using retinal images from a single baseline examination to identify infants who will develop treatment-requiring (TR)-ROP in LMIC telemedicine programs.

Design, setting, and participants: In this diagnostic study conducted from February 1, 2019, to June 30, 2021, retinal fundus images were collected from infants as part of an Indian ROP telemedicine screening program. An artificial intelligence (AI)-derived vascular severity score (VSS) was obtained from images from the first examination after 30 weeks' postmenstrual age. Using 5-fold cross-validation, logistic regression models were trained on 2 variables (gestational age and VSS) for prediction of TR-ROP. The model was externally validated on test data sets from India, Nepal, and Mongolia. Data were analyzed from October 20, 2021, to April 20, 2022.

Main outcomes and measures: Primary outcome measures included sensitivity, specificity, positive predictive value, and negative predictive value for predictions of future occurrences of TR-ROP; the number of weeks before clinical diagnosis when a prediction was made; and the potential reduction in number of examinations required.

Results: A total of 3760 infants (median [IQR] postmenstrual age, 37 [5] weeks; 1950 male infants [51.9%]) were included in the study. The diagnostic model had a sensitivity and specificity, respectively, for each of the data sets as follows: India, 100.0% (95% CI, 87.2%-100.0%) and 63.3% (95% CI, 59.7%-66.8%); Nepal, 100.0% (95% CI, 54.1%-100.0%) and 77.8% (95% CI, 72.9%-82.2%); and Mongolia, 100.0% (95% CI, 93.3%-100.0%) and 45.8% (95% CI, 39.7%-52.1%). With the AI model, infants with TR-ROP were identified a median (IQR) of 2.0 (0-11) weeks before TR-ROP diagnosis in India, 0.5 (0-2.0) weeks before TR-ROP diagnosis in Nepal, and 0 (0-5.0) weeks before TR-ROP diagnosis in Mongolia. If low-risk infants were never screened again, the population could be effectively screened with 45.0% (India, 664/1476), 38.4% (Nepal, 151/393), and 51.3% (Mongolia, 266/519) fewer examinations required.

Conclusions and relevance: Results of this diagnostic study suggest that there were 2 advantages to implementation of this risk model: (1) the number of examinations for low-risk infants could be reduced without missing cases of TR-ROP, and (2) high-risk infants could be identified and closely monitored before development of TR-ROP.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest Disclosures: The i-ROP DL system has been licensed by Massachusetts General Hospital and Oregon Health & Science University to Boston AI and Siloam Vision and may result in royalties to Drs. Coyner, Chan, Kalpathy-Cramer, and Campbell in the future. These conflicts are managed by each respective institution. Dr Hallak reported being an employee of AbbVie outside the submitted work. Dr Chan reported being an equity owner of Siloam Vision outside the submitted work. Dr Chiang reported receiving research support from the National Institutes of Health, the National Science Foundation, and Genentech; personal fees from Novartis; and having equity from InTeleretina outside the submitted work. Dr Kalpathy-Cramer reported receiving grants from the National Institutes of Health, GE, Bayer, and Genentech outside the submitted work. Dr Campbell reported receiving research support from the National Institutes of Health, the National Science Foundation, and Genentech outside the submitted work; consulting fees from Boston AI; and being an equity owner of Siloam Vision outside the submitted work. No other disclosures were reported.

Figures

Figure.
Figure.. Vascular Severity Score vs Postmenstrual Age
Beginning at approximately 30 weeks’ postmenstrual age, eyes from the India training data set that eventually developed treatment-requiring retinopathy of maturity and were treated had vascular severity scores that began to increase and diverge from those who did not.
See this image and copyright information in PMC

Comment in

References

    1. Good WV, Hardy RJ, Dobson V, et al. ; Early Treatment for Retinopathy of Prematurity Cooperative Group . The incidence and course of retinopathy of prematurity: findings from the early treatment for retinopathy of prematurity study. Pediatrics. 2005;116(1):15-23. doi:10.1542/peds.2004-1413 - DOI - PubMed
    1. Early Treatment for Retinopathy Of Prematurity Cooperative Group . Revised indications for the treatment of retinopathy of prematurity: results of the early treatment for retinopathy of prematurity randomized trial. Arch Ophthalmol. 2003;121(12):1684-1694. doi:10.1001/archopht.121.12.1684 - DOI - PubMed
    1. Fierson WM; American Academy of Pediatrics Section on Ophthalmology; American Academy of Ophthalmology; American Association for Pediatric Ophthalmology and Strabismus; American Association of Certified Orthoptists . Screening examination of premature infants for retinopathy of prematurity. Pediatrics. 2018;142(6):e20183061. doi:10.1542/peds.2018-3061 - DOI - PubMed
    1. Chiang MF, Quinn GE, Fielder AR, et al. . International classification of retinopathy of prematurity, third edition. Ophthalmology. 2021;128(10):e51-e68. doi:10.1016/j.ophtha.2021.05.031 - DOI - PMC - PubMed
    1. Lawn JE, Davidge R, Paul VK, et al. . Born too soon: care for the preterm baby. Reprod Health. 2013;10(suppl 1):S5. doi:10.1186/1742-4755-10-S1-S5 - DOI - PMC - PubMed

Publication types