Validation of DIGIROP models and decision support tool for prediction of treatment for retinopathy of prematurity on a contemporary Swedish cohort

Abstract

Background/aims: Retinopathy of prematurity (ROP) is currently diagnosed through repeated eye examinations to find the low percentage of infants that fulfil treatment criteria to reduce vision loss. A prediction model for severe ROP requiring treatment that might sensitively and specifically identify infants that develop severe ROP, DIGIROP-Birth, was developed using birth characteristics. DIGIROP-Screen additionally incorporates first signs of ROP in different models over time. The aim was to validate DIGIROP-Birth, DIGIROP-Screen and their decision support tool on a contemporary Swedish cohort.

Methods: Data were retrieved from the Swedish national registry for ROP (2018-2019) and two Swedish regions (2020), including 1082 infants born at gestational age (GA) 24 to <31 weeks. The predictors were GA at birth, sex, standardised birth weight and age at the first sign of ROP. The outcome was ROP treatment. Sensitivity, specificity and area under the receiver operating characteristic curve (AUC) with 95% CI were described.

Results: For DIGIROP-Birth, the AUC was 0.93 (95% CI 0.90 to 0.95); for DIGIROP-Screen, it ranged between 0.93 and 0.97. The specificity was 49.9% (95% CI 46.7 to 53.0) and the sensitivity was 96.5% (95% CI 87.9 to 99.6) for the tool applied at birth. For DIGIROP-Screen, the cumulative specificity ranged between 50.0% and 78.7%. One infant with Beckwith-Wiedemann syndrome who fulfilled criteria for ROP treatment and had no missed/incomplete examinations was incorrectly flagged as not needing screening.

Conclusions: DIGIROP-Birth and DIGIROP-Screen showed high predictive ability in a contemporary Swedish cohort. At birth, 50% of the infants born at 24 to <31 weeks of gestation were predicted to have low risk of severe ROP and could potentially be released from ROP screening examinations. All routinely screened treated infants, excluding those screened for clinical indications of severe illness, were correctly flagged as needing ROP screening.

Keywords: diagnostic tests/investigation; retina.

Figure 1

DIGIROP-Birth risk estimates and achieved sensitivity and specificity (Swedish Contemporary Validation Cohort 2018–2020). ROP, retinopathy of prematurity.

Figure 2

Specificity and cumulative specificity for DIGIROP decision support tool, (A) by gestational age at birth based on DIGIROP-Birth and (B) by postnatal age based on DIGIROP-Birth and DIGIROP-Screen (Swedish Contemporary Validation Cohort 2018–2020).

Figure 3

Receiver operating characteristic (ROC) curves for estimates obtained by DIGIROP-Birth and DIGIROP-Screen at PNAs 8, 10, 12 and 14 weeks (Swedish Contemporary Validation Cohort 2018–2020). AUC, area under the curve; PNA, postnatal age.