P Score: A Reference Image-Based Clinical Grading Scale for Vascular Change in Retinopathy of Prematurity

Abstract

Purpose: The International Classification of Retinopathy of Prematurity, Third Edition (ICROP3), acknowledged that plus-like retinopathy of prematurity (ROP) vascular changes occurs along a spectrum. Historically, clinician-experts demonstrate variable agreement for plus diagnosis. We developed a 9-photograph reference image set for grading plus-like changes and compared intergrader agreement of the set with standard grading with no plus, preplus, and plus disease.

Design: Retinal photographic grading and expert consensus opinion.

Participants: The development set included 34 international ICROP3 committee members. The validation set included 30 ophthalmologists with ROP expertise (15 ICROP3 committee members and 15 non-ICROP3 members) METHODS: Nine ROP fundus images (P1 through P9) representing increasing degrees of zone I vascular tortuosity and dilation, based on the 34 ICROP3 committee members' gradings and consensus image reviews, were used to establish standard photographs for the plus (P) score. Study participants graded 150 fundus photographs 2 ways, separated by a 1-week washout period: (1) no plus, preplus, or plus disease and (2) choosing the closest P score image.

Main outcome measures: Intergrader agreement measured by intraclass correlation coefficient.

Results: Intergrader agreement was higher using the P score (intraclass correlation coefficient, 0.75; 95% confidence interval, 0.71-0.79) than no plus, preplus, or plus disease (intraclass correlation coefficient, 0.67; 95% confidence interval, 0.62-0.72). Mean ± standard deviation P scores for images with mode gradings of no plus, preplus, and plus disease were 2.5 ± 0.7, 4.8 ± 0.8, and 7.4 ± 0.8, respectively.

Conclusions: Intergrader agreement of plus-like vascular change in ROP using the P score is high. We now incorporate this 9-image reference set into ICROP3 for use in clinician daily practice alongside zone, stage, and plus assessment. P score is not yet meant to replace plus diagnosis for treatment decisions, but its use at our institutions has permitted better comparison between examinations for progression and regression, communication between examiners, and documentation of vascular change without fundus imaging. P score also could provide more detailed ROP classification for clinical trials, consistent with the spectrum of plus-like change that is now formally part of the International Classification of Retinopathy of Prematurity.

Financial disclosure(s): Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Keywords: ICROP; ICROP3; P score; Plus disease; Retinopathy of prematurity.

Figure 1.

P-score reference images. The images are labelled P1 through P9, featuring increasing degrees of posterior pole retinal vasculature tortuosity and dilation.

Figure 2.

The proportion of P-score gradings that were within 1 P-score level (Y-axis), as a function of the mean P-score (X-axis), for each of 150 images graded by 30 graders. Gradings within 1 P score level of each other could be 1 lower, the same, or 1 higher P-score level, which ranges from P1 to P9.

Figure 3.

The mean P-scores, which range from P1 to P9 (X-axis), for 150 images grouped by mode no plus (n=68 images), pre-plus (n=62 images), or plus disease (n=20 images) grading (Y-axis). The P-scores and no/pre/plus gradings were made by 30 graders. The mode diagnosis on the Y axis refers to the most frequent diagnosis, no/pre/plus, for each image.

Figure 4.

Individual P-scores, which range from P1 to P9 (X-axis) for 150 images grouped by mode no plus, pre-plus, or plus disease grading (Y-axis). The P-scores and no/pre/plus gradings were made by 30 graders. The mode diagnosis on the Y axis refers to the most frequent diagnosis, no/pre/plus, for each image.

Figure 5.

Three images most commonly graded as P4, 3 images most commonly graded as P5, and and 3 images most commonly graded as P6 among 150 images classified by 30 graders using the 9 P-score reference images.