Optical coherence tomography grading reproducibility during the Comparison of Age-related Macular Degeneration Treatments Trials

Abstract

Objective: To report reading center reproducibility during grading of Stratus optical coherence tomography (OCT) (Carl Zeiss Meditec, Dublin, CA) images obtained during the Comparison of Age-Related Macular Degeneration Treatments Trials (CATT).

Design: Prospective, clinical trial.

Participants: Independent reading teams reevaluated 270 OCT scans randomly sampled from the first 2 years of CATT enrollment. To assess temporal drift, a cohort of 23 scans submitted during the initial portion of the CATT study was longitudinally followed with serial reproducibility analysis.

Intervention: The CATT readers performed standardized grading of OCT images. A reader team, composed of 2 independent readers and a senior reader, evaluated each scan. Grading included the CATT OCT end points of total thickness at the foveal center point and intraretinal fluid (IRF), subretinal fluid (SRF), and subretinal pigment epithelium (RPE) fluid. Independent reading teams masked to the results of initial grading reevaluated scans to determine the reproducibility of qualitative grading and measurements.

Main outcome measures: Categorical grading agreement was reported using percent agreement and kappa statistic, and measurement agreement was reported using intraclass correlations and paired differences.

Results: Reading center teams reproducibly graded IRF (percent agreement = 73%, kappa = 0.48; 95% confidence interval [CI], 0.38-0.58), SRF (percent agreement = 90%; kappa = 0.80; 95% CI, 0.73-0.87), and sub-RPE fluid (percent agreement 88%; kappa = 0.75; 95% CI, 0.67-0.83). For independent reading center team measurements of total thickness at the foveal center point, the intraclass correlation was 0.99 (95% CI, 0.99-0.99), and the mean paired difference between reading center teams was 4 μm (95% limits of agreement, -55 to 47 μm). There was no qualitative or quantitative grading drift.

Conclusions: The standardized protocols used to evaluate OCT scans from the CATT study were reproducible. The methods used are suitable to monitor OCT imaging data from a large, neovascular age-related macular degeneration, interventional, multicenter study.

Financial disclosure(s): The author(s) have no proprietary or commercial interest in any materials discussed in this article.

Figure 1

a – f: Representative morphologic features from Optical Coherence Tomography (OCT) images produced by the macular thickness map (MTM) protocol: 1a. Obvious vitreomacular attachment (VMA) - vitreous attachment and focal separation from the inner retina within a 3 mm diameter horizontal region centered at the middle of the fovea, 1b. Subtle VMA, 1c. Obvious epiretinal membrane (ERM), 1d. Subtle ERM, 1e. Obvious intraretinal fluid (IRF), and 1f. Subtle IRF.

Figure 2

a – f: Representative morphologic features from Optical Coherence Tomography (OCT) images produced by the macular thickness map (MTM) protocol: 2a. Obvious subretinal fluid (SRF), 2b. Subtle SRF, 2c. Obvious subretinal hyperreflective material (SHRM), 2d. Subtle SHRM, 2e. Obvious retinal pigment epithelium elevation (RPEE), 2f. Subtle RPEE.

Figure 3

a – i: Examples of grading subcategories for morphologic features noted on Optical Coherence Tomography (OCT) images produced by the macular thickness map (MTM) protocol: 3a. Epiretinal membrane (ERM) present with any deformation of the central 1 mm (horizontal dimension) of the retina, 3b. Vitreomacular attachment (VMA) present with any deformation of the central 1 mm of the retina, 3c. Any intraretinal fluid (IRF) present within central 1 mm of the retina, 3d. IRF present at the foveal center point, 3e. Any subretinal fluid (SRF) present within central 1 mm of the retina, 3f. SRF present at the foveal center point, 3g. Any sub-retinal pigment epithelium (RPE) fluid present within central 1 mm of the retina, 3h. Sub-RPE fluid present at the foveal center point, 3i. Any subretinal hyperreflective material (SHRM) present within central 1 mm of the retina.

Figure 4

Representative scan demonstrating choroidal neovascular membrane – pigment epithelial detachment (CNV – PED) thickness measurement at foveal center point. The measurement was performed from the outer boundary of Bruch’s membrane to the inner boundary of the CNV.

Figure 5

OCT scan work-flow demonstrated for reproducibility analysis between hypothetical Reading Center “Team A” and “Team B” when grading the same scan. All team analyses were performed more than 5 weeks after initial reading. solid lines = initial grading, dashed lines = reproducibility analysis. *Readers were randomly selected from pool of 8 readers, and no individual reader was part of both team A and B. **Senior Readers were randomly selected from pool of 4 Senior Readers, and no individual Senior Reader was part of both team A and B.

Figure 6

a – b: Comparison of single largest intraretinal cystoid hyporeflective cross sectional area from optical coherence tomography scans: 6a.Median single largest intraretinal cystoid hyporeflective cross sectional area where only one Reading Team reported intraretinal fluid as present, 6b. Median single largest intraretinal cystoid hyporeflective cross sectional area for scans where both Reading Teams agreed on presence of intraretinal fluid.