Interobserver agreement in the interpretation of single-field digital fundus images for diabetic retinopathy screening

Abstract

Purpose: To assess agreement among a group of ophthalmic care providers, including ophthalmologists and trained nonphysician personnel, in the interpretation of single-field digital fundus images for diabetic retinopathy screening.

Design: Interobserver reliability study.

Participants: Twelve ophthalmic care personnel, including 3 retina specialists, 3 general ophthalmologists, 3 ophthalmic nurses, and 3 ophthalmic photographers.

Methods: All participants were to read 400 good single-field digital fundus images of diabetic patients from a community hospital. The nonphysician personnel group read the images 1 month after attending a 2-day intensive instruction course regarding diabetic retinopathy screening. The ophthalmologists read the images without additional training. The 3 retina specialists read the images again together 2 months later to form a consensus regarding retinopathy severity and macular edema for each case. All readers used the Early Treatment Diabetic Retinopathy Study standard photographs as guidelines.

Main outcome measures: The kappa statistic was used for the reliability assessment of the diabetic retinopathy severity and macular edema, and for the identification of cases that needed referral to ophthalmologists.

Results: There is only fair agreement among all readers. The multirater kappa coefficient for retinopathy severity is 0.34; for macular edema, 0.27; and for referral cases, 0.28. Retina specialists have the best agreement among all groups (kappa = 0.58 for retinopathy severity or macular edema, kappa = 0.63 for referrals). There is also fair agreement when all readers are compared with the consensus of retina specialists (kappas = 0.35, 0.28, and 0.29 for retinopathy severity, macular edema, and referrals, respectively), and the retina specialist group also has the best agreement (kappas = 0.63, 0.65, and 0.67 for retinopathy severity, macular edema, and referrals).

Conclusions: Without additional training, retina specialists may be the most reliable personnel to interpret single-field digital fundus images for diabetic retinopathy screening. For other ophthalmic care personnel to achieve comparable reliability, a comprehensive instruction course with specific continuing education is essential. Authorized nonphysician interpreters should be experts, and new standard photographs for single-field digital fundus image interpretation may also be required to improve interobserver reliability.