Telemedicine diagnosis of cytomegalovirus retinitis by nonophthalmologists

Abstract

Importance: Cytomegalovirus (CMV) retinitis continues to be a leading cause of blindness in many developing countries. Telemedicine holds the potential to increase the number of people screened for CMV retinitis, but it is unclear whether nonophthalmologists could be responsible for interpreting fundus photographs captured in a telemedicine program.

Objective: To determine the accuracy of nonophthalmologist photographic graders in diagnosing CMV retinitis from digital fundus photographs.

Design, setting, and participants: Fifteen nonexpert graders each evaluated 182 mosaic retinal images taken from the eyes of patients with AIDS who were evaluated at the Ocular Infectious Diseases Clinic at Chiang Mai University in Chiang Mai, Thailand.

Main outcomes and measures: Graders diagnosed each image as CMV retinitis present, CMV retinitis absent, or unknown. The results from each grader were compared with those of an indirect ophthalmoscopic examination from an experienced on-site ophthalmologist as well as with the consensus grade given by a panel of CMV retinitis experts.

Results: Relative to the on-site ophthalmologist, the sensitivity of remote CMV retinitis diagnosis by nonexpert graders ranged from 64.0% to 95.5% (mean, 84.1%; 95% CI, 78.6%-89.6%)), and the specificity ranged from 65.6% to 92.5% (mean, 82.3%; 95% CI, 76.6%-88.0%)). Agreement between nonexpert and expert graders was high: the mean sensitivity and specificity values of nonexpert diagnosis using expert consensus as the reference standard were 93.2% (95% CI, 90.6%-95.8%) and 88.4% (95% CI, 85.4%-91.1%), respectively. Mean intrarater reliability also was high (mean Cohen κ, 0.83; 95% CI, 0.78-0.87).

Conclusions and relevance: The sensitivity and specificity of remote diagnosis of CMV retinitis by nonexpert graders was variable, although several nonexperts achieved a level of accuracy comparable to that of CMV retinitis experts. More intensive training and periodic evaluations would be required if nonexperts are to be used in clinical practice.

Figure 1.. Sensitivity and Specificity of Remote Nonexpert Diagnosis of Cytomegalovirus (CMV) Retinitis on Mosaic Fundus Photographs

Estimates are shown for each of 15 nonexperts and 3 CMV retinitis experts as well as the mean of the nonexperts. Graders had clinical experience (clinical), research experience (research), or neither clinical nor research experience (neither).

Figure 2.. Positive Predictive Values (PPVs) for Cytomegalovirus Retinitis Present (+CMV Retinitis) and Unknown, and Negative Predictive Values (NPVs) for CMV Retinitis Absent (−CMV Retinitis)

Estimates are shown over a range of likely CMV retinitis prevalence values for each of the 15 nonexpert graders (solid thin lines), the mean of nonexpert graders (solid thick lines), and the mean of the expert graders (dashed lines).

Figure 3.. Intrarater Reliability for Nonexpert Telemedicine Diagnosis of Cytomegalovirus Retinitis

The Cohen κ value is shown for each nonexpert grader based on grades from 50 randomly selected duplicate images; graders had clinical experience (clinical), research experience (research), or neither clinical nor research experience (neither). Limit lines indicate 95% CI.

Figure 4.. Sensitivity and Specificity of Remote Nonexpert Grading to Diagnose Cytomegalovirus Retinitis Using Expert Consensus as the Reference Standard

Estimates are shown for each of 15 nonexperts as well as the mean of the nonexperts. Graders had clinical experience (clinical), research experience (research), or neither clinical nor research experience (neither).