Evaluation of Sensitivity and Specificity of Acuity 360 Telemedicine Vision Screening System

Abstract

Purpose: Our aim is to evaluate the sensitivity and specificity of Acuity 360 telemedicine system, as compared to in-person clinic examination, in identifying clinically significant eye disease. Acuity 360 is a combination of commercially available ocular imaging devices used together to provide a comprehensive evaluation of the structures and diseases of the eye.

Methods: Observational cross-sectional study of consecutively examined patients where 19 remote examiners analyzed 80 patients using Acuity 360 images. The examiners' diagnoses were compared to the diagnosis obtained from in-person clinic examination of all the patients. The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), false-negative rate, and inter-rater agreement were calculated.

Results: Compared to in-person clinic examination, the sensitivity, specificity, PPV, NPV, and false-negative rate for the optical coherence tomography (OCT) of optic nerve were 95.8%, 98.5%, 86.8%, 99.6%, 0.39% and for the combined retina examination (OCT of the macula and widefield fundus photography) were 93.4%, 88.5%, 80.2%, 96.4%, 2.19%, respectively. The median inter-rater agreement the OCT of optic nerve and the combined retina examination were each 95%.

Conclusion: The Acuity 360 telemedicine system has a low false-negative rate and is highly sensitive and specific when compared to an in-person clinic examination. It can determine the necessity for specialist referral and triage the patients that require urgent treatment. High inter-rater agreement shows that it is effective with minimal variability in analyzing the Acuity 360 images by remote examiners.

Keywords: optical coherence tomography; sensitivity; specificity; tele-ophthalmology; telemedicine.

Figure 1

Box and whisker plots of the inter-rater agreement of the imaging modalities used in Acuity 360 vision screening system. OCT: optical coherence tomography. The top and bottom of the boxes are the 75th and 25th percentiles of the data. There are no top whiskers as third quartile is equal to the maximum data value in all the three imaging modalities. The line drawn through the middle of the box is the median (50th percentile). The box contains the middle 50% of the data. The length of the box is the inter-quartile range. Bottom whiskers indicate the minimum data values. Round dots are the outliers.