Predicting High Coronary Artery Calcium Score From Retinal Fundus Images With Deep Learning Algorithms

Abstract

Purpose: To evaluate high accumulation of coronary artery calcium (CAC) from retinal fundus images with deep learning technologies as an inexpensive and radiation-free screening method.

Methods: Individuals who underwent bilateral retinal fundus imaging and CAC score (CACS) evaluation from coronary computed tomography scans on the same day were identified. With this database, performances of deep learning algorithms (inception-v3) to distinguish high CACS from CACS of 0 were evaluated at various thresholds for high CACS. Vessel-inpainted and fovea-inpainted images were also used as input to investigate areas of interest in determining CACS.

Results: A total of 44,184 images from 20,130 individuals were included. A deep learning algorithm for discrimination of no CAC from CACS >100 achieved area under receiver operating curve (AUROC) of 82.3% (79.5%-85.0%) and 83.2% (80.2%-86.3%) using unilateral and bilateral fundus images, respectively, under a 5-fold cross validation setting. AUROC increased as the criterion for high CACS was increased, showing a plateau at 100 and losing significant improvement thereafter. AUROC decreased when fovea was inpainted and decreased further when vessels were inpainted, whereas AUROC increased when bilateral images were used as input.

Conclusions: Visual patterns of retinal fundus images in subjects with CACS > 100 could be recognized by deep learning algorithms compared with those with no CAC. Exploiting bilateral images improves discrimination performance, and ablation studies removing retinal vasculature or fovea suggest that recognizable patterns reside mainly in these areas.

Translational relevance: Retinal fundus images can be used by deep learning algorithms for prediction of high CACS.

Keywords: coronary artery calcium score; deep learning; retinal fundus images.

Figure 1.

Comparison of area under the receiver operating curve (AUROC) with different input types and varying thresholds for high coronary artery calcium score (CACS). While the threshold of high CACS was set to 0, 100, 200, 300, and 400, normal CACS was defined as CACS = 0. AUROC increased monotonically as vessels-inpainted < fovea-inpainted < fundus (one eye) < fundus (two eyes) at all threshold values, and statistical significance was maintained for distant pairs of input types. For instance, if AUROC differed significantly between vessels-inpainted images and fovea-inpainted images, AUROC of vessel-inpainted images also differed significantly compared with that of unilateral fundus images as well as bilateral fundus images. For the sake of visual clarity, the nearest pairs are marked with asterisks when there exist statistically significant margins between two different input types (P < 0.05).

Figure 2.

Receiver operating curve curves for (a) various threshold values for high coronary artery calcium score (CACS) when input is retinal fundus image, and (b) various input types when high CACS threshold was set to 100 (no CACS vs. CACS > 100). Discriminating regime is magnified.

Figure 3.

Exemplar heatmaps for deep learning algorithms that discriminate coronary artery calcium score (CACS) = 0 from CACS > 100 with different input types. Heatmaps for fovea-inpainted and vessels-inpainted images were compared with those of intact fundus images. Regardless of input types, the deep learning algorithms mainly attend to central main retinal branches to make the binary decision with respect to the abnormality in CACS. The attention tends to be more diffused when vessels were removed and inpainted.