Artificial intelligence method based on multi-feature fusion for automatic macular edema (ME) classification on spectral-domain optical coherence tomography (SD-OCT) images

Abstract

Purpose: A common ocular manifestation, macular edema (ME) is the primary cause of visual deterioration. In this study, an artificial intelligence method based on multi-feature fusion was introduced to enable automatic ME classification on spectral-domain optical coherence tomography (SD-OCT) images, to provide a convenient method of clinical diagnosis.

Methods: First, 1,213 two-dimensional (2D) cross-sectional OCT images of ME were collected from the Jiangxi Provincial People's Hospital between 2016 and 2021. According to OCT reports of senior ophthalmologists, there were 300 images with diabetic (DME), 303 images with age-related macular degeneration (AMD), 304 images with retinal-vein occlusion (RVO), and 306 images with central serous chorioretinopathy (CSC). Then, traditional omics features of the images were extracted based on the first-order statistics, shape, size, and texture. After extraction by the alexnet, inception_v3, resnet34, and vgg13 models and selected by dimensionality reduction using principal components analysis (PCA), the deep-learning features were fused. Next, the gradient-weighted class-activation map (Grad-CAM) was used to visualize the-deep-learning process. Finally, the fusion features set, which was fused from the traditional omics features and the deep-fusion features, was used to establish the final classification models. The performance of the final models was evaluated by accuracy, confusion matrix, and the receiver operating characteristic (ROC) curve.

Results: Compared with other classification models, the performance of the support vector machine (SVM) model was best, with an accuracy of 93.8%. The area under curves AUC of micro- and macro-averages were 99%, and the AUC of the AMD, DME, RVO, and CSC groups were 100, 99, 98, and 100%, respectively.

Conclusion: The artificial intelligence model in this study could be used to classify DME, AME, RVO, and CSC accurately from SD-OCT images.

Keywords: SD-OCT images; artificial intelligence; classification models; macular edema; multi-feature fusion.

FIGURE 1

The flowchart of this study.

FIGURE 2

The pie chart for traditional omics features distribution.

FIGURE 3

Gradient-weighted class-activation map (Grad-CAM) visualization of deep learning feature extraction: CSC (A); AMD (B); DME (C); RVO (D). The blue part that gathers inward from the red part is active, indicating that the model pays particular attention to this area (Huang et al., 2022).

FIGURE 4

Feature visualization by t-distributed stochastic neighbor embedding (t-SNE): AMD (red); DME (green); RVO (blue); CSC (purple).

FIGURE 5

Feature selection in the lasso model: (A) Lasso coefficient profiles of the 162 fusion features, where each curve corresponds to one feature, the vertical black line indicates an optimal λ. (B) Curve of binomial deviation varied with parameter λ, where value of the optimal log (λ) is marked by vertical dashed lines.

FIGURE 6

Confusion matrix and ROC curve of the different models: (A1–E1): Confusion matrix of the SVM, LR, KNN, MLP, and ExtraTrees model, respectively. Each row of the matrix represents the actual class and each column indicates the predicted class. (A2–E2): ROC curve of SVM, LR, KNN, MLP, and ExtraTrees model, respectively. Label 0 for AMD, label 1 for DME, label 2 for RVO, label 3 for CSC.