Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Jun 16:16:876927.
doi: 10.3389/fninf.2022.876927. eCollection 2022.

FN-OCT: Disease Detection Algorithm for Retinal Optical Coherence Tomography Based on a Fusion Network

Zhuang Ai  1 , Xuan Huang  2   3 , Jing Feng  2 , Hui Wang  2 , Yong Tao  2 , Fanxin Zeng  4 , Yaping Lu  1
Affiliations

FN-OCT: Disease Detection Algorithm for Retinal Optical Coherence Tomography Based on a Fusion Network

Zhuang Ai et al. Front Neuroinform. .

Abstract

Optical coherence tomography (OCT) is a new type of tomography that has experienced rapid development and potential in recent years. It is playing an increasingly important role in retinopathy diagnoses. At present, due to the uneven distributions of medical resources in various regions, the uneven proficiency levels of doctors in grassroots and remote areas, and the development needs of rare disease diagnosis and precision medicine, artificial intelligence technology based on deep learning can provide fast, accurate, and effective solutions for the recognition and diagnosis of retinal OCT images. To prevent vision damage and blindness caused by the delayed discovery of retinopathy, a fusion network (FN)-based retinal OCT classification algorithm (FN-OCT) is proposed in this paper to improve upon the adaptability and accuracy of traditional classification algorithms. The InceptionV3, Inception-ResNet, and Xception deep learning algorithms are used as base classifiers, a convolutional block attention mechanism (CBAM) is added after each base classifier, and three different fusion strategies are used to merge the prediction results of the base classifiers to output the final prediction results (choroidal neovascularization (CNV), diabetic macular oedema (DME), drusen, normal). The results show that in a classification problem involving the UCSD common retinal OCT dataset (108,312 OCT images from 4,686 patients), compared with that of the InceptionV3 network model, the prediction accuracy of FN-OCT is improved by 5.3% (accuracy = 98.7%, area under the curve (AUC) = 99.1%). The predictive accuracy and AUC achieved on an external dataset for the classification of retinal OCT diseases are 92 and 94.5%, respectively, and gradient-weighted class activation mapping (Grad-CAM) is used as a visualization tool to verify the effectiveness of the proposed FNs. This finding indicates that the developed fusion algorithm can significantly improve the performance of classifiers while providing a powerful tool and theoretical support for assisting with the diagnosis of retinal OCT.

Keywords: attention mechanism; fusion network; model interpretability; optical coherence tomography; retinal disease.

PubMed Disclaimer

Conflict of interest statement

ZA and YL were employed by Sinopharm Genomics Technology Co., Ltd. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as potential conflicts of interest.

Figures

Figure 1
Figure 1
FN. ‘Class’ is the number of categories required by the algorithm, ‘CBAM’ is the attention mechanism, ‘Fc + Softmax’ is the fully connected output layer, “Add” is an addition operation, and ‘Concatenate’ is the splicing operation. (A) Is the first fusion (FN-F1-OCT), (B) is the second fusion (FN-Weight-OCT), and (C) is the third fusion (FN-Auto-OCT). The network architecture is drawn by using ‘PlotNeuralNet’ (https://github.com/HarisIqbal88/PlotNeuralNet).
Figure 2
Figure 2
Model building process.
Figure 3
Figure 3
Dataset preparation. (A) Retinal OCT staging distribution of the samples; (B) representative fundus photographs of each sampling category according to their clinical diagnoses.
Figure 4
Figure 4
Training processes of the three fusion strategies. (A,C,E) represent the loss value and accuracy changes yielded on the training set and test set by the Inception, Inception-ResNet, and Xception base classifiers in FN-F1-OCT during transfer learning. (B,D,F) represent the fine-tuning of the models according to (A,C,E) in the training process, which is done to obtain the loss value and accuracy changes induced on the training set and test set. (G,I) represent the loss value and accuracy changes induced on the training set and test set by the FN-Weight-OCT and FN-Auto-OCT fusion strategies during transfer learning. (H,J) indicate that the models are fine-tuned according to (G,I) during the training process to obtain the loss value and accuracy changes induced on the training set and test set.
Figure 5
Figure 5
ROC curves of the three fusion strategies. (A–C) represent the ROC curves of FN-F1-OCT, FN-Weight-OCT, and FN-Auto-OCT, respectively.
Figure 6
Figure 6
Comparison of different fusion strategies on external test datasets.
Figure 7
Figure 7
Localization map visualization.
See this image and copyright information in PMC

References

    1. Al-Antari M. A., Al-Masni M. A., Kim T. S. (2020). Deep learning computer-aided diagnosis for breast lesion in digital mammogram. Adv. Exp. Med. Biol. 1213, 59–72. 10.1007/978-3-030-33128-3_4 - DOI - PubMed
    1. Albarrak A., Coenen F., Zheng Y. (2013). Age-related macular degeneration identification in volumetric optical coherence tomography using decomposition and local feature extraction, in Proceedings of 2013 International Conference on Medical Image, Understanding and Analysis (Birmingham: University of Birmingham; ), 59–64.
    1. Ali I., Muzammil M., Haq I. U., Khaliq A. A., Abdullah S. (2021). Deep feature selection and decision level fusion for lungs nodule classification. IEEE Access 9, 18962–18973. 10.1109/ACCESS.2021.3054735 - DOI
    1. Alsaih K., Lemaître G., Vall J. M., Rastgoo M., Sidib,é D., Wong T. Y., et al. (2016). Classification of sd-oct volumes with multi pyramids, lbp and hog descriptors: application to dme detections, in 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (Orlando, FL: IEEE; ), 1344–1347. - PubMed
    1. Apostolopoulos S., Ciller C., Sznitman R., Zanet S. D. (2017). Simultaneous classification and segmentation of cysts in retinal oct, in Proceedings of MICCAI Retinal OCT Fluid Challenge (RETOUCH) (Montreal, QC: ), 22–29.