DeepFundus: A flow-cytometry-like image quality classifier for boosting the whole life cycle of medical artificial intelligence

Abstract

Medical artificial intelligence (AI) has been moving from the research phase to clinical implementation. However, most AI-based models are mainly built using high-quality images preprocessed in the laboratory, which is not representative of real-world settings. This dataset bias proves a major driver of AI system dysfunction. Inspired by the design of flow cytometry, DeepFundus, a deep-learning-based fundus image classifier, is developed to provide automated and multidimensional image sorting to address this data quality gap. DeepFundus achieves areas under the receiver operating characteristic curves (AUCs) over 0.9 in image classification concerning overall quality, clinical quality factors, and structural quality analysis on both the internal test and national validation datasets. Additionally, DeepFundus can be integrated into both model development and clinical application of AI diagnostics to significantly enhance model performance for detecting multiple retinopathies. DeepFundus can be used to construct a data-driven paradigm for improving the entire life cycle of medical AI practice.

Keywords: artificial intelligence; image quality; retinal diseases.

Graphical abstract

Figure 1

Overall study design (A) Data collection, including retinal fundus images and slit-lamp images (obtained only from TAH), was conducted from 27 distinct cohorts for 4 years. (B) Fundus images are used to develop DeepFundus, a deep-learning-based system for the identification of fundus images with 13 different types of quality defects. (C) DeepFundus is integrated into an AI diagnostic system for removing unqualified fundus images before diagnosis, and its performances in good-quality and poor-quality image groups are compared. SCES, South China Eye Screening Program; NOS, Guangdong Neuro-ophthalmology Study; TAH, The Third Affiliated Hospital, Sun Yat-sen University; XJH, The People’s Hospital of Xinjiang Uygur Autonomous Region; AI, artificial intelligence.

Figure 2

Performance of DeepFundus on the national validation dataset (A) After model development, DeepFundus was externally tested using a national validation dataset prospectively collected from 16 clinic-based and 6 community-based cohorts across China. (B) On the national validation dataset, DeepFundus achieved AUCs of 0.911–0.985 for detecting images of poor overall quality; AUCs of 0.965–0.997 for detecting images of poor position concerning the overall image; AUCs of 0.924–0.991 for detecting images of poor illumination concerning the overall image; and AUCs of 0.922–0.971 for detecting images of poor clarity concerning the overall image in 7 different Chinese regions. AUC, area under the receiver operating characteristic curve.

Figure 3

Performance of established models in the detection of retinal diseases using different model architectures (A–C) ROCs for the detection of optic disc edema using InceptionV3, InceptionResNetV2, and DenseNet. (D–F) ROCs for the detection of drusen using InceptionV3, InceptionResNetV2, and DenseNet. (G–I) ROCs for the detection of diabetic retinopathy using InceptionV3, InceptionResNetV2, and DenseNet. ROC, receiver operating characteristic; AUC, area under the receiver operating characteristic curve.

Figure 4

Clinical application of DeepFundus Each fundus photograph collected in real-world settings will receive DeepFundus classification in terms of clinical quality factors, refractive media opacity, and structural quality analysis before entering downstream analysis. This system can also provide effective adjustments in real time for image acquisition based on quality analysis. These functions allow DeepFundus to serve as a data management tool in the whole life cycle of medical AI.