Vision transformer based interpretable metabolic syndrome classification using retinal Images

Abstract

Metabolic syndrome is leading to an increased risk of diabetes and cardiovascular disease. Our study developed a model using retinal image data from fundus photographs taken during comprehensive health check-ups to classify metabolic syndrome. The model achieved an AUC of 0.7752 (95% CI: 0.7719-0.7786) using retinal images, and an AUC of 0.8725 (95% CI: 0.8669-0.8781) when combining retinal images with basic clinical features. Furthermore, we propose a method to improve the interpretability of the relationship between retinal image features and metabolic syndrome by visualizing metabolic syndrome-related areas in retinal images. The results highlight the potential of retinal images in classifying metabolic syndrome.

Fig. 1. Model architecture.

The left and right retinal images of each participant were learned separately using RETFound, a Vision Transformer (ViT)-based model, to extract image features. These features are then concatenated with clinical features, and the combined features are used to classify them as either having metabolic syndrome or being normal through Logistic Regression. The model also employs a transformer-based explainability method to visualize which parts of the images are most indicative of metabolic syndrome, enhancing the interpretability of our model.

Fig. 2. BMI distribution for normal and metabolic syndrome participants, categorized into four groups based on their classification outcomes.

Group A is participants who were correctly classified both when using clinical features alone and when using combined clinical and retinal image features. Group B is participants misclassified both when using only clinical features and when using the combined clinical and retinal image features. Group C are participants correctly classified when using only clinical features but misclassified when using combined clinical and retinal image features. Group D are participants correctly classified when using combined features but misclassified when using only clinical features. The red line indicates a BMI of 25 kg/m².

Fig. 3. Feature importance scores for predicting normal or metabolic syndrome in four representative individuals.

Feature importance is expressed as the sum of SHAP values for each feature. A positive SHAP value indicates a positive correlation with the ground truth, while a negative SHAP value indicates a negative correlation. Information outside the plot represents the participant’s BMI, ground truth, and the results and probabilities predicted by the model (Pred: Prediction, Pred Prob: Prediction Probability). The red line indicates a BMI of 25 kg/m².

Fig. 4. Classification performance for associated diseases using clinical and retinal image features.

Classification of diseases associated with metabolic syndrome (hypertension, diabetes, dyslipidemia) was performed using clinical and retinal image features. The classification performances were evaluated using AUC. The $p$-value was used to show the statistical significance of combining clinical and retinal features, comparing the AUCs of models using retinal features alone, clinical features alone, and the combined model.

Fig. 5. Explainable visualizations of normal and metabolic syndrome.

Results were generated using the transformer-based explainability method. Each row shows a participant’s retinal images (left and right) and their explainable visualizations according to normal (a–e) or metabolic syndrome (f–j). The redder regions correspond to the greater relevance of each participant’s condition.