Development of machine learning models for detection of vision threatening Behçet's disease (BD) using Egyptian College of Rheumatology (ECR)-BD cohort

Abstract

Background: Eye lesions, occur in nearly half of patients with Behçet's Disease (BD), can lead to irreversible damage and vision loss; however, limited studies are available on identifying risk factors for the development of vision-threatening BD (VTBD). Using an Egyptian college of rheumatology (ECR)-BD, a national cohort of BD patients, we examined the performance of machine-learning (ML) models in predicting VTBD compared to logistic regression (LR) analysis. We identified the risk factors for the development of VTBD.

Methods: Patients with complete ocular data were included. VTBD was determined by the presence of any retinal disease, optic nerve involvement, or occurrence of blindness. Various ML-models were developed and examined for VTBD prediction. The Shapley additive explanation value was used for the interpretability of the predictors.

Results: A total of 1094 BD patients [71.5% were men, mean ± SD age 36.1 ± 10 years] were included. 549 (50.2%) individuals had VTBD. Extreme Gradient Boosting was the best-performing ML model (AUROC 0.85, 95% CI 0.81, 0.90) compared with logistic regression (AUROC 0.64, 95%CI 0.58, 0.71). Higher disease activity, thrombocytosis, ever smoking, and daily steroid dose were the top factors associated with VTBD.

Conclusions: Using information obtained in the clinical settings, the Extreme Gradient Boosting identified patients at higher risk of VTBD better than the conventional statistical method. Further longitudinal studies to evaluate the clinical utility of the proposed prediction model are needed.

Keywords: Behçet’s disease; Machine learning; SHAP analysis; Vision-threatening BD.

Fig. 1

Schematic presentation of the main path of machine learning process used in the current study

Fig. 2

Overall SHAP values for the variables in Shapely plots to display both the feature importance and feature contribution to the model prediction. Shapley plots show the SHAP values in the order of the important variables that contribute to VTBD. The x-axis represents the marginal contribution of a feature to the change in the predicted probability of development of VTBD. Colors indicate the value of the variable: red represents higher numerical values of the variable and blue represent lower numerical values. As all categorical variables were converted into binary indicators, zero (i.e., absence) is indicated with blue dots and one (i.e., presence) is represented by red dots

Fig. 3

XGBoost variable importance for predicting VTBD. The x-axis shows how much each feature added or subtracted to the final probability value for VTBD development. Please note that the numbers presented are average contributions for each feature to the model prediction

Fig. 4

XGBoost variable importance and overall SHAP values for predicting VTBD among women and men with BD. Figure shows degree and direction of contribution of the variables to VTBD from the Shapley plot separated by gender. Shapley plots show the SHAP values in the order of the important variables that contribute to VTBD (left side) and by the direction of the contribution (right side). A Represents the variables importance in women, while B shows the average probability value of each variables in the contribution of VTBD in women. C Represents the variables importance and D shows the average probability value of each variables in the contribution of VTBD in men