Validity of Machine Learning in Predicting Giant Cell Arteritis Flare After Glucocorticoids Tapering

Abstract

Background: Inferential statistical methods failed in identifying reliable biomarkers and risk factors for relapsing giant cell arteritis (GCA) after glucocorticoids (GCs) tapering. A ML approach allows to handle complex non-linear relationships between patient attributes that are hard to model with traditional statistical methods, merging them to output a forecast or a probability for a given outcome.

Objective: The objective of the study was to assess whether ML algorithms can predict GCA relapse after GCs tapering.

Methods: GCA patients who underwent GCs therapy and regular follow-up visits for at least 12 months, were retrospectively analyzed and used for implementing 3 ML algorithms, namely, Logistic Regression (LR), Decision Tree (DT), and Random Forest (RF). The outcome of interest was disease relapse within 3 months during GCs tapering. After a ML variable selection method, based on a XGBoost wrapper, an attribute core set was used to train and test each algorithm using 5-fold cross-validation. The performance of each algorithm in both phases was assessed in terms of accuracy and area under receiver operating characteristic curve (AUROC).

Results: The dataset consisted of 107 GCA patients (73 women, 68.2%) with mean age ( ± SD) 74.1 ( ± 8.5) years at presentation. GCA flare occurred in 40/107 patients (37.4%) within 3 months after GCs tapering. As a result of ML wrapper, the attribute core set with the least number of variables used for algorithm training included presence/absence of diabetes mellitus and concomitant polymyalgia rheumatica as well as erythrocyte sedimentation rate level at GCs baseline. RF showed the best performance, being significantly superior to other algorithms in accuracy (RF 71.4% vs LR 70.4% vs DT 62.9%). Consistently, RF precision (72.1%) was significantly greater than those of LR (62.6%) and DT (50.8%). Conversely, LR was superior to RF and DT in recall (RF 60% vs LR 62.5% vs DT 47.5%). Moreover, RF AUROC (0.76) was more significant compared to LR (0.73) and DT (0.65).

Conclusions: RF algorithm can predict GCA relapse after GCs tapering with sufficient accuracy. To date, this is one of the most accurate predictive modelings for such outcome. This ML method represents a reproducible tool, capable of supporting clinicians in GCA patient management.

Keywords: algorithm; giant cell (temporal) arteritis; glucocorticoids; machine learning; precision medicine.

Figure 1

Attribute core set used for training and validation of the algorithms ranked for feature importance score. DM, Diabetes mellitus; ESR, Erythrocyte Sedimentation Rate; PMR, Polymyalgia Rheumatica.

Figure 2

Receiver operating characteristic curves of the assessed algorithms. AUROC, Area under the receiver operating characteristic curve; DT, Decision tree; LR, Logistic Regression; RF, Random Forest.

Figure 3

A sample decision tree among those included into RF. At each node data are split according to ESR, presence/absence DM or presence/absence of PMR. DM, Diabetes mellitus; ESR, Erythrocyte Sedimentation Rate; PMR, Polymyalgia Rheumatica.

Figure 4

Calibration curve before (upper panel) and after (lower panel) isotonic calibration. After calibration, GCA flare roughly happened with an observed relative frequency (dotted line) consistent with the forecast value (solid orange line), showing an acceptable calibration.