Identifying relapse predictors in individual participant data with decision trees

Abstract

Background: Depression is a highly common and recurrent condition. Predicting who is at most risk of relapse or recurrence can inform clinical practice. Applying machine-learning methods to Individual Participant Data (IPD) can be promising to improve the accuracy of risk predictions.

Methods: Individual data of four Randomized Controlled Trials (RCTs) evaluating antidepressant treatment compared to psychological interventions with tapering ([Formula: see text]) were used to identify predictors of relapse and/or recurrence. Ten baseline predictors were assessed. Decision trees with and without gradient boosting were applied. To study the robustness of decision-tree classifications, we also performed a complementary logistic regression analysis.

Results: The combination of age, age of onset of depression, and depression severity significantly enhances the prediction of relapse risk when compared to classifiers solely based on depression severity. The studied decision trees can (i) identify relapse patients at intake with an accuracy, specificity, and sensitivity of about 55% (without gradient boosting) and 58% (with gradient boosting), and (ii) slightly outperform classifiers that are based on logistic regression.

Conclusions: Decision tree classifiers based on multiple-rather than single-risk indicators may be useful for developing treatment stratification strategies. These classification models have the potential to contribute to the development of methods aimed at effectively prioritizing treatment for those individuals who require it the most. Our results also underline the existing gaps in understanding how to accurately predict depressive relapse.

Keywords: Decision tree; Depression; Gradient boosting; Individual participant data; Logistic regression; Machine learning; Meta analysis; Relapse.

Fig. 1

Decision tree–based multi-factor analysis. a A decision tree with a depth of three was trained on a dataset of 380 patient samples. In each node, the notation “X vs. Y samples” represents the counts of X non-relapse and Y relapse patients. The nodes are color-coded as orange or blue, denoting the dominant group in terms of non-relapse or relapse patients. The leaf nodes display the labels “relapse” or “no relapse”, indicating the predictions associated with the corresponding decision paths. The values of “age” and “age of onset” are provided in years. b Normalized confusion matrix associated with the decision tree shown in panel (a)

Fig. 2

Performance comparison. a, b Accuracy (black disks), specificity (blue diamonds), and sensitivity (red squares) as a function of tree depth [in (a) for basic decision trees and in (b) for gradient-boosted trees]. Dashed lines in panels (a) and (b) represent the corresponding performance indicators of a classifier that is based on the HAMD score at intake and logistic regression, respectively. The training and test datasets consist of 380 and 163 samples, respectively. Markers in panels (a, b) indicate mean values that have been obtained using 1000 cross-validation realizations. Error bars indicate the corresponding standard errors. c–f Distributions of accuracy, specificity, and sensitivity for different classifiers

Fig. 3

Decision tree feature importance. a Feature importance (i.e., the relative frequency at which a certain feature occurs in a trained decision-tree classifier) associated with a decision tree with a depth of three. b Feature importance associated with a gradient-boosted tree of depth one. The shown results are based on 1000 cross-validation realizations. The training dataset consist of 380 samples. In both box plots, red lines show the median feature importance. Outliers are represented by black circles