Individual-Level Prediction of Exposure Therapy Outcome Using Structural and Functional MRI Data in Spider Phobia: A Machine-Learning Study

Abstract

Machine-learning prediction studies have shown potential to inform treatment stratification, but recent efforts to predict psychotherapy outcomes with clinical routine data have only resulted in moderate prediction accuracies. Neuroimaging data showed promise to predict treatment outcome, but previous prediction attempts have been exploratory and reported small clinical sample sizes. Herein, we aimed to examine the incremental predictive value of neuroimaging data in contrast to clinical and demographic data alone (for which results were previously published), using a two-level multimodal ensemble machine-learning strategy. We used pretreatment structural and task-based fMRI data to predict virtual reality exposure therapy outcome in a bicentric sample of N = 190 patients with spider phobia. First, eight 1st-level random forest classifications were conducted using separate data modalities (clinical questionnaire scores and sociodemographic data, cortical thickness and gray matter volumes, functional activation, connectivity, connectivity-derived graph metrics, and BOLD signal variance). Then, the resulting predictions were used to train a 2nd-level classifier that produced a final prediction. No 1st-level or 2nd-level classifier performed above chance level except BOLD signal variance, which showed potential as a contributor to higher-level prediction from multiple regions across the brain (1st-level balanced accuracy = 0.63). Overall, neuroimaging data did not provide any incremental accuracy for treatment outcome prediction in patients with spider phobia with respect to clinical and sociodemographic data alone. Thus, we advise caution in the interpretation of prediction performances from small-scale, single-site patient samples. Larger multimodal datasets are needed to further investigate individual-level neuroimaging predictors of therapy response in anxiety disorders.

Trial registration: ClinicalTrials.gov NCT03208400.

Figure 1

Ensemble machine-learning classification pipeline. RF: random forest; ROI: region of interest; gPPI: generalized psychophysiological interaction; MRI: magnetic resonance imaging; BOLD: blood-oxygen-level-dependent.

Figure 2

Area under the receiving operating curves for treatment outcome classification. (a) 1st-level classification results. (b) 2nd-level classification results. gPPI: generalized psychophysiological interaction.

Figure 3

The Shapley (SHAP) values and feature importance of the 1st-level variance classifier in the responder (N = 103) vs. nonresponder (N = 87) prediction using functional features across ROIs covering the whole brain. Positive SHAP values indicate a contribution of feature value in favour of the positive class prediction (future responder); negative Shapley values are in favour of the negative class prediction (future nonresponder). Larger absolute Shapley values indicate a larger impact on the model output. The 20 most contributing features are shown and ranked in decreasing order of mean absolute SHAP value. Horizontal violin plots on the left represent the distribution of all individuals in the test set across all cross-validation iterations. For each feature, relative values are represented on the left by a color gradient.