Multimodal workflows optimally predict response to repetitive transcranial magnetic stimulation in patients with schizophrenia: a multisite machine learning analysis

Abstract

The response variability to repetitive transcranial magnetic stimulation (rTMS) challenges the effective use of this treatment option in patients with schizophrenia. This variability may be deciphered by leveraging predictive information in structural MRI, clinical, sociodemographic, and genetic data using artificial intelligence. We developed and cross-validated rTMS response prediction models in patients with schizophrenia drawn from the multisite RESIS trial. The models incorporated pre-treatment sMRI, clinical, sociodemographic, and polygenic risk score (PRS) data. Patients were randomly assigned to receive active (N = 45) or sham (N = 47) rTMS treatment. The prediction target was individual response, defined as ≥20% reduction in pre-treatment negative symptom sum scores of the Positive and Negative Syndrome Scale. Our multimodal sequential prediction workflow achieved a balanced accuracy (BAC) of 94% (non-responders: 92%, responders: 95%) in the active-treated group and 50% in the sham-treated group. The clinical, clinical + PRS, and sMRI-based classifiers yielded BACs of 65%, 76%, and 80%, respectively. Apparent sadness, inability to feel, educational attainment PRS, and unemployment were most predictive of non-response in the clinical + PRS model, while grey matter density reductions in the default mode, limbic networks, and the cerebellum were most predictive in the sMRI model. Our sequential modelling approach provided superior predictive performance while minimising the diagnostic burden in the clinical setting. Predictive patterns suggest that rTMS responders may have higher levels of brain grey matter in the default mode and salience networks which increases their likelihood of profiting from plasticity-inducing brain stimulation methods, such as rTMS. The future clinical implementation of our models requires findings to be replicated at the international scale using stratified clinical trial designs.

Fig. 1. Schematic diagram of the main analysis design of our study.

Boxes represent key analysis stages of our study, arrows represent the order of the analysis stages.

Fig. 2. Feature importance and predictive patterns extracted from the Clinical+PRS model and sMRI model.

A All features from Clinical + PRS model ranked by absolute CVR values in ascending order. The vertical red lines indicate CVR value at −2 and 2 which are equivalent to p = 0.05. B Predictive pattern analysis for the clinical and sociodemographic features used in the Clinical + PRS model, ranked by absolute CVR values in ascending order. CVR subplot vertical red lines: CVR equivalence to alpha level of 0.05 (|2.2|), Sign-based consistency subplot vertical red line: −log10 p equivalence to alpha level of 0.05 (1.3). C Predictive pattern analysis for the PRS features used in the Clinical + PRS model, ranked by absolute CVR values in ascending order. CVR subplot vertical red lines: CVR equivalence to alpha level of 0.05 (−2.2), Sign consistency subplot vertical red line: −log10 p equivalence to alpha level of 0.05 (1.3). All exact values can be found in Supplementary S15. D The reliability of the Grey Matter Density (GMD) pattern elements was measured in terms of a Cross-Validation Ratio (CVR) map (CVR = mean(w)/standard error(w)], where w are the weight vectors of the 5054 Support Vector Machine (SVM) models generated in the study’s repeated nested cross-validation setup). The CVR map was thresholded at CVR value ranges corresponding to an alpha level of 0.01 (CVR ≤ −3, CVR ≥ 3). Reliable areas of GMD increase in predicting responders to active rTMS are shaded in red colours, whereas areas of GMD increments predicting non-responders to active rTMS are painted in green. The open-source 3D rendering software MRIcroGL (C. Rohrden) available at https://www.nitrc.org/projects/mricrogl/ was used to overlay the CVR map on the MNI single-subject template.

Fig. 3. Post-hoc model performance analyses for all RESIS active models.

A Model performance measures for all RESIS active models. B Step-wise BAC performance increase observed in the models in the active treatment group vs. models trained in the sham treatment group in both pooled CV and leave-one-site-out CV (Sequential model not included). C Comparison of linear correlations between patients’ predicted likelihood of non-response to rTMS treatment from sMRI model and sequential model and PANSS-NS score reduction from baseline to 21 days after rTMS treatment (upper R-squared: sMRI model, lower R-squared: sequential model). D Comparison of linear correlations between patients’ prediction decision scores from sMRI model and sequential model and PANSS-NS score reduction from baseline to 21 days after rTMS treatment (upper R-squared: sMRI model, lower R-squared: sequential model). E Model performance measures for each prognostic node of the sequential prognostic system (Supplementary S8). F The percentage of cases which are propagated at each step of the step-wise sequential model trained on the active treatment group.