Prediction Models of Functional Outcomes for Individuals in the Clinical High-Risk State for Psychosis or With Recent-Onset Depression: A Multimodal, Multisite Machine Learning Analysis

Abstract

Importance: Social and occupational impairments contribute to the burden of psychosis and depression. There is a need for risk stratification tools to inform personalized functional-disability preventive strategies for individuals in at-risk and early phases of these illnesses.

Objective: To determine whether predictors associated with social and role functioning can be identified in patients in clinical high-risk (CHR) states for psychosis or with recent-onset depression (ROD) using clinical, imaging-based, and combined machine learning; assess the geographic, transdiagnostic, and prognostic generalizability of machine learning and compare it with human prognostication; and explore sequential prognosis encompassing clinical and combined machine learning.

Design, setting, and participants: This multisite naturalistic study followed up patients in CHR states, with ROD, and with recent-onset psychosis, and healthy control participants for 18 months in 7 academic early-recognition services in 5 European countries. Participants were recruited between February 2014 and May 2016, and data were analyzed from April 2017 to January 2018.

ain outcomes and measures: Performance and generalizability of prognostic models.

Results: A total of 116 individuals in CHR states (mean [SD] age, 24.0 [5.1] years; 58 [50.0%] female) and 120 patients with ROD (mean [SD] age, 26.1 [6.1] years; 65 [54.2%] female) were followed up for a mean (SD) of 329 (142) days. Machine learning predicted the 1-year social-functioning outcomes with a balanced accuracy of 76.9% of patients in CHR states and 66.2% of patients with ROD using clinical baseline data. Balanced accuracy in models using structural neuroimaging was 76.2% in patients in CHR states and 65.0% in patients with ROD, and in combined models, it was 82.7% for CHR states and 70.3% for ROD. Lower functioning before study entry was a transdiagnostic predictor. Medial prefrontal and temporo-parieto-occipital gray matter volume (GMV) reductions and cerebellar and dorsolateral prefrontal GMV increments had predictive value in the CHR group; reduced mediotemporal and increased prefrontal-perisylvian GMV had predictive value in patients with ROD. Poor prognoses were associated with increased risk of psychotic, depressive, and anxiety disorders at follow-up in patients in the CHR state but not ones with ROD. Machine learning outperformed expert prognostication. Adding neuroimaging machine learning to clinical machine learning provided a 1.9-fold increase of prognostic certainty in uncertain cases of patients in CHR states, and a 10.5-fold increase of prognostic certainty for patients with ROD.

Conclusions and relevance: Precision medicine tools could augment effective therapeutic strategies aiming at the prevention of social functioning impairments in patients with CHR states or with ROD.

Figure 1.. Comparison of Functional Baseline and Combined Model Signatures

The predictive value of baseline global functioning scores used by models with significant associations with functional outcomes was measured in terms of the variable selection frequency across all the support-vector machine models generated in the nested leave-site-out cross-validation experiment. A value of 1 indicates that all models had retained the given variable during sequential backward feature elimination. Horizontal bar plots show the variable selection profiles of the clinical models making predictions of social functioning scores (A) and role functioning scores (B) in the group in the clinical high-risk (CHR) state and the model trained on social functioning scores from patients with recent-onset depression (ROD) (C), with orange lines at 0.5, which equals 50% of support-vector machine models’ selected given variable. Reliability profiles of the combined social functioning model, trained in the patients in CHR states (D) and patients with ROD (E); and a reliability profile model trained on role functioning scores from patients with ROD (F), with orange lines at a cross-validation ratio of 2, which indicates 95% confidence in the reliable involvement of given variable in the model's decision rule. MRI indicates magnetic resonance imaging.

Figure 2.. Comparison of Predictive Neuroanatomical Baseline Signatures in Patient Groups, Detected by the Structural Magnetic Resonance Imaging–Based Model

The reliability of predictive voxels in significant models was measured via a cross-validation ratio map with a threshold of ± 2, which corresponded to an α level of .05. Color scales indicate increased vs decreased gray matter volume in individuals in the clinical high-risk state or with recent-onset depression who were impaired on follow-up, compared with patients with no impairment on follow-up. The open-source 3-dimensional rendering software MRIcroGL (McCausland Center for Brain Imaging, University of South Carolina; https://www.nitrc.org/projects/mricrogl/) was used to overlay the cross-validation ratio maps on the Montreal Neurological Institute single-participant template and produce 3-dimensional renderings and axial mosaic slices. The cool color scale indicates increased gray matter volume and the warm color scale reduced gray matter volume in individuals in clinical high-risk states or with recent-onsent depression who were impaired on follow-up, compared with patients with no impairment at follow-up.