Data analysis strategies for the Accelerating Medicines Partnership® Schizophrenia Program

Abstract

The Accelerating Medicines Partnership® Schizophrenia (AMP® SCZ) project assesses a large sample of individuals at clinical high-risk for developing psychosis (CHR) and community controls. Subjects are enrolled in 43 sites across 5 continents. The assessments include domains similar to those acquired in previous CHR studies along with novel domains that are collected longitudinally across a period of 2 years. In parallel with the data acquisition, multidisciplinary teams of experts have been working to formulate the data analysis strategy for the AMP SCZ project. Here, we describe the key principles for the data analysis. The primary AMP SCZ analysis aim is to use baseline clinical assessments and multimodal biomarkers to predict clinical endpoints of CHR individuals. These endpoints are defined for the AMP SCZ study as transition to psychosis (i.e., conversion), remission from CHR syndrome, and persistent CHR syndrome (non-conversion/non-remission) obtained at one year and two years after baseline assessment. The secondary aim is to use longitudinal clinical assessments and multimodal biomarkers from all time points to identify clinical trajectories that differentiate subgroups of CHR individuals. The design of the analysis plan is informed by reviewing legacy data and the analytic approaches from similar international CHR studies. In addition, we consider properties of the newly acquired data that are distinct from the available legacy data. Legacy data are used to assist analysis pipeline building, perform benchmark experiments, quantify clinical concepts and to make design decisions meant to overcome the challenges encountered in previous studies. We present the analytic design of the AMP SCZ project, mitigation strategies to address challenges related to the analysis plan, provide rationales for key decisions, and present examples of how the legacy data have been used to support design decisions for the analysis of the multimodal and longitudinal data. Watch Prof. Ofer Pasternak discuss his work and this article: https://vimeo.com/1023394132?share=copy#t=0 .

Fig. 1. Potential utility of individual-level inference models for the AMP SCZ study.

A central goal of AMP SCZ analyses is the development and validation of quantitative models with straightforward translatability to clinically useful applications. Group-based statistical approaches quantify group differences in observed variables to make statements about the distribution of those variables in a target population. In contrast, models for individual-level inferences combine observations at baseline, or over time, to make statements about previously unseen individuals. Here, several applications of individual-level inference models relevant to the AMP SCZ study are shown.

Fig. 2. Feature dimensionality and frequency in the AMP SCZ study.

The figure captures the dimensionality and the frequency of the different data domains within the AMP SCZ study for raw data (left) and after preprocessing and feature extraction (right). Measurement frequency and number of dimensions are presented on a logarithmic scale for clarity. Abbreviations: magnetic resonance imaging (MRI), electroencephalography (EEG), ecological momentary assessment (EMA), Positive SYmptoms and Diagnostic Criteria for the CAARMS Harmonized with the SIPS (PSYCHS).

Fig. 3. Key considerations in making an analysis pipeline.

A typical analysis pipeline involves several different steps. Each step depends on key considerations and requires optimization of performance that is specific for the study needs.

Fig. 4. Sample size analysis for predictive models.

The curves in the figure show the dependency of significance on sample size for 4 model scenarios differing by AUC value and conversion rate. Significance, quantified by a z-score, represents the test with a null hypothesis that the AUC is the same as chance level AUC (0.5). The associated significant levels of p = 0.05 and p = 10⁻³ are marked. The model scenarios are characterized by a minimal AUC (analogous to the effect size in traditional power calculations) and by a minimal conversion rate. For example, the red line describes a model with an AUC of 0.7, and conversion rate of 10%.

Fig. 5. Multimodal fusion of four modalities to predict development of psychosis.

A We used multiple kernel learning (MKL) to combine signals from four modalities, to predict development of psychosis from baseline data in individuals at CHR from the SHARP study. Our analysis fused demographic information with EEG, cognitive scores, and mean diffusivity maps obtained from diffusion weighted imaging (DWI-MD). B Using 25 repetitions of 4-fold validation we observed an advantage of MKL (AUC = 0.73) over individual modalities (logistic regression with elastic net, AUC = 0.69 for the best modality, which was Cognition). C Linear MKL provides a characteristic feature weight vector for each modality that is used to project individual samples. Stability is quantified by the variability in the feature weight vector across 100 training folds. D Weight vectors for DWI-MD from MKL show more stability (vary less) and additional information (less overlap) compared with weight vectors from DWI-MD unimodal models, and DWI-MD t-statistics vectors derived from univariate group comparisons. E Visualization of the correlation between prediction signals from different modalities (within converters). F Visualization of DWI-MD weight feature vectors derived from MKL provides interpretability (e.g., locations on the brain). Units are z-scores across training folds.

Fig. 6. Longitudinal analyses.

An example using dynamic time warping in the NAPLS-3 dataset. A Dynamic time warping (DTW) is a method that quantifies the dissimilarity (distance) of pairs of time series. Here, we provide a schematic diagram about DTW: Its key feature is that it is sensitive to the overall shape of trajectories, accommodating temporal delays between them or differences in number of time points. It uses a “stretching” and “compressing” mechanism to match corresponding time points across time series. Given a list of time courses, DTW produces a matrix of pairwise distances. B Multidimensional scaling (MDS) representation, a dimensionality reduction method based on preservation of pairwise distances, for individuals at CHR (blue: non-converters within years, red: converters) of SIPS-based positive symptoms at the baseline visit in NAPLS. C MDS representations of SIPS at baseline and month 2 visits in the same individuals, showing nascent differentiation between the groups, with converters away from the center. D MDS representation of whole SIPS trajectories using DTW shows most pronounced grouping of converters.