Toward Addiction Prediction: An Overview of Cross-Validated Predictive Modeling Findings and Considerations for Future Neuroimaging Research

Abstract

Substance use is a leading cause of disability and death worldwide. Despite the existence of evidence-based treatments, clinical outcomes are highly variable across individuals, and relapse rates following treatment remain high. Within this context, methods to identify individuals at particular risk for unsuccessful treatment (i.e., limited within-treatment abstinence), or for relapse following treatment, are needed to improve outcomes. Cumulatively, the literature generally supports the hypothesis that individual differences in brain function and structure are linked to differences in treatment outcomes, although anatomical loci and directions of associations have differed across studies. However, this work has almost entirely used methods that may overfit the data, leading to inflated effect size estimates and reduced likelihood of reproducibility in novel clinical samples. In contrast, cross-validated predictive modeling (i.e., machine learning) approaches are designed to overcome limitations of traditional approaches by focusing on individual differences and generalization to novel subjects (i.e., cross-validation), thereby increasing the likelihood of replication and potential translation to novel clinical settings. Here, we review recent studies using these approaches to generate brain-behavior models of treatment outcomes in addictions and provide recommendations for further work using these methods.

Keywords: Abstinence; Biomarker; Classification; Connectivity; Regression; Substance use disorders.

Figure 1 –

Example workflow for clinical predictive modeling in addiction Summary of basic steps of predictive modeling (17), modified to emphasize specific aspects of predictive modeling in addiction: Neuroimaging data and clinical data are separated into independent training and testing datasets. Training data are then submitted to a predictive modeling algorithm to identify the most relevant features from the data (e.g., brain regions associated with the clinical outcome). Using the identified features, a mathematical model is generated to map the (typically high dimensional) neuroimaging features onto the (typically low dimensional) clinical outcome. The model is then applied to previously unseen data from the testing dataset to generate individual-level clinical predictions. Finally, the model’s performance is evaluated by comparing predicted and actual clinical outcomes in the testing dataset. Results are used to update current neurobiological models and to inform development of novel treatments. Tx=treatment; ROI=region of interest; NOI-network of interest; LOOCV=leave-one-out cross validation; CV=cross validation

Figure 2 –

Steps to maximize neurobiological interpretation of clinical neuroimaging models To maximize interpretability, model features should be considered across multiple levels. Left: Model features are summarized across descending levels of dimensionality. Individual connections (edge level) are summarized by: (i) overlap with macroscale brain regions (node level), (ii) overlap with canonical neural networks (network level) and (iii) a simplified network model of core systems contributing to cocaine abstinence (theory level). Right: Recommended steps for maximizing interpretation of neurobiological models for region-of-interest (ROI, top) and whole-brain (connectome-based, bottom) approaches are summarized. In both cases, post-hoc analyses involving ‘virtual lesioning’ of selected features is strongly recommended to guide interpretation of model elements. Elements of this figure adapted from (9) and are reprinted with permission from the American Journal of Psychiatry, (Copyright ©2019). American Psychiatric Association. All Rights Reserved.

Figure 3 –

Sensitivity vs. specificity within the context of clinical addiction prediction The relative importance of sensitivity versus specificity may depend on the clinical outcome. Confusion matrices showing rates of true positives (TP), false positives (FP), false negatives (FN) and true negatives (TN) are shown for models (1) with high sensitivity and low specificity (left) and; (2) with low sensitivity and high specificity (right) are illustrated for two different clinical outcomes: (1) assignment to active treatment (top) and; (2) termination of active treatment (bottom). Cells in red bold font correspond to individuals at increased risk for overdose. For outcome 1, overdose risk is minimized when sensitivity is maximized. For outcome 2, overdose risk is minimized when specificity is maximized.