Ensemble machine learning prediction of posttraumatic stress disorder screening status after emergency room hospitalization

Abstract

Posttraumatic stress disorder (PTSD) develops in a substantial minority of emergency room admits. Inexpensive and accurate person-level assessment of PTSD risk after trauma exposure is a critical precursor to large-scale deployment of early interventions that may reduce individual suffering and societal costs. Toward this aim, we applied ensemble machine learning to predict PTSD screening status three months after severe injury using cost-effective and minimally invasive data. Participants (N = 271) were recruited at a Level 1 Trauma Center where they provided variables routinely collected at the hospital, including pulse, injury severity, and demographics, as well as psychological variables, including self-reported current depression, psychiatric history, and social support. Participant zip codes were used to extract contextual variables including population total and density, average annual income, and health insurance coverage rates from publicly available U.S. Census data. Machine learning yielded good prediction of PTSD screening status 3 months post-hospitalization, AUC = 0.85 95% CI [0.83, 0.86], and significantly outperformed all benchmark comparison models in a cross-validation procedure designed to yield an unbiased estimate of performance. These results demonstrate that good prediction can be attained from variables that individually have relatively weak predictive value, pointing to the promise of ensemble machine learning approaches that do not rely on strong isolated risk factors.

Keywords: Computational psychiatry; Emergency room; Machine learning; PTSD; Personalized prognosis; Precision medicine; Prevention; Trauma.

Figure 1.

Average gain across cross-validated tests of the full model indicates the proportion that a feature contributed toward the ensemble’s prediction of PTSD screening status. Shading indicates the source of each feature, and the error bars show 95% CI. Hospital features were extracted from the hospital intake. Census features were extracted from publicly available census data based on participant zip codes. Psychological features were extracted from the following: patients self-reported history of mood and anxiety diagnoses, current depression was assessed with the Patient Health Questionnaire-8, physical and mental health functioning was assessed with the Veterans RAND 12-item Health survey, social support was assessed with the Social Provisions Scale, resilience was assessed with the Connor Davidson Resilience Scale, and alcohol use was assessed with the Alcohol Use Disorder Identification Test-Consumption.

Figure 2.

Nine features with gain > 0.05. Locally weighted smoothing (loess) curves show how the probability of PTSD+ prediction (y-axis) varies as a function of a feature (x-axis) after controlling for all other features. Note the y-axis range, which was selected to illustrate the narrow range of influence that single features have on prediction.

Figure 3.

A) Sensitivity as a function of 1-specificity for the full model (all features) and comparison models including logistic regression with PTSD severity at the hospital as the only predictor, and machine learning with only features routinely collected at the hospital. AUC statistics refer to area under these curves, which is significantly higher for the full model. The non-overlapping shaded areas represent improvement in prediction. The dashed line corresponds to a no-information model, which yields AUC = 0.50. B) Distribution of predicted probabilities of developing PTSD from the full model. The x-axis refers to mean test-sample prediction for each patient, and frequency bars are shaded by actual PTSD screening status at 3 months. The gray bars to the left of the 50% threshold represent the number of accurate PTSD− predictions, and the black bars to the right represent accurate PTSD+ predictions. The models yield a wide arrange of personalized predictions that are highly accurate at the extreme ends but less accurate near the 50% threshold, which should be considered for guiding treatment decisions.