A Pilot Study Using Frequent Inpatient Assessments of Suicidal Thinking to Predict Short-Term Postdischarge Suicidal Behavior

Abstract

Importance: The weeks following discharge from psychiatric hospitalization are the highest-risk period for suicide attempts. Real-time monitoring of suicidal thoughts via smartphone prompts may be more indicative of short-term risk than a single, cross-sectional assessment.

Objective: To test whether modeling dynamic changes in real-time suicidal thoughts during psychiatric hospitalization can improve predictions of postdischarge suicide attempts vs using only baseline (ie, admission) data or using the mean level of real-time suicidal thoughts during hospitalization.

Design, setting, and participants: In this prognostic study, 83 adults recruited from the inpatient psychiatric unit at Massachusetts General Hospital completed ecological momentary assessment surveys of suicidal thinking 4 to 6 times per day during hospitalization as well as brief follow-up surveys assessing suicide attempts at 2 and 4 weeks after discharge. Participants completed at least 3 real-time monitoring surveys. Inclusion criteria included hospitalization for suicidal thoughts and/or behaviors and English fluency. Data were collected from January 2016 to December 2018 and analyzed from January to December 2020.

Main outcomes and measures: The primary outcome was suicide attempt in the month after discharge.

Results: Of 83 participants (mean [SD] age, 38.4 [13.6] years; 43 [51.8%] male participants; 69 [83.1%] White individuals), 9 (10.8%) made a suicide attempt in the month after discharge. Mean cross-validated AUC for elastic net models revealed predictive accuracy was fair for the model using baseline data (area under the curve [AUC], 0.71; first to third quartile, 0.55-0.88), good for the model using the mean level of real-time suicidal thoughts during hospitalization (AUC, 0.81; first to third quartile, 0.67-0.91), and best for the model using dynamic changes in real-time suicidal thoughts during hospitalization (AUC, 0.89; first to third quartile, 0.81-0.97); this pattern of results held for other classification metrics (eg, accuracy, positive predictive value, Brier score) and when using different cross-validation procedures. Features assessing rapid fluctuations in suicidal thinking emerged as the strongest predictors of posthospital suicide attempts. A final set of models incorporating percentage missingness further improved both the mean (mean AUC, 0.93; first to third quartile, 0.90-1.00) and dynamic feature (mean AUC, 0.93; first to third quartile, 0.88-1.00) models.

Conclusions and relevance: In this study, collecting real-time data about suicidal thinking during the course of hospitalization significantly improved short-term prediction of posthospitalization suicide attempts. Models including dynamic changes in suicidal thinking over time yielded the best prediction; features that captured rapid changes in suicidal thoughts were particularly strong predictors. Survey noncompletion also emerged as an important predictor of posthospitalization suicide attempts.

Figure 1.. Suicide Attempt Prediction Model Metrics

Each panel presents descriptive visualizations (violin plots) of the distribution of cross-validated model performance metrics. The darker shaded boxes within each violin plot are box plots, with the lower and upper hinges representing the first and third quartiles, respectively, and the whiskers extending to the most extreme data points within 1.5 times of the interquartile range. Data points beyond the whiskers are plotted as separate points. AUC indicates area under the curve; AUPRC, area under the precision-recall curve; and PPV, positive predictive value.

Figure 2.. Dynamic Feature Model Variable Importance

Importance scores are coefficients for the final model scaled from 0 to 100, with 0 indicating least important and 100 indicating most important. Max indicates maximum; min, minimum; PAC, probability of acute change; PUV, percentage unique values.

Figure 3.. Suicide Attempt Prediction Model Metrics, With Missingness Added

Each panel presents descriptive visualizations (violin plots) of the distribution of cross-validated model performance metrics. The darker shaded boxes within each violin plot are box plots, with the lower and upper hinges representing the first and third quartiles, respectively, and the whiskers extending to the most extreme data points within 1.5 times of the interquartile range. Data points beyond the whiskers are plotted as separate points. AUC indicates area under the curve; AUPRC, area under the precision-recall curve; and PPV, positive predictive value.