Predicting Suicides Among US Army Soldiers After Leaving Active Service

Abstract

Importance: The suicide rate of military servicemembers increases sharply after returning to civilian life. Identifying high-risk servicemembers before they leave service could help target preventive interventions.

Objective: To develop a model based on administrative data for regular US Army soldiers that can predict suicides 1 to 120 months after leaving active service.

Design, setting, and participants: In this prognostic study, a consolidated administrative database was created for all regular US Army soldiers who left service from 2010 through 2019. Machine learning models were trained to predict suicides over the next 1 to 120 months in a random 70% training sample. Validation was implemented in the remaining 30%. Data were analyzed from March 2023 through March 2024.

Main outcome and measures: The outcome was suicide in the National Death Index. Predictors came from administrative records available before leaving service on sociodemographics, Army career characteristics, psychopathologic risk factors, indicators of physical health, social networks and supports, and stressors.

Results: Of the 800 579 soldiers in the cohort (84.9% male; median [IQR] age at discharge, 26 [23-33] years), 2084 suicides had occurred as of December 31, 2019 (51.6 per 100 000 person-years). A lasso model assuming consistent slopes over time discriminated as well over all but the shortest risk horizons as more complex stacked generalization ensemble machine learning models. Test sample area under the receiver operating characteristic curve ranged from 0.87 (SE = 0.06) for suicides in the first month after leaving service to 0.72 (SE = 0.003) for suicides over 120 months. The 10% of soldiers with highest predicted risk accounted for between 30.7% (SE = 1.8) and 46.6% (SE = 6.6) of all suicides across horizons. Calibration was for the most part better for the lasso model than the super learner model (both estimated over 120-month horizons.) Net benefit of a model-informed prevention strategy was positive compared with intervene-with-all or intervene-with-none strategies over a range of plausible intervention thresholds. Sociodemographics, Army career characteristics, and psychopathologic risk factors were the most important classes of predictors.

Conclusions and relevance: These results demonstrated that a model based on administrative variables available at the time of leaving active Army service can predict suicides with meaningful accuracy over the subsequent decade. However, final determination of cost-effectiveness would require information beyond the scope of this report about intervention content, costs, and effects over relevant horizons in relation to the monetary value placed on preventing suicides.

Figure 1.. Age Period Cohort Hex Map of Suicides per 100 000 Years Among the 800 579 Regular Army Soldiers Who Separated 2010 to 2019

P indicates calendar year; A, years since leaving the regular Army (0 = [0, 1], 1 = [1, 2], 2 = [2, 3], etc); C, approximation of separation cohort (eg, C: 2011 may include someone who left September 2010, died April 2011, and was out of service less than 1 year at time of death).

Figure 2.. Decision Curves for Net Benefit (NB) of the Lasso Model Trained Over a 120-Month Horizon Relative to the Strategies of Intervene-With-All and Intervene-With-None for a Range of Risk Horizons

The x-axis is expressed in terms of standardized suicide risk at the intervention threshold, where 1 indicates that the intervention is delivered to individuals with a suicide risk greater than the average for the population over the horizon. Note that population risk per 100 000 person-years differs across horizons. The y-axis expresses NB in terms of the number of individuals receiving the intervention who would otherwise die by suicide per person-year out of 100 000 receiving the intervention discounted by the number of individuals receiving the intervention who would not otherwise die by suicide. As NB is expressed per year and cumulative risk increases with time, a preventive intervention with a stable effect would be expected to prevent roughly 10 times as many postseparation suicides over 120 months, as over 12 months discounted by the decrease in PPV at longer horizons and by the decrease in the denominator population because of deaths due to competing risks. So, for example, a preventive intervention delivered shortly before separation to the 5% of soldiers at highest predicted risk could prevent no more than 21.1 suicides per 100 000 soldiers during the first month after leaving service but up to 16.0 per month, or a total of 1920, over 10 years. As a result, the appropriate decision threshold for an intervention will depend not only on the expected effect size but also on the expected duration and decay of the effect over time. The discount rate is defined by p / (1 − p) at the intervention threshold, where p is the unstandardized intervention threshold (ie, absolute suicide risk). The NB curve for intervening with all also differs across risk horizons but this difference is invisible to the naked eye because there is no significant variation in comparative prediction effects by time since leaving active service.

Figure 3.. Predictor Importance Based on Kernel Shapley Additive Explanations (SHAP) Values of the Lasso Model Trained Over a 120-Month Risk Horizon

^aOdds ratios (ORs) are for standardized versions of the predictors (ie, transformed to a mean of 0 and variance of 1) with odds of suicide. ^bSHAP values were estimated in the training sample based on a logistic regression model estimated in the test sample that included the 65 predictors selected by the lasso model in the training sample to predict individual-level predicted probabilities of suicide based on the balanced weighted training sample in which mean predicted probability of suicide was 0.50. The SHAP value for the total model was 0.102, which means that the average predicted probabilities if all 65 predictors were set to their means would have been in the range 0.398 to 0.602. The SHAP_P values reported here for the most important predictors in each category are expressed as proportions of 0.102. Only the 5 predictors with highest SHAP_P in each category are reported along with SHAP_P values for all predictors in the category. Only category totals are reported for the 2 categories where none of the predictors had SHAP_P values of at least 5.0%. ^cCollege graduate or more = 3, some college = 2, high school diploma = 1, less than high school, including General Educational Diploma = 0. ^dDirect combat arms (eg, infantry) = 3, indirect combat arms (eg, artillery) = 2, combat support (eg, combat engineer) = 1, combat service support (eg, quartermaster) = 0. ^eArmy Forces Qualifying Test (AFQT) score is expressed in percentiles based on equal weighting of percentile scores on separate tests of arithmetical reasoning, mathematics knowledge, word knowledge, and reading comprehension. ^fInternational Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis codes 291.XX, 303.XX, or 305.0X or International Statistical Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) diagnosis codes F10.10, F10.12X, F10.14, F10.15X, F10.18X, F10.19, F10.2XX, F10.92X, or F10.94-F10.99 associated with the visit. ^gICD-9-CM diagnosis codes 290.XX-319 or ICD-10-CM diagnosis code F01.XXX-F99 associated with the admission/visit. ^hAny of the following on record since 2004: a US Department of Defense Suicide Event Report record for suicidal ideation; A US Department of Defense Suicide Event Report record for suicide attempt resulting in hospitalization, evacuation from combat theater, or other attempt related action without a method recorded and no ICD-9-CM/ICD-10-CM code for self-harm or suspicious injury within 4 weeks (before/after) of the event; or an ICD-9-CM/ICD-10-CM code for suicidal ideation (V62.84 or R45.851) recorded in the medical data. ⁱAny inpatient or outpatient visit with the residual category of the Pain Condition Crosswalk (eTable 2 in Supplement 1). ^jAny inpatient or outpatient visit with ICD-9-CM diagnosis codes 240.X-278.XX or ICD-10-CM diagnosis codes E00.X-E89.XXX for “Endocrine, nutritional, and metabolic diseases.” ^kAny inpatient or outpatient visit with ICD-9-CM diagnosis codes 630.X-679.XX or ICD-10-CM diagnosis codes O00.XXX-O9A.XXX for “Pregnancy, childbirth, and the puerperium.”