Development of an early-warning system for high-risk patients for suicide attempt using deep learning and electronic health records

Abstract

Suicide is the tenth leading cause of death in the United States (US). An early-warning system (EWS) for suicide attempt could prove valuable for identifying those at risk of suicide attempts, and analyzing the contribution of repeated attempts to the risk of eventual death by suicide. In this study we sought to develop an EWS for high-risk suicide attempt patients through the development of a population-based risk stratification surveillance system. Advanced machine-learning algorithms and deep neural networks were utilized to build models with the data from electronic health records (EHRs). A final risk score was calculated for each individual and calibrated to indicate the probability of a suicide attempt in the following 1-year time period. Risk scores were subjected to individual-level analysis in order to aid in the interpretation of the results for health-care providers managing the at-risk cohorts. The 1-year suicide attempt risk model attained an area under the curve (AUC ROC) of 0.792 and 0.769 in the retrospective and prospective cohorts, respectively. The suicide attempt rate in the "very high risk" category was 60 times greater than the population baseline when tested in the prospective cohorts. Mental health disorders including depression, bipolar disorders and anxiety, along with substance abuse, impulse control disorders, clinical utilization indicators, and socioeconomic determinants were recognized as significant features associated with incident suicide attempt.

Fig. 1. Development of risk of suicide attempt early-warning system.

The system is consist of the deep learning live engine and the decision interpretation live engine. The deep learning engine is design to provide a real-time risk stratification for the whole population, so that the high-risk population can be found in advance. The decision interpretation live engine is used to analyze the driving features of the high-risk population and help provide insight for individual intervention.

Fig. 2. Workflow diagram depicting model construction and evaluation.

The retrospective cohort consisted of 118,252 individuals with EHR profiles extracted from 2015, 255 of whom (cases) attempted suicide in 2016. The validation cohort consisted of 118,095 individuals, with EHR profiles extracted from 2014, 203 of who were admitted for suicide attempt in 2017.

Fig. 3. ROC curves of three different algorithms applied on the prospective cohort.

The AUC of the deep learning model is 0.769, the AUC of the logistic regression model is 0.604, and the AUC of the XGBoost model is 0.702. The deep learning model has highest AUC and best performance compared to the logistic regression model and the XGBoost model.

Fig. 4. Mental illness subgroup’s average risk against the PPV.

The centers of the circles were the mean risk and PPV values. The radius represented the number of individuals in each subgroup, the bigger the radius was, the more individuals the subgroup had. Each circle was also a pie chart, that represented the gender distribution in each subgroup.

Fig. 5. Forest plot of odds ratios (and their 95% confidence intervals and p value < 0.01, the size of the square is proportional to the negative log p value) for the comparison between the <25-year-old age group and the 25–54-year-old age group.

Past suicide attempt is the strongest risk factor for the age group of 25–54, while mental disorders like personality disorder, substance abuse, and bipolar disorder also have more influence on this age group. In the younger group, the physiological defects and the pain-related disorders are stronger predictors.