Using Neural Networks with Routine Health Records to Identify Suicide Risk: Feasibility Study

Abstract

Background: Each year, approximately 800,000 people die by suicide worldwide, accounting for 1-2 in every 100 deaths. It is always a tragic event with a huge impact on family, friends, the community and health professionals. Unfortunately, suicide prevention and the development of risk assessment tools have been hindered by the complexity of the underlying mechanisms and the dynamic nature of a person's motivation and intent. Many of those who die by suicide had contact with health services in the preceding year but identifying those most at risk remains a challenge.

Objective: To explore the feasibility of using artificial neural networks with routinely collected electronic health records to support the identification of those at high risk of suicide when in contact with health services.

Methods: Using the Secure Anonymised Information Linkage Databank UK, we extracted the data of those who died by suicide between 2001 and 2015 and paired controls. Looking at primary (general practice) and secondary (hospital admissions) electronic health records, we built a binary feature vector coding the presence of risk factors at different times prior to death. Risk factors included: general practice contact and hospital admission; diagnosis of mental health issues; injury and poisoning; substance misuse; maltreatment; sleep disorders; and the prescription of opiates and psychotropics. Basic artificial neural networks were trained to differentiate between the suicide cases and paired controls. We interpreted the output score as the estimated suicide risk. System performance was assessed with 10x10-fold repeated cross-validation, and its behavior was studied by representing the distribution of estimated risk across the cases and controls, and the distribution of factors across estimated risks.

Results: We extracted a total of 2604 suicide cases and 20 paired controls per case. Our best system attained a mean error rate of 26.78% (SD 1.46; 64.57% of sensitivity and 81.86% of specificity). While the distribution of controls was concentrated around estimated risks < 0.5, cases were almost uniformly distributed between 0 and 1. Prescription of psychotropics, depression and anxiety, and self-harm increased the estimated risk by ~0.4. At least 95% of those presenting these factors were identified as suicide cases.

Conclusions: Despite the simplicity of the implemented system, the proposed methodology obtained an accuracy like other published methods based on specialized questionnaire generated data. Most of the errors came from the heterogeneity of patterns shown by suicide cases, some of which were identical to those of the paired controls. Prescription of psychotropics, depression and anxiety, and self-harm were strongly linked with higher estimated risk scores, followed by hospital admission and long-term drug and alcohol misuse. Other risk factors like sleep disorders and maltreatment had more complex effects.

Keywords: artificial neural networks; electronic health records; machine learning; risk assessment; routine data; suicide prevention.

Figure 1

Structure of an artificial neural network with 1 input layer (red), 2 hidden layers (green) and one output layer (blue) all fully connected.

Figure 2

Receiving operating characteristics (ROC) curve for nn0, nn50 and nn10-10. FPR: false positive rate; TPR: true positive rate; nn0: no hidden layers; nn50: 1 hidden layer with 50 neurons; nn10-10: 2 hidden layers with 10 neurons.

Figure 3

Distribution of cases and controls across estimated risk score levels. Those with risk score >0.5 were identified as “cases.”

Figure 4

Histogram of the difference in estimated risk score when turning specific factors ‘on’ and ‘off’ across the whole dataset. CLD: contact leading up to death.

Figure 5

Samples presenting a specific factor and their distribution across cases and controls, and across estimated risks from very low (VLR) to very high (VHR). To the left of each bar group, the total number of individuals presenting the factor (sample size). At the top, the distribution of the full population. VHR: very high risk (r>0.83); HR: high risk (0.67<r ≤0.83); MHR: moderate-high risk (0.5<r ≤0.67); MLR: moderate-low risk (0.33<r ≤0.5); LR: low risk (0.17<r ≤0.33); VLR: very low risk (r ≤0.17).

Figure 6

Incidence of factors for cases, controls and estimated risk levels from very low (VLR) to very high (VHR). Panels on the right hand column (shaded) have y-axis limits between 0% and 30% to facilitate visualization. VHR: very high risk (r>0.83); HR: high risk (0.67<r ≤0.83); MHR: moderate-high risk (0.5<r ≤0.67); MLR: moderate-low risk (0.33<r ≤0.5); LR: low risk (0.17<r ≤0.33); VLR: very low risk (r ≤0.17).