Prediction Models for Suicide Attempts among Adolescents Using Machine Learning Techniques

Abstract

Objective: Suicide attempts (SAs) in adolescents are difficult to predict although it is a leading cause of death among adolescents. This study aimed to develop and evaluate SA prediction models based on six different machine learning (ML) algorithms for Korean adolescents using data from online surveys.

Methods: Data were extracted from the 2011-2018 Korea Youth Risk Behavior Survey (KYRBS), an ongoing annual national survey. The participants comprised 468,482 nationally representative adolescents from 400 middle and 400 high schools, aged 12 to 18. The models were trained using several classic ML methods and then tested on internal and external independent datasets; performance metrics were calculated. Data analysis was performed from March 2020 to June 2020.

Results: Among the 468,482 adolescents included in the analysis, 15,012 cases (3.2%) were identified as having made an SA. Three features (suicidal ideation, suicide planning, and grade) were identified as the most important predictors. The performance of the six ML models on the internal testing dataset was good, with both the area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC) ranging from 0.92 to 0.94. Although the AUROC of all models on the external testing dataset (2018 KYRBS) ranged from 0.93 to 0.95, the AUPRC of the models was approximately 0.5.

Conclusion: The developed and validated SA prediction models can be applied to detect high risks of SA. This approach could facilitate early intervention in the suicide crisis and may ultimately contribute to suicide prevention for adolescents.

Keywords: Adolescents; Attempted suicide; Machine learning; Suicide.

Fig. 1

Schematic of prediction model development. KYRBS, Korea Youth Risk Behavior Survey; ROC, receiver operating characteristic; PR, precision−recall; LR, logistic regression; RF, random forest; ANN, artificial neural network; SVM, support vector machine; XGB, extreme gradient boosting; SMOTE, synthetic minority oversampling technique; CML, classic machine learning; DNN, deep neural network.

Fig. 2

Multivariate LR analysis to identify variables associated with SA. The forest plots indicate the odds ratios and confidence intervals of the variables associated with SA. The black dots indicate the adjusted odds ratios for the variables (p < 0.05) and the error bars indicate 95% confidence intervals. CI, confidence interval; MSG, middle school graduate or lower; LR, logistic regression; SA, suicide attempt; SES, socioeconomic status.

Fig. 3

ROC and PR curves plotted from internal balanced testing dataset. (A) ROC curves (B) PR curves. ROC, receiver operating characteristic; PR, precision−recall; AUROC, area under the ROC curve; AUPRC, area under the PR curve; LR, logistic regression; RF, random forest; ANN, artificial neural networks; SVM, support vector machines; XGB, extreme gradient boosting; DNN: deep neural network.

Fig. 4

Relative feature importance computed by Boruta algorithm. The blue violin plots correspond to the minimal, average, and maximum Z scores of a shadow attribute. The red and green violin plots represent the Z scores of rejected and confirmed attributes, respectively. The black dots and horizontal lines inside each violin plot represent the mean and median values, respectively. All fea-tures that received a lower relative feature importance than the shadow feature were defined as irrelevant for the prediction. For the training data-set, the irrelevant features (marked in red) were city type, asthma, and atopic dermatitis. BMI, body mass index; SES, socio-economic status.

Fig. 5

Performance of learning models evaluated with external imbalanced testing dataset. (A) ROC curves (B) PR curves. ROC, receiver operating characteristic; PR, precision−recall; AUROC, area under the ROC curve; AUPRC, area under the PR curve; LR, logistic regression; RF, random forest; ANN, artificial neural networks; SVM, support vector machines; XGB, extreme gradient boosting; DNN: deep neural network.