Building and Validation of an Acute Event Prediction Model for Severe Mental Disorders

Abstract

Background: The global incidence of acute events in psychiatric patients is intensifying, and models to successfully predict acute events have attracted much attention.

Objective: To explore the influence factors of acute incident severe mental disorders (SMDs) and the application of Rstudio statistical software, and build and verify a nomogram prediction model.

Methods: SMDs were taken as research objects. The questionnaire survey method was adopted to collect data. Patients with acute event independent factors were screened. R software multivariable Logistic regression model was constructed and a nomogram was drawn.

Results: A total of 342 patients with SMDs were hospitalized, and the number of patients who encountered acute events was 64, which accounted for 18.70% of all patients. Statistical significances were found in many aspects (all P ˂ 0.05). Such aspects included Medication adherence, disease diagnosis, marital status, caregivers, social support and the hospitalization environment (odds ratio (OR) = 4.08, 11.62, 12.06, 10.52, 0.04 and 0.61, respectively) were independent risk factors for the acute events of patients with SMDs. The prediction model was modeled, and the AUC was 0.77 and 0.80. The calibration curve shows that the model has good calibration. The clinical decision curve shows that the model has a good clinical effect.

Conclusion: The constructed risk prediction model shows good prediction effectiveness in the acute events of patients with SMDs, which is helpful for the early detection of clinical mental health staff at high risk of acute events.

Keywords: SMDs; acute event; influencing factors; predictive model and nomogram.

Figure 1

The nomogram for predicting acute event risk of patients. Based on the Logistic regression model: Z = 1.80–0.16 *MMAS-8 −0.90 *Disease diagnosis −1.28 *Marital status +1.73 * caregivers +5.80 *SSRS +8.43 *The hospital environment, R software was used to build and visualize a nomogram.

Figure 2

ROC curve of the hospitalized SMD patients encountering acute events. Prediction variables were used as a test, and hospitalized SMD patients encountering acute events were taken as state variables to draw the ROC curve. The modeling prediction model set of the AUC value is 0.77. When the best cutoff value was taken, the Yoden index was 0.62, and sensitivity and specificity were 0.90 and 0.72, respectively. The AUC value of the prediction model in the validation group was 0.80. When the best cutoff value index was 0.57, sensitivity and specificity were 0.81 and 0.76, respectively.

Figure 3

The calibration curves of the nomogram for the risk of acute event. The calibration ability of the risk prediction model for the risk of acute events in hospitalized patients with SMDs was evaluated using the Hosmer-Lemeshow (H-L) goodness-of-fit test. The result showed x² = 8.08, p = 0.43. In the prediction of hospitalized SMD patients encountering acute events, the predicted and actual occurrence probabilities of the model showed no statistical difference.

Figure 4

Decision curve analysis (DCA) for detection of patients. DCA is used to evaluate the clinical efficacy of the risk prediction model for the acute events of hospitalized patients with SMDs. It is also used to determine the use of the clinical prediction model to inform whether clinical decisions do more harm than good. The higher clinical utility can be obtained when the probability is 0.01 ~ 0.90.