A prediction model with measured sentiment scores for the risk of in-hospital mortality in acute pancreatitis: a retrospective cohort study

Abstract

Background: Accurate and prompt clinical assessment of the severity and prognosis of patients with acute pancreatitis (AP) is critical, particularly during hospitalization. Natural language processing algorithms gain an opportunity from the growing number of free-text notes in electronic health records to mine this unstructured data, e.g., nursing notes, to detect and predict adverse outcomes. However, the predictive value of nursing notes for AP prognosis is unclear. In this study, a predictive model for in-hospital mortality in AP was developed using measured sentiment scores in nursing notes.

Methods: The data of AP patients in the retrospective cohort study were collected from the Medical Information Mart for Intensive Care III (MIMIC-III) database. Sentiments in nursing notes were assessed by sentiment analysis. For each individual clinical note, sentiment polarity and sentiment subjectivity scores were assigned. The in-hospital mortality of AP patients was the outcome. A predictive model was built based on clinical information and sentiment scores, and its performance and clinical value were evaluated using the area under curves (AUCs) and decision-making curves, respectively.

Results: Of the 631 AP patients included, 88 cases (13.9%) cases were dead in hospital. When various confounding factors were adjusted, the mean sentiment polarity was associated with a reduced risk of in-hospital mortality in AP [odds ratio (OR): 0.448; 95% confidence interval (CI): 0.233-0.833; P=0.014]. A predictive model was established in the training group via multivariate logistic regression analysis, including 12 independent variables. In the testing group, the model showed an AUC of 0.812, which was significantly greater than the sequential organ failure assessment (SOFA) of 0.732 and the simplified acute physiology score-II (SAPS-II) of 0.792 (P<0.05). When the same level of risk was considered, the clinical benefits of the predictive model were found to be the highest compared with SOFA and SAPS-II scores.

Conclusions: The model combined sentiment scores in nursing notes showed well predictive performance and clinical value in in-hospital mortality of AP patients.

Keywords: Acute pancreatitis; Medical Information Mart for Intensive Care III (MIMIC-III); in-hospital mortality; predictive model; sentiment.

Figure 1

The receiver operating characteristic curves of different models in the prediction of in-hospital mortality. (A) Training group; (B) testing group. AUC, area under the curve; SOFA, sequential organ failure assessment; SAPS-II, Simplified Acute Physiology Score-II.

Figure 2

Comparison of the clinical value among different models. SOFA, sequential organ failure assessment; SAPS-II, Simplified Acute Physiology Score-II.

Figure 3

A nomogram to predict the risk of in-hospital mortality in acute pancreatitis. *, P<0.05; **, P<0.01; ***, P<0.001.

Figure 4

A schematic nomogram for predicting the risk of in-hospital mortality in a specific patient with acute pancreatitis. *, P<0.05; **, P<0.01; ***, P<0.001.