Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 May 15:2023:6042762.
doi: 10.1155/2023/6042762. eCollection 2023.

Comparing In-Hospital Mortality Prediction by Senior Emergency Resident's Judgment and Prognostic Models in the Emergency Department

Zahra Rahmatinejad  1 , Samira Peiravi  2 , Benyamin Hoseini  3 , Fatemeh Rahmatinejad  4 , Saeid Eslami  1   3   5 , Ameen Abu-Hanna  5 , Hamidreza Reihani  2
Affiliations

Comparing In-Hospital Mortality Prediction by Senior Emergency Resident's Judgment and Prognostic Models in the Emergency Department

Zahra Rahmatinejad et al. Biomed Res Int. .

Abstract

Background: A comparison of emergency residents' judgments and two derivatives of the Sequential Organ Failure Assessment (SOFA), namely, the mSOFA and the qSOFA, was conducted to determine the accuracy of predicting in-hospital mortality among critically ill patients in the emergency department (ED).

Methods: A prospective cohort research was performed on patients over 18 years of age presented to the ED. We used logistic regression to develop a model for predicting in-hospital mortality by using qSOFA, mSOFA, and residents' judgment scores. We compared the accuracy of prognostic models and residents' judgment in terms of the overall accuracy of the predicted probabilities (Brier score), discrimination (area under the ROC curve), and calibration (calibration graph). Analyses were carried out using R software version R-4.2.0.

Results: In the study, 2,205 patients with median age of 64 (IQR: 50-77) years were included. There were no significant differences between the qSOFA (AUC 0.70; 95% CI: 0.67-0.73) and physician's judgment (AUC 0.68; 0.65-0.71). Despite this, the discrimination of mSOFA (AUC 0.74; 0.71-0.77) was significantly higher than that of the qSOFA and residents' judgments. Additionally, the AUC-PR of mSOFA, qSOFA, and emergency resident's judgments was 0.45 (0.43-0.47), 0.38 (0.36-0.40), and 0.35 (0.33-0.37), respectively. The mSOFA appears stronger in terms of overall performance: 0.13 vs. 0.14 and 0.15. All three models showed good calibration.

Conclusion: The performance of emergency residents' judgment and the qSOFA was the same in predicting in-hospital mortality. However, the mSOFA predicted better-calibrated mortality risk. Large-scale studies should be conducted to determine the utility of these models.

PubMed Disclaimer

Conflict of interest statement

There is no conflict of interest to declare.

Figures

Figure 1
Figure 1
An overview of the enrollment process for the study population.
Figure 2
Figure 2
AUC-ROC (a) and AUC-PR (b) for the three models evaluated. In both cases, a higher area under the curve indicates better predictions. The mSOFA model performs better than other models.
Figure 3
Figure 3
The calibration graphs reveal that the actual mortality is similar to the predicted one by the prediction models (qSOFA, mSOFA). Without recalibration by regressing the outcome on the predictions, physicians' prognoses indicate overpredictions of 15 to 25% and underpredictions above 30%. After recalibration, in the fourth graph, the calibration is improved.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Rahmatinejad Z., Rahmatinejad F., Sezavar M., Tohidinezhad F., Abu-Hanna A., Eslami S. Internal validation and evaluation of the predictive performance of models based on the PRISM-3 (Pediatric Risk of Mortality) and PIM-3 (Pediatric Index of Mortality) scoring systems for predicting mortality in Pediatric Intensive Care Units (PICUs) BMC Pediatrics . 2022;22(1):p. 199. doi: 10.1186/s12887-022-03228-y. - DOI - PMC - PubMed
    1. Yarmohammadian M. H., Rezaei F., Haghshenas A., Tavakoli N. Overcrowding in emergency departments: a review of strategies to decrease future challenges. Journal of research in medical sciences: the official journal of Isfahan University of Medical Sciences . 2017;22(1):p. 23. doi: 10.4103/1735-1995.200277. - DOI - PMC - PubMed
    1. Goshayeshi L., Akbari Rad M., Bergquist R., Allahyari A., Hashemzadeh K., Hoseini B. Demographic and clinical characteristics of severe Covid-19 infections: a cross-sectional study from Mashhad University of Medical Sciences, Iran. BMC Infectious Diseases . 2021;21(1):p. 656. doi: 10.1186/s12879-021-06363-6. - DOI - PMC - PubMed
    1. Coslovsky M., Takala J., Exadaktylos A. K., Martinolli L., Merz T. M. A clinical prediction model to identify patients at high risk of death in the emergency department. Intensive Care Medicine . 2015;41(6):1029–1036. doi: 10.1007/s00134-015-3737-x. - DOI - PMC - PubMed
    1. Khodashahi R., Naderi H. R., Sedaghat A., et al. Intravenous immunoglobulin for treatment of patients with COVID-19: a case-control study. Archives of Clinical Infectious Diseases . 2021;16(1, article e108068) doi: 10.5812/archcid.108068. - DOI