Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Apr 18;17(1):109.
doi: 10.1186/s13104-024-06773-0.

Accuracy of automated computer-aided risk scoring systems to estimate the risk of COVID-19: a retrospective cohort study

Muhammad Faisal  1   2 , Mohammed Amin Mohammed  3   4 , Donald Richardson  5 , Massimo Fiori  6 , Kevin Beatson  6
Affiliations

Accuracy of automated computer-aided risk scoring systems to estimate the risk of COVID-19: a retrospective cohort study

Muhammad Faisal et al. BMC Res Notes. .

Abstract

Background: In the UK National Health Service (NHS), the patient's vital signs are monitored and summarised into a National Early Warning Score (NEWS) score. A set of computer-aided risk scoring systems (CARSS) was developed and validated for predicting in-hospital mortality and sepsis in unplanned admission to hospital using NEWS and routine blood tests results. We sought to assess the accuracy of these models to predict the risk of COVID-19 in unplanned admissions during the first phase of the pandemic.

Methods: Adult ( > = 18 years) non-elective admissions discharged (alive/deceased) between 11-March-2020 to 13-June-2020 from two acute hospitals with an index NEWS electronically recorded within ± 24 h of admission. We identified COVID-19 admission based on ICD-10 code 'U071' which was determined by COVID-19 swab test results (hospital or community). We assessed the performance of CARSS (CARS_N, CARS_NB, CARM_N, CARM_NB) for predicting the risk of COVID-19 in terms of discrimination (c-statistic) and calibration (graphically).

Results: The risk of in-hospital mortality following emergency medical admission was 8.4% (500/6444) and 9.6% (620/6444) had a diagnosis of COVID-19. For predicting COVID-19 admissions, the CARS_N model had the highest discrimination 0.73 (0.71 to 0.75) and calibration slope 0.81 (0.72 to 0.89) compared to other CARSS models: CARM_N (discrimination:0.68 (0.66 to 0.70) and calibration slope 0.47 (0.41 to 0.54)), CARM_NB (discrimination:0.68 (0.65 to 0.70) and calibration slope 0.37 (0.31 to 0.43)), and CARS_NB (discrimination:0.68 (0.66 to 0.70) and calibration slope 0.56 (0.47 to 0.64)).

Conclusions: The CARS_N model is reasonably accurate for predicting the risk of COVID-19. It may be clinically useful as an early warning system at the time of admission especially to triage large numbers of unplanned admissions because it requires no additional data collection and is readily automated.

Keywords: COVID-19; Computer-aided risk scoring systems; Mortality risk; National early warning score.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing interests.

Figures

Fig. 1
Fig. 1
Receiver Operating Characteristic curve for four CARSS models in predicting the risk of COVID-19. CARM_N: for predicting mortality with NEWS data only; CARM_NB: for predicting mortality with NEWS and Blood test results data; CARS_N: for predicting sepsis with NEWS data only; CARS_NB: for predicting sepsis with NEWS and Blood test results data
Fig. 2
Fig. 2
External validation of CARSS models, respectively for predicting the risk of COVID-19. NB: We limit the risk of COVID-19 to 0.40 for visualisation purpose because beyond this point, we have few patients
See this image and copyright information in PMC

Similar articles

See all similar articles

References

    1. Gorbalenya AE, Baker SC, Baric RS, de Groot RJ, Drosten C, Gulyaeva AA, et al. The species severe acute respiratory syndrome-related coronavirus: classifying 2019-nCoV and naming it SARS-CoV-2. Nat Microbiol. 2020;5:536–44. doi: 10.1038/s41564-020-0695-z. - DOI - PMC - PubMed
    1. Onder G, Rezza G, Brusaferro S. Case-fatality rate and characteristics of patients dying in relation to COVID-19 in Italy. JAMA - J Am Med Association. 2020;323:1775–6. doi: 10.1001/jama.2020.4683. - DOI - PubMed
    1. Vincent JL, Taccone FS. Understanding pathways to death in patients with COVID-19. Lancet Respiratory Med. 2020;8:430–2. doi: 10.1016/S2213-2600(20)30165-X. - DOI - PMC - PubMed
    1. Hao B, Sotudian S, Wang T, Xu T, Hu Y, Gaitanidis A, et al. Early prediction of level-of-care requirements in patients with COVID-19. Elife. 2020;9:1–23. doi: 10.7554/ELIFE.60519. - DOI - PMC - PubMed
    1. Wang T, Paschalidis A, Liu Q, Liu Y, Yuan Y, Paschalidis IC. Predictive models of mortality for hospitalized patients with COVID-19: Retrospective Cohort Study. JMIR Med Inf. 2020;8. 10.2196/21788. - PMC - PubMed