Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Feb 15;5(4):1697.
doi: 10.23889/ijpds.v5i4.1697. eCollection 2020.

Validating the QCOVID risk prediction algorithm for risk of mortality from COVID-19 in the adult population in Wales, UK

Jane Lyons  1 , Vahé Nafilyan  2 , Ashley Akbari  1 , Gareth Davies  1 , Rowena Griffiths  1 , Ewen M Harrison  3 , Julia Hippisley-Cox  4 , Joe Hollinghurst  1 , Kamlesh Khunti  5 , Laura North  1 , Aziz Sheikh  6 , Fatemeh Torabi  1 , Ronan A Lyons  1
Affiliations

Validating the QCOVID risk prediction algorithm for risk of mortality from COVID-19 in the adult population in Wales, UK

Jane Lyons et al. Int J Popul Data Sci. .

Abstract

Introduction: COVID-19 risk prediction algorithms can be used to identify at-risk individuals from short-term serious adverse COVID-19 outcomes such as hospitalisation and death. It is important to validate these algorithms in different and diverse populations to help guide risk management decisions and target vaccination and treatment programs to the most vulnerable individuals in society.

Objectives: To validate externally the QCOVID risk prediction algorithm that predicts mortality outcomes from COVID-19 in the adult population of Wales, UK.

Methods: We conducted a retrospective cohort study using routinely collected individual-level data held in the Secure Anonymised Information Linkage (SAIL) Databank. The cohort included individuals aged between 19 and 100 years, living in Wales on 24th January 2020, registered with a SAIL-providing general practice, and followed-up to death or study end (28th July 2020). Demographic, primary and secondary healthcare, and dispensing data were used to derive all the predictor variables used to develop the published QCOVID algorithm. Mortality data were used to define time to confirmed or suspected COVID-19 death. Performance metrics, including R2 values (explained variation), Brier scores, and measures of discrimination and calibration were calculated for two periods (24th January-30th April 2020 and 1st May-28th July 2020) to assess algorithm performance.

Results: 1,956,760 individuals were included. 1,192 (0.06%) and 610 (0.03%) COVID-19 deaths occurred in the first and second time periods, respectively. The algorithms fitted the Welsh data and population well, explaining 68.8% (95% CI: 66.9-70.4) of the variation in time to death, Harrell's C statistic: 0.929 (95% CI: 0.921-0.937) and D statistic: 3.036 (95% CI: 2.913-3.159) for males in the first period. Similar results were found for females and in the second time period for both sexes.

Conclusions: The QCOVID algorithm developed in England can be used for public health risk management for the adult Welsh population.

Keywords: COVID-19 outcomes; QCOVID algorithm; SAIL Databank; population data-linkage; risk prediction models.

PubMed Disclaimer

Conflict of interest statement

Conflicts of interest: AS is a member of the Scottish Government’s COVID-19 Chief Medical Officer’s Advisory Group and its Standing Committee on Pandemics; he is also a member of NERVTAG’s Risk Stratification Subgroup. KK is member of NERVTAG subgroup and member of the Scientific Advisory Group for Emergencies (SAGE). JHC reports grants from National Institute for Health Research (NIHR) Biomedical Research Centre, Oxford, grants from John Fell Oxford University Press Research Fund, grants from Cancer Research UK (CR-UK) grant number C5255/A18085, through the Cancer Research UK Oxford Centre, grants from the Oxford Wellcome Institutional Strategic Support Fund (204826/Z/16/Z) and other research councils, during the conduct of the study. JHC is an unpaid director of QResearch, a not-for-profit organisation which is a partnership between the University of Oxford and EMIS Health who supply the QResearch database used for this work. JHC is a founder and shareholder of ClinRisk ltd and was its medical director until 31st May 2019. ClinRisk Ltd produces open and closed source software to implement clinical risk algorithms (outside this work) into clinical computer systems. JHC is chair of the NERVTAG risk stratification subgroup and a member of SAGE COVID-19 groups and the NHS group advising on prioritisation of use of monoclonal antibodies in COVID-19 infection. RAL is a member of the Welsh Government COVID-19 Technical Advisory Group.

Figures

Figure 1: The concordance index by sex and age group in the first time period (24<sup>th</sup> January–30<sup>th</sup> April 2020)
Figure 1: The concordance index by sex and age group in the first time period (24th January–30th April 2020)
Figure 2: The concordance index by sex and age group in the second time period (1<sup>st</sup> May–28<sup>th</sup> July 2020)
Figure 2: The concordance index by sex and age group in the second time period (1st May–28th July 2020)
Figure 3: Predicted and observed risk of COVID-19-related death in the first time period (24<sup>th</sup> January–30<sup>th</sup> April 2020)
Figure 3: Predicted and observed risk of COVID-19-related death in the first time period (24th January–30th April 2020)
Figure 4: Sensitivity for COVID-19-related death in the first (24<sup>th</sup> January–30<sup>th</sup> April 2020) and second (1st May–28th July 2020) time periods
Figure 4: Sensitivity for COVID-19-related death in the first (24th January–30th April 2020) and second (1st May–28th July 2020) time periods
See this image and copyright information in PMC

References

    1. Huang C, Wang Y, Li X, Ren L, Zhao J, Hu Y, et al.. Clinical features of patients infected With 2019 novel coronavirus in Wuhan, China. The Lancet. 2020;395(10223):497–506. 10.1016/S0140-6736(20)30183-5 - DOI - PMC - PubMed
    1. Coronavirus cases: [Internet]. Worldometer. [cited 2021. Aug 23]. Available from: https://www.worldometers.info/coronavirus/
    1. Clift AK, Coupland CA, Keogh RH, Diaz-Ordaz K, Williamson E, Harrison EM, et al.. Living risk prediction algorithm (QCOVID) for risk of hospital admission and mortality from coronavirus 19 in adults: National derivation and Validation cohort study. BMJ. 2020;371:m3731. 10.1136/bmj.m3731 - DOI - PMC - PubMed
    1. Zhou F, Yu T, Du R, Fan G, Liu Y, Liu Z, et al.. Clinical course and risk factors for mortality of adult inpatients with Covid 19 in Wuhan, China A retrospective cohort study. The Lancet. 2020;395(10229):1054–62. 10.1016/S0140-6736(20)30566-3 - DOI - PMC - PubMed
    1. Harrison SL, Fazio-Eynullayeva E, Lane DA, Underhill P, Lip GY. Comorbidities associated with mortality in 31,461 adults with COVID-19 in the United states: A federated electronic medical record analysis. PLOS Medicine. 2020;17(9). 10.1371/journal.pmed.1003321 - DOI - PMC - PubMed

LinkOut - more resources