Development and validation of a model to predict ceiling of care in COVID-19 hospitalized patients

Abstract

Background: Therapeutic ceiling of care is the maximum level of care deemed appropiate to offer to a patient based on their clinical profile and therefore their potential to derive benefit, within the context of the availability of resources. To our knowledge, there are no models to predict ceiling of care decisions in COVID-19 patients or other acute illnesses. We aimed to develop and validate a clinical prediction model to predict ceiling of care decisions using information readily available at the point of hospital admission.

Methods: We studied a cohort of adult COVID-19 patients who were hospitalized in 5 centres of Catalonia between 2020 and 2021. All patients had microbiologically proven SARS-CoV-2 infection at the time of hospitalization. Their therapeutic ceiling of care was assessed at hospital admission. Comorbidities collected at hospital admission, age and sex were considered as potential factors for predicting ceiling of care. A logistic regression model was used to predict the ceiling of care. The final model was validated internally and externally using a cohort obtained from the Leeds Teaching Hospitals NHS Trust. The TRIPOD Checklist for Prediction Model Development and Validation from the EQUATOR Network has been followed to report the model.

Results: A total of 5813 patients were included in the development cohort, of whom 31.5% were assigned a ceiling of care at the point of hospital admission. A model including age, COVID-19 wave, chronic kidney disease, dementia, dyslipidaemia, heart failure, metastasis, peripheral vascular disease, chronic obstructive pulmonary disease, and stroke or transient ischaemic attack had excellent discrimination and calibration. Subgroup analysis by sex, age group, and relevant comorbidities showed excellent figures for calibration and discrimination. External validation on the Leeds Teaching Hospitals cohort also showed good performance.

Conclusions: Ceiling of care can be predicted with great accuracy from a patient's clinical information available at the point of hospital admission. Cohorts without information on ceiling of care could use our model to estimate the probability of ceiling of care. In future pandemics, during emergency situations or when dealing with frail patients, where time-sensitive decisions about the use of life-prolonging treatments are required, this model, combined with clinical expertise, could be valuable. However, future work is needed to evaluate the use of this prediction tool outside COVID-19.

Keywords: Bootstrapping; COVID-19; Calibration; Discrimination; Prediction model; Therapeutic ceiling of care.

Fig. 1

Flow chart of patients

Fig. 2

Observed vs predicted risk of the ceiling of care in the development cohort. Calibration Plot (left). The predicted event rate for each decile of predicted risk is the average of the probabilities of the patients in the decile (dark blue line). The observed event rate is the number of patients with a ceiling of care in the decile divided by the number of patients in the decile (light blue line). Ideally, the expected and observed rate lines should be overlapped. Calibration Plot (right): The calibration curve shows the relationship between the predicted and observed ceiling of care risk by deciles of predicted risk. The diagonal line represents the performance of an ideal model. The blue line represents the actual model performance comparing the predicted and observed risk. Points below the diagonal line correspond to over-prediction, whereas points above the diagonal line correspond to under-prediction

Fig. 3

Observed vs predicted risk of the ceiling of care in the external cohort. Calibration Plot (left). The predicted event rate for each decile of predicted risk is the average of the probabilities of the patients in the decile (dark blue line). The observed event rate is the number of patients with a ceiling of care in the decile divided by the number of patients in the decile (light blue line). Ideally, the expected and observed rate lines should be overlapped. Calibration Plot (right): The calibration curve shows the relationship between the predicted and observed ceiling of care risk by deciles of predicted risk. The diagonal line represents the performance of an ideal model. The blue line represents the actual model performance comparing the predicted and observed risk. Points below the diagonal line correspond to over-prediction, whereas points above the diagonal line correspond to under-prediction