Comprehensive statistical analysis reveals significant benefits of COVID-19 vaccination in hospitalized patients: propensity score, covariate adjustment, and feature importance by permutation

Abstract

Background: COVID-19 vaccines effectively prevent infection and hospitalization. However, few population-based studies have compared the clinical characteristics and outcomes of patients hospitalized for COVID-19 using advanced statistical methods. Our objective is to address this evidence gap by comparing vaccinated and unvaccinated patients hospitalized for COVID-19.

Methods: This retrospective cohort included adult COVID-19 patients admitted from March 2021 to August 2022 from 27 hospitals. Clinical characteristics, vaccination status, and outcomes were extracted from medical records. Vaccinated and unvaccinated patients were compared using propensity score analyses, calculated based on variables associated with vaccination status and/or outcomes, including waves. The vaccination effect was also assessed by covariate adjustment and feature importance by permutation.

Results: From the 3,188 patients, 1,963 (61.6%) were unvaccinated and 1,225 (38.4%) were fully vaccinated. Among these, 558 vaccinated individuals were matched with 558 unvaccinated ones. Vaccinated patients had lower rates of mortality (19.4% vs. 33.3%), invasive mechanical ventilation (IMV-18.3% vs. 34.6%), noninvasive mechanical ventilation (NIMV-10.6% vs. 22.0%), intensive care unit admission (ICU-32.0% vs. 44.1%) vasoactive drug use (21.1% vs. 32.6%), dialysis (8.2% vs. 14.7%) hospital length of stay (7.0 vs. 9.0 days), and thromboembolic events (3.9% vs.7.7%), p < 0.05 for all. Risk-adjusted multivariate analysis demonstrated a significant inverse association between vaccination and in-hospital mortality (adjusted odds ratio [aOR] = 0.42, 95% confidence interval [CI]: 0.31-0.56; p < 0.001) as well as IMV (aOR = 0.40, 95% CI: 0.30-0.53; p < 0.001). These results were consistent in all analyses, including feature importance by permutation.

Conclusion: Vaccinated patients admitted to hospital with COVID-19 had significantly lower mortality and other severe outcomes than unvaccinated ones during the Delta and Omicron waves. These findings have important implications for public health strategies and support the critical importance of vaccination efforts, particularly in low-income countries, where vaccination coverage remains suboptimal.

Keywords: Brazil; COVID-19; Hospitalizations; Machine learning; Mortality; Propensity score; SARS-CoV-2; Severe illness; Vaccine.

Fig. 1

Flowchart of COVID-19 patients included in the study

Fig. 2

Effects of vaccination on mortality (A) and IMV (B) in an original unmatched cohort (crude), covariate adjustment, matching (PSM) (crude and doubly robust), IPW (crude and doubly robust) and PS as a covariate (crude and doubly robust). Forest plots on the log scale show unadjusted and multivariable-adjusted odds ratios (ORs; indicated by diamonds) and 95% confidence intervals (CIs; indicated by the horizontal bars). The variables included in the multivariable regression models were age, sex, hospital of care, comorbidities (hypertension, coronary artery disease, heart failure, atrial fibrillation, stroke, asthma, COPD, pulmonary fibrosis, diabetes mellitus, obesity [body mass index > 30 kg/m²], chronic kidney disease, dialysis, rheumatologic disease, HIV, cancer, post-transplant, and cirrhosis), and home medications (anticoagulation, oral corticosteroids, and immunosuppressants). Abbreviations: IMV: invasive mechanical ventilation; PSM: propensity score matching; IPW: inverse probability weighting; PS: propensity score; OR: odds ratio; CI: confidence intervals

Fig. 3

The most important variables for the predictive models of mortality (A) and IMV (B). In the graph, the size of each bar reflects the importance of the variable for classifying instances for the outcome of interest, while the direction of the bar indicates the association of the variable with the outcome - whether the variable worsens or minimizes the outcome. The direction of the bar was determined based on coefficients from a logistic regression trained to predict outcomes. If a variable’s bar is to the right and is red, it means that the variable is associated with an increase in the probability of the outcome occurring. If the bar is to the left and is blue, this indicates that the variable is associated with a decrease in the probability of the outcome. The values on the X-axis are the measures of loss of effectiveness of the model when performing the permutation. Abbreviations: COPD, chronic obstructive pulmonary disease; CKD; HIV, human immunodeficiency virus