Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Meta-Analysis
. 2023 Jun;28(22):2200706.
doi: 10.2807/1560-7917.ES.2023.28.22.2200706.

Assessing the robustness of COVID-19 vaccine efficacy trials: systematic review and meta-analysis, January 2023

Thi Ngoc Anh Hoang  1 , Ha-Linh Quach  2   3   4 , Van Ngoc Hoang  5 , Van Thien Tran  6 , Quang Thai Pham  3   7 , Florian Vogt  2   8
Affiliations
Meta-Analysis

Assessing the robustness of COVID-19 vaccine efficacy trials: systematic review and meta-analysis, January 2023

Thi Ngoc Anh Hoang et al. Euro Surveill. 2023 Jun.

Abstract

BackgroundVaccines play a crucial role in the response to COVID-19 and their efficacy is thus of great importance.AimTo assess the robustness of COVID-19 vaccine efficacy (VE) trial results using the fragility index (FI) and fragility quotient (FQ) methodology.MethodsWe conducted a Cochrane and PRISMA-compliant systematic review and meta-analysis of COVID-19 VE trials published worldwide until 22 January 2023. We calculated the FI and FQ for all included studies and assessed their associations with selected trial characteristics using Wilcoxon rank sum tests and Kruskal-Wallis H tests. Spearman correlation coefficients and scatter plots were used to quantify the strength of correlation of FIs and FQs with trial characteristics.ResultsOf 6,032 screened records, we included 40 trials with 54 primary outcomes, comprising 909,404 participants with a median sample size per outcome of 13,993 (interquartile range (IQR): 8,534-25,519). The median FI and FQ was 62 (IQR: 22-123) and 0.50% (IQR: 0.24-0.92), respectively. FIs were positively associated with sample size (p < 0.001), and FQs were positively associated with type of blinding (p = 0.023). The Spearman correlation coefficient for FI with sample size was moderately strong (0.607), and weakly positive for FI and FQ with VE (0.138 and 0.161, respectively).ConclusionsThis was the largest study on trial robustness to date. Robustness of COVID-19 VE trials increased with sample size and varied considerably across several other important trial characteristics. The FI and FQ are valuable complementary parameters for the interpretation of trial results and should be reported alongside established trial outcome measures.

Keywords: COVID-19; SARS-CoV-2; fragility index; fragility quotient; randomized controlled trial; robustness; vaccine efficacy.

PubMed Disclaimer

Conflict of interest statement

Conflict of interest: None declared.

Figures

Figure 1
Figure 1
Selection of randomised controlled trial studies on COVID-19 vaccine efficacy, up to 22 January 2023 (n = 40 studies selected)
Figure 2
Figure 2
Fragility index and fragility quotient of outcomes against sample size, for studies included in this analysis up to 22 January 2023 (n = 40 studies selected)
Figure 3
Figure 3
Fragility index and fragility quotient of outcomes against percentage lost to follow-up, for studies included in this analysis up to 22 January 2023 (n = 40 studies selected)
Figure 4
Figure 4
Fragility index and fragility quotient for outcomes, against vaccine efficacy for studies included in this analysis up to 22 January 2023 (n = 40 studies selected)
Figure 5
Figure 5
Classification, according to risk of bias, of studies included in this analysis up to 22 January 2023
See this image and copyright information in PMC

References

    1. Au WY, Cheung PP-H. Effectiveness of heterologous and homologous covid-19 vaccine regimens: living systematic review with network meta-analysis. BMJ. 2022;377:e069989. 10.1136/bmj-2022-069989 - DOI - PMC - PubMed
    1. Rotshild V, Hirsh-Raccah B, Miskin I, Muszkat M, Matok I. Comparing the clinical efficacy of COVID-19 vaccines: a systematic review and network meta-analysis. Sci Rep. 2021;11(1):22777. 10.1038/s41598-021-02321-z - DOI - PMC - PubMed
    1. Xing K, Tu XY, Liu M, Liang ZW, Chen JN, Li JJ, et al. Efficacy and safety of COVID-19 vaccines: a systematic review. Zhongguo Dang Dai Er Ke Za Zhi. 2021;23(3):221-8. 10.7499/j.issn.1008-8830.2101133 - DOI - PMC - PubMed
    1. Ghazy RM, Ashmawy R, Hamdy NA, Elhadi YAM, Reyad OA, Elmalawany D, et al. Efficacy and Effectiveness of SARS-CoV-2 Vaccines: A Systematic Review and Meta-Analysis. Vaccines (Basel). 2022;10(3):350. 10.3390/vaccines10030350 - DOI - PMC - PubMed
    1. Armitage P. The design and analysis of clinical trials. Handbook of Statistics. 1996;13(7049):1-29. 10.1016/S0169-7161(96)13003-0 - DOI

Substances