Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Multicenter Study
. 2025 Jan 1;43(Pt 2):126492.
doi: 10.1016/j.vaccine.2024.126492. Epub 2024 Nov 7.

Assessment and mitigation of bias in influenza and COVID-19 vaccine effectiveness analyses - IVY Network, September 1, 2022-March 30, 2023

Nathaniel M Lewis  1 , Elizabeth J Harker  2 , Aleda Leis  3 , Yuwei Zhu  4 , H Keipp Talbot  4 , Carlos G Grijalva  4 , Natasha Halasa  4 , James D Chappell  4 , Cassandra A Johnson  4 , Todd W Rice  4 , Jonathan D Casey  4 , Adam S Lauring  3 , Manjusha Gaglani  5 , Shekhar Ghamande  6 , Cristie Columbus  7 , Jay S Steingrub  8 , Nathan I Shapiro  9 , Abhijit Duggal  10 , Jamie Felzer  11 , Matthew E Prekker  12 , Ithan D Peltan  13 , Samuel M Brown  13 , David N Hager  14 , Michelle N Gong  15 , Amira Mohamed  15 , Matthew C Exline  16 , Akram Khan  17 , Jennifer G Wilson  18 , Jarrod Mosier  19 , Nida Qadir  20 , Steven Y Chang  20 , Adit A Ginde  21 , Nicholas M Mohr  22 , Christopher Mallow  23 , Estelle S Harris  24 , Nicholas J Johnson  25 , Vasisht Srinivasan  25 , Kevin W Gibbs  26 , Jennie H Kwon  27 , Ivana A Vaughn  28 , Mayur Ramesh  28 , Basmah Safdar  29 , Jennifer DeCuir  2 , Diya Surie  30 , Fatimah S Dawood  30 , Sascha Ellington  2 , Wesley H Self  4 , Emily T Martin  3
Affiliations
Multicenter Study

Assessment and mitigation of bias in influenza and COVID-19 vaccine effectiveness analyses - IVY Network, September 1, 2022-March 30, 2023

Nathaniel M Lewis et al. Vaccine. .

Abstract

Background: In test-negative studies of vaccine effectiveness (VE), including patients with co-circulating, vaccine-preventable, respiratory pathogens in the control group for the pathogen of interest can introduce a downward bias on VE estimates.

Methods: A multicenter sentinel surveillance network in the US prospectively enrolled adults hospitalized with acute respiratory illness from September 1, 2022-March 31, 2023. We evaluated bias in estimates of VE against influenza-associated and COVID-19-associated hospitalization based on: inclusion vs exclusion of patients with a co-circulating virus among VE controls; observance of VE against the co-circulating virus (rather than the virus of interest), unadjusted and adjusted for vaccination against the virus of interest; and observance of influenza or COVID-19 against a sham outcome of respiratory syncytial virus (RSV).

Results: Overall VE against influenza-associated hospitalizations was 6 percentage points lower when patients with COVID-19 were included in the control group, and overall VE against COVID-19-associated hospitalizations was 2 percentage points lower when patients with influenza were included in the control group. Analyses of VE against the co-circulating virus and against the sham outcome of RSV showed that downward bias was largely attributable the correlation of vaccination status across pathogens, but also potentially attributable to other sources of residual confounding in VE models.

Conclusion: Excluding cases of confounding respiratory pathogens from the control group in VE analysis for a pathogen of interest can reduce downward bias. This real-world analysis demonstrates that such exclusion is a helpful bias mitigation strategy, especially for measuring influenza VE, which included a high proportion of COVID-19 cases among controls.

Keywords: Bias; COVID-19; Estimation; Influenza; RSV; Vaccine effectiveness.

PubMed Disclaimer

Conflict of interest statement

Declaration of Competing Interest The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: All authors have completed and submitted the International Committee of Medical Journal Editors form for disclosure of potential conflicts of interest. Michelle Ng Gong reports grants from NHLBI as a HART co-investigator and APS steering committee co-chair, support for travel to the ATS meeting, participating on an advisory board from Novartis, Philips Healthcase, Regeneron, and Radiomater, and participating on the DSMB for Best clinical trials, outside the submitted work. Carlos G. Grijalva reports receiving compensation for participation in an advisory board for Merck, and receiving research support from CDC, NIH, FDA, AHRQ and SyneosHealth, outside the submitted work. Natasha Halasa receives current support from Merck (investigator-initiated grant), past support from Sanofi and Quidel, and served on an advisory board for Seqirus, outside the submitted work. Akram Khan reports receiving research funding from Dompe pharmaceuticals, 4D Medical, Eli Lilly, and Direct Biologics for patient enrollment in studies, outside the submitted work. Adam S. Lauring reports consulting fees and research support from Roche, and research funding from NIAID, NSF, CDC, and Burroughs Wellcome Fund, outside the submitted work. Ithan Peltan reports funding from NIH (R35GM151147), funding from NHLBI and Janssen, and payments to his institution from Regeneron, Bluejay Diagnostics, and Novartis, outside the submitted work. Mayur Ramesh reports participating on an advisory board for AstraZeneca, Moderna, and Pfizer, outside the submitted work. Ivana A Vaughn reports funding from CDC via University of Michigan for US Flu VE Network, funding from eMaxHealth through her institution, and funding from Lilly USA through her institution, outside the submitted work. No other potential conflicts of interest were disclosed.

References

    1. Sullivan SG, Tchetgen Tchetgen EJ, Cowling BJ. Theoretical Basis of the Test-Negative Study Design for Assessment of Influenza Vaccine Effectiveness. Am J Epidemiol 2016;184:345–353. - PMC - PubMed
    1. Jackson ML, Nelson JC. The test-negative design for estimating influenza vaccine effectiveness. Vaccine 2013;31:2165–2168. - PubMed
    1. Leuchter RK, Jackson NJ, Mafi JN, Sarkisian CA. Association between Covid-19 vaccination and influenza vaccination rates. N Engl J Med 2022;386:2531–2532. - PMC - PubMed
    1. Doll MK, Pettigrew SM, Ma J, Verma A. Effects of Confounding Bias in Coronavirus Disease 2019 (COVID-19) and Influenza Vaccine Effectiveness Test-Negative Designs Due to Correlated Influenza and COVID-19 Vaccination Behaviors. Clin Infect Dis 2022;75:e564–e571. - PMC - PubMed
    1. CDC. Urgent Need to Increase Immunization Coverage for Influenza, COVID-19, and RSV and Use of Authorized/Approved Therapeutics in the Setting of Increased Respiratory Disease Activity During the 2023–2024 Winter Season. https://emergency.cdc.gov/han/2023/han00503.asp.

Publication types

MeSH terms

LinkOut - more resources