Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 Feb 10;20(2):e0316198.
doi: 10.1371/journal.pone.0316198. eCollection 2025.

Classifying COVID-19 hospitalizations in epidemiology cohort studies: The C4R study

Elizabeth C Oelsner  1 , Akshaya Krishnaswamy  1 , Rafail Rustamov  2 , Pallavi P Balte  1 , Tauqeer Ali  3 , Norrina B Allen  4 , Howard F Andrews  5 , Pramod Anugu  6 , Alexander Arynchyn  7 , Lori A Bateman  8 , Jianwen Cai  8 , Harry Chang  1 , Lucas Chen  9 , Mitchell S V Elkind  10   11 , James S Floyd  12   13 , Kelley Pettee Gabriel  14 , Sina A Gharib  15 , Jose D Gutierrez  10 , Karen Hinckley Stukovsky  16 , Virginia J Howard  14 , Carmen R Isasi  17 , Lauren Jager  16 , Ling Jin  9 , Suzanne E Judd  18 , Alka M Kanaya  19 , Namratha R Kandula  4   20 , Maureen R Kelly  1 , Sadiya S Khan  21 , Anna Kucharska-Newton  22 , Joyce S Lee  23 , Emily B Levitan  14 , Cora E Lewis  14 , Barry J Make  24 , Kimberly Malloy  3 , Jennifer J Manly  10 , David Mauger  25 , Yuan-I Min  6 , Joanne M Murabito  9   26 , Charles G Murphy  27 , Arnita F Norwood  6 , George T O'Connor  9   26 , Victor E Ortega  28 , Ashmi A Patel  1 , Amber Pirzada  29 , Elizabeth A Regan  30 , Kimberly B Ring  8 , Wayne D Rosamond  22 , David A Schwartz  23 , James M Shikany  7 , Daniela Sotres-Alvarez  8 , Cheryl Tarlton  3 , Janis Tse  9 , Elman M Urbina Meneses  31 , Maya Vankineni  1 , Sally E Wenzel  32 , Prescott G Woodruff  33 , Vanessa Xanthakis  9   34 , Ji Hyun Yang  9   35 , Neil A Zakai  36   37 , Ying Zhang  3 , Wendy S Post  38
Affiliations

Classifying COVID-19 hospitalizations in epidemiology cohort studies: The C4R study

Elizabeth C Oelsner et al. PLoS One. .

Abstract

Rationale: Robust COVID-19 outcomes classification is important for ongoing epidemiology research on acute and post-acute COVID-19 conditions. Protocolized medical record review is an established method to validate endpoints for clinical trials and cardiovascular epidemiology cohorts; however, a protocol to adjudicate hospitalizations for COVID-19 among epidemiology cohorts was lacking.

Objectives: We developed a protocol to ascertain and adjudicate hospitalized COVID-19 across a meta-cohort of 14 US prospective cohort studies. This report describes the first three years of protocol implementation (October 1, 2020-October 1, 2023) and evaluates its repeatability and performance compared to classification by administrative codes.

Methods: The protocol was adapted from cohort approaches to clinical cardiovascular events ascertainment and adjudication. Potential COVID-19 hospitalizations and deaths were identified by self-/proxy-report and, in some cases, active surveillance. Medical records were requested from hospitals and adjudicated for COVID-19 outcomes by clinically trained personnel according to a standardized rubric. Inter-rater agreement was assessed. The sensitivity and specificity of discharge diagnosis codes was compared to adjudicated diagnoses.

Measurements and main results: The study obtained medical records for 1,167 potential COVID-19 hospitalizations, which underwent protocolized adjudication. Adjudication confirmed COVID-19 infection was present for 1,030 (88%) events, of which COVID-19 was not the cause of hospitalization for 78 (8%). Of 952 hospitalizations determined by adjudicators to be caused by COVID-19, 319 (34%) participants were critically ill and 210 (22%) died. Pneumonia was confirmed in 822 (86%) and acute kidney injury in 350 (37%); other cardiovascular and thrombotic complications were rare (2-5%). Interrater reliability among adjudicators was high (kappa = 0.85-1.00) except for myocardial infarction (kappa = 0.60). Compared to adjudication, sensitivity of discharge diagnosis codes was higher for pneumonia (84%) and pulmonary embolism (81%) than for other complications (48-70%).

Conclusions: Protocolized adjudication confirmed four out of five COVID-19 hospitalizations in a US meta-cohort and confirmed cases of pneumonia, pulmonary embolism, and other conditions that were not indicated by discharge diagnosis codes. These results highlight the importance of validating health outcomes for use in research on COVID-19 and post-COVID-19 conditions, and some limitations of claims-based data.

PubMed Disclaimer

Conflict of interest statement

A Krishnaswamy left Columbia University and joined Merck (Epidemiology) as a project manager following completion of her work on this paper. Mitchell Elkind receives royalties from UpToDate for a chapter on neurological complications of COVID-19; receives study drug in kind from the BMS-Pfizer Alliance for Eliquis and ancillary funding from Roche, both for an NIH-funded trial of stroke prevention. E B Levitan receives research funding from Amgen (to UAB) unrelated to the current work and personal fees for serving on a DSMB from University of Pittsburgh for an NIH funded trial. David Schwartz is founder and chief scientific officer of Eleven P15, a company dedicated to the diagnosis, prevention, and treatment of early presentations of pulmonary fibrosis. Joyce Lee serves as a consultant for Eleven P15. Sally Wenzel receives funding for consulting and clinical trials from AstraZeneca, GSK, Sanofi-Genzyme, Novartis, Knopp; she also receives research support from Pieris and Regeneron. PG Woodruff has served as a consultant for Roche, Sanofi, Amegne and Astra Zeneca outside the context of this paper. This does not alter our adherence to PLOS ONE policies on sharing data and materials.

Figures

Fig 1
Fig 1. Incidence of COVID-19-related hospitalization and/or death over C4R follow-up, United States, January 2020—June 2023.
Incidence is calculated per month and based on date of event.
Fig 2
Fig 2. Consort diagram of participants with COVID-19-related hospitalizations and deaths ascertained by C4R, January 2020—June 2023.
See this image and copyright information in PMC

References

    1. Leading Causes of Death. National Center for Health Statistics, Centers for Disease Control and Prevention
    1. Shiels MS, Haque AT, Berrington de González A, Freedman ND. Leading Causes of Death in the US During the COVID-19 Pandemic, March 2020 to October 2021. JAMA Internal Medicine. 2022;182(8):883–6. doi: 10.1001/jamainternmed.2022.2476 - DOI - PMC - PubMed
    1. Chen C, Haupert SR, Zimmermann L, Shi X, Fritsche LG, Mukherjee B. Global Prevalence of Post-Coronavirus Disease 2019 (COVID-19) Condition or Long COVID: A Meta-Analysis and Systematic Review The Journal of Infectious Diseases. 2022;226(9):1593–607. doi: 10.1093/infdis/jiac136 - DOI - PMC - PubMed
    1. Oelsner EC, Krishnaswamy A, Balte PP, Allen NB, Ali T, Anugu P, et al.. Collaborative Cohort of Cohorts for COVID-19 Research (C4R) Study: Study Design. American Journal of Epidemiology. 2022;191(7):1153–73. doi: 10.1093/aje/kwac032 - DOI - PMC - PubMed
    1. The Atherosclerosis Risk in Communities (ARIC) Study: design and objectives. The ARIC investigators. Am J Epidemiol. 1989;129(4):687–702. Epub 1989/04/01. . - PubMed

Grants and funding