Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Jul 10;7(1):e175.
doi: 10.1017/cts.2023.549. eCollection 2023.

A framework for future national pediatric pandemic respiratory disease severity triage: The HHS pediatric COVID-19 data challenge

Timothy Bergquist #  1 , Marie Wax #  2 , Tellen D Bennett  3 , Richard A Moffitt  4 , Jifan Gao  5 , Guanhua Chen  5 , Amalio Telenti  6 , M Cyrus Maher  6 , Istvan Bartha  6 , Lorne Walker  7 , Benjamin E Orwoll  7 , Meenakshi Mishra  7 , Joy Alamgir  8 , Bruce L Cragin  9 , Christopher H Ferguson  2 , Hui-Hsing Wong  2 , Anne Deslattes Mays  10 , Leonie Misquitta  11 , Kerry A DeMarco  2 , Kimberly L Sciarretta  2 , Sandeep A Patel  2
Affiliations

A framework for future national pediatric pandemic respiratory disease severity triage: The HHS pediatric COVID-19 data challenge

Timothy Bergquist et al. J Clin Transl Sci. .

Abstract

Introduction: With persistent incidence, incomplete vaccination rates, confounding respiratory illnesses, and few therapeutic interventions available, COVID-19 continues to be a burden on the pediatric population. During a surge, it is difficult for hospitals to direct limited healthcare resources effectively. While the overwhelming majority of pediatric infections are mild, there have been life-threatening exceptions that illuminated the need to proactively identify pediatric patients at risk of severe COVID-19 and other respiratory infectious diseases. However, a nationwide capability for developing validated computational tools to identify pediatric patients at risk using real-world data does not exist.

Methods: HHS ASPR BARDA sought, through the power of competition in a challenge, to create computational models to address two clinically important questions using the National COVID Cohort Collaborative: (1) Of pediatric patients who test positive for COVID-19 in an outpatient setting, who are at risk for hospitalization? (2) Of pediatric patients who test positive for COVID-19 and are hospitalized, who are at risk for needing mechanical ventilation or cardiovascular interventions?

Results: This challenge was the first, multi-agency, coordinated computational challenge carried out by the federal government as a response to a public health emergency. Fifty-five computational models were evaluated across both tasks and two winners and three honorable mentions were selected.

Conclusion: This challenge serves as a framework for how the government, research communities, and large data repositories can be brought together to source solutions when resources are strapped during a pandemic.

Keywords: COVID-19; Pediatrics; community challenges; evaluation; machine learning.

PubMed Disclaimer

Conflict of interest statement

The views expressed are solely those of the authors and do not necessarily represent those of the U.S. Department of Health and Human Services. Timothy Bergquist, Tellen Bennett, and Richard Moffitt disclosed that this work was performed by Sage Bionetworks and its subcontractors under a grant with the National Institute of Health (U24TR002306). Additional funding for Timothy Bergquist was provided through the Bill and Melinda Gates Foundation (INV- 018455). Marie Wax and Hui-Hsing Wong disclose that they are government support contractors employed by Aveshka Inc. and Tunnell Government Services Inc. respectively, which receives funds from the U.S. government under contract to provide technical and programmatic support for HHS-BARDA. Joy Alamgir discloses that he is a founder and a shareholder of ARIScience. Tellen Bennett has received funding from the National Institutes of Health – NCATS, Eunice Kennedy Shriver NICHD, and National Heart, Lung and Blood Institute (NHLBI).

Figures

Figure 1.
Figure 1.
Time to outcome assessments for the top 10 quantitative scoring models in Task 1 predicting hospitalization by weeks 1–4 from the time of the outpatient visit. All team names have been removed except for the winning team. Outcome time bins are defined as noncumulative counts of patients who have the outcome in question within the given time window. Patients included in week 1 are not included in week 4 and are excluded from AUROC calculation. The boxplot represents the bootstrapped distribution of the AUROC (n = 100) for the given time window. The bar chart on the right shows the number of patients who have the outcome of interest during the given time window.
Figure 2.
Figure 2.
Time to outcome assessments for the top 10 quantitative scoring models in Task 2 predicting cardiovascular interventions and mechanical ventilation separately by days 0–4 from the time of hospitalization. All team names have been removed except for the winning team. Outcome time bins are defined as noncumulative counts of patients who have the outcome in question within the given time window. Patients included in day 0 are not included in day 4 and are excluded from AUROC calculation. The boxplot represents the bootstrapped distribution of the AUROC (n = 100) for the given time window. The bar chart on the right shows the number of patients who have the outcome of interest during the given time window.
Figure 3.
Figure 3.
Results of evaluating the winners and honorable mentions on N3C data from the time period when the omicron variant was the most prevalent strain of COVID-19. The main challenge data was used to evaluate models during the challenge. Each boxplot compares the bootstrapped distributions (n = 100) of models performance using AUROC between the main challenge data and the omicron data. The results from Task 1 are in the left column and the results from Task 2 are in the right column.
See this image and copyright information in PMC

References

    1. Kristie E.N. Seroprevalence of Infection-Induced SARS-CoV-2 Antibodies — United States, September 2021–February 2022. https://www.cdc.gov/mmwr/volumes/71/wr/mm7117e3.htm. Accessed April 29, 2022. - PMC - PubMed
    1. Children and COVID-19: State-Level Data Report. American Academy of Pediatrics. https://www.aap.org/en/pages/2019-novel-coronavirus-covid-19-infections/.... Accessed December 29, 2022.
    1. Delahoy MJ, Ujamaa D, Taylor CA, et al. Comparison of influenza and COVID-19-associated hospitalizations among children < 18 years old in the United States-FluSurv-NET (October-April 2017-2021) and COVID-NET (October 2020-September 2021). Clin Infect Dis: An Offic Pub Infect Dis Soc Am 2023; 76(3): e450–e459. - PMC - PubMed
    1. Feldstein LR, Rose EB, Horwitz SM, et al. Multisystem inflammatory syndrome in U.S. children and adolescents. N Engl J Med. 2020;383(4):334–346. - PMC - PubMed
    1. Feldstein LR, Tenforde MW, Friedman KG, et al. Characteristics and outcomes of US children and adolescents with multisystem inflammatory syndrome in children (MIS-C) compared with severe acute COVID-19. JAMA. 2021;325(11):1074–1087. - PMC - PubMed

LinkOut - more resources