Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Feb 27;331(8):675-686.
doi: 10.1001/jama.2024.0196.

Development and Validation of the Phoenix Criteria for Pediatric Sepsis and Septic Shock

Affiliations

Development and Validation of the Phoenix Criteria for Pediatric Sepsis and Septic Shock

L Nelson Sanchez-Pinto et al. JAMA. .

Abstract

Importance: The Society of Critical Care Medicine Pediatric Sepsis Definition Task Force sought to develop and validate new clinical criteria for pediatric sepsis and septic shock using measures of organ dysfunction through a data-driven approach.

Objective: To derive and validate novel criteria for pediatric sepsis and septic shock across differently resourced settings.

Design, setting, and participants: Multicenter, international, retrospective cohort study in 10 health systems in the US, Colombia, Bangladesh, China, and Kenya, 3 of which were used as external validation sites. Data were collected from emergency and inpatient encounters for children (aged <18 years) from 2010 to 2019: 3 049 699 in the development (including derivation and internal validation) set and 581 317 in the external validation set.

Exposure: Stacked regression models to predict mortality in children with suspected infection were derived and validated using the best-performing organ dysfunction subscores from 8 existing scores. The final model was then translated into an integer-based score used to establish binary criteria for sepsis and septic shock.

Main outcomes and measures: The primary outcome for all analyses was in-hospital mortality. Model- and integer-based score performance measures included the area under the precision recall curve (AUPRC; primary) and area under the receiver operating characteristic curve (AUROC; secondary). For binary criteria, primary performance measures were positive predictive value and sensitivity.

Results: Among the 172 984 children with suspected infection in the first 24 hours (development set; 1.2% mortality), a 4-organ-system model performed best. The integer version of that model, the Phoenix Sepsis Score, had AUPRCs of 0.23 to 0.38 (95% CI range, 0.20-0.39) and AUROCs of 0.71 to 0.92 (95% CI range, 0.70-0.92) to predict mortality in the validation sets. Using a Phoenix Sepsis Score of 2 points or higher in children with suspected infection as criteria for sepsis and sepsis plus 1 or more cardiovascular point as criteria for septic shock resulted in a higher positive predictive value and higher or similar sensitivity compared with the 2005 International Pediatric Sepsis Consensus Conference (IPSCC) criteria across differently resourced settings.

Conclusions and relevance: The novel Phoenix sepsis criteria, which were derived and validated using data from higher- and lower-resource settings, had improved performance for the diagnosis of pediatric sepsis and septic shock compared with the existing IPSCC criteria.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest Disclosures: Dr Sanchez-Pinto reported receipt of grants from the National Institutes of Health (NIH)/National Institute of General Medical Sciences outside the submitted work. Dr Bennett reported receipt of grants from the NIH/National Center for Advancing Translational Sciences and NIH/National Heart, Lung, and Blood Institute (NHLBI) outside the submitted work. Dr Martin reported receipt of grants from the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) and the Thrasher Research Fund outside the submitted work. Dr Balamuth reported receipt of grants from the NIH and various federal and foundation grants to study sepsis and other infectious emergencies outside the submitted work. Dr Bembea reported receipt of grants with funds paid to institution from the NIH/National Institute of Neurological Disorders and Stroke (NINDS), the NIH/NICHD, Grifols, the Department of Defense, and the NIH/NHLBI. Dr Horvat reported grants from the NICHD and NINDS outside the submitted work. Dr Zimmerman reported receipt of grants from Immunexpress and personal fees from Elsevier outside the submitted work. Dr Schlapbach reported receipt of grants from Medical Research Future Funds, the National Health and Medical Research Council, and the Swiss Personalized Health Network outside the submitted work. Dr Randolph reported receipt of grants to the institution from the NIH and the Centers for Disease Control and Prevention; receipt of personal fees from UpToDate; receipt of travel funds for advisory board meetings from Volition Inc, Thermo Fisher, and bioMérieux; and being a scientific advisor from Inotrem outside the submitted work. Dr Carrol reported being a specialist committee member, scientific advisory board member, and/or panel/group member for the UK National Institute for Health and Care Excellence (NICE) Diagnostics Advisory Committee, the NICE Sepsis Guideline Development Group, BioFire Diagnostics, the NICE Quality Standards Committee for Sepsis, the Surviving Sepsis Campaign Pediatrics Guideline Panel, and the Society of Critical Care Medicine Paediatric Sepsis Definition Task Force and an investigator in the UK National Institute for Health and Care Research (NIHR)–funded studies BATCH, PRONTO, and PEACH and the H2020-funded studies PERFORM and DIAMONDS. Dr Chiotos reported being a member of the Infectious Diseases Society of America Sepsis Taskforce. Dr Hall reported receipt of personal fees for serving as a data and safety monitoring board member from AbbVie and nonfinancial support from Partner Therapeutics and Sobi. Dr Inwald reported receipt of grants from the NIHR as chief investigator of the PRESSURE trial. Dr Peters reported receipt of grants from the NIHR outside the submitted work. Dr Tissieres reported receipt of grants from Baxter and personal fees from Baxter, Sanofi, Thermo Fisher, and Sedana. Dr Wynn reported receipt of personal fees from Sobi. No other disclosures were reported.

Figures

Figure 1.
Figure 1.. In-Hospital Mortality Associated With the Phoenix Sepsis Score in Patients in Higher-Resource Settings With Suspected Infection in the First 24 Hours
This figure shows calibration of the Phoenix Sepsis Score in higher-resource settings (sites with more technological resources, eg, laboratory equipment, ventilators, and kidney replacement therapy devices, to support organ dysfunction). For patients with suspected infection who have each possible integer value of the Phoenix Sepsis Score in the first 24 hours of the encounter, mortality among those at the development, internal validation, and external validation sites is shown. Binomial confidence intervals (whiskers) for the mortality point estimate in each group are also shown.
Figure 2.
Figure 2.. In-Hospital Mortality Associated With the Phoenix Sepsis Score in Patients in Lower-Resource Settings With Suspected Infection in the First 24 Hours
This figure shows the calibration of the Phoenix Sepsis Score in lower-resource settings (sites with fewer technological resources to support organ dysfunction). For patients with suspected infection who have each possible integer value of the Phoenix Sepsis Score in the first 24 hours of the encounter, mortality among those at the development, internal validation, and external validation sites is shown. Binomial confidence intervals (whiskers) for the mortality point estimate in each group are also shown. At lower-resource sites, some variables were rarely available (eg, D-dimer and fibrinogen for coagulation dysfunction), even when other variables for the same organ systems were recorded (eg, platelet count and international normalized ratio); thus, the maximum cumulative score achieved at lower-resource sites was 9, instead of the maximum possible of 13.
Figure 3.
Figure 3.. Mortality Prediction Performance of the Phoenix Sepsis Score and Organ Dysfunction Scores
IPSCC indicates International Pediatric Sepsis Consensus Conference; PELOD-2, Pediatric Logistic Organ Dysfunction version 2; PODIUM, Pediatric Organ Dysfunction Information Update Mandate; and pSOFA, pediatric Sequential Organ Failure Assessment. This figure compares the performance of the Phoenix Sepsis Score with validated pediatric organ dysfunction scores and criteria to predict mortality in patients with suspected infection in the first 24 hours. Equivalent performance metrics for the secondary outcome, early death or extracorporeal membrane oxygenation, are shown in eFigure 7 in Supplement 1. All types of organ dysfunction are evaluated across their respective full ranges, with higher scores indicating more organ dysfunction burden. The scores for IPSCC, Proulx, and PODIUM are based on the counts of organ dysfunction (eAppendix 1 and eTable 2 in Supplement 1). Performance is presented as both quantitative with 95% CIs (calculated using logit transform), as well as visually using a color heat map. Shading indicates highest (darkest) to lowest (lightest) in each row. The AUPRC is the area under a curve drawn with sensitivity (also referred to as “recall”) and positive predictive value (also referred to as “precision”) across all potential thresholds for the points in the scores. The AUPRC is a more reliable classifier performance metric than the AUROC when the classes are imbalanced, for example, when mortality is very low, as in this study. The AUROC is the area under a curve drawn with the false-positive rate on the x-axis and the true-positive rate on the y-axis. In this study, it is an indicator of how well a classifier can rank encounters with respect to mortality risk.
Figure 4.
Figure 4.. Comparison of Sensitivity and PPV of Novel Phoenix Sepsis Criteria With Current IPSCC Sepsis and Severe Sepsis Criteria Across Outcomes and Patient Subgroups in the Internal Validation Sets
The positive predictive value (PPV, or precision) and sensitivity for the Phoenix vs 2005 International Pediatric Sepsis Consensus Conference (IPSCC) criteria for sepsis in children with suspected infection are shown. The Phoenix sepsis criteria are based on achieving ≥2 points in the Phoenix Sepsis Score among patients with suspected infection in the first 24 hours of an encounter. The IPSCC sepsis and severe sepsis criteria are based on systemic inflammatory response syndrome (SIRS) and IPSCC-based organ dysfunction among patients with suspected infection in the first 24 hours of an encounter. Baseline rates of the outcome in each group (death, or early death or extracorporeal membrane oxygenation [ECMO]) are shown as horizontal dashed lines. 95% CIs are shown as bands from each point in the plane representing that component (eg, CIs for PPV are parallel to the y-axis). Confidence bands that are not visible are narrow enough to be completely hidden by the point. These figures are similar to area under the precision recall curves except at a single threshold for criteria that generate a binary response (eg, yes/no sepsis criteria met) instead of across the range of possible points in the curve (eg, 0-13 points in the Phoenix Sepsis Score; see Figure 3). Better-performing criteria are closer to the top right corner. A trade-off exists between sensitivity and PPV, with more sensitive criteria usually having lower PPV and more specific criteria usually having higher PPV and lower sensitivity. Criteria that are close to the baseline outcome rate have poor predictive value. aAt lower-resource site 2, some Phoenix Sepsis Score and IPSCC data inputs (eg, invasive mechanical ventilation, Glasgow Coma Scale score) are not recorded even when they are performed; thus, assessment of criteria performance is limited. Lower-resource site 1 and all higher-resource sites have inputs for all relevant organ systems in the criteria. Comparison of sepsis criteria in the external validation sites is shown in eFigure 10 in Supplement 1 with similar results. Diagnostic performance measures for this comparison are shown in eTable 7 in Supplement 1.

Comment in

Similar articles

Cited by

References

    1. Rudd KE, Johnson SC, Agesa KM, et al. Global, regional, and national sepsis incidence and mortality, 1990-2017. Lancet. 2020;395(10219):200-211. doi: 10.1016/S0140-6736(19)32989-7 - DOI - PMC - PubMed
    1. Goldstein B, Giroir B, Randolph A, et al. International pediatric sepsis consensus conference: definitions for sepsis and organ dysfunction in pediatrics. Pediatr Crit Care Med. 2005;6(1):2-8. doi: 10.1097/01.PCC.0000149131.72248.E6 - DOI - PubMed
    1. Weiss SL, Fitzgerald JC, Maffei FA, et al. Discordant identification of pediatric severe sepsis by research and clinical definitions in the SPROUT international point prevalence study. Crit Care. 2015;19(1):325. doi: 10.1186/s13054-015-1055-x - DOI - PMC - PubMed
    1. Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of clinical criteria for sepsis: for the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8):762-774. doi: 10.1001/jama.2016.0288 - DOI - PMC - PubMed
    1. Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8):801-810. doi: 10.1001/jama.2016.0287 - DOI - PMC - PubMed

Publication types