Development and validation of a diagnostic model for early differentiation of sepsis and non-infectious SIRS in critically ill children - a data-driven approach using machine-learning algorithms

Abstract

Background: Since early antimicrobial therapy is mandatory in septic patients, immediate diagnosis and distinction from non-infectious SIRS is essential but hampered by the similarity of symptoms between both entities. We aimed to develop a diagnostic model for differentiation of sepsis and non-infectious SIRS in critically ill children based on routinely available parameters (baseline characteristics, clinical/laboratory parameters, technical/medical support).

Methods: This is a secondary analysis of a randomized controlled trial conducted at a German tertiary-care pediatric intensive care unit (PICU). Two hundred thirty-eight cases of non-infectious SIRS and 58 cases of sepsis (as defined by IPSCC criteria) were included. We applied a Random Forest approach to identify the best set of predictors out of 44 variables measured at the day of onset of the disease. The developed diagnostic model was validated in a temporal split-sample approach.

Results: A model including four clinical (length of PICU stay until onset of non-infectious SIRS/sepsis, central line, core temperature, number of non-infectious SIRS/sepsis episodes prior to diagnosis) and four laboratory parameters (interleukin-6, platelet count, procalcitonin, CRP) was identified in the training dataset. Validation in the test dataset revealed an AUC of 0.78 (95% CI: 0.70-0.87). Our model was superior to previously proposed biomarkers such as CRP, interleukin-6, procalcitonin or a combination of CRP and procalcitonin (maximum AUC = 0.63; 95% CI: 0.52-0.74). When aiming at a complete identification of sepsis cases (100%; 95% CI: 87-100%), 28% (95% CI: 20-38%) of non-infectious SIRS cases were assorted correctly.

Conclusions: Our approach allows early recognition of sepsis with an accuracy superior to previously described biomarkers, and could potentially reduce antibiotic use by 30% in non-infectious SIRS cases. External validation studies are necessary to confirm the generalizability of our approach across populations and treatment practices.

Trial registration: ClinicalTrials.gov number: NCT00209768; registration date: September 21, 2005.

Keywords: Diagnosis; Intensive care unit; Pediatric; Random Forest; SIRS; Sepsis.

Fig. 1

Flow diagram showing the selection criteria for included non-infectious SIRS and sepsis episodes. Sepsis and non-infectious SIRS were discriminated according to the International Pediatric Sepsis Consensus Conference (IPSCC) criteria [1, 3], and were confirmed by two blinded experienced pediatric intensive care physicians. Each episode of disease was assigned to either non-infectious SIRS or sepsis without ambiguity

Fig. 2

Graphical illustration of the backward variable selection process based on the out-of-bag area under the curve (OOB-AUC). Left panel: Area under the curve (AUC) based permutation variable importance measure (VIM) ordered by importance of included variable; the VIM is a proxy for the importance of the variable for correct outcome prediction, but has not the same meaning as classic influence measures based on distributional statistics (like effect sizes (e.g. Odds Ratios) or p values). Right panel: Areas under the curve by number of included predictor variables (as determined by out-of-bag area under the curve (OOB-AUC) procedure). Corresponding variables can be found in Additional file 1: Table S3

Fig. 3

ROC analysis comparing the diagnostic performance of the developed model against previously proposed biomarkers. Left panel: The ROC curve of our proposed model (solid black line; AUC: 0.78; 95% CI: 0.70–0.87) was compared against previously proposed single biomarkers in the test data set. C-reactive protein (CRP, solid grey line; AUC = 0.57; 95% CI: 0.47–0.68), interleukin-6 (IL-6, dot-dashed black line; AUC = 0.63; 95% CI: 0.52–0.74) and procalcitonin (PCT, dashed grey line; AUC = 0.55; 95% CI: 0.34–0.56). Specificity represents the correct identification of sepsis, sensitivity the correct identification of SIRS cases. Right Panel: The ROC curve of our proposed model (solid black line; AUC: 0.78; 95% CI: 0.70–0.87) was compared against previously proposed combinations of biomarkers. CRP and PCT based on a logistic regression model allowing (dot-dashed black line; AUC = 0.54; 95% CI: 0.43–0.65) and not allowing for interaction (solid grey line; AUC = 0.56; 95% CI: 0.45–0.66). Specificity represents the correct identification of sepsis, sensitivity the correct identification of SIRS cases