Integrating Clinical Signs at Presentation and Clinician's Non-analytical Reasoning in Prediction Models for Serious Bacterial Infection in Febrile Children Presenting to Emergency Department

Abstract

Objective: Development and validation of clinical prediction model (CPM) for serious bacterial infections (SBIs) in children presenting to the emergency department (ED) with febrile illness, based on clinical variables, clinician's "gut feeling," and "sense of reassurance.

Materials and methods: Febrile children presenting to the ED of Children's Clinical University Hospital (CCUH) between April 1, 2017 and December 31, 2018 were enrolled in a prospective observational study. Data on clinical signs and symptoms at presentation, together with clinician's "gut feeling" of something wrong and "sense of reassurance" were collected as candidate variables for CPM. Variable selection for the CPM was performed using stepwise logistic regression (forward, backward, and bidirectional); Akaike information criterion was used to limit the number of parameters and simplify the model. Bootstrapping was applied for internal validation. For external validation, the model was tested in a separate dataset of patients presenting to six regional hospitals between January 1 and March 31, 2019.

Results: The derivation cohort consisted of 517; 54% (n = 279) were boys, and the median age was 58 months. SBI was diagnosed in 26.7% (n = 138). Validation cohort included 188 patients; the median age was 28 months, and 26.6% (n = 50) developed SBI. Two CPMs were created, namely, CPM1 consisting of six clinical variables and CPM2 with four clinical variables plus "gut feeling" and "sense of reassurance." The area under the curve (AUC) for receiver operating characteristics (ROC) curve of CPM1 was 0.744 (95% CI, 0.683-0.805) in the derivation cohort and 0.692 (95% CI, 0.604-0.780) in the validation cohort. AUC for CPM2 was 0.783 (0.727-0.839) and 0.752 (0.674-0.830) in derivation and validation cohorts, respectively. AUC of CPM2 in validation population was significantly higher than that of CPM1 [p = 0.037, 95% CI (-0.129; -0.004)]. A clinical evaluation score was derived from CPM2 to stratify patients in "low risk," "gray area," and "high risk" for SBI.

Conclusion: Both CPMs had moderate ability to predict SBI and acceptable performance in the validation cohort. Adding variables "gut feeling" and "sense of reassurance" in CPM2 improved its ability to predict SBI. More validation studies are needed for the assessment of applicability to all febrile patients presenting to ED.

Keywords: fever; gut feeling; non-analytical reasoning; prediction model; serious bacterial infection.

Figure 1

Receiver operating characteristic curves of clinical prediction model 1 (CPM 1) for risk of serious bacterial infections (SBIs) in derivation (A) and validation (B) populations. The dots on the curves represent sensitivity and specificity at different cut-off points.

Figure 2

Receiver operating characteristic curves of clinical prediction model 2 (CPM 2) for risk of serious bacterial infections (SBIs) in derivation (A) and validation (B) populations. The dots on the curves represent sensitivity and specificity at different cut-off points.

Figure 3

Confusion matrix for discrimination between subjects with SBI and without SBI by clinical prediction model 1 (CPM 1) in research (A) and validation (B) populations with the chosen cut-off value of 0.219. Symbols: ▽ true positives; + false negatives; x false positives; ♢ true negatives. The horizontal line represents the cut-off value.

Figure 4

Confusion matrix for discrimination between subjects with SBI and without SBI by clinical prediction model 2 (CPM 2) in research (A) and validation (B) populations with the chosen cut-off value of 0.283. Symbols: ▽ true positives; + false negatives; x false positives; ♢ true negatives. The horizontal line represents the cut-off value.

Figure 5

Categorization of patients with and without serious bacterial infection (SBI) in derivation and validation cohorts according to scoring system based on CPM 2.

Figure 6

Composition of patients with and without serious bacterial infection (SBI) within low-risk, “gray area,” and high-risk categories in derivation and validation cohorts. (A) Derivation cohort (CCUH). (B) Validation cohort (Regional hospitals).