A generalizable 29-mRNA neural-network classifier for acute bacterial and viral infections

Abstract

Improved identification of bacterial and viral infections would reduce morbidity from sepsis, reduce antibiotic overuse, and lower healthcare costs. Here, we develop a generalizable host-gene-expression-based classifier for acute bacterial and viral infections. We use training data (N = 1069) from 18 retrospective transcriptomic studies. Using only 29 preselected host mRNAs, we train a neural-network classifier with a bacterial-vs-other area under the receiver-operating characteristic curve (AUROC) 0.92 (95% CI 0.90-0.93) and a viral-vs-other AUROC 0.92 (95% CI 0.90-0.93). We then apply this classifier, inflammatix-bacterial-viral-noninfected-version 1 (IMX-BVN-1), without retraining, to an independent cohort (N = 163). In this cohort, IMX-BVN-1 AUROCs are: bacterial-vs.-other 0.86 (95% CI 0.77-0.93), and viral-vs.-other 0.85 (95% CI 0.76-0.93). In patients enrolled within 36 h of hospital admission (N = 70), IMX-BVN-1 AUROCs are: bacterial-vs.-other 0.92 (95% CI 0.83-0.99), and viral-vs.-other 0.91 (95% CI 0.82-0.98). With further study, IMX-BVN-1 could provide a tool for assessing patients with suspected infection and sepsis at hospital admission.

Fig. 1. HiCV schematic and results.

a schematic of hierarchical cross-validation (HiCV). The 18 studies (colored bands) of the IMX dataset are initially partitioned (first arrow) into three roughly equal groups of studies or folds. To simulate model selection and external validation, two of the three folds (inner) are grouped (second set of arrows) and used for cross-validation and training with the remaining fold (outer) used as a test set. This procedure is performed three times, once with each of the partitions of the IMX data treated as a test set. b–m HiCV analysis of bias/overfitting using 6-GM scores. b–d LR, logistic regression; (e–g) SVM, support vector machine; (h–j) XGBoost, extreme gradient-boosted trees; (k–m) MLP, multi-layer perceptrons. Each row contains HiCV results for outer folds 1 (b, e, h, k), 2 (c, f, i, l) or 3 (d, g, j, m). The x-axis is the difference between outer fold APA and inner fold CV APA. The blue density plots correspond to this difference for the top 50 models ranked by LOSO CV on the inner fold. Orange density plots show this difference for the top 50 models ranked by 5-fold CV on the inner fold. The vertical dashed line indicates equality between inner fold and outer fold APA (low bias), and density plots closer to this line highlight CV methods that potentially result in classifiers with lower generalization bias. Negative values indicate that inner fold APA was higher than outer fold APA, suggesting overfitting during training.

Fig. 2. Distribution of IMX-BVN-1 predicted bacterial and viral probabilities in IMX LOSO and Stanford ICU validation.

a,d Each dot in the scatter plot corresponds to a sample (x-axis: bacterial predicted probability, y-axis: viral predicted probability). The histogram/density plot above the scatter plot shows bacterial probabilities while the plot to the right of the scatter plot shows viral probabilities. The dotted lines indicate cutoffs for the lower and upper quartiles. b, c, e, f Receiver-operating characteristic (ROC) curves for IMX-BVN-1 in IMX (b,c) and Stanford ICU (e,f) data for bacterial-vs-other (b,e) and viral-vs-other (c,f) comparisons. AUC, area under the ROC curve.

Fig. 3. IMX-BVN-1 predicted probabilities in the Stanford ICU cohort across all clinical adjudication outcomes for bacterial and viral infections.

The gray oval in (a) highlights a small number of patients with low bacterial scores, all of whom received antibiotics and were admitted >36 h prior to enrollment. X-axis categories indicate adjudicated infection status. Open circles indicate admission timing (black/closed ≤ 36 h, white/open >36 h). For each boxplot, the box shows the median and 25^th–75^th quartile range (IQR), and the whiskers extend to the most extreme data point no further from the box than 1.5 times the IQR. Adj., adjudication; Micro., microbiology.