Integration and validation of host transcript signatures, including a novel 3-transcript tuberculosis signature, to enable one-step multiclass diagnosis of childhood febrile disease

Abstract

Background: Whole blood host transcript signatures show great potential for diagnosis of infectious and inflammatory illness, with most published signatures performing binary classification tasks. Barriers to clinical implementation include validation studies, and development of strategies that enable simultaneous, multiclass diagnosis of febrile illness based on gene expression.

Methods: We validated five distinct diagnostic signatures for paediatric infectious diseases in parallel using a single NanoString nCounter® experiment. We included a novel 3-transcript signature for childhood tuberculosis, and four published signatures which differentiate bacterial infection, viral infection, or Kawasaki disease from other febrile illnesses. Signature performance was assessed using receiver operating characteristic curve statistics. We also explored conceptual frameworks for multiclass diagnostic signatures, including additional transcripts found to be significantly differentially expressed in previous studies. Relaxed, regularised logistic regression models were used to derive two novel multiclass signatures: a mixed One-vs-All model (MOVA), running multiple binomial models in parallel, and a full-multiclass model. In-sample performance of these models was compared using radar-plots and confusion matrix statistics.

Results: Samples from 91 children were included in the study: 23 bacterial infections (DB), 20 viral infections (DV), 14 Kawasaki disease (KD), 18 tuberculosis disease (TB), and 16 healthy controls. The five signatures tested demonstrated cross-platform performance similar to their primary discovery-validation cohorts. The signatures could differentiate: KD from other diseases with area under ROC curve (AUC) of 0.897 [95% confidence interval: 0.822-0.972]; DB from DV with AUC of 0.825 [0.691-0.959] (signature-1) and 0.867 [0.753-0.982] (signature-2); TB from other diseases with AUC of 0.882 [0.787-0.977] (novel signature); TB from healthy children with AUC of 0.910 [0.808-1.000]. Application of signatures outside of their designed context reduced performance. In-sample error rates for the multiclass models were 13.3% for the MOVA model and 0.0% for the full-multiclass model. The MOVA model misclassified DB cases most frequently (18.7%) and TB cases least (2.7%).

Conclusions: Our study demonstrates the feasibility of NanoString technology for cross-platform validation of multiple transcriptomic signatures in parallel. This external cohort validated performance of all five signatures, including a novel sparse TB signature. Two exploratory multi-class models showed high potential accuracy across four distinct diagnostic groups.

Keywords: Bacterial infection; Diagnostics; Gene expression; Kawasaki disease; Multiclass diagnostics; Tuberculosis; Viral infection.

Fig. 1

Approaches to Diagnostic testing. A Simple schematic approach to clinical diagnostics, with reliance on traditional microbiological testing. B Application of binary diagnostic signatures in parallel, demonstrating problems relating to overlapping or contradictory results. C Application of binary signatures in series, demonstrating carry-forward error in classification. D Application of multiclass signature, avoiding these limitations. Created with BioRender.com

Fig. 2

Study overview. Schematic overview of study recruitment and analysis. Created with BioRender.com

Fig. 3

Performance of existing signatures. Plots of Disease Risk Scores by category (left) and ROC-curves (right) for five signatures. A and B Wright13 signature, with boxplots of the DRS by category A and ROC curves of the DRS and LR-probability (B). C and D Herberg2 signature, with boxplots of the DRS by category C and ROC curves of the DRS, LR-probability, and individual transcripts (D). E and F Pennisi2 signature, with boxplots of the DRS by category (E) and ROC curves of the DRS, LR-probability, and individual transcripts (F). G and H TB3 signature, with boxplots of the DRS by category (G) and ROC curves of the DRS, LR-probability, and individual transcripts (H). I and J BATF2, with boxplots of expression by category (I) and ROC-curves of BATF2 expression tasked with differentiating active TB from either controls or other disease groups J. 95% confidence intervals for ROC-curves are included for the DRS and LR-probability only in panels B, D, F and H

Fig. 4

TB3 signature performance in microarray training and validation cohorts. Boxplots of the DRS A of the TB3 signature and the correspondent ROC curves B in the training, test, and validation sets

Fig. 5

In-sample radar plots for multiclass signatures. Radar plots showing the probabilities of each class predicted by A the MOVA-model, B the Multiclass model. Probabilities separated and coloured by actual disease: Red = DB; Blue = DV; Purple = TB; Yellow = KD