Prediction of pyrazinamide resistance in Mycobacterium tuberculosis using structure-based machine-learning approaches

Abstract

Background: Pyrazinamide is one of four first-line antibiotics used to treat tuberculosis; however, antibiotic susceptibility testing for pyrazinamide is challenging. Resistance to pyrazinamide is primarily driven by genetic variation in pncA, encoding an enzyme that converts pyrazinamide into its active form.

Methods: We curated a dataset of 664 non-redundant, missense amino acid mutations in PncA with associated high-confidence phenotypes from published studies and then trained three different machine-learning models to predict pyrazinamide resistance. All models had access to a range of protein structural-, chemical- and sequence-based features.

Results: The best model, a gradient-boosted decision tree, achieved a sensitivity of 80.2% and a specificity of 76.9% on the hold-out test dataset. The clinical performance of the models was then estimated by predicting the binary pyrazinamide resistance phenotype of 4027 samples harbouring 367 unique missense mutations in pncA derived from 24 231 clinical isolates.

Conclusions: This work demonstrates how machine learning can enhance the sensitivity/specificity of pyrazinamide resistance prediction in genetics-based clinical microbiology workflows, highlights novel mutations for future biochemical investigation, and is a proof of concept for using this approach in other drugs.

Figure 1.

Distribution of PncA mutations from published datasets. (a) Barplot of the impact of possible missense mutations in PncA by amino acid position. High-confidence resistant (red) and susceptible (blue) mutations are overlaid on the possible missense mutations whose effect on resistance is unknown or unclear (grey). (b) Distribution of the types of mutations reported by the CRyPTIC consortium et al. (c) Missense mutations from the dataset plotted onto the PncA structure (PDB ID: 3PL1) in dark grey. A pyrazinamide molecule (orange) has been modelled into the active site.

Figure 2.

Structural and evolutionary traits correlate with mutational impact on pyrazinamide susceptibility. (a) Amino acids where >80% of mutations confer resistance are more likely to be found in the core of PncA. (b) There is only a moderate correlation between RaSP and DeepDDG, which both predict the effect of a mutation on protein stability, and MAPP and SNAP2. Resistant and susceptible mutations are plotted in red and blue, respectively. (c) The performance of individual features, as measured by the AUC of a univariable logistic regression. The dashed line denotes random guessing.

Figure 3.

Machine-learning models predict pyrazinamide resistance from structural, chemical and evolutionary features. Performance of logistic regression (LR), a simple neural network (NN) and gradient-boosted decision tree (XB) models on the (a) Training and (b) Test sets. Error bars represent 95% CIs from bootstrapping (n = 10) and brackets indicate a significant difference (z-test, P < 0.05) (c) Confusion matrices are shown for the Test set. VMEs are considered worse than MEs and hence VMEs and MEs are shaded red and pink, respectively.

Figure 4.

VMEs are concentrated on the surface of PncA. (a) The majority of VMEs and MEs are shared between the three models. (b) PncA with the corresponding residues highlighted where the shared VMEs (orange) and MEs (blue) are found. (c) The shared VMEs and MEs are predicted to have less and more effect, respectively, on the stability of the protein, as exemplified by DeepDDG and the function of the protein, according to SNAP2.

Figure 5.

Performance on a real set of clinical samples. (a) Whilst the sensitivity is high, the specificity of the gradient-boosted decision tree model on the Validation dataset is lower than observed on the Test dataset. (b) Removing samples containing a mutation that has an experimentally inconsistent phenotype increases the specificity. As expected, splitting into samples whose mutation either (c) belongs or (d) does not belong to the Train dataset further stratifies performance.