Harmonisation of in-silico next-generation sequencing based methods for diagnostics and surveillance

Abstract

Improvements in cost and speed of next generation sequencing (NGS) have provided a new pathway for delivering disease diagnosis, molecular typing, and detection of antimicrobial resistance (AMR). Numerous published methods and protocols exist, but a lack of harmonisation has hampered meaningful comparisons between results produced by different methods/protocols vital for global genomic diagnostics and surveillance. As an exemplar, this study evaluated the sensitivity and specificity of five well-established in-silico AMR detection software where the genotype results produced from running a panel of 436 Escherichia coli were compared to their AMR phenotypes, with the latter used as gold-standard. The pipelines exploited previously known genotype-phenotype associations. No significant differences in software performance were observed. As a consequence, efforts to harmonise AMR predictions from sequence data should focus on: (1) establishing universal minimum to assess performance thresholds (e.g. a control isolate panel, minimum sensitivity/specificity thresholds); (2) standardising AMR gene identifiers in reference databases and gene nomenclature; (3) producing consistent genotype/phenotype correlations. The study also revealed limitations of in-silico technology on detecting resistance to certain antimicrobials due to lack of specific fine-tuning options in bioinformatics tool or a lack of representation of resistance mechanisms in reference databases. Lastly, we noted user friendliness of tools was also an important consideration. Therefore, our recommendations are timely for widespread standardisation of bioinformatics for genomic diagnostics and surveillance globally.

Figure 1

Graphical representation of the sensitivity and specificity values (and 95% confidence intervals) for each antimicrobial detected by the 5 pipelines on the receiver operating characteristic (ROC) coordinate system. As there is one isolate with resistance to tigecycline, its sensitivity value was equal to 1 for all the pipelines.

Figure 1

Graphical representation of the sensitivity and specificity values (and 95% confidence intervals) for each antimicrobial detected by the 5 pipelines on the receiver operating characteristic (ROC) coordinate system. As there is one isolate with resistance to tigecycline, its sensitivity value was equal to 1 for all the pipelines.

Figure 1

Graphical representation of the sensitivity and specificity values (and 95% confidence intervals) for each antimicrobial detected by the 5 pipelines on the receiver operating characteristic (ROC) coordinate system. As there is one isolate with resistance to tigecycline, its sensitivity value was equal to 1 for all the pipelines.