Benchmarking of two bioinformatic workflows for the analysis of whole-genome sequenced Staphylococcus aureus collected from patients with suspected sepsis

Abstract

Background: The rapidly growing area of sequencing technologies, and more specifically bacterial whole-genome sequencing, could offer applications in clinical microbiology, including species identification of bacteria, prediction of genetic antibiotic susceptibility and virulence genes simultaneously. To accomplish the aforementioned points, the commercial cloud-based platform, 1928 platform (1928 Diagnostics, Gothenburg, Sweden) was benchmarked against an in-house developed bioinformatic pipeline as well as to reference methods in the clinical laboratory.

Methods: Whole-genome sequencing data retrieved from 264 Staphylococcus aureus isolates using the Illumina HiSeq X next-generation sequencing technology was used. The S. aureus isolates were collected during a prospective observational study of community-onset severe sepsis and septic shock in adults at Skaraborg Hospital, in the western region of Sweden. The collected isolates were characterized according to accredited laboratory methods i.e., species identification by MALDI-TOF MS analysis and phenotypic antibiotic susceptibility testing (AST) by following the EUCAST guidelines. Concordance between laboratory methods and bioinformatic tools, as well as concordance between the bioinformatic tools was assessed by calculating the percent of agreement.

Results: There was an overall high agreement between predicted genotypic AST and phenotypic AST results, 98.0% (989/1006, 95% CI 97.3-99.0). Nevertheless, the 1928 platform delivered predicted genotypic AST results with lower very major error rates but somewhat higher major error rates compared to the in-house pipeline. There were differences in processing times i.e., minutes versus hours, where the 1928 platform delivered the results faster. Furthermore, the bioinformatic workflows showed overall 99.4% (1267/1275, 95% CI 98.7-99.7) agreement in genetic prediction of the virulence gene characteristics and overall 97.9% (231/236, 95% CI 95.0-99.2%) agreement in predicting the sequence types (ST) of the S. aureus isolates.

Conclusions: Altogether, the benchmarking disclosed that both bioinformatic workflows are able to deliver results with high accuracy aiding diagnostics of severe infections caused by S. aureus. It also illustrates the need of international agreement on quality control and metrics to facilitate standardization of analytical approaches for whole-genome sequencing based predictions.

Keywords: Antimicrobial susceptibility; Benchmarking; Clinical microbiology; Illumina sequencing; S. aureus; Species identification; Virulence genes; Whole-genome sequencing.

Fig.1

Overview of the bacterial isolates in the study. During a prospective observational study of community-onset severe sepsis and septic shock in adults conducted at Skaraborg Hospital, Sweden Ljungstrom [8] approximately 1,800 bacterial isolates were recovered. Definite species identification of the collected isolates was performed by MALDI-TOF MS, identifying 272 bacterial isolates as S. aureus. Five isolates could not be recovered after freezing. In all, 267 isolates were prepared for DNA extraction and WGS. The output FASTQ pair-ended (PE) files for three of the isolates were excluded from the dataset after quality control of the raw data and the remaining 264 S. aureus were used as input into the bioinformatic analysis in the in-house pipeline and 1928 platform. During the benchmarking of species identification (step 1) nine FASTQ files did not pass the quality control levels in the 1928 platform. Only when benchmarking species identification, the depth/coverage was lowered to 11-29X. During the benchmarking of antibiotic sensitivity, virulence genes and ST (step 2) output from 255 isolates identified as S. aureus both phenotypically and genotypically were included

Fig. 2

Overview of the in-house pipeline. The in-house pipeline consists of a number of manual steps; input of raw data, preprocessing of paired-end (PE) FASTQ files, assembly and scaffolding followed by annotation of the assembled contigs in FASTA format. The outputs from the annotation; sequence type (MLST), virulence gene characterization (VirulenceFinder), species identification (JSpeciesWS) and presence of genes conferring antibiotic resistance (ResFinder) were manually sorted and collected into a summary (Excel-format). Circle represents data files and each box represents a component corresponding to a series of tasks that provide a certain well-defined functionality (indicated in bold). Bioinformatics tool employed in each module are also mentioned (indicated in italics)