An inter-laboratory study to investigate the impact of the bioinformatics component on microbiome analysis using mock communities

Abstract

Despite the advent of whole genome metagenomics, targeted approaches (such as 16S rRNA gene amplicon sequencing) continue to be valuable for determining the microbial composition of samples. Amplicon microbiome sequencing can be performed on clinical samples from a normally sterile site to determine the aetiology of an infection (usually single pathogen identification) or samples from more complex niches such as human mucosa or environmental samples where multiple microorganisms need to be identified. The methodologies are frequently applied to determine both presence of micro-organisms and their quantity or relative abundance. There are a number of technical steps required to perform microbial community profiling, many of which may have appreciable precision and bias that impacts final results. In order for these methods to be applied with the greatest accuracy, comparative studies across different laboratories are warranted. In this study we explored the impact of the bioinformatic approaches taken in different laboratories on microbiome assessment using 16S rRNA gene amplicon sequencing results. Data were generated from two mock microbial community samples which were amplified using primer sets spanning five different variable regions of 16S rRNA genes. The PCR-sequencing analysis included three technical repeats of the process to determine the repeatability of their methods. Thirteen laboratories participated in the study, and each analysed the same FASTQ files using their choice of pipeline. This study captured the methods used and the resulting sequence annotation and relative abundance output from bioinformatic analyses. Results were compared to digital PCR assessment of the absolute abundance of each target representing each organism in the mock microbial community samples and also to analyses of shotgun metagenome sequence data. This ring trial demonstrates that the choice of bioinformatic analysis pipeline alone can result in different estimations of the composition of the microbiome when using 16S rRNA gene amplicon sequencing data. The study observed differences in terms of both presence and abundance of organisms and provides a resource for ensuring reproducible pipeline development and application. The observed differences were especially prevalent when using custom databases and applying high stringency operational taxonomic unit (OTU) cut-off limits. In order to apply sequencing approaches with greater accuracy, the impact of different analytical steps needs to be clearly delineated and solutions devised to harmonise microbiome analysis results.

Figure 1

(A–D) The % family abundance reported by each laboratory including the nominal and dPCR reported composition for the two materials; MCM2α, variable regions V1–2 (A) and V4–6 (B) and MCM2β, variable regions, V1–2 (C) and V4–6 (D).

Figure 2

(A–C) NDMS (non-metric multidimensional scaling) plots to see if results from the 13 laboratories cluster according to OTU assignment tool (A), OTU assignment database (B) and taxa assignment database (C). OTU assignment database was included for laboratories that used closed reference OTU picking. They generally compared their sequences against a reference database of sequences that clustered the reads into OTUs based on sequence similarity. Later, many laboratories assigned taxonomic identifiers to each of these OTUs using a separate database which had sequence data (sometimes not clustered into OTUs) and which taxa that sequence originated from.