Impact of study design, contamination, and data characteristics on results and interpretation of microbiome studies

Abstract

Advances in high-throughput molecular techniques have enabled microbiome studies in low-biomass environments, which pose unique challenges due to contamination risks. While best-practice guidelines can reduce contamination by over 90%, the impact of residual contamination and data set variability on statistical outcomes remains understudied. Here, we quantitatively assessed how study design factors influence microbiome analyses using simulated and real-world data sets. Alpha diversity was affected by sample number and community dissimilarity, but not by the number of unique taxa. Beta diversity was influenced primarily by unique taxa and group dissimilarity, with a marginal effect of sample number. The number of differentially abundant taxa depended on the number of unique taxa but was also influenced by sample number, depending on thealgorithm. Notably, contamination had a marginal impact on weighted beta diversity but altered the number of differentially abundant taxa when at least 10 contaminants were present, with a greater effect as contamination increased. Findings closely mirrored results from seven real-world low-biomass data sets. Overall, group dissimilarity and the number of unique taxa were the primary drivers of statistical outcomes. The DESeq2 algorithm outperformed ANCOM-BC when exposed to stochastically distributed contamination, but algorithms were equivocal under contamination weighted toward one group. In all cases, the rate of false positives in differential abundance analyses was <15%. Importantly, in both simulated and real-world data, contamination rarely impacts whether microbiome differences were detected but did affect the number of differentially abundant taxa. Thus, when validated protocols with internal negative controls are used, residual contamination minimally impacts statistical outcomes. Alpha diversity was affected by sample number and community dissimilarity, but not by the number of unique taxa. Beta diversity was influenced primarily by unique taxa and group dissimilarity, with a marginal effect of sample number. The number of differentially abundant taxa depended on the number of unique taxa but was also influenced by sample number, depending on the algorithm. Notably, contamination had a marginal impact on weighted beta diversity but altered the number of differentially abundant taxa when at least 10 contaminants were present, with a greater effect as contamination increased. Findings closely mirrored results from seven real-world low-biomass data sets. Overall, group dissimilarity and the number of unique taxa were the primary drivers of statistical outcomes. The DESeq2 algorithm outperformed ANCOM-BC when exposed to stochastically distributed contamination, but algorithms were equivocal under contamination weighted toward one group. In all cases, the rate of false positives in differential abundance analyses was <15%. Importantly, in both simulated and real-world data, contamination rarely whether microbiome differences were detected but did affect the number of differentially abundant taxa. Thus, when validated protocols with internal negative controls are used, residual contamination minimally impacts statistical outcomes.IMPORTANCEMicrobiome studies in low-biomass environments face challenges due to contamination. However, even after implementing strict contamination prevention, control, and analysis measures, the impact of residual contamination on the validity of statistical outcomes in such studies remains a topic of ongoing discussion. Our analyses reveal that key drivers of microbiome study outcomes are group dissimilarity and the number of unique taxa, while contamination has minimal impact on statistical outcomes, primarily limited to the number of differentially abundant taxa detected. A common approach to contamination control involves removing taxa based on published contaminant lists. However, our analysis shows that these lists are highly inconsistent across studies, limiting reliability. Instead, our results support the use of internal negative controls as the most robust means of identifying and mitigating contamination. Collectively, data show that low-biomass microbiome studies have reduced power to detect differences between groups. However, when differences are observed, they are unlikely to be contamination-driven. By prioritizing validated protocols that prevent, assess, and eliminate contaminants through the use of internal negative controls, researchers can minimize the impact of contamination and improve the reliability of results.

Keywords: contamination; low microbial biomass; microbiome; real-world data; simulated data; study design.