Massive metagenomic data analysis using abundance-based machine learning

Abstract

Background: Metagenomics is the application of modern genomic techniques to investigate the members of a microbial community directly in their natural environments and is widely used in many studies to survey the communities of microbial organisms that live in diverse ecosystems. In order to understand the metagenomic profile of one of the densest interaction spaces for millions of people, the public transit system, the MetaSUB international Consortium has collected and sequenced metagenomes from subways of different cities across the world. In collaboration with CAMDA, MetaSUB has made the metagenomic samples from these cities available for an open challenge of data analysis including, but not limited in scope to, the identification of unknown samples.

Results: To distinguish the metagenomic profiling among different cities and also predict unknown samples precisely based on the profiling, two different approaches are proposed using machine learning techniques; one is a read-based taxonomy profiling of each sample and prediction method, and the other is a reduced representation assembly-based method. Among various machine learning techniques tested, the random forest technique showed promising results as a suitable classifier for both approaches. Random forest models developed from read-based taxonomic profiling could achieve an accuracy of 91% with 95% confidence interval between 80 and 93%. The assembly-based random forest model prediction also reached 90% accuracy. However, both models achieved roughly the same accuracy on the testing test, whereby they both failed to predict the most abundant label.

Conclusion: Our results suggest that both read-based and assembly-based approaches are powerful tools for the analysis of metagenomics data. Moreover, our results suggest that reduced representation assembly-based methods are able to simultaneous provide high-accuracy prediction on available data. Overall, we show that metagenomic samples can be traced back to their location with careful generation of features from the composition of microbes and utilizing existing machine learning algorithms. Proposed approaches show high accuracy of prediction, but require careful inspection before making any decisions due to sample noise or complexity.

Reviewers: This article was reviewed by Eugene V. Koonin, Jing Zhou and Serghei Mangul.

Keywords: CAMDA; Machine learning; MetaSUB; Metagenomics; Taxonomy profiling.

Fig. 1

The analysis pipeline presented in this paper. Here we show the two-pronged approach used in this analysis. The data were analyzed under a read-based and assembly-based approach. In the read-based approach, we used taxonomic profiling for the generation of machine learning features for city prediction. In the assembly-based approach, we used two different reduced representation paradigms to generate features for machine learning features

Fig. 2

LDA plots of the read-based approach. a LDA with all species. b LDA with rare species (present in < 5% of samples) removed

Fig. 3

Confusion matrices for the read-based approach. a Confusion matrix for the random forest model trained on a random 70/30 train/test data partition. b Confusion matrix for the random forest model trained on a random 70/30 train/test data partition of the rare-species-removed data set

Fig. 4

LDA of the assembly-based approach. a LDA of the random paired-end subset assembly (PP). b LDA of the left-only subset assembly (PL)

Fig. 5

Confusion matrices for the assembly-based approach. a Confusion matrix for the random forest model trained on a random 70/30 train/test data partition in the random paired-end subset assembly. b Confusion matrix for the random forest model trained on a random 70/30 train/test data partition of the left-only assembly