Compositionality, sparsity, spurious heterogeneity, and other data-driven challenges for machine learning algorithms within plant microbiome studies

Abstract

The plant-associated microbiome is a key component of plant systems, contributing to their health, growth, and productivity. The application of machine learning (ML) in this field promises to help untangle the relationships involved. However, measurements of microbial communities by high-throughput sequencing pose challenges for ML. Noise from low sample sizes, soil heterogeneity, and technical factors can impact the performance of ML. Additionally, the compositional and sparse nature of these datasets can impact the predictive accuracy of ML. We review recent literature from plant studies to illustrate that these properties often go unmentioned. We expand our analysis to other fields to quantify the degree to which mitigation approaches improve the performance of ML and describe the mathematical basis for this. With the advent of accessible analytical packages for microbiome data including learning models, researchers must be familiar with the nature of their datasets.

Keywords: Compositional data analysis; Deep learning; Machine learning; Plant-associated microbiome.

Figure 1.. The emergence of compositionality and sparsity in high-throughput sequencing-based microbiome studies.

We present a hypothetical comparison of two microbial populations with distinct absolute abundances (all values are arbitrary units). Investigators collect and sequence one sample per population, which results in some species with low abundance (Yellow Pentagon in Population 2) to be excluded from the collected sample, leading to a value of zero for said community. Unequal sequencing depths lead to non-quantification of other species (Green Hexagon in Population 2) uniquely due to the lack of corresponding sequencing counts. Finally, the compositional nature of the experimental setup and the resulting dataset leads to observed changes caused by a difference in relative abundance only, leading to bias in differential abundance compared to absolute changes.

Figure 2.. Typical Steps in Preparing Microbiome Sequencing Data for ML.

We show a common sequence of processes done to prepare microbiome sequencing data for use in machine learning models, with the stages of this preparatory process where the impacts of compositionally need to be accounted for and may be mitigated through selection of appropriate techniques.