Systematic evaluation of supervised classifiers for fecal microbiota-based prediction of colorectal cancer

Abstract

Predicting colorectal cancer (CRC) based on fecal microbiota presents a promising method for non-invasive screening of CRC, but the optimization of classification models remains an unaddressed question. The purpose of this study was to systematically evaluate the effectiveness of different supervised machine-learning models in predicting CRC in two independent eastern and western populations. The structures of intestinal microflora in feces in Chinese population (N = 141) were determined by 454 FLX pyrosequencing, and different supervised classifiers were employed to predict CRC based on fecal microbiota operational taxonomic unit (OTUs). As a result, Bayes Net and Random Forest displayed higher accuracies than other algorithms in both populations, although Bayes Net was found with a lower false negative rate than that of Random Forest. Gut microbiota-based prediction was more accurate than the standard fecal occult blood test (FOBT), and the combination of both approaches further improved the prediction accuracy. Moreover, when unclassified OTUs were used as input, the BayesDMNB text algorithm achieved higher accuracy in the Chinese population (AUC=0.994). Taken together, our results suggest that Bayes Net classification model combined with unclassified OTUs may present an accurate method for predicting CRC based on the compositions of gut microbiota.

Keywords: CRC; gut microbiota; prediction; supervised classifier.

Figure 1. The CRC prediction performance based on fecal microbiome varied among different classifiers and populations

(A) The test performance of different models using fecal microbiome on study population A was displayed using area under roc curve (AUC). (B) The test performance of different models using fecal microbiome on study population B was displayed using AUC. (C) ROC curves show the prediction ability of the Bayes Net and Random Forest models on study population A and B, respectively. TPR means true positive rate (sensitivity). (D) ROC curves of Simple Logistic and LMT algorithms on study population A and B. (E) Confusion Matrix of Bayes Net algorithm in population A. (F) Confusion Matrix of Random Forest algorithm in population A. *P < 0.05, **P < 0.01, the Bayes Net model was used as the control model.

Figure 2. Unclassified OTUs increased the test performance compared with classified OTUs

(A) The test performance of different models using OTUs dataset classified or not on study population A was displayed using AUC. (B) ROC curves show the test performance of the Bayes DMNB text algorithm using OTUs dataset classified or not. TPR (sensitivity) and FPR (false positive rate=1-sensitivity) were also shown. (C) Confusion Matrix of Bayes.DMNB text algorithm using OTUs unclassified in population A. *P < 0.05, **P < 0.01.

Figure 3. Fecal microbiome combined with FOBT moderately improved the test prediction ability

(A) The test performance of different classifiers using fecal microbiome combined with FOBT on study population A was displayed with AUC. (B) The test performance of different models using fecal microbiome combined with FOBT on study population B was displayed using AUC. (C and D) ROC curves show the test performance of the Bayes Net and Random Forest models using fecal microbiome in combination with FOBT on study population A and B, respectively. The test performance of FOBT alone was also shown. TPR means true positive rate (sensitivity). *P < 0.05.

Figure 4. Interpretation of gut microbial species in the prediction model

(A) The resulting decision tree of J48 model on study population A. Only the leaf nodes at the higher levels are displayed while the rest are indicated by dashes. The leaf nodes that are assigned by one or two numbers where the former and latter indicate the number of correctly and incorrectly classified samples, respectively. (B) Relative abundances of 6 gut microbial species, collectively associated with CRC analyzed by J48 model, are displayed as heat map. (C) The resulting decision tree of J48 model on study population B. (D) Relative abundances of 7 gut microbial species, collectively associated with CRC analyzed by J48 model, are displayed as heat map.