Diagnostic biases in translational bioinformatics

Abstract

Background: With the surge of translational medicine and computational omics research, complex disease diagnosis is more and more relying on massive omics data-driven molecular signature detection. However, how to detect and prevent possible diagnostic biases in translational bioinformatics remains an unsolved problem despite its importance in the coming era of personalized medicine.

Methods: In this study, we comprehensively investigate the diagnostic bias problem by analyzing benchmark gene array, protein array, RNA-Seq and miRNA-Seq data under the framework of support vector machines for different model selection methods. We further categorize the diagnostic biases into different types by conducting rigorous kernel matrix analysis and provide effective machine learning methods to conquer the diagnostic biases.

Results: In this study, we comprehensively investigate the diagnostic bias problem by analyzing benchmark gene array, protein array, RNA-Seq and miRNA-Seq data under the framework of support vector machines. We have found that the diagnostic biases happen for data with different distributions and SVM with different kernels. Moreover, we identify total three types of diagnostic biases: overfitting bias, label skewness bias, and underfitting bias in SVM diagnostics, and present corresponding reasons through rigorous analysis. Compared with the overfitting and underfitting biases, the label skewness bias is more challenging to detect and conquer because it can be easily confused as a normal diagnostic case from its deceptive accuracy. To tackle this problem, we propose a derivative component analysis based support vector machines to conquer the label skewness bias by achieving the rivaling clinical diagnostic results.

Conclusions: Our studies demonstrate that the diagnostic biases are mainly caused by the three major factors, i.e. kernel selection, signal amplification mechanism in high-throughput profiling, and training data label distribution. Moreover, the proposed DCA-SVM diagnosis provides a generic solution for the label skewness bias overcome due to the powerful feature extraction capability from derivative component analysis. Our work identifies and solves an important but less addressed problem in translational research. It also has a positive impact on machine learning for adding new results to kernel-based learning for omics data.

Fig. 1

The kernel matrices of the overfitting bias. The first row illustrates the box-plots of all pairwise sample distance squares in each data. The second row lists the kernel matrices of the three data sets under the ‘rbf’ kernel (σ=1), where each data is viewed as the population of training data, are identity matrices

Fig. 2

The distributions of α values. The distributions of α values of each diagnostic trial in the 5-fold cross validation for three data sets. The skewness of sample label distribution leads to the skewness of the distributions of α values of the diagnoses of the BreastIBC and Kidney data sets. The signs of the α values indicate the group property of corresponding support vectors. As such, more support vectors can be found for the majority-count type, which will increase the likelihood of an unknown sample to be detected as the majority-count type in diagnosis

Fig. 3

The comparisons of the kernel matrices in the label skewness and underfitting biases. The comparisons of the kernel matrices of the underfitting bias (‘mlp’ kernels) and those of the linear kernels for the three data sets. The linear kernel matrices appear to be normal ones though the label skewness bias happens to the BreastIBC and Kidney data

Fig. 4

ROC plots. The ROC plots of DCA-SVM, SVM, PCA-SVM, ICA-SVM diagnoses under the 5-fold cross validation for the BreastIBC and Kidney data

Fig. 5

The phenotype separation. The phenotype separation for four different data sets: GliomaRNASeq (LGG RNA-Seq), GliomaMiRNASeq (LGG MiRNA-Seq), Kidney (Kidney (KIRC) RNA-Seq), and HCC (HCC MALDI-TOF) by using the top three biomarkers