The impact of sample imbalance on identifying differentially expressed genes

Abstract

Background: Recently several statistical methods have been proposed to identify genes with differential expression between two conditions. However, very few studies consider the problem of sample imbalance and there is no study to investigate the impact of sample imbalance on identifying differential expression genes. In addition, it is not clear which method is more suitable for the unbalanced data.

Results: Based on random sampling, two evaluation models are proposed to investigate the impact of sample imbalance on identifying differential expression genes. Using the proposed evaluation models, the performances of six famous methods are compared on the unbalanced data. The experimental results indicate that the sample imbalance has a great influence on selecting differential expression genes. Furthermore, different methods have very different performances on the unbalanced data. Among the six methods, the welch t-test appears to perform best when the size of samples in the large variance group is larger than that in the small one, while the Regularized t-test and SAM outperform others on the unbalanced data in other cases.

Conclusion: Two proposed evaluation models are effective and sample imbalance should be taken into account in microarray experiment design and gene expression data analysis. The results and two proposed evaluation models can provide some help in selecting suitable method to process the unbalanced data.

Figure 1

The results on prostate and liver datasets under the evaluation model 1. The expected Overlap Rates of six methods as well as their error limits on prostate and liver datasets under the evaluation model 1, where the sizes of samples in Class C₁ of the artificial data, which are created from the liver data and the prostate data, are all fixed at 60.

Figure 2

The results on prostate and liver datasets under the evaluation model 2. The expected Overlap Rates of six methods as well as their error limits on prostate and liver datasets under the evaluation model 2, where the number of overall samples in the artificial data from liver data is fixed at 120 and that from the prostate data is fixed at 60.

Figure 3

The expected performances of six methods on the simulated data with equal variances, i.e. σ₁ = σ₂= 0.5. The expected Precision Rates and Recall Rates of six methods as well as their error limits on the simulated data with equal variances (σ₁ = σ₂ = 0.5), where the number of samples of class C₁ is fixed at 60 in the evaluation model 1 and the number of overall samples is fixed at 60 in the evaluation model 2.

Figure 4

The expected performances of six methods under the evaluation model 1 on the simulated data with unequal variances, where σ₁ = 0.5, σ₂ = 1. The expected Precision Rates and Recall Rates of six methods as well as their error limits on the simulated data with unequal variances (σ₁ = 0.5, σ₂ = 1) in the evaluation model 1, where the numbers of samples of class C₁ and class C₂ are fixed at 60, respectively.

Figure 5

The expected performances of six methods under the evaluation model 2 on the simulated data with unequal variances, where σ₁ = 0.5, σ₂ = 1 and n₁ + n₂ ≡ 60. The expected Precision Rates and Recall Rates of six methods as well as their error limits on the simulated data with unequal variances (σ₁ = 0.5,σ₂ = 1) in the evaluation model 2, where the number of overall samples is fixed at 60.

Figure 6

The average performance Regularized t-test minus the corresponding performance of Welch t-test on the simulated data with varied variance σ₂, where n₁ + n₂ ≡ 60 and σ₁ ≡ 0.5. The average Precision Rate and Recall Rate of Regularized t-test minus that of Welch t-test on the simulated data with varied variance σ₂, where σ₁ ≡ 0.5 and n₁ + n₂ ≡ 60.