Improving the power of gene set enrichment analyses

Abstract

Background: Set enrichment methods are commonly used to analyze high-dimensional molecular data and gain biological insight into molecular or clinical phenotypes. One important category of analysis methods employs an enrichment score, which is created from ranked univariate correlations between phenotype and each molecular attribute. Estimates of the significance of the associations are determined via a null distribution generated from phenotype permutation. We investigate some statistical properties of this method and demonstrate how alternative assessments of enrichment can be used to increase the statistical power of such analyses to detect associations between phenotype and biological processes and pathways.

Results: For this category of set enrichment analysis, the null distribution is largely independent of the number of samples with available molecular data. Hence, providing the sample cohort is not too small, we show that increased statistical power to identify associations between biological processes and phenotype can be achieved by splitting the cohort into two halves and using the average of the enrichment scores evaluated for each half as an alternative test statistic. Further, we demonstrate that this principle can be extended by averaging over multiple random splits of the cohort into halves. This enables the calculation of an enrichment statistic and associated p value of arbitrary precision, independent of the exact random splits used.

Conclusions: It is possible to increase the statistical power of gene set enrichment analyses that employ enrichment scores created from running sums of univariate phenotype-attribute correlations and phenotype-permutation generated null distributions. This increase can be achieved by using alternative test statistics that average enrichment scores calculated for splits of the dataset. Apart from the special case of a close balance between up- and down-regulated genes within a gene set, statistical power can be improved, or at least maintained, by this method down to small sample sizes, where accurate assessment of univariate phenotype-gene correlations becomes unfeasible.

Keywords: Enrichment analysis; Gene set enrichment analysis; Statistical power.

Fig. 1

Null distribution for ES and ES_avg for N = 20, 40, 60, 80, 100, and 200. a HALLMARKS_MYC_TARGETS_V1, b HALLMARKS_ALLOGRAFT_REJECTION. Distributions for ES are shown in blue and those for ES_avg are shown in red

Fig. 2

Sampling distribution for ES and ES_avg for N = 20, 40, 60, 80, 100, and 200. a HALLMARKS_MYC_TARGETS_V1, b HALLMARKS_ALLOGRAFT_REJECTION

Fig. 3

Power to detect association of phenotype with HALLMARKS_MYC_TARGETS_V1 (blue) and HALLMARKS_ALLOGRAFT_REJECTION (red) with α = 0.05. Power is shown as a function of N for ES (dotted line) and ES_avg (solid line)

Fig. 4

Null distributions for ES and for avg>. Null distributions for<ES_avg > are shown for one split (ES_avg = <ES_avg>), two splits, and 25 splits. All distributions are generated for one subset of 200 samples drawn from the 294-patient cohort

Fig. 5

Distribution of ES_avg, and < ES_avg > (two splits and 25 splits) for 1000 random splitting averages. All distributions are for a single subset of 200 samples using the MYC_TARGETS_V1 gene set

Fig. 6

p value distributions over dataset realizations for ES, ES_avg and < ES_avg > for control gene sets. a Gene Set a, b Gene Set j