Systematic permutation testing in GWAS pathway analyses: identification of genetic networks in dilated cardiomyopathy and ulcerative colitis

Abstract

Background: Genome wide association studies (GWAS) are applied to identify genetic loci, which are associated with complex traits and human diseases. Analogous to the evolution of gene expression analyses, pathway analyses have emerged as important tools to uncover functional networks of genome-wide association data. Usually, pathway analyses combine statistical methods with a priori available biological knowledge. To determine significance thresholds for associated pathways, correction for multiple testing and over-representation permutation testing is applied.

Results: We systematically investigated the impact of three different permutation test approaches for over-representation analysis to detect false positive pathway candidates and evaluate them on genome-wide association data of Dilated Cardiomyopathy (DCM) and Ulcerative Colitis (UC). Our results provide evidence that the gold standard - permuting the case-control status - effectively improves specificity of GWAS pathway analysis. Although permutation of SNPs does not maintain linkage disequilibrium (LD), these permutations represent an alternative for GWAS data when case-control permutations are not possible. Gene permutations, however, did not add significantly to the specificity. Finally, we provide estimates on the required number of permutations for the investigated approaches.

Conclusions: To discover potential false positive functional pathway candidates and to support the results from standard statistical tests such as the Hypergeometric test, permutation tests of case control data should be carried out. The most reasonable alternative was case-control permutation, if this is not possible, SNP permutations may be carried out. Our study also demonstrates that significance values converge rapidly with an increasing number of permutations. By applying the described statistical framework we were able to discover axon guidance, focal adhesion and calcium signaling as important DCM-related pathways and Intestinal immune network for IgA production as most significant UC pathway.

Figure 1

The two distributions represent the result of the column and row I permutation test approach. The original data set revealed a total of 6,226 significantly associated genes (dashed line). Following permutations of the case–control status (red), a significantly decreased number of genes is discovered to be significant. Following the SNP permutations (row permutations I), a significantly increased number of genes was discovered to be significant. The second row based permutation strategy preserved the number of genes (6,226). The respective gene sets have been used as input for the pathway analysis.

Figure 2

Venn diagram showing the overlap between the three different approaches.

Figure 3

Overview on the 20 significant pathways across all approaches (Figure 3A), in both permutation tests (Figure 3B) and just in original calculations (Figure 3C). The figure presents the significance values for the 20 pathways (ordered clockwise according to decreasing significance as calculated by the Hypergeometric test), showing p-values < 0.05 for all three approaches. The further away from the middle the higher the significance scores (on a logarithmic scale). The grey shaded area in the middle corresponds to non-significant pathways. Significance values have been cut at 10^-5.

Figure 4

Difference between row- and column permutations. The histograms in panel A and B show for two pathways the significance values as calculated for row and column permutations, respectively. Panels C and D present the respective pathways as provided by KEGG. Here, red marked genes correspond to significant genes in our GWAS.

Figure 5

Comparison between enriched and depleted pathways. Each dot corresponds to one pathway. Red dots correspond to depleted and green dots to enriched pathways.

Figure 6

Influence of the number of permutations. The upper panel of the figure shows for column (red) and row (green) permutation tests the average significance value and the standard deviation for “Pathways in cancer”. The lower panel shows the coefficient of variation (CV) for both approaches.