An adaptive permutation approach for genome-wide association study: evaluation and recommendations for use

Abstract

Background: Permutation testing is a robust and popular approach for significance testing in genomic research, which has the broad advantage of estimating significance non-parametrically, thereby safe guarding against inflated type I error rates. However, the computational efficiency remains a challenging issue that limits its wide application, particularly in genome-wide association studies (GWAS). Because of this, adaptive permutation strategies can be employed to make permutation approaches feasible. While these approaches have been used in practice, there is little research into the statistical properties of these approaches, and little guidance into the proper application of such a strategy for accurate p-value estimation at the GWAS level.

Methods: In this work, we advocate an adaptive permutation procedure that is statistically valid as well as computationally feasible in GWAS. We perform extensive simulation experiments to evaluate the robustness of the approach to violations of modeling assumptions and compare the power of the adaptive approach versus standard approaches. We also evaluate the parameter choices in implementing the adaptive permutation approach to provide guidance on proper implementation in real studies. Additionally, we provide an example of the application of adaptive permutation testing on real data.

Results: The results provide sufficient evidence that the adaptive test is robust to violations of modeling assumptions. In addition, even when modeling assumptions are correct, the power achieved by adaptive permutation is identical to the parametric approach over a range of significance thresholds and effect sizes under the alternative. A framework for proper implementation of the adaptive procedure is also generated.

Conclusions: While the adaptive permutation approach presented here is not novel, the current study provides evidence of the validity of the approach, and importantly provides guidance on the proper implementation of such a strategy. Additionally, tools are made available to aid investigators in implementing these approaches.

Figure 1

Quantile-quantile plots for ANOVA, standard and adaptive permutation under the normal and the Student's t -distribution (df = 5) null model, with 10,000 replications and 9,900 permutations.

Figure 2

Quartile-quartile plots and comparison for ANOVA and adaptive permutation under the Student's t-distribution (df = 5) null model, with 1,000,000 replications and 1,999,999,900 permutations.

Figure 3

Computation time comparison of standard and adaptive permutation for varying effect size, with 1,000 replications and 9,900 permutations.