Permutation testing in the presence of polygenic variation

Abstract

This article discusses problems with and solutions to performing valid permutation tests for quantitative trait loci in the presence of polygenic effects. Although permutation testing is a popular approach for determining statistical significance of a test statistic with an unknown distribution--for instance, the maximum of multiple correlated statistics or some omnibus test statistic for a gene, gene-set, or pathway--naive application of permutations may result in an invalid test. The risk of performing an invalid permutation test is particularly acute in complex trait mapping where polygenicity may combine with a structured population resulting from the presence of families, cryptic relatedness, admixture, or population stratification. I give both analytical derivations and a conceptual understanding of why typical permutation procedures fail and suggest an alternative permutation-based algorithm, MVNpermute, that succeeds. In particular, I examine the case where a linear mixed model is used to analyze a quantitative trait and show that both phenotype and genotype permutations may result in an invalid permutation test. I provide a formula that predicts the amount of inflation of the type 1 error rate depending on the degree of misspecification of the covariance structure of the polygenic effect and the heritability of the trait. I validate this formula by doing simulations, showing that the permutation distribution matches the theoretical expectation, and that my suggested permutation-based test obtains the correct null distribution. Finally, I discuss situations where naive permutations of the phenotype or genotype are valid and the applicability of the results to other test statistics.

Keywords: QTL; family studies; permutation test; polygenic effect; population structure; type I error rate.

Figure 1

QQ plots under naive phenotype residual permutations. In both plots the expected quantiles are for a ${\chi }_1^2$ distribution and the shaded area is the 95% confidence region. (A) The observed quantiles are the values of the test statistic under permutations of the trait values. (B) The observed quantiles are the values in (A) divided by the theoretical inflation factor.

Figure 2

QQ plot of the MVNpermute method. The observed quantiles are the values of the test statitic from 10,000 MVNpermutations, whereas the theoretical quantiles are those from a ${\chi }_1^2$ distribution. The shaded region is the 95% confidence bounds.

Figure 3

QQ plot of the empiric null distribution for the OLS statistic against the expected null distribution. The expected null distribution is a sample obtained by doing gene dropping. The solid line is the y = x line.