Estimation of significance thresholds for genomewide association scans

Abstract

The question of what significance threshold is appropriate for genomewide association studies is somewhat unresolved. Previous theoretical suggestions have yet to be validated in practice, whereas permutation testing does not resolve a discrepancy between the genomewide multiplicity of the experiment and the subset of markers actually tested. We used genotypes from the Wellcome Trust Case-Control Consortium to estimate a genomewide significance threshold for the UK Caucasian population. We subsampled the genotypes at increasing densities, using permutation to estimate the nominal P-value for 5% family-wise error. By extrapolating to infinite density, we estimated the genomewide significance threshold to be about 7.2 x 10(-8). To reduce the computation time, we considered Patterson's eigenvalue estimator of the effective number of tests, but found it to be an order of magnitude too low for multiplicity correction. However, by fitting a Beta distribution to the minimum P-value from permutation replicates, we showed that the effective number is a useful heuristic and suggest that its estimation in this context is an open problem. We conclude that permutation is still needed to obtain genomewide significance thresholds, but with subsampling, extrapolation and estimation of an effective number of tests, the threshold can be standardized for all studies of the same population.

Fig. 1

(a) Significance threshold as a function of marker density in combined NBS and 58BC sample from permutation procedure. At current density (359K single nucleotide polymorphisms typed) the significance threshold is about 2.2 × 10⁻⁷. The dotted line shows the estimated asymptote of 7.2 × 10⁻⁸. (b) Fitted Monod function to the effective number of tests associated with the significance threshold. At infinite density the number of tests is estimated at 693,138 giving the asymptote in (a).

Fig. 2

(a) Significance thresholds from permutation procedure and Patterson's estimate of the effective number of tests. At current marker density, the estimates differ by an order of magnitude. (b) The effective numbers of tests based on the permutation procedure and Patterson's estimator. At current marker density, Patterson's estimate is too low (33,279) compared to that of the permutation procedure (227,838).

Fig. 3

Quantile–xsquantile plot comparing fitted Beta distributions with minimum P-values from permutation replicates.