The functional false discovery rate with applications to genomics

Abstract

The false discovery rate (FDR) measures the proportion of false discoveries among a set of hypothesis tests called significant. This quantity is typically estimated based on p-values or test statistics. In some scenarios, there is additional information available that may be used to more accurately estimate the FDR. We develop a new framework for formulating and estimating FDRs and q-values when an additional piece of information, which we call an "informative variable", is available. For a given test, the informative variable provides information about the prior probability a null hypothesis is true or the power of that particular test. The FDR is then treated as a function of this informative variable. We consider two applications in genomics. Our first application is a genetics of gene expression (eQTL) experiment in yeast where every genetic marker and gene expression trait pair are tested for associations. The informative variable in this case is the distance between each genetic marker and gene. Our second application is to detect differentially expressed genes in an RNA-seq study carried out in mice. The informative variable in this study is the per-gene read depth. The framework we develop is quite general, and it should be useful in a broad range of scientific applications.

Keywords: q-value; FDR; Functional data analysis; Genetics of gene expression; Kernel density estimation; Local false discovery rate; Multiple hypothesis testing; RNA-seq; Sequencing depth; eQTL.

Fig. 1.

(a) P-value histograms of Wilcoxon tests for genetic association between genes and SNPs for the eQTL experiment in Smith and Kruglyak (2008), divided into six strata based on the gene–SNP basepair distance indicated by the strip names. The null hypothesis is “no association between a gene–SNP pair”. (b) P-value histograms for assessing differential gene expression in the RNA-seq study in Bottomly and others (2011), divided into six strata based on per-gene read depth indicated by the strip names. The null hypothesis is “no differential expression (for a gene) between two conditions”. In each subplot, the estimated proportion of true null hypotheses for all hypotheses in the corresponding stratum is based on Storey (2002) and indicated by the horizontal dashed line. It can be seen that gene–SNP genetic distance or per-gene read depth affects the prior probability of a gene–SNP association or differential gene expression.

Fig. 2.

Estimate of the functional null proportion for the eQTL and RNA-seq studies, using the GLM, GAM, or Kernel method. Each plot shows the estimate for different values of the tuning parameter , where the solid curve corresponds to the chosen tuning parameter value. The tuning parameter is chosen to balance the trade-off between the integrated bias and variance of the function ; details on how to choose are given in Section 2 of the supplementary material available at Biostatistics online.

Fig. 3.

The fFDR method applied for multiple testing in the eQTL and RNA-seq analyses. (a) Number of significant hypothesis tests at various target FDRs. The fFDR method (func FDR) has more significant tests than the standard FDR method (std FDR) at all target FDRs. (b) The significance regions of the fFDR method for various target FDRs, indicated by scatter plots of the p-values and informative variable. The horizontal lines indicate the significance thresholds that would be used by the standard FDR method at the same target FDRs. Clearly, these lines do not take the informative variable into account. (c) A scatter plot comparing the q-values for the standard FDR method ( axis) to the q-values for the fFDR method ( axis), colored based on the informative variable with reference line in red. It is clear that the fFDR method re-ranks the significance of hypotheses tests.