A critical review of the first 10 years of candidate gene-by-environment interaction research in psychiatry

Abstract

Objective: Gene-by-environment interaction (G×E) studies in psychiatry have typically been conducted using a candidate G×E (cG×E) approach, analogous to the candidate gene association approach used to test genetic main effects. Such cG×E research has received widespread attention and acclaim, yet cG×E findings remain controversial. The authors examined whether the many positive cG×E findings reported in the psychiatric literature were robust or if, in aggregate, cG×E findings were consistent with the existence of publication bias, low statistical power, and a high false discovery rate.

Method: The authors conducted analyses on data extracted from all published studies (103 studies) from the first decade (2000-2009) of cG×E research in psychiatry.

Results: Ninety-six percent of novel cG×E studies were significant compared with 27% of replication attempts. These findings are consistent with the existence of publication bias among novel cG×E studies, making cG×E hypotheses appear more robust than they actually are. There also appears to be publication bias among replication attempts because positive replication attempts had smaller average sample sizes than negative ones. Power calculations using observed sample sizes suggest that cG×E studies are underpowered. Low power along with the likely low prior probability of a given cG×E hypothesis being true suggests that most or even all positive cG×E findings represent type I errors.

Conclusions: In this new era of big data and small effects, a recalibration of views about groundbreaking findings is necessary. Well-powered direct replications deserve more attention than novel cG×E findings and indirect replications.

FIGURE 1. Boxplots of Sample Sizes for Three Classifications of Replication Studies in Candidate Gene-by-Environment (cG×E) Interaction Research^a

^a Positive replications significantly replicated (p<0.05) a previous cG×E effect. Negative replications failed to replicate a previous cG×E effect. Pure negative replications (a subset of negative replication attempts) failed to replicate a previous cG×E effect and were published alone. Boxes are first and third quartiles; black lines represent whiskers (maximum and minimum non-outlier values). Outliers (values beyond 1.5 box lengths from the first or third quartile) are shown as points.

FIGURE 2. Testing for Publication Bias in Replication Attempts of Candidate Gene-by-Environment (cG×E) Interaction Research^a

^a The graphs show power as a function of sample size for three potential cG×E effect sizes (panel A) and distribution of observed sample sizes in the cG×E literature (panel B).

FIGURE 3. The False Discovery Rate As a Function of Statistical Power and the Prior (the Percentage of Hypotheses That Are True)^a

^a α=0.05.