Transferring entropy to the realm of GxG interactions

Abstract

Genome-wide association studies are moving to genome-wide interaction studies, as the genetic background of many diseases appears to be more complex than previously supposed. Thus, many statistical approaches have been proposed to detect gene-gene (GxG) interactions, among them numerous information theory-based methods, inspired by the concept of entropy. These are suggested as particularly powerful and, because of their nonlinearity, as better able to capture nonlinear relationships between genetic variants and/or variables. However, the introduced entropy-based estimators differ to a surprising extent in their construction and even with respect to the basic definition of interactions. Also, not every entropy-based measure for interaction is accompanied by a proper statistical test. To shed light on this, a systematic review of the literature is presented answering the following questions: (1) How are GxG interactions defined within the framework of information theory? (2) Which entropy-based test statistics are available? (3) Which underlying distribution do the test statistics follow? (4) What are the given strengths and limitations of these test statistics?

Keywords: entropy; estimation; genetic interactions; information theory.

Figure 1

$H(X_1)$: Entropy of a random variable X₁$:=\mathbb{E}\left[\hspace{0.17em}{\text{log}}_2\left(\frac{1}{\mathbf{P}(X)}\right)\right]=-{\sum }_{i=1}^m{p}_i\hspace{0.17em}{\text{log}}_2\left(p_i\right)$

Figure 2

$H(X_1,X_2)$: Joint entropy of the two random variables X₁ and X₂$:=\mathbb{E}\left[\hspace{0.17em}{\text{log}}_2\left(\frac{1}{\mathbf{P}(X_1,X_2)}\right)\right]=-{\sum }_{i=1}^m{\sum }_{j=1}^{m\prime }{p}_{ij}\hspace{0.17em}{\text{log}}_2\left(p_{ij}\right)$

Figure 3

$H(X_1|X_2)$: Conditional entropy of the variable X₁ given the variable X₂$:=\mathbb{E}\left[\hspace{0.17em}{\text{log}}_2\left(\frac{\mathbf{P}(X_2)}{\mathbf{P}(X_1,X_2)}\right)\right]=-{\sum }_{i=1}^m{\sum }_{j=1}^{m\prime }{p}_{ij}\hspace{0.17em}{\text{log}}_2\left(\frac{p_{ij}}{p_{·j}}\right)=H(X_1,X_2)-H(X_2)$

Figure 4

$I(X_1,X_2)$: Mutual information of the variables X₁ and X₂$:=\mathbb{E}\left[\hspace{0.17em}{\text{log}}_2\left(\frac{\mathbf{P}(X_1,X_2)}{\mathbf{P}(X_1)\mathbf{P}(X_2)}\right)\right]={\sum }_{i=1}^m{\sum }_{j=1}^{m\prime }{p}_{ij}\hspace{0.17em}{\text{log}}_2\left(\frac{p_{ij}}{p_{i·}{p}_{·j}}\right)=H(X_2)H(X_2|X_1)=H(X_1)H(X_1|X_2)$

Figure 5

$TCI(X_1,X_2,X_3)$: Total correlation information of three random variables $H(X_1)+H(X_2)+H(X_3)-H(X_1,X_2,X_3)$

Figure 6

$3WII(X_1,X_2,X_3)$: Three-way interaction information of three random variables $H(X_1,X_2)+H(X_1,X_3)+H(X_2,X_3)-H(X_1)-H(X_2)-H(X_3)-H(X_1,X_2,X_3)$

Figure 7

Flow diagram of the search process.