A bi-Poisson model for clustering gene expression profiles by RNA-seq

Abstract

With the availability of gene expression data by RNA-seq, powerful statistical approaches for grouping similar gene expression profiles across different environments have become increasingly important. We describe and assess a computational model for clustering genes into distinct groups based on the pattern of gene expression in response to changing environment. The model capitalizes on the Poisson distribution to capture the count property of RNA-seq data. A two-stage hierarchical expectation–maximization (EM) algorithm is implemented to estimate an optimal number of groups and mean expression amounts of each group across two environments. A procedure is formulated to test whether and how a given group shows a plastic response to environmental changes. The impact of gene–environment interactions on the phenotypic plasticity of the organism can also be visualized and characterized. The model was used to analyse an RNA-seq dataset measured from two cell lines of breast cancer that respond differently to an anti-cancer drug, from which genes associated with the resistance and sensitivity of the cell lines are identified. We performed simulation studies to validate the statistical behaviour of the model. The model provides a useful tool for clustering gene expression data by RNA-seq, facilitating our understanding of gene functions and networks.

Figure 1:

Plot of BIC values over the number of clusters calculated from a transcripteomic study involving two breast cancer cell lines that are sensitive and resistant to tamoxifen, respectively.

Figure 2:

Mean values of gene expression for 10 distinct groups in sensitive and resistant cell lines of breast cancer. (A) Absolute values of gene expression in the two cell lines. (B) Differences of gene expression from sensitive to resistant cell lines.

Figure 3:

Comparison of estimated gene expression (solid) with true values (broken) for 10 distinct groups from simulated data by mimicking the transcripteomic study of breast cancer. (A) Absolute values of gene expression in the two cell lines. (B) Differences of gene expression from sensitive to resistant cell lines.