Estimation of Transcription Factor Activity in Knockdown Studies

Abstract

Numerous methods have been developed trying to infer actual regulatory events in a sample. A prominent class of methods model genome-wide gene expression as linear equations derived from a transcription factor (TF) - gene network and optimizes parameters to fit the measured expression intensities. We apply four such methods on experiments with a TF-knockdown (KD) in human and E. coli. The transcriptome data provides clear expression signals and thus represents an extremely favorable test setting. The methods estimate activity changes of all TFs, which we expect to be highest in the KD TF. However, only in 15 out of 54 cases, the KD TFs ranked in the top 5%. We show that this poor overall performance cannot be attributed to a low effectiveness of the knockdown or the specific regulatory network provided as background knowledge. Further, the ranks of regulators related to the KD TF by the network or pathway are not significantly different from a random selection. In general, the result overlaps of different methods are small, indicating that they draw very different conclusions when presented with the same, presumably simple, inference problem. These results show that the investigated methods cannot yield robust TF activity estimates in knockdown schemes.

Figure 1

Boxplots of log2 normalized expression values for all human KD TFs, comparing respective case and control groups. For the double KD C/EBPβ & STAT3, separate boxplots for both TFs are shown. In all experiments, expression in case samples is significantly lower than in control samples, except for C/EBPβ (single and double KD) in BTICs and RUNX1 KD.

Figure 2

Boxplots of log2 normalized expression values for all E. coli KD TFs, comparing respective case and control groups. For the double KD ArcA & Fnr, separate boxplots for both TFs are shown.

Figure 3

Ranks of knocked down TFs and total number of ranked TFs per method and data set. Ranks in the top 5% of all ranked TFs are marked in green and ranks in the top 5–10% in light green. Two ranks in one table cell refer to a combined knockdown of two TFs and are given in the order of the TFs at the beginning of the table row. An empty table cell (in ISMARA column) indicates that the method was not applicable to the data set. A dash is shown when a TF was not ranked by a method (see text for explanation of different numbers of ranked genes).

Figure 4

For experiment GSE17172: Ranks of MYB (bold) and related TFs, total number of ranked TFs per method and p-value indicating significance of test whether the mean of the ranks of all related TFs is smaller than the average rank. Ranks of TFs in the top 5% of all ranked TFs are marked in dark green, ranks in the top 5–10% in green and ranks in the top 10–20% in light green. When a TF was not ranked, “−” is shown.

Figure 5

Number of overlapping TFs in the top 100 by estimating TF activity with different methods. Venn diagrams are shown for FOXM1 knockdown in human (left) and for the combined ArcA & Fnr knockdown in E. coli for the anaerobic condition (right). For RABIT and RACER, the total number of ranked TFs was below 100 in some cases (see Fig. 3).

Figure 6

Restricted network for FOSL2. The color of the inner circle corresponds to the differential expression of case vs control samples from GSE19114, SNB19 cell line with FOSL2 knockdown (log2 fold changes): blue colors correspond to downregulated, red colors to upregulated genes in the case samples; genes with missing expression are colored in grey. The color of the outer circle corresponds to the inferred activity score from biRte, ranging from 0 (no activity, white) to 1 (high activity, dark green). The edge width corresponds to the absolute correlation of the expression values between the two adjacent nodes: small absolute correlation values are marked with a thin line, higher absolute correlation values with bolder lines. Edges with missing correlation values and self-correlation were given the thinnest line width.

Figure 7

Ranks of KD TFs and total number of ranked TFs per method and data set for the restricted networks. Ranks of KD TFs in the top 5% of all ranked TFs are marked in green and ranks in the top 5–10% in light green. When a TF was not ranked, “−” is shown.