Analysis and computational dissection of molecular signature multiplicity

Abstract

Molecular signatures are computational or mathematical models created to diagnose disease and other phenotypes and to predict clinical outcomes and response to treatment. It is widely recognized that molecular signatures constitute one of the most important translational and basic science developments enabled by recent high-throughput molecular assays. A perplexing phenomenon that characterizes high-throughput data analysis is the ubiquitous multiplicity of molecular signatures. Multiplicity is a special form of data analysis instability in which different analysis methods used on the same data, or different samples from the same population lead to different but apparently maximally predictive signatures. This phenomenon has far-reaching implications for biological discovery and development of next generation patient diagnostics and personalized treatments. Currently the causes and interpretation of signature multiplicity are unknown, and several, often contradictory, conjectures have been made to explain it. We present a formal characterization of signature multiplicity and a new efficient algorithm that offers theoretical guarantees for extracting the set of maximally predictive and non-redundant signatures independent of distribution. The new algorithm identifies exactly the set of optimal signatures in controlled experiments and yields signatures with significantly better predictivity and reproducibility than previous algorithms in human microarray gene expression datasets. Our results shed light on the causes of signature multiplicity, provide computational tools for studying it empirically and introduce a framework for in silico bioequivalence of this important new class of diagnostic and personalized medicine modalities.

Figure 1. The figure describes a class of Bayesian networks that share the same pathway structure (with 3 gene variables A, B, C and a phenotypic response variable T) and their joint probability distribution obeys the constraints shown below the structure.

Red dashed arrows denote nonzero conditional probabilities of each variable given its direct causes, and the absence of red dashed arrows denotes that these conditional probabilities are zero. For example, P(T = 0 | A = 1)≠0 while P(T = 0 | A = 2)  = 0. Genes A, B and phenotypic response variable T take 3 values {0, 1, 2}, while gene C takes two values {0, 1}.

Figure 2. High-level pseudocode of the TIE* algorithm.

Non-redundancy is not explicitly checked during the operation of TIE* but is a required property of the base Markov boundary algorithm. Details are provided in Text S3.

Figure 3. Plot of classification performance (AUC) in the validation dataset versus classification performance in the discovery dataset for each signature output by each method for the Leukemia 5 yr. Prognosis task.

Each dot in the graph corresponds to a signature (SVM computational model of the phenotype).

Figure 4. Plot of classification performance (AUC) in the validation dataset versus classification performance in the discovery dataset averaged over 6 pairs of datasets.

Axes are magnified for better visualization.