Statistical controversies in clinical research: prognostic gene signatures are not (yet) useful in clinical practice

Abstract

With the genomic revolution and the era of targeted therapy, prognostic and predictive gene signatures are becoming increasingly important in clinical research. They are expected to assist prognosis assessment and therapeutic decision making. Notwithstanding, an evidence-based approach is needed to bring gene signatures from the laboratory to clinical practice. In early breast cancer, multiple prognostic gene signatures are commercially available without having formally reached the highest levels of evidence-based criteria. We discuss specific concepts for developing and validating a prognostic signature and illustrate them with contemporary examples in breast cancer. When a prognostic signature has not been developed for predicting the magnitude of relative treatment benefit through an interaction effect, it may be wishful thinking to test its predictive value. We propose that new gene signatures be built specifically for predicting treatment effects for future patients and outline an approach for this using a cross-validation scheme in a standard phase III trial. Replication in an independent trial remains essential.

Keywords: clinical utility; evidence based; gene signature; predictive; prognostic.

Figure 1.

Example of survival curves in experimental (Exp) versus control (Ctrl) arms for patients with a high gene signature score (High score) versus patients with a low gene signature score (Low score) in the case of a prognostic gene signature (top left) or a predictive gene signature, with either quantitative (bottom left) or qualitative (bottom right) interaction.

Figure 2.

Receiver-operating characteristics curves when adding a proliferation and immune gene signature to a clinico-pathological (CP) model for pathological complete response in 845 early breast cancer patients treated with neoadjuvant anthracycline-based chemotherapy.

Figure 3.

Permutation scheme for computing the P-value of a global interaction test to evaluate the ability of a gene signature to be associated with the magnitude of treatment benefit.

Figure 4.

K-fold cross-validation process to develop a signature and to limit overfitting in the evaluation of the magnitude of treatment benefit according to gene signature values, when only one single randomized controlled clinical trial is available.