Comprehensive evaluation of published gene expression prognostic signatures for biomarker-based lung cancer clinical studies

Abstract

Background: A more accurate prognosis for non-small-cell lung cancer (NSCLC) patients could aid in the identification of patients at high risk for recurrence. Many NSCLC mRNA expression signatures claiming to be prognostic have been reported in the literature. The goal of this study was to identify the most promising mRNA prognostic signatures in NSCLC for further prospective clinical validation.

Experimental design: We carried out a systematic review and meta-analysis of published mRNA prognostic signatures for resected NSCLC. The prognostic performance of each signature was evaluated via a meta-analysis of 1927 early stage NSCLC patients collected from 15 studies using three evaluation metrics (hazard ratios, concordance scores, and time-dependent receiver-operating characteristic curves). The performance of each signature was then evaluated against 100 random signatures. The prognostic power independent of clinical risk factors was assessed by multivariate Cox models.

Results: Through a literature search, we identified 42 lung cancer prognostic signatures derived from genome-wide expression profiling analysis. Based on meta-analysis, 25 signatures were prognostic for survival after adjusting for clinical risk factors and 18 signatures carried out significantly better than random signatures. When analyzing histology types separately, 17 signatures and 8 signatures are prognostic for adenocarcinoma and squamous cell lung cancer, respectively. Despite little overlap among published gene signatures, the top-performing signatures are highly concordant in predicted patient outcomes.

Conclusions: Based on this large-scale meta-analysis, we identified a set of mRNA expression prognostic signatures appropriate for further validation in prospective clinical studies.

Keywords: meta-analysis; non-small-cell lung cancer; prognostic gene signatures.

Figure 1.

Flow chart summarizing the strategy used for the development (A) and validation (B) of the risk prediction models obtained from published gene signatures. Basically, 9 different superPCA predictors were trained for each of the 42 signatures using the nine training sets, respectively. The performance of a specific superPCA predictor was then evaluated based on a subset of 15 available data sets. The criteria to select the subset were as follows: (i), the data set was not the training set and (ii) the dataset was not used for developing the specific signature used in this superPCA predictor. Detailed descriptions of procedures and data analysis are provided in the supplementary material, available at Annals of Oncology online.

Figure 2.

Evaluation of 42 gene expression signatures via random effect meta-analysis. (A) Prognostic performance of the Shedden et al. gene signature using the Director’s Consortium dataset as the training set (10). Hazard ratios (HRs) between the two predicted risk groups varied widely when using different data as the test sets. (B and C) Heatmap of the HR and Concordance index (C.index) estimates based on meta-analysis using different prediction models and gene signatures. SuperPCA models were trained by 1 of the 9 training sets, and tested on the remaining 14 data sets. The name of each training set is shown in the column title column. The PCA model was developed using the first principal component of the signature in the test set. (D) Meta-analysis results from all SuperPCA models using all pairs of training and test sets. In (B) and (D), the x-axis denotes the HR of survival differences between two predicted risk groups, with black squares denoting the summarized HR from different training and testing strategies, and the black solid line denoting the confidence interval. For each gene signature, datasets used in signature development were excluded from meta-analysis.

Figure 3.

Over 50% of published signatures perform better than random signatures when using SuperPCA prediction models (A) and 20% perform better when using PCA prediction models (B). The x-axis denotes the P-values comparing gene signatures against 100 random signatures, based on linear mixed-effect models. Red and black bars indicate signatures that carried out better or worse than random signatures, respectively.

Figure 4.

Meta-analysis results for clinical risk factor-adjusted prognostic power. (A) Heatmap of the meta-analysis results based on different prediction models and gene signatures. SuperPCA models were trained by one of the nine training sets and tested on the remaining 14 datasets. The names of the training sets are shown in the column titles. The PCA model was developed using the first principal component of the signature in the test set. (B). Meta-analysis results from all SuperPCA models using any one of the training sets and any one of the test sets. The x-axis denotes the HR of survival differences between the two predicted risk groups. Black squares denote the summarized HR from different training and testing strategies, while the black broken solid line denotes the confidence interval. For each gene signature, datasets used in signature development were excluded from meta-analysis.