Testing the additional predictive value of high-dimensional molecular data

Abstract

Background: While high-dimensional molecular data such as microarray gene expression data have been used for disease outcome prediction or diagnosis purposes for about ten years in biomedical research, the question of the additional predictive value of such data given that classical predictors are already available has long been under-considered in the bioinformatics literature.

Results: We suggest an intuitive permutation-based testing procedure for assessing the additional predictive value of high-dimensional molecular data. Our method combines two well-known statistical tools: logistic regression and boosting regression. We give clear advice for the choice of the only method parameter (the number of boosting iterations). In simulations, our novel approach is found to have very good power in different settings, e.g. few strong predictors or many weak predictors. For illustrative purpose, it is applied to the two publicly available cancer data sets.

Conclusions: Our simple and computationally efficient approach can be used to globally assess the additional predictive power of a large number of candidate predictors given that a few clinical covariates or a known prognostic index are already available. It is implemented in the R package "globalboosttest" which is publicly available from R-forge and will be sent to the CRAN as soon as possible.

Figure 1

Choice of m_stop. Negative log-likelihood for the original data (red) and the permuted data (black) against the number of iterations m_stop. (a) μ_X = 5, p* = 1. (b) μ_X = 0.2, p* = 200.

Figure 2

Boxplots of p-values. Boxplots of the p-values for the eight settings described in the Section 'Simulation design' using our new method with m_stop = 100, 500, 1000 and AIC-optimized m_stop (grey boxes) and using Goeman's global test (white boxes) for comparison.

Figure 3

Negative binomial log-likelihood in the real data study. Negative binomial log-likelihood as a function of m_stop for the original data sets (black) and for the permuted data sets (grey).