Resampling and cross-validation techniques: a tool to reduce bias caused by model building?

Abstract

The process of model building involved in the analysis of many medical studies may lead to a considerable amount of over-optimism with respect to the predictive ability of the 'final' regression model. In this paper we illustrate this phenomenon in a simple cutpoint model and explore to what extent bias can be reduced by using cross-validation and bootstrap resampling. These computer intensive methods are compared to an ad hoc approach and to a heuristic method. Besides illustrating all proposals with the data from a breast cancer study we perform a simulation study in order to assess the quality of the methods.