Statistical methods for assessment of added usefulness of new biomarkers

Abstract

The discovery and development of new biomarkers continues to be an exciting and promising field. Improvement in prediction of risk of developing disease is one of the key motivations in these pursuits. Appropriate statistical measures are necessary for drawing meaningful conclusions about the clinical usefulness of these new markers. In this review, we present several novel metrics proposed to serve this purpose. We use reclassification tables constructed on the basis of clinically meaningful disease risk categories to discuss the concepts of calibration, risk separation, risk discrimination, and risk classification accuracy. We discuss the notion that the net reclassification improvement (NRI) is a simple yet informative way to summarize information contained in risk reclassification tables. In the absence of meaningful risk categories, we suggest a 'category-less' version of the NRI and integrated discrimination improvement as metrics to summarize the incremental value of new biomarkers. We also suggest that predictiveness curves be preferred to receiver operating characteristic curves as visual descriptors of a statistical model's ability to separate predicted probabilities of disease events. Reporting of standard metrics, including measures of relative risk and the c statistic, is still recommended. These concepts are illustrated with a risk prediction example using data from the Framingham Heart Study.

Figure 1

Increase in area under the curve (c statistic) as a function of baseline c statistic for three different effect sizes

Figure 2

Predictiveness curves for coronary heart disease risk prediction model without (old model) and with HDL cholesterol (new model)