Adding tests to risk based guidelines: evaluating improvements in prediction for an intermediate risk group

Abstract

Using an additional test in patients who are not at high enough risk of developing disease to confidently treat, or low enough risk to confidently not treat would seem to be a straightforward way to resolve the clinical equipoise and is recommended by many guidelines. However, clear methods for evaluating the population utility of additional tests in this group have not been established. This paper includes worked examples and simulation data to show that focus on the intermediate risk group alone can be misleading and that population utility is best evaluated across the full range of risk.

Fig 1

Calculation of the expected table of predicted risk under the null for bias correction. The expected table is based on a symmetry assumption using the whole dataset and is constructed separately for the cases and controls. We use the high density lipoprotein cholesterol example in the Women’s Health Study to work through the process. HDL=high density lipoprotein; CVD=cardiovascular disease; NRI=net reclassification improvement; RI=reclassification improvement

Fig 2

Distribution of measures using cardiovascular disease cut points in a group at intermediate risk when the new marker has a true association (odds ratio of 2 for a 2 standard deviation difference) with the outcome (panel A) and no association (panel B). The boxes show the results when different populations are used to calculate the risk model with the new marker: dark blue boxes use the full population, light blue boxes use the scaled population (a random sample of the full population with the same number of participants as the intermediate risk population), and white boxes use the intermediate risk population as determined by the original model only. NRI=net reclassification improvement; RI=reclassification improvement