Methods for Assessing Improvement in Specificity when a Biomarker is Combined with a Standard Screening Test

Abstract

Biomarkers that can be used in combination with established screening tests to reduce false positive rates are in considerable demand. In this article, we present methods for evaluating the diagnostic performance of combination tests that require positivity on a biomarker test in addition to a standard screening test. These methods rely on relative true and false positive rates to measure the loss in sensitivity and gain in specificity associated with the combination relative to the standard test. Inference about the relative rates follows from noting their interpretation as conditional probabilities. These methods are extended to evaluate combinations with continuous biomarker tests by introducing a new statistical entity, the relative receiver operating characteristic (rROC) curve. The rROC curve plots the relative true positive rate versus the relative false positive rate as the biomarker threshold for positivity varies. Inference can be made by applying existing ROC methodology. We illustrate the methods with two examples: a breast cancer biomarker study proposed by the Early Detection Research Network (EDRN) and a prostate cancer case-control study examining the ability of free prostate-specific antigen (PSA) to improve the specificity of the standard PSA test.

Figure 1

The empirical relative ROC curve (rROC) for the combined PSA and FPSA test compared with PSA alone. The point estimate for $\text{r}{\widehat{\text{ROC}}}^{-1}$ (0.90) = 0.59 is marked with a solid vertical line. The 90% confidence interval for $\text{r}{\widehat{\text{ROC}}}^{-1}$ (0.90) is also shown.

Figure 2

Age-specific empirical relative ROC (rROC) curves for the combined PSA and FPSA test compared with PSA alone. A 90% confidence interval for $\text{r}{\widehat{\text{ROC}}}^{-1}$ (0.90) is also shown for the age-specific groups.

Figure 3

The grey line shows the empirical rROC curve for the restricted combination, PSA combined with FPSA when 4 ≤ PSA ≤ 10 ng/mL, compared with PSA alone. A 90% confidence interval for $\text{r}{\widehat{\text{ROC}}}^{-1}$ (.90) is also shown for the restricted combination. The rROC curve for the simple ‘and combination’ is overlaid (black line) for comparison.