Statistical Inference for Box-Cox based Receiver Operating Characteristic Curves

Abstract

Receiver operating characteristic (ROC) curve analysis is widely used in evaluating the effectiveness of a diagnostic test/biomarker or classifier score. A parametric approach for statistical inference on ROC curves based on a Box-Cox transformation to normality has frequently been discussed in the literature. Many investigators have highlighted the difficulty of taking into account the variability of the estimated transformation parameter when carrying out such an analysis. This variability is often ignored and inferences are made by considering the estimated transformation parameter as fixed and known. In this paper, we will review the literature discussing the use of the Box-Cox transformation for ROC curves and the methodology for accounting for the estimation of the Box-Cox transformation parameter in the context of ROC analysis, and detail its application to a number of problems. We present a general framework for inference on any functional of interest, including common measures such as the AUC, the Youden index, and the sensitivity at a given specificity (and vice versa). We further developed a new R package (named 'rocbc') that carries out all discussed approaches and is available in CRAN.

Keywords: Box–Cox; ROC; correlated biomarkers; delta method; sensitivity; smooth ROC; specificity.

Fig. A1

Distribution and qq-plot for the scores of the non-diseased group of marker 1.

Fig. A2

Distribution and qq-plot for the scores of the diseased group of marker 1.

Fig. A3

Distribution and qq-plot for the scores of the non-diseased group of marker 2.

Fig. A4

Distribution and qq-plot for the scores of the diseased group of marker 2.

Figure 1

Summary results of the simulations when the data are generated from three distributional scenarios (Upper row: Normal, Middle row: Gamma, Lower row: Log-Normal). Left: ISE comparisons of the empirical ROC estimator against the Box-Cox-based one and the corresponding comparison of the MRMCoav-binormal against the Box-Cox. Results that lie above the reference horizontal line in red color are favorable to the Box-Cox approach. Middle: The corresponding comparisons in terms of the MSE of the AUC estimates. Right: The corresponding comparisons in terms of the TPR estimates at a fixed FPR=0.10.

Figure 2

Output of the ’checkboxcox’ routine. The left four panels refer to the non-diseased group (histograms and QQ plots of the original and the transformed marker scores). The four panels on the right are the corresponding plots for the diseased group. The transformed scores for the control group and the diseased group are returned in the transx and transy arguments.

Figure 3

The empirical ROC estimate as well as the Box-Cox based ROC are automatically generated after running checkboxcox.

Figure 4

The egg-shaped confidence region around the Youden index-based optimal operating point as generated by rocboxcox (Bantis et al. 2014). The legend of the plot is automatically generated with an informative output.

Figure 5

Left: Graphical output of rocboxcoxCI(antib2, D, givenSP, NA, 0.05, "on"). It illustrates the ($1-a$)% CI of the sensitivities at the given pre-determined specificities provided in vector givenSP. Right: Graphical output of rocboxcoxCI(antib2, D, NA, givenSE, 0.05, "on"). It illustrates the ($1-a$)% CI of the specificities at the given pre- determined sensitivities provided in vector givenSE.

Figure 6

Left: Bivariate normal density of the transformed scores for the group of non-prior COVID19. Right: Bivariate normal density of the transformed scores for the prior COVID19 group.

Figure 7

Box-Cox-based ROC and empirical ROC curve for marker 1 (left panel) and marker 2 (right panel)

Figure 8

(a): The resulting plot obtained by comparebcJ. Comparison is made with respect to the two correlated Youden indices. (b): The resulting plot obtained by comparebcAUC. Comparison is made with respect to the two correlated AUCs. (c): The resulting plot obtained by comparebcSens. Comparison is made with respect to the two correlated Sensitivities at a given Specificity of 0.8. (d) The resulting plot obtained by comparebcSpec. Comparison is made with respect to the two correlated Sensitivities at a given Sensitivity of 0.8.

Figure 9

Left: Visual result of three ROC estimates as provided by the function ’threerocs’ for Example 1. Here the use of the Box-Cox-based ROC is not recommended as can also be seen by the ’checkboxcox’ function (p-values of normality tests are <0.001 for both groups before and after Box-Cox transforming the data). The Metz approach also fails to provide a reasonable fit since it does not adequately match the empirical ROC. Right: Visual result of three ROC estimates as provided by the function ’threerocs’ for Example 1. Here the use of the Box-Cox-based ROC is not recommended as can also be seen by the ’checkboxcox’ function (p-values of normality tests are <0.05 for the diseased group before and after Box-Cox transforming the data). The Metz approach seems to provide a reasonable fit since it adequately matches the empirical ROC.

Figure 10

Result of "threerocs1" for the one-marker case of the COVID data. This plot visualizes the fit of the Box-Cox-based ROC curves in contrast to the empirical ROC estimates of both markers and in contrast to the binormal Metz approach as given by the MRMCoav package using the binormal option. The empirical ROC curve for this plot is plotted as a benchmark of the fit.

Figure 11

Result of "threerocs2" for the two-marker case of the COVID data. This plot visualizes the fit of the Box-Cox-based ROC curves in contrast to the empirical ROC estimates of both markers and in contrast to the binormal Metz approach as given by the MRMCoav package using the binormal option. The empirical ROC curves for this plot are obtained separately for each marker and are plotted as a benchmark of the fit.