Estimation and construction of confidence intervals for biomarker cutoff-points under the shortest Euclidean distance from the ROC surface to the perfection corner

Abstract

Pancreatic ductal adenocarcinoma (PDAC) is an aggressive type of cancer with a 5-year survival rate of less than 5%. As in many other diseases, its diagnosis might involve progressive stages. It is common that in biomarker studies referring to PDAC, recruitment involves three groups: healthy individuals, patients that suffer from chronic pancreatitis, and PDAC patients. Early detection and accurate classification of the state of the disease are crucial for patients' successful treatment. ROC analysis is the most popular way to evaluate the performance of a biomarker and the Youden index is commonly employed for cutoff derivation. The so-called generalized Youden index has a drawback in the three-class case of not accommodating the full data set when estimating the optimal cutoffs. In this article, we explore the use of the Euclidean distance of the ROC to the perfection corner for the derivation of cutoffs in trichotomous settings. We construct an inferential framework that involves both parametric and nonparametric techniques. Our methods can accommodate the full information of a given data set and thus provide more accurate estimates in terms of the decision-making cutoffs compared with a Youden-based strategy. We evaluate our approaches through extensive simulations and illustrate them on a PDAC biomarker study.

Keywords: 3-class; Box-Cox; Euclidean distance; ROC; Youden index; cutoffs; kernels; perfection corner.

FIGURE 1

Plots comparing the Euclidean distance and the Youden index of a biomarker score viewed from different perspectives for convenience. Euclidean distance is represented by the red line segment and Youden index is represented by the blue line segment. The useless biomarker plane is represented by the diagonal gridded plane.

FIGURE 2

Plot of coverage for c₁ and c₂ derived using the Euclidean method for all scenarios. The four plots, in order from left to right contain coverages for the following methods: (1) the delta method approach based on the assumption of normally distributed biomarker scores, (2) the Box-Cox approach, which assumes biomarker scores can be transformed to normality, (3) the naive kernel-based approach without the coverage correction of added noise for the bootstrap, and (4) the proposed kernel-based method with the coverage correction (denoted kernels w/𝜖). Each subplot contains four shaded regions, representing sample sizes of (n₁,n₂,n₃) = (50,50,50), (n₁,n₂,n₃) = (100,100,100), (n₁,n₂,n₃) = (200,200,200), and (n₁,n₂,n₃) = (150,100,50). Each shaded region contains two vertical bands of points, the left of which corresponds to the coverages of the Euclidean method, and the right of which corresponds to the Youden index, as illustrated in the leftmost subplot.

FIGURE 3

Interval widths for biomarker scores for all simulated scenarios. Scenarios include data generated from normal, lognormal, gamma, and normal mixture distributions. Sample sizes of (n₁,n₂,n₃) = (50,50,50), (n₁,n₂,n₃) = (100,100,100), (n₁,n₂,n₃) = (200,200,200), and (n₁,n₂,n₃) = (150,100,50) are included, as well as scenarios corresponding to VUS = 0.4, and VUS = 0.6. We include interval widths for the delta method approach (Del), the Box-Cox approach (BC), and the proposed kernel-based approach with added noise that was described in Section 4 (K w/ є).

FIGURE 4

The plots are histograms of the percent difference in total classification (sum of TCRs) for the Euclidean method and Youden index. Optimal cutoff values were estimated using the Euclidean and Youden methods based on training data. The estimated cutoff values were then used to derive true classification rates using the true distributions of the biomarker scores. The area to the right of 0 corresponds to a higher sum of true classification rates for the Euclidean method than for the Youden index. Plots (a)-(d) correspond to biomarkers simulated from normal distributions with equally spaced means of (μ₁,μ₂,μ₃) = (5,5.6592,6.3184), and standard deviations of (σ₁,σ₂,σ₃) = (1,1,1). Plots (e)-(h) correspond to normal distributions with means of (μ₁,μ₂,μ₃) = (10,10.8,12), and standard deviations of (σ₁,σ₂,σ₃) = (1,1,1.7)

FIGURE 5

Plots of biomarker scores and ROC surfaces for each of the biomarkers.