Estimating transformations for evaluating diagnostic tests with covariate adjustment

Abstract

Receiver operating characteristic analysis is one of the most popular approaches for evaluating and comparing the accuracy of medical diagnostic tests. Although various methodologies have been developed for estimating receiver operating characteristic curves and their associated summary indices, there is no consensus on a single framework that can provide consistent statistical inference while handling the complexities associated with medical data. Such complexities might include non-normal data, covariates that influence the diagnostic potential of a test, ordinal biomarkers or censored data due to instrument detection limits. We propose a regression model for the transformed test results which exploits the invariance of receiver operating characteristic curves to monotonic transformations and accommodates these features. Simulation studies show that the estimates based on transformation models are unbiased and yield coverage at nominal levels. The methodology is applied to a cross-sectional study of metabolic syndrome where we investigate the covariate-specific performance of weight-to-height ratio as a non-invasive diagnostic test. Software implementations for all the methods described in the article are provided in the tram add-on package to the R system for statistical computing and graphics.

Keywords: Transformation model; Youden index; area under the receiver operating characteristic curve; censoring; diagnostic test; distribution regression; limit of detection; ordinal outcome; overlapping coefficient; receiver operating characteristic curve.

Figure 1.

Density functions for the model used to generate the data for the simulations. The nondiseased test results followed a standard normal distribution corresponding to an $\text{AUC}=0.5$ . The diseased test results varied with three choices of $F_{Z_{\text{DGP}}}$ : ${\mathrm{probit}}^{-1}$ , ${\mathrm{logit}}^{-1}$ , and ${\mathrm{cloglog}}^{-1}$ each of which had an AUC of 0.5, 0.65, 0.8, and 0.95. DGP: data generating process; AUC: area under the receiver operating characteristic curve.

Figure 2.

Distribution of bias from the simulation study for estimation of the AUC. The DGP for nondiseased results was $F_0(y)=\mathrm{\Phi }(y)$ and for diseased results $F_1(y)=F_{Z_{\text{DGP}}}(F_{Z_{\text{DGP}}}^{-1}(\mathrm{\Phi }(y))-\delta )$ . We varied $F_{Z_{\text{DGP}}}\in \{{\mathrm{probit}}^{-1}$ , ${\mathrm{logit}}^{-1}$ , ${\mathrm{cloglog}}^{-1}\}$ , AUC and sample size. The proposed methods also varied by the same inverse link functions. An alignment of colors in the column (DGP) and the fill of the box plot is indicative that the method is correctly specified for the DGP. DGP: data generating process; AUC: area under the receiver operating characteristic curve.

Figure 3.

Estimates from the linear transformation model with a single shift parameter, $h(Y)=\delta d+Z$ , where $Z$ is chosen to be a standard logistic distribution. (A) Density functions of WHtR for the workers who were diagnosed with MetS (dotted line) and those who were not (solid line). (B) ROC curve for WHtR as a marker of MetS with 95% uniform score confidence bands are represented by gray shaded areas. MetS: metabolic syndrome; WHtR: waist-to-height ratio.

Figure 4.

Estimated covariate-specific ROC curves for WHtR as a marker of MetS for female (solid line) and male workers (dashed line). Vertical panels represent a specific age (30, 40, 50) and horizontal panels smoking status. ROC: receiver operating characteristic; MetS: metabolic syndrome; WHtR: waist-to-height ratio.

Figure 5.

Age-based AUC and Youden indices where WHtR is used as a marker to detect MetS for non-smoking female (solid line) and male (dashed line) workers. 95% Wald pointwise confidence bands are represented by gray shaded areas. AUC: area under the receiver operating characteristic curve; MetS: metabolic syndrome; WHtR: waist-to-height ratio.