Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2013 Dec:68:10.1016/j.csda.2013.07.007.
doi: 10.1016/j.csda.2013.07.007.

Estimating confidence intervals for the difference in diagnostic accuracy with three ordinal diagnostic categories without a gold standard

Le Kang  1 , Chengjie Xiong  2 , Lili Tian  3
Affiliations

Estimating confidence intervals for the difference in diagnostic accuracy with three ordinal diagnostic categories without a gold standard

Le Kang et al. Comput Stat Data Anal. 2013 Dec.

Abstract

With three ordinal diagnostic categories, the most commonly used measures for the overall diagnostic accuracy are the volume under the ROC surface (VUS) and partial volume under the ROC surface (PVUS), which are the extensions of the area under the ROC curve (AUC) and partial area under the ROC curve (PAUC), respectively. A gold standard (GS) test on the true disease status is required to estimate the VUS and PVUS. However, oftentimes it may be difficult, inappropriate, or impossible to have a GS because of misclassification error, risk to the subjects or ethical concerns. Therefore, in many medical research studies, the true disease status may remain unobservable. Under the normality assumption, a maximum likelihood (ML) based approach using the expectation-maximization (EM) algorithm for parameter estimation is proposed. Three methods using the concepts of generalized pivot and parametric/nonparametric bootstrap for confidence interval estimation of the difference in paired VUSs and PVUSs without a GS are compared. The coverage probabilities of the investigated approaches are numerically studied. The proposed approaches are then applied to a real data set of 118 subjects from a cohort study in early stage Alzheimer's disease (AD) from the Washington University Knight Alzheimer's Disease Research Center to compare the overall diagnostic accuracy of early stage AD between two different pairs of neuropsychological tests.

Keywords: EM algorithm; Generalized pivot; Gold standard; Parametric bootstrap; Volume under the ROC surface.

PubMed Disclaimer

Figures

Fig. 1
Fig. 1
Density contours for Config. 1: (a) with GS (left); (b) without GS (right, estimated from EM algorithm).
Fig. 2
Fig. 2
Density contours for Config. 2: (a) with GS (left); (b) without GS (right, estimated from EM algorithm).
Fig. 3
Fig. 3
Density contours for Config. 3: (a) with GS (left); (b) without GS (right, estimated from EM algorithm).
Fig. 4
Fig. 4
Density contours for Config. 4: (a) with GS (left); (b) without GS (right, estimated from EM algorithm).
See this image and copyright information in PMC

References

    1. Beiden SV, Campbell G, Meier KL, Wagner RF. On the problem of ROC analysis without truth: the EM algorithm and the information matrix. Proceedings of the Society of Photo-Optical Instrumentation Engineers (SPIE) 2000;3981:126–134.
    1. Benton D, Krishnamoorthy K. Performance of the parametric bootstrap method in small sample interval estimates. Advances and Applications in Statistics. 2002;2:269–285.
    1. Efron B. Bootstrap methods: another look at the jackknife. Annals of Statistics. 1979;7:1–26.
    1. Efron B, Tibshirani R. An Introduction to the Bootstrap. London: Chapman & Hall; 1993.
    1. Henkelman RM, Kay I, Bronskill MJ. Receiver operator characteristic (ROC) analysis without truth. Medical Decision Making. 1990;10:24–29. - PubMed

LinkOut - more resources