An Integrated Bayesian Nonparametric Approach for Stochastic and Variability Orders in ROC Curve Estimation: An Application to Endometriosis Diagnosis

Abstract

In estimating ROC curves of multiple tests, some a priori constraints may exist, either between the healthy and diseased populations within a test or between tests within a population. In this paper, we proposed an integrated modeling approach for ROC curves that jointly accounts for stochastic and variability orders. The stochastic order constrains the distributional centers of the diseased and healthy populations within a test, while the variability order constrains the distributional spreads of the tests within each of the populations. Under a Bayesian nonparametric framework, we used features of the Dirichlet process mixture to incorporate these order constraints in a natural way. We applied the proposed approach to data from the Physician Reliability Study that investigated the accuracy of diagnosing endometriosis using different clinical information. To address the issue of no gold standard in the real data, we used a sensitivity analysis approach that exploited diagnosis from a panel of experts. To demonstrate the performance of the methodology, we conducted simulation studies with varying sample sizes, distributional assumptions and order constraints. Supplementary materials for this article are available online.

Keywords: Area under the curve; Dirichlet process mixture; Gold standard; Order restricted analysis.

Figure 1

Histograms of rASRM scores in the PRS by scale and setting. The scores are not separated by disease status.

Figure 2

Histograms of log rASRM scores in the PRS by disease status and setting. The disease status is from posterior estimates in Section 5.

Figure 3

Scatter plots of REs’ rASRM scores and IEs’ diagnosis in the PRS. The fitted lines are all significant with p < 0.01.

Figure 4

Estimated ROC curves and AUCs (95% CI) from the initial analysis by setting and model.

Figure 5

The estimated predictive densities (and histograms) of log rASRM scores in the PRS by disease status and setting. The posterior mean of the density from the JO model is denoted by solid line for each population at each setting. For comparison, densities from NO (dotted line), SO (dashed line) and VO (dash-dotted line) models are provided.

Figure 6

The estimated ROC curve of log rASRM scores in the PRS by setting. The posterior mean of the ROC curve from the JO model is denoted by solid line at each setting. For comparison, the estimated ROC curves from NO (dotted line), SO (dashed line) and VO (dash-dotted line) models are provided.