Receiver operator characteristic (ROC) analysis without truth

Abstract

Receiver operator characteristic (ROC) analysis, the preferred method of evaluating diagnostic imaging tests, requires an independent assessment of the true state of disease, which can be difficult to obtain and is often of questionable accuracy. A new method of analysis is described which does not require independent truth data and which can be used when several accurate tests are being compared. This method uses correlative information to estimate the underlying model of multivariate normal distributions of disease-positive and disease-negative patients. The method is shown to give results equivalent to conventional ROC analysis in a comparison of computed tomography, radionuclide scintigraphy, and magnetic resonance imaging for liver metastasis. When independent truth is available, the method can be extended to incorporate truth data or to evaluate the consistency of the truth data with the imaging data.