Three-way ROCs

Abstract

Receiver operating characteristic (ROC) analysis traditionally has dealt with dichotomous diagnostic tasks (e.g., determining whether a disorder is present or absent). Often, however, medical problems involve distinguishing among more than two diagnostic alternatives. This article extends ROC concepts to diagnostic enterprises with three possible outcomes. For a trichotomous decision task, one can plot a ROC surface on three-dimensional coordinates; the volume under the ROC surface (VUS) equals the probability that test values will allow a decision maker to correctly sort a trio of items containing a randomly-selected member from each of three populations. Thus, the VUS summarizes global diagnostic accuracy for trichotomous tests, just as the area under a ROC curve does for a two-alternative diagnostic task. Information gain at points on the surface can be calculated just as is done for two-dimensional ROC curves, and investigators can thus compare three-way ROCs by comparing maximum information gain on each ROC surface.