Extending the c-statistic to nominal polytomous outcomes: the Polytomous Discrimination Index

Abstract

Diagnostic problems in medicine are sometimes polytomous, meaning that the outcome has more than two distinct categories. For example, ovarian tumors can be benign, borderline, primary invasive, or metastatic. Extending the main measure of binary discrimination, the c-statistic or area under the ROC curve, to nominal polytomous settings is not straightforward. This paper reviews existing measures and presents the polytomous discrimination index (PDI) as an alternative. The PDI assesses all sets of k cases consisting of one case from each outcome category. For each category i (i = 1, … ,k), it is assessed whether the risk of category i is highest for the case from category i. A score of 1∕k is given per category for which this holds, yielding a set score between 0 and 1 to indicate the level of discrimination. The PDI is the average set score and is interpreted as the probability to correctly identify a case from a randomly selected category within a set of k cases. This probability can be split up by outcome category, yielding k category-specific values that result in the PDI when averaged. We demonstrate the measures on two diagnostic problems (residual mass histology after chemotherapy for testicular cancer; diagnosis of ovarian tumors). We compare the behavior of the measures on theoretical data, showing that PDI is more strongly influenced by simultaneous discrimination between all categories than by partial discrimination between pairs of categories. In conclusion, the PDI is attractive because it better matches the requirements of a measure to summarize polytomous discrimination.