Regret graphs, diagnostic uncertainty and Youden's Index

Abstract

Regret is the difference in outcome between the action we took and the best action we could, in retrospect, have taken. 'Tent graphs', representing decision problems under uncertainty in terms of expected regret, offer an instructive geometric supplement to standard formulae, allow instant calculations, and suggest, as we shall illustrate, novel algebraic properties. For instance, the problem of finding the pre-test probability range in which it is worthwhile doing a diagnostic test becomes trivial, even when test costs depend on the unknown disease status; we take the opportunity to explore the (non-trivial) algebraic properties of this problem. The graphs also provide a simple way to illustrate and read off the expected value of information and expected value of perfect information. This property is used to derive a clinical interpretation of Youden's Index (sensitivity + specificity - 1), namely, it is the maximum proportional reduction in expected regret achieved by the test. Finally, we examine the relationship between Youden's Index and the area under an ROC curve.