Reducing cost in sequential testing: a limit of indifference approach

Abstract

In noninferiority studies, a limit of indifference is used to express a tolerance in results such that the clinician would regard such results as being acceptable or 'not worse'. We applied this concept to a measure of accuracy, the Receiver Operating Characteristic (ROC) curve, for a sequence of tests. We expressed a limit of indifference for the range of acceptable sensitivity values and examined the associated cost of testing within this range. In doing so, we generated the minimum cost maximum ROC (MCMROC) curve, which reflects the reduced sensitivity and cost of testing. We compared the MCMROC and its associated cost curve between limits of indifference set to 0.999 [a 0.1% reduction in true positive rate (TPR)], 0.95 (a 5% reduction in TPR), and 1 (no reduction in TPR). The limit of indifference tended to have less of an effect on the MCMROC curves than on the associated cost curves that were greatly affected. Cost was reduced at high false positive rates (FPRs) at higher limit of indifference (0.999) and at small FPRs as the limit of indifference decreased (0.95). These patterns were also observed as applied to sequential strategies used to diagnose diabetes in the Pima Indians.