A description of mixed group validation

Abstract

Mixed group validation (MGV) is a statistical model for estimating the diagnostic accuracy of tests. Unlike the more common approach to estimating criterion-related validity, known group validation (KGV), MGV does not require a perfect external validity criterion. The present article describes MGV by (a) specifying both the standard error associated with MGV validity estimates and the effect of assumption violation, (b) recommending required sample sizes under various study conditions, (c) evaluating whether assumption violation can be identified, and (d) providing a simulated example of an MGV with imperfect base rate estimates. It is concluded that MGV will always have a wider margin of error than KGV, MGV performs best when the research design approximates a KGV design, the effect of assumption violation depends on the severity of the assumption violation and also the value of the base rates, and that assumption violation may only be detected in severe cases.

Keywords: construct validity; criterion-related validity; mixed group validation; sensitivity; specificity.