Measuring precision in bioassays: Rethinking assay validation

Abstract

The m:n:θ_b procedure is often used for validating an assay for precision, where m levels of an analyte are measured with n replicates at each level, and if all m estimates of coefficient of variation (CV) are less than θ_b , then the assay is declared validated for precision. The statistical properties of the procedure are unknown so there is no clear statistical statement of precision upon passing. Further, it is unclear how to modify the procedure for relative potency assays in which the constant standard deviation (SD) model fits much better than the traditional constant CV model. We use simple normal error models to show that under constant CV across the m levels, the probability of passing when the CV is θ_b is about 10% to 20% for some recommended implementations; however, for extreme heterogeniety of CV when the largest CV is θ_b , the passing probability can be greater than 50%. We derive 100q% upper confidence limits on the CV under constant CV models and derive analogous limits for the SD under a constant SD model. Additionally, for a post-validation assay output of y, we derive 68.27% confidence intervals on either the mean or log geometric mean of the assay output using either y±s (for the constant SD model) or log(y)±rG (for the constant CV model), where s and r_G are constants that do not depend on y. We demonstrate the methods on a growth inhibition assay used to measure biologic activity of antibodies against the malaria parasite.

Keywords: assay qualification; coefficient of variation; functional assay; relative potency assay; standard deviation interval.

Figure 1

Plot of individual GIA values from each sample by the mean GIA for each sample, for both 3D7 (gray triangles) and FVO (black circles). Line is line of equality.