Variance and confidence limits in validation studies based on comparison between three different types of measurements

Abstract

Background: The methods used in epidemiological studies to assess exposure are often affected by a conspicuous amount of measurement error. Exposure-measurement error is recognised to cause attenuation in the association between exposure and disease. Among different possible approaches, the validity coefficient of a measurement can be estimated by a comparison of three types of measurements, using either structural equation models or factor analysis (the triads method). These approaches assume that the measurements are linearly related to true intake and have independent random errors.

Methods: In this paper we present an estimator of the variance of the estimated validity coefficient to compute the associated confidence intervals. Standard error for the validity coefficient allows the efficiency of validation studies to be evaluated. Our work was motivated by the fact that existing software does not provide correct standard errors for the estimated validity coefficient. The approach is illustrated using selected examples from dietary validation studies.

Results: The accuracy of our formula is evaluated by comparison with the results of a simulation study, which shows that our variance estimator provides good results for sample sizes of at least n = 100 and when the expected value of the validity coefficient is not too close to 1.0, independent of the sample size. Our estimator formula performs better than either a naïve approach, that computes the standard error for a validity coefficient as if it is a straightforward correlation coefficient, or the SAS-CALIS procedure, which uses a maximum likelihood method.

Conclusions: In evaluating the validity of the type of measurement chosen to assess exposure in an epidemiological study, it is important to provide an estimate of the precision of the validity coefficient of the measurement. Our variance estimator may help calculate sample size requirements for validation studies.