Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Jun 27;191(7):1290-1299.
doi: 10.1093/aje/kwac025.

Sampling Validation Data to Achieve a Planned Precision of the Bias-Adjusted Estimate of Effect

Lindsay J Collin, Richard F MacLehose, Thomas P Ahern, Jaimie L Gradus, Darios Getahun, Michael J Silverberg, Michael Goodman, Timothy L Lash

Sampling Validation Data to Achieve a Planned Precision of the Bias-Adjusted Estimate of Effect

Lindsay J Collin et al. Am J Epidemiol. .

Abstract

Data collected from a validation substudy permit calculation of a bias-adjusted estimate of effect that is expected to equal the estimate that would have been observed had the gold standard measurement been available for the entire study population. In this paper, we develop and apply a framework for adaptive validation to determine when sufficient validation data have been collected to yield a bias-adjusted effect estimate with a prespecified level of precision. Prespecified levels of precision are decided a priori by the investigator, based on the precision of the conventional estimate and allowing for wider confidence intervals that would still be substantively meaningful. We further present an applied example of the use of this method to address exposure misclassification in a study of transmasculine/transfeminine youth and self-harm. Our method provides a novel approach to effective and efficient estimation of classification parameters as validation data accrue, with emphasis on the precision of the bias-adjusted estimate. This method can be applied within the context of any parent epidemiologic study design in which validation data will be collected and modified to meet alternative criteria given specific study or validation study objectives.

Keywords: epidemiologic methods; quantitative bias analysis; study design; validation substudies.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Classification parameters for exposure misclassification in the STRONG youth cohort. Se, sensitivity; Sp, specificity; TF,transfeminine; TM, transmasculine; STRONG, Study of Transition, Outcomes and Gender.
Figure 2
Figure 2
Estimates of positive predictive value (PPV) and negative predictive value (NPV) within strata of self-inflicted injury from the adaptive validation sampling approach as compared with the full validation data, STRONG youth cohort, 2006–2014. Bars represent 95% confidence intervals. STRONG, Study of Transition, Outcomes and Gender.
Figure 3
Figure 3
P value functions for the conventional and bias-adjusted odds ratios and 95% confidence intervals associating transmasculine/transfeminine status with self-inflicted injury under the adaptive validation sampling approach using the positive and negative predictive values as compared with the full validation data, STRONG youth cohort, 2006–2014. STRONG, Study of Transition, Outcomes and Gender.
Figure 4
Figure 4
Estimates of sensitivity (Se) and specificity (Sp) from the adaptive validation sampling approach as compared with the full validation data, STRONG youth cohort, 2006–2014. Bars represent 95% confidence intervals. STRONG, Study of Transition, Outcomes and Gender.
Figure 5
Figure 5
P value functions for the conventional and bias-adjusted odds ratios and 95% confidence intervals associating transmasculine/transfeminine status with self-inflicted injury under the adaptive validation sampling approach using sensitivity and specificity as compared with the full validation data, STRONG youth cohort, 2006–2014. STRONG, Study of Transition, Outcomes and Gender.
See this image and copyright information in PMC

Similar articles

See all similar articles

References

    1. Rothman KJ, Greenland S. Planning study size based on precision rather than power. Epidemiology. 2018;29(5):599–603. - PubMed
    1. von Elm E, Altman DG, Egger M, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Int J Surg. 2014;12(12):1495–1499. - PubMed
    1. Holcroft CA, Spiegelman D. Design of validation studies for estimating the odds ratio of exposure-disease relationships when exposure is misclassified. Biometrics. 1999;55(4):1193–1201. - PubMed
    1. Spiegelman D, Rosner B, Logan R. Estimation and inference for logistic regression with covariate misclassification and measurement error in main study/validation study designs. J Am Stat Assoc. 2000;95(449):51–61.
    1. Greenland S. Variance estimation for epidemiologic effect estimates under misclassification. Stat Med. 1988;7(7):745–757. - PubMed

Publication types