Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2009 Jun 1;104(486):512-523.
doi: 10.1198/jasa.2009.0017.

Random Effects Models in a Meta-Analysis of the Accuracy of Two Diagnostic Tests Without a Gold Standard

Haitao Chu  1 , Sining Chen, Thomas A Louis
Affiliations

Random Effects Models in a Meta-Analysis of the Accuracy of Two Diagnostic Tests Without a Gold Standard

Haitao Chu et al. J Am Stat Assoc. .

Abstract

In studies of the accuracy of diagnostic tests, it is common that both the diagnostic test itself and the reference test are imperfect. This is the case for the microsatellite instability test, which is routinely used as a prescreening procedure to identify individuals with Lynch syndrome, the most common hereditary colorectal cancer syndrome. The microsatellite instability test is known to have imperfect sensitivity and specificity. Meanwhile, the reference test, mutation analysis, is also imperfect. We evaluate this test via a random effects meta-analysis of 17 studies. Study-specific random effects account for between-study heterogeneity in mutation prevalence, test sensitivities and specificities under a nonlinear mixed effects model and a Bayesian hierarchical model. Using model selection techniques, we explore a range of random effects models to identify a best-fitting model. We also evaluate sensitivity to the conditional independence assumption between the microsatellite instability test and the mutation analysis by allowing for correlation between them. Finally, we use simulations to illustrate the importance of including appropriate random effects and the impact of overfitting, underfitting, and misfitting on model performance. Our approach can be used to estimate the accuracy of two imperfect diagnostic tests from a meta-analysis of multiple studies or a multicenter study when the prevalence of disease, test sensitivities and/or specificities may be heterogeneous among studies or centers.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Posterior distributions of MSI and MUT sensitivities (A), MSI and MUT specificities (B). It is based on the kernel smoothed density estimation of 400,000 Monte Carlo samples. Solid lines are for MSI, dashed lines are for MUT.
Figure 2
Figure 2
Study-specific posterior means with 95% equal tail credible sets of the prevalence of family (A), and registry (B), recruitment groups, MSI (C), and MUT (D) sensitivities based on the Bayesian hierarchical model IVa. Large dots and bold lines are population averaged posterior estimates with their corresponding 95% credible intervals.
Figure 3
Figure 3
Model-based Kappa versus observed Kappa statistics for assessing the conditional dependence assumption.
See this image and copyright information in PMC

References

    1. Albert PS, McShane LM, Shih JH. Latent Class Modeling Approaches for Assessing Diagnostic Error without a Gold Standard: with Applications to P53 Immunohistochemical Assays in Bladder Tumors. Biometrics. 2001;57:610–619. - PubMed
    1. Albert PS, Dodd LE. A Cautionary Note on the Robustness of Latent Class Models for Estimating Diagnostic Error without a Gold Standard. Biometrics. 2004;60:427–435. - PubMed
    1. Albert PS, Dodd LE. On Estimating Diagnostic Accuracy from Studies with Multiple Raters and Partial Gold Standard Evaluation. Journal of the American Statistical Association. 2008;103:61–73. - PMC - PubMed
    1. Andersen S. Re: Bayesian Estimation of Disease Prevalence and the Parameters of Diagnostic Tests in the Absence of a Gold Standard. American Journal of Epidemiology. 1997;145:290–291. - PubMed
    1. Boland CR, Thibodeau SN, Hamilton SR, Sidransky D, Eshleman JR, Burt RW, Meltzer SJ, Rodriguez-Bigas MA, Fodde R, Ranzani GN, Srivastava S. A National Cancer Institute Workshop on Microsatellite Instability for Cancer Detection and Familial Predisposition: Development of International Criteria for the Determination of Microsatellite Instability in Colorectal Cancer. Cancer Research. 1998;58:5248–5257. - PubMed

Publication types

LinkOut - more resources