Utility of composite reference standards and latent class analysis in evaluating the clinical accuracy of diagnostic tests for pertussis

Abstract

Numerous evaluations of the clinical sensitivity and specificity of PCR and serologic assays for Bordetella pertussis have been hampered by the low sensitivity of culture, the gold standard test, which leads to biased accuracy estimates. The bias can be reduced by using statistical approaches such as the composite reference standard (CRS) (e.g., positive if culture or serology positive; negative otherwise) or latent class analysis (LCA), an internal reference standard based on a statistical model. We illustrated the benefits of the CRS and LCA approaches by reanalyzing data from a 1995 to 1996 study of cough illness among 212 patients. The accuracy of PCR in this study was evaluated using three reference standards: culture, CRS, and LCA. Using specimens obtained 0 to 34 days after cough onset, estimates of the sensitivity of PCR obtained using CRS (47%) and LCA (34%) were lower than the culture-based estimate (62%). The CRS and LCA approaches, which utilized more than one diagnostic marker of pertussis, likely produced more accurate reference standards than culture alone. In general, the CRS approach is simple, with a well-defined disease status. LCA requires statistical modeling but incorporates more indicators of disease than CRS. When three or more indicators of pertussis are available, these approaches should be used in evaluations of pertussis diagnostic tests.

FIG. 1.

Example of the observed specificity of a diagnostic test versus the prevalence of disease, by the sensitivity of culture, assuming a culture specificity of 100% and conditional independence between the diagnostic test and culture. Adjustment formulas for the observed specificity also assume a diagnostic test specificity of 85% and a diagnostic test sensitivity of 75% (46).

FIG. 2.

(a) Example of the observed sensitivity of a diagnostic test versus the prevalence of disease, when the assumption of conditional independence between the diagnostic test and an imperfect gold standard may or may not be met. The solid line indicates the assumed level of sensitivity of the diagnostic test. The short-dashed line indicates the observed sensitivity assuming conditional independence calculated by adjustment formulas (46). The long-dashed lines indicate the pointwise nonsampling intervals for the observed sensitivity assuming diagnostic test-gold standard dependence. Formulas for the nonsampling intervals assume a diagnostic test sensitivity of 75%, a gold standard sensitivity of 95%, and a gold standard specificity of 95% (50). (b) Example of the observed specificity of a diagnostic test versus the prevalence of disease, when the assumption of conditional independence between the diagnostic test and an imperfect gold standard may or may not be met. The solid line indicates the assumed level of specificity of the diagnostic test. The short-dashed line indicates the observed specificity assuming conditional independence calculated by adjustment formulas (46). The long-dashed lines indicate the pointwise nonsampling intervals for the observed specificity assuming diagnostic test-gold standard dependence. Formulas for the nonsampling intervals assume a diagnostic test specificity of 85%, a gold standard sensitivity of 95%, and a gold standard specificity of 95% (50).

FIG. 3.

(a) Conditional independence latent class model. The observed measurements made by diagnostic tests (Y_j; j = 1,…, J) are independent, given the common latent variable for subject disease status (X). If an association is observed among diagnostic tests, then it is entirely attributable to the common factor X. (b) Example of a dependence latent class model that includes a bivariate association between Y₁ and Y₂. The observed association between Y₁ and Y₂ cannot be entirely explained by the common factor X.