Variable performance of weaning-predictor tests: role of Bayes' theorem and spectrum and test-referral bias

Abstract

Objective: We examined whether variation in reported reliability of the frequency-to-tidal volume ratio (f/V(T)) in predicting weaning success is explained by spectrum and test-referral bias, as reflected by variation in pretest probability of success.

Design: Two authors extracted data from all studies on reliability of f/V(T) as a weaning predictor.

Results: Prevalence of successful weaning in studies of f/V(T) revealed significant heterogeneity; mean success rate was 0.75. The heterogeneity and high success rate reflects occurrence of spectrum bias, suggested by the lower value of f/V(T) in subsequent studies than in the original report (77.4 vs. 89.1) and test-referral bias, suggested by lower specificity of f/V(T) in subsequent studies than in the original report (0.52 vs. 0.64). When data from studies in the ACCP Task Force's meta-analysis of studies on f/V(T) were entered into a Bayesian model with pretest probability (prevalence of success) as the operating point, observed posttest probabilities were closely correlated with values predicted by the original report on f/V(T): positive-predictive value r = 0.86 and negative-predictive value r = 0.82. Average sensitivity, the most precise measure of screening-test reliability, was 0.87 +/- 0.14 and average specificity 0.52 +/- 0.26.

Conclusions: Much of the heterogeneity in performance of f/V(T) can be explained by variation in pretest probability of successful outcome, which may be secondary to spectrum and test-referral bias. The average sensitivity of 0.87 indicates that f/V(T) is a reliable screening test for successful weaning.