Pretest probability estimates: a pitfall to the clinical utility of evidence-based medicine?

Abstract

Objectives: The Bayesian application of likelihood ratios has become incorporated into evidence-based medicine (EBM). This approach uses clinicians' pretest estimates of disease along with the results of diagnostic tests to generate individualized posttest disease probabilities for a given patient. To date, there is minimum scientific validation for the clinical application of this approach. This study is designed to evaluate variability in the initial step of this process, clinicians' estimates of pretest probability of disease, to assess whether this approach can be expected to yield consistent posttest disease estimates.

Methods: This cross-sectional cohort study was conducted at an urban county teaching hospital by using a sample of emergency and internal medicine residents and faculty, as well as emergency department (ED) midlevel practitioners. Participants read clinical vignettes designed to raise consideration for common ED disorders and were asked to estimate the likelihood of the suggested diagnosis based on the history and physical examination findings alone. No information about laboratory results or imaging studies was provided.

Results: Mean pretest probability estimates of disease ranged from 42% (95% confidence interval [95% CI] = 36.6% to 47.4%) to 77% (95% CI = 72.9% to 81.1%). The smallest difference in pretest probability magnitude for a single vignette was 70% (range 30-100%; interquartile range [IQR] 64-80%), whereas the largest was 95% (range 3-98%; IQR 30-60%).

Conclusions: Wide variability in clinicians' pretest probability estimates of disease may present a possible concern about decision-making models based on Bayes' theorem, because it may ultimately yield inconsistent posttest disease estimates.