Academic calculations versus clinical judgments: practicing physicians' use of quantitative measures of test accuracy

Abstract

Purpose: To determine how often practicing physicians use the customarily recommended quantitative methods that include sensitivity, specificity, and likelihood ratio indexes; receiver operator characteristic (ROC) curves; and Bayesian diagnostic calculations.

Participants and methods: A random sample of 300 practicing physicians (stratified by specialty to include family physicians, general internists, general surgeons, pediatricians, obstetrician/gynecologists, and internal medicine subspecialists) were briefly interviewed in a telephone survey. They were asked about the frequency with which they used the formal methods, the reasons for non-use, and if they employed alternative strategies when appraising tests' diagnostic accuracy.

Results: Of the 300 surveyed physicians, 8 (3%) used the recommended formal Bayesian calculations, 3 used ROC curves, and 2 used likelihood ratios. The main reasons cited for non-use included impracticality of the Bayesian method (74%), and nonfamiliarity with ROC curves and likelihood ratios (97%). Of the 174 physicians who said they used sensitivity and specificity indexes, 165 (95%) did not do so in the recommended formal manner. Instead, the physicians directly estimated tests' diagnostic accuracy by determining how often the test results were correct in groups of patients later found to have, or to be free of, the selected disease.

Conclusions: The results indicate that most practicing physicians do not use the formal recommended quantitative methods to appraise tests' diagnostic accuracy, and instead report using an alternative direct approach. Although additional training might make physicians use the formal methods more often, the physicians' direct method merits further evaluation as a potentially pragmatic tool for the determination of tests' diagnostic accuracy in clinical practice.