Context bias. A problem in diagnostic radiology

Abstract

Objective: To determine whether radiologists' interpretations of images are biased by their context and by prevalence of disease in other recently observed cases.

Methods: A test set of 24 right pulmonary arteriograms with a 33% prevalence of pulmonary emboli (PE) was assembled and embedded in 2 larger groups of films. Group A contained 16 additional arteriograms, all showing PE involving the right lung, so that total prevalence was 60%. Group B contained 16 additional arteriograms without PE so that total prevalence was 20%. Six radiologists were randomly assigned to see either group first and then "cross over" to review the other group after a hiatus of at least 8 weeks. The direction of changes in a 5-point rating scale for the 2 readings of each film in the test set was compared with the sign test; mean sensitivity, specificity, and areas under receiver operating characteristic (ROC) curves were compared with the paired t test.

Results: In the context of group A's higher disease prevalence, radiologists shifted more of their diagnoses toward higher suspicion than expected by chance (P=.03, sign test). In group A, mean sensitivity for diagnosing PE was significantly higher (75% vs 60%; P=.04), and area under the ROC curve was significantly larger (0.88 vs 0.82; P=.02).

Conclusions: Radiologists' diagnoses are significantly influenced by the context of interpretation, even when spectrum and verification bias are avoided. This "context bias" effect is unique to the evaluation of subjectively interpreted tests, and illustrates the difficulty of obtaining unbiased estimates of diagnostic accuracy for both new and existing technologies.