An interactive web-based tool for detecting verification (work-up) bias in studies of the efficacy of diagnostic imaging

Abstract

Rationale and objectives: Diagnostic tests are validated by comparison against a "gold standard" reference test. When the reference test is invasive or expensive, it may not be applied to all patients. This results in biased estimates of the sensitivity and specificity of the diagnostic test. This type of bias is called "verification bias" and is a common problem in imaging research. The purpose of this study was to create an interactive web-based tool to help detect the presence of prevalence of verification bias in data.

Materials and methods: The web-based tool was written using well-documented and freely available open-source software (HTML, Apache, Ruby, and R). Our calculator is currently hosted on a Mac OS X server, but should run with minor changes on most other platforms.

Results: Our program applies user-supplied study data to generate a global sensitivity analysis plot containing noncorrected and corrected values of sensitivity and specificity and 95% confidence intervals for these estimates. Interpretation of the global sensitivity analysis (GSA) plot is relatively simple. Because the characteristic GSA "butterfly" plot contains all possible values of sensitivity and specificity, any indices lying outside of this butterfly are therefore incompatible with the data, confirming that they are biased.

Conclusions: Our online calculator makes it easy for investigators to use global sensitivity analysis to demonstrate the presence of significant verification bias and to estimate bias-corrected values of sensitivity and specificity.