Use of methodological standards in diagnostic test research. Getting better but still not good

Abstract

Objective: To determine the frequency and temporal changes in application of seven accepted methodological standards for the evaluation of diagnostic tests.

Data sources: A search of the MEDLINE database yielded 1302 articles about diagnostic test studies, during a 16-year secular interval, 1978 through 1993, in four prominent general medical journals.

Study selection: In the 112 eligible studies, the test was intended for clinical use, indexes of accuracy (sensitivity and specificity or likelihood ratios) were provided, and more than 10 patients were enrolled.

Data extraction: Although each study was critically reviewed by one primary observer, a subset was independently evaluated for interrater consistency.

Data synthesis: The percentage of studies that fulfilled criteria for each of the seven methodological standards are as follows: (1) specify spectrum of evaluated patients, 27%; (2) report test indexes for clinical subgroups, 8%; (3) avoid workup bias, 46%; (4) avoid review bias, 38%; (5) provide numerical precision for test indexes, 11%; (6) report frequency and management of indeterminate results when calculating test indexes, 22%; and (7) specify test reproducibility, 23%. Secular increases were found for six of the seven standards in ranges of use from 14% to 31% during 1978-1981 to 1990-1993. Nevertheless, only one standard, avoidance of workup bias, was fulfilled by more than 50% of studies in the most recent secular interval.

Conclusions: These results indicate that most diagnostic tests are still inadequately appraised. The routine demand for methodological standards could raise the quality of diagnostic test information, and the careful predissemination evaluation of diagnostic tests could eliminate useless tests before they receive widespread application.