Compliance with methodological standards when evaluating ophthalmic diagnostic tests

Abstract

Purpose: To draw attention to the importance of methodological standards when carrying out evaluations of ophthalmic diagnostic tests by reviewing the extent of compliance with these standards in reports of evaluations published within the ophthalmic literature.

Methods: Twenty published evaluations of ophthalmic screening/diagnostic tests or technologies were independently assessed by two reviewers for compliance with the following methodological standards: specification of the spectrum composition for populations used in the evaluation, analysis of pertinent subgroups, avoidance of work-up (verification) bias, avoidance of review bias, presentation of precision of results for test accuracy, presentation of indeterminate test results, and presentation of test reproducibility.

Results: Compliance ranged from just 10% (95%CI, 1%-32%) for presentation of test reproducibility data and avoidance of review bias to 70% (95%CI, 46%-88%) for avoidance of work-up bias and presentation of indeterminate test results. Only 5 of the 20 evaluations complied with four or more of the methodological standards and none with more than five of the standards.

Conclusions: The evaluations of ophthalmic diagnostic tests discussed in this article show limited compliance with accepted methodological standards but are no worse than previously described for evaluations published in general medical journals. Adherence to these standards by researchers can improve the study design and reporting of evaluations of new diagnostic techniques. Limited compliance, combined with a lack of awareness of the standards among users of research evidence, may lead to the inappropriate adoption of new diagnostic technologies, with a consequent waste of health care resources.