Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2006 Jul 18:6:33.
doi: 10.1186/1471-2288-6-33.

An alternative to the hand searching gold standard: validating methodological search filters using relative recall

Margaret Sampson  1 , Li Zhang, Andra Morrison, Nicholas J Barrowman, Tammy J Clifford, Robert W Platt, Terry P Klassen, David Moher
Affiliations

An alternative to the hand searching gold standard: validating methodological search filters using relative recall

Margaret Sampson et al. BMC Med Res Methodol. .

Abstract

Background: Search filters or hedges play an important role in evidence-based medicine but their development depends on the availability of a "gold standard"--a reference standard against which to establish the performance of the filter. We demonstrate the feasibility of using relative recall of included studies from multiple systematic reviews to validate methodological search filters as an alternative to validation against a gold standard formed through hand searching.

Methods: We identified 105 Cochrane reviews that used the Highly Sensitive Search Strategy (HSSS), included randomized or quasi-randomized controlled trials, and reported their included studies. We measured the ability of two published and one novel variant of the HSSS to retrieve the MEDLINE-index studies included in these reviews.

Results: The systematic reviews were comprehensive in their searches. 72% of included primary studies were indexed in MEDLINE. Relative recall of the three strategies ranged from .98 to .91 across all reviews and more comprehensive strategies showed higher recall.

Conclusion: An approach using relative recall instead of a hand searching gold standard proved feasible and produced recall figures that were congruent with previously published figures for the HSSS. This technique would permit validation of a methodological filter using a collection of approximately 100 studies of the chosen design drawn from the included studies of multiple systematic reviews that used comprehensive search strategies.

PubMed Disclaimer

Figures

Figure 1
Figure 1
QUOROM Flow Diagram.
See this image and copyright information in PMC

References

    1. Zaroukian MHM. PubMed clinical queries: a Web tool for filtered retrieval of citations relevant to evidence-based practice. ACP Journal Club. 2001;134:A15.
    1. Glanville JM, Lefebvre C, Miles JN, Camosso-Stefinovic J. How to identify randomized controlled trials in MEDLINE: ten years on. J Med Libr Assoc. 2006;94:130–136. - PMC - PubMed
    1. Clarke M and Oxman AD, editor. The Cochrane Collaboration Handbook 411. Oxford, Update Software; 2000.
    1. Jadad AR, Moher D, Klassen TP. Guides for reading and interpreting systematic reviews: II. How did the authors find the studies and assess their quality? Arch Pediatr Adolesc Med. 1998;152:812–817. - PubMed
    1. Dickersin K, Scherer R, Lefebvre C. Identifying relevant studies for systematic reviews. BMJ. 1994;309:1286–1291. http://bmj.com/cgi/content/abstract/309/6964/1286 - PMC - PubMed

Publication types

LinkOut - more resources