Simple search techniques in PubMed are potentially suitable for evaluating the completeness of systematic reviews

Abstract

Objective: The Institute for Quality and Efficiency in Health Care (IQWiG) assesses the added benefit of new drugs by means of company dossiers. The pharmaceutical company performs the information retrieval, which is then assessed by IQWiG. Our aim was to determine whether PubMed's Related Citations (RelCits) and/or a simple-structured Boolean search (SSBS) are efficient and reliable search techniques to assess the completeness of an evidence base consisting of published randomized controlled trials (RCTs).

Study design and setting: Retrospective analysis of citations included as relevant in systematic reviews (SRs) of drugs. The proportion of relevant citations identified by the above-mentioned search techniques was determined. Relative sensitivity, precision, and the number needed to read (NNR) were then calculated.

Results: A total of 19 SRs included 330 relevant PubMed citations. The single techniques yielded either insufficient completeness, reliability, or efficiency. The first 20 RelCits plus SSBS achieved high completeness and reliability (sensitivity: 98.1%, range: 80-100%) and sufficient efficiency (precision: 5.0%, NNR: 25). The first 50 RelCit plus SSBS achieved slightly better completeness and reliability, but slightly worse efficiency.

Conclusion: Combining the first 20 RelCits and an SSBS in PubMed is a suitable method to assess the completeness of an evidence base of published RCTs.