Precision and recall of search strategies for identifying studies on return-to-work in Medline

Abstract

Introduction: The purpose of this study was to report on the qualities of various search strategies and keywords to find return to work (RTW) studies in the Medline bibliographic database.

Methods: We searched Medline for articles on RTW published in 2003, using multiple search strings, and hand searched 16 major periodicals of rehabilitation or occupational medicine. Among the retrieved articles, those considered to be relevant, were pooled in a Gold Standard Database. From this database, we identified candidate text words or MeSH terms for search strategies using a word frequency analysis of the abstracts and a MEDLINE categorization algorithm. According to the frequency of identified terms, searches were run for each term independently and in combination. We computed Recall, Precision, and number needed to read (NNR = 1/Precision) of each keyword or combination of keywords.

Results: Among the 8,073 articles examined, 314 (3.9%) were considered relevant and included in the Gold Standard Database. The search strings ("Rehabilitation, Vocational" [MeSH]), ("Return to work"[All]) and ("Back to work"[All]) had Recall/Precision ratio of 30.46/19.11, 59.55/87.38 and 3.18/90.91%, respectively. Their combination with the Boolean operator OR yielded to a Recall/Precision ratio of 73.89/58.44% and a NNR of 1.7. For the end user requiring comprehensive literature search, the best string was ("Return to work" OR "Back to work" OR "Rehabilitation, vocational"[MeSH] OR "rehabilitation"[Subheading]), with a Recall of 88.22% and a NNR of 18.

Conclusions: No single MeSH term is available to help the physician to identify relevant studies on RTW in Medline. Locating these types of studies requires the use of various MeSH and non-MeSH terms in combination to obtain a satisfactory Recall. Nevertheless, enhancing the Recall of search strategies may lead to lower Precision, and higher NNR, although with a non linear trend. This factor must be taken into consideration by the end user in order to improve the cost-effectiveness ratio of the search in Medline.