Added Value of Medical Subject Headings Terms in Search Strategies of Systematic Reviews: Comparative Study

Abstract

Background: The massive increase in the number of published scientific articles enhances knowledge but makes it more complicated to summarize results. The Medical Subject Headings (MeSH) thesaurus was created in the mid-20th century with the aim of systematizing article indexing and facilitating their retrieval. Despite the advent of search engines, few studies have questioned the relevance of the MeSH thesaurus, and none have done so systematically.

Objective: The objective of this study was to estimate the added value of using MeSH terms in PubMed queries for systematic reviews (SRs).

Methods: SRs published in 4 high-impact medical journals in general medicine over the past 10 years were selected. Only SRs for which a PubMed query was provided were included. Each query was transformed to obtain 3 versions: the original query (V1), the query with free-text terms only (V2), and the query with MeSH terms only (V3). These 3 queries were compared with each other based on their sensitivity and positive predictive values.

Results: In total, 59 SRs were included. The suppression of MeSH terms had an impact on the number of relevant articles retrieved for 24 (41%) out of 59 SRs. The median (IQR) sensitivities of queries V1 and V2 were 77.8% (62.1%-95.2%) and 71.4% (42.6%-90%), respectively. V1 queries provided an average of 2.62 additional relevant papers per SR compared with V2 queries. However, an additional 820.29 papers had to be screened. The cost of screening an additional collected paper was therefore 313.09, which was slightly more than triple the mean reading cost associated with V2 queries (88.67).

Conclusions: Our results revealed that removing MeSH terms from a query decreases sensitivity while slightly increasing the positive predictive value. Queries containing both MeSH and free-text terms yielded more relevant articles but required screening many additional papers. Despite this additional workload, MeSH terms remain indispensable for SRs.

Keywords: MeSH; MeSH thesaurus; Medical Subject Headings; PPV; PubMed; comparative analysis; literature review; medical knowledge; positive predictive value; review; scientific knowledge; search strategy; systematic literature review; systematic review; utility.

Figure 1

Example of an automated transformation of queries. MeSH: Medical Subject Headings.

Figure 2

Flowchart for the selection of systematic reviews. MeSH: Medical Subject Heading; SR: systematic review.

Figure 3

Contribution of Medical Subject Headings (MeSH) terms to the queries. The orange circles correspond to V2 (free-text terms only), and the blue dots correspond to V1 (free-text terms and MeSH terms). PPV: positive predictive value.

Figure 4

Distribution of the ratio between the number of relevant articles found by V1 and the number found by V2. The pink bar corresponds to the interval (1-1.05). For 2 cases, the ratio corresponded to the division of 0 by 0, and we considered that the result was 1. For other 2 cases, the result of the ratio corresponded to infinity (division by 0).

Figure 5

Forest plot of the odds ratio (OR) for V2 versus V1. An OR>1 means that V2 was better than V1 and so that inclusion of the Medical Subject Headings (MeSH) terms was harmful. An OR<1 means that V2 was worse than V1 and so that MeSH terms were useful. FPV1: false positive V1; FPV2: false positive V2; GS: gold standard; PMID: PubMed Identifier; PPV: positive predictive value; TPV1: true positive V1; TPV2: true positive V2.