Searching for clinical prediction rules in MEDLINE

Abstract

Objectives: Clinical prediction rules have been advocated as a possible mechanism to enhance clinical judgment in diagnostic, therapeutic, and prognostic assessment. Despite renewed interest in the their use, inconsistent terminology makes them difficult to index and retrieve by computerized search systems. No validated approaches to locating clinical prediction rules appear in the literature. The objective of this study was to derive and validate an optimal search filter for retrieving clinical prediction rules, using the National Library of Medicine's MEDLINE database.

Design: A comparative, retrospective analysis was conducted. The "gold standard" was established by a manual search of all articles from select print journals for the years 1991 through 1998, which identified articles covering various aspects of clinical prediction rules such as derivation, validation, and evaluation. Search filters were derived, from the articles in the July through December issues of the journals (derivation set), by analyzing the textwords (words in the title and abstract) and the medical subject heading (from the MeSH Thesaurus) used to index each article. The accuracy of these filters in retrieving clinical prediction rules was then assessed using articles in the January through June issues (validation set).

Measurements: The sensitivity, specificity, positive predictive value, and positive likelihood ratio of several different search filters were measured.

Results: The filter "predict$ OR clinical$ OR outcome$ OR risk$" retrieved 98 percent of clinical prediction rules. Four filters, such as "predict$ OR validat$ OR rule$ OR predictive value of tests," had both sensitivity and specificity above 90 percent. The top-performing filter for positive predictive value and positive likelihood ratio in the validation set was "predict$.ti. AND rule$."

Conclusions: Several filters with high retrieval value were found. Depending on the goals and time constraints of the searcher, one of these filters could be used.

Figure 1

Flow diagram of derivation and validation phases.

Figure 2

Calculation of performance measures.Sensitivityequals a/(a+c), the proportion of articles with clinical prediction rules that were retrieved by filter. Specificityequals d/(b+d), the proportion of articles without clinical prediction rules that were not retrieved by filter. Positive predictive valueequals a/(a+b), the proportion of retrieved articles that contained clinical prediction rules. Positive likelihood ratio equals (a/[a+c])/(b/[b+d]), the ratio of sensitivity to 1–specificity, which was the proportion of desired articles retrieved compared with the proportion of undesired articles retrieved.