A practical method for transforming free-text eligibility criteria into computable criteria

Abstract

Formalizing eligibility criteria in a computer-interpretable language would facilitate eligibility determination for study subjects and the identification of studies on similar patient populations. Because such formalization is extremely labor intensive, we transform the problem from one of fully capturing the semantics of criteria directly in a formal expression language to one of annotating free-text criteria in a format called ERGO annotation. The annotation can be done manually, or it can be partially automated using natural-language processing techniques. We evaluated our approach in three ways. First, we assessed the extent to which ERGO annotations capture the semantics of 1000 eligibility criteria randomly drawn from ClinicalTrials.gov. Second, we demonstrated the practicality of the annotation process in a feasibility study. Finally, we demonstrate the computability of ERGO annotation by using it to (1) structure a library of eligibility criteria, (2) search for studies enrolling specified study populations, and (3) screen patients for potential eligibility for a study. We therefore demonstrate a new and practical method for incrementally capturing the semantics of free-text eligibility criteria into computable form.

Figure 1

The GELLO expression language assumes a patient data model that may includes an Observation class that has properties `effectiveTime' (an interval with high and low limits), `code' consisting of a coded concept (e.g., terminology and code), and `value' that may be a coded concept or a physical quantity that has a value and unit.

Figure 2

ERGO Annotations. (a) A simple statement annotation for a noun phrase composed of five terms combined together using OR; (b) a comparison statement annotation for an assessment of Framingham risk score greater than 0.2.

Figure 3

(a) A possible user interface for specifying a conjunction of disjunctive queries. Clicking on the “More” button refreshes the screen and creates a new set of OR subqueries. The button allows the creation of a new comparison ERGO Annotation as shown in Figure 3 (b) or a new noun phrase ERGO Annotation as shown in Figure 3 (c). Figure 3 (c). A possible user-interface for specifying a noun phrase such as “Tuberculosis of intrathoracic lymph nodes, confirmed histologically,” initially consisting of a primitive noun and its adjectival modifiers. By clicking on the “More” button, a user can specify additional modifier attributes and noun phrases. Tool support can easily facilitate the task of searching terminological references such as primitive nouns and adjectival modifiers.

Figure 4

Steps in automated generation of ERGO Annotations.

Figure 5

Predefined OWL ontology to illustrate how ERGO Annotations may be used to classify criteria and to search for.