A case study evaluating the portability of an executable computable phenotype algorithm across multiple institutions and electronic health record environments

Abstract

Electronic health record (EHR) algorithms for defining patient cohorts are commonly shared as free-text descriptions that require human intervention both to interpret and implement. We developed the Phenotype Execution and Modeling Architecture (PhEMA, http://projectphema.org) to author and execute standardized computable phenotype algorithms. With PhEMA, we converted an algorithm for benign prostatic hyperplasia, developed for the electronic Medical Records and Genomics network (eMERGE), into a standards-based computable format. Eight sites (7 within eMERGE) received the computable algorithm, and 6 successfully executed it against local data warehouses and/or i2b2 instances. Blinded random chart review of cases selected by the computable algorithm shows PPV ≥90%, and 3 out of 5 sites had >90% overlap of selected cases when comparing the computable algorithm to their original eMERGE implementation. This case study demonstrates potential use of PhEMA computable representations to automate phenotyping across different EHR systems, but also highlights some ongoing challenges.

Figure 1.

BPH case algorithm. ICD = International Classification of Diseases, CPT = Current Procedural Terminology: ICD and CPT codes available in the Value Sets posted at: https://github.com/PheMA/bph-use-case.

Figure 2.

KNIME workflows that read data from either an i2b2 instance (A), or an LDW (B), and execute the BPH case algorithm. After users complete configurations specific to their sites within these workflows, including database connection details (server address, username, password, etc.) and any necessary data customization, the user executes the entire KNIME workflow in one step. Upon execution, each QDM Data Element node reads in the i2b2, or LDW, connection details, and i2b2 ontology mapping, or LDW, queries the users modified and extracts the relevant data, after which the subsequent nodes execute the algorithm logic. On the far right of each workflow in the XLS Writer node: users can specify a filename to which to write the identifiers of the patients with BPH as found by the algorithm. A: i2b2: Table Creator node on the top left is where users enter their i2b2 connection details. Below that node, the “OIDs to i2b2 ONT” metanode is where users make adjustments, if necessary, to the i2b2 ontology mapping. This workflow that executes against the publicly available demonstration version of i2b2 is available at: https://github.com/PheMA/bph-use-caseB: LDW: Database Connector node on the top left is where users enter their LDW connection details. Then, within each QDM Data Element metanode, users open a Database Table Selector node, in which the value set for that data element is available as a variable, and edit the template query in that node to query the appropriate data for that data element from their LDW [eg, edit the template query “select diagnois_code_column from diagnois_table where diagnosis in (diagnois_list_variable)” by replacing the column and table names (in italics) with the column and tables names in the LDW].

Figure 3.

Comparison of eMERGE implementation to PhEMA implementation results illustrating overlap, for sites that successfully executed both the eMERGE and PhEMA implementations and thus had results to compare. Each circle in each Venn diagram is colored to represent the number of patients found by the algorithm to have BPH as follows: green depicts those found by both implementations (the overlap), yellow depicts those found only by the eMERGE implementation, and blue depicts those found only by the PhEMA implementation. Both the number (N) and percentage of patients are shown for each circle. Venn diagrams drawn by the Pacific Northwest National Laboratory’s Venn Diagram Plotter, freely available from: https://omics.pnl.gov/software/venn-diagram-plotter.