High-throughput phenotyping with electronic medical record data using a common semi-supervised approach (PheCAP)

Abstract

Phenotypes are the foundation for clinical and genetic studies of disease risk and outcomes. The growth of biobanks linked to electronic medical record (EMR) data has both facilitated and increased the demand for efficient, accurate, and robust approaches for phenotyping millions of patients. Challenges to phenotyping with EMR data include variation in the accuracy of codes, as well as the high level of manual input required to identify features for the algorithm and to obtain gold standard labels. To address these challenges, we developed PheCAP, a high-throughput semi-supervised phenotyping pipeline. PheCAP begins with data from the EMR, including structured data and information extracted from the narrative notes using natural language processing (NLP). The standardized steps integrate automated procedures, which reduce the level of manual input, and machine learning approaches for algorithm training. PheCAP itself can be executed in 1-2 d if all data are available; however, the timing is largely dependent on the chart review stage, which typically requires at least 2 weeks. The final products of PheCAP include a phenotype algorithm, the probability of the phenotype for all patients, and a phenotype classification (yes or no).

Figure 1.. PheCAP Overview.

Starting with all EMR data, a sensitive filter (Procedure step #1) such as a diagnosis code is used to create a data mart (Procedure step #2) containing all patients who may potentially have the phenotype. Codified data, such as diagnoses codes or medication prescriptions related to the phenotype are extracted from the data mart (Procedure steps #5-6). Additionally, concepts or terms related to the phenotype are extracted using natural language processing (NLP) (Procedure steps #10-15). The NLP dictionary can be developed manually or using an automated process. These data are combined into a patient level data table (Procedure step #7). In parallel, a random sample of patients is selected for chart review to provide gold standard labels (Procedure steps #3-4). Sparse machine learning is applied in two steps: an unsupervised (Procedure steps #28-35) and a supervised step (Procedure steps #36-41) to identify the important features of interest. The output of the pipeline is a phenotype algorithm, a probability of the phenotype for all subjects in the data mart, and a classification of the phenotype for each subject (yes/no) (Procedure steps #42-43).

Figure 2.. Creating an NLP dictionary.

Automated process to generate an NLP dictionary by processing knowledge sources using NLP (Procedure steps #10-14).

Figure 3.. Unsupervised Feature Learning.

Steps to identify informative codified and NLP features for the algorithm prior to supervised training of the algorithm with gold standard labels (Procedure steps #28-35).

Figure 4.. Detailed flow of PheCAP protocol.

User input required at various steps in the PheCAP protocol are specified at the top of the figure as the protocol moves from data extraction, data processing, through algorithm training and validation, and the final outputs: a phenotype algorithm, a probability of the phenotype for all subjects in the data mart, and a classification of the phenotype for each subject (yes or no). Numbers in the figure correspond to Procedure steps.

Figure 5.

Clinical terms identified by MetaMAP along with their mapped CUIs from a Wikipedia article on coronary artery disease (example of results obtained from Procedure step #13).

Figure 6.

Output from parsing the notes using after processing the i2b2 NLP Research Data Set using NILE (example of results obtained from Procedure step #15).

Figure 7.

Output from the supervised algorithm training step depicting (a) SAFE selected features with coefficients from the supervised training step (example of results obtained from Procedure step 38); (b) estimated percent of patients classified as cases (pos.rate), false positive rate (FPR), true positive rate (TPR), PPV, NPV, and F-score over a range of cut-off values from validating the algorithm (example of results obtained from Procedure step 41); (c) predicted probability of being a case for patients in the data mart along with their predicted case status, 1=case, 0=non-case (example of results obtained from Procedure steps 42-43).