Automatic identification of heart failure diagnostic criteria, using text analysis of clinical notes from electronic health records

Abstract

Objective: Early detection of Heart Failure (HF) could mitigate the enormous individual and societal burden from this disease. Clinical detection is based, in part, on recognition of the multiple signs and symptoms comprising the Framingham HF diagnostic criteria that are typically documented, but not necessarily synthesized, by primary care physicians well before more specific diagnostic studies are done. We developed a natural language processing (NLP) procedure to identify Framingham HF signs and symptoms among primary care patients, using electronic health record (EHR) clinical notes, as a prelude to pattern analysis and clinical decision support for early detection of HF.

Design: We developed a hybrid NLP pipeline that performs two levels of analysis: (1) At the criteria mention level, a rule-based NLP system is constructed to annotate all affirmative and negative mentions of Framingham criteria. (2) At the encounter level, we construct a system to label encounters according to whether any Framingham criterion is asserted, denied, or unknown.

Measurements: Precision, recall, and F-score are used as performance metrics for criteria mention extraction and for encounter labeling.

Results: Our criteria mention extractions achieve a precision of 0.925, a recall of 0.896, and an F-score of 0.910. Encounter labeling achieves an F-score of 0.932.

Conclusion: Our system accurately identifies and labels affirmations and denials of Framingham diagnostic criteria in primary care clinical notes and may help in the attempt to improve the early detection of HF. With adaptation and tooling, our development methodology can be repeated in new problem settings.

Keywords: Diagnostic criteria; Electronic health records; Heart failure; Natural language processing; Text mining.

Fig. 1

High-level PredMED text analysis pipeline.

Fig. 2

User interface for the annotation tool, which was used to manage expert annotations during IAR. Criteria abbreviations are as given in Table 1. Note the misspellings and the contradictions in the annotations for DOExertion and AnkleEdema.

Fig. 3

Evaluation flow.

Fig. 4

Precision and recall for individual criteria. Criteria abbreviations are as given in Table 1. Each circle represents a criterion and its size reflects the criterion’s occurrence frequency in the extracted results.

Fig. 5

PredMED extractions vs. gold standard annotations – a detailed performance analysis, presented as a confusion matrix over assertions and denials of Framingham criteria. Criteria abbreviations are as given in Table 1. Denials are marked with a “-Neg” suffix. Zero values off the diagonal have been blanked, for readability.