Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Multicenter Study
. 2012 Sep-Oct;19(5):859-66.
doi: 10.1136/amiajnl-2011-000535. Epub 2012 Mar 21.

Automated extraction of ejection fraction for quality measurement using regular expressions in Unstructured Information Management Architecture (UIMA) for heart failure

Jennifer H Garvin  1 , Scott L DuVall, Brett R South, Bruce E Bray, Daniel Bolton, Julia Heavirland, Steve Pickard, Paul Heidenreich, Shuying Shen, Charlene Weir, Matthew Samore, Mary K Goldstein
Affiliations
Multicenter Study

Automated extraction of ejection fraction for quality measurement using regular expressions in Unstructured Information Management Architecture (UIMA) for heart failure

Jennifer H Garvin et al. J Am Med Inform Assoc. 2012 Sep-Oct.

Abstract

Objectives: Left ventricular ejection fraction (EF) is a key component of heart failure quality measures used within the Department of Veteran Affairs (VA). Our goals were to build a natural language processing system to extract the EF from free-text echocardiogram reports to automate measurement reporting and to validate the accuracy of the system using a comparison reference standard developed through human review. This project was a Translational Use Case Project within the VA Consortium for Healthcare Informatics.

Materials and methods: We created a set of regular expressions and rules to capture the EF using a random sample of 765 echocardiograms from seven VA medical centers. The documents were randomly assigned to two sets: a set of 275 used for training and a second set of 490 used for testing and validation. To establish the reference standard, two independent reviewers annotated all documents in both sets; a third reviewer adjudicated disagreements.

Results: System test results for document-level classification of EF of <40% had a sensitivity (recall) of 98.41%, a specificity of 100%, a positive predictive value (precision) of 100%, and an F measure of 99.2%. System test results at the concept level had a sensitivity of 88.9% (95% CI 87.7% to 90.0%), a positive predictive value of 95% (95% CI 94.2% to 95.9%), and an F measure of 91.9% (95% CI 91.2% to 92.7%).

Discussion: An EF value of <40% can be accurately identified in VA echocardiogram reports.

Conclusions: An automated information extraction system can be used to accurately extract EF for quality measurement.

PubMed Disclaimer

Conflict of interest statement

Competing interests: None.

Figures

Figure 1
Figure 1
Degree of structure: unstructured (synthetic document).
Figure 2
Figure 2
Degree of structure: semi-structured (synthetic document).
Figure 3
Figure 3
Degree of structure: structured (synthetic document).
Figure 4
Figure 4
Development of annotation guideline and schema for establishment of a reference standard. IAA, inter-annotator agreement.
Figure 5
Figure 5
EF system components and use. EF, ejection fraction.
Figure 6
Figure 6
Rules and sequence of use. EF, ejection fraction.
See this image and copyright information in PMC

References

    1. Being a Meaningful User of Electronic Health Records. http://healthit.hhs.gov/portal/server.pt/community/healthit_hhs_gov__mea... (accessed 7 Jul 2011).
    1. Vest JR, Jasperson J. What should we measure? Conceptualizing usage in health information exchange. J Am Med Inform Assoc 2010;17:302–7 - PMC - PubMed
    1. CMS National Quality Forum Guide for Reading the EHR Incentive Program EP measures. 2010. http://www.cms.gov/QualityMeasures/Downloads/QMGuideForReadingEHR.pdf (accessed 6 Jan 2011).
    1. Bloomrosen M, Starren J, Lorenzi NM, et al. Anticipating and addressing the unintended consequences of health IT and policy: a report from the AMIA 2009 Health Policy Meeting. J Am Med Inform Assoc 2011;18:82–90 - PMC - PubMed
    1. The Official Web site for the Medicare and Medicaid electronic health records (EHR) Incentive Programs. https://www.cms.gov/ehrincentiveprograms/ (accessed 6 Jan 2011).

Publication types