Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2005 Jul-Aug;12(4):448-57.
doi: 10.1197/jamia.M1794. Epub 2005 Mar 31.

Automated detection of adverse events using natural language processing of discharge summaries

Genevieve B Melton  1 , George Hripcsak
Affiliations

Automated detection of adverse events using natural language processing of discharge summaries

Genevieve B Melton et al. J Am Med Inform Assoc. 2005 Jul-Aug.

Abstract

Objective: To determine whether natural language processing (NLP) can effectively detect adverse events defined in the New York Patient Occurrence Reporting and Tracking System (NYPORTS) using discharge summaries.

Design: An adverse event detection system for discharge summaries using the NLP system MedLEE was constructed to identify 45 NYPORTS event types. The system was first applied to a random sample of 1,000 manually reviewed charts. The system then processed all inpatient cases with electronic discharge summaries for two years. All system-identified events were reviewed, and performance was compared with traditional reporting.

Measurements: System sensitivity, specificity, and predictive value, with manual review serving as the gold standard.

Results: The system correctly identified 16 of 65 events in 1,000 charts. Of 57,452 total electronic discharge summaries, the system identified 1,590 events in 1,461 cases, and manual review verified 704 events in 652 cases, resulting in an overall sensitivity of 0.28 (95% confidence interval [CI]: 0.17-0.42), specificity of 0.985 (CI: 0.984-0.986), and positive predictive value of 0.45 (CI: 0.42-0.47) for detecting cases with events and an average specificity of 0.9996 (CI: 0.9996-0.9997) per event type. Traditional event reporting detected 322 events during the period (sensitivity 0.09), of which the system identified 110 as well as 594 additional events missed by traditional methods.

Conclusion: NLP is an effective technique for detecting a broad range of adverse events in text documents and outperformed traditional and previous automated adverse event detection methods.

PubMed Disclaimer

References

    1. Kohn LT, Corrigan JM, Donaldson MS, editors. To err is human: building a safer health system. Washington, DC: National Academy Press, 2000. - PubMed
    1. Zapt D, Reason JT. Introduction to error handling. Appl Psychol. 1994;43:427–32.
    1. Benson M, Junger A, Fuchs C, Quinzio L, Bottger S, Jost A, et al. Using an anesthesia information management system to prove a deficit in voluntary reporting of adverse events in a quality assurance program. J Clin Monit Comput. 2000;16:211–7. - PubMed
    1. Cullen DJ, Sweitzer BJ, Bates DW, Burdick E, Edmondson A, Leape LL. Preventable adverse drug events in hospitalized patients: a comparative study of intensive care and general care units. Crit Care Med. 1997;25:1289–97. - PubMed
    1. Field TS, Gurwitz JH, Harrold LR, Rothschild JM, Debellis K, Seger AC, et al. Strategies for detecting adverse drug events among older persons in the ambulatory setting. J Am Med Inform Assoc. 2004;11:25–80. - PMC - PubMed

Publication types