Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2013 May 1;20(3):413-9.
doi: 10.1136/amiajnl-2012-000930. Epub 2012 Oct 31.

Combing signals from spontaneous reports and electronic health records for detection of adverse drug reactions

Affiliations

Combing signals from spontaneous reports and electronic health records for detection of adverse drug reactions

Rave Harpaz et al. J Am Med Inform Assoc. .

Abstract

Objective: Data-mining algorithms that can produce accurate signals of potentially novel adverse drug reactions (ADRs) are a central component of pharmacovigilance. We propose a signal-detection strategy that combines the adverse event reporting system (AERS) of the Food and Drug Administration and electronic health records (EHRs) by requiring signaling in both sources. We claim that this approach leads to improved accuracy of signal detection when the goal is to produce a highly selective ranked set of candidate ADRs.

Materials and methods: Our investigation was based on over 4 million AERS reports and information extracted from 1.2 million EHR narratives. Well-established methodologies were used to generate signals from each source. The study focused on ADRs related to three high-profile serious adverse reactions. A reference standard of over 600 established and plausible ADRs was created and used to evaluate the proposed approach against a comparator.

Results: The combined signaling system achieved a statistically significant large improvement over AERS (baseline) in the precision of top ranked signals. The average improvement ranged from 31% to almost threefold for different evaluation categories. Using this system, we identified a new association between the agent, rasburicase, and the adverse event, acute pancreatitis, which was supported by clinical review.

Conclusions: The results provide promising initial evidence that combining AERS with EHRs via the framework of replicated signaling can improve the accuracy of signal detection for certain operating scenarios. The use of additional EHR data is required to further evaluate the capacity and limits of this system and to extend the generalizability of these results.

PubMed Disclaimer

Figures

Figure 1
Figure 1
A comparison of two signal-detection strategies: the proposed system, AERS∩HER, which includes signals common to both AERS and EHRs, and its comparator, AERS. The claim being made is that the top set of K ranked signals retrieved from the AERS∩EHR system contain more true positives (true ADRs) than the set of top K signals retrieved from AERS when used by itself (currently the standard approach in signal detection). During evaluation, the two sets of K signals will be compared with respect to a gold standard. The pattern of circle overlaps displayed reflects typical outcomes, where, for most values of K, the signals produced by AERS∩EHR will be richer with true ADRs. ADR, adverse drug reaction; AERS, adverse event reporting system; EHR, electronic health record.
Figure 2
Figure 2
Processing pipeline for generating and evaluating signals produced by adverse event reporting system (AERS), electronic health records (EHRs), and the combined system. Disproportionality analysis refers to a class of methods used to generate adverse drug reaction signals. Unstructured EHR data in this context refers to EHR narratives, which are processed using the natural language processing system MedLEE. Structured EHR data in this context refers to laboratory test results, which are linked to each narrative.
Figure 3
Figure 3
Performance comparison based on the precision at K statistic for different values of K (amount of signals selected). Error bars reflect 95% CIs for precision at each point evaluated. Non-overlapping CIs across the two systems are an indicator of statistically significant (0.05 level or less) performance differences across the two systems. The underlying reference standard consisted of the union of the established and plausible classes of adverse drug reactions. PPV, positive predictive value.

References

    1. Schneeweiss S, Hasford J, Gottler M, et al. Admissions caused by adverse drug events to internal medicine and emergency departments in hospitals: a longitudinal population-based study. Eur J Clin Pharmacol 2002;58:285–91 - PubMed
    1. Classen DC, Pestotnik SL, Evans RS, et al. Adverse drug events in hospitalized patients. Excess length of stay, extra costs, and attributable mortality. JAMA 1997;277:301–6 - PubMed
    1. Bates DW, Spell N, Cullen DJ, et al. The costs of adverse drug events in hospitalized patients. Adverse Drug Events Prevention Study Group. JAMA 1997;277:307–11 - PubMed
    1. Adverse Event Reporting System http://www.fda.gov/cder/aers/default.htm (accessed Jul 2007).
    1. Szarfman A, Machado SG, O'Neill RT. Use of screening algorithms and computer systems to efficiently signal higher-than-expected combinations of drugs and events in the US FDA's spontaneous reports database. Drug Saf 2002;25:381–92 - PubMed

Publication types

MeSH terms