Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2011 Apr;46(2):654-78.
doi: 10.1111/j.1475-6773.2010.01156.x. Epub 2010 Aug 16.

Performance characteristics of a methodology to quantify adverse events over time in hospitalized patients

Paul J Sharek  1 , Gareth Parry, Donald Goldmann, Kate Bones, Andrew Hackbarth, Roger Resar, Frances A Griffin, Dale Rhoda, Cathy Murphy, Christopher P Landrigan
Affiliations

Performance characteristics of a methodology to quantify adverse events over time in hospitalized patients

Paul J Sharek et al. Health Serv Res. 2011 Apr.

Abstract

Objective: To assess the performance characteristics of the Institute for Healthcare Improvement Global Trigger Tool (GTT) to determine its reliability for tracking local and national adverse event rates.

Data sources: Primary data from 2008 chart reviews.

Study design: A retrospective study in a stratified random sample of 10 North Carolina hospitals. Hospital-based (internal) and contract research organization-hired (external) reviewers used the GTT to identify adverse events in the same 10 randomly selected medical records per hospital in each quarter from January 2002 through December 2007.

Data collection/extraction: Interrater and intrarater reliability was assessed using κ statistics on 10 percent and 5 percent, respectively, of selected medical records. Additionally, experienced GTT users reviewed 10 percent of records to calculate internal and external teams' sensitivity and specificity.

Principal findings: Eighty-eight to 98 percent of the targeted 2,400 medical records were reviewed. The reliability of the GTT to detect the presence, number, and severity of adverse events varied from κ=0.40 to 0.60. When compared with a team of experienced reviewers, the internal teams' sensitivity (49 percent) and specificity (94 percent) exceeded the external teams' (34 and 93 percent), as did their performance on all other metrics.

Conclusions: The high specificity, moderate sensitivity, and favorable interrater and intrarater reliability of the GTT make it appropriate for tracking local and national adverse event rates. The strong performance of hospital-based reviewers supports their use in future studies.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Description and Number of Medical Record Reviews, by Reviewer Type (AE, Adverse Event)
Figure 2
Figure 2
Level of Agreement within the Internal and External Reviewer Teams Using the Same Subjects Note. Each bar shown is centered on the weighted κ estimate and extends to illustrate the associated 95 percent confidence interval.
Figure 3
Figure 3
Level of Agreement between the Internal and External Reviewer Teams
Figure 4
Figure 4
Level of Agreement between Internal and External Review Teams and the Experienced Review Team
See this image and copyright information in PMC

Comment in

References

    1. Agency for Healthcare Research and Quality. 2009. “Healthcare Cost and Utilization Project (HCUP)” [accessed on December 28, 2009]. Available at http://hcupnet.ahrq.gov/HCUPnet.jsp?Id=537A491834DBC7EC&Form=SelDB&JS=Y&.... - PubMed
    1. AHA Annual Survey Database for Fiscal Year 2005 and AHA Annual Survey Database for Fiscal Year 2004. [accessed on July 27, 2010]. Available at http://www.ahadata.com/ahadata/html/AHASurvey.html.
    1. Baker GR, Norton PG, Flintoft V, Blais R, Brown A, Cox J, Etchells E, Ghali WA, Hébert P, Majumdar SR, O'Beirne M, Palacios-Derflingher L, Reid RJ, Sheps S, Tamblyn R. The Canadian Adverse Event Study: The Incidence of Adverse Events among Hospital Patients in Canada. Canadian Medical Association Journal. 2004;170(11):1678–86. - PMC - PubMed
    1. Berwick DM, Calkins DR, McCannon CJ, Hackbarth AD. The 100,000 Lives Campaign: Setting a Goal and a Deadline for Improving Health Care Quality. Journal of the American Medical Association. 2006;295(3):324–7. - PubMed
    1. Brennan TA, Leape LL, Laird NM, Hebert L, Localio AR, Lawthers AG, Newhouse JP, Weiler PC, Hyatt HH. Incidence of Adverse Events and Negligence in Hospitalized Patients. Results of the Harvard Medical Practice Study I. New England Journal of Medicine. 1991;324(6):370–7. - PubMed

Publication types