Assessment of the global trigger tool to measure, monitor and evaluate patient safety in cancer patients: reliability concerns are raised

Abstract

Background: Countries around the world are currently aiming to improve patient safety by means of the Institute for Healthcare Improvement global trigger tool (GTT), which is considered a valid tool for evaluating and measuring patient safety within organisations. So far, only few data on the measurement properties and utility of the GTT have been published.

Aims: To determine and evaluate the effect of interrater variation between review teams on the standard outcome measures of the GTT and to assess and quantify measurement error of the GTT.

Methods: Retrospective chart reviews were conducted on identical charts by two independent review teams in 2010 at a department of oncology in a university hospital. Standard GTT outcome measurements were obtained and compared between teams using statistical process control (SPC) charts. A Bland-Altman plot assessed measurement error and limits of agreement.

Results: Only 31% of adverse events (AE) were identified by both teams, and further differences in categorisation of identical events was found. Moderate interrater agreement (κ=0.45) between teams gave rise to different conclusions on the patient safety process when monitoring using SPC charts. The Bland-Altman plot suggests little systematic error but large random error.

Conclusions: Review teams may identify different AE and reach different conclusions on the safety process when using the GTT on identical charts. Tracking true change in the safety level is difficult due to measurement error of the GTT. The results do not encourage further use of the GTT until additional evaluation studies on the measurement properties of the GTT have been conducted.

Keywords: Adverse events, epidemiology and detection; Patient safety; Performance measures; Statistical process control; Trigger tools.