Assessment of procedural skills training and performance in anesthesia using cumulative sum analysis (cusum)

Abstract

Purpose: The current methods (work based assessments and logbooks) used to assess procedural competency and performance have well-documented deficiencies. Cumulative sum analysis (cusum), a statistical method that generates performance graphs over time, is an alternative tool that is not currently widely used. The purpose of this review is to investigate its current role in anesthetic procedural skills training and performance.

Source: A literature search of MEDLINE(®), EMBASE™, BNI, CINAHL(®), the Cochrane Library, NHS Evidence, and the Trip database was performed in October 2012. All papers using cusum to investigate performance in anesthetic procedural skills were included. Their references were searched manually to identify any additional studies.

Principal findings: Thirteen papers were identified. The procedural skills they investigated could be split broadly into three groups: ultrasound skills, airway and cannulation, and regional anesthesia. All papers had small sample sizes (< 30), with most researching novice trainee performance. Wide ranges were seen in the number of procedures required to achieve cusum-defined procedural competency. These were due to differences in definitions of success/failure of a procedure, the acceptable and unacceptable failure rates used for the initial cusum calculation, and individual trainee performance.

Conclusion: Cusum can be used to assess procedural competency, but several problems need to be overcome before it can become a universally accepted method. It is ideally placed to be used as a quality control tool for a trained individual and could also be used to assess the impact of new training methods or equipment on performance.