Assessing doctors' competence: application of CUSUM technique in monitoring doctors' performance

Abstract

Background: Quality assurance of medical practice requires assessment of doctors' performance, whether informally via a system such as peer review or more formally via one such as credentialing. Current methods of assessment are, however, subjective or implicit. More objective methods of assessment based on statistical process control technique such as cumulative sum (CUSUM) procedure may be helpful.

Objective: To determine the usefulness and acceptability of CUSUM charting for assessing doctors' performance.

Method: We applied CUSUM charting to assess doctors' performance of endoscopic retrograde pancreatography, renal and breast biopsies, thyroidectomy, and instrumental deliver. A CUSUM chart is a graphical representation of the trend in the outcome of a series of consecutive procedures. At acceptable levels of performance, the CUSUM curve is flat, while at unacceptable levels of performance, the curve slopes upward and eventually crosses a decision interval. When this occurs, the CUSUM chart indicates unsatisfactory performance. Thus, it provides an early warning of an adverse trend.

Results: All participating doctors found the technique useful to objectively measure their proficiency. CUSUM charts showed the progress of trainees in acquiring new skills. As they become more skillful with training, their CUSUM curves flatten. Among consultants, level CUSUM curves demonstrated ongoing maintenance of competence. All participants found the technique acceptable as a self-assessment tool. They were, however, less certain of its acceptability as a basis for credentialing.

Conclusion: We recommend the use of CUSUM charting as a tool for personal audit at an individual level. It may also be used to show proof of technical competence for the purpose of credentialing.