Risk-adjusted observed minus expected cumulative sum (RA O-E CUSUM) chart for visualisation and monitoring of surgical outcomes

Abstract

To improve patient safety, surgeons can continually monitor the surgical outcomes of their patients. To this end, they can use statistical process control tools, which primarily originated in the manufacturing industry and are now widely used in healthcare. These tools belong to a broad family, making it challenging to identify the most suitable methodology to monitor surgical outcomes. The selected tools must balance statistical rigour with surgeon usability, enabling both statistical interpretation of trends over time and comprehensibility for the surgeons, their primary users. On one hand, the observed minus expected (O-E) chart is a simple and intuitive tool that allows surgeons without statistical expertise to view and interpret their activity; however, it may not possess the sophisticated algorithms required to accurately identify important changes in surgical performance. On the other hand, a statistically robust tool like the cumulative sum (CUSUM) method can be helpful but may be too complex for surgeons to interpret and apply in practice without proper statistical training. To address this issue, we developed a new risk-adjusted (RA) O-E CUSUM chart that aims to provide a balanced solution, integrating the visualisation strengths of a user-friendly O-E chart with the statistical interpretation capabilities of a CUSUM chart. With the RA O-E CUSUM chart, surgeons can effectively monitor patients' outcomes and identify sequences of statistically abnormal changes, indicating either deterioration or improvement in surgical outcomes. They can also quantify potentially preventable or avoidable adverse events during these sequences. Subsequently, surgical teams can try implementing changes to potentially improve their performance and enhance patient safety over time. This paper outlines the methodology for building the tool and provides a concrete example using real surgical data to demonstrate its application.

Keywords: Adverse events, epidemiology and detection; Healthcare quality improvement; Statistical process control; Surgery.

Figure 1. Example of a risk-adjusted observed minus expected (O-E) chart in thyroid surgery. The chart displays the occurrence of recurrent laryngeal nerve palsy after thyroidectomy for a single surgeon. The curve moves upward if the number of operations with recurrent laryngeal nerve palsy increases above that predicted by the risk model and moves downward if the number decreases. Data originated from a previous study: Duclos, A., Lifante, J.-C., Ducarroz, S., Soardo, P., Colin, C. and Peix, J.- L. (2011), Influence of Intraoperative Neuromonitoring on Surgeons’ Technique During Thyroidectomy. World J Surg, 35: 773-778 2014.https://doi.org/10.1007/soo268-011-0963-4.

Figure 2. Example of a risk-adjusted cumulative sum (CUSUM) chart in thyroid surgery. The chart displays the occurrence of recurrent laryngeal nerve palsy after thyroidectomy for a single surgeon. The upper curve detects signals of deterioration in surgical performance, while the lower curve detects signals of improvement. When a curve crosses over its control limit, a signal is detected (shown with circles), indicating a statistically significant change in surgical performance. The graph is subsequently reset to allow further monitoring. In this example, deteriorations were detected at procedures 57 and 192. Data originated from a previous study: Duclos, A., Lifante, J.-C., Ducarroz, S., Soardo, P., Colin, C. and Peix, J.- L. (2011), Influence of Intraoperative Neuromonitoring on Surgeons’ Technique During Thyroidectomy. World J Surg, 35: 773-778 2014. https://doi.org/10.1007/s00268-011-0963-4.

Figure 3. Risk-adjusted (A) O-E, (B) CUSUM and (C) O-E CUSUM charts for a single digestive surgeon (n=145 procedures performed between 1 November 2020 and 31 December 2021). The outcome was a composite morbidity–mortality measure of major adverse events occurring either during or in the 30 days following the surgery. (A) In the risk-adjusted O-E chart, the curve first moves downward since the number of major adverse events decreases below that predicted by the risk model, then slowly moves upward while the number of major adverse events increases, and finally moves downward again, with a final difference between observed and expected events of −2.5, that is, 2.5 events potentially avoided during the whole 14-month period. (B) In the risk-adjusted CUSUM chart, the coloured areas mark the sequences of deterioration and improvement, including all procedures between the signal and the last zero value of the subscore. The upper curve detects signals of deterioration in surgical performance at procedures 52 and 90, while the lower curve detects signals of improvement at procedures 33, 101 and 128. (C) In the risk-adjusted O-E CUSUM chart, sequences of deterioration and improvement are reported from the CUSUM chart to the O-E chart. First, the O-E CUSUM chart detects an improvement sequence (procedures 16–33, three events potentially avoided), followed by two distinct sequences of deterioration (procedures 41–52 and 67–90, respectively, two and three potentially avoidable events) and two consecutive sequences of improvement (procedures 92–101 and 102–128, respectively, two and three potentially avoided events).