Effect of monitoring surgical outcomes using control charts to reduce major adverse events in patients: cluster randomised trial

Abstract

Objective: To determine the effect of introducing prospective monitoring of outcomes using control charts and regular feedback on indicators to surgical teams on major adverse events in patients.

Design: National, parallel, cluster randomised trial embedding a difference-in-differences analysis.

Setting: 40 surgical departments of hospitals across France.

Participants: 155 362 adults who underwent digestive tract surgery. 20 of the surgical departments were randomised to prospective monitoring of outcomes using control charts with regular feedback on indicators (intervention group) and 20 to usual care only (control group).

Interventions: Prospective monitoring of outcomes using control charts, provided in sets quarterly, with regular feedback on indicators (intervention hospitals). To facilitate implementation of the programme, study champion partnerships were established at each site, comprising a surgeon and another member of the surgical team (surgeon, anaesthetist, or nurse), and were trained to conduct team meetings, display posters in operating rooms, maintain a logbook, and devise an improvement plan.

Main outcome measures: The primary outcome was a composite of major adverse events (inpatient death, intensive care stay, reoperation, and severe complications) within 30 days after surgery. Changes in surgical outcomes were compared before and after implementation of the programme between intervention and control hospitals, with adjustment for patient mix and clustering.

Results: 75 047 patients were analysed in the intervention hospitals (37 579 before and 37 468 after programme implementation) versus 80 315 in the control hospitals (41 548 and 38 767). After introduction of the control chart, the absolute risk of a major adverse event was reduced by 0.9% (95% confidence interval 0.4% to 1.4%) in intervention compared with control hospitals, corresponding to 114 patients (70 to 280) who needed to receive the intervention to prevent one major adverse event. A significant decrease in major adverse events (adjusted ratio of odds ratios 0.89, 95% confidence interval 0.83 to 0.96), patient death (0.84, 0.71 to 0.99), and intensive care stay (0.85, 0.76 to 0.94) was found in intervention compared with control hospitals. The same trend was observed for reoperation (0.91, 0.82 to 1.00), whereas severe complications remained unchanged (0.96, 0.87 to 1.07). Among the intervention hospitals, the effect size was proportional to the degree of control chart implementation witnessed. Highly compliant hospitals experienced a more important reduction in major adverse events (0.84, 0.77 to 0.92), patient death (0.78, 0.63 to 0.97), intensive care stay (0.76, 0.67 to 0.87), and reoperation (0.84, 0.74 to 0.96).

Conclusions: The implementation of control charts with feedback on indicators to surgical teams was associated with concomitant reductions in major adverse events in patients. Understanding variations in surgical outcomes and how to provide safe surgery is imperative for improvements.

Trial registration: ClinicalTrials.gov NCT02569450.

Fig 1

Trial profile. Of the 155 362 included patients, 79 127 were assigned to the 2014-15 pre-implementation period (37 579 patients in intervention hospitals, 41 548 in control hospitals) and 76 235 to the 2017-18 implementation period (37 468 and 38 767, respectively)

Fig 2

Primary and secondary outcomes by implementation of control charts. Highly compliant with implementation of control charts=scores 5-6, moderately compliant=scores 3-4, and poorly compliant=score 2. The adjusted ratio of odds ratios (ROR) captured the effect of the control chart on outcomes from the pre-implementation to implementation period between highly compliant intervention and control hospitals, and between moderate to poor compliance intervention and control hospitals. A ROR less than unity indicated an improvement caused by control chart use. Bars denote 95% confidence intervals that considered patient risk score and clustering at the hospital level

Fig 3

Signal detection on control charts between hospitals groups. A total of 640 quarters corresponding to 16 quarters for each of the 40 hospitals were included in the analysis. Rates of signal detection were calculated as the total number of signals detected for all quarters divided by the number of interpretable indicator variations on control charts provided for all hospitals. Indicator variation was considered interpretable when the warning or control limits were not equal to 0% or 100%. The signal detection of variation in a special cause was defined as a single point outside the control limits or two of three successive points outside the warning limits. Deterioration and improvement signals were studied separately. Deterioration (improvement) signals were counted as the number of upward (downward) signals regardless of the surgical outcome and operative procedure. P values are for rates ratios estimated using mixed effect Poisson regression models to compare rates of signals between the pre-implementation and implementation periods in intervention and control hospitals