Association of Surgical Skill Assessment With Clinical Outcomes in Cancer Surgery

Abstract

Importance: Complex surgical interventions are inherently prone to variation yet they are not objectively measured. The reasons for outcome differences following cancer surgery are unclear.

Objective: To quantify surgical skill within advanced laparoscopic procedures and its association with histopathological and clinical outcomes.

Design, setting, and participants: This analysis of data and video from the Australasian Laparoscopic Cancer of Rectum (ALaCaRT) and 2-dimensional/3-dimensional (2D3D) multicenter randomized laparoscopic total mesorectal excision trials, which were conducted at 28 centers in Australia, the United Kingdom, and New Zealand, was performed from 2018 to 2019 and included 176 patients with clinical T1 to T3 rectal adenocarcinoma 15 cm or less from the anal verge. Case videos underwent blinded objective analysis using a bespoke performance assessment tool developed with a 62-international expert Delphi exercise and workshop, interview, and pilot phases.

Interventions: Laparoscopic total mesorectal excision undertaken with curative intent by 34 credentialed surgeons.

Main outcomes and measures: Histopathological (plane of mesorectal dissection, ALaCaRT composite end point success [mesorectal fascial plane, circumferential margin, ≥1 mm; distal margin, ≥1 mm]) and 30-day morbidity. End points were analyzed using surgeon quartiles defined by tool scores.

Results: The laparoscopic total mesorectal excision performance tool was produced and shown to be reliable and valid for the specialist level (intraclass correlation coefficient, 0.889; 95% CI, 0.832-0.926; P < .001). A substantial variation in tool scores was recorded (range, 25-48). Scores were associated with the number of intraoperative errors, plane of mesorectal dissection, and short-term patient morbidity, including the number and severity of complications. Upper quartile-scoring surgeons obtained excellent results compared with the lower quartile (mesorectal fascial plane: 93% vs 59%; number needed to treat [NNT], 2.9, P = .002; ALaCaRT end point success, 83% vs 58%; NNT, 4; P = .03; 30-day morbidity, 23% vs 50%; NNT, 3.7; P = .03).

Conclusions and relevance: Intraoperative surgical skill can be objectively and reliably measured in complex cancer interventions. Substantial variation in technical performance among credentialed surgeons is seen and significantly associated with clinical and pathological outcomes.

Figure 1.. The Laparoscopic Total Mesorectal Excision (TME) Performance Tool (LapTMEpt)

The accompanying manual is provided in the eAppendix in the Supplement. It consists of 4 vertical columns representing task areas and 4 horizontal rows representing the performance domains, creating 16 separate items that are scored on a scale of 1 to 4, in which a higher score indicates a more proficient technical performance and a total score of 64 indicates a perfect and proficient performance. Nv indicates neurovascular.

Figure 2.. Laparoscopic Total Mesorectal Excision (TME) Performance Tool Score Analyses

A, Scattergraph displaying number of error events identified from observational clinical human reliability analysis (OCHRA) review with line of best fit and 95% CI. A moderate negative correlation is observed (r^s = −0.515; P < .001) and is comparable with the previously reported laparoscopic colonic competency assessment tool concurrent validity. Each additional error event was associated with a 2-point drop in tool scores. B, Bar graph displaying the distribution of tool scores from the 176 cases. Substantial variation is observed despite both randomized clinical trials using surgeon-credentialing policies. C, Box-whisker plot comparing scores between the 3 case complexity grades. Lines represent the median and interquartile range with whiskers depicting the 95% CI. A significant decrease is observed with grade increase (43 [95% CI, 40-46] vs 39 [95% CI, 36-42] vs 36 [95% CI, 32-38]; P < .001).