Development and Validation of a Surgical Performance Assessment Scale for Lung Lobectomy and Lymph Node Dissection

Abstract

Objectives: Assessing surgical trainees' performance requires objective, validated, and reliable evaluation tools. We developed and validated the Lung Lobectomy Performance Assessment Scale Score (LOB-PASS) to objectively evaluate lung lobectomy performance, including lymph node dissection, across different surgical approaches (postero-lateral thoracotomy, video-assisted, and robot-assisted).

Methods: The LOB-PASS scale was developed using the Delphi consensus method and response process testing in simulation sessions. Validation was performed on 48 thoracic surgery residents, assessed by independent observers through simulation-based response testing. Internal structure was evaluated using Cronbach's alpha, intraclass correlation coefficient (ICC), correlation scores from linear regression (R2), and Bland-Altman analysis. Inter-observer score comparisons were conducted using the Wilcoxon test and the F-test.

Results: The final scale included 52 items categorized into seven domains: opening (n = 6), cavity exploration and lymph node dissection (n = 12), vein time (n = 5), arterial time (n = 8), bronchial time (n = 6), parenchymal time (n = 7), and closure (n = 8), with a total score of 104 points. Internal structure analysis was based on 48 simulation procedures: Cronbach's alpha = 0.89, ICC = 0.92, R2 = 0.84. No significant inter-observer differences were found (P > .05). Bland-Altman analysis demonstrated a mean difference of 0.19, with 95% limits of agreement ranging from -13.3 to 13.7.

Conclusions: LOB-PASS is a valid and reliable tool for assessing lung lobectomy performance with lymph node dissection across all surgical approaches. Integrating LOB-PASS into surgical training curricula could standardize competency assessment and optimize technical skill acquisition in thoracic surgery.

Keywords: assessment; lung lobectomy; performance; reliability; scale; simulation; validity.