Non-Technical Skills for Surgeons (NOTSS): Critical appraisal of its measurement properties

Abstract

Background: To critically appraise the development and measurement properties, including sensibility, reliability, and validity of the Non-Technical Skills of Surgeons (NOTSS) system.

Data sources: Articles that described development process of the NOTSS system were identified. Relevant primary studies that presented evidence of reliability and validity were identified through a comprehensive literature review.

Results: NOTSS was developed through robust item generation and reduction strategies. It was shown to have good content validity, acceptability, and feasibility. Inter-rater reliability increased with greater expertise and number of assessors. Studies demonstrated evidence of cross-sectional construct validity, in that the tool was able to differentiate known groups of varied non-technical skill levels. Evidence of longitudinal construct validity also existed to demonstrate that NOTSS detected changes in non-technical skills before and after targeted training.

Conclusion: In populations and settings presented in our critical appraisal, NOTSS provided reliable and valid measurements of intraoperative non-technical skills of surgeons.

Keywords: Critical appraisal; Measurement property; Non-technical skill; Structured feedback; Surgeon evaluation.