Global Rating Scales for the Assessment of Arthroscopic Surgical Skills: A Systematic Review

Abstract

Purpose: To evaluate whether sufficient validity and reliability evidence exists to support the use of global rating scales (GRS) as evaluation tools in both formative assessment and competency assessment of arthroscopic procedures.

Methods: A search of PubMed, Embase, and Scopus was conducted for articles published between 1990 and 2018. Studies reporting measures of validity and reliability of GRS relating to arthroscopic skills were included. Procedural checklists and other assessment tools were excluded.

Results: A total of 39 articles met the inclusion criteria. In total, 7 de novo GRS specific for arthroscopic education and 3 pre-existing GRS repurposed 4 times for arthroscopic education were identified in the literature. The 11 GRS were used to assess 1175 surgeons 3890 times. Three GRS tools explicitly defined an arbitrary minimum competency threshold, 6 of 11 tools demonstrated construct validity-the ability to significantly discriminate between groups of differing experience-and 5 of 11 tools assessed inter-rater reliability, but only the Arthroscopic Surgical Skills Evaluation Tool demonstrated excellent inter-rater reliability. The Arthroscopic Surgical Skills Evaluation Tool was validated by 16 articles for a total of 537 surgeons for hip, knee, shoulder, and ankle arthroscopy in both simulated and clinical environments but was found to be invalid in wrist arthroscopy. The Basic Arthroscopic Knee Skill Scoring System was validated by 15 articles for a total of 497 surgeons for knee, hip, and shoulder in both clinical and simulated environments. The remaining 9 GRS were validated by 2 or fewer studies.

Conclusions: Overall, GRS have contributed to training, feedback, and formative assessment practices. The GRS reviewed demonstrate both construct and concurrent validity as well as reliability in multiple arthroscopic procedures in multiple joints. Currently, there is sufficient evidence to use GRS as a feedback tool. However, there is insufficient evidence for its use in high-stakes examinations or as a minimum competency assessment.

Level of evidence: Level III, systematic review of level I to III studies.