Efficacy and Validity of Orthopaedic Simulators in Surgical Training: A Systematic Review and Meta-Analysis of Randomized Controlled Trials

Abstract

Introduction: The purpose of this study was to perform a systematic review and meta-analysis of the effects of training simulators on surgical skill measures across randomized controlled trials. The authors hypothesized that simulated training would (1) result in objective improvements in skill acquisition and (2) be heterogeneous regarding the outcomes and types of validity assessed.

Methods: The Cochrane Database of Systematic Reviews, the Central Register of Controlled Trials, PubMed, EMBASE, and MEDLINE databases were queried for Level I studies on training simulators between 2007 and 2019 in accordance with the 2009 Preferred Reporting Items for Systematic Review and Meta-Analysis statement. Exclusion criteria were studies without discrete assessment of skills acquisition after surgical simulator training and level of evidence II to V. The Jadad scale was used to assess the methodological quality of all included articles. Data pertaining to patient demographics, validity measures, simulator types, and study-specific outcome measures were extracted. Meta-analyses adjusted for random effects and heterogeneity analyses (I) were used to compare pooled time-to-completion and performance outcomes among included studies.

Results: A total of 24 studies with 494 participants were identified. The most common simulator type involved knee arthroscopy (11 studies, 45.8%). Eight studies reporting time-to-task completion and performance scores were included in the meta-analysis. Virtual reality training was favored in time-to-task completion (mean difference = -82.25 seconds, P = 0.002) and improvement in objective performance scores (mean difference = 1.24, P = 0.02) relative to traditional training. Sensitivity analysis of time-to-task completion based on the length of training interval revealed a mean difference of -45.24 (P = 0.07) and -137.74 (P < 0.001) seconds for the short-term and immediate posttesting subgroups, respectively.

Conclusion: Overall, improved task efficiency and performance were observed with the use of orthopaedic simulators. However, simulator type, training protocols, and outcome measures were heterogeneous. Future studies are warranted to evaluate financial cost and longitudinal training programs and to standardize outcomes regarding the use of simulators in orthopaedic education.

Level of evidence: Level I.