Construct validity of the ProMIS laparoscopic simulator

Abstract

Background: Laparoscopic surgery challenges both the surgical novice and experienced open surgeon with unique psychomotor adaptations. Surgical skills assessment has historically relied on subjective opinion and case experience. Objective performance metrics have stimulated much interest in surgical education over the last decade and proficiency-based simulation has been proposed as a paradigm shift in surgical skills training. New assessment tools must be subjected to scientific validation. This study examined the construct validity of a hybrid laparoscopic simulator with in-built motion tracking technology.

Methods: Volunteers were recruited from four experience groups (consultant surgeon, senior trainee, junior trainee, medical student). All subjects completed questionnaires and three tasks on the ProMIS laparoscopic simulator (laparoscope orientation, object positioning, sharp dissection). Motion analysis data was obtained via optical tracking of instrument movements. Objective metrics included time, path length (economy of movement), smoothness (controlled handling) and observer-recorded penalty scores.

Results: One hundred and sixty subjects completed at least one of the three tasks. Significant group differences were confirmed for number of years qualified, age and case experience. Significant differences were found between experts and novices in all three tasks. Sharp dissection was the strongest discriminator of four recognised laparoscopic skill groups: consultants outperformed students and juniors in all three performance metrics and objective penalty score (p < 0.05), and only accuracy of dissection did not distinguish them from senior trainees (p = 0.261). Seniors dissected faster, more efficiently and more accurately than juniors and students (p < 0.05).

Conclusions: ProMIS provides a construct valid laparoscopic simulator and is a feasible tool to assess skills in a cross-section of surgical experience groups. ProMIS has the potential to objectively measure pre-theatre dexterity practice until an agreed proficiency level of dexterity is achieved. Future work should now examine whether training to expert criterion levels on ProMIS correlates with actual operative performance.