Assessment of innate ability and skills for endoscopic manipulations by the Advanced Dundee Endoscopic Psychomotor Tester: predictive and concurrent validity

Abstract

Background: This work was undertaken to evaluate the concurrent and predictive validity of the Advanced Dundee Endoscopic Psychomotor Tester (ADEPT) for clinical competence in endoscopic manipulations.

Method: Ten specialist registrars completed 200 structured tasks on ADEPT involving manipulation of switches and dials using standard endoscopic imaging and surgical instruments. This performance was correlated with blind ratings on clinical endoscopic operative competence by four consultant surgeons.

Results: The learning curve on the use of the ADEPT was quick, proficiency in using the system was achieved by all candidates after 20 tasks by the end of the first run when the individual's performance reached its peak. Individual error rates did not improve from the start. A high correlation (0.909) was observed between success rates on ADEPT tasks and error rates. The overall performance on ADEPT correlated well with independent blind assessment of clinical competence (r = 0.789) as did the number of error-free runs (r = 0.740).

Conclusions: The study confirms a good correlation between performance on ADEPT and independent consultant clinical assessment of operative skills (concurrent validity). The system identifies aspects of performance that do not improve with practice (innate abilities) and thus could be used to predict ultimate level of operative skill. If confirmed by larger studies, ADEPT could be used as an aptitude tester for trainee selection in minimal access surgery and interventional radiology. By providing objective feedback on task performance, the system facilitates reflective skill acquisition and its assessment by trainees.