Quantitative assessment of a new preparatory tool for board certification in urology

Abstract

Objective: To analyse the performance of candidates in a Canadian national mock-examination for final-year urology residents with respect to North American speciality examinations in urology.

Methods: In 1997 the Queen's Urology Examination Skills Training Program (QUEST) was established as an annual national mock examination for final-year Canadian urology residents. It consists of a short answer question component and an objective structured clinical examination. During the 5-year period (1997-2001), 91 final-year residents from all 11 Canadian urology residency-training programmes participated in QUEST and the Royal College of Physicians and Surgeons of Canada certifying examinations (RCPSCE); 43 (47%) of candidates also attempted the American Board of Urology part 1 qualifying examinations (ABU 1). Performance on QUEST was correlated with the RCPSCE and ABU 1 in a blinded fashion after submitting QUEST scores to governing bodies. Thresholds were determined to help to predict a candidate's performance on the RCPSCE and ABU 1, based on QUEST scores.

Results: There was a moderately close correlation between overall QUEST and RCPSCE performance (r = 0.68, P < 0.001) and a moderate correlation between overall QUEST and ABU 1 performance (r = 0.42, P = 0.005). For the following QUEST scores, the probability of success on the RCPSCE was: < 65%, 67% pass; 66-75%, 80% pass; > 75%, 100% pass (P = 0.002). For ABU 1, QUEST overall score of 80% gave a 69% probability of scoring > or = 70% on ABU 1 (P = 0.003).

Conclusions: QUEST is a moderate predictor of performance on speciality examinations in urology. We consider that the time, effort and expense to maintain QUEST are justified.