Performance of a dangerous answer subtest within a subspecialty certifying examination

Abstract

Previous studies have been reported suggesting that a population of candidates for specialty board certification can be identified who pass the certification examination but who give an unduly high number of 'dangerous' responses, indicating their acceptance of actively harmful actions. To confirm these results, a retrospective analysis was undertaken of a Subspecialty Board of Nephrology certification examination. Experts identified a subtest of 75 dangerous answers. The performance of candidates on this subtest was compared with their performance on the total examination. The subtest was moderately reliable, ranked criterion groups appropriately, had a correlation with the total test of 0.71, and using the standard used for the total examination, identified the certification status of 84% of candidates. However, when the correlation was corrected for unreliability, the correlation became 1.0, indicating that a dangerous answer subtest, at least in this examination and population, does not identify a unique population of certified but 'dangerous' doctors.