Reliability of peer and self-assessment scores compared with trainers' scores following third molar surgery

Abstract

Context: It is sometimes claimed that self-assessment is inaccurate and that clinicians over-rate their performance. There is a need to find out why this should be. Is poor self-assessment caused by some clinicians' inability to accurately judge performance? Or does over-scoring result from a desire to convey a more favourable impression? Peer assessment is widely advocated and is said to be of benefit to both assessor and assessee.

Methods: In this study, we wanted to see if postgraduates were able to peer-assess and if this form of assessment was more reliable than self-assessment when compared with assessment by a trainer. We used checklist and global rating scales to evaluate surgical skills in removing a mandibular third molar tooth.

Results: There was no statistically significant difference between peer-assessed and trainer-assessed scores. We found that, on average, peer assessment (especially global rating scales) reflected trainer scores more accurately than self-assessment of surgical skills. Self-assessment scores were significantly higher on average than those given in peer assessment.

Discussion: Although peers and trainee surgeons came from the same group, the surgeons were more likely to over-score when measuring their own performances. The greatest variability (and over-scoring) between assessor and trainee surgeon appeared to occur in those with lower mean scores. Formative peer assessment may be a useful and less stressful mechanism for encouraging reflection.