Using a 'peer assessment questionnaire' in primary medical care

Abstract

Background: Periodic assessment of clinician performance or 'revalidation' is being actively considered to reassure the public that doctors are 'up to date and fit to practice'. There is, therefore, increasing interest in how to assess individual clinician performance in a valid and reliable way. The use of peer assessment questionnaires is one of the methods being considered and investigated by the General Medical Council in the UK.

Aim: To test the feasibility of using a peer assessment questionnaire in a primary care setting, and consider the related issues of validity and reliability and compare the results to previous studies.

Design: Cross-sectional survey in a volunteer sample.

Setting: General practice in the UK.

Method: GPs who volunteered to take part in an evaluation of a pilot appraisal implementation scheme were recruited by appraisers. These volunteers (GP subjects) chose 15 colleagues to complete a 'peer assessment' questionnaire that asked peers to make judgements about their clinical skills and other characteristics, such as 'compassion', 'integrity' and 'responsibility'.

Results: Of the 207 practitioners that agreed to be appraised, 113 completed the optional task of implementing the peer questionnaire. Of the 1271 raters, 1189 provided data about their roles and 33.6% of these were GPs. The data revealed significant levels of items where peers were 'unable to evaluate' the issues posed in the questionnaire (ranging from 13.7-1.8%). These rates differed from those obtained in studies based in the US where mean scores were slightly higher. Although the overall results are broadly similar to those previously obtained, there are sufficient differences to suggest that there are contextual issues influencing the interpretation of the items and therefore the scoring process.

Conclusion: The volunteer sample in this study found no major obstacles to the implementation of the peer assessment questionnaire. While it is not possible to generalise from this selected volunteer sample, the use of peer assessment questionnaires appears feasible and may be acceptable to clinical practitioners. However, concern remains about the validity of such instruments and that their development did not fully consider issues of procedural justice or whether the overall purpose of the tools was to be formative, summative, or both.