Audit and summative assessment: a criterion-referenced marking schedule

Abstract

Background: 1996 will see the introduction of summative assessment of general practitioner registrars (trainees). One part of this assessment is the written submission of practical work. In the west of Scotland, audit projects have been chosen as the format for practical work. A valid and reliable marking schedule for such projects is needed.

Aim: A study was undertaken to develop a criterion-referenced marking schedule for assessing registrars' audit projects for summative assessment.

Method: Medical and non-medical professionals, in a series of workshops, compiled a list of essential elements of good audit. These features were tested and refined using registrars' audit projects. All trainers in the west of Scotland were then sent a list of 14 criteria and asked to indicate whether each criterion was an essential or desirable component of a registrar's audit project for summative assessment. A final workshop was held to develop an audit marking schedule.

Results: Of 155 trainers in the west of Scotland, 135 replied to the list of criteria for registrar audit projects (87%). Ten criteria were deemed essential or desirable by 80% or more of the respondents. Participants in the final workshop selected five criteria which would form the audit project marking schedule for registrars undergoing summative assessment. These were: defined reason for choice of audit project, relevance of criteria chosen, appropriate preparation and planning, appropriate interpretation of relevant data and detailed proposals for change. For an audit project to pass assessment all five criteria must be present.

Conclusion: A criterion-referenced approach to assessing registrars' audit projects, developed from their trainers' opinions of essential or desirable criteria for good audit, is described. Further evaluation of the criterion-referenced marking schedule is required.