A pilot study of the mental workload of objective structured clinical examination examiners

Abstract

Context: Mental workload is a concept used in many industries to investigate operator performance, but it has only recently been used in the educational setting. It has been suggested that excessive mental workload in assessors may impair the validity of objective structured clinical examination (OSCE)-type assessments.

Objectives: This study aimed to establish the feasibility of measuring the mental workload of examiners during an OSCE assessment and to establish methodologies and baseline values to guide future study design.

Methods: Two previously validated methods of measuring mental workload, the NASA-Task Load Index rating scale and use of a secondary task (response to a prompt from a vibrotactile device), were used to measure the workload of 10 subjects during a formative OSCE. Trainee anaesthetists (n = 24) working in an operating theatre were used as a control group.

Results: The mental workload of examiners exceeded that of controls on both measures. Although there was marked inter-subject variability, reliability between stations for individual examiners was robust (α = 0.922).

Conclusions: These data suggest that mental workload is excessive in OSCE examiners. Further studies are required to measure the effect of changes in assessment design and examiner training.