Measuring clinical skills in agenda-mapping (EAGL-I)

Abstract

Objective: To develop and validate the Evaluation of AGenda-mapping skilL Instrument (EAGL-I).

Methods: EAGL-I was constructed after a literature review and piloting. Simulated consultation recordings were collected in a workshop with third-year medical students at three time points: once pre-teaching, twice post-teaching. Three raters used EAGL-I to assess student agenda-mapping. We examined reliability, ability to detect change and predict full expression of patients' agendas.

Results: EAGL-I scores represented reliable assessment of agenda-mapping (Ep(2)=0.832; φ=0.675). Generalizability coefficients across items (Ep(2)=0.836) and raters (φ=0.797 two raters) were acceptable. A one-way repeated measure ANOVA with post hoc analysis found a statistically significant difference between the pre-teaching occasion of measurement and each post-teaching occasion (p<0.001) and no significant difference between the two post-teaching occasions (p=0.085). Multilevel logistic regression show scores predict expression of scripted hidden agendas irrespective of occasions, or patient scenario (n=60, p=0.005).

Conclusion: Evidence of measure validation is shown. Reliability is optimised when two or more raters use EAGL-I and agenda-mapping has been taught. EAGL-I appears sensitive to change. Higher scores predict the likelihood that a patient will disclose their full agenda in a simulated environment.

Practice implications: A validated tool for measuring agenda-mapping in teaching and research is now available.

Keywords: Agenda-mapping; Establish focus; Measure development; Measure validation; Skills assessment; Teaching.