Development and pilot testing of an OSCE for difficult conversations in surgical intensive care

Abstract

Objectives: To describe the development and results of an Objective Structured Clinical Exam (OSCE) for leading family conferences in the surgical intensive care unit (SICU).

Design: Pilot demonstration and reliability assessment.

Setting: General surgery residency program at a major academic teaching hospital.

Participants: PGY-2 and PGY-4 categorical general surgery residents (n=8).

Results: The SICU Family Conference OSCE consists of two 20-minute stations, one requiring residents to lead an end-of-life discussion and the other to disclose an iatrogenic complication. Actual case scenarios and trained actors were used; the examinations were videotaped in a standardized setting. Two professional raters as well as the participating actors assessed each resident performance using rating tools developed for each station and based on guiding principles gleaned from the literature. Resident debriefings and evaluation surveys were also conducted. Resident perception of the OSCE overall was positive. Analysis of the videotapes revealed the need for greater standardization of the actors' roles. The rating tools showed strong internal consistency (0.77-0.85), but inter-rater agreement of scores was generally low (<0.70) within rater groups. Family actors consistently gave residents higher global assessment scores than did the professional raters. Second- and fourth-year residents scored equally well on the examination.

Conclusions: This pilot provided residents with a positive learning experience and valid formative feedback. Case materials developed for each station served their function well. More work in actor and rater training is needed before the examination scores can be reliably used in summative evaluation.